Based on #2480
This actually indexes events based on their event type. They are removed
from the index if we receive a `m.room.redaction` event on the
`OutputRoomEvent` stream.
An admin endpoint is added to reindex all existing events.
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
Some tweaks for the send-to-device consumers/producers:
- use `json.RawMessage` without marshalling it first
- try further devices (if available) if we failed to `PublishMsg` in the
producers
- some logging changes (to better debug E2EE issues)
This makes the following changes:
* The various `Defaults` functions are now responsible for setting sane defaults if `generate` is specified, rather than hiding them in `generate-config`
* Some configuration options have been marked as `omitempty` so that they don't appear in generated configs unnecessarily (monolith-specific vs. polylith-specific options)
* A new option `-polylith` has been added to `generate-config` to create a config that makes sense for polylith deployments (i.e. including the internal/external API listeners and per-component database sections)
* A new option `-normalise` has been added to `generate-config` to take an existing file and add any missing options and/or defaults
* CS API changes
* Query remote profiles
* Add passing tests
* Don't create a new FullyQualifiedProfile
* Handle sql.ErrNoRows
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
* Get all account data on CompleteSync
* Revert "Get all account data on CompleteSync"
This reverts commit 44a3e566d8.
* Use /_synapse/admin/v1/register to create account
* Linting
* Linter again :)
* Update docs
* Use HTTP API to reset password, add option to User API `PerformPasswordUpdate` to invalidate sessions
* Fix routing name
* Tell me more about what went wrong
* Deprecate the `-reset-password` flag, document the new API
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
* Generic-based internal HTTP API (tested out on a few endpoints in the federation API)
* Add `PerformInvite`
* More tweaks
* Fix metric name
* Fix LookupStateIDs
* Lots of changes to clients
* Some serverside stuff
* Some error handling
* Use paths as metric names
* Revert "Use paths as metric names"
This reverts commit a9323a6a34.
* Namespace metric names
* Remove duplicate entry
* Remove another duplicate entry
* Tweak error handling
* Some more tweaks
* Update error behaviour
* Some more error tweaking
* Fix API path for `PerformDeleteKeys`
* Fix another path
* Tweak federation client proxying
* Fix another path
* Don't return typed nils
* Some more tweaks, not that it makes any difference
* Tweak federation client proxying
* Maybe fix the key backup test
This should avoid coercions between signed and unsigned ints which might fix problems like `sql: converting argument $5 type: uint64 values with high bit set are not supported`.
* Add `evacuateUser` endpoint, use it when deactivating accounts
* Populate the API
* Clean up user devices when deactivating
* Include invites, delete pushers
* Check state before event
* Tweaks
* Refactor a bit, include in output events
* Don't waste time if soft failed either
* Tweak control flow, comments, use GMSL history visibility type
* Rate limiting changes
This makes the following changes:
* For logged in users, the rate limiting now applies to the device session rather than the remote IP address;
* For non-logged in users, the rate limiting continues to apply to remote address as it does today;
* It is now possible to add user IDs to the `exempt_user_ids` option under `rate_limiting` to exclude bots from rate limiting;
* Admin and appservice users are now exempt from rate limiting by default.
* Fix build with media API
* Add `QueryRestrictedJoinAllowed`
* Add `Resident` flag to `QueryRestrictedJoinAllowedResponse`
* Check restricted joins on federation API
* Return `Restricted` to determine if the room was restricted or not
* Populate `AuthorisedVia` properly
* Sign the event on `/send_join`, return it in the `/send_join` response in the `"event"` key
* Kick back joins with invalid authorising user IDs, use event from `"event"` key if returned in `RespSendJoin`
* Use invite helper in `QueryRestrictedJoinAllowed`
* Only use users with the power to invite, change error bubbling a bit
* Placate the almighty linter
One day I will nuke `gocyclo` from orbit and everything in the world will be much better for it.
* Review comments
* tidy up interfaces
* remove unused GetCreatorIDForAlias
* Add RoomserverUserAPI interface
* Define more interfaces
* Use AppServiceInternalAPI for consistent naming
* clean up federationapi constructor a bit
* Fix monolith in -http mode
* Specify interfaces used by appservice, do half of clientapi
* convert more deps of clientapi to finer-grained interfaces
* Convert mediaapi and rest of clientapi
* Somehow this got missed
* Update to matrix-org/gomatrixserverlib#303
* Use `gomatrixserverlib.Client` for phone-home stats
* Use `gomatrixserverlib.Client` for push notifications
* Use `gomatrixserverlib.Client` for appservices
* Use `gomatrixserverlib.Client` for three-PID invites
* syncapi: use finer-grained interfaces when making the syncapi
* Use specific interfaces for syncapi-roomserver interactions
* Define query access token api for shared http auth code
* Simplify federation API `AddPublicRoutes`
* Simplify client API `AddPublicRoutes`
* Simplify media API `AddPublicRoutes`
* Simplify sync API `AddPublicRoutes`
* Simplify `AddAllPublicRoutes`
* Add new endpoint to allow admins to evacuate the local server from the room
* Guard endpoint
* Use right prefix
* Auth API
* More useful return error rather than a panic
* More useful return value again
* Update the path
* Try using inputer instead
* oh provide the config
* Try that again
* Return affected user IDs
* Don't create so many forward extremities
* Add missing `Path` to name
Co-authored-by: Till <2353100+S7evinK@users.noreply.github.com>
* Add response size and requests total to internal handler
* Move MustRegister calls to New* funcs
* Move MustRegister back to init
* Init at some place, minimize changes
* Add ignore users
* Ignore users in pushrules
Add passing tests
* Update sytest lists
* Store ignore knowledge in the sync API
* Fix copyrights
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
* Added /upgrade endpoint
* fix
* Fix lints
* More lint lifex
* Move room upgrading to the roomserver
* Remove extraneous arg
* Fix HTTP API for `PerformUpgrade`
* Reduce number of API calls in `generateInitialEvents`, preserve membership fields
* Refactor `generateInitialEvents` to preserve old state events for all but the essential room setup events
* Handle ban events in the state transfer
* Refactor and comment `createTemporaryPowerLevels`
* Only send two power levels if we needed to override the levels, preserve miscellaneous fields in the create event
* Fix copyrights
* Review comments @S7evinK
* Update sytest whitelist
* Specify empty state keys, use `EventLevel`, remove unnecessary check on state copy
* Add comment to `restrictOldRoomPowerLevels`
* Ensure canonical aliases exist before clearing
* Copy invites as well as bans
* Fix return error on `m.room.tombstone` handling in client API
* Relax checks for well-formedness of join rules, membership event etc
Co-authored-by: Alex Kursell <alex@awk.run>
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
Co-authored-by: kegsay <kegan@matrix.org>
* Move receipt sending to own JetStream producer
* Move SendToDevice to producer
* Remove most parts of the EDU server
* Fix SendToDevice & copyrights
* Move structs, cleanup EDU Server traces
* Use HeadersOnly subscription
* Missing file
* Fix linter issues
* Move consumers to own files
* Rename durable consumer; Consumer cleanup
* Docs/config cleanup
* Allow stored session parameters to be overwritten in the registration request
* Remove logging
* Close request body
* Use `httputil.UnmarshalJSON` as that should enforce UTF-8 correctness
* Return `M_NOT_JSON` on read error
* Whoops, return the value of `httputil.UnmarshalJSON`
* Remove redundant comment
* Refactor user api internal
* Refactor clientapi to use internal userapi
* Use internal userapi instead of user DB directly
* Remove AccountDB dependency
* Fix linter issues
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
