diff --git a/CHANGES.md b/CHANGES.md index ee608194d..eb365baf1 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -1,5 +1,31 @@ # Changelog +## Dendrite 0.6.5 (2022-03-04) + +### Features + +* Early support for push notifications has been added, with support for push rules, pushers, HTTP push gateways and the `/notifications` endpoint (contributions by [danpe](https://github.com/danpe), [PiotrKozimor](https://github.com/PiotrKozimor) and [tommie](https://github.com/tommie)) +* Spaces Summary (MSC2946) is now correctly supported (when `msc2946` is enabled in the config) +* All media API endpoints are now available under the `/v3` namespace +* Profile updates (display name and avatar) are now sent asynchronously so they shouldn't block the client for a very long time +* State resolution v2 has been optimised further to considerably reduce the number of memory allocations +* State resolution v2 will no longer duplicate events unnecessarily when calculating the auth difference +* The `create-account` tool now has a `-reset-password` option for resetting the passwords of existing accounts +* The `/sync` endpoint now calculates device list changes much more quickly with less RAM used +* The `/messages` endpoint now lazy-loads members correctly + +### Fixes + +* Read receipts now work correctly by correcting bugs in the stream positions and receipt coalescing +* Topological sorting of state and join responses has been corrected, which should help to reduce the number of auth problems when joining new federated rooms +* Media thumbnails should now work properly after having unnecessarily strict rate limiting removed +* The roomserver no longer holds transactions for as long when processing input events +* Uploading device keys and cross-signing keys will now correctly no-op if there were no changes +* Parameters are now remembered correctly during registration +* Devices can now only be deleted within the appropriate UIA flow +* The `/context` endpoint now returns 404 instead of 500 if the event was not found +* SQLite mode will no longer leak memory as a result of not closing prepared statements + ## Dendrite 0.6.4 (2022-02-21) ### Features @@ -210,9 +236,9 @@ ### Fixes -- **SECURITY:** A bug in SQLite mode which could cause the registration flow to complete unexpectedly for existing accounts has been fixed (PostgreSQL deployments are not affected) -- A panic in the federation sender has been fixed when shutting down destination queues -- The `/keys/upload` endpoint now correctly returns the number of one-time keys in response to an empty upload request +* **SECURITY:** A bug in SQLite mode which could cause the registration flow to complete unexpectedly for existing accounts has been fixed (PostgreSQL deployments are not affected) +* A panic in the federation sender has been fixed when shutting down destination queues +* The `/keys/upload` endpoint now correctly returns the number of one-time keys in response to an empty upload request ## Dendrite 0.3.10 (2021-02-17) @@ -534,4 +560,4 @@ First versioned release of Dendrite. * Typing: Yes. * Presence: No. * Receipts: No. -* OpenID: No. \ No newline at end of file +* OpenID: No. diff --git a/README.md b/README.md index a077788cf..d3a862587 100644 --- a/README.md +++ b/README.md @@ -2,26 +2,29 @@ Dendrite is a second-generation Matrix homeserver written in Go. It intends to provide an **efficient**, **reliable** and **scalable** alternative to [Synapse](https://github.com/matrix-org/synapse): - - Efficient: A small memory footprint with better baseline performance than an out-of-the-box Synapse. - - Reliable: Implements the Matrix specification as written, using the + +- Efficient: A small memory footprint with better baseline performance than an out-of-the-box Synapse. +- Reliable: Implements the Matrix specification as written, using the [same test suite](https://github.com/matrix-org/sytest) as Synapse as well as a [brand new Go test suite](https://github.com/matrix-org/complement). - - Scalable: can run on multiple machines and eventually scale to massive homeserver deployments. +- Scalable: can run on multiple machines and eventually scale to massive homeserver deployments. As of October 2020, Dendrite has now entered **beta** which means: + - Dendrite is ready for early adopters. We recommend running in Monolith mode with a PostgreSQL database. - Dendrite has periodic semver releases. We intend to release new versions as we land significant features. - Dendrite supports database schema upgrades between releases. This means you should never lose your messages when upgrading Dendrite. - Breaking changes will not occur on minor releases. This means you can safely upgrade Dendrite without modifying your database or config file. This does not mean: - - Dendrite is bug-free. It has not yet been battle-tested in the real world and so will be error prone initially. - - All of the CS/Federation APIs are implemented. We are tracking progress via a script called 'Are We Synapse Yet?'. In particular, + +- Dendrite is bug-free. It has not yet been battle-tested in the real world and so will be error prone initially. +- All of the CS/Federation APIs are implemented. We are tracking progress via a script called 'Are We Synapse Yet?'. In particular, presence and push notifications are entirely missing from Dendrite. See [CHANGES.md](CHANGES.md) for updates. - - Dendrite is ready for massive homeserver deployments. You cannot shard each microservice, only run each one on a different machine. +- Dendrite is ready for massive homeserver deployments. You cannot shard each microservice, only run each one on a different machine. Currently, we expect Dendrite to function well for small (10s/100s of users) homeserver deployments as well as P2P Matrix nodes in-browser or on mobile devices. -In the future, we will be able to scale up to gigantic servers (equivalent to matrix.org) via polylith mode. +In the future, we will be able to scale up to gigantic servers (equivalent to matrix.org) via polylith mode. If you have further questions, please take a look at [our FAQ](docs/FAQ.md) or join us in: @@ -31,14 +34,16 @@ If you have further questions, please take a look at [our FAQ](docs/FAQ.md) or j ## Requirements -To build Dendrite, you will need Go 1.16 or later. +To build Dendrite, you will need Go 1.16 or later. For a usable federating Dendrite deployment, you will also need: -- A domain name (or subdomain) + +- A domain name (or subdomain) - A valid TLS certificate issued by a trusted authority for that domain - SRV records or a well-known file pointing to your deployment Also recommended are: + - A PostgreSQL database engine, which will perform better than SQLite with many users and/or larger rooms - A reverse proxy server, such as nginx, configured [like this sample](https://github.com/matrix-org/dendrite/blob/master/docs/nginx/monolith-sample.conf) @@ -76,30 +81,32 @@ Then point your favourite Matrix client at `http://localhost:8008` or `https://l We use a script called Are We Synapse Yet which checks Sytest compliance rates. Sytest is a black-box homeserver test rig with around 900 tests. The script works out how many of these tests are passing on Dendrite and it -updates with CI. As of January 2022 we're at around 65% CS API coverage and 92% Federation coverage, though check +updates with CI. As of March 2022 we're at around 76% CS API coverage and 95% Federation coverage, though check CI for the latest numbers. In practice, this means you can communicate locally and via federation with Synapse servers such as matrix.org reasonably well. There's a long list of features that are not implemented, notably: - - Push - - Search and Context - - User Directory - - Presence - - Guests + +- Search +- User Directory +- Presence We are prioritising features that will benefit single-user homeservers first (e.g Receipts, E2E) rather than features that massive deployments may be interested in (User Directory, OpenID, Guests, Admin APIs, AS API). This means Dendrite supports amongst others: - - Core room functionality (creating rooms, invites, auth rules) - - Federation in rooms v1-v7 - - Backfilling locally and via federation - - Accounts, Profiles and Devices - - Published room lists - - Typing - - Media APIs - - Redaction - - Tagging - - E2E keys and device lists - - Receipts +- Core room functionality (creating rooms, invites, auth rules) +- Federation in rooms v1-v7 +- Backfilling locally and via federation +- Accounts, Profiles and Devices +- Published room lists +- Typing +- Media APIs +- Redaction +- Tagging +- Context +- E2E keys and device lists +- Receipts +- Push +- Guests ## Contributing @@ -112,6 +119,7 @@ For example, if the test `Local device key changes get to remote servers` was ma test file (e.g via `grep` or via the [CI log output](https://buildkite.com/matrix-dot-org/dendrite/builds/2826#39cff5de-e032-4ad0-ad26-f819e6919c42) it's `tests/50federation/40devicelists.pl` ) then to run Sytest: + ``` docker run --rm --name sytest -v "/Users/kegan/github/sytest:/sytest" @@ -121,10 +129,12 @@ docker run --rm --name sytest -e "POSTGRES=1" -e "DENDRITE_TRACE_HTTP=1" matrixdotorg/sytest-dendrite:latest tests/50federation/40devicelists.pl ``` + See [sytest.md](docs/sytest.md) for the full description of these flags. You can try running sytest outside of docker for faster runs, but the dependencies can be temperamental and we recommend using docker where possible. + ``` cd sytest export PERL5LIB=$HOME/lib/perl5 @@ -149,8 +159,9 @@ Dendrite in Monolith + SQLite works in a range of environments including iOS and For small homeserver installations joined on ~10s rooms on matrix.org with ~100s of users in those rooms, including some encrypted rooms: - - Memory: uses around 100MB of RAM, with peaks at around 200MB. - - Disk space: After a few months of usage, the database grew to around 2GB (in Monolith mode). - - CPU: Brief spikes when processing events, typically idles at 1% CPU. + +- Memory: uses around 100MB of RAM, with peaks at around 200MB. +- Disk space: After a few months of usage, the database grew to around 2GB (in Monolith mode). +- CPU: Brief spikes when processing events, typically idles at 1% CPU. This means Dendrite should comfortably work on things like Raspberry Pis. diff --git a/appservice/consumers/roomserver.go b/appservice/consumers/roomserver.go index 7b59e3704..9d723bed1 100644 --- a/appservice/consumers/roomserver.go +++ b/appservice/consumers/roomserver.go @@ -88,7 +88,16 @@ func (s *OutputRoomEventConsumer) onMessage(ctx context.Context, msg *nats.Msg) } events := []*gomatrixserverlib.HeaderedEvent{output.NewRoomEvent.Event} - events = append(events, output.NewRoomEvent.AddStateEvents...) + if len(output.NewRoomEvent.AddsStateEventIDs) > 0 { + eventsReq := &api.QueryEventsByIDRequest{ + EventIDs: output.NewRoomEvent.AddsStateEventIDs, + } + eventsRes := &api.QueryEventsByIDResponse{} + if err := s.rsAPI.QueryEventsByID(s.ctx, eventsReq, eventsRes); err != nil { + return false + } + events = append(events, eventsRes.Events...) + } // Send event to any relevant application services if err := s.filterRoomserverEvents(context.TODO(), events); err != nil { diff --git a/build/docker/config/dendrite.yaml b/build/docker/config/dendrite.yaml index ebae50132..c01ab62e7 100644 --- a/build/docker/config/dendrite.yaml +++ b/build/docker/config/dendrite.yaml @@ -6,7 +6,7 @@ # # At a minimum, to get started, you will need to update the settings in the # "global" section for your deployment, and you will need to check that the -# database "connection_string" line in each component section is correct. +# database "connection_string" line in each component section is correct. # # Each component with a "database" section can accept the following formats # for "connection_string": @@ -21,13 +21,13 @@ # small number of users and likely will perform worse still with a higher volume # of users. # -# The "max_open_conns" and "max_idle_conns" settings configure the maximum +# The "max_open_conns" and "max_idle_conns" settings configure the maximum # number of open/idle database connections. The value 0 will use the database # engine default, and a negative value will use unlimited connections. The # "conn_max_lifetime" option controls the maximum length of time a database # connection can be idle in seconds - a negative value is unlimited. -# The version of the configuration file. +# The version of the configuration file. version: 2 # Global Matrix configuration. This configuration applies to all components. @@ -59,8 +59,8 @@ global: # Lists of domains that the server will trust as identity servers to verify third # party identifiers such as phone numbers and email addresses. trusted_third_party_id_servers: - - matrix.org - - vector.im + - matrix.org + - vector.im # Configuration for NATS JetStream jetstream: @@ -76,7 +76,7 @@ global: # should not be used on a real world Dendrite deployment. in_memory: false - # Persistent directory to store JetStream streams in. This directory + # Persistent directory to store JetStream streams in. This directory # should be preserved across Dendrite restarts. storage_path: ./ @@ -138,13 +138,13 @@ client_api: # Whether to require reCAPTCHA for registration. enable_registration_captcha: false - # Settings for ReCAPTCHA. + # Settings for ReCAPTCHA. recaptcha_public_key: "" recaptcha_private_key: "" recaptcha_bypass_secret: "" recaptcha_siteverify_api: "" - # TURN server information that this homeserver should send to clients. + # TURN server information that this homeserver should send to clients. turn: turn_user_lifetime: "" turn_uris: [] @@ -153,7 +153,7 @@ client_api: turn_password: "" # Settings for rate-limited endpoints. Rate limiting will kick in after the - # threshold number of "slots" have been taken by requests from a specific + # threshold number of "slots" have been taken by requests from a specific # host. Each "slot" will be released after the cooloff time in milliseconds. rate_limiting: enabled: true @@ -204,13 +204,13 @@ federation_api: # be required to satisfy key requests for servers that are no longer online when # joining some rooms. key_perspectives: - - server_name: matrix.org - keys: - - key_id: ed25519:auto - public_key: Noi6WqcDj0QmPxCNQqgezwTlBKrfqehY1u2FyWP9uYw - - key_id: ed25519:a_RXGa - public_key: l8Hft5qXKn1vfHrg3p4+W8gELQVo8N13JkluMfmn2sQ - + - server_name: matrix.org + keys: + - key_id: ed25519:auto + public_key: Noi6WqcDj0QmPxCNQqgezwTlBKrfqehY1u2FyWP9uYw + - key_id: ed25519:a_RXGa + public_key: l8Hft5qXKn1vfHrg3p4+W8gELQVo8N13JkluMfmn2sQ + # This option will control whether Dendrite will prefer to look up keys directly # or whether it should try perspective servers first, using direct fetches as a # last resort. @@ -255,15 +255,15 @@ media_api: # A list of thumbnail sizes to be generated for media content. thumbnail_sizes: - - width: 32 - height: 32 - method: crop - - width: 96 - height: 96 - method: crop - - width: 640 - height: 480 - method: scale + - width: 32 + height: 32 + method: crop + - width: 96 + height: 96 + method: crop + - width: 640 + height: 480 + method: scale # Configuration for experimental MSC's mscs: @@ -295,7 +295,7 @@ sync_api: listen: http://0.0.0.0:7773 connect: http://sync_api:7773 external_api: - listen: http://0.0.0.0:8073 + listen: http://0.0.0.0:8073 database: connection_string: postgresql://dendrite:itsasecret@postgres/dendrite_syncapi?sslmode=disable max_open_conns: 10 @@ -312,11 +312,6 @@ user_api: max_open_conns: 10 max_idle_conns: 2 conn_max_lifetime: -1 - device_database: - connection_string: postgresql://dendrite:itsasecret@postgres/dendrite_userapi_devices?sslmode=disable - max_open_conns: 10 - max_idle_conns: 2 - conn_max_lifetime: -1 # Configuration for the Push Server API. push_server: @@ -348,7 +343,7 @@ tracing: # Logging configuration, in addition to the standard logging that is sent to # stdout by Dendrite. logging: -- type: file - level: info - params: - path: /var/log/dendrite + - type: file + level: info + params: + path: /var/log/dendrite diff --git a/build/docker/postgres/create_db.sh b/build/docker/postgres/create_db.sh index 9093738e4..27d2a4df4 100755 --- a/build/docker/postgres/create_db.sh +++ b/build/docker/postgres/create_db.sh @@ -1,5 +1,5 @@ #!/bin/sh -for db in userapi_accounts userapi_devices mediaapi syncapi roomserver keyserver federationapi appservice mscs; do +for db in userapi_accounts mediaapi syncapi roomserver keyserver federationapi appservice mscs; do createdb -U dendrite -O dendrite dendrite_$db done diff --git a/clientapi/jsonerror/jsonerror.go b/clientapi/jsonerror/jsonerror.go index 97c597030..1fc1c0c01 100644 --- a/clientapi/jsonerror/jsonerror.go +++ b/clientapi/jsonerror/jsonerror.go @@ -58,6 +58,11 @@ func BadJSON(msg string) *MatrixError { return &MatrixError{"M_BAD_JSON", msg} } +// BadAlias is an error when the client supplies a bad alias. +func BadAlias(msg string) *MatrixError { + return &MatrixError{"M_BAD_ALIAS", msg} +} + // NotJSON is an error when the client supplies something that is not JSON // to a JSON endpoint. func NotJSON(msg string) *MatrixError { diff --git a/clientapi/routing/directory.go b/clientapi/routing/directory.go index e408c264f..ac355b5d4 100644 --- a/clientapi/routing/directory.go +++ b/clientapi/routing/directory.go @@ -139,11 +139,17 @@ func SetLocalAlias( // TODO: This code should eventually be refactored with: // 1. The new method for checking for things matching an AS's namespace // 2. Using an overall Regex object for all AS's just like we did for usernames - + reqUserID, _, err := gomatrixserverlib.SplitID('@', device.UserID) + if err != nil { + return util.JSONResponse{ + Code: http.StatusBadRequest, + JSON: jsonerror.BadJSON("User ID must be in the form '@localpart:domain'"), + } + } for _, appservice := range cfg.Derived.ApplicationServices { // Don't prevent AS from creating aliases in its own namespace // Note that Dendrite uses SenderLocalpart as UserID for AS users - if device.UserID != appservice.SenderLocalpart { + if reqUserID != appservice.SenderLocalpart { if aliasNamespaces, ok := appservice.NamespaceMap["aliases"]; ok { for _, namespace := range aliasNamespaces { if namespace.Exclusive && namespace.RegexpObject.MatchString(alias) { diff --git a/clientapi/routing/sendevent.go b/clientapi/routing/sendevent.go index 23935b5d9..3d5993718 100644 --- a/clientapi/routing/sendevent.go +++ b/clientapi/routing/sendevent.go @@ -16,6 +16,8 @@ package routing import ( "context" + "encoding/json" + "fmt" "net/http" "sync" "time" @@ -120,6 +122,40 @@ func SendEvent( } timeToGenerateEvent := time.Since(startedGeneratingEvent) + // validate that the aliases exists + if eventType == gomatrixserverlib.MRoomCanonicalAlias && stateKey != nil && *stateKey == "" { + aliasReq := api.AliasEvent{} + if err = json.Unmarshal(e.Content(), &aliasReq); err != nil { + return util.ErrorResponse(fmt.Errorf("unable to parse alias event: %w", err)) + } + if !aliasReq.Valid() { + return util.JSONResponse{ + Code: http.StatusBadRequest, + JSON: jsonerror.InvalidParam("Request contains invalid aliases."), + } + } + aliasRes := &api.GetAliasesForRoomIDResponse{} + if err = rsAPI.GetAliasesForRoomID(req.Context(), &api.GetAliasesForRoomIDRequest{RoomID: roomID}, aliasRes); err != nil { + return jsonerror.InternalServerError() + } + var found int + requestAliases := append(aliasReq.AltAliases, aliasReq.Alias) + for _, alias := range aliasRes.Aliases { + for _, altAlias := range requestAliases { + if altAlias == alias { + found++ + } + } + } + // check that we found at least the same amount of existing aliases as are in the request + if aliasReq.Alias != "" && found < len(requestAliases) { + return util.JSONResponse{ + Code: http.StatusBadRequest, + JSON: jsonerror.BadAlias("No matching alias found."), + } + } + } + var txnAndSessionID *api.TransactionID if txnID != nil { txnAndSessionID = &api.TransactionID{ diff --git a/docs/INSTALL.md b/docs/INSTALL.md index 686ae1dbb..a7b2e67f2 100644 --- a/docs/INSTALL.md +++ b/docs/INSTALL.md @@ -87,7 +87,7 @@ On macOS, omit `sudo -u postgres` from the below commands. * If you want to run each Dendrite component with its own database: ```bash - for i in mediaapi syncapi roomserver federationapi appservice keyserver userapi_accounts userapi_devices; do + for i in mediaapi syncapi roomserver federationapi appservice keyserver userapi_accounts; do sudo -u postgres createdb -O dendrite dendrite_$i done ``` diff --git a/federationapi/consumers/roomserver.go b/federationapi/consumers/roomserver.go index 173dcff01..989f7cf49 100644 --- a/federationapi/consumers/roomserver.go +++ b/federationapi/consumers/roomserver.go @@ -146,7 +146,28 @@ func (s *OutputRoomEventConsumer) processInboundPeek(orp api.OutputNewInboundPee // processMessage updates the list of currently joined hosts in the room // and then sends the event to the hosts that were joined before the event. func (s *OutputRoomEventConsumer) processMessage(ore api.OutputNewRoomEvent) error { - addsJoinedHosts, err := joinedHostsFromEvents(gomatrixserverlib.UnwrapEventHeaders(ore.AddsState())) + eventsRes := &api.QueryEventsByIDResponse{} + if len(ore.AddsStateEventIDs) > 0 { + eventsReq := &api.QueryEventsByIDRequest{ + EventIDs: ore.AddsStateEventIDs, + } + if err := s.rsAPI.QueryEventsByID(s.ctx, eventsReq, eventsRes); err != nil { + return fmt.Errorf("s.rsAPI.QueryEventsByID: %w", err) + } + + found := false + for _, event := range eventsRes.Events { + if event.EventID() == ore.Event.EventID() { + found = true + break + } + } + if !found { + eventsRes.Events = append(eventsRes.Events, ore.Event) + } + } + + addsJoinedHosts, err := joinedHostsFromEvents(gomatrixserverlib.UnwrapEventHeaders(eventsRes.Events)) if err != nil { return err } diff --git a/go.mod b/go.mod index 525950daa..9957a5b6c 100644 --- a/go.mod +++ b/go.mod @@ -1,6 +1,6 @@ module github.com/matrix-org/dendrite -replace github.com/nats-io/nats-server/v2 => github.com/neilalexander/nats-server/v2 v2.7.4-0.20220302103432-6b04b9f12740 +replace github.com/nats-io/nats-server/v2 => github.com/neilalexander/nats-server/v2 v2.7.2-0.20220217100407-087330ed46ad replace github.com/nats-io/nats.go => github.com/neilalexander/nats.go v1.11.1-0.20220104162523-f4ddebe1061c @@ -41,7 +41,7 @@ require ( github.com/matrix-org/go-sqlite3-js v0.0.0-20210709140738-b0d1ba599a6d github.com/matrix-org/gomatrix v0.0.0-20210324163249-be2af5ef2e16 github.com/matrix-org/gomatrixserverlib v0.0.0-20220301141554-e124bd7d7902 - github.com/matrix-org/pinecone v0.0.0-20220223104432-0f0afd1a46aa + github.com/matrix-org/pinecone v0.0.0-20220308124038-cfde1f8054c5 github.com/matrix-org/util v0.0.0-20200807132607-55161520e1d4 github.com/mattn/go-sqlite3 v1.14.10 github.com/morikuni/aec v1.0.0 // indirect diff --git a/go.sum b/go.sum index 25697f59c..951c0eba9 100644 --- a/go.sum +++ b/go.sum @@ -480,6 +480,8 @@ github.com/golang/protobuf v1.5.0/go.mod h1:FsONVRAS9T7sI+LIUmWTfcYkHO4aIWwzhcaS github.com/golang/protobuf v1.5.2 h1:ROPKBNFfQgOUMifHyP+KYbvpjbdoFNs+aK7DXlji0Tw= github.com/golang/protobuf v1.5.2/go.mod h1:XVQd3VNwM+JqD3oG2Ue2ip4fOMUkwXdXDdiuN0vRsmY= github.com/golang/snappy v0.0.0-20180518054509-2e65f85255db/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q= +github.com/golang/snappy v0.0.3 h1:fHPg5GQYlCeLIPB9BZqMVR5nR9A+IM5zcgeTdjMYmLA= +github.com/golang/snappy v0.0.3/go.mod h1:/XxbfmMg8lxefKM7IXC3fBNl/7bRcc72aCRzEWrmP2Q= github.com/gologme/log v1.2.0/go.mod h1:gq31gQ8wEHkR+WekdWsqDuf8pXTUZA9BnnzTuPz1Y9U= github.com/gologme/log v1.3.0 h1:l781G4dE+pbigClDSDzSaaYKtiueHCILUa/qSDsmHAo= github.com/gologme/log v1.3.0/go.mod h1:yKT+DvIPdDdDoPtqFrFxheooyVmoqi0BAsw+erN3wA4= @@ -711,8 +713,8 @@ github.com/klauspost/compress v1.9.7/go.mod h1:RyIbtBH6LamlWaDj8nUwkbUhJ87Yi3uG0 github.com/klauspost/compress v1.10.3/go.mod h1:aoV0uJVorq1K+umq18yTdKaF57EivdYsUV+/s2qKfXs= github.com/klauspost/compress v1.11.3/go.mod h1:aoV0uJVorq1K+umq18yTdKaF57EivdYsUV+/s2qKfXs= github.com/klauspost/compress v1.11.13/go.mod h1:aoV0uJVorq1K+umq18yTdKaF57EivdYsUV+/s2qKfXs= -github.com/klauspost/compress v1.14.4 h1:eijASRJcobkVtSt81Olfh7JX43osYLwy5krOJo6YEu4= -github.com/klauspost/compress v1.14.4/go.mod h1:/3/Vjq9QcHkK5uEr5lBEmyoZ1iFhe47etQ6QUkpK6sk= +github.com/klauspost/compress v1.13.4 h1:0zhec2I8zGnjWcKyLl6i3gPqKANCCn5e9xmviEEeX6s= +github.com/klauspost/compress v1.13.4/go.mod h1:8dP1Hq4DHOhN9w426knH3Rhby4rFm6D8eO+e+Dq5Gzg= github.com/klauspost/cpuid v1.2.1/go.mod h1:Pj4uuM528wm8OyEC2QMXAi2YiTZ96dNQPGgoMS4s3ek= github.com/konsorten/go-windows-terminal-sequences v1.0.1/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= github.com/konsorten/go-windows-terminal-sequences v1.0.2/go.mod h1:T0+1ngSBFLxvqU3pZ+m/2kptfBszLMUkC4ZK/EgS/cQ= @@ -983,8 +985,8 @@ github.com/matrix-org/gomatrix v0.0.0-20210324163249-be2af5ef2e16 h1:ZtO5uywdd5d github.com/matrix-org/gomatrix v0.0.0-20210324163249-be2af5ef2e16/go.mod h1:/gBX06Kw0exX1HrwmoBibFA98yBk/jxKpGVeyQbff+s= github.com/matrix-org/gomatrixserverlib v0.0.0-20220301141554-e124bd7d7902 h1:WHlrE8BYh/hzn1RKwq3YMAlhHivX47jQKAjZFtkJyPE= github.com/matrix-org/gomatrixserverlib v0.0.0-20220301141554-e124bd7d7902/go.mod h1:+WF5InseAMgi1fTnU46JH39IDpEvLep0fDzx9LDf2Bo= -github.com/matrix-org/pinecone v0.0.0-20220223104432-0f0afd1a46aa h1:rMYFNVto66gp+eWS8XAUzgp4m0qmUBid6l1HX3mHstk= -github.com/matrix-org/pinecone v0.0.0-20220223104432-0f0afd1a46aa/go.mod h1:r6dsL+ylE0yXe/7zh8y/Bdh6aBYI1r+u4yZni9A4iyk= +github.com/matrix-org/pinecone v0.0.0-20220308124038-cfde1f8054c5 h1:7viLTiLAA2MtGKY+uf14j6TjfKvvGLAMj/qdm70jJuQ= +github.com/matrix-org/pinecone v0.0.0-20220308124038-cfde1f8054c5/go.mod h1:r6dsL+ylE0yXe/7zh8y/Bdh6aBYI1r+u4yZni9A4iyk= github.com/matrix-org/util v0.0.0-20190711121626-527ce5ddefc7/go.mod h1:vVQlW/emklohkZnOPwD3LrZUBqdfsbiyO3p1lNV8F6U= github.com/matrix-org/util v0.0.0-20200807132607-55161520e1d4 h1:eCEHXWDv9Rm335MSuB49mFUK44bwZPFSDde3ORE3syk= github.com/matrix-org/util v0.0.0-20200807132607-55161520e1d4/go.mod h1:vVQlW/emklohkZnOPwD3LrZUBqdfsbiyO3p1lNV8F6U= @@ -1027,8 +1029,8 @@ github.com/miekg/dns v1.1.31/go.mod h1:KNUDUusw/aVsxyTYZM1oqvCicbwhgbNgztCETuNZ7 github.com/miekg/pkcs11 v1.0.3/go.mod h1:XsNlhZGX73bx86s2hdc/FuaLm2CPZJemRLMA+WTFxgs= github.com/minio/blake2b-simd v0.0.0-20160723061019-3f5f724cb5b1 h1:lYpkrQH5ajf0OXOcUbGjvZxxijuBwbbmlSxLiuofa+g= github.com/minio/blake2b-simd v0.0.0-20160723061019-3f5f724cb5b1/go.mod h1:pD8RvIylQ358TN4wwqatJ8rNavkEINozVn9DtGI3dfQ= -github.com/minio/highwayhash v1.0.2 h1:Aak5U0nElisjDCfPSG79Tgzkn2gl66NxOMspRrKnA/g= -github.com/minio/highwayhash v1.0.2/go.mod h1:BQskDq+xkJ12lmlUUi7U0M5Swg3EWR+dLTk+kldvVxY= +github.com/minio/highwayhash v1.0.1 h1:dZ6IIu8Z14VlC0VpfKofAhCy74wu/Qb5gcn52yWoz/0= +github.com/minio/highwayhash v1.0.1/go.mod h1:BQskDq+xkJ12lmlUUi7U0M5Swg3EWR+dLTk+kldvVxY= github.com/minio/sha256-simd v0.0.0-20190131020904-2d45a736cd16/go.mod h1:2FMWW+8GMoPweT6+pI63m9YE3Lmw4J71hV56Chs1E/U= github.com/minio/sha256-simd v0.0.0-20190328051042-05b4dd3047e5/go.mod h1:2FMWW+8GMoPweT6+pI63m9YE3Lmw4J71hV56Chs1E/U= github.com/minio/sha256-simd v0.1.0/go.mod h1:2FMWW+8GMoPweT6+pI63m9YE3Lmw4J71hV56Chs1E/U= @@ -1130,8 +1132,8 @@ github.com/nbio/st v0.0.0-20140626010706-e9e8d9816f32/go.mod h1:9wM+0iRr9ahx58uY github.com/ncw/swift v1.0.47/go.mod h1:23YIA4yWVnGwv2dQlN4bB7egfYX6YLn0Yo/S6zZO/ZM= github.com/neelance/astrewrite v0.0.0-20160511093645-99348263ae86/go.mod h1:kHJEU3ofeGjhHklVoIGuVj85JJwZ6kWPaJwCIxgnFmo= github.com/neelance/sourcemap v0.0.0-20151028013722-8c68805598ab/go.mod h1:Qr6/a/Q4r9LP1IltGz7tA7iOK1WonHEYhu1HRBA7ZiM= -github.com/neilalexander/nats-server/v2 v2.7.4-0.20220302103432-6b04b9f12740 h1:RJrc+z35RHZlrjR6UBt9UmVRAlFh4SgYyEA0YpQdPHM= -github.com/neilalexander/nats-server/v2 v2.7.4-0.20220302103432-6b04b9f12740/go.mod h1:eJUrA5gm0ch6sJTEv85xmXIgQWsB0OyjkTsKXvlHbYc= +github.com/neilalexander/nats-server/v2 v2.7.2-0.20220217100407-087330ed46ad h1:Z2nWMQsXWWqzj89nW6OaLJSdkFknqhaR5whEOz4++Y8= +github.com/neilalexander/nats-server/v2 v2.7.2-0.20220217100407-087330ed46ad/go.mod h1:tckmrt0M6bVaDT3kmh9UrIq/CBOBBse+TpXQi5ldaa8= github.com/neilalexander/nats.go v1.11.1-0.20220104162523-f4ddebe1061c h1:G2qsv7D0rY94HAu8pXmElMluuMHQ85waxIDQBhIzV2Q= github.com/neilalexander/nats.go v1.11.1-0.20220104162523-f4ddebe1061c/go.mod h1:BPko4oXsySz4aSWeFgOHLZs3G4Jq4ZAyE6/zMCxRT6w= github.com/neilalexander/utp v0.1.1-0.20210622132614-ee9a34a30488/go.mod h1:NPHGhPc0/wudcaCqL/H5AOddkRf8GPRhzOujuUKGQu8= diff --git a/internal/version.go b/internal/version.go index 2ea1c5201..2e8de4175 100644 --- a/internal/version.go +++ b/internal/version.go @@ -17,7 +17,7 @@ var build string const ( VersionMajor = 0 VersionMinor = 6 - VersionPatch = 4 + VersionPatch = 5 VersionTag = "" // example: "rc1" ) diff --git a/mediaapi/routing/routing.go b/mediaapi/routing/routing.go index fc2136bb6..0e1583991 100644 --- a/mediaapi/routing/routing.go +++ b/mediaapi/routing/routing.go @@ -120,13 +120,16 @@ func makeDownloadAPI( w.Header().Set("Content-Type", "application/json") // Ratelimit requests - if r := rateLimits.Limit(req); r != nil { - if err := json.NewEncoder(w).Encode(r); err != nil { - w.WriteHeader(http.StatusInternalServerError) + // NOTSPEC: The spec says everything at /media/ should be rate limited, but this causes issues with thumbnails (#2243) + if name != "thumbnail" { + if r := rateLimits.Limit(req); r != nil { + if err := json.NewEncoder(w).Encode(r); err != nil { + w.WriteHeader(http.StatusInternalServerError) + return + } + w.WriteHeader(http.StatusTooManyRequests) return } - w.WriteHeader(http.StatusTooManyRequests) - return } vars, _ := httputil.URLDecodeMapValues(mux.Vars(req)) diff --git a/roomserver/api/alias.go b/roomserver/api/alias.go index df69e5b4d..be37333b6 100644 --- a/roomserver/api/alias.go +++ b/roomserver/api/alias.go @@ -14,6 +14,8 @@ package api +import "regexp" + // SetRoomAliasRequest is a request to SetRoomAlias type SetRoomAliasRequest struct { // ID of the user setting the alias @@ -84,3 +86,20 @@ type RemoveRoomAliasResponse struct { // Did we remove it? Removed bool `json:"removed"` } + +type AliasEvent struct { + Alias string `json:"alias"` + AltAliases []string `json:"alt_aliases"` +} + +var validateAliasRegex = regexp.MustCompile("^#.*:.+$") + +func (a AliasEvent) Valid() bool { + for _, alias := range a.AltAliases { + if !validateAliasRegex.MatchString(alias) { + return false + } + } + return a.Alias == "" || validateAliasRegex.MatchString(a.Alias) +} + diff --git a/roomserver/api/alias_test.go b/roomserver/api/alias_test.go new file mode 100644 index 000000000..680493b7b --- /dev/null +++ b/roomserver/api/alias_test.go @@ -0,0 +1,62 @@ +package api + +import "testing" + +func TestAliasEvent_Valid(t *testing.T) { + type fields struct { + Alias string + AltAliases []string + } + tests := []struct { + name string + fields fields + want bool + }{ + { + name: "empty alias", + fields: fields{ + Alias: "", + }, + want: true, + }, + { + name: "empty alias, invalid alt aliases", + fields: fields{ + Alias: "", + AltAliases: []string{ "%not:valid.local"}, + }, + }, + { + name: "valid alias, invalid alt aliases", + fields: fields{ + Alias: "#valid:test.local", + AltAliases: []string{ "%not:valid.local"}, + }, + }, + { + name: "empty alias, invalid alt aliases", + fields: fields{ + Alias: "", + AltAliases: []string{ "%not:valid.local"}, + }, + }, + { + name: "invalid alias", + fields: fields{ + Alias: "%not:valid.local", + AltAliases: []string{ }, + }, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + a := AliasEvent{ + Alias: tt.fields.Alias, + AltAliases: tt.fields.AltAliases, + } + if got := a.Valid(); got != tt.want { + t.Errorf("Valid() = %v, want %v", got, tt.want) + } + }) + } +} diff --git a/roomserver/api/output.go b/roomserver/api/output.go index d60d1cc86..767611ec4 100644 --- a/roomserver/api/output.go +++ b/roomserver/api/output.go @@ -105,7 +105,7 @@ type OutputNewRoomEvent struct { Event *gomatrixserverlib.HeaderedEvent `json:"event"` // Does the event completely rewrite the room state? If so, then AddsStateEventIDs // will contain the entire room state. - RewritesState bool `json:"rewrites_state"` + RewritesState bool `json:"rewrites_state,omitempty"` // The latest events in the room after this event. // This can be used to set the prev events for new events in the room. // This also can be used to get the full current state after this event. @@ -113,16 +113,9 @@ type OutputNewRoomEvent struct { // The state event IDs that were added to the state of the room by this event. // Together with RemovesStateEventIDs this allows the receiver to keep an up to date // view of the current state of the room. - AddsStateEventIDs []string `json:"adds_state_event_ids"` - // All extra newly added state events. This is only set if there are *extra* events - // other than `Event`. This can happen when forks get merged because state resolution - // may decide a bunch of state events on one branch are now valid, so they will be - // present in this list. This is useful when trying to maintain the current state of a room - // as to do so you need to include both these events and `Event`. - AddStateEvents []*gomatrixserverlib.HeaderedEvent `json:"adds_state_events"` - + AddsStateEventIDs []string `json:"adds_state_event_ids,omitempty"` // The state event IDs that were removed from the state of the room by this event. - RemovesStateEventIDs []string `json:"removes_state_event_ids"` + RemovesStateEventIDs []string `json:"removes_state_event_ids,omitempty"` // The ID of the event that was output before this event. // Or the empty string if this is the first event output for this room. // This is used by consumers to check if they can safely update their @@ -145,10 +138,10 @@ type OutputNewRoomEvent struct { // // The state is given as a delta against the current state because they are // usually either the same state, or differ by just a couple of events. - StateBeforeAddsEventIDs []string `json:"state_before_adds_event_ids"` + StateBeforeAddsEventIDs []string `json:"state_before_adds_event_ids,omitempty"` // The state event IDs that are part of the current state, but not part // of the state at the event. - StateBeforeRemovesEventIDs []string `json:"state_before_removes_event_ids"` + StateBeforeRemovesEventIDs []string `json:"state_before_removes_event_ids,omitempty"` // The server name to use to push this event to other servers. // Or empty if this event shouldn't be pushed to other servers. // @@ -167,27 +160,7 @@ type OutputNewRoomEvent struct { SendAsServer string `json:"send_as_server"` // The transaction ID of the send request if sent by a local user and one // was specified - TransactionID *TransactionID `json:"transaction_id"` -} - -// AddsState returns all added state events from this event. -// -// This function is needed because `AddStateEvents` will not include a copy of -// the original event to save space, so you cannot use that slice alone. -// Instead, use this function which will add the original event if it is present -// in `AddsStateEventIDs`. -func (ore *OutputNewRoomEvent) AddsState() []*gomatrixserverlib.HeaderedEvent { - includeOutputEvent := false - for _, id := range ore.AddsStateEventIDs { - if id == ore.Event.EventID() { - includeOutputEvent = true - break - } - } - if !includeOutputEvent { - return ore.AddStateEvents - } - return append(ore.AddStateEvents, ore.Event) + TransactionID *TransactionID `json:"transaction_id,omitempty"` } // An OutputOldRoomEvent is written when the roomserver receives an old event. diff --git a/roomserver/internal/alias.go b/roomserver/internal/alias.go index 7995279d2..5c1c04f01 100644 --- a/roomserver/internal/alias.go +++ b/roomserver/internal/alias.go @@ -16,12 +16,18 @@ package internal import ( "context" + "database/sql" + "errors" "fmt" - - "github.com/matrix-org/dendrite/roomserver/api" - "github.com/matrix-org/gomatrixserverlib" + "time" asAPI "github.com/matrix-org/dendrite/appservice/api" + "github.com/matrix-org/dendrite/internal/eventutil" + "github.com/matrix-org/dendrite/roomserver/api" + "github.com/matrix-org/dendrite/roomserver/internal/helpers" + "github.com/matrix-org/gomatrixserverlib" + "github.com/tidwall/gjson" + "github.com/tidwall/sjson" ) // RoomserverInternalAPIDatabase has the storage APIs needed to implement the alias API. @@ -183,6 +189,57 @@ func (r *RoomserverInternalAPI) RemoveRoomAlias( } } + ev, err := r.DB.GetStateEvent(ctx, roomID, gomatrixserverlib.MRoomCanonicalAlias, "") + if err != nil && err != sql.ErrNoRows { + return err + } else if ev != nil { + stateAlias := gjson.GetBytes(ev.Content(), "alias").Str + // the alias to remove is currently set as the canonical alias, remove it + if stateAlias == request.Alias { + res, err := sjson.DeleteBytes(ev.Content(), "alias") + if err != nil { + return err + } + + sender := request.UserID + if request.UserID != ev.Sender() { + sender = ev.Sender() + } + + builder := &gomatrixserverlib.EventBuilder{ + Sender: sender, + RoomID: ev.RoomID(), + Type: ev.Type(), + StateKey: ev.StateKey(), + Content: res, + } + + eventsNeeded, err := gomatrixserverlib.StateNeededForEventBuilder(builder) + if err != nil { + return fmt.Errorf("gomatrixserverlib.StateNeededForEventBuilder: %w", err) + } + if len(eventsNeeded.Tuples()) == 0 { + return errors.New("expecting state tuples for event builder, got none") + } + + stateRes := &api.QueryLatestEventsAndStateResponse{} + if err := helpers.QueryLatestEventsAndState(ctx, r.DB, &api.QueryLatestEventsAndStateRequest{RoomID: roomID, StateToFetch: eventsNeeded.Tuples()}, stateRes); err != nil { + return err + } + + newEvent, err := eventutil.BuildEvent(ctx, builder, r.Cfg.Matrix, time.Now(), &eventsNeeded, stateRes) + if err != nil { + return err + } + + err = api.SendEvents(ctx, r.RSAPI, api.KindNew, []*gomatrixserverlib.HeaderedEvent{newEvent}, r.ServerName, r.ServerName, nil, false) + if err != nil { + return err + } + + } + } + // Remove the alias from the database if err := r.DB.RemoveRoomAlias(ctx, request.Alias); err != nil { return err diff --git a/roomserver/internal/input/input_events.go b/roomserver/internal/input/input_events.go index 531d6959e..1acbf0443 100644 --- a/roomserver/internal/input/input_events.go +++ b/roomserver/internal/input/input_events.go @@ -294,7 +294,7 @@ func (r *Inputer) processRoomEvent( } // Store the event. - _, _, stateAtEvent, redactionEvent, redactedEventID, err := r.DB.StoreEvent(ctx, event, authEventNIDs, isRejected) + _, _, stateAtEvent, redactionEvent, redactedEventID, err := r.DB.StoreEvent(ctx, event, authEventNIDs, isRejected || softfail) if err != nil { return fmt.Errorf("updater.StoreEvent: %w", err) } diff --git a/roomserver/internal/input/input_latest_events.go b/roomserver/internal/input/input_latest_events.go index f4a52031a..7e58ef9d0 100644 --- a/roomserver/internal/input/input_latest_events.go +++ b/roomserver/internal/input/input_latest_events.go @@ -365,6 +365,7 @@ func (u *latestEventsUpdater) makeOutputNewRoomEvent() (*api.OutputEvent, error) LastSentEventID: u.lastEventIDSent, LatestEventIDs: latestEventIDs, TransactionID: u.transactionID, + SendAsServer: u.sendAsServer, } eventIDMap, err := u.stateEventMap() @@ -384,51 +385,17 @@ func (u *latestEventsUpdater) makeOutputNewRoomEvent() (*api.OutputEvent, error) ore.StateBeforeAddsEventIDs = append(ore.StateBeforeAddsEventIDs, eventIDMap[entry.EventNID]) } - ore.SendAsServer = u.sendAsServer - - // include extra state events if they were added as nearly every downstream component will care about it - // and we'd rather not have them all hit QueryEventsByID at the same time! - if len(ore.AddsStateEventIDs) > 0 { - var err error - if ore.AddStateEvents, err = u.extraEventsForIDs(u.roomInfo.RoomVersion, ore.AddsStateEventIDs); err != nil { - return nil, fmt.Errorf("failed to load add_state_events from db: %w", err) - } - } - return &api.OutputEvent{ Type: api.OutputTypeNewRoomEvent, NewRoomEvent: &ore, }, nil } -// extraEventsForIDs returns the full events for the event IDs given, but does not include the current event being -// updated. -func (u *latestEventsUpdater) extraEventsForIDs(roomVersion gomatrixserverlib.RoomVersion, eventIDs []string) ([]*gomatrixserverlib.HeaderedEvent, error) { - var extraEventIDs []string - for _, e := range eventIDs { - if e == u.event.EventID() { - continue - } - extraEventIDs = append(extraEventIDs, e) - } - if len(extraEventIDs) == 0 { - return nil, nil - } - extraEvents, err := u.updater.UnsentEventsFromIDs(u.ctx, extraEventIDs) - if err != nil { - return nil, err - } - var h []*gomatrixserverlib.HeaderedEvent - for _, e := range extraEvents { - h = append(h, e.Headered(roomVersion)) - } - return h, nil -} - // retrieve an event nid -> event ID map for all events that need updating func (u *latestEventsUpdater) stateEventMap() (map[types.EventNID]string, error) { - var stateEventNIDs []types.EventNID - var allStateEntries []types.StateEntry + cap := len(u.added) + len(u.removed) + len(u.stateBeforeEventRemoves) + len(u.stateBeforeEventAdds) + stateEventNIDs := make(types.EventNIDs, 0, cap) + allStateEntries := make([]types.StateEntry, 0, cap) allStateEntries = append(allStateEntries, u.added...) allStateEntries = append(allStateEntries, u.removed...) allStateEntries = append(allStateEntries, u.stateBeforeEventRemoves...) @@ -436,12 +403,6 @@ func (u *latestEventsUpdater) stateEventMap() (map[types.EventNID]string, error) for _, entry := range allStateEntries { stateEventNIDs = append(stateEventNIDs, entry.EventNID) } - stateEventNIDs = stateEventNIDs[:util.SortAndUnique(eventNIDSorter(stateEventNIDs))] + stateEventNIDs = stateEventNIDs[:util.SortAndUnique(stateEventNIDs)] return u.updater.EventIDs(u.ctx, stateEventNIDs) } - -type eventNIDSorter []types.EventNID - -func (s eventNIDSorter) Len() int { return len(s) } -func (s eventNIDSorter) Less(i, j int) bool { return s[i] < s[j] } -func (s eventNIDSorter) Swap(i, j int) { s[i], s[j] = s[j], s[i] } diff --git a/roomserver/internal/query/query.go b/roomserver/internal/query/query.go index 70cc5d62c..471c6fb42 100644 --- a/roomserver/internal/query/query.go +++ b/roomserver/internal/query/query.go @@ -610,6 +610,14 @@ func (r *Queryer) QueryPublishedRooms( req *api.QueryPublishedRoomsRequest, res *api.QueryPublishedRoomsResponse, ) error { + if req.RoomID != "" { + visible, err := r.DB.GetPublishedRoom(ctx, req.RoomID) + if err == nil && visible { + res.RoomIDs = []string{req.RoomID} + return nil + } + return err + } rooms, err := r.DB.GetPublishedRooms(ctx) if err != nil { return err diff --git a/roomserver/storage/interface.go b/roomserver/storage/interface.go index a2b22b401..a98fda073 100644 --- a/roomserver/storage/interface.go +++ b/roomserver/storage/interface.go @@ -139,6 +139,8 @@ type Database interface { PublishRoom(ctx context.Context, roomID string, publish bool) error // Returns a list of room IDs for rooms which are published. GetPublishedRooms(ctx context.Context) ([]string, error) + // Returns whether a given room is published or not. + GetPublishedRoom(ctx context.Context, roomID string) (bool, error) // TODO: factor out - from currentstateserver diff --git a/roomserver/storage/shared/storage.go b/roomserver/storage/shared/storage.go index dd49f35ca..68fd3867c 100644 --- a/roomserver/storage/shared/storage.go +++ b/roomserver/storage/shared/storage.go @@ -669,6 +669,10 @@ func (d *Database) PublishRoom(ctx context.Context, roomID string, publish bool) }) } +func (d *Database) GetPublishedRoom(ctx context.Context, roomID string) (bool, error) { + return d.PublishedTable.SelectPublishedFromRoomID(ctx, nil, roomID) +} + func (d *Database) GetPublishedRooms(ctx context.Context) ([]string, error) { return d.PublishedTable.SelectAllPublishedRooms(ctx, nil, true) } diff --git a/roomserver/storage/sqlite3/event_state_keys_table.go b/roomserver/storage/sqlite3/event_state_keys_table.go index bf12d5b83..8af40024a 100644 --- a/roomserver/storage/sqlite3/event_state_keys_table.go +++ b/roomserver/storage/sqlite3/event_state_keys_table.go @@ -154,6 +154,7 @@ func (s *eventStateKeyStatements) BulkSelectEventStateKey( if err != nil { return nil, err } + defer selectPrep.Close() stmt := sqlutil.TxStmt(txn, selectPrep) rows, err := stmt.QueryContext(ctx, iEventStateKeyNIDs...) if err != nil { diff --git a/roomserver/storage/sqlite3/event_types_table.go b/roomserver/storage/sqlite3/event_types_table.go index f2c9c42fe..f794a3d0e 100644 --- a/roomserver/storage/sqlite3/event_types_table.go +++ b/roomserver/storage/sqlite3/event_types_table.go @@ -140,6 +140,7 @@ func (s *eventTypeStatements) BulkSelectEventTypeNID( if err != nil { return nil, err } + defer selectPrep.Close() stmt := sqlutil.TxStmt(txn, selectPrep) /////////////// diff --git a/roomserver/storage/sqlite3/events_table.go b/roomserver/storage/sqlite3/events_table.go index 969a10ce5..2ab1151d5 100644 --- a/roomserver/storage/sqlite3/events_table.go +++ b/roomserver/storage/sqlite3/events_table.go @@ -198,11 +198,12 @@ func (s *eventStatements) BulkSelectStateEventByID( iEventIDs[k] = v } selectOrig := strings.Replace(bulkSelectStateEventByIDSQL, "($1)", sqlutil.QueryVariadic(len(iEventIDs)), 1) - selectStmt, err := s.db.Prepare(selectOrig) + selectPrep, err := s.db.Prepare(selectOrig) if err != nil { return nil, err } - selectStmt = sqlutil.TxStmt(txn, selectStmt) + defer selectPrep.Close() // nolint:errcheck + selectStmt := sqlutil.TxStmt(txn, selectPrep) /////////////// rows, err := selectStmt.QueryContext(ctx, iEventIDs...) @@ -266,11 +267,12 @@ func (s *eventStatements) BulkSelectStateEventByNID( } } selectOrig += " ORDER BY event_type_nid, event_state_key_nid ASC" - selectStmt, err := s.db.Prepare(selectOrig) + selectPrep, err := s.db.Prepare(selectOrig) if err != nil { return nil, fmt.Errorf("s.db.Prepare: %w", err) } - selectStmt = sqlutil.TxStmt(txn, selectStmt) + defer selectPrep.Close() // nolint:errcheck + selectStmt := sqlutil.TxStmt(txn, selectPrep) rows, err := selectStmt.QueryContext(ctx, params...) if err != nil { return nil, fmt.Errorf("selectStmt.QueryContext: %w", err) @@ -307,11 +309,12 @@ func (s *eventStatements) BulkSelectStateAtEventByID( iEventIDs[k] = v } selectOrig := strings.Replace(bulkSelectStateAtEventByIDSQL, "($1)", sqlutil.QueryVariadic(len(iEventIDs)), 1) - selectStmt, err := s.db.Prepare(selectOrig) + selectPrep, err := s.db.Prepare(selectOrig) if err != nil { return nil, err } - selectStmt = sqlutil.TxStmt(txn, selectStmt) + defer selectPrep.Close() // nolint:errcheck + selectStmt := sqlutil.TxStmt(txn, selectPrep) /////////////// rows, err := selectStmt.QueryContext(ctx, iEventIDs...) if err != nil { @@ -390,10 +393,11 @@ func (s *eventStatements) BulkSelectStateAtEventAndReference( if err != nil { return nil, err } - selectPrep = sqlutil.TxStmt(txn, selectPrep) + defer selectPrep.Close() // nolint:errcheck + selectStmt := sqlutil.TxStmt(txn, selectPrep) ////////////// - rows, err := sqlutil.TxStmt(txn, selectPrep).QueryContext(ctx, iEventNIDs...) + rows, err := sqlutil.TxStmt(txn, selectStmt).QueryContext(ctx, iEventNIDs...) if err != nil { return nil, fmt.Errorf("sqlutil.TxStmt.QueryContext: %w", err) } @@ -441,6 +445,7 @@ func (s *eventStatements) BulkSelectEventReference( if err != nil { return nil, err } + defer selectPrep.Close() // nolint:errcheck /////////////// selectStmt := sqlutil.TxStmt(txn, selectPrep) @@ -471,11 +476,12 @@ func (s *eventStatements) BulkSelectEventID(ctx context.Context, txn *sql.Tx, ev iEventNIDs[k] = v } selectOrig := strings.Replace(bulkSelectEventIDSQL, "($1)", sqlutil.QueryVariadic(len(iEventNIDs)), 1) - selectStmt, err := s.db.Prepare(selectOrig) + selectPrep, err := s.db.Prepare(selectOrig) if err != nil { return nil, err } - selectStmt = sqlutil.TxStmt(txn, selectStmt) + defer selectPrep.Close() // nolint:errcheck + selectStmt := sqlutil.TxStmt(txn, selectPrep) /////////////// rows, err := selectStmt.QueryContext(ctx, iEventNIDs...) @@ -526,11 +532,12 @@ func (s *eventStatements) bulkSelectEventNID(ctx context.Context, txn *sql.Tx, e } else { selectOrig = strings.Replace(bulkSelectEventNIDSQL, "($1)", sqlutil.QueryVariadic(len(iEventIDs)), 1) } - selectStmt, err := s.db.Prepare(selectOrig) + selectPrep, err := s.db.Prepare(selectOrig) if err != nil { return nil, err } - selectStmt = sqlutil.TxStmt(txn, selectStmt) + defer selectPrep.Close() // nolint:errcheck + selectStmt := sqlutil.TxStmt(txn, selectPrep) /////////////// rows, err := selectStmt.QueryContext(ctx, iEventIDs...) if err != nil { @@ -560,6 +567,7 @@ func (s *eventStatements) SelectMaxEventDepth(ctx context.Context, txn *sql.Tx, if err != nil { return 0, err } + defer sqlPrep.Close() err = sqlutil.TxStmt(txn, sqlPrep).QueryRowContext(ctx, iEventIDs...).Scan(&result) if err != nil { return 0, fmt.Errorf("sqlutil.TxStmt.QueryRowContext: %w", err) @@ -575,12 +583,13 @@ func (s *eventStatements) SelectRoomNIDsForEventNIDs( if err != nil { return nil, err } - sqlPrep = sqlutil.TxStmt(txn, sqlPrep) + defer sqlPrep.Close() + sqlStmt := sqlutil.TxStmt(txn, sqlPrep) iEventNIDs := make([]interface{}, len(eventNIDs)) for i, v := range eventNIDs { iEventNIDs[i] = v } - rows, err := sqlPrep.QueryContext(ctx, iEventNIDs...) + rows, err := sqlStmt.QueryContext(ctx, iEventNIDs...) if err != nil { return nil, err } diff --git a/roomserver/storage/sqlite3/rooms_table.go b/roomserver/storage/sqlite3/rooms_table.go index 5413475e2..a81b78148 100644 --- a/roomserver/storage/sqlite3/rooms_table.go +++ b/roomserver/storage/sqlite3/rooms_table.go @@ -233,12 +233,13 @@ func (s *roomStatements) SelectRoomVersionsForRoomNIDs( if err != nil { return nil, err } - sqlPrep = sqlutil.TxStmt(txn, sqlPrep) + defer sqlPrep.Close() // nolint:errcheck + sqlStmt := sqlutil.TxStmt(txn, sqlPrep) iRoomNIDs := make([]interface{}, len(roomNIDs)) for i, v := range roomNIDs { iRoomNIDs[i] = v } - rows, err := sqlPrep.QueryContext(ctx, iRoomNIDs...) + rows, err := sqlStmt.QueryContext(ctx, iRoomNIDs...) if err != nil { return nil, err } diff --git a/roomserver/storage/sqlite3/state_block_table.go b/roomserver/storage/sqlite3/state_block_table.go index d51fc492d..3c829cdcd 100644 --- a/roomserver/storage/sqlite3/state_block_table.go +++ b/roomserver/storage/sqlite3/state_block_table.go @@ -108,11 +108,12 @@ func (s *stateBlockStatements) BulkSelectStateBlockEntries( intfs[i] = int64(stateBlockNIDs[i]) } selectOrig := strings.Replace(bulkSelectStateBlockEntriesSQL, "($1)", sqlutil.QueryVariadic(len(intfs)), 1) - selectStmt, err := s.db.Prepare(selectOrig) + selectPrep, err := s.db.Prepare(selectOrig) if err != nil { return nil, err } - selectStmt = sqlutil.TxStmt(txn, selectStmt) + defer selectPrep.Close() // nolint:errcheck + selectStmt := sqlutil.TxStmt(txn, selectPrep) rows, err := selectStmt.QueryContext(ctx, intfs...) if err != nil { return nil, err diff --git a/roomserver/storage/sqlite3/state_snapshot_table.go b/roomserver/storage/sqlite3/state_snapshot_table.go index 01df31e90..1f5e9ee3b 100644 --- a/roomserver/storage/sqlite3/state_snapshot_table.go +++ b/roomserver/storage/sqlite3/state_snapshot_table.go @@ -113,11 +113,12 @@ func (s *stateSnapshotStatements) BulkSelectStateBlockNIDs( nids[k] = v } selectOrig := strings.Replace(bulkSelectStateBlockNIDsSQL, "($1)", sqlutil.QueryVariadic(len(nids)), 1) - selectStmt, err := s.db.Prepare(selectOrig) + selectPrep, err := s.db.Prepare(selectOrig) if err != nil { return nil, err } - selectStmt = sqlutil.TxStmt(txn, selectStmt) + defer selectPrep.Close() // nolint:errcheck + selectStmt := sqlutil.TxStmt(txn, selectPrep) rows, err := selectStmt.QueryContext(ctx, nids...) if err != nil { diff --git a/setup/config/config_test.go b/setup/config/config_test.go index 6467b7c82..e6f0a493e 100644 --- a/setup/config/config_test.go +++ b/setup/config/config_test.go @@ -200,11 +200,6 @@ user_api: max_open_conns: 100 max_idle_conns: 2 conn_max_lifetime: -1 - device_database: - connection_string: file:userapi_devices.db - max_open_conns: 100 - max_idle_conns: 2 - conn_max_lifetime: -1 pusher_database: connection_string: file:pushserver.db max_open_conns: 100 diff --git a/setup/jetstream/helpers.go b/setup/jetstream/helpers.go index 544b5f0c3..d444272d2 100644 --- a/setup/jetstream/helpers.go +++ b/setup/jetstream/helpers.go @@ -4,6 +4,7 @@ import ( "context" "fmt" + "github.com/getsentry/sentry-go" "github.com/nats-io/nats.go" "github.com/sirupsen/logrus" ) @@ -29,6 +30,7 @@ func JetStreamConsumer( name := durable + "Pull" sub, err := js.PullSubscribe(subj, name, opts...) if err != nil { + sentry.CaptureException(err) return fmt.Errorf("nats.SubscribeSync: %w", err) } go func() { @@ -55,6 +57,7 @@ func JetStreamConsumer( } } else { // Something else went wrong, so we'll panic. + sentry.CaptureException(err) logrus.WithContext(ctx).WithField("subject", subj).Fatal(err) } } @@ -64,15 +67,18 @@ func JetStreamConsumer( msg := msgs[0] if err = msg.InProgress(); err != nil { logrus.WithContext(ctx).WithField("subject", subj).Warn(fmt.Errorf("msg.InProgress: %w", err)) + sentry.CaptureException(err) continue } if f(ctx, msg) { if err = msg.Ack(); err != nil { logrus.WithContext(ctx).WithField("subject", subj).Warn(fmt.Errorf("msg.Ack: %w", err)) + sentry.CaptureException(err) } } else { if err = msg.Nak(); err != nil { logrus.WithContext(ctx).WithField("subject", subj).Warn(fmt.Errorf("msg.Nak: %w", err)) + sentry.CaptureException(err) } } } diff --git a/setup/mscs/msc2946/msc2946.go b/setup/mscs/msc2946/msc2946.go index 7ab50c32e..7fb043366 100644 --- a/setup/mscs/msc2946/msc2946.go +++ b/setup/mscs/msc2946/msc2946.go @@ -199,13 +199,14 @@ func (w *walker) storePaginationCache(paginationToken string, cache paginationIn } type roomVisit struct { - roomID string - depth int - vias []string // vias to query this room by + roomID string + parentRoomID string + depth int + vias []string // vias to query this room by } func (w *walker) walk() util.JSONResponse { - if !w.authorised(w.rootRoomID) { + if authorised, _ := w.authorised(w.rootRoomID, ""); !authorised { if w.caller != nil { // CS API format return util.JSONResponse{ @@ -238,8 +239,9 @@ func (w *walker) walk() util.JSONResponse { w.paginationToken = tok // Begin walking the graph starting with the room ID in the request in a queue of unvisited rooms c.unvisited = append(c.unvisited, roomVisit{ - roomID: w.rootRoomID, - depth: 0, + roomID: w.rootRoomID, + parentRoomID: "", + depth: 0, }) } @@ -277,23 +279,8 @@ func (w *walker) walk() util.JSONResponse { // If we know about this room and the caller is authorised (joined/world_readable) then pull // events locally - if w.roomExists(rv.roomID) && w.authorised(rv.roomID) { - // Get all `m.space.child` state events for this room - events, err := w.childReferences(rv.roomID) - if err != nil { - util.GetLogger(w.ctx).WithError(err).WithField("room_id", rv.roomID).Error("failed to extract references for room") - continue - } - discoveredChildEvents = events - - pubRoom := w.publicRoomsChunk(rv.roomID) - - discoveredRooms = append(discoveredRooms, gomatrixserverlib.MSC2946Room{ - PublicRoom: *pubRoom, - RoomType: roomType, - ChildrenState: events, - }) - } else { + roomExists := w.roomExists(rv.roomID) + if !roomExists { // attempt to query this room over federation, as either we've never heard of it before // or we've left it and hence are not authorised (but info may be exposed regardless) fedRes, err := w.federatedRoomInfo(rv.roomID, rv.vias) @@ -312,6 +299,29 @@ func (w *walker) walk() util.JSONResponse { // as these children may be rooms we do know about. roomType = ConstCreateEventContentValueSpace } + } else if authorised, isJoinedOrInvited := w.authorised(rv.roomID, rv.parentRoomID); authorised { + // Get all `m.space.child` state events for this room + events, err := w.childReferences(rv.roomID) + if err != nil { + util.GetLogger(w.ctx).WithError(err).WithField("room_id", rv.roomID).Error("failed to extract references for room") + continue + } + discoveredChildEvents = events + + pubRoom := w.publicRoomsChunk(rv.roomID) + + discoveredRooms = append(discoveredRooms, gomatrixserverlib.MSC2946Room{ + PublicRoom: *pubRoom, + RoomType: roomType, + ChildrenState: events, + }) + // don't walk children if the user is not joined/invited to the space + if !isJoinedOrInvited { + continue + } + } else { + // room exists but user is not authorised + continue } // don't walk the children @@ -332,9 +342,10 @@ func (w *walker) walk() util.JSONResponse { ev := discoveredChildEvents[i] _ = json.Unmarshal(ev.Content, &spaceContent) unvisited = append(unvisited, roomVisit{ - roomID: ev.StateKey, - depth: rv.depth + 1, - vias: spaceContent.Via, + roomID: ev.StateKey, + parentRoomID: rv.roomID, + depth: rv.depth + 1, + vias: spaceContent.Via, }) } } @@ -465,25 +476,29 @@ func (w *walker) roomExists(roomID string) bool { } // authorised returns true iff the user is joined this room or the room is world_readable -func (w *walker) authorised(roomID string) bool { +func (w *walker) authorised(roomID, parentRoomID string) (authed, isJoinedOrInvited bool) { if w.caller != nil { - return w.authorisedUser(roomID) + return w.authorisedUser(roomID, parentRoomID) } - return w.authorisedServer(roomID) + return w.authorisedServer(roomID), false } // authorisedServer returns true iff the server is joined this room or the room is world_readable func (w *walker) authorisedServer(roomID string) bool { - // Check history visibility first + // Check history visibility / join rules first hisVisTuple := gomatrixserverlib.StateKeyTuple{ EventType: gomatrixserverlib.MRoomHistoryVisibility, StateKey: "", } + joinRuleTuple := gomatrixserverlib.StateKeyTuple{ + EventType: gomatrixserverlib.MRoomJoinRules, + StateKey: "", + } var queryRoomRes roomserver.QueryCurrentStateResponse err := w.rsAPI.QueryCurrentState(w.ctx, &roomserver.QueryCurrentStateRequest{ RoomID: roomID, StateTuples: []gomatrixserverlib.StateKeyTuple{ - hisVisTuple, + hisVisTuple, joinRuleTuple, }, }, &queryRoomRes) if err != nil { @@ -497,29 +512,46 @@ func (w *walker) authorisedServer(roomID string) bool { return true } } - // check if server is joined to the room - var queryRes fs.QueryJoinedHostServerNamesInRoomResponse - err = w.fsAPI.QueryJoinedHostServerNamesInRoom(w.ctx, &fs.QueryJoinedHostServerNamesInRoomRequest{ - RoomID: roomID, - }, &queryRes) - if err != nil { - util.GetLogger(w.ctx).WithError(err).Error("failed to QueryJoinedHostServerNamesInRoom") - return false + + // check if this room is a restricted room and if so, we need to check if the server is joined to an allowed room ID + // in addition to the actual room ID (but always do the actual one first as it's quicker in the common case) + allowJoinedToRoomIDs := []string{roomID} + joinRuleEv := queryRoomRes.StateEvents[joinRuleTuple] + if joinRuleEv != nil { + allowJoinedToRoomIDs = append(allowJoinedToRoomIDs, w.restrictedJoinRuleAllowedRooms(joinRuleEv, "m.room_membership")...) } - for _, srv := range queryRes.ServerNames { - if srv == w.serverName { - return true + + // check if server is joined to any allowed room + for _, allowedRoomID := range allowJoinedToRoomIDs { + var queryRes fs.QueryJoinedHostServerNamesInRoomResponse + err = w.fsAPI.QueryJoinedHostServerNamesInRoom(w.ctx, &fs.QueryJoinedHostServerNamesInRoomRequest{ + RoomID: allowedRoomID, + }, &queryRes) + if err != nil { + util.GetLogger(w.ctx).WithError(err).Error("failed to QueryJoinedHostServerNamesInRoom") + continue + } + for _, srv := range queryRes.ServerNames { + if srv == w.serverName { + return true + } } } + return false } -// authorisedUser returns true iff the user is joined this room or the room is world_readable -func (w *walker) authorisedUser(roomID string) bool { +// authorisedUser returns true iff the user is invited/joined this room or the room is world_readable. +// Failing that, if the room has a restricted join rule and belongs to the space parent listed, it will return true. +func (w *walker) authorisedUser(roomID, parentRoomID string) (authed bool, isJoinedOrInvited bool) { hisVisTuple := gomatrixserverlib.StateKeyTuple{ EventType: gomatrixserverlib.MRoomHistoryVisibility, StateKey: "", } + joinRuleTuple := gomatrixserverlib.StateKeyTuple{ + EventType: gomatrixserverlib.MRoomJoinRules, + StateKey: "", + } roomMemberTuple := gomatrixserverlib.StateKeyTuple{ EventType: gomatrixserverlib.MRoomMember, StateKey: w.caller.UserID, @@ -528,28 +560,79 @@ func (w *walker) authorisedUser(roomID string) bool { err := w.rsAPI.QueryCurrentState(w.ctx, &roomserver.QueryCurrentStateRequest{ RoomID: roomID, StateTuples: []gomatrixserverlib.StateKeyTuple{ - hisVisTuple, roomMemberTuple, + hisVisTuple, joinRuleTuple, roomMemberTuple, }, }, &queryRes) if err != nil { util.GetLogger(w.ctx).WithError(err).Error("failed to QueryCurrentState") - return false + return false, false } memberEv := queryRes.StateEvents[roomMemberTuple] - hisVisEv := queryRes.StateEvents[hisVisTuple] if memberEv != nil { membership, _ := memberEv.Membership() if membership == gomatrixserverlib.Join || membership == gomatrixserverlib.Invite { - return true + return true, true } } + hisVisEv := queryRes.StateEvents[hisVisTuple] if hisVisEv != nil { hisVis, _ := hisVisEv.HistoryVisibility() if hisVis == "world_readable" { - return true + return true, false } } - return false + joinRuleEv := queryRes.StateEvents[joinRuleTuple] + if parentRoomID != "" && joinRuleEv != nil { + allowedRoomIDs := w.restrictedJoinRuleAllowedRooms(joinRuleEv, "m.room_membership") + // check parent is in the allowed set + var allowed bool + for _, a := range allowedRoomIDs { + if parentRoomID == a { + allowed = true + break + } + } + if allowed { + // ensure caller is joined to the parent room + var queryRes2 roomserver.QueryCurrentStateResponse + err = w.rsAPI.QueryCurrentState(w.ctx, &roomserver.QueryCurrentStateRequest{ + RoomID: parentRoomID, + StateTuples: []gomatrixserverlib.StateKeyTuple{ + roomMemberTuple, + }, + }, &queryRes2) + if err != nil { + util.GetLogger(w.ctx).WithError(err).WithField("parent_room_id", parentRoomID).Warn("failed to check user is joined to parent room") + } else { + memberEv = queryRes2.StateEvents[roomMemberTuple] + if memberEv != nil { + membership, _ := memberEv.Membership() + if membership == gomatrixserverlib.Join { + return true, false + } + } + } + } + } + return false, false +} + +func (w *walker) restrictedJoinRuleAllowedRooms(joinRuleEv *gomatrixserverlib.HeaderedEvent, allowType string) (allows []string) { + rule, _ := joinRuleEv.JoinRule() + if rule != "restricted" { + return nil + } + var jrContent gomatrixserverlib.JoinRuleContent + if err := json.Unmarshal(joinRuleEv.Content(), &jrContent); err != nil { + util.GetLogger(w.ctx).Warnf("failed to check join_rule on room %s: %s", joinRuleEv.RoomID(), err) + return nil + } + for _, allow := range jrContent.Allow { + if allow.Type == allowType { + allows = append(allows, allow.RoomID) + } + } + return } // references returns all child references pointing to or from this room. diff --git a/syncapi/consumers/roomserver.go b/syncapi/consumers/roomserver.go index 159657f9f..640c505c2 100644 --- a/syncapi/consumers/roomserver.go +++ b/syncapi/consumers/roomserver.go @@ -154,7 +154,42 @@ func (s *OutputRoomEventConsumer) onNewRoomEvent( ctx context.Context, msg api.OutputNewRoomEvent, ) error { ev := msg.Event - addsStateEvents := msg.AddsState() + + addsStateEvents := []*gomatrixserverlib.HeaderedEvent{} + foundEventIDs := map[string]bool{} + if len(msg.AddsStateEventIDs) > 0 { + for _, eventID := range msg.AddsStateEventIDs { + foundEventIDs[eventID] = false + } + foundEvents, err := s.db.Events(ctx, msg.AddsStateEventIDs) + if err != nil { + return fmt.Errorf("s.db.Events: %w", err) + } + for _, event := range foundEvents { + foundEventIDs[event.EventID()] = true + } + eventsReq := &api.QueryEventsByIDRequest{} + eventsRes := &api.QueryEventsByIDResponse{} + for eventID, found := range foundEventIDs { + if !found { + eventsReq.EventIDs = append(eventsReq.EventIDs, eventID) + } + } + if err = s.rsAPI.QueryEventsByID(ctx, eventsReq, eventsRes); err != nil { + return fmt.Errorf("s.rsAPI.QueryEventsByID: %w", err) + } + for _, event := range eventsRes.Events { + eventID := event.EventID() + foundEvents = append(foundEvents, event) + foundEventIDs[eventID] = true + } + for eventID, found := range foundEventIDs { + if !found { + return fmt.Errorf("event %s is missing", eventID) + } + } + addsStateEvents = foundEvents + } ev, err := s.updateStateEvent(ev) if err != nil { diff --git a/syncapi/routing/messages.go b/syncapi/routing/messages.go index 7cd54eef0..9aef5db14 100644 --- a/syncapi/routing/messages.go +++ b/syncapi/routing/messages.go @@ -184,16 +184,18 @@ func OnIncomingMessagesRequest( // at least fetch the membership events for the users returned in chunk if LazyLoadMembers is set state := []gomatrixserverlib.ClientEvent{} if filter.LazyLoadMembers { - memberShipToUser := make(map[string]*gomatrixserverlib.HeaderedEvent) + membershipToUser := make(map[string]*gomatrixserverlib.HeaderedEvent) for _, evt := range clientEvents { - memberShip, err := db.GetStateEvent(req.Context(), roomID, gomatrixserverlib.MRoomMember, evt.Sender) + membership, err := db.GetStateEvent(req.Context(), roomID, gomatrixserverlib.MRoomMember, evt.Sender) if err != nil { util.GetLogger(req.Context()).WithError(err).Error("failed to get membership event for user") continue } - memberShipToUser[evt.Sender] = memberShip + if membership != nil { + membershipToUser[evt.Sender] = membership + } } - for _, evt := range memberShipToUser { + for _, evt := range membershipToUser { state = append(state, gomatrixserverlib.HeaderedToClientEvent(evt, gomatrixserverlib.FormatAll)) } } diff --git a/sytest-blacklist b/sytest-blacklist index becc500ec..cee2406e5 100644 --- a/sytest-blacklist +++ b/sytest-blacklist @@ -24,6 +24,7 @@ Local device key changes get to remote servers with correct prev_id # Flakey Local device key changes appear in /keys/changes +/context/ with lazy_load_members filter works # we don't support groups Remove group category @@ -31,8 +32,11 @@ Remove group role # Flakey AS-ghosted users can use rooms themselves +/context/ with lazy_load_members filter works +AS-ghosted users can use rooms via AS +Events in rooms with AS-hosted room aliases are sent to AS server # Flakey, need additional investigation Messages that notify from another user increment notification_count Messages that highlight from another user increment unread highlight count -Notifications can be viewed with GET /notifications \ No newline at end of file +Notifications can be viewed with GET /notifications diff --git a/sytest-whitelist b/sytest-whitelist index 63d779bf1..6c4745b32 100644 --- a/sytest-whitelist +++ b/sytest-whitelist @@ -648,7 +648,6 @@ Device list doesn't change if remote server is down /context/ on joined room works /context/ on non world readable room does not work /context/ returns correct number of events -/context/ with lazy_load_members filter works GET /rooms/:room_id/messages lazy loads members correctly Can query remote device keys using POST after notification Device deletion propagates over federation @@ -659,4 +658,9 @@ registration accepts non-ascii passwords registration with inhibit_login inhibits login The operation must be consistent through an interactive authentication session Multiple calls to /sync should not cause 500 errors -/context/ with lazy_load_members filter works +Canonical alias can be set +Canonical alias can include alt_aliases +Can delete canonical alias +Multiple calls to /sync should not cause 500 errors +AS can make room aliases +Accesing an AS-hosted room alias asks the AS server