* Correctly redact events over federation (#2526)
* Ensure we check powerlevel/origin before redacting an event
* Add passing test
* Use pl.UserLevel
* Make check more readable, also check for the sender
* Add new next steps page to the documentation
* Highlighting in docs
* Rename the page to "Optimise your installation"
* Attempt to raise the file descriptor limit at startup (#2527)
* Add `--difference` to `resolve-state` tool
* Make the linter happy again
* generic CaddyFile in front of Dendrite (monolith) (#2531)
for Caddy 2.5.x
Co-authored-by: emanuele.aliberti <emanuele.aliberti@mtka.eu>
* Handle state before, send history visibility in output (#2532)
* Check state before event
* Tweaks
* Refactor a bit, include in output events
* Don't waste time if soft failed either
* Tweak control flow, comments, use GMSL history visibility type
* Fix rare panic when returning user devices over federation (#2534)
* Add `InputDeviceListUpdate` to the keyserver, remove old input API (#2536)
* Add `InputDeviceListUpdate` to the keyserver, remove old input API
* Fix copyright
* Log more information when a device list update fails
* Fix nats.go commit (#2540)
Signed-off-by: Jean Lucas <jean@4ray.co>
* Don't return `end` if there are not more messages (#2542)
* Be more spec compliant
* Move lazyLoadMembers to own method
* Return an error if trying to invite a malformed user ID (#2543)
* Add `evacuateUser` endpoint, use it when deactivating accounts (#2545)
* Add `evacuateUser` endpoint, use it when deactivating accounts
* Populate the API
* Clean up user devices when deactivating
* Include invites, delete pushers
* Silence presence logs (#2547)
* Blacklist `Guest users can join guest_access rooms` test until it can be investigated
* Disable WebAssembly builds for now
* Try to fix backfilling (#2548)
* Try to fix backfilling
* Return start/end to not confuse clients
* Update GMSL
* Update GMSL
* Roomserver producers package (#2546)
* Give the roomserver a producers package
* Change init point
* Populate ACLs API
* Fix build issues
* `RoomEventProducer` naming
* Version 0.8.9 (#2549)
* Version 0.8.9
* Update changelog
* feat+fix: Ignore unknown keys and verify required fields are present in appservice registration files (#2550)
* fix: ignore unknown keys in appservice configs
fixesmatrix-org/dendrite#1567
* feat: verify required fields in appservice configs
* Use new testrig for key changes tests (#2552)
* Use new testrig for tests
* Log the error message
* Fix QuerySharedUsers for the SyncAPI keychange consumer (#2554)
* Make more use of base.BaseDendrite
* Fix QuerySharedUsers if no UserIDs are supplied
* Return clearer error when no state NID exists for an event (#2555)
* Wrap error from `SnapshotNIDFromEventID`
* Hopefully fix read receipts timestamps (#2557)
This should avoid coercions between signed and unsigned ints which might fix problems like `sql: converting argument $5 type: uint64 values with high bit set are not supported`.
* Fix nil pointer access when redacting events (#2560)
* Fix issue `uint64 values with high bit are not supported` in presence (#2562)
* Fix issue #2528
* Use gomatrixserverlib.Timestamp
* Use ParseUint instead of ParseInt
* Update Pinecone to matrix-org/pinecone@1ce778f
* Ristretto cache (#2563)
* Try Ristretto cache
* Tweak
* It's beautiful
* Update GMSL
* More strict keyable interface
* Fix that some more
* Make less panicky
* Don't enforce mutability checks for now
* Determine mutability using deep equality
* Tweaks
* Namespace keys
* Make federation caches mutable
* Update cost estimation, add metric
* Update GMSL
* Estimate cost for metrics better
* Reduce counters a bit
* Try caching events
* Some guards
* Try again
* Try this
* Use separate caches for hopefully better hash distribution
* Fix bug with admitting events into cache
* Try to fix bugs
* Check nil
* Try that again
* Preserve order jeezo this is messy
* thanks VS Code for doing exactly the wrong thing
* Try this again
* Be more specific
* aaaaargh
* One more time
* That might be better
* Stronger sorting
* Cache expiries, async publishing of EDUs
* Put it back
* Use a shared cache again
* Cost estimation fixes
* Update ristretto
* Reduce counters a bit
* Clean up a bit
* Update GMSL
* 1GB
* Configurable cache sizees
* Tweaks
* Add `config.DataUnit` for specifying friendly cache sizes
* Various tweaks
* Update GMSL
* Add back some lazy loading caching
* Include key in cost
* Include key in cost
* Tweak max age handling, config key name
* Only register prometheus metrics if requested
* Review comments @S7evinK
* Don't return errors when creating caches (it is better just to crash since otherwise we'll `nil`-pointer exception everywhere)
* Review comments
* Update sample configs
* Update GHA Workflow
* Update Complement images to Go 1.18
* Remove the cache test from the federation API as we no longer guarantee immediate cache admission
* Don't check the caches in the renewal test
* Possibly fix the upgrade tests
* Update to matrix-org/gomatrixserverlib#322
* Update documentation to refer to Go 1.18
* Minor SendToDevice fix (#2565)
* Avoid unnecessary marshalling if sending to the local server
* Fix ordering of ToDevice messages
* Revive SendToDevice test
* Use `/v3` to request media from remote servers (update to matrix-org/gomatrixserverlib#324)
* Pointerise `types.RoomInfo` in the cache so we can update it in-place in the latest events updater
* Add a Troubleshooting page
* Update `sytest-whitelist`
* Use sync API database in `filterSharedUsers` (#2572)
* Add function to the sync API storage package for filtering shared users
* Use the database instead of asking the RS API
* Fix unit tests
* Fix map handling in `filterSharedUsers`
* Update 1_createusers.md (#2571)
* Update 1_createusers.md
Added description on how to create user accounts when running in docker.
* Update 1_createusers.md
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
* Fix connection_string format in dendrite-sample.polylith.yaml (#2574)
* History visibility database changes (#2533)
* Add new history_visibility column
* Update SQL queries to include history_visibility
* Store the history visibilty calculated by the roomserver
* Update GMSL
* Update migrations
* Fix migration
* Update GMSL
* Fix `go.sum`
* Update GMSL to use sql.Scanner & sql.Valuer
* Re-order migration/table creation
* Update gomatrixserverlib
* Add history_visibility column to current_room_state
* Fix migrations
* Return error instead of Fatal log
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
* Tweak cache counters (#2575)
* Tweak cache counters
This makes the number of counters relative to the
maximum cache size. Since the counters
effectively manage the size of the bloom filter,
larger caches need more counters and smaller
caches need less.
10 counters per 1KB data means that the default
cache size of 1GB should result in a bloom filter
and TinyLRU admission set of about 16MB
estimated.
* Remove line left by accident
* Set historyVisibility in rowsToStreamEvents
* Update FAQ
* Add event state key cache (#2576)
* Explain how SRV works in Matrix and discourage using it (#2577)
* Explain how SRV works in Matrix and discourage using it
* Minor tweaks to formatting
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
* Fix issue with membership event_nid being 0 (#2580)
* docs: Add build page; correct proxy info; fix Caddy example (#2579)
* Add build page; correct proxy info; fix Caddy example
* Improve Caddyfile example
* Apply review comments; add polylith Caddyfile
* Bump tzinfo from 1.2.9 to 1.2.10 in /docs (#2584)
Bumps [tzinfo](https://github.com/tzinfo/tzinfo) from 1.2.9 to 1.2.10.
- [Release notes](https://github.com/tzinfo/tzinfo/releases)
- [Changelog](https://github.com/tzinfo/tzinfo/blob/master/CHANGES.md)
- [Commits](https://github.com/tzinfo/tzinfo/compare/v1.2.9...v1.2.10)
---
updated-dependencies:
- dependency-name: tzinfo
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Membership updater refactoring (#2541)
* Membership updater refactoring
* Pass in membership state
* Use membership check rather than referring to state directly
* Delete irrelevant membership states
* We don't need the leave event after all
* Tweaks
* Put a log entry in that I might stand a chance of finding
* Be less panicky
* Tweak invite handling
* Don't freak if we can't find the event NID
* Use event NID from `types.Event`
* Clean up
* Better invite handling
* Placate the almighty linter
* Blacklist a Sytest which is otherwise fine under Complement for reasons I don't understand
* Fix the sytest after all (thanks @S7evinK for the spot)
* Try to fix HTTP 500s on `/members` (#2581)
* Update database migrations, remove goose (#2264)
* Add new db migration
* Update migrations
Remove goose
* Add possibility to test direct upgrades
* Try to fix WASM test
* Add checks for specific migrations
* Remove AddMigration
Use WithTransaction
Add Dendrite version to table
* Fix linter issues
* Update tests
* Update comments, outdent if
* Namespace migrations
* Add direct upgrade tests, skipping over one version
* Split migrations
* Update go version in CI
* Fix copy&paste mistake
* Use contexts in migrations
Co-authored-by: kegsay <kegan@matrix.org>
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
* Add .well-known/matrix/client to clientapi (#2551)
Signed-off-by: Jonathan Bartlett <jonathan@jonnobrow.co.uk>
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
* Remove `room_id` field from MSC2946 stripped events (closes#2588)
* Remove `goose` from Dockerfiles
* Make the User API responsible for sending account data output events (#2592)
* Make the User API responsible for sending account data output events
* Clean up producer
* Review comments
* Update NATS Server and nats.go to use upstream
* Set CORS headers for HTTP 404 and 405 errors (#2599)
* Set CORS headers for the 404s
* Use custom handlers, plus one for HTTP 405 too
* Tweak setup
* Add to muxes too
* Tidy up some more
* Use built-in HTTP 404 handler
* Don't bother setting it for federation-facing
* Optimise checking other servers allowed to see events (#2596)
* Try optimising checking if server is allowed to see event
* Fix error
* Handle case where snapshot NID is 0
* Fix query
* Update SQL
* Clean up `CheckServerAllowedToSeeEvent`
* Not supported on SQLite
* Maybe placate the unit tests
* Review comments
* De-race `types.RoomInfo` (#2600)
* De-race `CompleteSync` (#2601)
The `err` was coming from outside of the goroutine and being written to by concurrent goroutines.
* Version 0.9.0 (#2602)
Co-authored-by: Till <2353100+S7evinK@users.noreply.github.com>
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
Co-authored-by: Till Faelligen <davidf@element.io>
Co-authored-by: Emanuele Aliberti <dev@mtka.eu>
Co-authored-by: emanuele.aliberti <emanuele.aliberti@mtka.eu>
Co-authored-by: Jean Lucas <jean@4ray.co>
Co-authored-by: Kabir Kwatra <kabir@kwatra.me>
Co-authored-by: andreever <52261463+andreever@users.noreply.github.com>
Co-authored-by: Maximilian Gaedig <38767445+MaximilianGaedig@users.noreply.github.com>
Co-authored-by: Tulir Asokan <tulir@maunium.net>
Co-authored-by: Matt Holt <mholt@users.noreply.github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: kegsay <kegan@matrix.org>
Co-authored-by: Jonathan Bartlett <34320158+Jonnobrow@users.noreply.github.com>
* Return correct membership in GetMembership when user is invited
* Update whitelist
* Restore logging to file
* Fix linter issues
* Attempt to fix presence
* Allow invited users to fetch state events and events by ID
* Move failing test case to blacklist
* Fix /sync panicking due to presence udpates
* Use upload-artifact v3
* Yet another attempt to fix sytest artefacts
* Maybe this will work?
Squashed commit of the following:
commit b5c55faf98
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Fri Jul 1 12:00:32 2022 +0100
Version 0.8.9 (#2549)
* Version 0.8.9
* Update changelog
commit b50a24c666
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Fri Jul 1 10:54:07 2022 +0100
Roomserver producers package (#2546)
* Give the roomserver a producers package
* Change init point
* Populate ACLs API
* Fix build issues
* `RoomEventProducer` naming
commit 89cd0e8fc1
Author: Till <2353100+S7evinK@users.noreply.github.com>
Date: Fri Jul 1 11:49:26 2022 +0200
Try to fix backfilling (#2548)
* Try to fix backfilling
* Return start/end to not confuse clients
* Update GMSL
* Update GMSL
commit 086f182e24
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Fri Jul 1 09:50:06 2022 +0100
Disable WebAssembly builds for now
commit 54bed4c593
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Fri Jul 1 09:37:54 2022 +0100
Blacklist `Guest users can join guest_access rooms` test until it can be investigated
commit 561c159ad7
Author: Till <2353100+S7evinK@users.noreply.github.com>
Date: Thu Jun 30 12:34:37 2022 +0200
Silence presence logs (#2547)
commit 519bc1124b
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Wed Jun 29 15:29:39 2022 +0100
Add `evacuateUser` endpoint, use it when deactivating accounts (#2545)
* Add `evacuateUser` endpoint, use it when deactivating accounts
* Populate the API
* Clean up user devices when deactivating
* Include invites, delete pushers
commit 2dea466685
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Wed Jun 29 12:32:24 2022 +0100
Return an error if trying to invite a malformed user ID (#2543)
commit 2086992caf
Author: Till <2353100+S7evinK@users.noreply.github.com>
Date: Wed Jun 29 10:49:12 2022 +0200
Don't return `end` if there are not more messages (#2542)
* Be more spec compliant
* Move lazyLoadMembers to own method
commit 920a20821b
Author: Jean Lucas <jean@4ray.co>
Date: Mon Jun 27 04:15:19 2022 -0400
Fix nats.go commit (#2540)
Signed-off-by: Jean Lucas <jean@4ray.co>
commit 7120eb6bc9
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Wed Jun 15 14:27:07 2022 +0100
Add `InputDeviceListUpdate` to the keyserver, remove old input API (#2536)
* Add `InputDeviceListUpdate` to the keyserver, remove old input API
* Fix copyright
* Log more information when a device list update fails
commit 1b90cc9536
Author: Till <2353100+S7evinK@users.noreply.github.com>
Date: Wed Jun 15 12:50:02 2022 +0200
Fix rare panic when returning user devices over federation (#2534)
commit 4c2a10f1a6
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Mon Jun 13 15:11:10 2022 +0100
Handle state before, send history visibility in output (#2532)
* Check state before event
* Tweaks
* Refactor a bit, include in output events
* Don't waste time if soft failed either
* Tweak control flow, comments, use GMSL history visibility type
commit c500958583
Author: Emanuele Aliberti <dev@mtka.eu>
Date: Mon Jun 13 13:08:46 2022 +0200
generic CaddyFile in front of Dendrite (monolith) (#2531)
for Caddy 2.5.x
Co-authored-by: emanuele.aliberti <emanuele.aliberti@mtka.eu>
commit e1136f4d3e
Author: Till Faelligen <davidf@element.io>
Date: Mon Jun 13 11:46:59 2022 +0200
Make the linter happy again
commit 0a7f7dc716
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Mon Jun 13 10:16:30 2022 +0100
Add `--difference` to `resolve-state` tool
commit 89d2adadbd
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Fri Jun 10 10:58:04 2022 +0100
Attempt to raise the file descriptor limit at startup (#2527)
commit 1030072285
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Fri Jun 10 10:18:32 2022 +0100
Rename the page to "Optimise your installation"
commit 16ed1633b6
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Fri Jun 10 10:15:14 2022 +0100
Highlighting in docs
commit e2a64773ce
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Fri Jun 10 10:14:15 2022 +0100
Add new next steps page to the documentation
commit 660f7839f5
Author: Till <2353100+S7evinK@users.noreply.github.com>
Date: Thu Jun 9 18:38:07 2022 +0200
Correctly redact events over federation (#2526)
* Ensure we check powerlevel/origin before redacting an event
* Add passing test
* Use pl.UserLevel
* Make check more readable, also check for the sender
* Don't create fictitious presence entries for users that don't have any
* Update whitelist, since that test probably shouldn't be passing
* Fix panics
* Initial work on lazyloading
* Partially implement lazy loading on /sync
* Rename methods
* Make missing tests pass
* Preallocate slice, even if it will end up with fewer values
* Let the cache handle the user mapping
* Linter
* Cap cache growth
* Add ignore users
* Ignore users in pushrules
Add passing tests
* Update sytest lists
* Store ignore knowledge in the sync API
* Fix copyrights
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
* Added /upgrade endpoint
* fix
* Fix lints
* More lint lifex
* Move room upgrading to the roomserver
* Remove extraneous arg
* Fix HTTP API for `PerformUpgrade`
* Reduce number of API calls in `generateInitialEvents`, preserve membership fields
* Refactor `generateInitialEvents` to preserve old state events for all but the essential room setup events
* Handle ban events in the state transfer
* Refactor and comment `createTemporaryPowerLevels`
* Only send two power levels if we needed to override the levels, preserve miscellaneous fields in the create event
* Fix copyrights
* Review comments @S7evinK
* Update sytest whitelist
* Specify empty state keys, use `EventLevel`, remove unnecessary check on state copy
* Add comment to `restrictOldRoomPowerLevels`
* Ensure canonical aliases exist before clearing
* Copy invites as well as bans
* Fix return error on `m.room.tombstone` handling in client API
* Relax checks for well-formedness of join rules, membership event etc
Co-authored-by: Alex Kursell <alex@awk.run>
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
Co-authored-by: kegsay <kegan@matrix.org>
* Include joined and invite member counts in room summary
This should fix#2314 and also fix the problem where some clients like Element Android, Fluffychat etc would display the wrong member count for a given room.
* Improve SQLite query precision
* Check existence of state key for membership events
* Convert stream positions into topological positions for both `from` and `to` in `/messages`
* Hopefully it works now
* Remove unnecessary logging
* Return sane values if `StreamToTopologicalPosition` can't work out the right thing to do
* Revert logging change
* tweaks
* Fix `selectEventIDsInRangeASCSQL`
* Test `Getting messages going forward is limited for a departed room (SPEC-216)` was passing incorrectly so un-whitelist it
* Add membership events to the end of the list, to ensure Sytest sees them
* Move tests to allowlist
* Append to correct list, fix logging message
* Add flakey tests to blacklist
* Remove flakey tests from whitelist
* Add canonical support
* Add test
* Check that the send event is actually an m.room.canonical_alias
Check that we got an event from the database
* Update to get correct required events
* Add flakey test to blacklist
* Clean old notifications regularly
We'll keep highlights for a month and non-highlights for a day, to stop the `userapi_notifications` table from growing indefinitely.
We'll also allow storing events even if no pushers are present, because apparently Element Web expects to work that way.
* Fix the milliseconds
* Use process context
* Update sytest lists
* Fix build issue
* Add Pushserver component with Pushers API
Co-authored-by: Tommie Gannert <tommie@gannert.se>
Co-authored-by: Dan Peleg <dan@globekeeper.com>
* Wire Pushserver component
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
* Add PushGatewayClient.
The full event format is required for Sytest.
* Add a pushrules module.
* Change user API account creation to use the new pushrules module's defaults.
Introduces "scope" as required by client API, and some small field
tweaks to make some 61push Sytests pass.
* Add push rules query/put API in Pushserver.
This manipulates account data over User API, and fires sync messages
for changes. Those sync messages should, according to an existing TODO
in clientapi, be moved to userapi.
Forks clientapi/producers/syncapi.go to pushserver/ for later extension.
* Add clientapi routes for push rules to Pushserver.
A cleanup would be to move more of the name-splitting logic into
pushrules.go, to depollute routing.go.
* Output rooms.join.unread_notifications in /sync.
This is the read-side. Pushserver will be the write-side.
* Implement pushserver/storage for notifications.
* Use PushGatewayClient and the pushrules module in Pushserver's room consumer.
* Use one goroutine per user to avoid locking up the entire server for
one bad push gateway.
* Split pushing by format.
* Send one device per push. Sytest does not support coalescing
multiple devices into one push. Matches Synapse. Either we change
Sytest, or remove the group-by-url-and-format logic.
* Write OutputNotificationData from push server. Sync API is already
the consumer.
* Implement read receipt consumers in Pushserver.
Supports m.read and m.fully_read receipts.
* Add clientapi route for /unstable/notifications.
* Rename to UpsertPusher for clarity and handle pusher update
* Fix linter errors
* Ignore body.Close() error check
* Fix push server internal http wiring
* Add 40 newly passing 61push tests to whitelist
* Add next 12 newly passing 61push tests to whitelist
* Send notification data before notifying users in EDU server consumer
* NATS JetStream
* Goodbye sarama
* Fix `NewStreamTokenFromString`
* Consume on the correct topic for the roomserver
* Don't panic, NAK instead
* Move push notifications into the User API
* Don't set null values since that apparently causes Element upsetti
* Also set omitempty on conditions
* Fix bug so that we don't override the push rules unnecessarily
* Tweak defaults
* Update defaults
* More tweaks
* Move `/notifications` onto `r0`/`v3` mux
* User API will consume events and read/fully read markers from the sync API with stream positions, instead of consuming directly
Co-authored-by: Piotr Kozimor <p1996k@gmail.com>
Co-authored-by: Tommie Gannert <tommie@gannert.se>
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
* Only allow device deletion if the session matches
* Make the challenge response available to other packages
* Remove userID, as it's not in the spec
* Remove tests
* Add passing test & remove obsolete config
* Rename field, add comment
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
* Remember parameters for sessions
Cleanup sessions on successfully registering or after a while
* Add flakey test
* Update to use time.AfterFunc, add more tests
* Try to drain the channel, if possible
* Send device_list update to satisfy sytest
* Fix build issue from merged in change
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
* Add account_type for sqlite3
* Add account_type for postgres
* Remove CreateGuestAccount from interface
* Add new AccountTypes & update test
* Use newly added AccountType for account creation
* Add migrations
* Reuse type
* Add AccounnType to Device, so it can be verified on requests
* Rename migration, add missing update for appservices
* Rename sqlite3 migration
* Add missing AccountType to return value
* Update sqlite migration
Change allowance check on /admin/whois
* Fix migration, add IS NULL
* Move accountType to completeRegistration
* Fix migrations
* Add passing test
* Remove unneeded logging
* Add MasterKey & SelfSigningKey to update
Avoid panic if signatures are not present
* Add passing test
* Revert "Add MasterKey & SelfSigningKey to update"
This reverts commit 2c81b34884.
* Send MasterKey & SelfSigningKey with update
* Debugging
* Remove delete() so we also query signingkeys
* Put federation client functions into their own file
* Look for missing auth events in RS input
* Remove retrieveMissingAuthEvents from federation API
* Logging
* Sorta transplanted the code over
* Use event origin failing all else
* Don't get stuck on mutexes:
* Add verifier
* Don't mark state events with zero snapshot NID as not existing
* Check missing state if not an outlier before storing the event
* Reject instead of soft-fail, don't copy roominfo so much
* Use synchronous contexts, limit time to fetch missing events
* Clean up some commented out bits
* Simplify `/send` endpoint significantly
* Submit async
* Report errors on sending to RS input
* Set max payload in NATS to 16MB
* Tweak metrics
* Add `workerForRoom` for tidiness
* Try skipping unmarshalling errors for RespMissingEvents
* Track missing prev events separately to avoid calculating state when not possible
* Tweak logic around checking missing state
* Care about state when checking missing prev events
* Don't check missing state for create events
* Try that again
* Handle create events better
* Send create room events as new
* Use given event kind when sending auth/state events
* Revert "Use given event kind when sending auth/state events"
This reverts commit 089d64d271.
* Only search for missing prev events or state for new events
* Tweaks
* We only have missing prev if we don't supply state
* Room version tweaks
* Allow async inputs again
* Apply backpressure to consumers/synchronous requests to hopefully stop things being overwhelmed
* Set timeouts on roomserver input tasks (need to decide what timeout makes sense)
* Use work queue policy, deliver all on restart
* Reduce chance of duplicates being sent by NATS
* Limit the number of servers we attempt to reduce backpressure
* Some review comment fixes
* Tidy up a couple things
* Don't limit servers, randomise order using map
* Some context refactoring
* Update gmsl
* Don't resend create events
* Set stateIDs length correctly or else the roomserver thinks there are missing events when there aren't
* Exclude our own servername
* Try backing off servers
* Make excluding self behaviour optional
* Exclude self from g_m_e
* Update sytest-whitelist
* Update consumers for the roomserver output stream
* Remember to send outliers for state returned from /gme
* Make full HTTP tests less upsetti
* Remove 'If a device list update goes missing, the server resyncs on the next one' from the sytest blacklist
* Remove debugging test
* Fix blacklist again, remove unnecessary duplicate context
* Clearer contexts, don't use background in case there's something happening there
* Don't queue up events more than once in memory
* Correctly identify create events when checking for state
* Fill in gaps again in /gme code
* Remove `AuthEventIDs` from `InputRoomEvent`
* Remove stray field
Co-authored-by: Kegan Dougal <kegan@matrix.org>
Sytest had two tests with the name `Name/topic keys are correct`. https://github.com/matrix-org/sytest/pull/1098 edited one so that the names were unique.
It seems that this test name was in the dendrite Sytest whitelist. This PR adds the additional, new name to the whitelist as well. Time to see if CI passes!
* Handle other signatures
* Decorate key ID properly
* Match by key IDs
* Tweaks
* Fixes
* Fix /user/keys/query bug, review comments, update sytest-whitelist
* Various wtweaks
* Fix wiring for keyserver in API mode
* Additional fixes
* Cross-signing groundwork
* Update to matrix-org/gomatrixserverlib#274
* Fix gobind builds, which stops unit tests in CI from yelling
* Some changes from review comments
* Fix build by passing in UIA
* Update to matrix-org/gomatrixserverlib@bec8d22
* Process master/self-signing keys from devices call
* nolint
* Enum-ify the key type in the database
* Process self-signing key too
* Fix sanity check in device list updater
* Fix check
* Fix sytest, hopefully
* Fix build
