Squashed commit of the following:
commit b5c55faf98
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Fri Jul 1 12:00:32 2022 +0100
Version 0.8.9 (#2549)
* Version 0.8.9
* Update changelog
commit b50a24c666
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Fri Jul 1 10:54:07 2022 +0100
Roomserver producers package (#2546)
* Give the roomserver a producers package
* Change init point
* Populate ACLs API
* Fix build issues
* `RoomEventProducer` naming
commit 89cd0e8fc1
Author: Till <2353100+S7evinK@users.noreply.github.com>
Date: Fri Jul 1 11:49:26 2022 +0200
Try to fix backfilling (#2548)
* Try to fix backfilling
* Return start/end to not confuse clients
* Update GMSL
* Update GMSL
commit 086f182e24
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Fri Jul 1 09:50:06 2022 +0100
Disable WebAssembly builds for now
commit 54bed4c593
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Fri Jul 1 09:37:54 2022 +0100
Blacklist `Guest users can join guest_access rooms` test until it can be investigated
commit 561c159ad7
Author: Till <2353100+S7evinK@users.noreply.github.com>
Date: Thu Jun 30 12:34:37 2022 +0200
Silence presence logs (#2547)
commit 519bc1124b
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Wed Jun 29 15:29:39 2022 +0100
Add `evacuateUser` endpoint, use it when deactivating accounts (#2545)
* Add `evacuateUser` endpoint, use it when deactivating accounts
* Populate the API
* Clean up user devices when deactivating
* Include invites, delete pushers
commit 2dea466685
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Wed Jun 29 12:32:24 2022 +0100
Return an error if trying to invite a malformed user ID (#2543)
commit 2086992caf
Author: Till <2353100+S7evinK@users.noreply.github.com>
Date: Wed Jun 29 10:49:12 2022 +0200
Don't return `end` if there are not more messages (#2542)
* Be more spec compliant
* Move lazyLoadMembers to own method
commit 920a20821b
Author: Jean Lucas <jean@4ray.co>
Date: Mon Jun 27 04:15:19 2022 -0400
Fix nats.go commit (#2540)
Signed-off-by: Jean Lucas <jean@4ray.co>
commit 7120eb6bc9
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Wed Jun 15 14:27:07 2022 +0100
Add `InputDeviceListUpdate` to the keyserver, remove old input API (#2536)
* Add `InputDeviceListUpdate` to the keyserver, remove old input API
* Fix copyright
* Log more information when a device list update fails
commit 1b90cc9536
Author: Till <2353100+S7evinK@users.noreply.github.com>
Date: Wed Jun 15 12:50:02 2022 +0200
Fix rare panic when returning user devices over federation (#2534)
commit 4c2a10f1a6
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Mon Jun 13 15:11:10 2022 +0100
Handle state before, send history visibility in output (#2532)
* Check state before event
* Tweaks
* Refactor a bit, include in output events
* Don't waste time if soft failed either
* Tweak control flow, comments, use GMSL history visibility type
commit c500958583
Author: Emanuele Aliberti <dev@mtka.eu>
Date: Mon Jun 13 13:08:46 2022 +0200
generic CaddyFile in front of Dendrite (monolith) (#2531)
for Caddy 2.5.x
Co-authored-by: emanuele.aliberti <emanuele.aliberti@mtka.eu>
commit e1136f4d3e
Author: Till Faelligen <davidf@element.io>
Date: Mon Jun 13 11:46:59 2022 +0200
Make the linter happy again
commit 0a7f7dc716
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Mon Jun 13 10:16:30 2022 +0100
Add `--difference` to `resolve-state` tool
commit 89d2adadbd
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Fri Jun 10 10:58:04 2022 +0100
Attempt to raise the file descriptor limit at startup (#2527)
commit 1030072285
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Fri Jun 10 10:18:32 2022 +0100
Rename the page to "Optimise your installation"
commit 16ed1633b6
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Fri Jun 10 10:15:14 2022 +0100
Highlighting in docs
commit e2a64773ce
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date: Fri Jun 10 10:14:15 2022 +0100
Add new next steps page to the documentation
commit 660f7839f5
Author: Till <2353100+S7evinK@users.noreply.github.com>
Date: Thu Jun 9 18:38:07 2022 +0200
Correctly redact events over federation (#2526)
* Ensure we check powerlevel/origin before redacting an event
* Add passing test
* Use pl.UserLevel
* Make check more readable, also check for the sender
* syncapi: use finer-grained interfaces when making the syncapi
* Use specific interfaces for syncapi-roomserver interactions
* Define query access token api for shared http auth code
* Remove dependency on saramajetstream & sarama
Signed-off-by: Till Faelligen <tfaelligen@gmail.com>
* Remove internal.ContinualConsumer from federationapi
* Remove internal.ContinualConsumer from syncapi
* Remove internal.ContinualConsumer from keyserver
* Move to new Prepare function
* Remove saramajetstream & sarama dependency
* Delete unneeded file
* Remove duplicate import
* Log error instead of silently irgnoring it
* Move `OffsetNewest` and `OffsetOldest` into keyserver types, change them to be more sane values
* Fix comments
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
* go mod tidy
* Break complement to check it fails CI
* Remove partitioned stream positions
This was used by the device list stream position. The device list position
now corresponds to the `Offset`, and the partition is always 0, in prep
for removing reliance on Kafka topics for device list changes.
* Linting
* Migrate old style tokens to new style because element-web doesn't soft-logoout on 4xx errors on /sync
* Delete device keys/signatures from key server when deleting device from user API
* Move loop to within database transaction
* Don't fall over deleting no rows
* Enable unstable feature again
* Try to verify when a device signs a key
* Try to verify when a key signs a device
* It's the self-signing key, not the master key
* Fix error
* Try to verify master key uploads
* Actually we can't guarantee we can do that so nevermind
* Add signatures into /devices/list request
* Fix nil pointer
* Reprioritise map creation
* Don't skip devices that don't have signatures
* Add some debug logging
* Fix logic error in QuerySignatures
* Fix bugs
* Expose master and self-signing keys on /devices/list hopefully
* maps are tedious
* Expose signatures via /keys/query
* Upload signatures when uploading keys
* Fixes
* Disable the feature again
* Cross-signing groundwork
* Update to matrix-org/gomatrixserverlib#274
* Fix gobind builds, which stops unit tests in CI from yelling
* Some changes from review comments
* Fix build by passing in UIA
* Update to matrix-org/gomatrixserverlib@bec8d22
* Process master/self-signing keys from devices call
* nolint
* Enum-ify the key type in the database
* Process self-signing key too
* Fix sanity check in device list updater
* Fix check
* Fix sytest, hopefully
* Fix build
* It's half-alive
* Wakeups largely working
* Other tweaks, typing works
* Fix bugs, add receipt stream
* Delete notifier, other tweaks
* Dedupe a bit, add a template for the invite stream
* Clean up, add templates for other streams
* Don't leak channels
* Bring forward some more PDU logic, clean up other places
* Add some more wakeups
* Use addRoomDeltaToResponse
* Log tweaks, typing fixed?
* Fix timed out syncs
* Don't reset next batch position on timeout
* Add account data stream/position
* End of day
* Fix complete sync for receipt, typing
* Streams package
* Clean up a bit
* Complete sync send-to-device
* Don't drop errors
* More lightweight notifications
* Fix typing positions
* Don't advance position on remove again unless needed
* Device list updates
* Advance account data position
* Use limit for incremental sync
* Limit fixes, amongst other things
* Remove some fmt.Println
* Tweaks
* Re-add notifier
* Fix invite position
* Fixes
* Notify account data without advancing PDU position in notifier
* Apply account data position
* Get initial position for account data
* Fix position update
* Fix complete sync positions
* Review comments @Kegsay
* Room consumer parameters
* Update sync responses
* Fix positions, add ApplyUpdates
* Fix MarshalText as non-pointer, PrevBatch is optional
* Increment by number of read receipts
* Merge branch 'master' into neilalexander/devicelist
* Tweak typing
* Include keyserver position tweak
* Fix typing next position in all cases
* Tweaks
* Fix typo
* Tweaks, restore StreamingToken.MarshalText which somehow went missing?
* Rely on positions from notifier rather than manually advancing them
* Revert "Rely on positions from notifier rather than manually advancing them"
This reverts commit 53112a62cc.
* Give invites their own position, fix other things
* Fix test
* Fix invites maybe
* Un-whitelist tests that look to be genuinely wrong
* Use real receipt positions
* Ensure send-to-device uses real positions too
* Remove QueryBulkStateContent from current state server
Expected fail due to db impl not existing
* Implement query bulk state content
* Fix up rejecting invites over federation
* Fix bulk content marshalling
* First pass at server ACLs (not efficient)
* Use transaction origin, update whitelist
* Fix federation API test
It's sufficient for us to return nothing in response to current state, so that the server ACL check returns no ACLs.
* More efficient server ACLs - hopefully
* Fix queries
* Fix queries
* Avoid panics by nil pointers
* Bug fixes
* Fix state event type
* Fix mutex
* Update logging
* Ignore port when matching servername
* Use read mutex
* Fix bugs
* Fix sync API test
* Comments
* Add tests, tweaks to behaviour
* Fix test output
* Add tests for device list updates
* Add stale_device_lists table and use db before asking remote for device keys
* Fetch remote keys if all devices are requested
* Add display_name col to store remote device names
Few other tweaks to make `Server correctly handles incoming m.device_list_update`
pass.
* Fix sqlite otk bug
* Unbuffered channel to block /send causing sytest to not race anymore
* Linting and fix bug whereby we didn't send updated dl tokens to the client causing a tightloop on /sync sometimes
* No longer assert staleness as Update blocks on workers now
* Back out tweaks
* Bugfixes
* Add InputDeviceListUpdate
* Unbreak unit tests
* Process inbound device list updates from federation
- Persist the keys in the keyserver and produce key changes
- Does not currently fetch keys from the remote server if the prev IDs are missing
* Linting
* Add QueryDeviceMessages to serve up device keys and stream IDs
* Consume key change events in fedsender
Don't yet send them to destinations as we haven't worked them out yet
* Send device list updates to all required servers
* Glue it all together
* Fix New users appear in /keys/changes
* Create blank device keys when logging in on a new device
* Add PerformDeviceUpdate and fix a few bugs
- Correct device deletion query on sqlite
- Return no keys on /keys/query rather than an empty key
* Unbreak sqlite properly
* Use a real DB for currentstateserver integration tests
* Race fix
* Recheck device lists when join/leave events come in
* Add PerformDeviceDeletion
* Notify clients when devices are deleted
* Unbreak things
* Remove debug logging
