Commit graph

1756 commits

Author SHA1 Message Date
kegsay 7dc8fb1fe7
Add more logs (#2005)
* Add more logs

To help debug the migration issue in #1924 along with manual data-loss-inducing fixes.
Also log the origin server on processed txns to help debug buggy server origins.

* Fix query
2021-09-07 15:07:14 +01:00
Neil Alexander 51b119107c
Don't return nonsense canonical room aliases in the public rooms responses (#1992) 2021-08-27 16:50:30 +01:00
Neil Alexander 2dd5fd1fd6
publicRooms should accept POST as well as GET (#1991) 2021-08-27 15:48:27 +01:00
Neil Alexander 037ff4fb23
Version 0.5.0 2021-08-24 15:03:28 +01:00
Neil Alexander 4b0cc32af2
Hopefully fix UIA for cross-signing setup (closes #1983) (#1986) 2021-08-24 11:44:03 +01:00
Neil Alexander 329c9e671c
Update to matrix-org/pinecone@e692df1 2021-08-19 16:07:28 +01:00
Neil Alexander 9a2dcd6078
Update to matrix-org/pinecone@21e3b39228 2021-08-19 15:02:50 +01:00
Neil Alexander cd3823308c
Version 0.5.0rc1 2021-08-18 14:29:45 +01:00
Neil Alexander cf42ae6064
Build Docker images using Go 1.17 2021-08-18 14:13:38 +01:00
Kevin Liu fe5050c0a5
Add and use M_ROOM_IN_USE (#1972)
Signed-off-by: nivekuil <mail@nivekuil.com>

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2021-08-18 12:41:30 +01:00
Neil Alexander 614e67280d
Delete device keys/signatures from key server when deleting devices (#1979)
* Delete device keys/signatures from key server when deleting device from user API

* Move loop to within database transaction

* Don't fall over deleting no rows
2021-08-18 12:07:09 +01:00
Sambhav Saggi 1bee1ae204
Syslog integration (#1952)
* Syslog integration, part 1

* Add protocol, make sure syslog actually logs

* Make golangci-lint happy about shadow variables

* Add syslog tag, wrap syslog in logLevelHook
2021-08-17 16:18:51 +01:00
Neil Alexander ff21675c5b
Cross-signing fixes, notifications via sync, federation (#1974)
* Initial work on signing key update EDUs

* Fix build

* Produce/consume EDUs

* Producer logging

* Only produce key change notifications for local users

* Better naming

* Try to notify sync

* Enable feature

* Use key change topic

* Don't bother verifying signatures, validate key lengths if we can, notifier fixes

* Copyright notices

* Remove tests from whitelist until matrix-org/sytest#1117

* Some review comment fixes

* Update to matrix-org/gomatrixserverlib@f9416ac

* Remove unneeded parameter
2021-08-17 13:44:30 +01:00
S7evinK 8a4b90b7dd
Update golangci-lint (#1978)
* Update golangci-lint

* Use unconditional strings.TrimSuffix

* Add error checks

Signed-off-by: Till Faelligen <tfaelligen@gmail.com>

* Disable lint typecheck

* Fix ineffectual error assignment (#1976)

Was working on another PR and noticed that golangci-lint was failing
locally on `ineffassign`

Signed-off-by: Devon Mizelle <dev@devon.so>

* Revert "Disable lint typecheck"

This reverts commit 3c76f219d3.

Co-authored-by: Devon Mizelle <dev@devon.so>
2021-08-17 10:39:09 +01:00
Devon Mizelle c2b9ab7470
Fix ineffectual error assignment (#1976)
Was working on another PR and noticed that golangci-lint was failing
locally on `ineffassign`

Signed-off-by: Devon Mizelle <dev@devon.so>
2021-08-16 14:19:35 +01:00
Neil Alexander 125ea75b24
Add type field to DeviceMessage, allow fields to be nullable (#1969) 2021-08-11 09:44:14 +01:00
Neil Alexander b1377d991a
Cross-signing signature handling (#1965)
* Handle other signatures

* Decorate key ID properly

* Match by key IDs

* Tweaks

* Fixes

* Fix /user/keys/query bug, review comments, update sytest-whitelist

* Various wtweaks

* Fix wiring for keyserver in API mode

* Additional fixes
2021-08-09 14:35:24 +01:00
Neil Alexander e95b1fd238
Cross-signing validation for self-sigs, expose signatures over /user/keys/query and /user/devices/{userId} (#1962)
* Enable unstable feature again

* Try to verify when a device signs a key

* Try to verify when a key signs a device

* It's the self-signing key, not the master key

* Fix error

* Try to verify master key uploads

* Actually we can't guarantee we can do that so nevermind

* Add signatures into /devices/list request

* Fix nil pointer

* Reprioritise map creation

* Don't skip devices that don't have signatures

* Add some debug logging

* Fix logic error in QuerySignatures

* Fix bugs

* Expose master and self-signing keys on /devices/list hopefully

* maps are tedious

* Expose signatures via /keys/query

* Upload signatures when uploading keys

* Fixes

* Disable the feature again
2021-08-06 10:13:35 +01:00
Neil Alexander 8e5a0139b5
Update to matrix-org/gomatrixserverlib@03e40fa 2021-08-05 11:48:22 +01:00
Neil Alexander eb0efa4636
Cross-signing groundwork (#1953)
* Cross-signing groundwork

* Update to matrix-org/gomatrixserverlib#274

* Fix gobind builds, which stops unit tests in CI from yelling

* Some changes from review comments

* Fix build by passing in UIA

* Update to matrix-org/gomatrixserverlib@bec8d22

* Process master/self-signing keys from devices call

* nolint

* Enum-ify the key type in the database

* Process self-signing key too

* Fix sanity check in device list updater

* Fix check

* Fix sytest, hopefully

* Fix build
2021-08-04 17:56:29 +01:00
kegsay 4cc8b28b7f
Ensure all create events have a snapshot NID of 0 (#1961)
Fixes #1924 for postgres users, though the underlying cause of why
they aren't 0 in the first place is unresolved.
2021-08-04 17:48:23 +01:00
Neil Alexander 7a9a2547b3
Cross-signing storage code (#1959) 2021-08-04 17:31:18 +01:00
kegsay ed04eed441
Fix sqlite migration issues (#1960)
* Do not store 'null' in the database for empty JSON arrays

This can cause issues, though it should be noted that the majority
of the time this will marshal/unmarshal just fine, see
https://play.golang.org/p/Doe2NZUgv7Q

* bugfix: sqlite migration should handle create events as having no 'before' snapshot

The state snapshot for any given event in the roomserver represents the state _before_
the event. For the create event, this is nothing, so the state snapshot nid should be 0.

In some cases this wasn't happening, resulting in a nice mix of possible options including:
 - A state snapshot without any state blocks `[]` or `null`.
 - A state snapshot with a single state block with a single event, the create event, causing
   a circular loop. This is incorrect as it represents the state before the event, not after.

* Add state key check
2021-08-04 17:08:17 +01:00
Neil Alexander da101469fa
Minimum version Go 1.15 (#1957) 2021-08-03 11:35:29 +01:00
kegsay e3df612953
Add tracing to user API (#1948)
Use the trace version in tests so we can just implement the required API functions.
2021-08-03 11:23:25 +01:00
Neil Alexander d4aa311c74
Update containerd dependency 2021-08-03 11:17:27 +01:00
Neil Alexander a81c499f65
Update dependencies (#1956) 2021-08-03 11:15:17 +01:00
kegsay a22ab3eee6
Read the right username field when performing account deactivation (#1954)
`Login` has 2 username fields, and we were always checking the
deprecated one. Instead, check both.
2021-08-02 13:11:23 +01:00
Kegan Dougal ed4097825b Factor out StatementList to sqlutil and use it in userapi
It helps with the boilerplate.
2021-07-28 18:30:04 +01:00
Neil Alexander 9e4618000e
Alias key backup endpoints onto /unstable, fix key backup bugs (#1947)
* Default /unstable requests to stable endpoints if not overridden specifically with a custom route

* Rewrite URL

* Try something different

* Fix routing manually

* Fix selectLatestVersionSQL

* Don't return 0 if no backup version exists

* Log more useful error

* fix up replace keys check

* Don't enforce uniqueness on e2e_room_keys_versions_idx

Co-authored-by: kegsay <kegan@matrix.org>
2021-07-28 10:25:45 +01:00
Neil Alexander 3e01a88a0c
Update to neilalexander/utp@54ae7b1 2021-07-27 21:34:40 +01:00
kegsay 32bf14a37c
Key Backups (3/3) : Implement querying keys and various bugfixes (#1946)
* Add querying device keys

Makes a bunch of sytests pass

* Apparently only the current version supports uploading keys

* Linting
2021-07-27 19:29:32 +01:00
kegsay b3754d68fc
Key Backups (2/3) : Add E2E backup key tables (#1945)
* Add PUT key backup endpoints and glue them to PerformKeyBackup

* Add tables for storing backup keys and glue them into the user API

* Don't create tables whilst still WIPing

* writer on sqlite please

* Linting
2021-07-27 17:08:53 +01:00
kegsay a060df91e2
Use db writer on sqlite account table (#1944) 2021-07-27 12:47:50 +01:00
kegsay 32538640db
Key backups (1/2) : Add E2E session backup metadata tables (#1943)
* Initial key backup paths and userapi API

* Fix unit tests

* Add key backup table

* Glue REST API to database

* Linting

* use writer on sqlite
2021-07-27 12:47:32 +01:00
Neil Alexander 4704957dab
Merge branch 'master' into add-nats-support 2021-07-26 12:56:49 +01:00
Neil Alexander e3679799ea
Version 0.4.1 2021-07-26 12:52:11 +01:00
kegsay 16bf94f239
Not finding the snapshot is not fatal (#1940) 2021-07-26 12:30:44 +01:00
Till Faelligen a7ad4c182f Revert changes made to streams 2021-07-26 12:08:35 +02:00
Till Faelligen 5f9996b1c5 Revert "Remove unneeded TopicFor"
This reverts commit f5a4e4a339.
2021-07-26 12:06:09 +02:00
Meenal Trivedi fa1ec482a7
fix:Inviting to an unsupported room version return M_BAD_JSON instead of Incompatible_Version (#1930)
* fix:Inviting to an unsupported room version return M_BAD_JSON instead of M_UNSUPPORTED_ROOM_VERSION

Signed-off-by: Meenal Trivedi <meenaltrivedi6102@gmail.com>

* fix

Signed-off-by: Meenal Trivedi <meenaltrivedi6102@gmail.com>

* fix

Signed-off-by: Meenal Trivedi <meenaltrivedi6102@gmail.com>

* feat: make requested changes

Signed-off-by: Meenal Trivedi <meenaltrivedi6102@gmail.com>

* Use error typecast from matrix-org/gomatrixserverlib#272

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2021-07-26 10:41:58 +01:00
Till Faelligen 9ac699b77a Dep update 2021-07-24 12:28:51 +02:00
Till Faelligen f5a4e4a339 Remove unneeded TopicFor 2021-07-24 12:17:42 +02:00
Till Faelligen a833f5764a Merge branch 'master' of https://github.com/matrix-org/dendrite into add-nats-support 2021-07-24 11:27:24 +02:00
Till Faelligen 1436ac8a51 Adapt changes to config 2021-07-23 20:45:48 +02:00
Till Faelligen bc3fc705f7 Merge branch 'add-nats-support' of github.com:S7evinK/dendrite into add-nats-support 2021-07-23 20:35:11 +02:00
Neil Alexander 75d0f009ec
Don't set prev state when it is the same as the event it replaces (#1936) 2021-07-22 16:46:36 +01:00
Neil Alexander 39e8d1cc6f
Track knocking in membership updater (#1935)
* Topologically sort outliers in SendEventWithState

* Knock in membership updater

* Update gomatrixserverlib

* Update gomatrixserverlib

* Get the NID of the knock event properly for the membership updater
2021-07-22 12:26:58 +01:00
Neil Alexander 43ac66e0b4
Update to matrix-org/gomatrixserverlib@b9eb787 2021-07-22 09:46:31 +01:00
Neil Alexander 9008b1c111
Update to matrix-org/gomatrixserverlib@ae88543 2021-07-21 18:46:31 +01:00