Commit graph

1776 commits

Author SHA1 Message Date
Till Faelligen 6482630f7b Fix insert statement 2022-02-15 14:28:51 +01:00
Till Faelligen 2fc1c46743 Merge branch 'main' of github.com:matrix-org/dendrite into s7evink/consent-tracking 2022-02-15 14:16:37 +01:00
Till Faelligen 5a0ec6e443 Add policy version to create-account & mediaapi 2022-02-15 14:15:18 +01:00
Till Faelligen 535d388ec0 Add new login type "m.login.terms" 2022-02-15 14:14:39 +01:00
Till Faelligen cbdbbb0839 Make sure we use the correct login stages 2022-02-15 14:13:22 +01:00
Till Faelligen f8bebe5e5a Add policy_version to insertAccount statement 2022-02-15 14:10:50 +01:00
Neil Alexander e22e87c012
Update to matrix-org/gomatrixserverlib@20632dd 2022-02-15 11:14:43 +00:00
Till Faelligen d19518fca5 Add ConsentNotGiven error
Verify consent on desired endpoints
Store consent on POST requests
2022-02-15 11:07:24 +01:00
Till Faelligen 89340cfc52 Verify the user has given their consent, otherwise block access 2022-02-14 18:11:56 +01:00
Till Faelligen 11144de92f Implement consent tracking 2022-02-14 16:18:51 +01:00
Till Faelligen b2045c24cb Add missing yaml tag 2022-02-14 16:18:19 +01:00
Till Faelligen 097f1d4609 Add a way to update the policy_version for a user 2022-02-14 15:08:00 +01:00
Till Faelligen a505471c90 Add table migrations 2022-02-14 14:52:16 +01:00
Till Faelligen 3c5c3ea7fb Add methods to query the policy version 2022-02-14 14:03:30 +01:00
Till Faelligen 9583784e8a Add new coloumn to track accepted policy version 2022-02-14 14:02:13 +01:00
Till Faelligen b6ee34918c Add consent tracking endpoint 2022-02-14 13:41:21 +01:00
Till Faelligen ac343861ad Add missing form_secret
Add tests
2022-02-14 13:06:36 +01:00
Till Faelligen 4da7df5e3e Add consent tracking template 2022-02-14 13:01:26 +01:00
Till Faelligen ccc11f94f7 Add consentAPIMux to components 2022-02-14 13:00:07 +01:00
Till Faelligen 5702b84dae Add User consent configuration
Add consentAPIMux
2022-02-14 12:59:13 +01:00
Neil Alexander 5106cc807c
Ensure only one transaction is used for RS input per room (#2178)
* Ensure the input API only uses a single transaction

* Remove more of the dead query API call

* Tidy up

* Fix tests hopefully

* Don't do unnecessary work for rooms that don't exist

* Improve error, fix another case where transaction wasn't used properly

* Add a unit test for checking single transaction on RS input API

* Fix logic oops when deciding whether to use a transaction in storeEvent
2022-02-11 17:40:14 +00:00
S7evinK a4e7d471af
Remove FederationDisabled error type (#2174) 2022-02-11 18:15:44 +01:00
kegsay a566d53b0b
Don't allow parallel complement tests (#2169)
Fixes flakiness seemingly. See https://github.com/matrix-org/synapse/pull/11910
2022-02-11 16:26:23 +00:00
Neil Alexander 88b45d5cd2
Drop m.room.create events in federation /send transaction (#2179) 2022-02-11 15:18:14 +00:00
kegsay 4e75ab9930
Add postgres complement support (#2177) 2022-02-11 12:35:47 +00:00
kegsay f800cae6d2
Point to /complement/ca not /ca (#2172) 2022-02-10 18:12:11 +00:00
Neil Alexander 9ac27cabc5
Version 0.6.3 (#2170) 2022-02-10 13:50:13 +00:00
tommie c36e4546c3
Support for m.login.token (#2014)
* Add GOPATH to PATH in find-lint.sh.

The user doesn't necessarily have it in PATH.

* Refactor LoginTypePassword and Type to support m.login.token and m.login.sso.

For login token:

* m.login.token will require deleting the token after completeAuth has
  generated an access token, so a cleanup function is returned by
  Type.Login.
* Allowing different login types will require parsing the /login body
  twice: first to extract the "type" and then the type-specific parsing.
  Thus, we will have to buffer the request JSON in /login, like
  UserInteractive already does.

For SSO:

* NewUserInteractive will have to also use GetAccountByLocalpart. It
  makes more sense to just pass a (narrowed-down) accountDB interface
  to it than adding more function pointers.

Code quality:

* Passing around (and down-casting) interface{} for login request types
  has drawbacks in terms of type-safety, and no inherent benefits. We
  always decode JSON anyway. Hence renaming to Type.LoginFromJSON. Code
  that directly uses LoginTypePassword with parsed data can still use
  Login.
* Removed a TODO for SSO. This is already tracked in #1297.
* httputil.UnmarshalJSON is useful because it returns a JSONResponse.

This change is intended to have no functional changes.

* Support login tokens in User API.

This adds full lifecycle functions for login tokens: create, query, delete.

* Support m.login.token in /login.

* Fixes for PR review.

* Set @matrix-org/dendrite-core as repository code owner

* Return event NID from `StoreEvent`, match PSQL vs SQLite behaviour, tweak backfill persistence (#2071)

Co-authored-by: kegsay <kegan@matrix.org>
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-02-10 10:27:26 +00:00
S7evinK 432c35a307
Allow user to forget a room, even if they never were a member (#2166)
* Allow user to forget a room, even if they never were a member

* Return "M_UNKNOWN" as per the spec

Co-authored-by: kegsay <kegan@matrix.org>
2022-02-10 10:05:37 +00:00
Neil Alexander 2782ae3d56
Fix fetching missing state (#2163)
* Check that we have a populated state snapshot when determining if we closed the gap

* Do the same in the query API

* Use HasState more opportunistically

* Try to avoid falling down the hole of using a trustworthy but empty state snapshot for non-create events

* Refactor missing state and make sure that we really solve the problem for the new event

* Comments

* Review comments

* Tweak that check again

* Tidy up that create check further

* Fix build hopefully

* Update sendOutliers to use OrderAuthAndStateEvents

* Don't go out of bounds on missingEvents
2022-02-10 10:05:14 +00:00
kegsay 9130156b13
Make the Complement Dockerfile use a fresh directory for runtime (#2168) 2022-02-10 09:37:46 +00:00
Neil Alexander 37cbe263ce
Fix transaction issues in events table in PSQL (#2165)
* Revert "Revert "Fix storage bug in PSQL events table""

This reverts commit cf447dd52a.

* Membership updater to use updater

* Fix membership updater to use transactions properly
2022-02-10 09:30:16 +00:00
kegsay aa5c3b88de
Unmarshal events at the Dendrite level not GMSL level (#2164)
* Use new event json types in gmsl

* Fix EventJSON to actually unmarshal events

* Update GMSL

* Bump GMSL and improve error messages

* Send back the correct RespState

* Update GMSL
2022-02-09 20:31:24 +00:00
S7evinK cc688a9a38
Avoid unnecessary logs and marshaling (#2167)
Co-authored-by: kegsay <kegan@matrix.org>
2022-02-09 15:46:52 +01:00
S7evinK ac25065a54
Fix sytest uploading signed devices gets propagated over federation (#2162)
* Remove unneeded logging

* Add MasterKey & SelfSigningKey to update
Avoid panic if signatures are not present

* Add passing test

* Revert "Add MasterKey & SelfSigningKey to update"

This reverts commit 2c81b34884.

* Send MasterKey & SelfSigningKey with update

* Debugging

* Remove delete() so we also query signingkeys
2022-02-09 13:11:43 +01:00
Neil Alexander cf447dd52a
Revert "Fix storage bug in PSQL events table"
This reverts commit b4687f2ed2.
2022-02-09 11:41:21 +00:00
Neil Alexander b4687f2ed2
Fix storage bug in PSQL events table 2022-02-09 11:24:49 +00:00
S7evinK 2771d93748
Remove OutputKeyChangeEvent consumer on keyserver (#2160)
* Remove keyserver consumer

* Remove keyserver from eduserver

* Directly upload device keys without eduserver

* Add passing tests
2022-02-08 18:13:38 +01:00
Neil Alexander 457a07eac5
More relaxed auth event fetching (#2161)
* Tweaks around auth event fetching

* More tweaking
2022-02-08 17:06:13 +00:00
Neil Alexander a84f50f4fb
Demote logging entry for backoff 2022-02-08 16:49:49 +00:00
Neil Alexander bb39149ff8
Fix DendriteJS dockerfile 2022-02-08 16:18:16 +00:00
Neil Alexander 8a1dfffe3d
Various updates for renaming the master branch to main 2022-02-08 16:16:01 +00:00
Neil Alexander 0e26662a55
Allow events to be un-rejected (#2159)
* Allow un-rejecting an event later

* SQL

* Only un-reject, don't re-reject

* Clarify ambiguous column reference
2022-02-08 13:45:48 +00:00
Neil Alexander a2b4777ae5
Update to matrix-org/gomatrixserverlib@a05e156fd8 2022-02-08 09:30:21 +00:00
Neil Alexander a572f4db03
Fix bugs that could wedge rooms (#2154)
* Don't flake so badly for rejected events

* Moar

* Fix panic

* Don't count rejected events as missing

* Don't treat rejected events without state as missing

* Revert "Don't count rejected events as missing"

This reverts commit 4b6139b62e.

* Missing events should be KindOld

* If we have state, use it, regardless of memberships which could be stale now

* Fetch missing state for KindOld too

* Tweak the condition again

* Clean up a bit

* Use room updater to get latest events in a race-free way

* Return the correct error

* Improve errors
2022-02-07 19:10:01 +00:00
Neil Alexander 908d881a6e
Version 0.6.2 2022-02-04 17:49:01 +00:00
Neil Alexander 00cbe75150
Fix CPU spin from key change consumer when an invalid message is supplied (#2146) 2022-02-04 16:16:50 +00:00
Neil Alexander 585ced89bd
Version 0.6.1 (#2145) 2022-02-04 14:44:45 +00:00
S7evinK 9de7efa0b0
Remove sarama/saramajetstream dependencies (#2138)
* Remove dependency on saramajetstream & sarama

Signed-off-by: Till Faelligen <tfaelligen@gmail.com>

* Remove internal.ContinualConsumer from federationapi

* Remove internal.ContinualConsumer from syncapi

* Remove internal.ContinualConsumer from keyserver

* Move to new Prepare function

* Remove saramajetstream & sarama dependency

* Delete unneeded file

* Remove duplicate import

* Log error instead of silently irgnoring it

* Move `OffsetNewest` and `OffsetOldest` into keyserver types, change them to be more sane values

* Fix comments

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-02-04 13:08:13 +00:00
Neil Alexander 532f445c4e
Remove roomserver input deadlines (#2144)
It isn't really clear that the deadlines actually help in any way. Currently we can use up our 2 minutes doing something, run out of context time and then return an error which causes the transaction to rollback and forgetting everything we've done. If the message came to us from NATS then we probably will end up retrying just to be in the same situation. We'd be really a lot better if we just spent the time reconciling the problem in the first place, and then we're much less likely to need to fetch those missing auth or prev events in the future.

Also includes matrix-org/gomatrixserverlib#287 so we don't wait so long for servers that are obviously dead.
2022-02-04 12:13:07 +00:00