Commit graph

2596 commits

Author SHA1 Message Date
Neil Alexander 4c07374c42
Reduce allocations significantly in state res v2, which should help to keep memory down when joining rooms too (update to matrix-org/gomatrixserverlib@f6ab9c5) 2022-02-24 17:05:49 +00:00
Neil Alexander 4b01f1cd12
State resolution v2 micro-optimisations (#2226)
* Don't populate duplicates into auth events

* Only append the single event

* Potentially reduce number of iterations in `isInAllAuthLists
2022-02-24 11:09:01 +00:00
Neil Alexander fea8d152e7
Relax roomserver input transactional isolation (#2224)
* Don't force full transactional isolation on roomserver input

* Set succeeded

* Tweak `MissingAuthPrevEvents`
2022-02-23 15:41:32 +00:00
Neil Alexander b8a97b6ee0
Update to matrix-org/pinecone@0f0afd1a46 2022-02-23 10:45:07 +00:00
Neil Alexander 2b0a5adfaf
Version 0.6.4 (#2212)
* Version 0.6.4

* Tweaks

* Update changelog

* Update changelog one last time
2022-02-22 16:55:08 +00:00
S7evinK 41dc651b25
Send device update to local users if remote display name changes (#2215)
* Send device_list update to satisfy sytest

* Fix build issue from merged in change

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-02-22 16:34:53 +00:00
S7evinK cfff1b0aaa
Remote banned user is kicked and may not rejoin until unbanned (#2216)
* Remote banned user is kicked and may not rejoin until unbanned

* Use gmsl constant
2022-02-22 15:50:04 +00:00
Neil Alexander c7811e9d71
Add DeviceKeysEqual (#2219)
* Add `DeviceKeysEqual`

* Update check order

* Fix check

* Tweak conditions again

* One more time

* Single return value
2022-02-22 15:43:17 +00:00
Neil Alexander 34116178e8
Remove logging line in PerformInvite 2022-02-22 13:47:14 +00:00
Neil Alexander e9545dc12f
Remove error when state keys are missing for user NIDs (#2213)
* Remove error when state keys are missing for user NIDs

There is still an actual bug here somewhere in the membership updater, but this check does more harm than good, since it means that the key consumers don't actually distribute updates to *anyone*. It's better just to deal with this silently for now.

To find these broken rows:

```
SELECT * FROM roomserver_membership AS m WHERE NOT EXISTS (
	SELECT event_state_key_nid FROM roomserver_event_state_keys AS s
	WHERE m.sender_nid = s.event_state_key_nid
);
```

* Logging
2022-02-22 13:40:08 +00:00
Neil Alexander 600fbae31f
Only emit key change notifications from federation when changes are made (#2217)
* Only emit key changes when poked over federation

* Remove logging

* Fix unit test possibly
2022-02-22 13:35:06 +00:00
Neil Alexander bbe7d37928
Fix logic error on context history visibility (#2211) 2022-02-21 16:38:53 +00:00
Neil Alexander aa6bbf484a
Return ErrRoomNoExists if insufficient state is available for a buildEvent to succeed when joining a room (#2210)
This may help cases like #2206, since it should prompt us to try a federated join again instead.
2022-02-21 16:22:29 +00:00
S7evinK cf525d1f61
Implement /context (#2207)
* Add QueryEventsAfter

* Add /context

* Make all tests pass on sqlite

* Add queries to get the events for /context requests

* Move /context to the syncapi

* Revert "Add QueryEventsAfter"

This reverts commit 440a771d10.

* Simplify getting the required events

* Apply RoomEventFilter when getting events

* Add passing tests

* Remove logging

* Remove unused SQL statements
Update comments & add TODO
2022-02-21 17:12:22 +01:00
Benjamin Nater 280e9b19a1
Don't ignore config flag for yggdrasil demo if set (#2070)
* Don't ignore config flag for yggdrasil demo if set

Signed-off-by: bn4t <me@bn4t.me>

* run goimports

Signed-off-by: bn4t <me@bn4t.me>

* always override ServerName, PrivateKey and KeyID

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-02-21 15:36:03 +00:00
Neil Alexander a02dd7721d
Reset invalid state snapshots for events during state storage refactor migration (#2209)
This should help with #2204. We can't do this for rooms, only events.
2022-02-21 15:25:54 +00:00
Neil Alexander a386fbed2c
Delete one-time keys when deleting a device (#2208) 2022-02-21 12:30:43 +00:00
S7evinK 002429c9e2
Implement server notices (#2180)
* Add server_notices config

* Disallow rejecting "server notice" invites

* Update config

* Slightly refactor sendEvent and CreateRoom so it can be reused

* Implement unspecced server notices

* Validate the request

* Set the user api when starting

* Rename function/variables

* Update comments

* Update config

* Set the avatar on account creation

* Update test

* Only create the account when starting
Only add routes if sever notices are enabled

* Use reserver username
Check that we actually got roomData

* Add check for admin account
Enable server notices for CI
Return same values as Synapse

* Add custom error for rejecting server notice invite

* Move building an invite to it's own function, for reusability

* Don't create new rooms, use the existing one (follow Synapse behavior)

Co-authored-by: kegsay <kegan@matrix.org>
2022-02-18 16:05:03 +01:00
Neil Alexander dbded87525
Expose sync endpoints via /v3 (#2203) 2022-02-18 14:14:16 +00:00
Neil Alexander 9f4a39e8e0
Refactor user API storage (#2202)
* Refactor User API database

* Fix migration bugs
2022-02-18 13:51:59 +00:00
Neil Alexander 9bd5e414c9
Missing commit from #2186 2022-02-18 11:32:45 +00:00
Neil Alexander 153bfbbea5
Merge both user API databases into one (#2186)
* Merge user API databases into one

* Remove DeviceDatabase from config

* Fix tests

* Try that again

* Clean up keyserver device keys when the devices no longer exist in the user API

* Tweak ordering

* Fix UserExists flag, device check

* Allow including empty entries so we can clean them up

* Remove logging
2022-02-18 11:31:05 +00:00
kegsay 0a7dea4450
Update /whoami response to match Spec v1.2 (#2201)
Basically include `is_guest` and `device_id`. The latter is
needed for https://github.com/matrix-org/complement/pull/305
2022-02-18 11:28:02 +00:00
Neil Alexander 131bedc1a1
Remove event type and state key caches (#2200)
* Don't proactively cache event types and state keys when we don't know if the transaction has persisted yet

* Remove event type and state key caches altogether
2022-02-18 10:58:41 +00:00
Neil Alexander e1eb5807b6
Allow preventing guest registration (#2199)
* Allow disabling guest registration separately

* Update sample config

* Set `guests_disabled` to `true` in the sample config
2022-02-18 10:12:26 +00:00
kegsay 5dd203fde3
Listen for /v3 on CSAPI (#2197)
* Listen for /v3 on CSAPI

* Docs

* More docs

* Rename path variable to fix key backup tests

* Update routing.go

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-02-17 17:38:22 +00:00
Neil Alexander 0b123b29f5
Use process context for roomserver input (#2198) 2022-02-17 15:58:54 +00:00
Neil Alexander 140077265e
Make GetUserDevices logging entry more useful 2022-02-17 15:02:06 +00:00
Neil Alexander 7dfc7c3d70
Don't re-send sent events in add_state_events (#2195)
* Only add events to `add_state_events` that haven't already been sent to the roomserver output before

* Filter on event NIDs instead, hopefully bring joy to SQLite

* UnsentFilter, review comments
2022-02-17 13:53:48 +00:00
Neil Alexander 353168a9e9
Fix potential panic in NewStreamTokenFromString caused by off-by-one error (#2196)
Line 291 could panic when trying to set `positions[i]` if `i == len(positions)`.
2022-02-17 13:25:41 +00:00
S7evinK 89b7519089
Raise waitTime for network related issues (#2192) 2022-02-17 13:15:49 +00:00
Neil Alexander 934491eda5
Update NATS Server to v2.7.2 (#2193)
* Update NATS JetStream to v2.7.2

* Remove deprecated option
2022-02-17 13:15:35 +00:00
S7evinK f51e2a99e9
Remove outbound proxy, http.ProxyFromEnvironment is now used (#2191) 2022-02-17 13:54:29 +01:00
kegsay a4681bc7f7
Set 'complement' as the shared secret for CI (#2194) 2022-02-17 10:59:44 +00:00
S7evinK 5a39512f5f
Add account type (#2171)
* Add account_type for sqlite3

* Add account_type for postgres

* Remove CreateGuestAccount from interface

* Add new AccountTypes & update test

* Use newly added AccountType for account creation

* Add migrations

* Reuse type

* Add AccounnType to Device, so it can be verified on requests

* Rename migration, add missing update for appservices

* Rename sqlite3 migration

* Add missing AccountType to return value

* Update sqlite migration
Change allowance check on /admin/whois

* Fix migration, add IS NULL

* Move accountType to completeRegistration

* Fix migrations

* Add passing test
2022-02-16 18:55:38 +01:00
S7evinK e9b672a34e
Make "Device list doesn't change if remote server is down" pass (#2190) 2022-02-16 16:56:45 +00:00
Neil Alexander fa1e12b503
Don't panic on retiring an invite that we haven't seen yet (#2189) 2022-02-16 11:56:08 +00:00
kegsay f92b048fec
Add host mount compatible Complement image (#2187) 2022-02-15 17:40:48 +00:00
Neil Alexander 4c8c53244e
Update prometheus 2022-02-15 16:27:22 +00:00
Neil Alexander e22e87c012
Update to matrix-org/gomatrixserverlib@20632dd 2022-02-15 11:14:43 +00:00
Neil Alexander 5106cc807c
Ensure only one transaction is used for RS input per room (#2178)
* Ensure the input API only uses a single transaction

* Remove more of the dead query API call

* Tidy up

* Fix tests hopefully

* Don't do unnecessary work for rooms that don't exist

* Improve error, fix another case where transaction wasn't used properly

* Add a unit test for checking single transaction on RS input API

* Fix logic oops when deciding whether to use a transaction in storeEvent
2022-02-11 17:40:14 +00:00
S7evinK a4e7d471af
Remove FederationDisabled error type (#2174) 2022-02-11 18:15:44 +01:00
kegsay a566d53b0b
Don't allow parallel complement tests (#2169)
Fixes flakiness seemingly. See https://github.com/matrix-org/synapse/pull/11910
2022-02-11 16:26:23 +00:00
Neil Alexander 88b45d5cd2
Drop m.room.create events in federation /send transaction (#2179) 2022-02-11 15:18:14 +00:00
kegsay 4e75ab9930
Add postgres complement support (#2177) 2022-02-11 12:35:47 +00:00
kegsay f800cae6d2
Point to /complement/ca not /ca (#2172) 2022-02-10 18:12:11 +00:00
Neil Alexander 9ac27cabc5
Version 0.6.3 (#2170) 2022-02-10 13:50:13 +00:00
tommie c36e4546c3
Support for m.login.token (#2014)
* Add GOPATH to PATH in find-lint.sh.

The user doesn't necessarily have it in PATH.

* Refactor LoginTypePassword and Type to support m.login.token and m.login.sso.

For login token:

* m.login.token will require deleting the token after completeAuth has
  generated an access token, so a cleanup function is returned by
  Type.Login.
* Allowing different login types will require parsing the /login body
  twice: first to extract the "type" and then the type-specific parsing.
  Thus, we will have to buffer the request JSON in /login, like
  UserInteractive already does.

For SSO:

* NewUserInteractive will have to also use GetAccountByLocalpart. It
  makes more sense to just pass a (narrowed-down) accountDB interface
  to it than adding more function pointers.

Code quality:

* Passing around (and down-casting) interface{} for login request types
  has drawbacks in terms of type-safety, and no inherent benefits. We
  always decode JSON anyway. Hence renaming to Type.LoginFromJSON. Code
  that directly uses LoginTypePassword with parsed data can still use
  Login.
* Removed a TODO for SSO. This is already tracked in #1297.
* httputil.UnmarshalJSON is useful because it returns a JSONResponse.

This change is intended to have no functional changes.

* Support login tokens in User API.

This adds full lifecycle functions for login tokens: create, query, delete.

* Support m.login.token in /login.

* Fixes for PR review.

* Set @matrix-org/dendrite-core as repository code owner

* Return event NID from `StoreEvent`, match PSQL vs SQLite behaviour, tweak backfill persistence (#2071)

Co-authored-by: kegsay <kegan@matrix.org>
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-02-10 10:27:26 +00:00
S7evinK 432c35a307
Allow user to forget a room, even if they never were a member (#2166)
* Allow user to forget a room, even if they never were a member

* Return "M_UNKNOWN" as per the spec

Co-authored-by: kegsay <kegan@matrix.org>
2022-02-10 10:05:37 +00:00
Neil Alexander 2782ae3d56
Fix fetching missing state (#2163)
* Check that we have a populated state snapshot when determining if we closed the gap

* Do the same in the query API

* Use HasState more opportunistically

* Try to avoid falling down the hole of using a trustworthy but empty state snapshot for non-create events

* Refactor missing state and make sure that we really solve the problem for the new event

* Comments

* Review comments

* Tweak that check again

* Tidy up that create check further

* Fix build hopefully

* Update sendOutliers to use OrderAuthAndStateEvents

* Don't go out of bounds on missingEvents
2022-02-10 10:05:14 +00:00