Commit graph

205 commits

Author SHA1 Message Date
Till Faelligen 4b838672e8
PR review comments; Make request a POST, tests 2023-01-16 11:43:36 +01:00
Till Faelligen 2e5ef9c711
Make use of sqlutil.StatementList, dedupe NIDs, timeout test 2022-12-22 15:43:29 +01:00
Till Faelligen e7356f7e96
Minor cleanup 2022-12-22 14:54:13 +01:00
Till Faelligen e779b5b3a8
Initial purge room support for SQLite 2022-12-22 14:48:58 +01:00
Till Faelligen c10f563510
Minor fixes/changes; Initial test 2022-12-21 16:11:03 +01:00
Till Faelligen e9336df152
Merge branch 'main' of github.com:matrix-org/dendrite into neilalexander/purgeroom 2022-12-21 14:20:25 +01:00
Till 7d2344049d
Cleanup stale device lists for users we don't share a room with anymore (#2857)
The stale device lists table might contain entries for users we don't
share a room with anymore. This now asks the roomserver about left users
and removes those entries from the table.

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-12-12 08:20:59 +01:00
Neil Alexander 321e6a1e23
Merge branch 'main' into neilalexander/purgeroom 2022-12-01 09:50:27 +00:00
Till 2a77a910eb
Handle remote room upgrades (#2866)
Makes the following tests pass
```
/upgrade moves remote aliases to the new room
Local and remote users' homeservers remove a room from their public directory on upgrade
```
2022-11-14 12:07:13 +00:00
Till 1e79b0557e
Use a writer to assign state key NIDs (#2877) 2022-11-14 12:06:27 +00:00
Till Faelligen e177e0ae73
Fix oops, add simple UT 2022-11-11 16:44:59 +01:00
Till c648c671a3
Fix issue with missing user NIDs (#2874)
This should fix #2696 and possibly other related issues regarding
missing user NIDs.
(https://github.com/matrix-org/dendrite/issues/2094?)
2022-11-11 10:52:43 +01:00
Neil Alexander b7443aa77f
Merge branch 'main' into neilalexander/purgeroom 2022-11-02 11:09:23 +00:00
Neil Alexander 6663728eb1
Fix SQLite roomserver_published migration 2022-11-01 16:08:13 +00:00
Till 2acc1d65fb
Optimize history visibility checks (#2848)
This optimizes history visibility checks by (mostly) avoiding database
hits.
Possibly solves https://github.com/matrix-org/dendrite/issues/2777

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-11-01 15:07:17 +00:00
Till Faelligen a785532463
Fix upgrade appservices 2022-10-27 16:01:51 +02:00
Till 444b4bbdb8
Add AS specific public room list endpoints (#2836)
Adds `PUT
/_matrix/client/v3/directory/list/appservice/{networkId}/{roomId}` and
`DELTE
/_matrix/client/v3/directory/list/appservice/{networkId}/{roomId}`
support, as well as the ability to filter `/publicRooms` on networkID
and including all networks.
2022-10-27 14:40:35 +02:00
Neil Alexander af46301970
Merge branch 'main' into neilalexander/purgeroom 2022-10-13 14:53:04 +01:00
Till 3c1474f68f
Fix /get_missing_events for rooms with joined/invited history_visibility (#2787)
Sytest was using a wrong `history_visibility` for `invited`
(https://github.com/matrix-org/sytest/pull/1303), so `invited` was
passing for the wrong reason (-> defaulted to `shared`, as `invite`
wasn't understood).
This change now handles missing events like Synapse, if a server isn't
allowed to see the event, it gets a redacted version of it, making the
`get_missing_events` tests pass.
2022-10-11 16:04:02 +02:00
Neil Alexander 5163cad239
Merge branch 'main' into neilalexander/purgeroom 2022-10-11 11:20:20 +01:00
Till 1ca3f3efb5
Fix issue with DMs shown as normal rooms (#2776)
Fixes #2121, test added in
https://github.com/matrix-org/complement/pull/494
2022-10-07 16:00:12 +02:00
Neil Alexander 8e231130e9
Revert "tDatabase transaction tweaks in roomserver"
This reverts commit 8d8f4689a0.
2022-10-07 14:05:06 +01:00
Neil Alexander 8d8f4689a0
tDatabase transaction tweaks in roomserver 2022-10-07 12:21:55 +01:00
Neil Alexander c85bc3434f
Optimise QuerySharedUsers so that we can only work on local users (#2766)
Otherwise the sync API key change consumer wastes a lot of time trying
to wake up the notifiers for non-local users.
2022-10-05 12:47:53 +01:00
Neil Alexander abd6a6425a
Merge branch 'main' into neilalexander/purgeroom 2022-09-27 14:14:19 +01:00
Neil Alexander f022fc1397
Remove origin field from PDUs (#2737)
This nukes the `origin` field from PDUs as per
matrix-org/matrix-spec#998, matrix-org/gomatrixserverlib#341.
2022-09-26 17:35:35 +01:00
Neil Alexander 1ca58ec9d1
Merge branch 'main' into neilalexander/purgeroom 2022-09-13 11:15:14 +01:00
Till 100fa9b235
Check unique constraint errors when manually inserting migrations (#2712)
This should avoid unnecessary logging on startup if the migration (were
we need `InsertMigration`) was already executed.
This now checks for "unique constraint errors" for SQLite and Postgres
and fails the startup process if the migration couldn't be manually
inserted for some other reason.
2022-09-13 08:07:43 +02:00
Neil Alexander 47be521dd9
Merge branch 'main' into neilalexander/purgeroom 2022-09-11 20:42:32 +01:00
Neil Alexander c0e17bbe1b
Fix transactions around assigning NIDs 2022-09-09 13:30:09 +01:00
Till 8196b29657
Change detection of already executed migrations (#2665)
This changes the detection of already executed migrations for the
roomserver state block and keychange refactor. It now uses schema tables
provided by the database engine to check if the column was already
removed. We now also store the migration in the migrations table.

This should stop e.g. Postgres from logging errors like `ERROR: column
"event_nid" does not exist at character 8`.
2022-09-09 13:14:52 +01:00
Neil Alexander b7258dcc00
Let the roomserver work first 2022-08-25 12:11:00 +01:00
Neil Alexander c1d6e18152
Lock the room during a purge operation to prevent other input events from populating mid-operation 2022-08-25 12:09:34 +01:00
Neil Alexander 965f532bb0
Merge branch 'main' into neilalexander/purgeroom 2022-08-24 14:47:11 +01:00
Neil Alexander 522bd2999f
Allow un-rejecting events on reprocessing 2022-08-24 14:03:06 +01:00
Neil Alexander 14fea600bb
Detect types.MissingStateError in CheckServerAllowedToSeeEvent (#2667)
This will hopefully stop some 500 errors on `/event` where there is no state-before known.
2022-08-23 13:57:11 +01:00
Neil Alexander e31469682d
Send the output event first 2022-08-22 14:14:29 +01:00
Neil Alexander 4e4fc400a2
Consolidate purge statements as they need to be prepared after other tables are created 2022-08-22 13:48:09 +01:00
Neil Alexander cd81635cf4
Let us retry 2022-08-22 12:52:22 +01:00
Neil Alexander 4c4de0dfc6
Purge the room entry too 2022-08-22 12:05:38 +01:00
Neil Alexander b56011514b
Fix a query 2022-08-22 12:01:23 +01:00
Neil Alexander 83c7984dba
Fix table name 2022-08-22 11:58:17 +01:00
Neil Alexander 856fe9d62b
Try adding an endpoint for it 2022-08-22 11:55:22 +01:00
Neil Alexander c1dcd5b218
SQLite stubs 2022-08-22 11:49:46 +01:00
Neil Alexander d11c243599
More PostgreSQL purging 2022-08-22 11:46:44 +01:00
Neil Alexander 537f300edf
Some PostgreSQL purging 2022-08-22 11:26:18 +01:00
Neil Alexander 6b48ce0d75
State handling tweaks (#2652)
This tweaks how rejected events are handled in room state and also to not apply checks we can't complete to outliers.
2022-08-18 17:06:13 +01:00
Neil Alexander 59bc0a6f4e
Reprocess rejected input events (#2647)
* Reprocess outliers that were previously rejected

* Might as well do all events this way

* More useful errors

* Fix queries

* Tweak condition

* Don't wrap errors

* Report more useful error

* Flatten error on `r.Queryer.QueryStateAfterEvents`

* Some more debug logging

* Flatten error in `QueryRestrictedJoinAllowed`

* Revert "Flatten error in `QueryRestrictedJoinAllowed`"

This reverts commit 1238b4184c.

* Tweak `QueryStateAfterEvents`

* Handle MissingStateError too

* Scope to room

* Clean up

* Fix the error

* Only apply rejection check to outliers
2022-08-18 10:37:47 +01:00
Till 03ddd98f5e
Fix issues with migrations not getting executed (#2628)
* Fix issues with migrations not getting executed

* Check actual postgres error

* Return error if it's not "column does not exist"
2022-08-08 10:18:57 +02:00
Till 1b7f84250a
Fix linter issues (#2624)
* Try that again

* All hail the mighty linter?

* And once again

* goimport all the things
2022-08-05 11:12:41 +02:00