Commit graph

450 commits

Author SHA1 Message Date
Till Faelligen 2e5ef9c711
Make use of sqlutil.StatementList, dedupe NIDs, timeout test 2022-12-22 15:43:29 +01:00
Till Faelligen e7356f7e96
Minor cleanup 2022-12-22 14:54:13 +01:00
Till Faelligen e779b5b3a8
Initial purge room support for SQLite 2022-12-22 14:48:58 +01:00
Till Faelligen 2b7d1023ba
Merge branch 'main' of github.com:matrix-org/dendrite into neilalexander/purgeroom 2022-12-22 13:13:42 +01:00
Till 5eed31fea3
Handle guest access [1/2?] (#2872)
Needs https://github.com/matrix-org/sytest/pull/1315, as otherwise the
membership events aren't persisted yet when hitting `/state` after
kicking guest users.

Makes the following tests pass:
```
Guest users denied access over federation if guest access prohibited
Guest users are kicked from guest_access rooms on revocation of guest_access
Guest users are kicked from guest_access rooms on revocation of guest_access over federation
```

Todo (in a follow up PR):
- Restrict access to CS API Endpoints as per
https://spec.matrix.org/v1.4/client-server-api/#client-behaviour-14

Co-authored-by: kegsay <kegan@matrix.org>
2022-12-22 13:05:59 +01:00
Till Faelligen 1e85ec6441
Wait for streams to process the event 2022-12-22 07:00:09 +01:00
Till Faelligen 23d1002142
Fix FederationAPI not processing purges; start SyncAPI/FedAPI consumer 2022-12-21 16:49:10 +01:00
Till Faelligen c10f563510
Minor fixes/changes; Initial test 2022-12-21 16:11:03 +01:00
Till Faelligen e9336df152
Merge branch 'main' of github.com:matrix-org/dendrite into neilalexander/purgeroom 2022-12-21 14:20:25 +01:00
Till 7d2344049d
Cleanup stale device lists for users we don't share a room with anymore (#2857)
The stale device lists table might contain entries for users we don't
share a room with anymore. This now asks the roomserver about left users
and removes those entries from the table.

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-12-12 08:20:59 +01:00
Till e245a26f6b
Enable/Disable internal metrics (#2899)
Basically enables us to use `test.WithAllDatabases` when testing
internal HTTP APIs, as this would otherwise result in Prometheus
complaining about already registered metric names.
2022-12-05 13:53:36 +01:00
Neil Alexander 321e6a1e23
Merge branch 'main' into neilalexander/purgeroom 2022-12-01 09:50:27 +00:00
Neil Alexander 9b8bb55430
Don't get blacklisted hosts when querying joined servers (#2880)
Otherwise we just waste time/CPU.
2022-11-15 17:21:16 +00:00
Neil Alexander 6650712a1c
Federation fixes for virtual hosting 2022-11-15 15:05:23 +00:00
Till 2a77a910eb
Handle remote room upgrades (#2866)
Makes the following tests pass
```
/upgrade moves remote aliases to the new room
Local and remote users' homeservers remove a room from their public directory on upgrade
```
2022-11-14 12:07:13 +00:00
Till 1e79b0557e
Use a writer to assign state key NIDs (#2877) 2022-11-14 12:06:27 +00:00
Till Faelligen e177e0ae73
Fix oops, add simple UT 2022-11-11 16:44:59 +01:00
Till c648c671a3
Fix issue with missing user NIDs (#2874)
This should fix #2696 and possibly other related issues regarding
missing user NIDs.
(https://github.com/matrix-org/dendrite/issues/2094?)
2022-11-11 10:52:43 +01:00
Till 0193549201
Send presence to newly added servers (#2869)
This should make `New federated private chats get full presence
information (SYN-115)` happy.
2022-11-11 10:35:17 +01:00
Neil Alexander 16c2a95900
Improve logging for processEventWithMissingState 2022-11-02 11:30:49 +00:00
Neil Alexander cb7100c556
Fix oops 2022-11-02 11:10:13 +00:00
Neil Alexander b7443aa77f
Merge branch 'main' into neilalexander/purgeroom 2022-11-02 11:09:23 +00:00
Neil Alexander 6663728eb1
Fix SQLite roomserver_published migration 2022-11-01 16:08:13 +00:00
Till 2acc1d65fb
Optimize history visibility checks (#2848)
This optimizes history visibility checks by (mostly) avoiding database
hits.
Possibly solves https://github.com/matrix-org/dendrite/issues/2777

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-11-01 15:07:17 +00:00
Neil Alexander 7bd6631935
Move code for calculating auth difference into GMSL 2022-11-01 10:12:11 +00:00
Neil Alexander f10c6f26e5
Add /_dendrite/admin/downloadState/{serverName}/{roomID} admin endpoint 2022-10-31 09:13:28 +00:00
Till Faelligen a785532463
Fix upgrade appservices 2022-10-27 16:01:51 +02:00
Till 444b4bbdb8
Add AS specific public room list endpoints (#2836)
Adds `PUT
/_matrix/client/v3/directory/list/appservice/{networkId}/{roomId}` and
`DELTE
/_matrix/client/v3/directory/list/appservice/{networkId}/{roomId}`
support, as well as the ability to filter `/publicRooms` on networkID
and including all networks.
2022-10-27 14:40:35 +02:00
Neil Alexander f6dea712d2
Initial support for multiple server names (#2829)
This PR is the first step towards virtual hosting by laying the
groundwork for multiple server names being configured.
2022-10-26 12:59:19 +01:00
Neboer 2a4c7f45b3
Add support for config "auto_join_rooms" (#2823)
Add support for config "auto_join_rooms". Now new accounts can join the
rooms in config file automatically.

### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have justified why this PR doesn't need tests.
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)

Signed-off-by: `Rubin Poster <rubinposter@gmail.com>`
2022-10-26 11:04:53 +02:00
Neil Alexander 2ca3774841
Merge branch 'main' into neilalexander/purgeroom 2022-10-21 10:57:33 +01:00
Neil Alexander cd8f7e1251
Set inactivity threshold on durable consumers in the roomserver input API (#2795)
This prevents us from holding onto durable consumers indefinitely for
rooms that have long since turned inactive, since they do have a bit of
a processing overhead in the NATS Server. If we clear up a consumer and
then a room becomes active again, the consumer gets recreated as needed.

The threshold is set to 24 hours for now, we can tweak it later if needs
be.
2022-10-14 15:14:29 +01:00
Till 088ad1dd21
Fix outliers whose auth_events are in a different room are correctly rejected (#2791)
Fixes `outliers whose auth_events are in a different room are correctly
rejected`, by validating that auth events are all from the same room and
not using rejected events for event auth.
2022-10-14 09:14:54 +02:00
Neil Alexander af46301970
Merge branch 'main' into neilalexander/purgeroom 2022-10-13 14:53:04 +01:00
Till 3c1474f68f
Fix /get_missing_events for rooms with joined/invited history_visibility (#2787)
Sytest was using a wrong `history_visibility` for `invited`
(https://github.com/matrix-org/sytest/pull/1303), so `invited` was
passing for the wrong reason (-> defaulted to `shared`, as `invite`
wasn't understood).
This change now handles missing events like Synapse, if a server isn't
allowed to see the event, it gets a redacted version of it, making the
`get_missing_events` tests pass.
2022-10-11 16:04:02 +02:00
Neil Alexander 5163cad239
Merge branch 'main' into neilalexander/purgeroom 2022-10-11 11:20:20 +01:00
Till 1ca3f3efb5
Fix issue with DMs shown as normal rooms (#2776)
Fixes #2121, test added in
https://github.com/matrix-org/complement/pull/494
2022-10-07 16:00:12 +02:00
Neil Alexander 8e231130e9
Revert "tDatabase transaction tweaks in roomserver"
This reverts commit 8d8f4689a0.
2022-10-07 14:05:06 +01:00
Neil Alexander 8d8f4689a0
tDatabase transaction tweaks in roomserver 2022-10-07 12:21:55 +01:00
Neil Alexander 3f82bceb70
Don't try to talk to ourselves when finding missing events 2022-10-06 10:51:06 +01:00
Neil Alexander c85bc3434f
Optimise QuerySharedUsers so that we can only work on local users (#2766)
Otherwise the sync API key change consumer wastes a lot of time trying
to wake up the notifiers for non-local users.
2022-10-05 12:47:53 +01:00
Till 8c0c3441d8
Add RoomEventType nats.Header to avoid unneeded unmarshalling (#2765) 2022-10-05 12:12:42 +02:00
Neil Alexander ae10aac456
Don't perform a federated join after invite if we are already joined to the room (#2762)
If we are already joined to the room then it shouldn't matter if you
were invited or not, so this looks like a bug.
2022-10-04 15:40:04 +01:00
Neil Alexander abd6a6425a
Merge branch 'main' into neilalexander/purgeroom 2022-09-27 14:14:19 +01:00
Neil Alexander f022fc1397
Remove origin field from PDUs (#2737)
This nukes the `origin` field from PDUs as per
matrix-org/matrix-spec#998, matrix-org/gomatrixserverlib#341.
2022-09-26 17:35:35 +01:00
Neil Alexander 40f00abd19
Merge branch 'main' into neilalexander/purgeroom 2022-09-20 10:09:40 +01:00
Neil Alexander fc1d8e479b
Ensure that all state event IDs are included in the added section when rewriting state (#2725)
This should hopefully fix an entire class of problems where components
downstream from the roomserver (i.e. the sync API) could just lose a
whole bunch of state after a rewrite operation like a federated join.

The root of the bug is that we set `RewritesState` in the output event
which instructs downstream components to purge their copy of any room
state, but then didn't send the entire state snapshot in
`adds_state_event_ids` so the downstream state ends up being incomplete
as a result.
2022-09-16 10:35:32 +01:00
Neil Alexander 7f89fed1e4
Revert 482914aef4 2022-09-14 09:55:50 +01:00
Neil Alexander 482914aef4
Use AckNone on the ephemeral room input consumer 2022-09-13 15:25:02 +01:00
Neil Alexander b05e028f7d
Tweak LoadMembershipAtEvent behaviour when state not known (#2716)
Previously `LoadMembershipAtEvent` would fail if the state before one of
the events was not known, i.e. because it was an outlier. This modifies
it so that it gracefully handles not knowing the state and returns no
memberships instead, so that history visibility doesn't freak out and
kill `/sync` requests dead.
2022-09-13 12:52:09 +01:00