Commit graph

256 commits

Author SHA1 Message Date
Till b732eede27
Fix spaces over federation (#3347)
Fixes #2504

 A few issues with the previous iteration:
- We never returned `inaccessible_children`, which (if I read the code
correctly), made Synapse raise an error and thus not returning the
requested rooms
- For restricted rooms, we didn't return the list of allowed rooms
2024-03-28 20:40:45 +01:00
Till d58daf9665
Update sentry reporting (#3305)
This hopefully reduces the garbage we currently produce.
(Using [GlitchTip](https://glitchtip.com/) on my personal instance, this
seems to look better)
2024-01-24 19:24:04 +01:00
Till 8e4dc6b4ae
Optimize PrevEventIDs when getting thousands of backwards extremeties (#3308)
Changes how many `PrevEventIDs` we send to other servers when
backfilling, capped to 100 events.

Unsure about how representative this benchmark is..
```
goos: linux
goarch: amd64
pkg: github.com/matrix-org/dendrite/roomserver/api
cpu: Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz
                            │    old.txt     │               new.txt               │
                            │     sec/op     │   sec/op     vs base                │
PrevEventIDs/Original1-8         264.9n ± 5%   237.4n ± 7%  -10.36% (p=0.000 n=10)
PrevEventIDs/Original10-8        3.101µ ± 4%   1.590µ ± 2%  -48.72% (p=0.000 n=10)
PrevEventIDs/Original100-8       44.32µ ± 2%   12.80µ ± 4%  -71.11% (p=0.000 n=10)
PrevEventIDs/Original500-8     263.835µ ± 4%   7.907µ ± 4%  -97.00% (p=0.000 n=10)
PrevEventIDs/Original1000-8    578.798µ ± 2%   7.620µ ± 2%  -98.68% (p=0.000 n=10)
PrevEventIDs/Original2000-8   1272.039µ ± 2%   8.241µ ± 9%  -99.35% (p=0.000 n=10)
geomean                          43.81µ        3.659µ       -91.65%

                            │    old.txt     │               new.txt                │
                            │      B/op      │     B/op      vs base                │
PrevEventIDs/Original1-8          72.00 ± 0%     48.00 ± 0%  -33.33% (p=0.000 n=10)
PrevEventIDs/Original10-8        1512.0 ± 0%     500.0 ± 0%  -66.93% (p=0.000 n=10)
PrevEventIDs/Original100-8     11.977Ki ± 0%   7.023Ki ± 0%  -41.36% (p=0.000 n=10)
PrevEventIDs/Original500-8     67.227Ki ± 0%   7.023Ki ± 0%  -89.55% (p=0.000 n=10)
PrevEventIDs/Original1000-8   163.227Ki ± 0%   7.023Ki ± 0%  -95.70% (p=0.000 n=10)
PrevEventIDs/Original2000-8   347.227Ki ± 0%   7.023Ki ± 0%  -97.98% (p=0.000 n=10)
geomean                         12.96Ki        1.954Ki       -84.92%

                            │   old.txt   │              new.txt               │
                            │  allocs/op  │ allocs/op   vs base                │
PrevEventIDs/Original1-8       2.000 ± 0%   1.000 ± 0%  -50.00% (p=0.000 n=10)
PrevEventIDs/Original10-8      6.000 ± 0%   2.000 ± 0%  -66.67% (p=0.000 n=10)
PrevEventIDs/Original100-8     9.000 ± 0%   3.000 ± 0%  -66.67% (p=0.000 n=10)
PrevEventIDs/Original500-8    12.000 ± 0%   3.000 ± 0%  -75.00% (p=0.000 n=10)
PrevEventIDs/Original1000-8   14.000 ± 0%   3.000 ± 0%  -78.57% (p=0.000 n=10)
PrevEventIDs/Original2000-8   16.000 ± 0%   3.000 ± 0%  -81.25% (p=0.000 n=10)
geomean                        8.137        2.335       -71.31%
```
2024-01-20 22:26:57 +01:00
Till 13c5173273
Fix notary keys requests for all keys (#3296)
This should be more spec compliant:
> If no key IDs are given to be queried, the notary server should query
for all keys.
2024-01-08 19:14:29 +01:00
Till 7863a405a5
Use IsBlacklistedOrBackingOff to determine if we should try to fetch devices (#3254)
Use `IsBlacklistedOrBackingOff` from the federation API to check if we
should fetch devices.

To reduce back pressure, we now only queue retrying servers if there's
space in the channel.
2023-11-09 08:43:27 +01:00
devonh 8245b24100
Update gmsl to use new validated RoomID on PDUs (#3200)
GMSL returns a `spec.RoomID` when calling `PDU.RoomID()`
2023-09-15 14:39:06 +00:00
Sam Wedgwood c7193e24d0
Use *spec.SenderID for QuerySenderIDForUser (#3164)
There are cases where a dendrite instance is unaware of a pseudo ID for
a user, the user is not a member of that room. To represent this case,
we currently use the 'zero' value, which is often not checked and so
causes errors later down the line. To make this case more explict, and
to be consistent with `QueryUserIDForSender`, this PR changes this to
use a pointer (and `nil` to mean no sender ID).

Signed-off-by: `Sam Wedgwood <sam@wedgwood.dev>`
2023-08-02 11:12:14 +01:00
Sam Wedgwood 9582827493
de-MSC-ifying space summaries (MSC2946) (#3134)
- This PR moves and refactors the
[code](https://github.com/matrix-org/dendrite/blob/main/setup/mscs/msc2946/msc2946.go)
for
[MSC2946](https://github.com/matrix-org/matrix-spec-proposals/pull/2946)
('Space Summaries') to integrate it into the rest of the codebase.
- Means space summaries are no longer hidden behind an MSC flag
- Solves #3096

Signed-off-by: Sam Wedgwood <sam@wedgwood.dev>
2023-07-20 15:06:05 +01:00
Till Faelligen 33ff309572
Don't HTTP500 if a profile does't exist 2023-07-14 14:24:31 +02:00
devonh d507c5fc95
Add pseudoID compatibility to Invites (#3126) 2023-07-06 15:15:24 +00:00
Till 23cd7877a1
Add MXIDMapping for pseudoID rooms (#3112)
Add `MXIDMapping` on membership events when
creating/joining rooms.
2023-06-28 20:29:49 +02:00
devonh e4665979bf
Merge SenderID & Per Room User Key work (#3109) 2023-06-14 14:23:46 +00:00
devonh 77d9e4e93d
Cleanup remaining statekey usage for senderIDs (#3106) 2023-06-12 11:19:25 +00:00
devonh 8ea1a11105
Use SenderID Type (#3105) 2023-06-07 17:14:35 +00:00
devonh 7a1fd7f512
PDU Sender split (#3100)
Initial cut of splitting PDU Sender into SenderID & looking up UserID where required.
2023-06-06 20:55:18 +00:00
Till 725ff5567d
Make StrictValidityChecking a function (#3092)
Companion PR to https://github.com/matrix-org/gomatrixserverlib/pull/388
2023-06-06 15:16:55 +02:00
devonh ea6b368ad4
Move Invite logic to GMSL (#3086)
This is both the federation receiving & sending side logic (which were
previously entangeld in a single function)
2023-05-31 16:33:49 +00:00
devonh cbdc601f1b
Move CreateRoom logic to Roomserver (#3093)
Move create room logic over to roomserver.
2023-05-31 15:27:08 +00:00
Till 5d6221d191
Move MakeLeave to GMSL (#3085)
Basically the same API shape as for `/make_join`
https://github.com/matrix-org/gomatrixserverlib/pull/385
2023-05-23 19:37:04 +02:00
devonh 2eae8dc489
Move SendJoin logic to GMSL (#3084)
Moves the core matrix logic for handling the send_join endpoint over to
gmsl.
2023-05-19 16:27:01 +00:00
devonh 67d6876857
Move MakeJoin logic to GMSL (#3081) 2023-05-17 00:33:27 +00:00
devonh 0489d16f95
Move json errors over to gmsl (#3080) 2023-05-09 22:46:49 +00:00
kegsay 2b34f88fde
Use ProtoEvent where needed instead of EventBuilder (#3075)
They are fundamentally different concepts, so should be represented as
such. Proto events are exchanged in /make_xxx calls over federation, and
made as "fledgling" events in /createRoom and general event sending.
*Building* events is a reasonably complex VERSION SPECIFIC process which
needs amongst other things, auth event providers, prev events, signing
keys, etc.

Requires https://github.com/matrix-org/gomatrixserverlib/pull/379
2023-05-04 11:17:42 +01:00
kegsay 6284790f98
Use PDU in even more places (#3074)
- No longer rely on *Event returning from NewEventFrom... functions
 
Requires https://github.com/matrix-org/gomatrixserverlib/pull/377
2023-05-03 10:21:27 +01:00
kegsay f5b3144dc3
Use PDU not *Event in HeaderedEvent (#3073)
Requires https://github.com/matrix-org/gomatrixserverlib/pull/376

This has numerous upsides:
 - Less type casting to `*Event` is required.
- Making Dendrite work with `PDU` interfaces means we can swap out Event
impls more easily.
 - Tests which represent weird event shapes are easier to write.

Part of a series of refactors on GMSL.
2023-05-02 15:03:16 +01:00
Till 6b47cf0f6a
Remove PerformError (#3066)
This removes `PerformError`, which was needed when we still had
polylith.

This removes quite a bunch of
```go
if err != nil {
	return err
}
if err := res.Error; err != nil {
	return err.JSONResponse()
}
```

Hopefully can be read commit by commit. [skip ci]
2023-04-28 17:46:01 +02:00
kegsay 6171310307
Use PDU interface (#3070)
We only use it in a few places currently, enough to get things to
compile and run. We should be using it in much more places.

Similarly, in some places we cast []PDU back to []*Event, we need to not
do that. Likewise, in some places we cast PDU to *Event, we need to not
do that. For now though, hopefully this is a start.
2023-04-27 16:35:19 +01:00
kegsay b189edf4f4
Remove gmsl.HeaderedEvent (#3068)
Replaced with types.HeaderedEvent _for now_. In reality we want to move
them all to gmsl.Event and only use HeaderedEvent when we _need_ to
bundle the version/event ID with the event (seriailsation boundaries,
and even then only when we don't have the room version).

Requires https://github.com/matrix-org/gomatrixserverlib/pull/373
2023-04-27 12:54:20 +01:00
Till 2475cf4b61
Add some roomserver UTs (#3067)
Adds tests for `QueryRestrictedJoinAllowed`, `IsServerAllowed` and
`PerformRoomUpgrade`. Refactors the `QueryRoomVersionForRoom` method to
accept a string and return a `gmsl.RoomVersion` instead of req/resp
structs.
Adds some more caching for `GetStateEvent`

This should also fix #2912 by ignoring state events belonging to other
users.
2023-04-27 08:07:13 +02:00
devonh ed19efc5d7
Move fedclient interface over to gmsl (#3061)
Companion PR: https://github.com/matrix-org/gomatrixserverlib/pull/366
2023-04-24 16:23:25 +00:00
kegsay 4679098a64
Use IRoomVersion (#3064)
This is a step towards allowing arbitrary room version impls.
2023-04-24 11:50:37 +01:00
kegsay 1647213fac
Implement new RoomVersionImpl API (#3062)
As outlined in https://github.com/matrix-org/gomatrixserverlib/pull/368

The main change Dendrite side is that `RoomVersion` no longer has any
methods on it. Instead, you need to bounce via `gmsl.GetRoomVersion`.

It's very interesting to see where exactly Dendrite cares about this.
For some places it's creating events (fine) but others are way more
specific. Those areas will need to migrate to GMSL at some point.
2023-04-21 17:06:29 +01:00
kegsay 71eeccf34a
refactor: funnel event creation through room versions (#3060)
In preparation of interfacing up the room version value.
2023-04-20 19:07:31 +01:00
kegsay 72285b2659
refactor: update GMSL (#3058)
Sister PR to https://github.com/matrix-org/gomatrixserverlib/pull/364

Read this commit by commit to avoid going insane.
2023-04-19 15:50:33 +01:00
kegsay 0db43f13a6
refactor: use latest GMSL which splits fed client from matrix room logic (#3051)
Part of a series of refactors on GMSL.
2023-04-06 09:55:01 +01:00
Till c2db38d295
Add user profile tests, refactor user API methods (#3030)
This adds tests for `/profile`.
Also, as a first change in this regard, refactors the methods defined on
the `UserInternalAPI` to not use structs as the request/response
parameters.
2023-04-03 20:19:26 +02:00
Till 5e85a00cb3
Remove BaseDendrite (#3023)
Removes `BaseDendrite` to, hopefully, make testing and composing of
components easier in the future.
2023-03-22 09:21:32 +01:00
Till 5579121c6f
Preparations for removing BaseDendrite (#3016)
Preparations to actually remove/replace `BaseDendrite`.
Quite a few changes:
- SyncAPI accepts an `fulltext.Indexer` interface (fulltext is removed
from `BaseDendrite`)
- Caches are removed from `BaseDendrite`
- Introduces a `Router` struct (likely to change)
  - also fixes #2903
- Introduces a `sqlutil.ConnectionManager`, which should remove
`base.DatabaseConnection` later on
- probably more
2023-03-17 11:09:45 +00:00
Till 6c20f8f742
Refactor StoreEvent, add MaybeRedactEvent, create an EventDatabase (#2989)
This PR changes the following:
- `StoreEvent` now only stores an event (and possibly prev event),
instead of also doing redactions
- Adds a `MaybeRedactEvent` (pulled out from `StoreEvent`), which should
be called after storing events
- a few other things
2023-03-01 17:06:47 +01:00
Till 4594233f89
Merge keyserver & userapi (#2972)
As discussed yesterday, a first draft of merging the keyserver and the
userapi.
2023-02-20 14:58:03 +01:00
devonh 5b73592f5a
Initial Store & Forward Implementation (#2917)
This adds store & forward relays into dendrite for p2p.
A few things have changed:
- new relay api serves new http endpoints for s&f federation
- updated outbound federation queueing which will attempt to forward
using s&f if appropriate
- database entries to track s&f relays for other nodes
2023-01-23 17:55:12 +00:00
Till ce2bfc3f2e
Make tests more reliable (#2948)
When using `testrig.CreateBase` and then using that base for other
`NewInternalAPI` calls, we never actually shutdown the components.
`testrig.CreateBase` returns a `close` function, which only removes the
database, so still running components have issues connecting to the
database, since we ripped it out underneath it - which can result in
"Disk I/O" or "pq deadlock detected" issues.
2023-01-20 12:45:56 +01:00
Neil Alexander 8299da5905
Fix registration for virtual hosting 2022-11-18 13:24:02 +00:00
Neil Alexander 16325203af
Try that again 2022-11-17 09:32:19 +00:00
Neil Alexander 607819f425
Fix /key/v2/server, add HTTP Host matching 2022-11-17 09:26:56 +00:00
Neil Alexander d558da1c87
Virtual host server name workaround 2022-11-16 09:34:09 +00:00
Neil Alexander deddf686b9
Tweak /key/v2/server 2022-11-16 09:16:07 +00:00
Neil Alexander 6650712a1c
Federation fixes for virtual hosting 2022-11-15 15:05:23 +00:00
Neil Alexander 529df30b56
Virtual hosting schema and logic changes (#2876)
Note that virtual users cannot federate correctly yet.
2022-11-11 16:41:37 +00:00
Till 444b4bbdb8
Add AS specific public room list endpoints (#2836)
Adds `PUT
/_matrix/client/v3/directory/list/appservice/{networkId}/{roomId}` and
`DELTE
/_matrix/client/v3/directory/list/appservice/{networkId}/{roomId}`
support, as well as the ability to filter `/publicRooms` on networkID
and including all networks.
2022-10-27 14:40:35 +02:00