Commit graph

206 commits

Author SHA1 Message Date
Till Faelligen 89d64c6dca
Add support for getting authenticated federation media requests 2024-06-18 15:20:10 +02:00
Till Faelligen 29ee5401ee
Add support for authenticated federation media requests 2024-06-18 12:36:35 +02:00
Till Faelligen 40eb7cbeef
Use httputil.Routers for the media API 2024-06-17 10:25:41 +02:00
0x1a8510f2 46902e5766
Take advantage of changes in recent Go versions (#3361)
Given that #2714 wasn't merged but we are now at a minimum supported Go
version of 1.20 (soon to be 1.21), I wanted to carry over some of the
changes. Namely:
- Fix the log typo
- Simplify build constraints for unix
- Use stdlib atomic package

### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `0x1a8510f2 <admin@0x1a8510f2.space>`

---------

Co-authored-by: devonh <devon.dmytro@gmail.com>
2024-05-01 00:38:36 +00:00
Till 81f73c9f8d
Reuse existing NATS connection (#3345)
If using external NATS, we opened unnecessary connections. This now
re-uses existing connections.

[skip ci]
2024-03-22 22:33:23 +01:00
Till 865fff5f03
Make usage of relays optional, avoid DB roundtrips (#3337)
This should avoid 2 additional DB roundtrips if we don't want to use
relays.

So instead of possibly doing roughly 20k trips to the DB, we are now
"only" doing ~6600.

---------

Co-authored-by: devonh <devon.dmytro@gmail.com>
2024-02-28 20:59:34 +01:00
Tulir Asokan 0f6b81f456
Modernize appservice paths and authentication (#3316)
This brings Dendrite's appservice spec support up to v1.4, from the
previous level of pre-release-spec support only (even r0.1.0 wasn't
supported for pushing transactions 🙃). There are config options to
revert to the old behavior, but the default is v1.4+ only. [Synapse also
does
that](https://element-hq.github.io/synapse/latest/usage/configuration/config_documentation.html#use_appservice_legacy_authorization)

mautrix bridges will drop support for legacy paths and authentication
soon (and possibly also require matrix v1.4 to be advertised, but I
might add some workaround to not require that for dendrite)

Signed-off-by: Tulir Asokan <tulir@maunium.net>
2024-02-03 18:56:13 +01:00
Till bebf701dce
Add login fallback (#3302)
Part of https://github.com/matrix-org/dendrite/issues/3216

The files are basically copied from Synapse, with minor changes to the
called endpoints. We never seem to have had the
`/_matrix/static/client/login/` endpoint, this adds it.
2024-01-17 17:08:57 +01:00
Till 1555b3542d
Introduce a new stream for the appservice consumer (#3277)
This introduces a new stream the syncAPI produces to once it processed a
`OutputRoomEvent` and the appservices consumes.
This is to work around a race condition where appservices receive an
event before the syncAPI has handled it, this can result in e.g. calls
to `/joined_members` returning a wrong membership list.
2023-12-12 12:13:55 +01:00
Joseph Alvarenga Beech fd11e65a9d
added a warning log , for well_known_server_name,well_known_server_name when they dont have prefix (#3205)
closing this https://github.com/matrix-org/dendrite/issues/3180

added a warning log when either well_known_server_name,
well_known_server_name: dont have a prefix in them

josephalvarengabeech@pm.me

---------

Co-authored-by: Till Faelligen <2353100+S7evinK@users.noreply.github.com>
2023-11-25 22:19:22 +01:00
Till 699f5ca8c1
More rows.Close() and rows.Err() (#3262)
Looks like we missed some `rows.Close()`

Even though `rows.Err()` is mostly not necessary, we should be more
consistent in the DB layer.

[skip ci]
2023-11-09 08:42:33 +01:00
Till da7bca0224
Some tweaks for the device list updater (#3251)
This makes the following changes:
- Adds two new metrics observing the usage of the `DeviceListUpdater`
workers
- Makes the number of workers configurable
- Adds a 30s timeout for DB requests when receiving a device list update
over federation
2023-10-31 16:39:45 +01:00
CicadaCinema 89482ad790
clean up dead links, fix typo (#3130)
I fixed any dead links beginning https://matrix.org/speculator and some
issues I found along the way.


https://web.archive.org/web/20190329152312/https://matrix.org/speculator/spec/HEAD/client_server/unstable.html#user-interactive-authentication-api
is now found at

https://spec.matrix.org/v1.7/client-server-api/#user-interactive-authentication-api


https://web.archive.org/web/20170620093435/https://matrix.org/speculator/spec/HEAD/client_server/unstable.html#post-matrix-client-unstable-register
is now found at

https://spec.matrix.org/v1.7/client-server-api/#post_matrixclientv3register


2a8d64fef7/specification/intro.rst?plain=1#L443
is now found at
https://spec.matrix.org/v1.7/appendices/#user-identifiers
2023-10-25 10:24:06 +02:00
devonh 8245b24100
Update gmsl to use new validated RoomID on PDUs (#3200)
GMSL returns a `spec.RoomID` when calling `PDU.RoomID()`
2023-09-15 14:39:06 +00:00
devonh 1c4ec67bb6
Add configuration option for sliding sync when hosting /.well-known/matrix/client (#3189)
Adds the `org.matrix.msc3575.proxy` field (used for configuring sliding
sync) to /.well-known/matrix/client when Dendrite is serving that
endpoint and `well_known_sliding_sync_proxy` has been configured.

ie. Config values of:
``` yaml
global:
    well_known_client_name: https://example.com
    well_known_sliding_sync_proxy: https://syncv3.example.com
```
results in a /.well-known/matrix/client of:
``` json
{
    "m.homeserver": {
        "base_url": "https://example.com"
    },
    "org.matrix.msc3575.proxy": {
        "url": "https://syncv3.example.com"
    }
}
```

If `well_known_sliding_sync_proxy` is not provided, the json provided by
/.well-known/matrix/client does not include the proxy field.
ie.
``` json
{
    "m.homeserver": {
        "base_url": "https://example.com"
    }
}
```
2023-08-24 21:08:40 +00:00
Sam Wedgwood 35804f8493
Add config key for default room version (#3171)
This PR adds a config key `room_server.default_config_key` to set the
default room version for the room server.

Signed-off-by: `Sam Wedgwood <sam@wedgwood.dev>`
2023-08-08 14:20:05 +01:00
Sam Wedgwood c7193e24d0
Use *spec.SenderID for QuerySenderIDForUser (#3164)
There are cases where a dendrite instance is unaware of a pseudo ID for
a user, the user is not a member of that room. To represent this case,
we currently use the 'zero' value, which is often not checked and so
causes errors later down the line. To make this case more explict, and
to be consistent with `QueryUserIDForSender`, this PR changes this to
use a pointer (and `nil` to mean no sender ID).

Signed-off-by: `Sam Wedgwood <sam@wedgwood.dev>`
2023-08-02 11:12:14 +01:00
Devon Hudson a48c7d33a5
Don't quit if unknown msc in config, log it and keep going 2023-07-21 13:08:28 -06:00
Sam Wedgwood 9582827493
de-MSC-ifying space summaries (MSC2946) (#3134)
- This PR moves and refactors the
[code](https://github.com/matrix-org/dendrite/blob/main/setup/mscs/msc2946/msc2946.go)
for
[MSC2946](https://github.com/matrix-org/matrix-spec-proposals/pull/2946)
('Space Summaries') to integrate it into the rest of the codebase.
- Means space summaries are no longer hidden behind an MSC flag
- Solves #3096

Signed-off-by: Sam Wedgwood <sam@wedgwood.dev>
2023-07-20 15:06:05 +01:00
Till 297479ea49
Use pointer when passing the connection manager around (#3152)
As otherwise existing connections aren't reused.
2023-07-19 13:37:04 +02:00
Neil e93bdd56fd
Set max age for roomserver input stream to avoid excessive interior deletes (#3145)
If old messages build up in the input stream and do not get processed
successfully, this can create a significant drift between the stream
first sequence and the consumer ack floors, which results in a slow and
expensive start-up when interest-based retention is in use.

If a message is sat in the stream for 24 hours, it's probably not going
to get processed successfully, so let NATS drop them instead. Dendrite
can reconcile by fetching missing events later if it needs to.

---------

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2023-07-07 19:59:34 +02:00
Till Faelligen e1d76de6c6
Increase NATS server startup timeout 2023-07-06 10:04:46 +02:00
santhoshivan23 45082d4dce
feat: admin APIs for token authenticated registration (#3101)
### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `Santhoshivan Amudhan santhoshivan23@gmail.com`
2023-06-22 16:37:21 +00:00
devonh e4665979bf
Merge SenderID & Per Room User Key work (#3109) 2023-06-14 14:23:46 +00:00
devonh 77d9e4e93d
Cleanup remaining statekey usage for senderIDs (#3106) 2023-06-12 11:19:25 +00:00
devonh 8ea1a11105
Use SenderID Type (#3105) 2023-06-07 17:14:35 +00:00
devonh 7a1fd7f512
PDU Sender split (#3100)
Initial cut of splitting PDU Sender into SenderID & looking up UserID where required.
2023-06-06 20:55:18 +00:00
devonh 67d6876857
Move MakeJoin logic to GMSL (#3081) 2023-05-17 00:33:27 +00:00
devonh 0489d16f95
Move json errors over to gmsl (#3080) 2023-05-09 22:46:49 +00:00
kegsay 2b34f88fde
Use ProtoEvent where needed instead of EventBuilder (#3075)
They are fundamentally different concepts, so should be represented as
such. Proto events are exchanged in /make_xxx calls over federation, and
made as "fledgling" events in /createRoom and general event sending.
*Building* events is a reasonably complex VERSION SPECIFIC process which
needs amongst other things, auth event providers, prev events, signing
keys, etc.

Requires https://github.com/matrix-org/gomatrixserverlib/pull/379
2023-05-04 11:17:42 +01:00
Devon Hudson d5c11a3c86
Fix flaky test in process context 2023-05-03 18:21:33 -06:00
kegsay 6284790f98
Use PDU in even more places (#3074)
- No longer rely on *Event returning from NewEventFrom... functions
 
Requires https://github.com/matrix-org/gomatrixserverlib/pull/377
2023-05-03 10:21:27 +01:00
kegsay f5b3144dc3
Use PDU not *Event in HeaderedEvent (#3073)
Requires https://github.com/matrix-org/gomatrixserverlib/pull/376

This has numerous upsides:
 - Less type casting to `*Event` is required.
- Making Dendrite work with `PDU` interfaces means we can swap out Event
impls more easily.
 - Tests which represent weird event shapes are easier to write.

Part of a series of refactors on GMSL.
2023-05-02 15:03:16 +01:00
kegsay 1432743d1a
Use PDU in more places (#3072) 2023-04-28 16:00:22 +01:00
kegsay 6171310307
Use PDU interface (#3070)
We only use it in a few places currently, enough to get things to
compile and run. We should be using it in much more places.

Similarly, in some places we cast []PDU back to []*Event, we need to not
do that. Likewise, in some places we cast PDU to *Event, we need to not
do that. For now though, hopefully this is a start.
2023-04-27 16:35:19 +01:00
kegsay b189edf4f4
Remove gmsl.HeaderedEvent (#3068)
Replaced with types.HeaderedEvent _for now_. In reality we want to move
them all to gmsl.Event and only use HeaderedEvent when we _need_ to
bundle the version/event ID with the event (seriailsation boundaries,
and even then only when we don't have the room version).

Requires https://github.com/matrix-org/gomatrixserverlib/pull/373
2023-04-27 12:54:20 +01:00
Till 2475cf4b61
Add some roomserver UTs (#3067)
Adds tests for `QueryRestrictedJoinAllowed`, `IsServerAllowed` and
`PerformRoomUpgrade`. Refactors the `QueryRoomVersionForRoom` method to
accept a string and return a `gmsl.RoomVersion` instead of req/resp
structs.
Adds some more caching for `GetStateEvent`

This should also fix #2912 by ignoring state events belonging to other
users.
2023-04-27 08:07:13 +02:00
devonh ed19efc5d7
Move fedclient interface over to gmsl (#3061)
Companion PR: https://github.com/matrix-org/gomatrixserverlib/pull/366
2023-04-24 16:23:25 +00:00
kegsay 72285b2659
refactor: update GMSL (#3058)
Sister PR to https://github.com/matrix-org/gomatrixserverlib/pull/364

Read this commit by commit to avoid going insane.
2023-04-19 15:50:33 +01:00
kegsay ca63b414da
Update GMSL: use static Check functions (#3052)
Sister PR to https://github.com/matrix-org/gomatrixserverlib/pull/359 

A nice side effect is that we don't need to re-parse the events in some
cases.
2023-04-14 12:32:42 +01:00
kegsay 0db43f13a6
refactor: use latest GMSL which splits fed client from matrix room logic (#3051)
Part of a series of refactors on GMSL.
2023-04-06 09:55:01 +01:00
kegsay 3691423626
Move GMSL client types to Dendrite (#3045)
GMSL is intended for Federation only. Sister PR to
https://github.com/matrix-org/gomatrixserverlib/pull/357
2023-04-04 17:16:53 +00:00
Boris Rybalkin 985298cfc4
app service unix socket support (#3022)
This is the last part of unix socket support to talk to app servers, go
based app services already support unix sockets:

5a68173fe3
```
appservice:
  # The address that the homeserver can use to connect to this appservice.
  address: unix:///var/snap/matrix/current/whatsapp.socket

  # The hostname and port where this appservice should listen.
  hostname: /var/snap/matrix/current/whatsapp.socket
  port: 0
```

### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `Boris Rybalkin <ribalkin@gmail.com>`
2023-04-04 09:42:46 +02:00
Till 5e85a00cb3
Remove BaseDendrite (#3023)
Removes `BaseDendrite` to, hopefully, make testing and composing of
components easier in the future.
2023-03-22 09:21:32 +01:00
Till 5579121c6f
Preparations for removing BaseDendrite (#3016)
Preparations to actually remove/replace `BaseDendrite`.
Quite a few changes:
- SyncAPI accepts an `fulltext.Indexer` interface (fulltext is removed
from `BaseDendrite`)
- Caches are removed from `BaseDendrite`
- Introduces a `Router` struct (likely to change)
  - also fixes #2903
- Introduces a `sqlutil.ConnectionManager`, which should remove
`base.DatabaseConnection` later on
- probably more
2023-03-17 11:09:45 +00:00
Boris Rybalkin d88f71ab71
simplify unix socket permission format (#3014)
### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `Boris Rybalkin <ribalkin@gmail.com>`
2023-03-16 08:51:21 +01:00
Till 689b5ee72f
Change default stats reporting endpoint (#3007)
It's the same instance we report to, only using the subdomain.
2023-03-10 12:27:08 +01:00
Boris Rybalkin 6b1c9eafa9
unix socket support (#2974)
### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `Boris Rybalkin <ribalkin@gmail.com>`

I need this for Syncloud project (https://github.com/syncloud/platform)
where I run multiple apps behind an nginx on the same RPi like device so
unix socket is very convenient to not have port conflicts between apps.
Also someone opened this Issue:
https://github.com/matrix-org/dendrite/issues/2924

---------

Co-authored-by: kegsay <kegan@matrix.org>
Co-authored-by: Till <2353100+S7evinK@users.noreply.github.com>
2023-03-01 22:57:30 +01:00
Till 6c20f8f742
Refactor StoreEvent, add MaybeRedactEvent, create an EventDatabase (#2989)
This PR changes the following:
- `StoreEvent` now only stores an event (and possibly prev event),
instead of also doing redactions
- Adds a `MaybeRedactEvent` (pulled out from `StoreEvent`), which should
be called after storing events
- a few other things
2023-03-01 17:06:47 +01:00
David Schneider e6aa0955ff
Unify logging by using logrus for jetstream logs (#2976)
I guess tests for the logging is rather unusual so I omitted tests for
this change.

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `David Schneider <dsbrng25b@gmail.com>`

---------

Signed-off-by: David Schneider <dsbrng25b@gmail.com>
2023-02-24 08:56:53 +01:00