Commit graph

2540 commits

Author SHA1 Message Date
Neil Alexander ca8bc87380
Multi-stage Docker builds (#2850)
This builds on @S7evinK's work to make multi-stage Docker builds. Now
that we can build SQLite without Cgo this should be much simpler and
should make Docker builds in CI significantly faster.

Co-authored-by: Till Faelligen <tfaelligen@gmail.com>
Co-authored-by: Till Faelligen <davidf@element.io>
Co-authored-by: Till Faelligen <2353100+S7evinK@users.noreply.github.com>
2022-11-02 14:04:08 +00:00
0x1a8510f2 51ab0a8ccf
Fix moderncsqlite errors and rebase onto main (#2832)
This is #2819 but rebased on latest `main`. This PR is against main too
as opposed to the `moderncsqlite` branch.

The main change here is simply:

```go
// add query parameters to the dsn
if strings.Contains(dsn, "?") {
	dsn += "&"
} else {
	dsn += "?"
}

// wait some time before erroring if the db is locked
// https://gitlab.com/cznic/sqlite/-/issues/106#note_1058094993
dsn += "_pragma=busy_timeout%3d10000"
```

### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added tests for PR _or_ I have justified why this PR
doesn't need tests.
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed off privately.

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-11-02 13:20:10 +00:00
Neil Alexander 16c2a95900
Improve logging for processEventWithMissingState 2022-11-02 11:30:49 +00:00
Till 86b25a6337
Add message stats to reporting (#2748)
Since we're now listening on the `OutputRoomEvent` stream, we are able
to store messages stats.
2022-11-02 10:18:11 +00:00
Till b367cfeddf
Implement /thirdparty endpoints (#2831)
Implements the following endpoints 
```
GET /_matrix/client/v3/thirdparty/protocols
GET /_matrix/client/v3/thirdparty/protocols/{protocol}
GET /_matrix/client/v3/thirdparty/location
GET /_matrix/client/v3/thirdparty/location/{protocol}
GET /_matrix/client/v3/thirdparty/user
GET /_matrix/client/v3/thirdparty/user/{protocol}
```
2022-11-02 10:17:53 +00:00
Tak Wai Wong 75a508cc27
Fix issue where a member is forced to leave a room when the invite is marked deleted (#2839)
Proposed fix for issue:
https://github.com/matrix-org/dendrite/issues/2838

Suppose bob received invites to spaceA and spaceB.
When Bob joins spaceA, we add an OutputEvent event to retire the invite.
This sets the invite to "deleted" in the database. This makes sense.

The bug is in stream_invites.go. Triggered when bob received a new
invite for spaceB, and does a client sync.

In the block (line 76)
`for roomID := range retiredInvites

   if _, ok := req.Response.Rooms.Invite[roomID]; ok {
	continue
   }

   if _, ok := req.Response.Rooms.Join[roomID]; ok {
	continue
  }
...
` 

Bob is not in either maps even though he had just accepted the invite
for spaceA. Consequently, the spaceA invite is treated as a retired
invite, and a membership Leave event is generated. What bob sees is that
after accepting the invite to spaceB, he lose access to spaceA.


### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [ ] I have added tests for PR _or_ I have justified why this PR
doesn't need tests.
* [x ] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `Tak Wai Wong <tak@hntlabs.com>`

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-11-02 10:02:23 +00:00
Neil Alexander 3db9e98456
Don't limit "state" (#2849)
This is apparently some incorrect behaviour that we built as a result of
a spec bug (matrix-org/matrix-spec#1314) where we were applying a filter
to the `"state"` section of the `/sync` response incorrectly. The client
then has no way to know that the state was limited.

This PR removes the state limiting, which probably also helps #2842.
2022-11-02 09:34:19 +00:00
Neil Alexander 8a1904ffe5
Update pull request template 2022-11-02 09:33:00 +00:00
Neil Alexander 52478dac3c
Version 0.10.6 2022-11-01 17:14:30 +00:00
Neil Alexander 501977f6fe
Fix a panic in ToClientEvents etc. 2022-11-01 16:58:51 +00:00
ash lea 5aaa60227a
return required room_id field in /members (#2846)
### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [ ] I have added tests for PR _or_ I have justified why this PR
doesn't need tests.
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `ash lea <example@thisismyactual.email>`
2022-11-01 16:42:07 +00:00
Neil Alexander 42d7e3ee0d
Update dependencies 2022-11-01 16:15:55 +00:00
Neil Alexander 6663728eb1
Fix SQLite roomserver_published migration 2022-11-01 16:08:13 +00:00
Till 2acc1d65fb
Optimize history visibility checks (#2848)
This optimizes history visibility checks by (mostly) avoiding database
hits.
Possibly solves https://github.com/matrix-org/dendrite/issues/2777

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-11-01 15:07:17 +00:00
Neil Alexander 0b21cb78aa
Try to fix a panic in the sync API PDU stream 2022-11-01 14:45:15 +00:00
Neil Alexander 7bd6631935
Move code for calculating auth difference into GMSL 2022-11-01 10:12:11 +00:00
Till Faelligen 869bf4d0ac
Fix flakey stats tests 2022-11-01 08:39:16 +01:00
Neil Alexander 8c7b274e4e
Version 0.10.5 (#2845)
Changelog and version bump.
2022-10-31 16:44:41 +00:00
Neil Alexander 4c38bd76ce
Fix go.mod 2022-10-31 15:15:18 +00:00
Neil Alexander 7307701a24
Tweak "state" and "timeline" filtering (#2844)
This should stop state events disappearing down a gap where we'd try to
separate out the sections *before* applying history visibility instead
of after.

This may be a better approach than #2843 but I hope @tak-hntlabs will
shout if it isn't.
2022-10-31 15:14:08 +00:00
Neil Alexander 66a82e0fa4
Update to matrix-org/gomatrixserverlib@0885c35 2022-10-31 15:12:05 +00:00
Neil Alexander f10c6f26e5
Add /_dendrite/admin/downloadState/{serverName}/{roomID} admin endpoint 2022-10-31 09:13:28 +00:00
Till 69aff372f3
Limit recent events when going backwards (#2840)
If we're going backwards, we were selecting potentially thousands of
events, which in turn were fed to history visibility checks, resulting
in bad sync performance.
2022-10-28 13:40:51 +02:00
X. Ding 0782011f54
Add hcaptcha support besides Google ReCaptcha (#2834)
### Pull Request Checklist
This PR add support for hcaptcha.com as an alternative to Google
ReCaptcha. It also makes possible for user to customize ReCaptcha URL
when needed. (Such as use recaptcha.net instead of www.google.com)

This feature needs manual test cuz it involves 3rd party _captcha_.

Signed-off-by: `Simon Ding <dxl@plotbridge.com>`

Co-authored-by: dxl <dxl@plotbridge.com>
2022-10-28 11:25:01 +01:00
Till Faelligen f6035822e7
Simplify error checking and check the correct error 2022-10-28 08:17:40 +02:00
Neil Alexander a2706e6498
Refactor claimRemoteKeys 2022-10-27 15:34:26 +01:00
Till Faelligen a785532463
Fix upgrade appservices 2022-10-27 16:01:51 +02:00
Till 444b4bbdb8
Add AS specific public room list endpoints (#2836)
Adds `PUT
/_matrix/client/v3/directory/list/appservice/{networkId}/{roomId}` and
`DELTE
/_matrix/client/v3/directory/list/appservice/{networkId}/{roomId}`
support, as well as the ability to filter `/publicRooms` on networkID
and including all networks.
2022-10-27 14:40:35 +02:00
Till a169a9121a
Fix /members (#2837)
Fixes a bug introduced in #2827, where the SyncAPI might not have all
requested eventIDs, resulting in too few members returned.
2022-10-27 14:18:22 +02:00
Till fa96811e64
Add scheduled tasks to run tests with race detection (#2814)
Needs https://github.com/matrix-org/sytest/pull/1308 to be actually
useful.
Not sure if we need to run Sytest in all combinations with enabled race
detection.

Closes https://github.com/matrix-org/dendrite/issues/491
2022-10-27 12:12:50 +02:00
Neil Alexander 238b6ef2cd
Update Yggdrasil demo 2022-10-26 18:37:01 +01:00
devonh 97491a174b
Associate events in db before queueing them to send (#2833)
Fixes a race condition between sending federation events and having them
fully associated in the database.
2022-10-26 17:35:01 +01:00
devonh a74aea0714
Add network interface callback to pinecone build (#2825)
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-10-26 17:25:57 +01:00
Neil Alexander 5298dd1133
Update federation API consumers 2022-10-26 14:52:33 +01:00
Neil Alexander f6dea712d2
Initial support for multiple server names (#2829)
This PR is the first step towards virtual hosting by laying the
groundwork for multiple server names being configured.
2022-10-26 12:59:19 +01:00
Neboer 2a4c7f45b3
Add support for config "auto_join_rooms" (#2823)
Add support for config "auto_join_rooms". Now new accounts can join the
rooms in config file automatically.

### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have justified why this PR doesn't need tests.
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)

Signed-off-by: `Rubin Poster <rubinposter@gmail.com>`
2022-10-26 11:04:53 +02:00
Till c62ac3d6ad
Fix Current state appears in timeline in private history with many messages after (#2830)
The problem was that we weren't getting enough recent events, as most of
them were removed by the history visibility filter. Now we're getting
all events between the given input range and re-slice the returned
values after applying history visibility.
2022-10-25 15:15:24 +02:00
Till Faelligen 8b7bf5e7d7
Return forbidden if not a member anymore (fix #2802) 2022-10-25 15:00:52 +02:00
Neil Alexander db6a214b04
Prettify unit test output 2022-10-25 12:28:34 +01:00
Till 313cb3fd19
Filter /members, return members at given point (#2827)
Makes the tests
```
Can get rooms/{roomId}/members at a given point
Can filter rooms/{roomId}/members
```
pass, by moving `/members` and `/joined_members` to the SyncAPI.
2022-10-25 12:39:10 +02:00
Till 7506e3303e
Get messages from before user left the room (#2824)
This is going to make `Can get rooms/{roomId}/messages for a departed
room (SPEC-216)` pass, since we now only grep events from before the
user left the room.
2022-10-24 17:03:04 +02:00
Neil Alexander a553fe7705
Fix slow querying of cross-signing signatures 2022-10-24 10:07:50 +01:00
Till Faelligen 0843bd776e
Fix wrong config key 2022-10-24 07:10:50 +02:00
Neil Alexander 411db6083b
Version 0.10.4 (#2822)
Changelog and version bump.
2022-10-21 15:00:51 +01:00
Till 3cf42a1d64
Add syncapi_memberships table tests (#2805) 2022-10-21 12:53:04 +02:00
Till 9e4c3171da
Optimize inserting pending PDUs/EDUs (#2821)
This optimizes the association of PDUs/EDUs to their destination by
inserting all destinations in one transaction.
2022-10-21 12:50:51 +02:00
Neil Alexander e98d75fd63
Verify room_id, type, sender and state_key field lengths using bytes rather than codepoints (update to matrix-org/gomatrixserverlib@7c772f1, reverts bbb3ade4a2) 2022-10-21 10:15:08 +01:00
Till e57b301722
Set display_name and/or avatar_url for server notices (#2820)
This should fix #2815 by making sure we actually set the `display_name`
and/or `avatar_url` and create the needed membership event.
To avoid creating a new membership event when starting Dendrite,
`SetAvatarURL` and `SetDisplayName` now return a `Changed` value, which
also makes the regular endpoints idempotent.
2022-10-21 10:48:25 +02:00
Till 40cfb9a4ea
Fix invite -> leave -> join dance when accepting invites (#2817)
As mentioned in
https://github.com/matrix-org/dendrite/issues/2361#issuecomment-1139394565
and observed by ourselves, this should fix the odd `invite -> leave ->
join` dance when accepting invites.
2022-10-21 09:26:22 +01:00
Neil Alexander 73e02463cf
Allow m.read.private to clear notifications (#2811)
Otherwise if a user switches to private read receipts, they may not be
able to clear notification counts.
2022-10-21 09:19:52 +01:00