Commit graph

2500 commits

Author SHA1 Message Date
Neil Alexander fb2e7d1b05
Put P2P Demos back into their own Dockerfiles 2022-11-03 13:57:44 +00:00
Neil Alexander 9625a79926
Update to matrix-org/pinecone@37f2e9b 2022-11-03 13:06:21 +00:00
gitlab-nickfreeman 23a25be904
Update README.md (#2855)
There only are 2, not 3 sample `docker-compose` files.
This is only a change to the README.md, no need for Go unit tests

### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `Nick Freeman <nick@nickfreeman.de>`
2022-11-03 12:18:25 +00:00
Neil Alexander 8704e84898
Tweak removeDuplicates calls to use events instead of recentEvents (#2853)
... since `events` is *after* history visibility filtering, not before
it.
2022-11-03 10:19:37 +00:00
0x1a8510f2 1fcbb9b5e5
Fix workdir in Dockerfile (and make it a volume) (#2852) 2022-11-03 08:37:58 +00:00
Till Faelligen 85d740ea1b
Add GHA permission to upload security events 2022-11-03 08:26:46 +01:00
Till Faelligen f5b11e30a4
Hopefully fix GHA sarif upload 2022-11-03 08:20:51 +01:00
devonh 4afadebd99
Add readme for dendrite-demo-pinecone (#2851) 2022-11-02 20:29:30 +00:00
Neil Alexander ef52731e9f
Tweak FLAGS in GHA Docker builds 2022-11-02 14:41:38 +00:00
Neil Alexander 9c0725feac
Maybe fix GHA 2022-11-02 14:09:19 +00:00
Neil Alexander ca8bc87380
Multi-stage Docker builds (#2850)
This builds on @S7evinK's work to make multi-stage Docker builds. Now
that we can build SQLite without Cgo this should be much simpler and
should make Docker builds in CI significantly faster.

Co-authored-by: Till Faelligen <tfaelligen@gmail.com>
Co-authored-by: Till Faelligen <davidf@element.io>
Co-authored-by: Till Faelligen <2353100+S7evinK@users.noreply.github.com>
2022-11-02 14:04:08 +00:00
0x1a8510f2 51ab0a8ccf
Fix moderncsqlite errors and rebase onto main (#2832)
This is #2819 but rebased on latest `main`. This PR is against main too
as opposed to the `moderncsqlite` branch.

The main change here is simply:

```go
// add query parameters to the dsn
if strings.Contains(dsn, "?") {
	dsn += "&"
} else {
	dsn += "?"
}

// wait some time before erroring if the db is locked
// https://gitlab.com/cznic/sqlite/-/issues/106#note_1058094993
dsn += "_pragma=busy_timeout%3d10000"
```

### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added tests for PR _or_ I have justified why this PR
doesn't need tests.
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed off privately.

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-11-02 13:20:10 +00:00
Neil Alexander 16c2a95900
Improve logging for processEventWithMissingState 2022-11-02 11:30:49 +00:00
Till 86b25a6337
Add message stats to reporting (#2748)
Since we're now listening on the `OutputRoomEvent` stream, we are able
to store messages stats.
2022-11-02 10:18:11 +00:00
Till b367cfeddf
Implement /thirdparty endpoints (#2831)
Implements the following endpoints 
```
GET /_matrix/client/v3/thirdparty/protocols
GET /_matrix/client/v3/thirdparty/protocols/{protocol}
GET /_matrix/client/v3/thirdparty/location
GET /_matrix/client/v3/thirdparty/location/{protocol}
GET /_matrix/client/v3/thirdparty/user
GET /_matrix/client/v3/thirdparty/user/{protocol}
```
2022-11-02 10:17:53 +00:00
Tak Wai Wong 75a508cc27
Fix issue where a member is forced to leave a room when the invite is marked deleted (#2839)
Proposed fix for issue:
https://github.com/matrix-org/dendrite/issues/2838

Suppose bob received invites to spaceA and spaceB.
When Bob joins spaceA, we add an OutputEvent event to retire the invite.
This sets the invite to "deleted" in the database. This makes sense.

The bug is in stream_invites.go. Triggered when bob received a new
invite for spaceB, and does a client sync.

In the block (line 76)
`for roomID := range retiredInvites

   if _, ok := req.Response.Rooms.Invite[roomID]; ok {
	continue
   }

   if _, ok := req.Response.Rooms.Join[roomID]; ok {
	continue
  }
...
` 

Bob is not in either maps even though he had just accepted the invite
for spaceA. Consequently, the spaceA invite is treated as a retired
invite, and a membership Leave event is generated. What bob sees is that
after accepting the invite to spaceB, he lose access to spaceA.


### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [ ] I have added tests for PR _or_ I have justified why this PR
doesn't need tests.
* [x ] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `Tak Wai Wong <tak@hntlabs.com>`

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-11-02 10:02:23 +00:00
Neil Alexander 3db9e98456
Don't limit "state" (#2849)
This is apparently some incorrect behaviour that we built as a result of
a spec bug (matrix-org/matrix-spec#1314) where we were applying a filter
to the `"state"` section of the `/sync` response incorrectly. The client
then has no way to know that the state was limited.

This PR removes the state limiting, which probably also helps #2842.
2022-11-02 09:34:19 +00:00
Neil Alexander 8a1904ffe5
Update pull request template 2022-11-02 09:33:00 +00:00
Neil Alexander 52478dac3c
Version 0.10.6 2022-11-01 17:14:30 +00:00
Neil Alexander 501977f6fe
Fix a panic in ToClientEvents etc. 2022-11-01 16:58:51 +00:00
ash lea 5aaa60227a
return required room_id field in /members (#2846)
### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [ ] I have added tests for PR _or_ I have justified why this PR
doesn't need tests.
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `ash lea <example@thisismyactual.email>`
2022-11-01 16:42:07 +00:00
Neil Alexander 42d7e3ee0d
Update dependencies 2022-11-01 16:15:55 +00:00
Neil Alexander 6663728eb1
Fix SQLite roomserver_published migration 2022-11-01 16:08:13 +00:00
Till 2acc1d65fb
Optimize history visibility checks (#2848)
This optimizes history visibility checks by (mostly) avoiding database
hits.
Possibly solves https://github.com/matrix-org/dendrite/issues/2777

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-11-01 15:07:17 +00:00
Neil Alexander 0b21cb78aa
Try to fix a panic in the sync API PDU stream 2022-11-01 14:45:15 +00:00
Neil Alexander 7bd6631935
Move code for calculating auth difference into GMSL 2022-11-01 10:12:11 +00:00
Till Faelligen 869bf4d0ac
Fix flakey stats tests 2022-11-01 08:39:16 +01:00
Neil Alexander 8c7b274e4e
Version 0.10.5 (#2845)
Changelog and version bump.
2022-10-31 16:44:41 +00:00
Neil Alexander 4c38bd76ce
Fix go.mod 2022-10-31 15:15:18 +00:00
Neil Alexander 7307701a24
Tweak "state" and "timeline" filtering (#2844)
This should stop state events disappearing down a gap where we'd try to
separate out the sections *before* applying history visibility instead
of after.

This may be a better approach than #2843 but I hope @tak-hntlabs will
shout if it isn't.
2022-10-31 15:14:08 +00:00
Neil Alexander 66a82e0fa4
Update to matrix-org/gomatrixserverlib@0885c35 2022-10-31 15:12:05 +00:00
Neil Alexander f10c6f26e5
Add /_dendrite/admin/downloadState/{serverName}/{roomID} admin endpoint 2022-10-31 09:13:28 +00:00
Till 69aff372f3
Limit recent events when going backwards (#2840)
If we're going backwards, we were selecting potentially thousands of
events, which in turn were fed to history visibility checks, resulting
in bad sync performance.
2022-10-28 13:40:51 +02:00
X. Ding 0782011f54
Add hcaptcha support besides Google ReCaptcha (#2834)
### Pull Request Checklist
This PR add support for hcaptcha.com as an alternative to Google
ReCaptcha. It also makes possible for user to customize ReCaptcha URL
when needed. (Such as use recaptcha.net instead of www.google.com)

This feature needs manual test cuz it involves 3rd party _captcha_.

Signed-off-by: `Simon Ding <dxl@plotbridge.com>`

Co-authored-by: dxl <dxl@plotbridge.com>
2022-10-28 11:25:01 +01:00
Till Faelligen f6035822e7
Simplify error checking and check the correct error 2022-10-28 08:17:40 +02:00
Neil Alexander a2706e6498
Refactor claimRemoteKeys 2022-10-27 15:34:26 +01:00
Till Faelligen a785532463
Fix upgrade appservices 2022-10-27 16:01:51 +02:00
Till 444b4bbdb8
Add AS specific public room list endpoints (#2836)
Adds `PUT
/_matrix/client/v3/directory/list/appservice/{networkId}/{roomId}` and
`DELTE
/_matrix/client/v3/directory/list/appservice/{networkId}/{roomId}`
support, as well as the ability to filter `/publicRooms` on networkID
and including all networks.
2022-10-27 14:40:35 +02:00
Till a169a9121a
Fix /members (#2837)
Fixes a bug introduced in #2827, where the SyncAPI might not have all
requested eventIDs, resulting in too few members returned.
2022-10-27 14:18:22 +02:00
Till fa96811e64
Add scheduled tasks to run tests with race detection (#2814)
Needs https://github.com/matrix-org/sytest/pull/1308 to be actually
useful.
Not sure if we need to run Sytest in all combinations with enabled race
detection.

Closes https://github.com/matrix-org/dendrite/issues/491
2022-10-27 12:12:50 +02:00
Neil Alexander 238b6ef2cd
Update Yggdrasil demo 2022-10-26 18:37:01 +01:00
devonh 97491a174b
Associate events in db before queueing them to send (#2833)
Fixes a race condition between sending federation events and having them
fully associated in the database.
2022-10-26 17:35:01 +01:00
devonh a74aea0714
Add network interface callback to pinecone build (#2825)
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-10-26 17:25:57 +01:00
Neil Alexander 5298dd1133
Update federation API consumers 2022-10-26 14:52:33 +01:00
Neil Alexander f6dea712d2
Initial support for multiple server names (#2829)
This PR is the first step towards virtual hosting by laying the
groundwork for multiple server names being configured.
2022-10-26 12:59:19 +01:00
Neboer 2a4c7f45b3
Add support for config "auto_join_rooms" (#2823)
Add support for config "auto_join_rooms". Now new accounts can join the
rooms in config file automatically.

### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have justified why this PR doesn't need tests.
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)

Signed-off-by: `Rubin Poster <rubinposter@gmail.com>`
2022-10-26 11:04:53 +02:00
Till c62ac3d6ad
Fix Current state appears in timeline in private history with many messages after (#2830)
The problem was that we weren't getting enough recent events, as most of
them were removed by the history visibility filter. Now we're getting
all events between the given input range and re-slice the returned
values after applying history visibility.
2022-10-25 15:15:24 +02:00
Till Faelligen 8b7bf5e7d7
Return forbidden if not a member anymore (fix #2802) 2022-10-25 15:00:52 +02:00
Neil Alexander db6a214b04
Prettify unit test output 2022-10-25 12:28:34 +01:00
Till 313cb3fd19
Filter /members, return members at given point (#2827)
Makes the tests
```
Can get rooms/{roomId}/members at a given point
Can filter rooms/{roomId}/members
```
pass, by moving `/members` and `/joined_members` to the SyncAPI.
2022-10-25 12:39:10 +02:00