Commit graph

1569 commits

Author SHA1 Message Date
Neil Alexander a92936ffa4
Merge branch 'master' into neilalexander/nats 2021-07-14 11:10:51 +01:00
Neil Alexander 5a76c3b07a
Update go.mod/go.sum 2021-07-14 10:49:22 +01:00
kegsay 2713d1935e
AWSY: update list and shuffle groups (#1919)
So it's more accurate.
2021-07-14 10:48:34 +01:00
kegsay e80098e186
bugfix: retire invites even when we cannot talk to the remote server to make/send_leave (#1918)
* bugfix: retire invites even when we cannot talk to the remote server to make/send_leave

Also modify the leave response in /sync to include a fake event as this is ultimately
what clients (and sytest) will use to determine leave-ness.

* hash the event ID

* Base64 not hex
2021-07-14 10:39:17 +01:00
Neil Alexander beb2ddf03d
Make sure NATS uses own persistent directory (TODO: make this configurable) 2021-07-14 10:23:04 +01:00
Neil Alexander e2d562257c
Try setting up with NATS 2021-07-13 13:34:41 +01:00
kegsay 7df3e691f2
Fix failing complement test (#1917)
Specifically `TestBannedUserCannotSendJoin`
2021-07-13 12:22:27 +01:00
kegsay f8ae391a5b
Expose more data when outputting output room events (#1916)
* Add more logging for content fields

* Fix fields
2021-07-13 11:19:21 +01:00
kegsay f698142073
update whitelist (#1914)
* update whitelist

* newline
2021-07-12 19:42:05 +01:00
Kegan Dougal 93cf1e7590 Actually bump GMSL 2021-07-12 18:58:25 +01:00
kegsay b130fab583
Update are we synapse groupings (#1913) 2021-07-12 16:27:13 +01:00
Kegan Dougal ea477d7d66 Update blacklist 2021-07-12 16:11:26 +01:00
Neil Alexander 913e4e651f
Merge branch 'add-nats-support' of github.com:s7evink/dendrite into add-nats-support 2021-07-12 14:02:09 +01:00
Neil Alexander a478b1682c
Merge branch 'master' into add-nats-support 2021-07-12 14:01:28 +01:00
Neil Alexander 48bdd79bde
Fix attribution in changelog 2021-07-12 11:54:11 +01:00
Neil Alexander 89a16bdcd9
Version 0.4.0 2021-07-12 11:48:29 +01:00
Kegan Dougal 0530302cd6 Add shared secret sytests to whitelist 2021-07-12 11:48:08 +01:00
Melroy van den Berg e48a08fef0
Propose config better (#1758)
Better explain where the config file are located and how to deal with the yml file.

Co-authored-by: kegsay <kegan@matrix.org>
2021-07-12 10:13:17 +01:00
Neil Alexander acec6fa979
Move a couple of callers to helpers.IsServerCurrentlyInRoom over to the query API (#1912) 2021-07-09 17:49:59 +01:00
kegsay 1ed732cc78
Implement /_synapse/admin/v1/register (#1911)
* Implement /_synapse/admin/v1/register

This is implemented identically to Synapse, so scripts which work
with Synapse should work with Dendrite.

```
    Test 27 POST /_synapse/admin/v1/register with shared secret... OK
    Test 28 POST /_synapse/admin/v1/register admin with shared secret... OK
    Test 29 POST /_synapse/admin/v1/register with shared secret downcases capitals... OK
    Test 30 POST /_synapse/admin/v1/register with shared secret disallows symbols... OK
```

Sytest however has `implementation_specific => "synapse"` which stops these
tests from running.

* Add missing muxes to gobind

* Linting
2021-07-09 16:52:31 +01:00
Neil Alexander c8408a6387
Add more optimised code path for checking if we're in a room (#1909)
* Add more optimised code path for checking if we're in a room

* Fix database queries

* Fix federation API test

* Fix logging

* Review comments

* Make separate API call for room membership
2021-07-09 16:36:45 +01:00
kegsay 3e50bac944
bugfix: order the state blocks so recreating state snapshots works correctly (#1908)
* Logging

* Revert "Logging"

This reverts commit 23ce334182.

* bugfix: order the state blocks so recreating state snapshots works correctly
2021-07-09 10:49:49 +01:00
Till Faelligen 8bdb508c84 Merge branch 'add-nats-support' of github.com:S7evinK/dendrite into add-nats-support 2021-07-09 08:52:26 +02:00
Till Faelligen 4f1f4c320a Merge branch 'master' of https://github.com/matrix-org/dendrite into add-nats-support 2021-07-09 08:18:19 +02:00
Till Faelligen 341351c75a Update saramajetstream 2021-07-09 08:16:33 +02:00
Neil Alexander 816e1a402b
Fix bug when rejecting invites (#1907)
* Fix rejecting invites maybe

* Remove comment that is no longer correct

* Review comment on performFederatedRejectInvite
2021-07-08 14:54:03 +01:00
kegsay 70e4bbda3b
Only log filename and not entire path (#1906) 2021-07-08 13:13:27 +01:00
kegsay ef331c52af
dendrite-upgrade-test: tweaks to get it to run under CI in docker (#1905)
* dendrite-upgrade-test: tweaks to get it to run under CI in docker

* Linting
2021-07-08 12:28:04 +01:00
Kegan Dougal 3fb5ee7e1c linting 2021-07-08 10:17:21 +01:00
Kegan Dougal 717d16345c Improve error handling and close files post-tarring 2021-07-08 10:17:21 +01:00
Neil Alexander fc5f8e1743
Merge branch 'master' into add-nats-support 2021-07-08 10:12:28 +01:00
Neil Alexander f2974721d5
Fix concurrent map reads/writes on t.hadEvents (#1902)
* Fix concurrent map reads/writes on t.hadEvents

* Add hadEvent function
2021-07-07 18:55:44 +01:00
kegsay 5a09290c32
db migration: handle create events with no state blocks from v0.1.0 (#1904) 2021-07-07 17:07:33 +01:00
kegsay bb6e4487dd
dendrite-upgrade-test: support --from HEAD-N, cleanup on startup (#1903) 2021-07-07 14:20:07 +01:00
kegsay d72d634391
Add dendrite-upgrade-test (#1901)
* Add WIP binary for testing dendrite version upgrades

* WIP dendrite upgrade work

* Finish dendrite upgrade checks

* go mod tidy

* Review comments; print container logs on failure

* Linting
2021-07-07 12:06:17 +01:00
Till Faelligen 1533c648e3 Merge branch 'master' of https://github.com/matrix-org/dendrite into add-nats-support 2021-07-06 20:03:51 +02:00
Neil Alexander bcd3ef38d0
Track expiry rate on pduCountTotal 2021-07-05 13:47:37 +01:00
Neil Alexander 99d8e1c107
Federation API fixes (#1899)
* Ensure worker has work before starting goroutine

* Revert "Remove processEventWithMissingStateMutex"

This reverts commit 7f02eab47d.

* Use request context when processing transactions

* Keep goroutine count down by not starting work for things where the caller gave up

* Remove mutex, start workers at correct time
2021-07-05 12:14:31 +01:00
Neil Alexander 7f02eab47d
Remove processEventWithMissingStateMutex 2021-07-05 09:14:24 +01:00
Till Faelligen 0c88ac6158 Use all configured NATS addresses 2021-07-04 22:11:00 +02:00
Till Faelligen c575f5501f Merge branch 'master' of https://github.com/matrix-org/dendrite into add-nats-support 2021-07-02 20:21:17 +02:00
Neil Alexander 355ce7ed30
Update to matrix-org/gomatrixserverlib#266 (+ missing file) 2021-07-02 16:30:37 +01:00
Neil Alexander 6bb994bcfa
Update to matrix-org/gomatrixserverlib#265 for faster power level content parsing 2021-07-02 16:13:56 +01:00
Neil Alexander 57320897cb
Federation API workers for /send to reduce memory usage (#1897)
* Try to process rooms concurrently in FS /send

* Clean up

* Use request context so that dead things don't linger for so long

* Remove mutex

* Free up pdus slice so only references remaining are in channel

* Revert "Remove mutex"

This reverts commit 8558075e8c.

* Process EDUs in parallel

* Try refactoring /send concurrency

* Fix waitgroup

* Release on waitgroup

* Respond to transaction

* Reduce CPU usage, fix unit tests

* Tweaks

* Move into one file
2021-07-02 12:33:27 +01:00
Neil Alexander 192a7a7923
Roomserver input backpressure metric
Squashed commit of the following:

commit 56e934ac0aeedcfb2c072010959ba49734d4e0cb
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Fri Jul 2 09:39:30 2021 +0100

    Fix metric

commit 3911f3a0c17b164b012e881c085ceca30f5de408
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Fri Jul 2 09:36:29 2021 +0100

    Register correct metric

commit a9ddbfaed421538a701151801e9451198a8be4f3
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Fri Jul 2 09:33:33 2021 +0100

    Try to capture RS input backpressure metric
2021-07-02 09:48:55 +01:00
Neil Alexander 2647f6e9c5
Fix concurrent map read/write on haveEvents (#1893) 2021-06-30 12:32:20 +01:00
Neil Alexander b7a2d369c0
Change how servers are selected for missing auth/prev events (#1892)
* Change how servers are selected for missing auth/prev events

* Shuffle order

* Move ServersInRoomProvider into api package
2021-06-30 12:05:58 +01:00
Neil Alexander 0e69212206
Give up on loops when the context expires (#1891) 2021-06-30 10:39:47 +01:00
Neil Alexander 3afb161352
Reduce memory usage in federation /send endpoint (#1890)
* More aggressive event caching

* Deduplicate /state results

* Deduplicate more

* Ensure we use the correct list of events when excluding repeated state

* Fixes

* Ensure we track all events we already knew about properly
2021-06-30 10:01:56 +01:00
kegsay c849e74dfc
db migration: fix #1844 and add additional assertions (#1889)
* db migration: fix #1844 and add additional assertions

- Migration scripts will now check to see if there are any unconverted
  snapshot IDs and fail the migration if there are any. This should
  prevent people from getting a corrupt database in the event the root
  cause is still unknown.
- Add an ORDER BY clause when doing batch queries in the postgres
  migration. LIMIT and OFFSET without ORDER BY are undefined and must
  not be relied upon to produce a deterministic ordering (e.g row order).
  See https://www.postgresql.org/docs/current/queries-limit.html

* Linting

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2021-06-29 11:25:17 +01:00