Commit graph

2538 commits

Author SHA1 Message Date
Neil Alexander 34e1dc210b
Use database writer in user API DeleteOldNotifications 2022-09-09 12:59:28 +01:00
Till 64472d9aab
Update getting pushrules, add tests, tweak pushrules (#2705)
This PR
- adds tests for `evaluatePushrules`
- removes the need for the UserAPI on the `OutputStreamEventConsumer`
(for easier testing)
- adds a method to get the pushrules from the database
- adds a new default pushrule for `m.reaction` events (and some other
tweaks)
2022-09-09 13:56:33 +02:00
Till 42a82091a8
Fix issue with stale device lists (#2702)
We were only sending the last entry to the worker, so most likely missed
updates.
2022-09-08 12:03:44 +02:00
Till d5876abbe9
Fulltext implementation incl. config (#2480)
This adds the main component of the fulltext search.
This PR doesn't do anything yet, besides creating an empty fulltextindex
folder if enabled. Indexing events is done in a separate PR.
2022-09-07 18:15:54 +02:00
Neil Alexander 31f4ae8997
Use a single context instead 2022-09-07 16:24:43 +01:00
Neil Alexander 5014b35bd7
Update state reset capture to Sentry 2022-09-07 16:23:22 +01:00
Till 0d697f6754
Add HTTP status code to FederationClientError (#2699)
Also ensures we wait on more HTTP status codes.
2022-09-07 16:14:09 +02:00
Till 7e8c605f98
Avoid unneeded JSON operations (#2698)
We were `json.Unmarshal`ing the EDU and `json.Marshal`ing right before
sending the EDU to the stream. Those are now removed and the consumer
does `json.Unmarshal` once.
2022-09-07 12:16:04 +02:00
Till Faelligen 4e352390b6
Re-add waitTime if we're not blacklisted and no RetryAfter was
specified.
2022-09-07 12:13:02 +02:00
Till 2cfcfddecc
Add a SigningKeyUpdate producer (#2697)
This adds a new stream for signing key updates, this should ensure we
don't lose any updates over federation.
2022-09-07 11:45:12 +02:00
Till 440eb0f3a2
Handle errors differently in the DeviceListUpdater (#2695)
`If a device list update goes missing, the server resyncs on the next
one` was failing because a previous test would receive a `waitTime` of
1h, resulting in the test timing out.
This now tries to handle the returned errors differently, e.g. by using
the default `waitTime` of 2s. Also doesn't try further users in the
list, if one of the errors would cause a longer `waitTime`.
2022-09-07 11:44:27 +02:00
Neil Alexander 847032df36
Update to matrix-org/gomatrixserverlib@637a173 2022-09-07 09:11:36 +01:00
Neil Alexander 489ccc1c60
Stronger default power levels 2022-09-06 17:25:17 +01:00
Neil Alexander 70cd9a902c
State resolution fixes in applyAndAuthEvent (update to matrix-org/gomatrixserverlib#336) 2022-09-06 17:19:21 +01:00
Neil Alexander fd7661f69c
Tweak Sentry again 2022-09-05 17:37:17 +01:00
Neil Alexander 5992b4c7ed
Set Sentry user where known 2022-09-05 17:35:43 +01:00
Neil Alexander cd22ba22b0
Improve Sentry reporting 2022-09-05 17:25:11 +01:00
Neil Alexander ecee5f10f4
Tweak logging for detected state resets 2022-09-05 17:08:54 +01:00
Neil Alexander bea3dbe77b
State and auth tweaks (update to matrix-org/gomatrixserverlib#335) 2022-09-05 17:00:54 +01:00
Neil Alexander 1c1d09abd4
Optimise resolve-state tool 2022-09-05 14:17:04 +01:00
Neil Alexander d1f87e63f1
Move SetLatestEvents call 2022-09-05 13:16:14 +01:00
Till 350a5e5393
Only send (invite) events to the AS if it's interested in it (#2692)
This should hopefully fix #2691 (and maybe also #2690)
2022-09-02 16:03:33 +02:00
Neil Alexander fea869b41f
Update P2P demos 2022-09-01 17:12:27 +01:00
Neil Alexander 304acd7adc
Version 0.9.6 (#2689)
Changelog and version bump.
2022-09-01 14:49:42 +01:00
Neil Alexander 51d229b025
Configuration tweaks (#2567)
This makes the following changes:

* The various `Defaults` functions are now responsible for setting sane defaults if `generate` is specified, rather than hiding them in `generate-config`
* Some configuration options have been marked as `omitempty` so that they don't appear in generated configs unnecessarily (monolith-specific vs. polylith-specific options)
* A new option `-polylith` has been added to `generate-config` to create a config that makes sense for polylith deployments (i.e. including the internal/external API listeners and per-component database sections)
* A new option `-normalise` has been added to `generate-config` to take an existing file and add any missing options and/or defaults
2022-09-01 14:15:41 +01:00
Neil Alexander ad6b902b84
Refactor appservices component (#2687)
This PR refactors the app services component. It makes the following changes:

* Each appservice now gets its own NATS JetStream consumer
* The appservice database is now removed entirely, since we just use JetStream as a data source instead
* The entire component is now much simpler and we deleted lots of lines of code 💅

The result is that it should be much lighter and hopefully much more performant.
2022-09-01 09:20:40 +01:00
Neil Alexander 175f65407a
Allow batching in JetStreamConsumer (#2686)
This allows us to receive more than one message from NATS at a time if we want.
2022-08-31 12:21:56 +01:00
Neil Alexander ba0b3adab4
Pinecone standalone refactoring (#2685)
This refactors the `dendrite-demo-pinecone` executable so that it:

1. Converts the old `.key` file into a standard `.pem` file
2. Allows passing in the `--config` option to supply a normal Dendrite configuration file, so that you can configure PostgreSQL instead of SQLite, appservices and all the other usual stuff
2022-08-31 10:41:32 +01:00
Neil Alexander 02ec00b1bb
Check for "creator" field in m.room.create during event auth (update to matrix-org/gomatrixserverlib@c71e518) 2022-08-30 17:40:46 +01:00
PiotrKozimor 2be43560ca
Index on syncapi_send_to_device table (#2684)
Introduced index improves select query performance. Example execution time of `selectSendToDeviceMessagesSQL` query dropped from 80 ms to 15 ms. No sytest modifications are required.

### Pull Request Checklist

* [x] I have added added tests for PR _or_ I have justified why this PR doesn't need tests.
* [x] Pull request includes a [sign off](https://github.com/matrix-org/dendrite/blob/main/docs/CONTRIBUTING.md#sign-off)

Signed-off-by: `Piotr Kozimor <p1996k@gmail.com>`
2022-08-30 14:47:54 +01:00
Till 93a6e2f4d3
Set the notification count when sending push notifications (#2683) 2022-08-30 14:54:57 +02:00
Till 7313f56f44
Use existing limit instead of default limit when lazy loading members (#2682)
This should fix an issue where we return less than the expected membership events, when doing an initial sync.
When doing an initial sync, the state limit is set to `math.MaxInt32`, while the default filter is set to 20.
2022-08-30 14:18:47 +02:00
Till b0e2ea0f37
Fix race condition on startup (#2679)
`SetupAndServeHTTP` would race in `configureHTTPErrors` and while configuring routes.
2022-08-30 13:59:13 +02:00
Neil Alexander bbb3ade4a2
Fix the Cat Overflow bug (update to matrix-org/gomatrixserverlib@2020bdc) 2022-08-30 11:44:26 +01:00
Till Faelligen 21ef487ff8
Log roomserver error 2022-08-29 14:18:33 +02:00
Brian Meek 704cc5c9f5
Race in keyserver intialization (#2619)
Signed-off-by: Brian Meek <brian@hntlabs.com>
2022-08-29 09:10:42 +02:00
Till Faelligen aba171d9bc
Default to limit of 20 state events 2022-08-29 07:23:46 +02:00
Neil Alexander a3eb4e5e98
Enforce type, sender, state_key and room_id lengths using codepoints rather than bytes (update to matrix-org/gomatrixserverlib@5f66df0, matrix-org/matrix-spec#1001) 2022-08-28 09:52:03 +01:00
Neil Alexander 38bed30b41
Restrict username length properly (#2676)
This fixes #2674 by fixing the username check to comply with the [spec appendices](https://spec.matrix.org/v1.3/appendices/#user-identifiers):

> The length of a user ID, including the @ sigil and the domain, MUST NOT exceed 255 characters.
2022-08-26 09:56:41 +01:00
Neil Alexander ed79e8626a
Version 0.9.5 (#2673)
Changelog and version bump.
2022-08-25 14:14:10 +01:00
Till 07dd9bd995
SyncAPI tweaks/fixes (#2671)
- Reverts 9dc57122d9 as it was causing issues https://github.com/matrix-org/dendrite/issues/2660
- Updates the GMSL `DefaultStateFilter` to use a limit of 20 events
- Uses the timeline events to determine the new position instead of the state events
2022-08-25 13:42:47 +01:00
Neil Alexander 8ff3f1a7c9
Remove a couple unnecessary Sentry captures from backfill 2022-08-25 11:01:07 +01:00
Neil Alexander cd7fa34595
Tweak logging and Sentry reporting for roomserver input 2022-08-25 10:57:27 +01:00
Neil Alexander 16156b0b09
Fix 500s on /state, /state_ids when state not known (#2672)
This was due to bad error bubbling.
2022-08-25 09:51:36 +01:00
Neil Alexander 522bd2999f
Allow un-rejecting events on reprocessing 2022-08-24 14:03:06 +01:00
Till 78e5d05efc
Only set backOffStarted to false if until is not zero (#2669) 2022-08-23 16:54:42 +02:00
Neil Alexander 14fea600bb
Detect types.MissingStateError in CheckServerAllowedToSeeEvent (#2667)
This will hopefully stop some 500 errors on `/event` where there is no state-before known.
2022-08-23 13:57:11 +01:00
Till 95a509757a
Complement QoL changes (#2663)
This PR does the following:
- adds a `keysize` parameter to `generate-keys`, so we can use lower sized keys when running in CI
- updates the Complement docker files to use BuildKit (requires Docker >18.09)
- uses `exec` when executing `dendrite-monotlith-server`, making it PID 1 inside docker, which results in Dendrite actually receiving the `SIGTERM` signal send by Docker. (Making it faster when running tests with Complement, as we don't take 10 seconds to timeout)
2022-08-23 13:10:29 +02:00
Till 33129c02f7
Add timeout parameter & trim URL (#2666)
A timeout of 10 seconds could cause issues with servers having a high `bcrypt_cost` configured in the config.
This adds a parameter to manually configure the timeout, defaults to 30 seconds.
2022-08-23 11:10:41 +02:00
Neil Alexander 2668050e53
Tweak soft-failure handling in roomserver
commit 1929b688e31987c46e0c8a546f0f9cb0a46bf9a3
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Mon Aug 22 10:09:44 2022 +0100

    Still process state-before for soft-failed events

commit e83c0b701d40d78b92072c4643f6bc6f71b72800
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Mon Aug 22 10:06:50 2022 +0100

    Improve logging

commit 29e26124bc27cb83d449de2a4214b253c594aa93
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Mon Aug 22 09:58:13 2022 +0100

    Don't store soft-failed events as rejected
2022-08-22 10:34:07 +01:00