Commit graph

2360 commits

Author SHA1 Message Date
Neil Alexander e6960d0b15
Update to matrix-org/pinecone@608215eb1b 2022-09-14 14:25:25 +01:00
Neil Alexander 7f89fed1e4
Revert 482914aef4 2022-09-14 09:55:50 +01:00
Neil Alexander 482914aef4
Use AckNone on the ephemeral room input consumer 2022-09-13 15:25:02 +01:00
Neil Alexander b05e028f7d
Tweak LoadMembershipAtEvent behaviour when state not known (#2716)
Previously `LoadMembershipAtEvent` would fail if the state before one of
the events was not known, i.e. because it was an outlier. This modifies
it so that it gracefully handles not knowing the state and returns no
memberships instead, so that history visibility doesn't freak out and
kill `/sync` requests dead.
2022-09-13 12:52:09 +01:00
Neil Alexander 3e55856254
Always resolve state in QueryStateAfterEvents 2022-09-13 09:37:38 +01:00
Till c366ccdfca
Send-to-device consumer/producer tweaks (#2713)
Some tweaks for the send-to-device consumers/producers:
- use `json.RawMessage` without marshalling it first
- try further devices (if available) if we failed to `PublishMsg` in the
producers
- some logging changes (to better debug E2EE issues)
2022-09-13 09:35:45 +02:00
Till 100fa9b235
Check unique constraint errors when manually inserting migrations (#2712)
This should avoid unnecessary logging on startup if the migration (were
we need `InsertMigration`) was already executed.
This now checks for "unique constraint errors" for SQLite and Postgres
and fails the startup process if the migration couldn't be manually
inserted for some other reason.
2022-09-13 08:07:43 +02:00
Neil Alexander 62afb936a5
Update to matrix-org/gomatrixserverlib@7b96db4 2022-09-12 15:27:45 +01:00
Neil Alexander 47b2a5d6b8
Version 0.9.8 2022-09-12 13:41:07 +01:00
Neil Alexander 2792d0490f
Fix missing signature check on the /get_missing_events response 2022-09-12 13:30:51 +01:00
Neil Alexander 7595fbf58c
Update to NATS Server v2.9.0 2022-09-12 13:24:03 +01:00
Neil Alexander e9af30b3fe
Update to matrix-org/pinecone@b215925 2022-09-12 10:35:04 +01:00
Neil Alexander 3a9dde28fd
P2P demo tweaks 2022-09-12 10:19:02 +01:00
Neil Alexander 5997c32452
Update to matrix-org/gomatrixserverlib@dec87dbaa4 2022-09-11 15:39:11 +01:00
Till Faelligen af9a204cc0 Only verify if fulltext indexing is enabled 2022-09-09 17:19:29 +02:00
Neil Alexander e8687f6f82
Version 0.9.7 (#2707)
Changelog and version bump.
2022-09-09 15:01:13 +01:00
Neil Alexander 955e69a3b7
Optimise SharedUsers again by using complete composite index 2022-09-09 14:18:45 +01:00
Neil Alexander 6ee758df63
Optimise shared users query in Synx API slightly by removing a potential sort 2022-09-09 13:50:50 +01:00
Neil Alexander e1bc4f6a1e
Fix database transaction for keyserver DeleteDeviceKeys 2022-09-09 13:31:55 +01:00
Neil Alexander c0e17bbe1b
Fix transactions around assigning NIDs 2022-09-09 13:30:09 +01:00
Till 8196b29657
Change detection of already executed migrations (#2665)
This changes the detection of already executed migrations for the
roomserver state block and keychange refactor. It now uses schema tables
provided by the database engine to check if the column was already
removed. We now also store the migration in the migrations table.

This should stop e.g. Postgres from logging errors like `ERROR: column
"event_nid" does not exist at character 8`.
2022-09-09 13:14:52 +01:00
Neil Alexander 646de03d60
More writer fixes in the Sync API 2022-09-09 13:06:42 +01:00
Neil Alexander 34e1dc210b
Use database writer in user API DeleteOldNotifications 2022-09-09 12:59:28 +01:00
Till 64472d9aab
Update getting pushrules, add tests, tweak pushrules (#2705)
This PR
- adds tests for `evaluatePushrules`
- removes the need for the UserAPI on the `OutputStreamEventConsumer`
(for easier testing)
- adds a method to get the pushrules from the database
- adds a new default pushrule for `m.reaction` events (and some other
tweaks)
2022-09-09 13:56:33 +02:00
Till 42a82091a8
Fix issue with stale device lists (#2702)
We were only sending the last entry to the worker, so most likely missed
updates.
2022-09-08 12:03:44 +02:00
Till d5876abbe9
Fulltext implementation incl. config (#2480)
This adds the main component of the fulltext search.
This PR doesn't do anything yet, besides creating an empty fulltextindex
folder if enabled. Indexing events is done in a separate PR.
2022-09-07 18:15:54 +02:00
Neil Alexander 31f4ae8997
Use a single context instead 2022-09-07 16:24:43 +01:00
Neil Alexander 5014b35bd7
Update state reset capture to Sentry 2022-09-07 16:23:22 +01:00
Till 0d697f6754
Add HTTP status code to FederationClientError (#2699)
Also ensures we wait on more HTTP status codes.
2022-09-07 16:14:09 +02:00
Till 7e8c605f98
Avoid unneeded JSON operations (#2698)
We were `json.Unmarshal`ing the EDU and `json.Marshal`ing right before
sending the EDU to the stream. Those are now removed and the consumer
does `json.Unmarshal` once.
2022-09-07 12:16:04 +02:00
Till Faelligen 4e352390b6
Re-add waitTime if we're not blacklisted and no RetryAfter was
specified.
2022-09-07 12:13:02 +02:00
Till 2cfcfddecc
Add a SigningKeyUpdate producer (#2697)
This adds a new stream for signing key updates, this should ensure we
don't lose any updates over federation.
2022-09-07 11:45:12 +02:00
Till 440eb0f3a2
Handle errors differently in the DeviceListUpdater (#2695)
`If a device list update goes missing, the server resyncs on the next
one` was failing because a previous test would receive a `waitTime` of
1h, resulting in the test timing out.
This now tries to handle the returned errors differently, e.g. by using
the default `waitTime` of 2s. Also doesn't try further users in the
list, if one of the errors would cause a longer `waitTime`.
2022-09-07 11:44:27 +02:00
Neil Alexander 847032df36
Update to matrix-org/gomatrixserverlib@637a173 2022-09-07 09:11:36 +01:00
Neil Alexander 489ccc1c60
Stronger default power levels 2022-09-06 17:25:17 +01:00
Neil Alexander 70cd9a902c
State resolution fixes in applyAndAuthEvent (update to matrix-org/gomatrixserverlib#336) 2022-09-06 17:19:21 +01:00
Neil Alexander fd7661f69c
Tweak Sentry again 2022-09-05 17:37:17 +01:00
Neil Alexander 5992b4c7ed
Set Sentry user where known 2022-09-05 17:35:43 +01:00
Neil Alexander cd22ba22b0
Improve Sentry reporting 2022-09-05 17:25:11 +01:00
Neil Alexander ecee5f10f4
Tweak logging for detected state resets 2022-09-05 17:08:54 +01:00
Neil Alexander bea3dbe77b
State and auth tweaks (update to matrix-org/gomatrixserverlib#335) 2022-09-05 17:00:54 +01:00
Neil Alexander 1c1d09abd4
Optimise resolve-state tool 2022-09-05 14:17:04 +01:00
Neil Alexander d1f87e63f1
Move SetLatestEvents call 2022-09-05 13:16:14 +01:00
Till 350a5e5393
Only send (invite) events to the AS if it's interested in it (#2692)
This should hopefully fix #2691 (and maybe also #2690)
2022-09-02 16:03:33 +02:00
Neil Alexander fea869b41f
Update P2P demos 2022-09-01 17:12:27 +01:00
Neil Alexander 304acd7adc
Version 0.9.6 (#2689)
Changelog and version bump.
2022-09-01 14:49:42 +01:00
Neil Alexander 51d229b025
Configuration tweaks (#2567)
This makes the following changes:

* The various `Defaults` functions are now responsible for setting sane defaults if `generate` is specified, rather than hiding them in `generate-config`
* Some configuration options have been marked as `omitempty` so that they don't appear in generated configs unnecessarily (monolith-specific vs. polylith-specific options)
* A new option `-polylith` has been added to `generate-config` to create a config that makes sense for polylith deployments (i.e. including the internal/external API listeners and per-component database sections)
* A new option `-normalise` has been added to `generate-config` to take an existing file and add any missing options and/or defaults
2022-09-01 14:15:41 +01:00
Neil Alexander ad6b902b84
Refactor appservices component (#2687)
This PR refactors the app services component. It makes the following changes:

* Each appservice now gets its own NATS JetStream consumer
* The appservice database is now removed entirely, since we just use JetStream as a data source instead
* The entire component is now much simpler and we deleted lots of lines of code 💅

The result is that it should be much lighter and hopefully much more performant.
2022-09-01 09:20:40 +01:00
Neil Alexander 175f65407a
Allow batching in JetStreamConsumer (#2686)
This allows us to receive more than one message from NATS at a time if we want.
2022-08-31 12:21:56 +01:00
Neil Alexander ba0b3adab4
Pinecone standalone refactoring (#2685)
This refactors the `dendrite-demo-pinecone` executable so that it:

1. Converts the old `.key` file into a standard `.pem` file
2. Allows passing in the `--config` option to supply a normal Dendrite configuration file, so that you can configure PostgreSQL instead of SQLite, appservices and all the other usual stuff
2022-08-31 10:41:32 +01:00