Commit graph

169 commits

Author SHA1 Message Date
Devon Hudson 9d52bed9c1
Check error and log when closing full text search 2023-01-20 11:59:07 -07:00
Devon Hudson 33cdc5adb7
Close full text search index on dendrite shutdown 2023-01-20 11:52:10 -07:00
Devon Hudson 3b76c776f6
Merge branch 'main' into mailbox 2023-01-20 11:49:07 -07:00
Till ce2bfc3f2e
Make tests more reliable (#2948)
When using `testrig.CreateBase` and then using that base for other
`NewInternalAPI` calls, we never actually shutdown the components.
`testrig.CreateBase` returns a `close` function, which only removes the
database, so still running components have issues connecting to the
database, since we ripped it out underneath it - which can result in
"Disk I/O" or "pq deadlock detected" issues.
2023-01-20 12:45:56 +01:00
Devon Hudson 3c54ea1d56
Merge branch 'main' into mailbox 2023-01-19 14:47:14 -07:00
Andrew Morgan eeeb3017d6
Switch the default config option values for recaptcha_sitekey_class and recaptcha_form_field (#2939)
Attempting to use the [web auth fallback
mechanism](https://spec.matrix.org/v1.5/client-server-api/#fallback) for
Google ReCAPTCHA with the default setting for
`client_api.recaptcha_sitekey_class` of "g-recaptcha-response" results
in no captcha being rendered:


![image](https://user-images.githubusercontent.com/1342360/212482321-14980045-6e20-4d59-adaa-59a01ad88367.png)

I cross-checked the captcha code between [dendrite.matrix.org's fallback
page](https://dendrite.matrix.org/_matrix/client/r0/auth/m.login.recaptcha/fallback/web?session=asdhjaksd)
and [matrix-client.matrix.org's
one](https://matrix-client.matrix.org/_matrix/client/r0/auth/m.login.recaptcha/fallback/web?session=asdhjaksd)
(which both use the same captcha public key) and noticed a discrepancy
in the `class` attribute of the div that renders the captcha.
[ReCAPTCHA's docs
state](https://developers.google.com/recaptcha/docs/v3#automatically_bind_the_challenge_to_a_button)
to use "g-recaptcha" as the class for the submit button.

I noticed this when user `@parappanon:parappa.party` reported that they
were also seeing no captcha being rendered on their Dendrite instance.
Changing `client_api.recaptcha_sitekey_class` to "g-recaptcha" caused
their captcha to render properly as well.

There may have been a change in the class name from ReCAPTCHA v2 to v3?
The [docs for
v2](https://developers.google.com/recaptcha/docs/display#auto_render)
also request one uses "g-recaptcha" though.

Thus I propose changing the default setting to unbreak people's
recaptcha auth fallback pages. Should fix dendrite.matrix.org as well.
2023-01-16 12:52:30 +01:00
Devon Hudson 75bc8ab826
Various typos and whitespace cleanup 2023-01-12 14:16:23 -07:00
Devon Hudson 75fd677081
Merge branch 'main' into mailbox 2023-01-12 09:29:16 -07:00
Devon Hudson 07d2976854
Finish s&f rename refactor 2023-01-12 08:21:26 -07:00
Till d579ddb8e7
Add simplified helm chart (#2905)
As discussed yesterday, a simplified version of [my
helm](https://github.com/S7evinK/dendrite-helm) which deploys a monolith
with internal NATS and an optionally enabled PostgreSQL server. If the
PostgreSQL dependency is not enabled, a user specified connection string
is constructed.

Co-authored-by: kegsay <kegan@matrix.org>
2023-01-06 15:44:10 +01:00
Devon Hudson a5e012eda9
Merge branch 'main' into mailbox 2022-12-23 08:21:02 -07:00
Till f762ce1050
Add clientapi tests (#2916)
This PR
- adds several tests for the clientapi, mostly around `/register` and
auth fallback.
- removes the now deprecated `homeserver` field from responses to
`/register` and `/login`
- slightly refactors auth fallback handling
2022-12-23 14:11:11 +01:00
Devon Hudson eaa09f0d82
Goimports config test imports 2022-12-22 08:26:13 -07:00
devonh 044f1d1f37
Merge branch 'main' into mailbox 2022-12-22 15:22:41 +00:00
Till 5eed31fea3
Handle guest access [1/2?] (#2872)
Needs https://github.com/matrix-org/sytest/pull/1315, as otherwise the
membership events aren't persisted yet when hitting `/state` after
kicking guest users.

Makes the following tests pass:
```
Guest users denied access over federation if guest access prohibited
Guest users are kicked from guest_access rooms on revocation of guest_access
Guest users are kicked from guest_access rooms on revocation of guest_access over federation
```

Todo (in a follow up PR):
- Restrict access to CS API Endpoints as per
https://spec.matrix.org/v1.4/client-server-api/#client-behaviour-14

Co-authored-by: kegsay <kegan@matrix.org>
2022-12-22 13:05:59 +01:00
Devon Hudson 05769fa60e
Cleanup new file license headers 2022-12-21 10:42:35 -07:00
Devon Hudson 59a5dd2bfa
Add test coverage for config verification 2022-12-20 10:27:02 -07:00
Devon Hudson 30196ef083
Add tests for relayapi http endpoints 2022-12-16 16:13:57 -07:00
Devon Hudson 0875ba2073
Setup default relayapi config 2022-12-16 11:05:13 -07:00
Devon Hudson ed42b252ee
Shorten retry count until sending to relay 2022-12-15 14:28:18 -07:00
Devon Hudson ad53326ce8
Refactor all relay specific stuff into it's own component 2022-12-14 18:41:27 -07:00
Devon Hudson ee8a1c5680
Rename mailserver to relay server 2022-12-09 13:06:16 -07:00
Devon Hudson 98c7711b84
Merge branch 'main' into mailbox 2022-12-02 16:13:30 -07:00
Neil Alexander 934056f21f
Fix dendrite-demo-pinecone, /_dendrite namespace setup 2022-12-01 10:45:15 +00:00
Devon Hudson d4dde15113
Mark server as assumed offline with configurable value 2022-11-29 15:10:16 -07:00
Neil Alexander 1990c154e9
Update configuration 2022-11-29 11:11:08 +00:00
Neil Alexander 8299da5905
Fix registration for virtual hosting 2022-11-18 13:24:02 +00:00
Neil Alexander 607819f425
Fix /key/v2/server, add HTTP Host matching 2022-11-17 09:26:56 +00:00
Neil Alexander a916b041b1
Detect consumer being deleted in JetStreamConsumer 2022-11-16 10:28:22 +00:00
Neil Alexander 6650712a1c
Federation fixes for virtual hosting 2022-11-15 15:05:23 +00:00
Till d35a5642e8
Deny guest access on several endpoints (#2873)
Second part for guest access, this adds a `WithAllowGuests()` option to
`MakeAuthAPI`, allowing guests to access the specified endpoints.
Endpoints taken from the
[spec](https://spec.matrix.org/v1.4/client-server-api/#client-behaviour-14)
and by checking Synapse endpoints for `allow_guest=true`.
2022-11-11 10:52:08 +01:00
X. Ding 0782011f54
Add hcaptcha support besides Google ReCaptcha (#2834)
### Pull Request Checklist
This PR add support for hcaptcha.com as an alternative to Google
ReCaptcha. It also makes possible for user to customize ReCaptcha URL
when needed. (Such as use recaptcha.net instead of www.google.com)

This feature needs manual test cuz it involves 3rd party _captcha_.

Signed-off-by: `Simon Ding <dxl@plotbridge.com>`

Co-authored-by: dxl <dxl@plotbridge.com>
2022-10-28 11:25:01 +01:00
Neil Alexander f6dea712d2
Initial support for multiple server names (#2829)
This PR is the first step towards virtual hosting by laying the
groundwork for multiple server names being configured.
2022-10-26 12:59:19 +01:00
Neboer 2a4c7f45b3
Add support for config "auto_join_rooms" (#2823)
Add support for config "auto_join_rooms". Now new accounts can join the
rooms in config file automatically.

### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have justified why this PR doesn't need tests.
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)

Signed-off-by: `Rubin Poster <rubinposter@gmail.com>`
2022-10-26 11:04:53 +02:00
Till Faelligen 0843bd776e
Fix wrong config key 2022-10-24 07:10:50 +02:00
Neil Alexander d605d928bc
Allow specifying old signing keys with the public key and key ID only (#2770)
If the private key file is lost, it's often possible to retrieve the
public key from another server elsewhere, so we should make it possible
to configure it in that way.
2022-10-06 11:56:00 +01:00
Till 8c0c3441d8
Add RoomEventType nats.Header to avoid unneeded unmarshalling (#2765) 2022-10-05 12:12:42 +02:00
Neil Alexander 3da182212e
Track reasons why the process is in a degraded state 2022-10-04 13:02:41 +01:00
Neil Alexander 34993717fd
Update search docs 2022-09-27 17:10:47 +01:00
Till 87be32ca26
Fulltext implementation using Bleve (#2675)
Based on #2480

This actually indexes events based on their event type. They are removed
from the index if we receive a `m.room.redaction` event on the
`OutputRoomEvent` stream.
An admin endpoint is added to reindex all existing events.


Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-09-27 18:06:49 +02:00
Neil Alexander 6c67552bf9
Return M_UNRECOGNIZED for unknown CS API endpoints/actions (#2740)
Fixes #2739.
2022-09-27 15:50:22 +01:00
Till 249b32c4f3
Refactor notifications (#2688)
This PR changes the handling of notifications
- removes the `StreamEvent` and `ReadUpdate` stream
- listens on the `OutputRoomEvent` stream in the UserAPI to inform the
SyncAPI about unread notifications
- listens on the `OutputReceiptEvent` stream in the UserAPI to set
receipts/update notifications
- sets the `read_markers` directly from within the internal UserAPI

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-09-27 15:01:34 +02:00
Neil Alexander d8b19c857f
HTTP connection keepalives (#2730)
Beforehand we disabled HTTP keepalives to prevent ambient system
resources from being used by excess idle connections. Now that we've
fixed some bugs in the federation API and device list updater, this
situation is now much better and we don't open so many remote
connections anyway.

Keepalives allow us to not have to handshake TLS so often (which is
quite expensive) and reusing an idle connection is much faster than
having to open a new one. This can help with response times when talking
to remote federated servers.

This PR also adds a new option to disable keepalives if needed:

```
  # Disable HTTP keepalives, which also prevents connection reuse. Dendrite will typically
  # keep HTTP connections open to remote hosts for 5 minutes as they can be reused much
  # more quickly than opening new connections each time. Disabling keepalives will close
  # HTTP connections immediately after a successful request but may result in more CPU and
  # memory being used on TLS handshakes for each new connection instead.
  disable_http_keepalives: false
```
2022-09-20 17:17:44 +01:00
Neil Alexander bd39748b5c
Update dependencies (#2729)
This updates Dendrite dependencies.
2022-09-20 15:01:19 +01:00
Till Faelligen af9a204cc0 Only verify if fulltext indexing is enabled 2022-09-09 17:19:29 +02:00
Till d5876abbe9
Fulltext implementation incl. config (#2480)
This adds the main component of the fulltext search.
This PR doesn't do anything yet, besides creating an empty fulltextindex
folder if enabled. Indexing events is done in a separate PR.
2022-09-07 18:15:54 +02:00
Till 2cfcfddecc
Add a SigningKeyUpdate producer (#2697)
This adds a new stream for signing key updates, this should ensure we
don't lose any updates over federation.
2022-09-07 11:45:12 +02:00
Neil Alexander 51d229b025
Configuration tweaks (#2567)
This makes the following changes:

* The various `Defaults` functions are now responsible for setting sane defaults if `generate` is specified, rather than hiding them in `generate-config`
* Some configuration options have been marked as `omitempty` so that they don't appear in generated configs unnecessarily (monolith-specific vs. polylith-specific options)
* A new option `-polylith` has been added to `generate-config` to create a config that makes sense for polylith deployments (i.e. including the internal/external API listeners and per-component database sections)
* A new option `-normalise` has been added to `generate-config` to take an existing file and add any missing options and/or defaults
2022-09-01 14:15:41 +01:00
Neil Alexander ad6b902b84
Refactor appservices component (#2687)
This PR refactors the app services component. It makes the following changes:

* Each appservice now gets its own NATS JetStream consumer
* The appservice database is now removed entirely, since we just use JetStream as a data source instead
* The entire component is now much simpler and we deleted lots of lines of code 💅

The result is that it should be much lighter and hopefully much more performant.
2022-09-01 09:20:40 +01:00
Neil Alexander 175f65407a
Allow batching in JetStreamConsumer (#2686)
This allows us to receive more than one message from NATS at a time if we want.
2022-08-31 12:21:56 +01:00