From e48a08fef07e35a87d32591a44d2e416be3b12b1 Mon Sep 17 00:00:00 2001 From: Melroy van den Berg Date: Mon, 12 Jul 2021 11:13:17 +0200 Subject: [PATCH 1/4] Propose config better (#1758) Better explain where the config file are located and how to deal with the yml file. Co-authored-by: kegsay --- build/docker/README.md | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/build/docker/README.md b/build/docker/README.md index 6d3cd3dbd..19e4234c5 100644 --- a/build/docker/README.md +++ b/build/docker/README.md @@ -28,8 +28,8 @@ There are three sample `docker-compose` files: The `docker-compose` files refer to the `/etc/dendrite` volume as where the runtime config should come from. The mounted folder must contain: -- `dendrite.yaml` configuration file (based on the sample `dendrite-config.yaml` - in the `docker/config` folder in the [Dendrite repository](https://github.com/matrix-org/dendrite) +- `dendrite.yaml` configuration file (based on the [`dendrite-config.yaml`](https://raw.githubusercontent.com/matrix-org/dendrite/master/dendrite-config.yaml) + sample in the `build/docker/config` folder of this repository.) - `matrix_key.pem` server key, as generated using `cmd/generate-keys` - `server.crt` certificate file - `server.key` private key file for the above certificate @@ -50,8 +50,7 @@ The key files will now exist in your current working directory, and can be mount ## Starting Dendrite as a monolith deployment -Create your config based on the `dendrite.yaml` configuration file in the `docker/config` -folder in the [Dendrite repository](https://github.com/matrix-org/dendrite). +Create your config based on the [`dendrite-config.yaml`](https://raw.githubusercontent.com/matrix-org/dendrite/master/dendrite-config.yaml) configuration file in the `build/docker/config` folder of this repository. And rename the config file to `dendrite.yml` (and put it in your `config` directory). Once in place, start the PostgreSQL dependency: @@ -67,8 +66,7 @@ docker-compose -f docker-compose.monolith.yml up ## Starting Dendrite as a polylith deployment -Create your config based on the `dendrite.yaml` configuration file in the `docker/config` -folder in the [Dendrite repository](https://github.com/matrix-org/dendrite). +Create your config based on the [`dendrite-config.yaml`](https://raw.githubusercontent.com/matrix-org/dendrite/master/dendrite-config.yaml) configuration file in the `build/docker/config` folder of this repository. And rename the config file to `dendrite.yml` (and put it in your `config` directory). Once in place, start all the dependencies: @@ -84,10 +82,10 @@ docker-compose -f docker-compose.polylith.yml up ## Building the images -The `docker/images-build.sh` script will build the base image, followed by +The `build/docker/images-build.sh` script will build the base image, followed by all of the component images. -The `docker/images-push.sh` script will push them to Docker Hub (subject +The `build/docker/images-push.sh` script will push them to Docker Hub (subject to permissions). If you wish to build and push your own images, rename `matrixdotorg/dendrite` to From 0530302cd6e028f890227309bd221115da637325 Mon Sep 17 00:00:00 2001 From: Kegan Dougal Date: Mon, 12 Jul 2021 11:48:08 +0100 Subject: [PATCH 2/4] Add shared secret sytests to whitelist --- sytest-whitelist | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/sytest-whitelist b/sytest-whitelist index 8c4585716..55922f943 100644 --- a/sytest-whitelist +++ b/sytest-whitelist @@ -520,3 +520,7 @@ Inviting an AS-hosted user asks the AS server Can generate a openid access_token that can be exchanged for information about a user Invalid openid access tokens are rejected Requests to userinfo without access tokens are rejected +POST /_synapse/admin/v1/register with shared secret +POST /_synapse/admin/v1/register admin with shared secret +POST /_synapse/admin/v1/register with shared secret downcases capitals +POST /_synapse/admin/v1/register with shared secret disallows symbols From 89a16bdcd9897e9410b0a4b70b4049ebfd087318 Mon Sep 17 00:00:00 2001 From: Neil Alexander Date: Mon, 12 Jul 2021 11:48:29 +0100 Subject: [PATCH 3/4] Version 0.4.0 --- CHANGES.md | 41 +++++++++++++++++++++++++++++++++++++++++ internal/version.go | 4 ++-- 2 files changed, 43 insertions(+), 2 deletions(-) diff --git a/CHANGES.md b/CHANGES.md index c909c5715..a11326844 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -1,5 +1,46 @@ # Changelog +## Dendrite 0.4.0 (2021-07-12) + +### Features + +* All-new state storage in the roomserver, which dramatically reduces disk space utilisation + * State snapshots and blocks are now aggressively deduplicated and reused wherever possible, with state blocks being reduced by up to 15x and snapshot references being reduced up to 2x + * Dendrite will upgrade to the new state storage automatically on the first run after upgrade, although this may take some time depending on the size of the state storage +* Appservice support has been improved significantly, with many bridges now working correctly with Dendrite + * Events are now correctly sent to appservices based on room memberships + * Aliases and namespaces are now handled correctly, calling the appservice to query for aliases as needed + * Appservice user registrations are no longer being subject to incorrect validation checks +* Shared secret registration has now been implemented correctly +* The roomserver input API implements a new queuing system to reduce backpressure across rooms +* Checking if the local server is in a room has been optimised substantially, reducing CPU usage +* State resolution v2 has been optimised further by improving the power level checks, reducing CPU usage +* The federation API `/send` endpoint now deduplicates missing auth and prev events more aggressively to reduce memory usage +* The federation API `/send` endpoint now uses workers to reduce backpressure across rooms +* The bcrypt cost for password storage is now configurable with the `user_api.bcrypt_cost` option +* The federation API will now use significantly less memory when calling `/get_missing_events` +* MSC2946 Spaces endpoints have been updated to stable endpoint naming +* The media API can now be configured without a maximum file size +* A new `dendrite-upgrade-test` test has been added for verifying database schema upgrades across versions +* Added Prometheus metrics for roomserver backpressure, excessive device list updates and federation API event processing summaries +* Sentry support has been added for error reporting + +### Fixes + +* Removed the legacy `/v1` register endpoint. Dendrite only implements `/r0` of the CS API, and the legacy `/v1` endpoint had implementation errors which made it possible to bypass shared secret registration (thanks to Jakob Varmose Bentzen for reporting this) +* Attempting to register an account that already exists now returns a sensible error code rather than a HTTP 500 +* Dendrite will no longer attempt to `/make_join` with itself if listed in the request `server_names` +* `/sync` will no longer return immediately if there is nothing to sync, which happened particularly with new accounts, causing high CPU usage +* Malicious media uploads can no longer exhaust all available memory (contributed by [S7evinK](https://github.com/S7evinK)) +* Selecting one-time keys from the database has been optimised (contributed by [S7evinK](https://github.com/S7evinK)) +* The return code when trying to fetch missing account data has been fixed (contributed by [adamgreig](https://github.com/adamgreig)) +* Dendrite will no longer attempt to use `/make_leave` over federation when rejecting a local invite +* A panic has been fixed in `QueryMembershipsForRoom` +* A panic on duplicate membership events has been fixed in the federation sender +* A panic has been fixed in in `IsInterestedInRoomID` (contributed by [S7evinK](https://github.com/S7evinK)) +* A panic in the roomserver has been fixed when handling empty state sets +* A panic in the federation API has been fixed when handling cached events + ## Dendrite 0.3.11 (2021-03-02) ### Fixes diff --git a/internal/version.go b/internal/version.go index 0d3487799..37f0c30d3 100644 --- a/internal/version.go +++ b/internal/version.go @@ -16,8 +16,8 @@ var build string const ( VersionMajor = 0 - VersionMinor = 3 - VersionPatch = 11 + VersionMinor = 4 + VersionPatch = 0 VersionTag = "" // example: "rc1" ) From 48bdd79bdec8f9f13cbc07762060b31d1a2cc6cd Mon Sep 17 00:00:00 2001 From: Neil Alexander Date: Mon, 12 Jul 2021 11:54:11 +0100 Subject: [PATCH 4/4] Fix attribution in changelog --- CHANGES.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGES.md b/CHANGES.md index a11326844..27356b3cb 100644 --- a/CHANGES.md +++ b/CHANGES.md @@ -37,7 +37,7 @@ * Dendrite will no longer attempt to use `/make_leave` over federation when rejecting a local invite * A panic has been fixed in `QueryMembershipsForRoom` * A panic on duplicate membership events has been fixed in the federation sender -* A panic has been fixed in in `IsInterestedInRoomID` (contributed by [S7evinK](https://github.com/S7evinK)) +* A panic has been fixed in in `IsInterestedInRoomID` (contributed by [bodqhrohro](https://github.com/bodqhrohro)) * A panic in the roomserver has been fixed when handling empty state sets * A panic in the federation API has been fixed when handling cached events