resolve merge conflicts with main

.gitignore

@@ -41,6 +41,10 @@ _testmain.go
 *.test
 *.prof
 *.wasm
+*.aar
+*.jar
+*.framework
+*.xcframework
 
 # Generated keys
 *.pem
@@ -65,6 +69,9 @@ test/wasm/node_modules
 # Ignore complement folder when running locally
 complement/
 
+# Stuff from GitHub Pages
+docs/_site
+
 media_store/
 
 # Debug

CHANGES.md

@@ -1,5 +1,49 @@
 # Changelog
 
+## Dendrite 0.8.4 (2022-05-10)
+
+### Fixes
+
+* Fixes a regression introduced in the previous version where appservices, push and phone-home statistics would not work over plain HTTP
+* Adds missing indexes to the sync API output events table, which should significantly improve `/sync` performance and reduce database CPU usage
+* Building Dendrite with the `bimg` thumbnailer should now work again (contributed by [database64128](https://github.com/database64128))
+
+## Dendrite 0.8.3 (2022-05-09)
+
+### Features
+
+* Open registration is now harder to enable, which should reduce the chance that Dendrite servers will be used to conduct spam or abuse attacks
+  * Dendrite will only enable open registration if you pass the `--really-enable-open-registration` command line flag at startup
+  * If open registration is enabled but this command line flag is not passed, Dendrite will fail to start up
+* Dendrite now supports phone-home statistic reporting
+  * These statistics include things like the number of registered and active users, some configuration options and platform/environment details, to help us to understand how Dendrite is used
+  * This is not enabled by default — it must be enabled in the `global.report_stats` section of the config file
+* Monolith installations can now be configured with a single global database connection pool (in `global.database` in the config) rather than having to configure each component separately
+  * This also means that you no longer need to balance connection counts between different components, as they will share the same larger pool
+  * Specific components can override the global database settings by specifying their own `database` block
+  * To use only the global pool, you must configure `global.database` and then remove the `database` block from all of the component sections of the config file
+* A new admin API endpoint `/_dendrite/admin/evacuateRoom/{roomID}` has been added, allowing server admins to forcefully part all local users from a given room
+* The sync notifier now only loads members for the relevant rooms, which should reduce CPU usage and load on the database
+* A number of component interfaces have been refactored for cleanliness and developer ease
+* Event auth errors in the log should now be much more useful, including the reason for the event failures
+* The forward extremity calculation in the roomserver has been simplified
+* A new index has been added to the one-time keys table in the keyserver which should speed up key count lookups
+
+### Fixes
+
+* Dendrite will no longer process events for rooms where there are no local users joined, which should help to reduce CPU and RAM usage
+* A bug has been fixed in event auth when changing the user levels in `m.room.power_levels` events
+* Usernames should no longer be duplicated when no room name is set
+* Device display names should now be correctly propagated over federation
+* A panic when uploading cross-signing signatures has been fixed
+* Presence is now correctly limited in `/sync` based on the filters
+* The presence stream position returned by `/sync` will now be correct if no presence events were returned
+* The media `/config` endpoint will no longer return a maximum upload size field if it is configured to be unlimited in the Dendrite config
+* The server notices room will no longer produce "User is already joined to the room" errors
+* Consumer errors will no longer flood the logs during a graceful shutdown
+* Sync API and federation API consumers will no longer unnecessarily query added state events matching the one in the output event
+* The Sync API will no longer unnecessarily track invites for remote users
+
 ## Dendrite 0.8.2 (2022-04-27)
 
 ### Features

README.md

@@ -1,4 +1,5 @@
 # Dendrite
+
 [![Build status](https://github.com/matrix-org/dendrite/actions/workflows/dendrite.yml/badge.svg?event=push)](https://github.com/matrix-org/dendrite/actions/workflows/dendrite.yml) [![Dendrite](https://img.shields.io/matrix/dendrite:matrix.org.svg?label=%23dendrite%3Amatrix.org&logo=matrix&server_fqdn=matrix.org)](https://matrix.to/#/#dendrite:matrix.org) [![Dendrite Dev](https://img.shields.io/matrix/dendrite-dev:matrix.org.svg?label=%23dendrite-dev%3Amatrix.org&logo=matrix&server_fqdn=matrix.org)](https://matrix.to/#/#dendrite-dev:matrix.org)
 
 Dendrite is a second-generation Matrix homeserver written in Go.
@@ -52,7 +53,7 @@ The [Federation Tester](https://federationtester.matrix.org) can be used to veri
 
 ## Get started
 
-If you wish to build a fully-federating Dendrite instance, see [INSTALL.md](docs/INSTALL.md). For running in Docker, see [build/docker](build/docker).
+If you wish to build a fully-federating Dendrite instance, see [the Installation documentation](docs/installation). For running in Docker, see [build/docker](build/docker).
 
 The following instructions are enough to get Dendrite started as a non-federating test deployment using self-signed certificates and SQLite databases:
 
@@ -70,7 +71,7 @@ $ ./bin/generate-keys --tls-cert server.crt --tls-key server.key
 
 # Copy and modify the config file - you'll need to set a server name and paths to the keys
 # at the very least, along with setting up the database connection strings.
-$ cp dendrite-config.yaml dendrite.yaml
+$ cp dendrite-sample.monolith.yaml dendrite.yaml
 
 # Build and run the server:
 $ ./bin/dendrite-monolith-server --tls-cert server.crt --tls-key server.key --config dendrite.yaml

appservice/api/query.go

@@ -26,6 +26,23 @@ import (
 	"github.com/matrix-org/gomatrixserverlib"
 )
 
+// AppServiceInternalAPI is used to query user and room alias data from application
+// services
+type AppServiceInternalAPI interface {
+	// Check whether a room alias exists within any application service namespaces
+	RoomAliasExists(
+		ctx context.Context,
+		req *RoomAliasExistsRequest,
+		resp *RoomAliasExistsResponse,
+	) error
+	// Check whether a user ID exists within any application service namespaces
+	UserIDExists(
+		ctx context.Context,
+		req *UserIDExistsRequest,
+		resp *UserIDExistsResponse,
+	) error
+}
+
 // RoomAliasExistsRequest is a request to an application service
 // about whether a room alias exists
 type RoomAliasExistsRequest struct {
@@ -60,31 +77,14 @@ type UserIDExistsResponse struct {
 	UserIDExists bool `json:"exists"`
 }
 
-// AppServiceQueryAPI is used to query user and room alias data from application
-// services
-type AppServiceQueryAPI interface {
-	// Check whether a room alias exists within any application service namespaces
-	RoomAliasExists(
-		ctx context.Context,
-		req *RoomAliasExistsRequest,
-		resp *RoomAliasExistsResponse,
-	) error
-	// Check whether a user ID exists within any application service namespaces
-	UserIDExists(
-		ctx context.Context,
-		req *UserIDExistsRequest,
-		resp *UserIDExistsResponse,
-	) error
-}
-
 // RetrieveUserProfile is a wrapper that queries both the local database and
 // application services for a given user's profile
 // TODO: Remove this, it's called from federationapi and clientapi but is a pure function
 func RetrieveUserProfile(
 	ctx context.Context,
 	userID string,
-	asAPI AppServiceQueryAPI,
-	profileAPI userapi.UserProfileAPI,
+	asAPI AppServiceInternalAPI,
+	profileAPI userapi.ClientUserAPI,
 ) (*authtypes.Profile, error) {
 	localpart, _, err := gomatrixserverlib.SplitID('@', userID)
 	if err != nil {

appservice/appservice.go

@@ -34,12 +34,11 @@ import (
 	roomserverAPI "github.com/matrix-org/dendrite/roomserver/api"
 	"github.com/matrix-org/dendrite/setup/base"
 	"github.com/matrix-org/dendrite/setup/config"
-	"github.com/matrix-org/dendrite/setup/jetstream"
 	userapi "github.com/matrix-org/dendrite/userapi/api"
 )
 
 // AddInternalRoutes registers HTTP handlers for internal API calls
-func AddInternalRoutes(router *mux.Router, queryAPI appserviceAPI.AppServiceQueryAPI) {
+func AddInternalRoutes(router *mux.Router, queryAPI appserviceAPI.AppServiceInternalAPI) {
 	inthttp.AddRoutes(queryAPI, router)
 }
 
@@ -49,7 +48,7 @@ func NewInternalAPI(
 	base *base.BaseDendrite,
 	userAPI userapi.UserInternalAPI,
 	rsAPI roomserverAPI.RoomserverInternalAPI,
-) appserviceAPI.AppServiceQueryAPI {
+) appserviceAPI.AppServiceInternalAPI {
 	client := &http.Client{
 		Timeout: time.Second * 30,
 		Transport: &http.Transport{
@@ -57,12 +56,13 @@ func NewInternalAPI(
 			TLSClientConfig: &tls.Config{
 				InsecureSkipVerify: base.Cfg.AppServiceAPI.DisableTLSValidation,
 			},
+			Proxy: http.ProxyFromEnvironment,
 		},
 	}
-	js, _ := jetstream.Prepare(base.ProcessContext, &base.Cfg.Global.JetStream)
+	js, _ := base.NATS.Prepare(base.ProcessContext, &base.Cfg.Global.JetStream)
 
 	// Create a connection to the appservice postgres DB
-	appserviceDB, err := storage.NewDatabase(&base.Cfg.AppServiceAPI.Database)
+	appserviceDB, err := storage.NewDatabase(base, &base.Cfg.AppServiceAPI.Database)
 	if err != nil {
 		logrus.WithError(err).Panicf("failed to connect to appservice db")
 	}
@@ -117,7 +117,7 @@ func NewInternalAPI(
 // `sender_localpart` field of each application service if it doesn't
 // exist already
 func generateAppServiceAccount(
-	userAPI userapi.UserInternalAPI,
+	userAPI userapi.AppserviceUserAPI,
 	as config.ApplicationService,
 ) error {
 	var accRes userapi.PerformAccountCreationResponse

appservice/consumers/roomserver.go

@@ -37,7 +37,7 @@ type OutputRoomEventConsumer struct {
 	durable      string
 	topic        string
 	asDB         storage.Database
-	rsAPI        api.RoomserverInternalAPI
+	rsAPI        api.AppserviceRoomserverAPI
 	serverName   string
 	workerStates []types.ApplicationServiceWorkerState
 }
@@ -49,7 +49,7 @@ func NewOutputRoomEventConsumer(
 	cfg *config.Dendrite,
 	js nats.JetStreamContext,
 	appserviceDB storage.Database,
-	rsAPI api.RoomserverInternalAPI,
+	rsAPI api.AppserviceRoomserverAPI,
 	workerStates []types.ApplicationServiceWorkerState,
 ) *OutputRoomEventConsumer {
 	return &OutputRoomEventConsumer{

appservice/inthttp/client.go

@@ -29,7 +29,7 @@ type httpAppServiceQueryAPI struct {
 func NewAppserviceClient(
 	appserviceURL string,
 	httpClient *http.Client,
-) (api.AppServiceQueryAPI, error) {
+) (api.AppServiceInternalAPI, error) {
 	if httpClient == nil {
 		return nil, errors.New("NewRoomserverAliasAPIHTTP: httpClient is <nil>")
 	}

appservice/inthttp/server.go

@@ -11,7 +11,7 @@ import (
 )
 
 // AddRoutes adds the AppServiceQueryAPI handlers to the http.ServeMux.
-func AddRoutes(a api.AppServiceQueryAPI, internalAPIMux *mux.Router) {
+func AddRoutes(a api.AppServiceInternalAPI, internalAPIMux *mux.Router) {
 	internalAPIMux.Handle(
 		AppServiceRoomAliasExistsPath,
 		httputil.MakeInternalAPI("appserviceRoomAliasExists", func(req *http.Request) util.JSONResponse {

appservice/storage/postgres/storage.go

@@ -22,6 +22,7 @@ import (
 	// Import postgres database driver
 	_ "github.com/lib/pq"
 	"github.com/matrix-org/dendrite/internal/sqlutil"
+	"github.com/matrix-org/dendrite/setup/base"
 	"github.com/matrix-org/dendrite/setup/config"
 	"github.com/matrix-org/gomatrixserverlib"
 )
@@ -35,13 +36,12 @@ type Database struct {
 }
 
 // NewDatabase opens a new database
-func NewDatabase(dbProperties *config.DatabaseOptions) (*Database, error) {
+func NewDatabase(base *base.BaseDendrite, dbProperties *config.DatabaseOptions) (*Database, error) {
 	var result Database
 	var err error
-	if result.db, err = sqlutil.Open(dbProperties); err != nil {
+	if result.db, result.writer, err = base.DatabaseConnection(dbProperties, sqlutil.NewDummyWriter()); err != nil {
 		return nil, err
 	}
-	result.writer = sqlutil.NewDummyWriter()
 	if err = result.prepare(); err != nil {
 		return nil, err
 	}

appservice/storage/sqlite3/storage.go

@@ -21,6 +21,7 @@ import (
 
 	// Import SQLite database driver
 	"github.com/matrix-org/dendrite/internal/sqlutil"
+	"github.com/matrix-org/dendrite/setup/base"
 	"github.com/matrix-org/dendrite/setup/config"
 	"github.com/matrix-org/gomatrixserverlib"
 )
@@ -34,13 +35,12 @@ type Database struct {
 }
 
 // NewDatabase opens a new database
-func NewDatabase(dbProperties *config.DatabaseOptions) (*Database, error) {
+func NewDatabase(base *base.BaseDendrite, dbProperties *config.DatabaseOptions) (*Database, error) {
 	var result Database
 	var err error
-	if result.db, err = sqlutil.Open(dbProperties); err != nil {
+	if result.db, result.writer, err = base.DatabaseConnection(dbProperties, sqlutil.NewExclusiveWriter()); err != nil {
 		return nil, err
 	}
-	result.writer = sqlutil.NewExclusiveWriter()
 	if err = result.prepare(); err != nil {
 		return nil, err
 	}

appservice/storage/storage.go

@@ -22,17 +22,18 @@ import (
 
 	"github.com/matrix-org/dendrite/appservice/storage/postgres"
 	"github.com/matrix-org/dendrite/appservice/storage/sqlite3"
+	"github.com/matrix-org/dendrite/setup/base"
 	"github.com/matrix-org/dendrite/setup/config"
 )
 
 // NewDatabase opens a new Postgres or Sqlite database (based on dataSourceName scheme)
 // and sets DB connection parameters
-func NewDatabase(dbProperties *config.DatabaseOptions) (Database, error) {
+func NewDatabase(base *base.BaseDendrite, dbProperties *config.DatabaseOptions) (Database, error) {
 	switch {
 	case dbProperties.ConnectionString.IsSQLite():
-		return sqlite3.NewDatabase(dbProperties)
+		return sqlite3.NewDatabase(base, dbProperties)
 	case dbProperties.ConnectionString.IsPostgres():
-		return postgres.NewDatabase(dbProperties)
+		return postgres.NewDatabase(base, dbProperties)
 	default:
 		return nil, fmt.Errorf("unexpected database type")
 	}

appservice/storage/storage_wasm.go

@@ -18,13 +18,14 @@ import (
 	"fmt"
 
 	"github.com/matrix-org/dendrite/appservice/storage/sqlite3"
+	"github.com/matrix-org/dendrite/setup/base"
 	"github.com/matrix-org/dendrite/setup/config"
 )
 
-func NewDatabase(dbProperties *config.DatabaseOptions) (Database, error) {
+func NewDatabase(base *base.BaseDendrite, dbProperties *config.DatabaseOptions) (Database, error) {
 	switch {
 	case dbProperties.ConnectionString.IsSQLite():
-		return sqlite3.NewDatabase(dbProperties)
+		return sqlite3.NewDatabase(base, dbProperties)
 	case dbProperties.ConnectionString.IsPostgres():
 		return nil, fmt.Errorf("can't use Postgres implementation")
 	default:

build/docker/README.md

@@ -27,8 +27,7 @@ There are three sample `docker-compose` files:
 The `docker-compose` files refer to the `/etc/dendrite` volume as where the
 runtime config should come from. The mounted folder must contain:
 
-- `dendrite.yaml` configuration file (from the [Docker config folder](https://github.com/matrix-org/dendrite/tree/master/build/docker/config)
-   sample in the `build/docker/config` folder of this repository.)
+- `dendrite.yaml` configuration file (based on one of the sample config files)
 - `matrix_key.pem` server key, as generated using `cmd/generate-keys`
 - `server.crt` certificate file
 - `server.key` private key file for the above certificate
@@ -49,7 +48,7 @@ The key files will now exist in your current working directory, and can be mount
 
 ## Starting Dendrite as a monolith deployment
 
-Create your config based on the [`dendrite.yaml`](https://github.com/matrix-org/dendrite/tree/master/build/docker/config) configuration file in the `build/docker/config` folder of this repository.
+Create your config based on the [`dendrite-sample.monolith.yaml`](https://github.com/matrix-org/dendrite/blob/main/dendrite-sample.monolith.yaml) sample configuration file.
 
 Then start the deployment:
 
@@ -59,7 +58,7 @@ docker-compose -f docker-compose.monolith.yml up
 
 ## Starting Dendrite as a polylith deployment
 
-Create your config based on the [`dendrite-config.yaml`](https://github.com/matrix-org/dendrite/tree/master/build/docker/config) configuration file in the `build/docker/config` folder of this repository.
+Create your config based on the [`dendrite-sample.polylith.yaml`](https://github.com/matrix-org/dendrite/blob/main/dendrite-sample.polylith.yaml) sample configuration file.
 
 Then start the deployment:
 

build/docker/config/dendrite.yaml

@@ -1,348 +0,0 @@
-# This is the Dendrite configuration file.
-#
-# The configuration is split up into sections - each Dendrite component has a
-# configuration section, in addition to the "global" section which applies to
-# all components.
-#
-# At a minimum, to get started, you will need to update the settings in the
-# "global" section for your deployment, and you will need to check that the
-# database "connection_string" line in each component section is correct.
-#
-# Each component with a "database" section can accept the following formats
-# for "connection_string":
-#   SQLite:     file:filename.db
-#               file:///path/to/filename.db
-#   PostgreSQL: postgresql://user:pass@hostname/database?params=...
-#
-# SQLite is embedded into Dendrite and therefore no further prerequisites are
-# needed for the database when using SQLite mode. However, performance with
-# PostgreSQL is significantly better and recommended for multi-user deployments.
-# SQLite is typically around 20-30% slower than PostgreSQL when tested with a
-# small number of users and likely will perform worse still with a higher volume
-# of users.
-#
-# The "max_open_conns" and "max_idle_conns" settings configure the maximum
-# number of open/idle database connections. The value 0 will use the database
-# engine default, and a negative value will use unlimited connections. The
-# "conn_max_lifetime" option controls the maximum length of time a database
-# connection can be idle in seconds - a negative value is unlimited.
-
-# The version of the configuration file.
-version: 2
-
-# Global Matrix configuration. This configuration applies to all components.
-global:
-  # The domain name of this homeserver.
-  server_name: example.com
-
-  # The path to the signing private key file, used to sign requests and events.
-  private_key: matrix_key.pem
-
-  # The paths and expiry timestamps (as a UNIX timestamp in millisecond precision)
-  # to old signing private keys that were formerly in use on this domain. These
-  # keys will not be used for federation request or event signing, but will be
-  # provided to any other homeserver that asks when trying to verify old events.
-  # old_private_keys:
-  # - private_key: old_matrix_key.pem
-  #   expired_at: 1601024554498
-
-  # How long a remote server can cache our server signing key before requesting it
-  # again. Increasing this number will reduce the number of requests made by other
-  # servers for our key but increases the period that a compromised key will be
-  # considered valid by other homeservers.
-  key_validity_period: 168h0m0s
-
-  # The server name to delegate server-server communications to, with optional port
-  # e.g. localhost:443
-  well_known_server_name: ""
-
-  # Lists of domains that the server will trust as identity servers to verify third
-  # party identifiers such as phone numbers and email addresses.
-  trusted_third_party_id_servers:
-    - matrix.org
-    - vector.im
-
-  # Disables federation. Dendrite will not be able to make any outbound HTTP requests
-  # to other servers and the federation API will not be exposed.
-  disable_federation: false
-
-  # Configures the handling of presence events.
-  presence:
-    # Whether inbound presence events are allowed, e.g. receiving presence events from other servers
-    enable_inbound: false
-    # Whether outbound presence events are allowed, e.g. sending presence events to other servers
-    enable_outbound: false
-
-  # Configuration for NATS JetStream
-  jetstream:
-    # A list of NATS Server addresses to connect to. If none are specified, an
-    # internal NATS server will be started automatically when running Dendrite
-    # in monolith mode. It is required to specify the address of at least one
-    # NATS Server node if running in polylith mode.
-    addresses:
-      - jetstream:4222
-
-    # Keep all NATS streams in memory, rather than persisting it to the storage
-    # path below. This option is present primarily for integration testing and
-    # should not be used on a real world Dendrite deployment.
-    in_memory: false
-
-    # Persistent directory to store JetStream streams in. This directory
-    # should be preserved across Dendrite restarts.
-    storage_path: ./
-
-    # The prefix to use for stream names for this homeserver - really only
-    # useful if running more than one Dendrite on the same NATS deployment.
-    topic_prefix: Dendrite
-
-  # Configuration for Prometheus metric collection.
-  metrics:
-    # Whether or not Prometheus metrics are enabled.
-    enabled: false
-
-    # HTTP basic authentication to protect access to monitoring.
-    basic_auth:
-      username: metrics
-      password: metrics
-
-  # DNS cache options. The DNS cache may reduce the load on DNS servers
-  # if there is no local caching resolver available for use.
-  dns_cache:
-    # Whether or not the DNS cache is enabled.
-    enabled: false
-
-    # Maximum number of entries to hold in the DNS cache, and
-    # for how long those items should be considered valid in seconds.
-    cache_size: 256
-    cache_lifetime: 300
-
-# Configuration for the Appservice API.
-app_service_api:
-  internal_api:
-    listen: http://0.0.0.0:7777
-    connect: http://appservice_api:7777
-  database:
-    connection_string: postgresql://dendrite:itsasecret@postgres/dendrite_appservice?sslmode=disable
-    max_open_conns: 10
-    max_idle_conns: 2
-    conn_max_lifetime: -1
-
-  # Appservice configuration files to load into this homeserver.
-  config_files: []
-
-# Configuration for the Client API.
-client_api:
-  internal_api:
-    listen: http://0.0.0.0:7771
-    connect: http://client_api:7771
-  external_api:
-    listen: http://0.0.0.0:8071
-
-  # Prevents new users from being able to register on this homeserver, except when
-  # using the registration shared secret below.
-  registration_disabled: true
-
-  # If set, allows registration by anyone who knows the shared secret, regardless of
-  # whether registration is otherwise disabled.
-  registration_shared_secret: ""
-
-  # Whether to require reCAPTCHA for registration.
-  enable_registration_captcha: false
-
-  # Settings for ReCAPTCHA.
-  recaptcha_public_key: ""
-  recaptcha_private_key: ""
-  recaptcha_bypass_secret: ""
-  recaptcha_siteverify_api: ""
-
-  # TURN server information that this homeserver should send to clients.
-  turn:
-    turn_user_lifetime: ""
-    turn_uris: []
-    turn_shared_secret: ""
-    turn_username: ""
-    turn_password: ""
-
-  # Settings for rate-limited endpoints. Rate limiting will kick in after the
-  # threshold number of "slots" have been taken by requests from a specific
-  # host. Each "slot" will be released after the cooloff time in milliseconds.
-  rate_limiting:
-    enabled: true
-    threshold: 5
-    cooloff_ms: 500
-
-# Configuration for the Federation API.
-federation_api:
-  internal_api:
-    listen: http://0.0.0.0:7772
-    connect: http://federation_api:7772
-  external_api:
-    listen: http://0.0.0.0:8072
-  database:
-    connection_string: postgresql://dendrite:itsasecret@postgres/dendrite_federationapi?sslmode=disable
-    max_open_conns: 10
-    max_idle_conns: 2
-    conn_max_lifetime: -1
-
-  # How many times we will try to resend a failed transaction to a specific server. The
-  # backoff is 2**x seconds, so 1 = 2 seconds, 2 = 4 seconds, 3 = 8 seconds etc.
-  send_max_retries: 16
-
-  # Disable the validation of TLS certificates of remote federated homeservers. Do not
-  # enable this option in production as it presents a security risk!
-  disable_tls_validation: false
-
-  # Use the following proxy server for outbound federation traffic.
-  proxy_outbound:
-    enabled: false
-    protocol: http
-    host: localhost
-    port: 8080
-
-  # Perspective keyservers to use as a backup when direct key fetches fail. This may
-  # be required to satisfy key requests for servers that are no longer online when
-  # joining some rooms.
-  key_perspectives:
-    - server_name: matrix.org
-      keys:
-        - key_id: ed25519:auto
-          public_key: Noi6WqcDj0QmPxCNQqgezwTlBKrfqehY1u2FyWP9uYw
-        - key_id: ed25519:a_RXGa
-          public_key: l8Hft5qXKn1vfHrg3p4+W8gELQVo8N13JkluMfmn2sQ
-
-  # This option will control whether Dendrite will prefer to look up keys directly
-  # or whether it should try perspective servers first, using direct fetches as a
-  # last resort.
-  prefer_direct_fetch: false
-
-# Configuration for the Key Server (for end-to-end encryption).
-key_server:
-  internal_api:
-    listen: http://0.0.0.0:7779
-    connect: http://key_server:7779
-  database:
-    connection_string: postgresql://dendrite:itsasecret@postgres/dendrite_keyserver?sslmode=disable
-    max_open_conns: 10
-    max_idle_conns: 2
-    conn_max_lifetime: -1
-
-# Configuration for the Media API.
-media_api:
-  internal_api:
-    listen: http://0.0.0.0:7774
-    connect: http://media_api:7774
-  external_api:
-    listen: http://0.0.0.0:8074
-  database:
-    connection_string: postgresql://dendrite:itsasecret@postgres/dendrite_mediaapi?sslmode=disable
-    max_open_conns: 10
-    max_idle_conns: 2
-    conn_max_lifetime: -1
-
-  # Storage path for uploaded media. May be relative or absolute.
-  base_path: /var/dendrite/media
-
-  # The maximum allowed file size (in bytes) for media uploads to this homeserver
-  # (0 = unlimited).
-  max_file_size_bytes: 10485760
-
-  # Whether to dynamically generate thumbnails if needed.
-  dynamic_thumbnails: false
-
-  # The maximum number of simultaneous thumbnail generators to run.
-  max_thumbnail_generators: 10
-
-  # A list of thumbnail sizes to be generated for media content.
-  thumbnail_sizes:
-    - width: 32
-      height: 32
-      method: crop
-    - width: 96
-      height: 96
-      method: crop
-    - width: 640
-      height: 480
-      method: scale
-
-# Configuration for experimental MSC's
-mscs:
-  # A list of enabled MSC's
-  # Currently valid values are:
-  # - msc2836    (Threading, see https://github.com/matrix-org/matrix-doc/pull/2836)
-  # - msc2946    (Spaces Summary, see https://github.com/matrix-org/matrix-doc/pull/2946)
-  mscs: []
-  database:
-    connection_string: postgresql://dendrite:itsasecret@postgres/dendrite_mscs?sslmode=disable
-    max_open_conns: 5
-    max_idle_conns: 2
-    conn_max_lifetime: -1
-
-# Configuration for the Room Server.
-room_server:
-  internal_api:
-    listen: http://0.0.0.0:7770
-    connect: http://room_server:7770
-  database:
-    connection_string: postgresql://dendrite:itsasecret@postgres/dendrite_roomserver?sslmode=disable
-    max_open_conns: 10
-    max_idle_conns: 2
-    conn_max_lifetime: -1
-
-# Configuration for the Sync API.
-sync_api:
-  internal_api:
-    listen: http://0.0.0.0:7773
-    connect: http://sync_api:7773
-  external_api:
-    listen: http://0.0.0.0:8073
-  database:
-    connection_string: postgresql://dendrite:itsasecret@postgres/dendrite_syncapi?sslmode=disable
-    max_open_conns: 10
-    max_idle_conns: 2
-    conn_max_lifetime: -1
-
-# Configuration for the User API.
-user_api:
-  internal_api:
-    listen: http://0.0.0.0:7781
-    connect: http://user_api:7781
-  account_database:
-    connection_string: postgresql://dendrite:itsasecret@postgres/dendrite_userapi_accounts?sslmode=disable
-    max_open_conns: 10
-    max_idle_conns: 2
-    conn_max_lifetime: -1
-
-# Configuration for the Push Server API.
-push_server:
-  internal_api:
-    listen: http://localhost:7782
-    connect: http://localhost:7782
-  database:
-    connection_string: postgresql://dendrite:itsasecret@postgres/dendrite_pushserver?sslmode=disable
-    max_open_conns: 10
-    max_idle_conns: 2
-    conn_max_lifetime: -1
-
-# Configuration for Opentracing.
-# See https://github.com/matrix-org/dendrite/tree/master/docs/tracing for information on
-# how this works and how to set it up.
-tracing:
-  enabled: false
-  jaeger:
-    serviceName: ""
-    disabled: false
-    rpc_metrics: false
-    tags: []
-    sampler: null
-    reporter: null
-    headers: null
-    baggage_restrictions: null
-    throttler: null
-
-# Logging configuration, in addition to the standard logging that is sent to
-# stdout by Dendrite.
-logging:
-  - type: file
-    level: info
-    params:
-      path: /var/log/dendrite

build/gobind-pinecone/monolith.go

@@ -268,7 +268,6 @@ func (m *DendriteMonolith) Start() {
 	base := base.NewBaseDendrite(cfg, "Monolith")
 	defer base.Close() // nolint: errcheck
 
-	accountDB := base.CreateAccountsDB()
 	federation := conn.CreateFederationClient(base, m.PineconeQUIC)
 
 	serverKeyAPI := &signing.YggdrasilKeys{}
@@ -281,7 +280,7 @@ func (m *DendriteMonolith) Start() {
 	)
 
 	keyAPI := keyserver.NewInternalAPI(base, &base.Cfg.KeyServer, fsAPI)
-	m.userAPI = userapi.NewInternalAPI(base, accountDB, &cfg.UserAPI, cfg.Derived.ApplicationServices, keyAPI, rsAPI, base.PushGatewayHTTPClient())
+	m.userAPI = userapi.NewInternalAPI(base, &cfg.UserAPI, cfg.Derived.ApplicationServices, keyAPI, rsAPI, base.PushGatewayHTTPClient())
 	keyAPI.SetUserAPI(m.userAPI)
 
 	asAPI := appservice.NewInternalAPI(base, m.userAPI, rsAPI)
@@ -295,7 +294,6 @@ func (m *DendriteMonolith) Start() {
 
 	monolith := setup.Monolith{
 		Config:    base.Cfg,
-		AccountDB: accountDB,
 		Client:    conn.CreateClient(base, m.PineconeQUIC),
 		FedClient: federation,
 		KeyRing:   keyRing,
@@ -308,16 +306,7 @@ func (m *DendriteMonolith) Start() {
 		ExtPublicRoomsProvider:   roomProvider,
 		ExtUserDirectoryProvider: userProvider,
 	}
-	monolith.AddAllPublicRoutes(
-		base.ProcessContext,
-		base.PublicClientAPIMux,
-		base.PublicFederationAPIMux,
-		base.PublicKeyAPIMux,
-		base.PublicWellKnownAPIMux,
-		base.PublicMediaAPIMux,
-		base.SynapseAdminMux,
-		base.DendriteAdminMux,
-	)
+	monolith.AddAllPublicRoutes(base)
 
 	httpRouter := mux.NewRouter().SkipClean(true).UseEncodedPath()
 	httpRouter.PathPrefix(httputil.InternalPathPrefix).Handler(base.InternalAPIMux)

build/gobind-yggdrasil/monolith.go

@@ -107,7 +107,6 @@ func (m *DendriteMonolith) Start() {
 	m.processContext = base.ProcessContext
 	defer base.Close() // nolint: errcheck
 
-	accountDB := base.CreateAccountsDB()
 	federation := ygg.CreateFederationClient(base)
 
 	serverKeyAPI := &signing.YggdrasilKeys{}
@@ -120,7 +119,7 @@ func (m *DendriteMonolith) Start() {
 	)
 
 	keyAPI := keyserver.NewInternalAPI(base, &base.Cfg.KeyServer, federation)
-	userAPI := userapi.NewInternalAPI(base, accountDB, &cfg.UserAPI, cfg.Derived.ApplicationServices, keyAPI, rsAPI, base.PushGatewayHTTPClient())
+	userAPI := userapi.NewInternalAPI(base, &cfg.UserAPI, cfg.Derived.ApplicationServices, keyAPI, rsAPI, base.PushGatewayHTTPClient())
 	keyAPI.SetUserAPI(userAPI)
 
 	asAPI := appservice.NewInternalAPI(base, userAPI, rsAPI)
@@ -132,7 +131,6 @@ func (m *DendriteMonolith) Start() {
 
 	monolith := setup.Monolith{
 		Config:    base.Cfg,
-		AccountDB: accountDB,
 		Client:    ygg.CreateClient(base),
 		FedClient: federation,
 		KeyRing:   keyRing,
@@ -146,16 +144,7 @@ func (m *DendriteMonolith) Start() {
 			ygg, fsAPI, federation,
 		),
 	}
-	monolith.AddAllPublicRoutes(
-		base.ProcessContext,
-		base.PublicClientAPIMux,
-		base.PublicFederationAPIMux,
-		base.PublicKeyAPIMux,
-		base.PublicWellKnownAPIMux,
-		base.PublicMediaAPIMux,
-		base.SynapseAdminMux,
-		base.DendriteAdminMux,
-	)
+	monolith.AddAllPublicRoutes(base)
 
 	httpRouter := mux.NewRouter()
 	httpRouter.PathPrefix(httputil.InternalPathPrefix).Handler(base.InternalAPIMux)

clientapi/auth/auth.go

@@ -51,7 +51,7 @@ type AccountDatabase interface {
 // Note: For an AS user, AS dummy device is returned.
 // On failure returns an JSON error response which can be sent to the client.
 func VerifyUserFromRequest(
-	req *http.Request, userAPI api.UserInternalAPI,
+	req *http.Request, userAPI api.QueryAcccessTokenAPI,
 ) (*api.Device, *util.JSONResponse) {
 	// Try to find the Application Service user
 	token, err := ExtractAccessToken(req)

clientapi/auth/login.go

@@ -36,13 +36,12 @@ import (
 func LoginFromJSONReader(
 	ctx context.Context,
 	r io.Reader,
-	useraccountAPI uapi.UserAccountAPI,
+	useraccountAPI uapi.UserLoginAPI,
 	userAPI UserInternalAPIForLogin,
-	userRegisterAPI uapi.UserRegisterAPI,
+	clientUserAPI uapi.ClientUserAPI,
 	userInteractiveAuth *UserInteractive,
 	cfg *config.ClientAPI,
 ) (*Login, LoginCleanupFunc, *util.JSONResponse) {
-
 	reqBytes, err := ioutil.ReadAll(r)
 	if err != nil {
 		err := &util.JSONResponse{
@@ -77,7 +76,7 @@ func LoginFromJSONReader(
 		}
 	case header.Type == authtypes.LoginTypePublicKey && cfg.PublicKeyAuthentication.Enabled():
 		typ = &LoginTypePublicKey{
-			UserAPI:         userRegisterAPI,
+			UserAPI:         clientUserAPI,
 			UserInteractive: userInteractiveAuth,
 			Config:          cfg,
 		}

clientapi/auth/login_publickey.go

@@ -38,7 +38,7 @@ type LoginPublicKeyHandler interface {
 
 // LoginTypePublicKey implements https://matrix.org/docs/spec/client_server/..... (to be spec'ed)
 type LoginTypePublicKey struct {
-	UserAPI         userapi.UserRegisterAPI
+	UserAPI         userapi.ClientUserAPI
 	UserInteractive *UserInteractive
 	Config          *config.ClientAPI
 }

clientapi/auth/login_publickey_ethereum.go

@@ -42,7 +42,7 @@ type LoginPublicKeyEthereum struct {
 	HashFields    publicKeyEthereumHashFields `json:"hashFields"`
 	HashFieldsRaw string                      // Raw base64 encoded string of MessageFields for hash verification
 
-	userAPI userapi.UserRegisterAPI
+	userAPI userapi.ClientUserAPI
 	config  *config.ClientAPI
 }
 
@@ -63,7 +63,7 @@ type publicKeyEthereumRequiredFields struct {
 
 func CreatePublicKeyEthereumHandler(
 	reqBytes []byte,
-	userAPI userapi.UserRegisterAPI,
+	userAPI userapi.ClientUserAPI,
 	config *config.ClientAPI,
 ) (*LoginPublicKeyEthereum, *jsonerror.MatrixError) {
 	var pk LoginPublicKeyEthereum

clientapi/auth/login_test.go

@@ -176,8 +176,8 @@ func TestBadLoginFromJSONReader(t *testing.T) {
 
 type fakeUserInternalAPI struct {
 	UserInternalAPIForLogin
-	uapi.UserAccountAPI
-	uapi.UserRegisterAPI
+	uapi.UserLoginAPI
+	uapi.ClientUserAPI
 	DeletedTokens []string
 }
 
@@ -196,6 +196,10 @@ func (ua *fakeUserInternalAPI) PerformLoginTokenDeletion(ctx context.Context, re
 	return nil
 }
 
+func (ua *fakeUserInternalAPI) PerformLoginTokenCreation(ctx context.Context, req *uapi.PerformLoginTokenCreationRequest, res *uapi.PerformLoginTokenCreationResponse) error {
+	return nil
+}
+
 func (*fakeUserInternalAPI) QueryLoginToken(ctx context.Context, req *uapi.QueryLoginTokenRequest, res *uapi.QueryLoginTokenResponse) error {
 	if req.Token == "invalidtoken" {
 		return nil

clientapi/auth/user_interactive.go

@@ -112,8 +112,8 @@ type UserInteractive struct {
 }
 
 func NewUserInteractive(
-	userAccountAPI api.UserAccountAPI,
-	userRegisterAPI api.UserRegisterAPI,
+	userAccountAPI api.UserLoginAPI,
+	clientUserAPI api.ClientUserAPI,
 	cfg *config.ClientAPI,
 ) *UserInteractive {
 	userInteractive := UserInteractive{
@@ -134,7 +134,7 @@ func NewUserInteractive(
 
 	if cfg.PublicKeyAuthentication.Enabled() {
 		typePublicKey := &LoginTypePublicKey{
-			userRegisterAPI,
+			clientUserAPI,
 			&userInteractive,
 			cfg,
 		}

clientapi/auth/user_interactive_test.go

@@ -20,8 +20,8 @@ var (
 )
 
 type fakeAccountDatabase struct {
-	api.UserAccountAPI
-	api.UserRegisterAPI
+	api.UserLoginAPI
+	api.ClientUserAPI
 }
 
 func (d *fakeAccountDatabase) PerformPasswordUpdate(ctx context.Context, req *api.PerformPasswordUpdateRequest, res *api.PerformPasswordUpdateResponse) error {

clientapi/clientapi.go

@@ -15,7 +15,6 @@
 package clientapi
 
 import (
-	"github.com/gorilla/mux"
 	appserviceAPI "github.com/matrix-org/dendrite/appservice/api"
 	"github.com/matrix-org/dendrite/clientapi/api"
 	"github.com/matrix-org/dendrite/clientapi/producers"
@@ -24,32 +23,28 @@ import (
 	"github.com/matrix-org/dendrite/internal/transactions"
 	keyserverAPI "github.com/matrix-org/dendrite/keyserver/api"
 	roomserverAPI "github.com/matrix-org/dendrite/roomserver/api"
-	"github.com/matrix-org/dendrite/setup/config"
+	"github.com/matrix-org/dendrite/setup/base"
 	"github.com/matrix-org/dendrite/setup/jetstream"
-	"github.com/matrix-org/dendrite/setup/process"
 	userapi "github.com/matrix-org/dendrite/userapi/api"
 	"github.com/matrix-org/gomatrixserverlib"
 )
 
 // AddPublicRoutes sets up and registers HTTP handlers for the ClientAPI component.
 func AddPublicRoutes(
-	process *process.ProcessContext,
-	router *mux.Router,
-	synapseAdminRouter *mux.Router,
-	dendriteAdminRouter *mux.Router,
-	cfg *config.ClientAPI,
+	base *base.BaseDendrite,
 	federation *gomatrixserverlib.FederationClient,
-	rsAPI roomserverAPI.RoomserverInternalAPI,
-	asAPI appserviceAPI.AppServiceQueryAPI,
+	rsAPI roomserverAPI.ClientRoomserverAPI,
+	asAPI appserviceAPI.AppServiceInternalAPI,
 	transactionsCache *transactions.Cache,
-	fsAPI federationAPI.FederationInternalAPI,
-	userAPI userapi.UserInternalAPI,
-	userDirectoryProvider userapi.UserDirectoryProvider,
-	keyAPI keyserverAPI.KeyInternalAPI,
+	fsAPI federationAPI.ClientFederationAPI,
+	userAPI userapi.ClientUserAPI,
+	userDirectoryProvider userapi.QuerySearchProfilesAPI,
+	keyAPI keyserverAPI.ClientKeyAPI,
 	extRoomsProvider api.ExtraPublicRoomsProvider,
-	mscCfg *config.MSCs,
 ) {
-	js, natsClient := jetstream.Prepare(process, &cfg.Matrix.JetStream)
+	cfg := &base.Cfg.ClientAPI
+	mscCfg := &base.Cfg.MSCs
+	js, natsClient := base.NATS.Prepare(base.ProcessContext, &cfg.Matrix.JetStream)
 
 	syncProducer := &producers.SyncAPIProducer{
 		JetStream:              js,
@@ -63,7 +58,9 @@ func AddPublicRoutes(
 	}
 
 	routing.Setup(
-		router, synapseAdminRouter, dendriteAdminRouter,
+		base.PublicClientAPIMux,
+		base.SynapseAdminMux,
+		base.DendriteAdminMux,
 		cfg, rsAPI, asAPI,
 		userAPI, userDirectoryProvider, federation,
 		syncProducer, transactionsCache, fsAPI, keyAPI,

clientapi/producers/syncapi.go

@@ -38,7 +38,7 @@ type SyncAPIProducer struct {
 	TopicPresenceEvent     string
 	JetStream              nats.JetStreamContext
 	ServerName             gomatrixserverlib.ServerName
-	UserAPI                userapi.UserInternalAPI
+	UserAPI                userapi.ClientUserAPI
 }
 
 // SendData sends account data to the sync API server

clientapi/routing/account_data.go

@@ -33,7 +33,7 @@ import (
 
 // GetAccountData implements GET /user/{userId}/[rooms/{roomid}/]account_data/{type}
 func GetAccountData(
-	req *http.Request, userAPI api.UserInternalAPI, device *api.Device,
+	req *http.Request, userAPI api.ClientUserAPI, device *api.Device,
 	userID string, roomID string, dataType string,
 ) util.JSONResponse {
 	if userID != device.UserID {
@@ -76,7 +76,7 @@ func GetAccountData(
 
 // SaveAccountData implements PUT /user/{userId}/[rooms/{roomId}/]account_data/{type}
 func SaveAccountData(
-	req *http.Request, userAPI api.UserInternalAPI, device *api.Device,
+	req *http.Request, userAPI api.ClientUserAPI, device *api.Device,
 	userID string, roomID string, dataType string, syncProducer *producers.SyncAPIProducer,
 ) util.JSONResponse {
 	if userID != device.UserID {
@@ -152,7 +152,7 @@ type fullyReadEvent struct {
 // SaveReadMarker implements POST /rooms/{roomId}/read_markers
 func SaveReadMarker(
 	req *http.Request,
-	userAPI api.UserInternalAPI, rsAPI roomserverAPI.RoomserverInternalAPI,
+	userAPI api.ClientUserAPI, rsAPI roomserverAPI.ClientRoomserverAPI,
 	syncProducer *producers.SyncAPIProducer, device *api.Device, roomID string,
 ) util.JSONResponse {
 	// Verify that the user is a member of this room

clientapi/routing/admin.go

@@ -11,7 +11,7 @@ import (
 	"github.com/matrix-org/util"
 )
 
-func AdminEvacuateRoom(req *http.Request, device *userapi.Device, rsAPI roomserverAPI.RoomserverInternalAPI) util.JSONResponse {
+func AdminEvacuateRoom(req *http.Request, device *userapi.Device, rsAPI roomserverAPI.ClientRoomserverAPI) util.JSONResponse {
 	if device.AccountType != userapi.AccountTypeAdmin {
 		return util.JSONResponse{
 			Code: http.StatusForbidden,

clientapi/routing/admin_whois.go

@@ -44,7 +44,7 @@ type connectionInfo struct {
 
 // GetAdminWhois implements GET /admin/whois/{userId}
 func GetAdminWhois(
-	req *http.Request, userAPI api.UserInternalAPI, device *api.Device,
+	req *http.Request, userAPI api.ClientUserAPI, device *api.Device,
 	userID string,
 ) util.JSONResponse {
 	allowed := device.AccountType == api.AccountTypeAdmin || userID == device.UserID

clientapi/routing/aliases.go

@@ -28,7 +28,7 @@ import (
 
 // GetAliases implements GET /_matrix/client/r0/rooms/{roomId}/aliases
 func GetAliases(
-	req *http.Request, rsAPI api.RoomserverInternalAPI, device *userapi.Device, roomID string,
+	req *http.Request, rsAPI api.ClientRoomserverAPI, device *userapi.Device, roomID string,
 ) util.JSONResponse {
 	stateTuple := gomatrixserverlib.StateKeyTuple{
 		EventType: gomatrixserverlib.MRoomHistoryVisibility,

clientapi/routing/capabilities.go

@@ -26,7 +26,7 @@ import (
 // GetCapabilities returns information about the server's supported feature set
 // and other relevant capabilities to an authenticated user.
 func GetCapabilities(
-	req *http.Request, rsAPI roomserverAPI.RoomserverInternalAPI,
+	req *http.Request, rsAPI roomserverAPI.ClientRoomserverAPI,
 ) util.JSONResponse {
 	roomVersionsQueryReq := roomserverAPI.QueryRoomVersionCapabilitiesRequest{}
 	roomVersionsQueryRes := roomserverAPI.QueryRoomVersionCapabilitiesResponse{}

clientapi/routing/createroom.go

@@ -137,8 +137,8 @@ type fledglingEvent struct {
 func CreateRoom(
 	req *http.Request, device *api.Device,
 	cfg *config.ClientAPI,
-	profileAPI api.UserProfileAPI, rsAPI roomserverAPI.RoomserverInternalAPI,
-	asAPI appserviceAPI.AppServiceQueryAPI,
+	profileAPI api.ClientUserAPI, rsAPI roomserverAPI.ClientRoomserverAPI,
+	asAPI appserviceAPI.AppServiceInternalAPI,
 ) util.JSONResponse {
 	var r createRoomRequest
 	resErr := httputil.UnmarshalJSONRequest(req, &r)
@@ -164,8 +164,8 @@ func createRoom(
 	ctx context.Context,
 	r createRoomRequest, device *api.Device,
 	cfg *config.ClientAPI,
-	profileAPI api.UserProfileAPI, rsAPI roomserverAPI.RoomserverInternalAPI,
-	asAPI appserviceAPI.AppServiceQueryAPI,
+	profileAPI api.ClientUserAPI, rsAPI roomserverAPI.ClientRoomserverAPI,
+	asAPI appserviceAPI.AppServiceInternalAPI,
 	evTime time.Time,
 ) util.JSONResponse {
 	// TODO (#267): Check room ID doesn't clash with an existing one, and we
@@ -531,25 +531,23 @@ func createRoom(
 				gomatrixserverlib.NewInviteV2StrippedState(inviteEvent.Event),
 			)
 			// Send the invite event to the roomserver.
-			err = roomserverAPI.SendInvite(
-				ctx,
-				rsAPI,
-				inviteEvent.Headered(roomVersion),
-				inviteStrippedState,   // invite room state
-				cfg.Matrix.ServerName, // send as server
-				nil,                   // transaction ID
-			)
-			switch e := err.(type) {
-			case *roomserverAPI.PerformError:
-				return e.JSONResponse()
-			case nil:
-			default:
-				util.GetLogger(ctx).WithError(err).Error("roomserverAPI.SendInvite failed")
+			var inviteRes roomserverAPI.PerformInviteResponse
+			event := inviteEvent.Headered(roomVersion)
+			if err := rsAPI.PerformInvite(ctx, &roomserverAPI.PerformInviteRequest{
+				Event:           event,
+				InviteRoomState: inviteStrippedState,
+				RoomVersion:     event.RoomVersion,
+				SendAsServer:    string(cfg.Matrix.ServerName),
+			}, &inviteRes); err != nil {
+				util.GetLogger(ctx).WithError(err).Error("PerformInvite failed")
 				return util.JSONResponse{
 					Code: http.StatusInternalServerError,
 					JSON: jsonerror.InternalServerError(),
 				}
 			}
+			if inviteRes.Error != nil {
+				return inviteRes.Error.JSONResponse()
+			}
 		}
 	}
 

clientapi/routing/deactivate.go

@@ -15,7 +15,7 @@ import (
 func Deactivate(
 	req *http.Request,
 	userInteractiveAuth *auth.UserInteractive,
-	accountAPI api.UserAccountAPI,
+	accountAPI api.ClientUserAPI,
 	deviceAPI *api.Device,
 ) util.JSONResponse {
 	ctx := req.Context()

clientapi/routing/device.go

@@ -50,7 +50,7 @@ type devicesDeleteJSON struct {
 
 // GetDeviceByID handles /devices/{deviceID}
 func GetDeviceByID(
-	req *http.Request, userAPI api.UserInternalAPI, device *api.Device,
+	req *http.Request, userAPI api.ClientUserAPI, device *api.Device,
 	deviceID string,
 ) util.JSONResponse {
 	var queryRes api.QueryDevicesResponse
@@ -88,7 +88,7 @@ func GetDeviceByID(
 
 // GetDevicesByLocalpart handles /devices
 func GetDevicesByLocalpart(
-	req *http.Request, userAPI api.UserInternalAPI, device *api.Device,
+	req *http.Request, userAPI api.ClientUserAPI, device *api.Device,
 ) util.JSONResponse {
 	var queryRes api.QueryDevicesResponse
 	err := userAPI.QueryDevices(req.Context(), &api.QueryDevicesRequest{
@@ -118,7 +118,7 @@ func GetDevicesByLocalpart(
 
 // UpdateDeviceByID handles PUT on /devices/{deviceID}
 func UpdateDeviceByID(
-	req *http.Request, userAPI api.UserInternalAPI, device *api.Device,
+	req *http.Request, userAPI api.ClientUserAPI, device *api.Device,
 	deviceID string,
 ) util.JSONResponse {
 
@@ -161,7 +161,7 @@ func UpdateDeviceByID(
 
 // DeleteDeviceById handles DELETE requests to /devices/{deviceId}
 func DeleteDeviceById(
-	req *http.Request, userInteractiveAuth *auth.UserInteractive, userAPI api.UserInternalAPI, device *api.Device,
+	req *http.Request, userInteractiveAuth *auth.UserInteractive, userAPI api.ClientUserAPI, device *api.Device,
 	deviceID string,
 ) util.JSONResponse {
 	var (
@@ -242,7 +242,7 @@ func DeleteDeviceById(
 
 // DeleteDevices handles POST requests to /delete_devices
 func DeleteDevices(
-	req *http.Request, userAPI api.UserInternalAPI, device *api.Device,
+	req *http.Request, userAPI api.ClientUserAPI, device *api.Device,
 ) util.JSONResponse {
 	ctx := req.Context()
 	payload := devicesDeleteJSON{}

clientapi/routing/directory.go

@@ -46,8 +46,8 @@ func DirectoryRoom(
 	roomAlias string,
 	federation *gomatrixserverlib.FederationClient,
 	cfg *config.ClientAPI,
-	rsAPI roomserverAPI.RoomserverInternalAPI,
-	fedSenderAPI federationAPI.FederationInternalAPI,
+	rsAPI roomserverAPI.ClientRoomserverAPI,
+	fedSenderAPI federationAPI.ClientFederationAPI,
 ) util.JSONResponse {
 	_, domain, err := gomatrixserverlib.SplitID('#', roomAlias)
 	if err != nil {
@@ -117,7 +117,7 @@ func SetLocalAlias(
 	device *userapi.Device,
 	alias string,
 	cfg *config.ClientAPI,
-	rsAPI roomserverAPI.RoomserverInternalAPI,
+	rsAPI roomserverAPI.ClientRoomserverAPI,
 ) util.JSONResponse {
 	_, domain, err := gomatrixserverlib.SplitID('#', alias)
 	if err != nil {
@@ -199,7 +199,7 @@ func RemoveLocalAlias(
 	req *http.Request,
 	device *userapi.Device,
 	alias string,
-	rsAPI roomserverAPI.RoomserverInternalAPI,
+	rsAPI roomserverAPI.ClientRoomserverAPI,
 ) util.JSONResponse {
 	queryReq := roomserverAPI.RemoveRoomAliasRequest{
 		Alias:  alias,
@@ -237,7 +237,7 @@ type roomVisibility struct {
 
 // GetVisibility implements GET /directory/list/room/{roomID}
 func GetVisibility(
-	req *http.Request, rsAPI roomserverAPI.RoomserverInternalAPI,
+	req *http.Request, rsAPI roomserverAPI.ClientRoomserverAPI,
 	roomID string,
 ) util.JSONResponse {
 	var res roomserverAPI.QueryPublishedRoomsResponse
@@ -265,7 +265,7 @@ func GetVisibility(
 // SetVisibility implements PUT /directory/list/room/{roomID}
 // TODO: Allow admin users to edit the room visibility
 func SetVisibility(
-	req *http.Request, rsAPI roomserverAPI.RoomserverInternalAPI, dev *userapi.Device,
+	req *http.Request, rsAPI roomserverAPI.ClientRoomserverAPI, dev *userapi.Device,
 	roomID string,
 ) util.JSONResponse {
 	resErr := checkMemberInRoom(req.Context(), rsAPI, dev.UserID, roomID)

clientapi/routing/directory_public.go

@@ -50,7 +50,7 @@ type filter struct {
 
 // GetPostPublicRooms implements GET and POST /publicRooms
 func GetPostPublicRooms(
-	req *http.Request, rsAPI roomserverAPI.RoomserverInternalAPI,
+	req *http.Request, rsAPI roomserverAPI.ClientRoomserverAPI,
 	extRoomsProvider api.ExtraPublicRoomsProvider,
 	federation *gomatrixserverlib.FederationClient,
 	cfg *config.ClientAPI,
@@ -91,7 +91,7 @@ func GetPostPublicRooms(
 }
 
 func publicRooms(
-	ctx context.Context, request PublicRoomReq, rsAPI roomserverAPI.RoomserverInternalAPI, extRoomsProvider api.ExtraPublicRoomsProvider,
+	ctx context.Context, request PublicRoomReq, rsAPI roomserverAPI.ClientRoomserverAPI, extRoomsProvider api.ExtraPublicRoomsProvider,
 ) (*gomatrixserverlib.RespPublicRooms, error) {
 
 	response := gomatrixserverlib.RespPublicRooms{
@@ -229,7 +229,7 @@ func sliceInto(slice []gomatrixserverlib.PublicRoom, since int64, limit int16) (
 }
 
 func refreshPublicRoomCache(
-	ctx context.Context, rsAPI roomserverAPI.RoomserverInternalAPI, extRoomsProvider api.ExtraPublicRoomsProvider,
+	ctx context.Context, rsAPI roomserverAPI.ClientRoomserverAPI, extRoomsProvider api.ExtraPublicRoomsProvider,
 ) []gomatrixserverlib.PublicRoom {
 	cacheMu.Lock()
 	defer cacheMu.Unlock()

clientapi/routing/getevent.go

@@ -31,7 +31,6 @@ type getEventRequest struct {
 	roomID         string
 	eventID        string
 	cfg            *config.ClientAPI
-	federation     *gomatrixserverlib.FederationClient
 	requestedEvent *gomatrixserverlib.Event
 }
 
@@ -43,8 +42,7 @@ func GetEvent(
 	roomID string,
 	eventID string,
 	cfg *config.ClientAPI,
-	rsAPI api.RoomserverInternalAPI,
-	federation *gomatrixserverlib.FederationClient,
+	rsAPI api.ClientRoomserverAPI,
 ) util.JSONResponse {
 	eventsReq := api.QueryEventsByIDRequest{
 		EventIDs: []string{eventID},
@@ -72,7 +70,6 @@ func GetEvent(
 		roomID:         roomID,
 		eventID:        eventID,
 		cfg:            cfg,
-		federation:     federation,
 		requestedEvent: requestedEvent,
 	}
 

clientapi/routing/joinroom.go

@@ -29,8 +29,8 @@ import (
 func JoinRoomByIDOrAlias(
 	req *http.Request,
 	device *api.Device,
-	rsAPI roomserverAPI.RoomserverInternalAPI,
-	profileAPI api.UserProfileAPI,
+	rsAPI roomserverAPI.ClientRoomserverAPI,
+	profileAPI api.ClientUserAPI,
 	roomIDOrAlias string,
 ) util.JSONResponse {
 	// Prepare to ask the roomserver to perform the room join.

clientapi/routing/key_backup.go

@@ -55,7 +55,7 @@ type keyBackupSessionResponse struct {
 
 // Create a new key backup. Request must contain a `keyBackupVersion`. Returns a `keyBackupVersionCreateResponse`.
 // Implements  POST /_matrix/client/r0/room_keys/version
-func CreateKeyBackupVersion(req *http.Request, userAPI userapi.UserInternalAPI, device *userapi.Device) util.JSONResponse {
+func CreateKeyBackupVersion(req *http.Request, userAPI userapi.ClientUserAPI, device *userapi.Device) util.JSONResponse {
 	var kb keyBackupVersion
 	resErr := httputil.UnmarshalJSONRequest(req, &kb)
 	if resErr != nil {
@@ -89,7 +89,7 @@ func CreateKeyBackupVersion(req *http.Request, userAPI userapi.UserInternalAPI,
 
 // KeyBackupVersion returns the key backup version specified. If `version` is empty, the latest `keyBackupVersionResponse` is returned.
 // Implements GET  /_matrix/client/r0/room_keys/version and GET /_matrix/client/r0/room_keys/version/{version}
-func KeyBackupVersion(req *http.Request, userAPI userapi.UserInternalAPI, device *userapi.Device, version string) util.JSONResponse {
+func KeyBackupVersion(req *http.Request, userAPI userapi.ClientUserAPI, device *userapi.Device, version string) util.JSONResponse {
 	var queryResp userapi.QueryKeyBackupResponse
 	userAPI.QueryKeyBackup(req.Context(), &userapi.QueryKeyBackupRequest{
 		UserID:  device.UserID,
@@ -118,7 +118,7 @@ func KeyBackupVersion(req *http.Request, userAPI userapi.UserInternalAPI, device
 
 // Modify the auth data of a key backup. Version must not be empty. Request must contain a `keyBackupVersion`
 // Implements PUT  /_matrix/client/r0/room_keys/version/{version}
-func ModifyKeyBackupVersionAuthData(req *http.Request, userAPI userapi.UserInternalAPI, device *userapi.Device, version string) util.JSONResponse {
+func ModifyKeyBackupVersionAuthData(req *http.Request, userAPI userapi.ClientUserAPI, device *userapi.Device, version string) util.JSONResponse {
 	var kb keyBackupVersion
 	resErr := httputil.UnmarshalJSONRequest(req, &kb)
 	if resErr != nil {
@@ -159,7 +159,7 @@ func ModifyKeyBackupVersionAuthData(req *http.Request, userAPI userapi.UserInter
 
 // Delete a version of key backup. Version must not be empty. If the key backup was previously deleted, will return 200 OK.
 // Implements DELETE  /_matrix/client/r0/room_keys/version/{version}
-func DeleteKeyBackupVersion(req *http.Request, userAPI userapi.UserInternalAPI, device *userapi.Device, version string) util.JSONResponse {
+func DeleteKeyBackupVersion(req *http.Request, userAPI userapi.ClientUserAPI, device *userapi.Device, version string) util.JSONResponse {
 	var performKeyBackupResp userapi.PerformKeyBackupResponse
 	if err := userAPI.PerformKeyBackup(req.Context(), &userapi.PerformKeyBackupRequest{
 		UserID:       device.UserID,
@@ -194,7 +194,7 @@ func DeleteKeyBackupVersion(req *http.Request, userAPI userapi.UserInternalAPI,
 
 // Upload a bunch of session keys for a given `version`.
 func UploadBackupKeys(
-	req *http.Request, userAPI userapi.UserInternalAPI, device *userapi.Device, version string, keys *keyBackupSessionRequest,
+	req *http.Request, userAPI userapi.ClientUserAPI, device *userapi.Device, version string, keys *keyBackupSessionRequest,
 ) util.JSONResponse {
 	var performKeyBackupResp userapi.PerformKeyBackupResponse
 	if err := userAPI.PerformKeyBackup(req.Context(), &userapi.PerformKeyBackupRequest{
@@ -230,7 +230,7 @@ func UploadBackupKeys(
 
 // Get keys from a given backup version. Response returned varies depending on if roomID and sessionID are set.
 func GetBackupKeys(
-	req *http.Request, userAPI userapi.UserInternalAPI, device *userapi.Device, version, roomID, sessionID string,
+	req *http.Request, userAPI userapi.ClientUserAPI, device *userapi.Device, version, roomID, sessionID string,
 ) util.JSONResponse {
 	var queryResp userapi.QueryKeyBackupResponse
 	userAPI.QueryKeyBackup(req.Context(), &userapi.QueryKeyBackupRequest{

clientapi/routing/key_crosssigning.go

@@ -34,8 +34,8 @@ type crossSigningRequest struct {
 
 func UploadCrossSigningDeviceKeys(
 	req *http.Request, userInteractiveAuth *auth.UserInteractive,
-	keyserverAPI api.KeyInternalAPI, device *userapi.Device,
-	accountAPI userapi.UserAccountAPI, cfg *config.ClientAPI,
+	keyserverAPI api.ClientKeyAPI, device *userapi.Device,
+	accountAPI userapi.ClientUserAPI, cfg *config.ClientAPI,
 ) util.JSONResponse {
 	uploadReq := &crossSigningRequest{}
 	uploadRes := &api.PerformUploadDeviceKeysResponse{}
@@ -105,7 +105,7 @@ func UploadCrossSigningDeviceKeys(
 	}
 }
 
-func UploadCrossSigningDeviceSignatures(req *http.Request, keyserverAPI api.KeyInternalAPI, device *userapi.Device) util.JSONResponse {
+func UploadCrossSigningDeviceSignatures(req *http.Request, keyserverAPI api.ClientKeyAPI, device *userapi.Device) util.JSONResponse {
 	uploadReq := &api.PerformUploadDeviceSignaturesRequest{}
 	uploadRes := &api.PerformUploadDeviceSignaturesResponse{}
 

clientapi/routing/keys.go

@@ -31,7 +31,7 @@ type uploadKeysRequest struct {
 	OneTimeKeys map[string]json.RawMessage `json:"one_time_keys"`
 }
 
-func UploadKeys(req *http.Request, keyAPI api.KeyInternalAPI, device *userapi.Device) util.JSONResponse {
+func UploadKeys(req *http.Request, keyAPI api.ClientKeyAPI, device *userapi.Device) util.JSONResponse {
 	var r uploadKeysRequest
 	resErr := httputil.UnmarshalJSONRequest(req, &r)
 	if resErr != nil {
@@ -100,7 +100,7 @@ func (r *queryKeysRequest) GetTimeout() time.Duration {
 	return time.Duration(r.Timeout) * time.Millisecond
 }
 
-func QueryKeys(req *http.Request, keyAPI api.KeyInternalAPI, device *userapi.Device) util.JSONResponse {
+func QueryKeys(req *http.Request, keyAPI api.ClientKeyAPI, device *userapi.Device) util.JSONResponse {
 	var r queryKeysRequest
 	resErr := httputil.UnmarshalJSONRequest(req, &r)
 	if resErr != nil {
@@ -138,7 +138,7 @@ func (r *claimKeysRequest) GetTimeout() time.Duration {
 	return time.Duration(r.TimeoutMS) * time.Millisecond
 }
 
-func ClaimKeys(req *http.Request, keyAPI api.KeyInternalAPI) util.JSONResponse {
+func ClaimKeys(req *http.Request, keyAPI api.ClientKeyAPI) util.JSONResponse {
 	var r claimKeysRequest
 	resErr := httputil.UnmarshalJSONRequest(req, &r)
 	if resErr != nil {

clientapi/routing/leaveroom.go

@@ -26,7 +26,7 @@ import (
 func LeaveRoomByID(
 	req *http.Request,
 	device *api.Device,
-	rsAPI roomserverAPI.RoomserverInternalAPI,
+	rsAPI roomserverAPI.ClientRoomserverAPI,
 	roomID string,
 ) util.JSONResponse {
 	// Prepare to ask the roomserver to perform the room join.

clientapi/routing/login.go

@@ -59,7 +59,7 @@ func publicKeyLogin(f *flows) {
 // Login implements GET and POST /login
 func Login(
 	req *http.Request,
-	userAPI userapi.UserInternalAPI,
+	userAPI userapi.ClientUserAPI,
 	userInteractiveAuth *auth.UserInteractive,
 	cfg *config.ClientAPI,
 ) util.JSONResponse {
@@ -93,7 +93,7 @@ func Login(
 }
 
 func completeAuth(
-	ctx context.Context, serverName gomatrixserverlib.ServerName, userAPI userapi.UserInternalAPI, login *auth.Login,
+	ctx context.Context, serverName gomatrixserverlib.ServerName, userAPI userapi.ClientUserAPI, login *auth.Login,
 	ipAddr, userAgent string,
 ) util.JSONResponse {
 	token, err := auth.GenerateAccessToken()

clientapi/routing/logout.go

@@ -24,7 +24,7 @@ import (
 
 // Logout handles POST /logout
 func Logout(
-	req *http.Request, userAPI api.UserInternalAPI, device *api.Device,
+	req *http.Request, userAPI api.ClientUserAPI, device *api.Device,
 ) util.JSONResponse {
 	var performRes api.PerformDeviceDeletionResponse
 	err := userAPI.PerformDeviceDeletion(req.Context(), &api.PerformDeviceDeletionRequest{
@@ -44,7 +44,7 @@ func Logout(
 
 // LogoutAll handles POST /logout/all
 func LogoutAll(
-	req *http.Request, userAPI api.UserInternalAPI, device *api.Device,
+	req *http.Request, userAPI api.ClientUserAPI, device *api.Device,
 ) util.JSONResponse {
 	var performRes api.PerformDeviceDeletionResponse
 	err := userAPI.PerformDeviceDeletion(req.Context(), &api.PerformDeviceDeletionRequest{

clientapi/routing/membership.go

@@ -27,6 +27,7 @@ import (
 	"github.com/matrix-org/dendrite/clientapi/jsonerror"
 	"github.com/matrix-org/dendrite/clientapi/threepid"
 	"github.com/matrix-org/dendrite/internal/eventutil"
+	"github.com/matrix-org/dendrite/roomserver/api"
 	roomserverAPI "github.com/matrix-org/dendrite/roomserver/api"
 	"github.com/matrix-org/dendrite/setup/config"
 	userapi "github.com/matrix-org/dendrite/userapi/api"
@@ -38,9 +39,9 @@ import (
 var errMissingUserID = errors.New("'user_id' must be supplied")
 
 func SendBan(
-	req *http.Request, profileAPI userapi.UserProfileAPI, device *userapi.Device,
+	req *http.Request, profileAPI userapi.ClientUserAPI, device *userapi.Device,
 	roomID string, cfg *config.ClientAPI,
-	rsAPI roomserverAPI.RoomserverInternalAPI, asAPI appserviceAPI.AppServiceQueryAPI,
+	rsAPI roomserverAPI.ClientRoomserverAPI, asAPI appserviceAPI.AppServiceInternalAPI,
 ) util.JSONResponse {
 	body, evTime, roomVer, reqErr := extractRequestData(req, roomID, rsAPI)
 	if reqErr != nil {
@@ -80,10 +81,10 @@ func SendBan(
 	return sendMembership(req.Context(), profileAPI, device, roomID, "ban", body.Reason, cfg, body.UserID, evTime, roomVer, rsAPI, asAPI)
 }
 
-func sendMembership(ctx context.Context, profileAPI userapi.UserProfileAPI, device *userapi.Device,
+func sendMembership(ctx context.Context, profileAPI userapi.ClientUserAPI, device *userapi.Device,
 	roomID, membership, reason string, cfg *config.ClientAPI, targetUserID string, evTime time.Time,
 	roomVer gomatrixserverlib.RoomVersion,
-	rsAPI roomserverAPI.RoomserverInternalAPI, asAPI appserviceAPI.AppServiceQueryAPI) util.JSONResponse {
+	rsAPI roomserverAPI.ClientRoomserverAPI, asAPI appserviceAPI.AppServiceInternalAPI) util.JSONResponse {
 
 	event, err := buildMembershipEvent(
 		ctx, targetUserID, reason, profileAPI, device, membership,
@@ -124,9 +125,9 @@ func sendMembership(ctx context.Context, profileAPI userapi.UserProfileAPI, devi
 }
 
 func SendKick(
-	req *http.Request, profileAPI userapi.UserProfileAPI, device *userapi.Device,
+	req *http.Request, profileAPI userapi.ClientUserAPI, device *userapi.Device,
 	roomID string, cfg *config.ClientAPI,
-	rsAPI roomserverAPI.RoomserverInternalAPI, asAPI appserviceAPI.AppServiceQueryAPI,
+	rsAPI roomserverAPI.ClientRoomserverAPI, asAPI appserviceAPI.AppServiceInternalAPI,
 ) util.JSONResponse {
 	body, evTime, roomVer, reqErr := extractRequestData(req, roomID, rsAPI)
 	if reqErr != nil {
@@ -164,9 +165,9 @@ func SendKick(
 }
 
 func SendUnban(
-	req *http.Request, profileAPI userapi.UserProfileAPI, device *userapi.Device,
+	req *http.Request, profileAPI userapi.ClientUserAPI, device *userapi.Device,
 	roomID string, cfg *config.ClientAPI,
-	rsAPI roomserverAPI.RoomserverInternalAPI, asAPI appserviceAPI.AppServiceQueryAPI,
+	rsAPI roomserverAPI.ClientRoomserverAPI, asAPI appserviceAPI.AppServiceInternalAPI,
 ) util.JSONResponse {
 	body, evTime, roomVer, reqErr := extractRequestData(req, roomID, rsAPI)
 	if reqErr != nil {
@@ -187,6 +188,12 @@ func SendUnban(
 	if err != nil {
 		return util.ErrorResponse(err)
 	}
+	if !queryRes.RoomExists {
+		return util.JSONResponse{
+			Code: http.StatusForbidden,
+			JSON: jsonerror.Forbidden("room does not exist"),
+		}
+	}
 	// unban is only valid if the user is currently banned
 	if queryRes.Membership != "ban" {
 		return util.JSONResponse{
@@ -199,9 +206,9 @@ func SendUnban(
 }
 
 func SendInvite(
-	req *http.Request, profileAPI userapi.UserProfileAPI, device *userapi.Device,
+	req *http.Request, profileAPI userapi.ClientUserAPI, device *userapi.Device,
 	roomID string, cfg *config.ClientAPI,
-	rsAPI roomserverAPI.RoomserverInternalAPI, asAPI appserviceAPI.AppServiceQueryAPI,
+	rsAPI roomserverAPI.ClientRoomserverAPI, asAPI appserviceAPI.AppServiceInternalAPI,
 ) util.JSONResponse {
 	body, evTime, _, reqErr := extractRequestData(req, roomID, rsAPI)
 	if reqErr != nil {
@@ -233,12 +240,12 @@ func SendInvite(
 // sendInvite sends an invitation to a user. Returns a JSONResponse and an error
 func sendInvite(
 	ctx context.Context,
-	profileAPI userapi.UserProfileAPI,
+	profileAPI userapi.ClientUserAPI,
 	device *userapi.Device,
 	roomID, userID, reason string,
 	cfg *config.ClientAPI,
-	rsAPI roomserverAPI.RoomserverInternalAPI,
-	asAPI appserviceAPI.AppServiceQueryAPI, evTime time.Time,
+	rsAPI roomserverAPI.ClientRoomserverAPI,
+	asAPI appserviceAPI.AppServiceInternalAPI, evTime time.Time,
 ) (util.JSONResponse, error) {
 	event, err := buildMembershipEvent(
 		ctx, userID, reason, profileAPI, device, "invite",
@@ -259,37 +266,36 @@ func sendInvite(
 		return jsonerror.InternalServerError(), err
 	}
 
-	err = roomserverAPI.SendInvite(
-		ctx, rsAPI,
-		event,
-		nil, // ask the roomserver to draw up invite room state for us
-		cfg.Matrix.ServerName,
-		nil,
-	)
-	switch e := err.(type) {
-	case *roomserverAPI.PerformError:
-		return e.JSONResponse(), err
-	case nil:
-		return util.JSONResponse{
-			Code: http.StatusOK,
-			JSON: struct{}{},
-		}, nil
-	default:
-		util.GetLogger(ctx).WithError(err).Error("roomserverAPI.SendInvite failed")
+	var inviteRes api.PerformInviteResponse
+	if err := rsAPI.PerformInvite(ctx, &api.PerformInviteRequest{
+		Event:           event,
+		InviteRoomState: nil, // ask the roomserver to draw up invite room state for us
+		RoomVersion:     event.RoomVersion,
+		SendAsServer:    string(cfg.Matrix.ServerName),
+	}, &inviteRes); err != nil {
+		util.GetLogger(ctx).WithError(err).Error("PerformInvite failed")
 		return util.JSONResponse{
 			Code: http.StatusInternalServerError,
 			JSON: jsonerror.InternalServerError(),
 		}, err
 	}
+	if inviteRes.Error != nil {
+		return inviteRes.Error.JSONResponse(), inviteRes.Error
+	}
+
+	return util.JSONResponse{
+		Code: http.StatusOK,
+		JSON: struct{}{},
+	}, nil
 }
 
 func buildMembershipEvent(
 	ctx context.Context,
-	targetUserID, reason string, profileAPI userapi.UserProfileAPI,
+	targetUserID, reason string, profileAPI userapi.ClientUserAPI,
 	device *userapi.Device,
 	membership, roomID string, isDirect bool,
 	cfg *config.ClientAPI, evTime time.Time,
-	rsAPI roomserverAPI.RoomserverInternalAPI, asAPI appserviceAPI.AppServiceQueryAPI,
+	rsAPI roomserverAPI.ClientRoomserverAPI, asAPI appserviceAPI.AppServiceInternalAPI,
 ) (*gomatrixserverlib.HeaderedEvent, error) {
 	profile, err := loadProfile(ctx, targetUserID, cfg, profileAPI, asAPI)
 	if err != nil {
@@ -326,8 +332,8 @@ func loadProfile(
 	ctx context.Context,
 	userID string,
 	cfg *config.ClientAPI,
-	profileAPI userapi.UserProfileAPI,
-	asAPI appserviceAPI.AppServiceQueryAPI,
+	profileAPI userapi.ClientUserAPI,
+	asAPI appserviceAPI.AppServiceInternalAPI,
 ) (*authtypes.Profile, error) {
 	_, serverName, err := gomatrixserverlib.SplitID('@', userID)
 	if err != nil {
@@ -344,7 +350,7 @@ func loadProfile(
 	return profile, err
 }
 
-func extractRequestData(req *http.Request, roomID string, rsAPI roomserverAPI.RoomserverInternalAPI) (
+func extractRequestData(req *http.Request, roomID string, rsAPI roomserverAPI.ClientRoomserverAPI) (
 	body *threepid.MembershipRequest, evTime time.Time, roomVer gomatrixserverlib.RoomVersion, resErr *util.JSONResponse,
 ) {
 	verReq := roomserverAPI.QueryRoomVersionForRoomRequest{RoomID: roomID}
@@ -379,8 +385,8 @@ func checkAndProcessThreepid(
 	device *userapi.Device,
 	body *threepid.MembershipRequest,
 	cfg *config.ClientAPI,
-	rsAPI roomserverAPI.RoomserverInternalAPI,
-	profileAPI userapi.UserProfileAPI,
+	rsAPI roomserverAPI.ClientRoomserverAPI,
+	profileAPI userapi.ClientUserAPI,
 	roomID string,
 	evTime time.Time,
 ) (inviteStored bool, errRes *util.JSONResponse) {
@@ -418,7 +424,7 @@ func checkAndProcessThreepid(
 	return
 }
 
-func checkMemberInRoom(ctx context.Context, rsAPI roomserverAPI.RoomserverInternalAPI, userID, roomID string) *util.JSONResponse {
+func checkMemberInRoom(ctx context.Context, rsAPI roomserverAPI.ClientRoomserverAPI, userID, roomID string) *util.JSONResponse {
 	tuple := gomatrixserverlib.StateKeyTuple{
 		EventType: gomatrixserverlib.MRoomMember,
 		StateKey:  userID,
@@ -457,7 +463,7 @@ func checkMemberInRoom(ctx context.Context, rsAPI roomserverAPI.RoomserverIntern
 
 func SendForget(
 	req *http.Request, device *userapi.Device,
-	roomID string, rsAPI roomserverAPI.RoomserverInternalAPI,
+	roomID string, rsAPI roomserverAPI.ClientRoomserverAPI,
 ) util.JSONResponse {
 	ctx := req.Context()
 	logger := util.GetLogger(ctx).WithField("roomID", roomID).WithField("userID", device.UserID)
@@ -471,6 +477,12 @@ func SendForget(
 		logger.WithError(err).Error("QueryMembershipForUser: could not query membership for user")
 		return jsonerror.InternalServerError()
 	}
+	if !membershipRes.RoomExists {
+		return util.JSONResponse{
+			Code: http.StatusForbidden,
+			JSON: jsonerror.Forbidden("room does not exist"),
+		}
+	}
 	if membershipRes.IsInRoom {
 		return util.JSONResponse{
 			Code: http.StatusBadRequest,

clientapi/routing/memberships.go

@@ -55,7 +55,7 @@ type databaseJoinedMember struct {
 func GetMemberships(
 	req *http.Request, device *userapi.Device, roomID string, joinedOnly bool,
 	_ *config.ClientAPI,
-	rsAPI api.RoomserverInternalAPI,
+	rsAPI api.ClientRoomserverAPI,
 ) util.JSONResponse {
 	queryReq := api.QueryMembershipsForRoomRequest{
 		JoinedOnly: joinedOnly,
@@ -100,7 +100,7 @@ func GetMemberships(
 func GetJoinedRooms(
 	req *http.Request,
 	device *userapi.Device,
-	rsAPI api.RoomserverInternalAPI,
+	rsAPI api.ClientRoomserverAPI,
 ) util.JSONResponse {
 	var res api.QueryRoomsForUserResponse
 	err := rsAPI.QueryRoomsForUser(req.Context(), &api.QueryRoomsForUserRequest{

clientapi/routing/notification.go

@@ -27,7 +27,7 @@ import (
 // GetNotifications handles /_matrix/client/r0/notifications
 func GetNotifications(
 	req *http.Request, device *userapi.Device,
-	userAPI userapi.UserInternalAPI,
+	userAPI userapi.ClientUserAPI,
 ) util.JSONResponse {
 	var limit int64
 	if limitStr := req.URL.Query().Get("limit"); limitStr != "" {

clientapi/routing/openid.go

@@ -34,7 +34,7 @@ type openIDTokenResponse struct {
 // can supply to an OpenID Relying Party to verify their identity
 func CreateOpenIDToken(
 	req *http.Request,
-	userAPI api.UserInternalAPI,
+	userAPI api.ClientUserAPI,
 	device *api.Device,
 	userID string,
 	cfg *config.ClientAPI,

clientapi/routing/password.go

@@ -28,7 +28,7 @@ type newPasswordAuth struct {
 
 func Password(
 	req *http.Request,
-	userAPI api.UserInternalAPI,
+	userAPI api.ClientUserAPI,
 	device *api.Device,
 	cfg *config.ClientAPI,
 ) util.JSONResponse {

clientapi/routing/peekroom.go

@@ -26,7 +26,7 @@ import (
 func PeekRoomByIDOrAlias(
 	req *http.Request,
 	device *api.Device,
-	rsAPI roomserverAPI.RoomserverInternalAPI,
+	rsAPI roomserverAPI.ClientRoomserverAPI,
 	roomIDOrAlias string,
 ) util.JSONResponse {
 	// if this is a remote roomIDOrAlias, we have to ask the roomserver (or federation sender?) to
@@ -79,7 +79,7 @@ func PeekRoomByIDOrAlias(
 func UnpeekRoomByID(
 	req *http.Request,
 	device *api.Device,
-	rsAPI roomserverAPI.RoomserverInternalAPI,
+	rsAPI roomserverAPI.ClientRoomserverAPI,
 	roomID string,
 ) util.JSONResponse {
 	unpeekReq := roomserverAPI.PerformUnpeekRequest{

clientapi/routing/profile.go

@@ -35,9 +35,9 @@ import (
 
 // GetProfile implements GET /profile/{userID}
 func GetProfile(
-	req *http.Request, profileAPI userapi.UserProfileAPI, cfg *config.ClientAPI,
+	req *http.Request, profileAPI userapi.ClientUserAPI, cfg *config.ClientAPI,
 	userID string,
-	asAPI appserviceAPI.AppServiceQueryAPI,
+	asAPI appserviceAPI.AppServiceInternalAPI,
 	federation *gomatrixserverlib.FederationClient,
 ) util.JSONResponse {
 	profile, err := getProfile(req.Context(), profileAPI, cfg, userID, asAPI, federation)
@@ -64,8 +64,8 @@ func GetProfile(
 
 // GetAvatarURL implements GET /profile/{userID}/avatar_url
 func GetAvatarURL(
-	req *http.Request, profileAPI userapi.UserProfileAPI, cfg *config.ClientAPI,
-	userID string, asAPI appserviceAPI.AppServiceQueryAPI,
+	req *http.Request, profileAPI userapi.ClientUserAPI, cfg *config.ClientAPI,
+	userID string, asAPI appserviceAPI.AppServiceInternalAPI,
 	federation *gomatrixserverlib.FederationClient,
 ) util.JSONResponse {
 	profile, err := getProfile(req.Context(), profileAPI, cfg, userID, asAPI, federation)
@@ -91,8 +91,8 @@ func GetAvatarURL(
 
 // SetAvatarURL implements PUT /profile/{userID}/avatar_url
 func SetAvatarURL(
-	req *http.Request, profileAPI userapi.UserProfileAPI,
-	device *userapi.Device, userID string, cfg *config.ClientAPI, rsAPI api.RoomserverInternalAPI,
+	req *http.Request, profileAPI userapi.ClientUserAPI,
+	device *userapi.Device, userID string, cfg *config.ClientAPI, rsAPI api.ClientRoomserverAPI,
 ) util.JSONResponse {
 	if userID != device.UserID {
 		return util.JSONResponse{
@@ -193,8 +193,8 @@ func SetAvatarURL(
 
 // GetDisplayName implements GET /profile/{userID}/displayname
 func GetDisplayName(
-	req *http.Request, profileAPI userapi.UserProfileAPI, cfg *config.ClientAPI,
-	userID string, asAPI appserviceAPI.AppServiceQueryAPI,
+	req *http.Request, profileAPI userapi.ClientUserAPI, cfg *config.ClientAPI,
+	userID string, asAPI appserviceAPI.AppServiceInternalAPI,
 	federation *gomatrixserverlib.FederationClient,
 ) util.JSONResponse {
 	profile, err := getProfile(req.Context(), profileAPI, cfg, userID, asAPI, federation)
@@ -220,8 +220,8 @@ func GetDisplayName(
 
 // SetDisplayName implements PUT /profile/{userID}/displayname
 func SetDisplayName(
-	req *http.Request, profileAPI userapi.UserProfileAPI,
-	device *userapi.Device, userID string, cfg *config.ClientAPI, rsAPI api.RoomserverInternalAPI,
+	req *http.Request, profileAPI userapi.ClientUserAPI,
+	device *userapi.Device, userID string, cfg *config.ClientAPI, rsAPI api.ClientRoomserverAPI,
 ) util.JSONResponse {
 	if userID != device.UserID {
 		return util.JSONResponse{
@@ -325,9 +325,9 @@ func SetDisplayName(
 // Returns an error when something goes wrong or specifically
 // eventutil.ErrProfileNoExists when the profile doesn't exist.
 func getProfile(
-	ctx context.Context, profileAPI userapi.UserProfileAPI, cfg *config.ClientAPI,
+	ctx context.Context, profileAPI userapi.ClientUserAPI, cfg *config.ClientAPI,
 	userID string,
-	asAPI appserviceAPI.AppServiceQueryAPI,
+	asAPI appserviceAPI.AppServiceInternalAPI,
 	federation *gomatrixserverlib.FederationClient,
 ) (*authtypes.Profile, error) {
 	localpart, domain, err := gomatrixserverlib.SplitID('@', userID)
@@ -366,7 +366,7 @@ func buildMembershipEvents(
 	ctx context.Context,
 	roomIDs []string,
 	newProfile authtypes.Profile, userID string, cfg *config.ClientAPI,
-	evTime time.Time, rsAPI api.RoomserverInternalAPI,
+	evTime time.Time, rsAPI api.ClientRoomserverAPI,
 ) ([]*gomatrixserverlib.HeaderedEvent, error) {
 	evs := []*gomatrixserverlib.HeaderedEvent{}
 

clientapi/routing/pusher.go

@@ -28,7 +28,7 @@ import (
 // GetPushers handles /_matrix/client/r0/pushers
 func GetPushers(
 	req *http.Request, device *userapi.Device,
-	userAPI userapi.UserInternalAPI,
+	userAPI userapi.ClientUserAPI,
 ) util.JSONResponse {
 	var queryRes userapi.QueryPushersResponse
 	localpart, _, err := gomatrixserverlib.SplitID('@', device.UserID)
@@ -57,7 +57,7 @@ func GetPushers(
 // The behaviour of this endpoint varies depending on the values in the JSON body.
 func SetPusher(
 	req *http.Request, device *userapi.Device,
-	userAPI userapi.UserInternalAPI,
+	userAPI userapi.ClientUserAPI,
 ) util.JSONResponse {
 	localpart, _, err := gomatrixserverlib.SplitID('@', device.UserID)
 	if err != nil {

clientapi/routing/pushrules.go

@@ -30,7 +30,7 @@ func errorResponse(ctx context.Context, err error, msg string, args ...interface
 	return jsonerror.InternalServerError()
 }
 
-func GetAllPushRules(ctx context.Context, device *userapi.Device, userAPI userapi.UserInternalAPI) util.JSONResponse {
+func GetAllPushRules(ctx context.Context, device *userapi.Device, userAPI userapi.ClientUserAPI) util.JSONResponse {
 	ruleSets, err := queryPushRules(ctx, device.UserID, userAPI)
 	if err != nil {
 		return errorResponse(ctx, err, "queryPushRulesJSON failed")
@@ -41,7 +41,7 @@ func GetAllPushRules(ctx context.Context, device *userapi.Device, userAPI userap
 	}
 }
 
-func GetPushRulesByScope(ctx context.Context, scope string, device *userapi.Device, userAPI userapi.UserInternalAPI) util.JSONResponse {
+func GetPushRulesByScope(ctx context.Context, scope string, device *userapi.Device, userAPI userapi.ClientUserAPI) util.JSONResponse {
 	ruleSets, err := queryPushRules(ctx, device.UserID, userAPI)
 	if err != nil {
 		return errorResponse(ctx, err, "queryPushRulesJSON failed")
@@ -56,7 +56,7 @@ func GetPushRulesByScope(ctx context.Context, scope string, device *userapi.Devi
 	}
 }
 
-func GetPushRulesByKind(ctx context.Context, scope, kind string, device *userapi.Device, userAPI userapi.UserInternalAPI) util.JSONResponse {
+func GetPushRulesByKind(ctx context.Context, scope, kind string, device *userapi.Device, userAPI userapi.ClientUserAPI) util.JSONResponse {
 	ruleSets, err := queryPushRules(ctx, device.UserID, userAPI)
 	if err != nil {
 		return errorResponse(ctx, err, "queryPushRules failed")
@@ -75,7 +75,7 @@ func GetPushRulesByKind(ctx context.Context, scope, kind string, device *userapi
 	}
 }
 
-func GetPushRuleByRuleID(ctx context.Context, scope, kind, ruleID string, device *userapi.Device, userAPI userapi.UserInternalAPI) util.JSONResponse {
+func GetPushRuleByRuleID(ctx context.Context, scope, kind, ruleID string, device *userapi.Device, userAPI userapi.ClientUserAPI) util.JSONResponse {
 	ruleSets, err := queryPushRules(ctx, device.UserID, userAPI)
 	if err != nil {
 		return errorResponse(ctx, err, "queryPushRules failed")
@@ -98,7 +98,7 @@ func GetPushRuleByRuleID(ctx context.Context, scope, kind, ruleID string, device
 	}
 }
 
-func PutPushRuleByRuleID(ctx context.Context, scope, kind, ruleID, afterRuleID, beforeRuleID string, body io.Reader, device *userapi.Device, userAPI userapi.UserInternalAPI) util.JSONResponse {
+func PutPushRuleByRuleID(ctx context.Context, scope, kind, ruleID, afterRuleID, beforeRuleID string, body io.Reader, device *userapi.Device, userAPI userapi.ClientUserAPI) util.JSONResponse {
 	var newRule pushrules.Rule
 	if err := json.NewDecoder(body).Decode(&newRule); err != nil {
 		return errorResponse(ctx, err, "JSON Decode failed")
@@ -160,7 +160,7 @@ func PutPushRuleByRuleID(ctx context.Context, scope, kind, ruleID, afterRuleID,
 	return util.JSONResponse{Code: http.StatusOK, JSON: struct{}{}}
 }
 
-func DeletePushRuleByRuleID(ctx context.Context, scope, kind, ruleID string, device *userapi.Device, userAPI userapi.UserInternalAPI) util.JSONResponse {
+func DeletePushRuleByRuleID(ctx context.Context, scope, kind, ruleID string, device *userapi.Device, userAPI userapi.ClientUserAPI) util.JSONResponse {
 	ruleSets, err := queryPushRules(ctx, device.UserID, userAPI)
 	if err != nil {
 		return errorResponse(ctx, err, "queryPushRules failed")
@@ -187,7 +187,7 @@ func DeletePushRuleByRuleID(ctx context.Context, scope, kind, ruleID string, dev
 	return util.JSONResponse{Code: http.StatusOK, JSON: struct{}{}}
 }
 
-func GetPushRuleAttrByRuleID(ctx context.Context, scope, kind, ruleID, attr string, device *userapi.Device, userAPI userapi.UserInternalAPI) util.JSONResponse {
+func GetPushRuleAttrByRuleID(ctx context.Context, scope, kind, ruleID, attr string, device *userapi.Device, userAPI userapi.ClientUserAPI) util.JSONResponse {
 	attrGet, err := pushRuleAttrGetter(attr)
 	if err != nil {
 		return errorResponse(ctx, err, "pushRuleAttrGetter failed")
@@ -216,7 +216,7 @@ func GetPushRuleAttrByRuleID(ctx context.Context, scope, kind, ruleID, attr stri
 	}
 }
 
-func PutPushRuleAttrByRuleID(ctx context.Context, scope, kind, ruleID, attr string, body io.Reader, device *userapi.Device, userAPI userapi.UserInternalAPI) util.JSONResponse {
+func PutPushRuleAttrByRuleID(ctx context.Context, scope, kind, ruleID, attr string, body io.Reader, device *userapi.Device, userAPI userapi.ClientUserAPI) util.JSONResponse {
 	var newPartialRule pushrules.Rule
 	if err := json.NewDecoder(body).Decode(&newPartialRule); err != nil {
 		return util.JSONResponse{
@@ -266,7 +266,7 @@ func PutPushRuleAttrByRuleID(ctx context.Context, scope, kind, ruleID, attr stri
 	return util.JSONResponse{Code: http.StatusOK, JSON: struct{}{}}
 }
 
-func queryPushRules(ctx context.Context, userID string, userAPI userapi.UserInternalAPI) (*pushrules.AccountRuleSets, error) {
+func queryPushRules(ctx context.Context, userID string, userAPI userapi.ClientUserAPI) (*pushrules.AccountRuleSets, error) {
 	var res userapi.QueryPushRulesResponse
 	if err := userAPI.QueryPushRules(ctx, &userapi.QueryPushRulesRequest{UserID: userID}, &res); err != nil {
 		util.GetLogger(ctx).WithError(err).Error("userAPI.QueryPushRules failed")
@@ -275,7 +275,7 @@ func queryPushRules(ctx context.Context, userID string, userAPI userapi.UserInte
 	return res.RuleSets, nil
 }
 
-func putPushRules(ctx context.Context, userID string, ruleSets *pushrules.AccountRuleSets, userAPI userapi.UserInternalAPI) error {
+func putPushRules(ctx context.Context, userID string, ruleSets *pushrules.AccountRuleSets, userAPI userapi.ClientUserAPI) error {
 	req := userapi.PerformPushRulesPutRequest{
 		UserID:   userID,
 		RuleSets: ruleSets,

clientapi/routing/redaction.go

@@ -40,7 +40,7 @@ type redactionResponse struct {
 
 func SendRedaction(
 	req *http.Request, device *userapi.Device, roomID, eventID string, cfg *config.ClientAPI,
-	rsAPI roomserverAPI.RoomserverInternalAPI,
+	rsAPI roomserverAPI.ClientRoomserverAPI,
 	txnID *string,
 	txnCache *transactions.Cache,
 ) util.JSONResponse {

clientapi/routing/register.go

@@ -518,7 +518,7 @@ func validateApplicationService(
 // http://matrix.org/speculator/spec/HEAD/client_server/unstable.html#post-matrix-client-unstable-register
 func Register(
 	req *http.Request,
-	userAPI userapi.UserRegisterAPI,
+	userAPI userapi.ClientUserAPI,
 	cfg *config.ClientAPI,
 ) util.JSONResponse {
 	defer req.Body.Close() // nolint: errcheck
@@ -618,7 +618,7 @@ func handleGuestRegistration(
 	req *http.Request,
 	r registerRequest,
 	cfg *config.ClientAPI,
-	userAPI userapi.UserRegisterAPI,
+	userAPI userapi.ClientUserAPI,
 ) util.JSONResponse {
 	if cfg.RegistrationDisabled || cfg.GuestsDisabled {
 		return util.JSONResponse{
@@ -684,7 +684,7 @@ func handleRegistrationFlow(
 	r registerRequest,
 	sessionID string,
 	cfg *config.ClientAPI,
-	userAPI userapi.UserRegisterAPI,
+	userAPI userapi.ClientUserAPI,
 	accessToken string,
 	accessTokenErr error,
 ) util.JSONResponse {
@@ -785,7 +785,7 @@ func handleApplicationServiceRegistration(
 	req *http.Request,
 	r registerRequest,
 	cfg *config.ClientAPI,
-	userAPI userapi.UserRegisterAPI,
+	userAPI userapi.ClientUserAPI,
 ) util.JSONResponse {
 	// Check if we previously had issues extracting the access token from the
 	// request.
@@ -823,7 +823,7 @@ func checkAndCompleteFlow(
 	r registerRequest,
 	sessionID string,
 	cfg *config.ClientAPI,
-	userAPI userapi.UserRegisterAPI,
+	userAPI userapi.ClientUserAPI,
 ) util.JSONResponse {
 	if checkFlowCompleted(flow, cfg.Derived.Registration.Flows) {
 		// This flow was completed, registration can continue
@@ -850,7 +850,7 @@ func checkAndCompleteFlow(
 // not all
 func completeRegistration(
 	ctx context.Context,
-	userAPI userapi.UserRegisterAPI,
+	userAPI userapi.ClientUserAPI,
 	username, password, appserviceID, ipAddr, userAgent, sessionID string,
 	inhibitLogin eventutil.WeakBoolean,
 	displayName, deviceID *string,
@@ -1009,7 +1009,7 @@ type availableResponse struct {
 func RegisterAvailable(
 	req *http.Request,
 	cfg *config.ClientAPI,
-	registerAPI userapi.UserRegisterAPI,
+	registerAPI userapi.ClientUserAPI,
 ) util.JSONResponse {
 	username := req.URL.Query().Get("username")
 
@@ -1057,7 +1057,7 @@ func RegisterAvailable(
 	}
 }
 
-func handleSharedSecretRegistration(userAPI userapi.UserInternalAPI, sr *SharedSecretRegistration, req *http.Request) util.JSONResponse {
+func handleSharedSecretRegistration(userAPI userapi.ClientUserAPI, sr *SharedSecretRegistration, req *http.Request) util.JSONResponse {
 	ssrr, err := NewSharedSecretRegistrationRequest(req.Body)
 	if err != nil {
 		return util.JSONResponse{

clientapi/routing/register_publickey.go

@@ -36,7 +36,7 @@ func newPublicKeyAuthSession(request *registerRequest) {
 func handlePublicKeyRegistration(
 	cfg *config.ClientAPI,
 	reqBytes []byte,
-	userAPI userapi.UserRegisterAPI,
+	userAPI userapi.ClientUserAPI,
 ) (bool, authtypes.LoginType, *util.JSONResponse) {
 	if !cfg.PublicKeyAuthentication.Enabled() {
 		return false, "", &util.JSONResponse{

clientapi/routing/room_tagging.go

@@ -31,7 +31,7 @@ import (
 // GetTags implements GET /_matrix/client/r0/user/{userID}/rooms/{roomID}/tags
 func GetTags(
 	req *http.Request,
-	userAPI api.UserInternalAPI,
+	userAPI api.ClientUserAPI,
 	device *api.Device,
 	userID string,
 	roomID string,
@@ -62,7 +62,7 @@ func GetTags(
 // the tag to the "map" and saving the new "map" to the DB
 func PutTag(
 	req *http.Request,
-	userAPI api.UserInternalAPI,
+	userAPI api.ClientUserAPI,
 	device *api.Device,
 	userID string,
 	roomID string,
@@ -113,7 +113,7 @@ func PutTag(
 // the "map" and then saving the new "map" in the DB
 func DeleteTag(
 	req *http.Request,
-	userAPI api.UserInternalAPI,
+	userAPI api.ClientUserAPI,
 	device *api.Device,
 	userID string,
 	roomID string,
@@ -167,7 +167,7 @@ func obtainSavedTags(
 	req *http.Request,
 	userID string,
 	roomID string,
-	userAPI api.UserInternalAPI,
+	userAPI api.ClientUserAPI,
 ) (tags gomatrix.TagContent, err error) {
 	dataReq := api.QueryAccountDataRequest{
 		UserID:   userID,
@@ -194,7 +194,7 @@ func saveTagData(
 	req *http.Request,
 	userID string,
 	roomID string,
-	userAPI api.UserInternalAPI,
+	userAPI api.ClientUserAPI,
 	Tag gomatrix.TagContent,
 ) error {
 	newTagData, err := json.Marshal(Tag)

clientapi/routing/routing.go

@@ -50,15 +50,15 @@ import (
 func Setup(
 	publicAPIMux, synapseAdminRouter, dendriteAdminRouter *mux.Router,
 	cfg *config.ClientAPI,
-	rsAPI roomserverAPI.RoomserverInternalAPI,
-	asAPI appserviceAPI.AppServiceQueryAPI,
-	userAPI userapi.UserInternalAPI,
-	userDirectoryProvider userapi.UserDirectoryProvider,
+	rsAPI roomserverAPI.ClientRoomserverAPI,
+	asAPI appserviceAPI.AppServiceInternalAPI,
+	userAPI userapi.ClientUserAPI,
+	userDirectoryProvider userapi.QuerySearchProfilesAPI,
 	federation *gomatrixserverlib.FederationClient,
 	syncProducer *producers.SyncAPIProducer,
 	transactionsCache *transactions.Cache,
-	federationSender federationAPI.FederationInternalAPI,
-	keyAPI keyserverAPI.KeyInternalAPI,
+	federationSender federationAPI.ClientFederationAPI,
+	keyAPI keyserverAPI.ClientKeyAPI,
 	extRoomsProvider api.ExtraPublicRoomsProvider,
 	mscCfg *config.MSCs, natsClient *nats.Conn,
 ) {
@@ -325,7 +325,7 @@ func Setup(
 			if err != nil {
 				return util.ErrorResponse(err)
 			}
-			return GetEvent(req, device, vars["roomID"], vars["eventID"], cfg, rsAPI, federation)
+			return GetEvent(req, device, vars["roomID"], vars["eventID"], cfg, rsAPI)
 		}),
 	).Methods(http.MethodGet, http.MethodOptions)
 
@@ -897,7 +897,7 @@ func Setup(
 			if resErr := clientutil.UnmarshalJSONRequest(req, &postContent); resErr != nil {
 				return *resErr
 			}
-			return *SearchUserDirectory(
+			return SearchUserDirectory(
 				req.Context(),
 				device,
 				userAPI,

clientapi/routing/sendevent.go

@@ -70,7 +70,7 @@ func SendEvent(
 	device *userapi.Device,
 	roomID, eventType string, txnID, stateKey *string,
 	cfg *config.ClientAPI,
-	rsAPI api.RoomserverInternalAPI,
+	rsAPI api.ClientRoomserverAPI,
 	txnCache *transactions.Cache,
 ) util.JSONResponse {
 	verReq := api.QueryRoomVersionForRoomRequest{RoomID: roomID}
@@ -207,7 +207,7 @@ func generateSendEvent(
 	device *userapi.Device,
 	roomID, eventType string, stateKey *string,
 	cfg *config.ClientAPI,
-	rsAPI api.RoomserverInternalAPI,
+	rsAPI api.ClientRoomserverAPI,
 	evTime time.Time,
 ) (*gomatrixserverlib.Event, *util.JSONResponse) {
 	// parse the incoming http request

clientapi/routing/sendtyping.go

@@ -32,7 +32,7 @@ type typingContentJSON struct {
 // sends the typing events to client API typingProducer
 func SendTyping(
 	req *http.Request, device *userapi.Device, roomID string,
-	userID string, rsAPI roomserverAPI.RoomserverInternalAPI,
+	userID string, rsAPI roomserverAPI.ClientRoomserverAPI,
 	syncProducer *producers.SyncAPIProducer,
 ) util.JSONResponse {
 	if device.UserID != userID {

clientapi/routing/server_notices.go

@@ -21,6 +21,7 @@ import (
 	"net/http"
 	"time"
 
+	"github.com/matrix-org/dendrite/roomserver/version"
 	"github.com/matrix-org/gomatrix"
 	"github.com/matrix-org/gomatrixserverlib"
 	"github.com/matrix-org/gomatrixserverlib/tokens"
@@ -55,9 +56,9 @@ func SendServerNotice(
 	req *http.Request,
 	cfgNotices *config.ServerNotices,
 	cfgClient *config.ClientAPI,
-	userAPI userapi.UserInternalAPI,
-	rsAPI api.RoomserverInternalAPI,
-	asAPI appserviceAPI.AppServiceQueryAPI,
+	userAPI userapi.ClientUserAPI,
+	rsAPI api.ClientRoomserverAPI,
+	asAPI appserviceAPI.AppServiceInternalAPI,
 	device *userapi.Device,
 	senderDevice *userapi.Device,
 	txnID *string,
@@ -95,29 +96,16 @@ func SendServerNotice(
 	// get rooms for specified user
 	allUserRooms := []string{}
 	userRooms := api.QueryRoomsForUserResponse{}
-	if err := rsAPI.QueryRoomsForUser(ctx, &api.QueryRoomsForUserRequest{
-		UserID:         r.UserID,
-		WantMembership: "join",
-	}, &userRooms); err != nil {
-		return util.ErrorResponse(err)
+	// Get rooms the user is either joined, invited or has left.
+	for _, membership := range []string{"join", "invite", "leave"} {
+		if err := rsAPI.QueryRoomsForUser(ctx, &api.QueryRoomsForUserRequest{
+			UserID:         r.UserID,
+			WantMembership: membership,
+		}, &userRooms); err != nil {
+			return util.ErrorResponse(err)
+		}
+		allUserRooms = append(allUserRooms, userRooms.RoomIDs...)
 	}
-	allUserRooms = append(allUserRooms, userRooms.RoomIDs...)
-	// get invites for specified user
-	if err := rsAPI.QueryRoomsForUser(ctx, &api.QueryRoomsForUserRequest{
-		UserID:         r.UserID,
-		WantMembership: "invite",
-	}, &userRooms); err != nil {
-		return util.ErrorResponse(err)
-	}
-	allUserRooms = append(allUserRooms, userRooms.RoomIDs...)
-	// get left rooms for specified user
-	if err := rsAPI.QueryRoomsForUser(ctx, &api.QueryRoomsForUserRequest{
-		UserID:         r.UserID,
-		WantMembership: "leave",
-	}, &userRooms); err != nil {
-		return util.ErrorResponse(err)
-	}
-	allUserRooms = append(allUserRooms, userRooms.RoomIDs...)
 
 	// get rooms of the sender
 	senderUserID := fmt.Sprintf("@%s:%s", cfgNotices.LocalPart, cfgClient.Matrix.ServerName)
@@ -145,7 +133,7 @@ func SendServerNotice(
 
 	var (
 		roomID      string
-		roomVersion = gomatrixserverlib.RoomVersionV6
+		roomVersion = version.DefaultRoomVersion()
 	)
 
 	// create a new room for the user
@@ -194,14 +182,21 @@ func SendServerNotice(
 			// if we didn't get a createRoomResponse, we probably received an error, so return that.
 			return roomRes
 		}
-
 	} else {
 		// we've found a room in common, check the membership
 		roomID = commonRooms[0]
-		// re-invite the user
-		res, err := sendInvite(ctx, userAPI, senderDevice, roomID, r.UserID, "Server notice room", cfgClient, rsAPI, asAPI, time.Now())
+		membershipRes := api.QueryMembershipForUserResponse{}
+		err := rsAPI.QueryMembershipForUser(ctx, &api.QueryMembershipForUserRequest{UserID: r.UserID, RoomID: roomID}, &membershipRes)
 		if err != nil {
-			return res
+			util.GetLogger(ctx).WithError(err).Error("unable to query membership for user")
+			return jsonerror.InternalServerError()
+		}
+		if !membershipRes.IsInRoom {
+			// re-invite the user
+			res, err := sendInvite(ctx, userAPI, senderDevice, roomID, r.UserID, "Server notice room", cfgClient, rsAPI, asAPI, time.Now())
+			if err != nil {
+				return res
+			}
 		}
 	}
 
@@ -281,7 +276,7 @@ func (r sendServerNoticeRequest) valid() (ok bool) {
 // It returns an userapi.Device, which is used for building the event
 func getSenderDevice(
 	ctx context.Context,
-	userAPI userapi.UserInternalAPI,
+	userAPI userapi.ClientUserAPI,
 	cfg *config.ClientAPI,
 ) (*userapi.Device, error) {
 	var accRes userapi.PerformAccountCreationResponse

clientapi/routing/state.go

@@ -41,7 +41,7 @@ type stateEventInStateResp struct {
 // TODO: Check if the user is in the room. If not, check if the room's history
 // is publicly visible. Current behaviour is returning an empty array if the
 // user cannot see the room's history.
-func OnIncomingStateRequest(ctx context.Context, device *userapi.Device, rsAPI api.RoomserverInternalAPI, roomID string) util.JSONResponse {
+func OnIncomingStateRequest(ctx context.Context, device *userapi.Device, rsAPI api.ClientRoomserverAPI, roomID string) util.JSONResponse {
 	var worldReadable bool
 	var wantLatestState bool
 
@@ -56,6 +56,12 @@ func OnIncomingStateRequest(ctx context.Context, device *userapi.Device, rsAPI a
 		util.GetLogger(ctx).WithError(err).Error("queryAPI.QueryLatestEventsAndState failed")
 		return jsonerror.InternalServerError()
 	}
+	if !stateRes.RoomExists {
+		return util.JSONResponse{
+			Code: http.StatusForbidden,
+			JSON: jsonerror.Forbidden("room does not exist"),
+		}
+	}
 
 	// Look at the room state and see if we have a history visibility event
 	// that marks the room as world-readable. If we don't then we assume that
@@ -162,7 +168,7 @@ func OnIncomingStateRequest(ctx context.Context, device *userapi.Device, rsAPI a
 // is then (by default) we return the content, otherwise a 404.
 // If eventFormat=true, sends the whole event else just the content.
 func OnIncomingStateTypeRequest(
-	ctx context.Context, device *userapi.Device, rsAPI api.RoomserverInternalAPI,
+	ctx context.Context, device *userapi.Device, rsAPI api.ClientRoomserverAPI,
 	roomID, evType, stateKey string, eventFormat bool,
 ) util.JSONResponse {
 	var worldReadable bool

clientapi/routing/threepid.go

@@ -40,7 +40,7 @@ type threePIDsResponse struct {
 // RequestEmailToken implements:
 //     POST /account/3pid/email/requestToken
 //     POST /register/email/requestToken
-func RequestEmailToken(req *http.Request, threePIDAPI api.UserThreePIDAPI, cfg *config.ClientAPI) util.JSONResponse {
+func RequestEmailToken(req *http.Request, threePIDAPI api.ClientUserAPI, cfg *config.ClientAPI) util.JSONResponse {
 	var body threepid.EmailAssociationRequest
 	if reqErr := httputil.UnmarshalJSONRequest(req, &body); reqErr != nil {
 		return *reqErr
@@ -90,7 +90,7 @@ func RequestEmailToken(req *http.Request, threePIDAPI api.UserThreePIDAPI, cfg *
 
 // CheckAndSave3PIDAssociation implements POST /account/3pid
 func CheckAndSave3PIDAssociation(
-	req *http.Request, threePIDAPI api.UserThreePIDAPI, device *api.Device,
+	req *http.Request, threePIDAPI api.ClientUserAPI, device *api.Device,
 	cfg *config.ClientAPI,
 ) util.JSONResponse {
 	var body threepid.EmailAssociationCheckRequest
@@ -158,7 +158,7 @@ func CheckAndSave3PIDAssociation(
 
 // GetAssociated3PIDs implements GET /account/3pid
 func GetAssociated3PIDs(
-	req *http.Request, threepidAPI api.UserThreePIDAPI, device *api.Device,
+	req *http.Request, threepidAPI api.ClientUserAPI, device *api.Device,
 ) util.JSONResponse {
 	localpart, _, err := gomatrixserverlib.SplitID('@', device.UserID)
 	if err != nil {
@@ -182,7 +182,7 @@ func GetAssociated3PIDs(
 }
 
 // Forget3PID implements POST /account/3pid/delete
-func Forget3PID(req *http.Request, threepidAPI api.UserThreePIDAPI) util.JSONResponse {
+func Forget3PID(req *http.Request, threepidAPI api.ClientUserAPI) util.JSONResponse {
 	var body authtypes.ThreePID
 	if reqErr := httputil.UnmarshalJSONRequest(req, &body); reqErr != nil {
 		return *reqErr

clientapi/routing/upgrade_room.go

@@ -40,9 +40,9 @@ type upgradeRoomResponse struct {
 func UpgradeRoom(
 	req *http.Request, device *userapi.Device,
 	cfg *config.ClientAPI,
-	roomID string, profileAPI userapi.UserProfileAPI,
-	rsAPI roomserverAPI.RoomserverInternalAPI,
-	asAPI appserviceAPI.AppServiceQueryAPI,
+	roomID string, profileAPI userapi.ClientUserAPI,
+	rsAPI roomserverAPI.ClientRoomserverAPI,
+	asAPI appserviceAPI.AppServiceInternalAPI,
 ) util.JSONResponse {
 	var r upgradeRoomRequest
 	if rErr := httputil.UnmarshalJSONRequest(req, &r); rErr != nil {

clientapi/routing/userdirectory.go

@@ -34,13 +34,13 @@ type UserDirectoryResponse struct {
 func SearchUserDirectory(
 	ctx context.Context,
 	device *userapi.Device,
-	userAPI userapi.UserInternalAPI,
-	rsAPI api.RoomserverInternalAPI,
-	provider userapi.UserDirectoryProvider,
+	userAPI userapi.ClientUserAPI,
+	rsAPI api.ClientRoomserverAPI,
+	provider userapi.QuerySearchProfilesAPI,
 	serverName gomatrixserverlib.ServerName,
 	searchString string,
 	limit int,
-) *util.JSONResponse {
+) util.JSONResponse {
 	if limit < 10 {
 		limit = 10
 	}
@@ -58,8 +58,7 @@ func SearchUserDirectory(
 	}
 	userRes := &userapi.QuerySearchProfilesResponse{}
 	if err := provider.QuerySearchProfiles(ctx, userReq, userRes); err != nil {
-		errRes := util.ErrorResponse(fmt.Errorf("userAPI.QuerySearchProfiles: %w", err))
-		return &errRes
+		return util.ErrorResponse(fmt.Errorf("userAPI.QuerySearchProfiles: %w", err))
 	}
 
 	for _, user := range userRes.Profiles {
@@ -94,8 +93,7 @@ func SearchUserDirectory(
 		}
 		stateRes := &api.QueryKnownUsersResponse{}
 		if err := rsAPI.QueryKnownUsers(ctx, stateReq, stateRes); err != nil && err != sql.ErrNoRows {
-			errRes := util.ErrorResponse(fmt.Errorf("rsAPI.QueryKnownUsers: %w", err))
-			return &errRes
+			return util.ErrorResponse(fmt.Errorf("rsAPI.QueryKnownUsers: %w", err))
 		}
 
 		for _, user := range stateRes.Users {
@@ -114,7 +112,7 @@ func SearchUserDirectory(
 		response.Results = append(response.Results, result)
 	}
 
-	return &util.JSONResponse{
+	return util.JSONResponse{
 		Code: 200,
 		JSON: response,
 	}

clientapi/threepid/invites.go

@@ -86,7 +86,7 @@ var (
 func CheckAndProcessInvite(
 	ctx context.Context,
 	device *userapi.Device, body *MembershipRequest, cfg *config.ClientAPI,
-	rsAPI api.RoomserverInternalAPI, db userapi.UserProfileAPI,
+	rsAPI api.ClientRoomserverAPI, db userapi.ClientUserAPI,
 	roomID string,
 	evTime time.Time,
 ) (inviteStoredOnIDServer bool, err error) {
@@ -136,7 +136,7 @@ func CheckAndProcessInvite(
 // Returns an error if a check or a request failed.
 func queryIDServer(
 	ctx context.Context,
-	db userapi.UserProfileAPI, cfg *config.ClientAPI, device *userapi.Device,
+	userAPI userapi.ClientUserAPI, cfg *config.ClientAPI, device *userapi.Device,
 	body *MembershipRequest, roomID string,
 ) (lookupRes *idServerLookupResponse, storeInviteRes *idServerStoreInviteResponse, err error) {
 	if err = isTrusted(body.IDServer, cfg); err != nil {
@@ -152,7 +152,7 @@ func queryIDServer(
 	if lookupRes.MXID == "" {
 		// No Matrix ID matches with the given 3PID, ask the server to store the
 		// invite and return a token
-		storeInviteRes, err = queryIDServerStoreInvite(ctx, db, cfg, device, body, roomID)
+		storeInviteRes, err = queryIDServerStoreInvite(ctx, userAPI, cfg, device, body, roomID)
 		return
 	}
 
@@ -163,7 +163,7 @@ func queryIDServer(
 	if lookupRes.NotBefore > now || now > lookupRes.NotAfter {
 		// If the current timestamp isn't in the time frame in which the association
 		// is known to be valid, re-run the query
-		return queryIDServer(ctx, db, cfg, device, body, roomID)
+		return queryIDServer(ctx, userAPI, cfg, device, body, roomID)
 	}
 
 	// Check the request signatures and send an error if one isn't valid
@@ -205,7 +205,7 @@ func queryIDServerLookup(ctx context.Context, body *MembershipRequest) (*idServe
 // Returns an error if the request failed to send or if the response couldn't be parsed.
 func queryIDServerStoreInvite(
 	ctx context.Context,
-	db userapi.UserProfileAPI, cfg *config.ClientAPI, device *userapi.Device,
+	userAPI userapi.ClientUserAPI, cfg *config.ClientAPI, device *userapi.Device,
 	body *MembershipRequest, roomID string,
 ) (*idServerStoreInviteResponse, error) {
 	// Retrieve the sender's profile to get their display name
@@ -217,7 +217,7 @@ func queryIDServerStoreInvite(
 	var profile *authtypes.Profile
 	if serverName == cfg.Matrix.ServerName {
 		res := &userapi.QueryProfileResponse{}
-		err = db.QueryProfile(ctx, &userapi.QueryProfileRequest{UserID: device.UserID}, res)
+		err = userAPI.QueryProfile(ctx, &userapi.QueryProfileRequest{UserID: device.UserID}, res)
 		if err != nil {
 			return nil, err
 		}
@@ -337,7 +337,7 @@ func emit3PIDInviteEvent(
 	ctx context.Context,
 	body *MembershipRequest, res *idServerStoreInviteResponse,
 	device *userapi.Device, roomID string, cfg *config.ClientAPI,
-	rsAPI api.RoomserverInternalAPI,
+	rsAPI api.ClientRoomserverAPI,
 	evTime time.Time,
 ) error {
 	builder := &gomatrixserverlib.EventBuilder{

cmd/create-account/main.go

@@ -27,6 +27,7 @@ import (
 	"github.com/matrix-org/dendrite/setup"
 	"github.com/matrix-org/dendrite/setup/base"
 	"github.com/matrix-org/dendrite/userapi/api"
+	"github.com/matrix-org/dendrite/userapi/storage"
 	"github.com/sirupsen/logrus"
 	"golang.org/x/term"
 )
@@ -99,8 +100,24 @@ func main() {
 		}
 	}
 
-	b := base.NewBaseDendrite(cfg, "Monolith")
-	accountDB := b.CreateAccountsDB()
+	// avoid warning about open registration
+	cfg.ClientAPI.RegistrationDisabled = true
+
+	b := base.NewBaseDendrite(cfg, "")
+	defer b.Close() // nolint: errcheck
+
+	accountDB, err := storage.NewUserAPIDatabase(
+		b,
+		&cfg.UserAPI.AccountDatabase,
+		cfg.Global.ServerName,
+		cfg.UserAPI.BCryptCost,
+		cfg.UserAPI.OpenIDTokenLifetimeMS,
+		0, // TODO
+		cfg.Global.ServerNotices.LocalPart,
+	)
+	if err != nil {
+		logrus.WithError(err).Fatalln("Failed to connect to the database")
+	}
 
 	accType := api.AccountTypeUser
 	if *isAdmin {

cmd/dendrite-demo-pinecone/main.go

@@ -149,7 +149,6 @@ func main() {
 	base := base.NewBaseDendrite(cfg, "Monolith")
 	defer base.Close() // nolint: errcheck
 
-	accountDB := base.CreateAccountsDB()
 	federation := conn.CreateFederationClient(base, pQUIC)
 
 	serverKeyAPI := &signing.YggdrasilKeys{}
@@ -162,7 +161,7 @@ func main() {
 	)
 
 	keyAPI := keyserver.NewInternalAPI(base, &base.Cfg.KeyServer, fsAPI)
-	userAPI := userapi.NewInternalAPI(base, accountDB, &cfg.UserAPI, nil, keyAPI, rsAPI, base.PushGatewayHTTPClient())
+	userAPI := userapi.NewInternalAPI(base, &cfg.UserAPI, nil, keyAPI, rsAPI, base.PushGatewayHTTPClient())
 	keyAPI.SetUserAPI(userAPI)
 
 	asAPI := appservice.NewInternalAPI(base, userAPI, rsAPI)
@@ -174,7 +173,6 @@ func main() {
 
 	monolith := setup.Monolith{
 		Config:    base.Cfg,
-		AccountDB: accountDB,
 		Client:    conn.CreateClient(base, pQUIC),
 		FedClient: federation,
 		KeyRing:   keyRing,
@@ -187,16 +185,7 @@ func main() {
 		ExtPublicRoomsProvider:   roomProvider,
 		ExtUserDirectoryProvider: userProvider,
 	}
-	monolith.AddAllPublicRoutes(
-		base.ProcessContext,
-		base.PublicClientAPIMux,
-		base.PublicFederationAPIMux,
-		base.PublicKeyAPIMux,
-		base.PublicWellKnownAPIMux,
-		base.PublicMediaAPIMux,
-		base.SynapseAdminMux,
-		base.DendriteAdminMux,
-	)
+	monolith.AddAllPublicRoutes(base)
 
 	wsUpgrader := websocket.Upgrader{
 		CheckOrigin: func(_ *http.Request) bool {

cmd/dendrite-demo-pinecone/users/users.go

@@ -37,7 +37,7 @@ import (
 type PineconeUserProvider struct {
 	r         *pineconeRouter.Router
 	s         *pineconeSessions.Sessions
-	userAPI   userapi.UserProfileAPI
+	userAPI   userapi.QuerySearchProfilesAPI
 	fedClient *gomatrixserverlib.FederationClient
 }
 
@@ -46,7 +46,7 @@ const PublicURL = "/_matrix/p2p/profiles"
 func NewPineconeUserProvider(
 	r *pineconeRouter.Router,
 	s *pineconeSessions.Sessions,
-	userAPI userapi.UserProfileAPI,
+	userAPI userapi.QuerySearchProfilesAPI,
 	fedClient *gomatrixserverlib.FederationClient,
 ) *PineconeUserProvider {
 	p := &PineconeUserProvider{

cmd/dendrite-demo-yggdrasil/main.go

@@ -104,7 +104,6 @@ func main() {
 	base := base.NewBaseDendrite(cfg, "Monolith")
 	defer base.Close() // nolint: errcheck
 
-	accountDB := base.CreateAccountsDB()
 	federation := ygg.CreateFederationClient(base)
 
 	serverKeyAPI := &signing.YggdrasilKeys{}
@@ -117,7 +116,7 @@ func main() {
 	)
 	rsAPI := rsComponent
 
-	userAPI := userapi.NewInternalAPI(base, accountDB, &cfg.UserAPI, nil, keyAPI, rsAPI, base.PushGatewayHTTPClient())
+	userAPI := userapi.NewInternalAPI(base, &cfg.UserAPI, nil, keyAPI, rsAPI, base.PushGatewayHTTPClient())
 	keyAPI.SetUserAPI(userAPI)
 
 	asAPI := appservice.NewInternalAPI(base, userAPI, rsAPI)
@@ -130,7 +129,6 @@ func main() {
 
 	monolith := setup.Monolith{
 		Config:    base.Cfg,
-		AccountDB: accountDB,
 		Client:    ygg.CreateClient(base),
 		FedClient: federation,
 		KeyRing:   keyRing,
@@ -144,16 +142,7 @@ func main() {
 			ygg, fsAPI, federation,
 		),
 	}
-	monolith.AddAllPublicRoutes(
-		base.ProcessContext,
-		base.PublicClientAPIMux,
-		base.PublicFederationAPIMux,
-		base.PublicKeyAPIMux,
-		base.PublicWellKnownAPIMux,
-		base.PublicMediaAPIMux,
-		base.SynapseAdminMux,
-		base.DendriteAdminMux,
-	)
+	monolith.AddAllPublicRoutes(base)
 	if err := mscs.Enable(base, &monolith); err != nil {
 		logrus.WithError(err).Fatalf("Failed to enable MSCs")
 	}

cmd/dendrite-monolith-server/main.go

@@ -71,7 +71,6 @@ func main() {
 	base := basepkg.NewBaseDendrite(cfg, "Monolith", options...)
 	defer base.Close() // nolint: errcheck
 
-	accountDB := base.CreateAccountsDB()
 	federation := base.CreateFederationClient()
 
 	rsImpl := roomserver.NewInternalAPI(base)
@@ -90,6 +89,7 @@ func main() {
 	fsAPI := federationapi.NewInternalAPI(
 		base, federation, rsAPI, base.Caches, nil, false,
 	)
+	fsImplAPI := fsAPI
 	if base.UseHTTPAPIs {
 		federationapi.AddInternalRoutes(base.InternalAPIMux, fsAPI)
 		fsAPI = base.FederationAPIHTTPClient()
@@ -104,7 +104,7 @@ func main() {
 	}
 
 	pgClient := base.PushGatewayHTTPClient()
-	userImpl := userapi.NewInternalAPI(base, accountDB, &cfg.UserAPI, cfg.Derived.ApplicationServices, keyAPI, rsAPI, pgClient)
+	userImpl := userapi.NewInternalAPI(base, &cfg.UserAPI, cfg.Derived.ApplicationServices, keyAPI, rsAPI, pgClient)
 	userAPI := userImpl
 	if base.UseHTTPAPIs {
 		userapi.AddInternalRoutes(base.InternalAPIMux, userAPI)
@@ -135,26 +135,19 @@ func main() {
 
 	monolith := setup.Monolith{
 		Config:    base.Cfg,
-		AccountDB: accountDB,
 		Client:    base.CreateClient(),
 		FedClient: federation,
 		KeyRing:   keyRing,
 
-		AppserviceAPI: asAPI, FederationAPI: fsAPI,
+		AppserviceAPI: asAPI,
+		// always use the concrete impl here even in -http mode because adding public routes
+		// must be done on the concrete impl not an HTTP client else fedapi will call itself
+		FederationAPI: fsImplAPI,
 		RoomserverAPI: rsAPI,
 		UserAPI:       userAPI,
 		KeyAPI:        keyAPI,
 	}
-	monolith.AddAllPublicRoutes(
-		base.ProcessContext,
-		base.PublicClientAPIMux,
-		base.PublicFederationAPIMux,
-		base.PublicKeyAPIMux,
-		base.PublicWellKnownAPIMux,
-		base.PublicMediaAPIMux,
-		base.SynapseAdminMux,
-		base.DendriteAdminMux,
-	)
+	monolith.AddAllPublicRoutes(base)
 
 	if len(base.Cfg.MSCs.MSCs) > 0 {
 		if err := mscs.Enable(base, &monolith); err != nil {

cmd/dendrite-polylith-multi/main.go

@@ -31,7 +31,7 @@ import (
 type entrypoint func(base *base.BaseDendrite, cfg *config.Dendrite)
 
 func main() {
-	cfg := setup.ParseFlags(true)
+	cfg := setup.ParseFlags(false)
 
 	component := ""
 	if flag.NFlag() > 0 {
@@ -71,8 +71,8 @@ func main() {
 
 	logrus.Infof("Starting %q component", component)
 
-	base := base.NewBaseDendrite(cfg, component) // TODO
-	defer base.Close()                           // nolint: errcheck
+	base := base.NewBaseDendrite(cfg, component, base.PolylithMode) // TODO
+	defer base.Close()                                              // nolint: errcheck
 
 	go start(base, cfg)
 	base.WaitForShutdown()

cmd/dendrite-polylith-multi/personalities/clientapi.go

@@ -31,11 +31,9 @@ func ClientAPI(base *basepkg.BaseDendrite, cfg *config.Dendrite) {
 	keyAPI := base.KeyServerHTTPClient()
 
 	clientapi.AddPublicRoutes(
-		base.ProcessContext, base.PublicClientAPIMux,
-		base.SynapseAdminMux, base.DendriteAdminMux,
-		&base.Cfg.ClientAPI, federation, rsAPI, asQuery,
+		base, federation, rsAPI, asQuery,
 		transactions.New(), fsAPI, userAPI, userAPI,
-		keyAPI, nil, &cfg.MSCs,
+		keyAPI, nil,
 	)
 
 	base.SetupAndServeHTTP(

cmd/dendrite-polylith-multi/personalities/federationapi.go

@@ -29,10 +29,9 @@ func FederationAPI(base *basepkg.BaseDendrite, cfg *config.Dendrite) {
 	keyRing := fsAPI.KeyRing()
 
 	federationapi.AddPublicRoutes(
-		base.ProcessContext, base.PublicFederationAPIMux, base.PublicKeyAPIMux, base.PublicWellKnownAPIMux,
-		&base.Cfg.FederationAPI, userAPI, federation, keyRing,
-		rsAPI, fsAPI, keyAPI,
-		&base.Cfg.MSCs, nil,
+		base,
+		userAPI, federation, keyRing,
+		rsAPI, fsAPI, keyAPI, nil,
 	)
 
 	federationapi.AddInternalRoutes(base.InternalAPIMux, fsAPI)

cmd/dendrite-polylith-multi/personalities/mediaapi.go

@@ -24,7 +24,9 @@ func MediaAPI(base *basepkg.BaseDendrite, cfg *config.Dendrite) {
 	userAPI := base.UserAPIClient()
 	client := base.CreateClient()
 
-	mediaapi.AddPublicRoutes(base.PublicMediaAPIMux, &base.Cfg.MediaAPI, &base.Cfg.ClientAPI.RateLimiting, userAPI, client)
+	mediaapi.AddPublicRoutes(
+		base, userAPI, client,
+	)
 
 	base.SetupAndServeHTTP(
 		base.Cfg.MediaAPI.InternalAPI.Listen,

cmd/dendrite-polylith-multi/personalities/syncapi.go

@@ -22,15 +22,13 @@ import (
 
 func SyncAPI(base *basepkg.BaseDendrite, cfg *config.Dendrite) {
 	userAPI := base.UserAPIClient()
-	federation := base.CreateFederationClient()
 
 	rsAPI := base.RoomserverHTTPClient()
 
 	syncapi.AddPublicRoutes(
-		base.ProcessContext,
-		base.PublicClientAPIMux, userAPI, rsAPI,
+		base,
+		userAPI, rsAPI,
 		base.KeyServerHTTPClient(),
-		federation, &cfg.SyncAPI,
 	)
 
 	base.SetupAndServeHTTP(

cmd/dendrite-polylith-multi/personalities/userapi.go

@@ -21,10 +21,8 @@ import (
 )
 
 func UserAPI(base *basepkg.BaseDendrite, cfg *config.Dendrite) {
-	accountDB := base.CreateAccountsDB()
-
 	userAPI := userapi.NewInternalAPI(
-		base, accountDB, &cfg.UserAPI, cfg.Derived.ApplicationServices,
+		base, &cfg.UserAPI, cfg.Derived.ApplicationServices,
 		base.KeyServerHTTPClient(), base.RoomserverHTTPClient(),
 		base.PushGatewayHTTPClient(),
 	)

cmd/dendritejs-pinecone/main.go

@@ -180,7 +180,6 @@ func startup() {
 	base := base.NewBaseDendrite(cfg, "Monolith")
 	defer base.Close() // nolint: errcheck
 
-	accountDB := base.CreateAccountsDB()
 	federation := conn.CreateFederationClient(base, pSessions)
 	keyAPI := keyserver.NewInternalAPI(base, &base.Cfg.KeyServer, federation)
 
@@ -189,7 +188,7 @@ func startup() {
 
 	rsAPI := roomserver.NewInternalAPI(base)
 
-	userAPI := userapi.NewInternalAPI(base, accountDB, &cfg.UserAPI, nil, keyAPI, rsAPI, base.PushGatewayHTTPClient())
+	userAPI := userapi.NewInternalAPI(base, &cfg.UserAPI, nil, keyAPI, rsAPI, base.PushGatewayHTTPClient())
 	keyAPI.SetUserAPI(userAPI)
 
 	asQuery := appservice.NewInternalAPI(
@@ -201,7 +200,6 @@ func startup() {
 
 	monolith := setup.Monolith{
 		Config:    base.Cfg,
-		AccountDB: accountDB,
 		Client:    conn.CreateClient(base, pSessions),
 		FedClient: federation,
 		KeyRing:   keyRing,
@@ -214,16 +212,7 @@ func startup() {
 		//ServerKeyAPI:        serverKeyAPI,
 		ExtPublicRoomsProvider: rooms.NewPineconeRoomProvider(pRouter, pSessions, fedSenderAPI, federation),
 	}
-	monolith.AddAllPublicRoutes(
-		base.ProcessContext,
-		base.PublicClientAPIMux,
-		base.PublicFederationAPIMux,
-		base.PublicKeyAPIMux,
-		base.PublicWellKnownAPIMux,
-		base.PublicMediaAPIMux,
-		base.SynapseAdminMux,
-		base.DendriteAdminMux,
-	)
+	monolith.AddAllPublicRoutes(base)
 
 	httpRouter := mux.NewRouter().SkipClean(true).UseEncodedPath()
 	httpRouter.PathPrefix(httputil.InternalPathPrefix).Handler(base.InternalAPIMux)

cmd/resolve-state/main.go

@@ -45,7 +45,7 @@ func main() {
 		panic(err)
 	}
 
-	roomserverDB, err := storage.Open(&cfg.RoomServer.Database, cache)
+	roomserverDB, err := storage.Open(nil, &cfg.RoomServer.Database, cache)
 	if err != nil {
 		panic(err)
 	}

dendrite-sample.monolith.yaml

@@ -0,0 +1,279 @@
+# This is the Dendrite configuration file.
+#
+# The configuration is split up into sections - each Dendrite component has a
+# configuration section, in addition to the "global" section which applies to
+# all components.
+
+# The version of the configuration file.
+version: 2
+
+# Global Matrix configuration. This configuration applies to all components.
+global:
+  # The domain name of this homeserver.
+  server_name: localhost
+
+  # The path to the signing private key file, used to sign requests and events.
+  # Note that this is NOT the same private key as used for TLS! To generate a
+  # signing key, use "./bin/generate-keys --private-key matrix_key.pem".
+  private_key: matrix_key.pem
+
+  # The paths and expiry timestamps (as a UNIX timestamp in millisecond precision)
+  # to old signing private keys that were formerly in use on this domain. These
+  # keys will not be used for federation request or event signing, but will be
+  # provided to any other homeserver that asks when trying to verify old events.
+  old_private_keys:
+  #  - private_key: old_matrix_key.pem
+  #    expired_at: 1601024554498
+
+  # How long a remote server can cache our server signing key before requesting it
+  # again. Increasing this number will reduce the number of requests made by other
+  # servers for our key but increases the period that a compromised key will be
+  # considered valid by other homeservers.
+  key_validity_period: 168h0m0s
+
+  # Global database connection pool, for PostgreSQL monolith deployments only. If
+  # this section is populated then you can omit the "database" blocks in all other
+  # sections. For polylith deployments, or monolith deployments using SQLite databases,
+  # you must configure the "database" block for each component instead.
+  database:
+    connection_string: postgresql://username:password@hostname/dendrite?sslmode=disable
+    max_open_conns: 100
+    max_idle_conns: 5
+    conn_max_lifetime: -1
+
+  # The server name to delegate server-server communications to, with optional port
+  # e.g. localhost:443
+  well_known_server_name: ""
+
+  # Lists of domains that the server will trust as identity servers to verify third
+  # party identifiers such as phone numbers and email addresses.
+  trusted_third_party_id_servers:
+    - matrix.org
+    - vector.im
+
+  # Disables federation. Dendrite will not be able to communicate with other servers
+  # in the Matrix federation and the federation API will not be exposed.
+  disable_federation: false
+
+  # Configures the handling of presence events. Inbound controls whether we receive
+  # presence events from other servers, outbound controls whether we send presence
+  # events for our local users to other servers.
+  presence:
+    enable_inbound: false
+    enable_outbound: false
+
+  # Configures phone-home statistics reporting. These statistics contain the server
+  # name, number of active users and some information on your deployment config.
+  # We use this information to understand how Dendrite is being used in the wild.
+  report_stats:
+    enabled: false
+    endpoint: https://matrix.org/report-usage-stats/push
+
+  # Server notices allows server admins to send messages to all users on the server.
+  server_notices:
+    enabled: false
+    # The local part, display name and avatar URL (as a mxc:// URL) for the user that
+    # will send the server notices. These are visible to all users on the deployment.
+    local_part: "_server"
+    display_name: "Server Alerts"
+    avatar_url: ""
+    # The room name to be used when sending server notices. This room name will
+    # appear in user clients.
+    room_name: "Server Alerts"
+
+  # Configuration for NATS JetStream
+  jetstream:
+    # A list of NATS Server addresses to connect to. If none are specified, an
+    # internal NATS server will be started automatically when running Dendrite in
+    # monolith mode. For polylith deployments, it is required to specify the address
+    # of at least one NATS Server node.
+    addresses:
+    # - localhost:4222
+
+    # Persistent directory to store JetStream streams in. This directory should be
+    # preserved across Dendrite restarts.
+    storage_path: ./
+
+    # The prefix to use for stream names for this homeserver - really only useful
+    # if you are running more than one Dendrite server on the same NATS deployment.
+    topic_prefix: Dendrite
+
+  # Configuration for Prometheus metric collection.
+  metrics:
+    enabled: false
+    basic_auth:
+      username: metrics
+      password: metrics
+
+  # Optional DNS cache. The DNS cache may reduce the load on DNS servers if there
+  # is no local caching resolver available for use.
+  dns_cache:
+    enabled: false
+    cache_size: 256
+    cache_lifetime: "5m" # 5 minutes; https://pkg.go.dev/time@master#ParseDuration
+
+# Configuration for the Appservice API.
+app_service_api:
+  # Disable the validation of TLS certificates of appservices. This is
+  # not recommended in production since it may allow appservice traffic
+  # to be sent to an insecure endpoint.
+  disable_tls_validation: false
+
+  # Appservice configuration files to load into this homeserver.
+  config_files:
+  #  - /path/to/appservice_registration.yaml
+
+# Configuration for the Client API.
+client_api:
+  # Prevents new users from being able to register on this homeserver, except when
+  # using the registration shared secret below.
+  registration_disabled: true
+
+  # Prevents new guest accounts from being created. Guest registration is also
+  # disabled implicitly by setting 'registration_disabled' above.
+  guests_disabled: true
+
+  # If set, allows registration by anyone who knows the shared secret, regardless
+  # of whether registration is otherwise disabled.
+  registration_shared_secret: ""
+
+  # Whether to require reCAPTCHA for registration. If you have enabled registration
+  # then this is HIGHLY RECOMMENDED to reduce the risk of your homeserver being used
+  # for coordinated spam attacks.
+  enable_registration_captcha: false
+
+  # Settings for ReCAPTCHA.
+  recaptcha_public_key: ""
+  recaptcha_private_key: ""
+  recaptcha_bypass_secret: ""
+  recaptcha_siteverify_api: ""
+
+  # TURN server information that this homeserver should send to clients.
+  turn:
+    turn_user_lifetime: ""
+    turn_uris:
+    #  - turn:turn.server.org?transport=udp
+    #  - turn:turn.server.org?transport=tcp
+    turn_shared_secret: ""
+    turn_username: ""
+    turn_password: ""
+
+  # Settings for rate-limited endpoints. Rate limiting kicks in after the threshold
+  # number of "slots" have been taken by requests from a specific host. Each "slot"
+  # will be released after the cooloff time in milliseconds.
+  rate_limiting:
+    enabled: true
+    threshold: 5
+    cooloff_ms: 500
+
+# Configuration for the Federation API.
+federation_api:
+  # How many times we will try to resend a failed transaction to a specific server. The
+  # backoff is 2**x seconds, so 1 = 2 seconds, 2 = 4 seconds, 3 = 8 seconds etc. Once
+  # the max retries are exceeded, Dendrite will no longer try to send transactions to
+  # that server until it comes back to life and connects to us again.
+  send_max_retries: 16
+
+  # Disable the validation of TLS certificates of remote federated homeservers. Do not
+  # enable this option in production as it presents a security risk!
+  disable_tls_validation: false
+
+  # Perspective keyservers to use as a backup when direct key fetches fail. This may
+  # be required to satisfy key requests for servers that are no longer online when
+  # joining some rooms.
+  key_perspectives:
+    - server_name: matrix.org
+      keys:
+        - key_id: ed25519:auto
+          public_key: Noi6WqcDj0QmPxCNQqgezwTlBKrfqehY1u2FyWP9uYw
+        - key_id: ed25519:a_RXGa
+          public_key: l8Hft5qXKn1vfHrg3p4+W8gELQVo8N13JkluMfmn2sQ
+
+  # This option will control whether Dendrite will prefer to look up keys directly
+  # or whether it should try perspective servers first, using direct fetches as a
+  # last resort.
+  prefer_direct_fetch: false
+
+# Configuration for the Media API.
+media_api:
+  # Storage path for uploaded media. May be relative or absolute.
+  base_path: ./media_store
+
+  # The maximum allowed file size (in bytes) for media uploads to this homeserver
+  # (0 = unlimited). If using a reverse proxy, ensure it allows requests at least
+  #this large (e.g. the client_max_body_size setting in nginx).
+  max_file_size_bytes: 10485760
+
+  # Whether to dynamically generate thumbnails if needed.
+  dynamic_thumbnails: false
+
+  # The maximum number of simultaneous thumbnail generators to run.
+  max_thumbnail_generators: 10
+
+  # A list of thumbnail sizes to be generated for media content.
+  thumbnail_sizes:
+    - width: 32
+      height: 32
+      method: crop
+    - width: 96
+      height: 96
+      method: crop
+    - width: 640
+      height: 480
+      method: scale
+
+# Configuration for enabling experimental MSCs on this homeserver.
+mscs:
+  mscs:
+  #  - msc2836  # (Threading, see https://github.com/matrix-org/matrix-doc/pull/2836)
+  #  - msc2946  # (Spaces Summary, see https://github.com/matrix-org/matrix-doc/pull/2946)
+
+# Configuration for the Sync API.
+sync_api:
+  # This option controls which HTTP header to inspect to find the real remote IP
+  # address of the client. This is likely required if Dendrite is running behind
+  # a reverse proxy server.
+  # real_ip_header: X-Real-IP
+
+# Configuration for the User API.
+user_api:
+  # The cost when hashing passwords on registration/login. Default: 10. Min: 4, Max: 31
+  # See https://pkg.go.dev/golang.org/x/crypto/bcrypt for more information.
+  # Setting this lower makes registration/login consume less CPU resources at the cost
+  # of security should the database be compromised. Setting this higher makes registration/login
+  # consume more CPU resources but makes it harder to brute force password hashes. This value
+  # can be lowered if performing tests or on embedded Dendrite instances (e.g WASM builds).
+  bcrypt_cost: 10
+
+  # The length of time that a token issued for a relying party from
+  # /_matrix/client/r0/user/{userId}/openid/request_token endpoint
+  # is considered to be valid in milliseconds.
+  # The default lifetime is 3600000ms (60 minutes).
+  # openid_token_lifetime_ms: 3600000
+
+# Configuration for Opentracing.
+# See https://github.com/matrix-org/dendrite/tree/master/docs/tracing for information on
+# how this works and how to set it up.
+tracing:
+  enabled: false
+  jaeger:
+    serviceName: ""
+    disabled: false
+    rpc_metrics: false
+    tags: []
+    sampler: null
+    reporter: null
+    headers: null
+    baggage_restrictions: null
+    throttler: null
+
+# Logging configuration. The "std" logging type controls the logs being sent to
+# stdout. The "file" logging type controls logs being written to a log folder on
+# the disk. Supported log levels are "debug", "info", "warn", "error".
+logging:
+  - type: std
+    level: info
+  - type: file
+    level: info
+    params:
+      path: ./logs

dendrite-sample.polylith.yaml

@@ -3,29 +3,6 @@
 # The configuration is split up into sections - each Dendrite component has a
 # configuration section, in addition to the "global" section which applies to
 # all components.
-#
-# At a minimum, to get started, you will need to update the settings in the
-# "global" section for your deployment, and you will need to check that the
-# database "connection_string" line in each component section is correct.
-#
-# Each component with a "database" section can accept the following formats
-# for "connection_string":
-#   SQLite:     file:filename.db
-#               file:///path/to/filename.db
-#   PostgreSQL: postgresql://user:pass@hostname/database?params=...
-#
-# SQLite is embedded into Dendrite and therefore no further prerequisites are
-# needed for the database when using SQLite mode. However, performance with
-# PostgreSQL is significantly better and recommended for multi-user deployments.
-# SQLite is typically around 20-30% slower than PostgreSQL when tested with a
-# small number of users and likely will perform worse still with a higher volume
-# of users.
-#
-# The "max_open_conns" and "max_idle_conns" settings configure the maximum
-# number of open/idle database connections. The value 0 will use the database
-# engine default, and a negative value will use unlimited connections. The
-# "conn_max_lifetime" option controls the maximum length of time a database
-# connection can be idle in seconds - a negative value is unlimited.
 
 # The version of the configuration file.
 version: 2
@@ -44,9 +21,9 @@ global:
   # to old signing private keys that were formerly in use on this domain. These
   # keys will not be used for federation request or event signing, but will be
   # provided to any other homeserver that asks when trying to verify old events.
-  # old_private_keys:
-  # - private_key: old_matrix_key.pem
-  #   expired_at: 1601024554498
+  old_private_keys:
+  #  - private_key: old_matrix_key.pem
+  #    expired_at: 1601024554498
 
   # How long a remote server can cache our server signing key before requesting it
   # again. Increasing this number will reduce the number of requests made by other
@@ -64,96 +41,90 @@ global:
     - matrix.org
     - vector.im
 
-  # Disables federation. Dendrite will not be able to make any outbound HTTP requests
-  # to other servers and the federation API will not be exposed.
+  # Disables federation. Dendrite will not be able to communicate with other servers
+  # in the Matrix federation and the federation API will not be exposed.
   disable_federation: false
 
-  # Configures the handling of presence events.
+  # Configures the handling of presence events. Inbound controls whether we receive
+  # presence events from other servers, outbound controls whether we send presence
+  # events for our local users to other servers.
   presence:
-    # Whether inbound presence events are allowed, e.g. receiving presence events from other servers
     enable_inbound: false
-    # Whether outbound presence events are allowed, e.g. sending presence events to other servers
     enable_outbound: false
 
-  # Server notices allows server admins to send messages to all users.
+  # Configures phone-home statistics reporting. These statistics contain the server
+  # name, number of active users and some information on your deployment config.
+  # We use this information to understand how Dendrite is being used in the wild.
+  report_stats:
+    enabled: false
+    endpoint: https://matrix.org/report-usage-stats/push
+
+  # Server notices allows server admins to send messages to all users on the server.
   server_notices:
     enabled: false
-    # The server localpart to be used when sending notices, ensure this is not yet taken
+    # The local part, display name and avatar URL (as a mxc:// URL) for the user that
+    # will send the server notices. These are visible to all users on the deployment.
     local_part: "_server"
-    # The displayname to be used when sending notices
-    display_name: "Server alerts"
-    # The mxid of the avatar to use
+    display_name: "Server Alerts"
     avatar_url: ""
-    # The roomname to be used when creating messages
+    # The room name to be used when sending server notices. This room name will
+    # appear in user clients.
     room_name: "Server Alerts"
 
   # Configuration for NATS JetStream
   jetstream:
     # A list of NATS Server addresses to connect to. If none are specified, an
-    # internal NATS server will be started automatically when running Dendrite
-    # in monolith mode. It is required to specify the address of at least one
-    # NATS Server node if running in polylith mode.
+    # internal NATS server will be started automatically when running Dendrite in
+    # monolith mode. For polylith deployments, it is required to specify the address
+    # of at least one NATS Server node.
     addresses:
-    # - localhost:4222
+      - hostname:4222
 
-    # Keep all NATS streams in memory, rather than persisting it to the storage
-    # path below. This option is present primarily for integration testing and
-    # should not be used on a real world Dendrite deployment.
-    in_memory: false
-
-    # Persistent directory to store JetStream streams in. This directory
-    # should be preserved across Dendrite restarts.
-    storage_path: ./
-
-    # The prefix to use for stream names for this homeserver - really only
-    # useful if running more than one Dendrite on the same NATS deployment.
+    # The prefix to use for stream names for this homeserver - really only useful
+    # if you are running more than one Dendrite server on the same NATS deployment.
     topic_prefix: Dendrite
 
   # Configuration for Prometheus metric collection.
   metrics:
-    # Whether or not Prometheus metrics are enabled.
     enabled: false
-
-    # HTTP basic authentication to protect access to monitoring.
     basic_auth:
       username: metrics
       password: metrics
 
-  # DNS cache options. The DNS cache may reduce the load on DNS servers
-  # if there is no local caching resolver available for use.
+  # Optional DNS cache. The DNS cache may reduce the load on DNS servers if there
+  # is no local caching resolver available for use.
   dns_cache:
-    # Whether or not the DNS cache is enabled.
     enabled: false
-
-    # Maximum number of entries to hold in the DNS cache, and
-    # for how long those items should be considered valid in seconds.
     cache_size: 256
-    cache_lifetime: "5m" # 5minutes; see https://pkg.go.dev/time@master#ParseDuration for more
+    cache_lifetime: "5m" # 5 minutes; https://pkg.go.dev/time@master#ParseDuration
 
 # Configuration for the Appservice API.
 app_service_api:
   internal_api:
-    listen: http://localhost:7777 # Only used in polylith deployments
-    connect: http://localhost:7777 # Only used in polylith deployments
+    listen: http://[::]:7777 # The listen address for incoming API requests
+    connect: http://app_service_api:7777 # The connect address for other components to use
+
+  # Database configuration for this component.
   database:
-    connection_string: file:appservice.db
+    connection_string: postgresql://username@password:hostname/dendrite_appservice?sslmode=disable
     max_open_conns: 10
     max_idle_conns: 2
     conn_max_lifetime: -1
 
   # Disable the validation of TLS certificates of appservices. This is
   # not recommended in production since it may allow appservice traffic
-  # to be sent to an unverified endpoint.
+  # to be sent to an insecure endpoint.
   disable_tls_validation: false
 
   # Appservice configuration files to load into this homeserver.
-  config_files: []
+  config_files:
+  #  - /path/to/appservice_registration.yaml
 
 # Configuration for the Client API.
 client_api:
   internal_api:
-    listen: http://localhost:7771 # Only used in polylith deployments
-    connect: http://localhost:7771 # Only used in polylith deployments
+    listen: http://[::]:7771 # The listen address for incoming API requests
+    connect: http://client_api:7771 # The connect address for other components to use
   external_api:
     listen: http://[::]:8071
 
@@ -165,8 +136,8 @@ client_api:
   # disabled implicitly by setting 'registration_disabled' above.
   guests_disabled: true
 
-  # If set, allows registration by anyone who knows the shared secret, regardless of
-  # whether registration is otherwise disabled.
+  # If set, allows registration by anyone who knows the shared secret, regardless
+  # of whether registration is otherwise disabled.
   registration_shared_secret: ""
 
   # Disable password authentication.
@@ -179,7 +150,9 @@ client_api:
       version: 1
       chain_ids: []
 
-  # Whether to require reCAPTCHA for registration.
+  # Whether to require reCAPTCHA for registration. If you have enabled registration
+  # then this is HIGHLY RECOMMENDED to reduce the risk of your homeserver being used
+  # for coordinated spam attacks.
   enable_registration_captcha: false
 
   # Settings for ReCAPTCHA.
@@ -191,14 +164,16 @@ client_api:
   # TURN server information that this homeserver should send to clients.
   turn:
     turn_user_lifetime: ""
-    turn_uris: []
+    turn_uris:
+    #  - turn:turn.server.org?transport=udp
+    #  - turn:turn.server.org?transport=tcp
     turn_shared_secret: ""
     turn_username: ""
     turn_password: ""
 
-  # Settings for rate-limited endpoints. Rate limiting will kick in after the
-  # threshold number of "slots" have been taken by requests from a specific
-  # host. Each "slot" will be released after the cooloff time in milliseconds.
+  # Settings for rate-limited endpoints. Rate limiting kicks in after the threshold
+  # number of "slots" have been taken by requests from a specific host. Each "slot"
+  # will be released after the cooloff time in milliseconds.
   rate_limiting:
     enabled: true
     threshold: 5
@@ -207,18 +182,20 @@ client_api:
 # Configuration for the Federation API.
 federation_api:
   internal_api:
-    listen: http://localhost:7772 # Only used in polylith deployments
-    connect: http://localhost:7772 # Only used in polylith deployments
+    listen: http://[::]:7772 # The listen address for incoming API requests
+    connect: http://federation_api:7772 # The connect address for other components to use
   external_api:
     listen: http://[::]:8072
   database:
-    connection_string: file:federationapi.db
+    connection_string: postgresql://username@password:hostname/dendrite_federationapi?sslmode=disable
     max_open_conns: 10
     max_idle_conns: 2
     conn_max_lifetime: -1
 
   # How many times we will try to resend a failed transaction to a specific server. The
-  # backoff is 2**x seconds, so 1 = 2 seconds, 2 = 4 seconds, 3 = 8 seconds etc.
+  # backoff is 2**x seconds, so 1 = 2 seconds, 2 = 4 seconds, 3 = 8 seconds etc. Once
+  # the max retries are exceeded, Dendrite will no longer try to send transactions to
+  # that server until it comes back to life and connects to us again.
   send_max_retries: 16
 
   # Disable the validation of TLS certificates of remote federated homeservers. Do not
@@ -244,10 +221,10 @@ federation_api:
 # Configuration for the Key Server (for end-to-end encryption).
 key_server:
   internal_api:
-    listen: http://localhost:7779 # Only used in polylith deployments
-    connect: http://localhost:7779 # Only used in polylith deployments
+    listen: http://[::]:7779 # The listen address for incoming API requests
+    connect: http://key_server:7779 # The connect address for other components to use
   database:
-    connection_string: file:keyserver.db
+    connection_string: postgresql://username@password:hostname/dendrite_keyserver?sslmode=disable
     max_open_conns: 10
     max_idle_conns: 2
     conn_max_lifetime: -1
@@ -255,12 +232,12 @@ key_server:
 # Configuration for the Media API.
 media_api:
   internal_api:
-    listen: http://localhost:7774 # Only used in polylith deployments
-    connect: http://localhost:7774 # Only used in polylith deployments
+    listen: http://[::]:7774 # The listen address for incoming API requests
+    connect: http://media_api:7774 # The connect address for other components to use
   external_api:
     listen: http://[::]:8074
   database:
-    connection_string: file:mediaapi.db
+    connection_string: postgresql://username@password:hostname/dendrite_mediaapi?sslmode=disable
     max_open_conns: 5
     max_idle_conns: 2
     conn_max_lifetime: -1
@@ -269,8 +246,8 @@ media_api:
   base_path: ./media_store
 
   # The maximum allowed file size (in bytes) for media uploads to this homeserver
-  # (0 = unlimited). If using a reverse proxy, ensure it allows requests at
-  # least this large (e.g. client_max_body_size in nginx.)
+  # (0 = unlimited). If using a reverse proxy, ensure it allows requests at least
+  #this large (e.g. the client_max_body_size setting in nginx).
   max_file_size_bytes: 10485760
 
   # Whether to dynamically generate thumbnails if needed.
@@ -291,15 +268,13 @@ media_api:
       height: 480
       method: scale
 
-# Configuration for experimental MSC's
+# Configuration for enabling experimental MSCs on this homeserver.
 mscs:
-  # A list of enabled MSC's
-  # Currently valid values are:
-  # - msc2836    (Threading, see https://github.com/matrix-org/matrix-doc/pull/2836)
-  # - msc2946    (Spaces Summary, see https://github.com/matrix-org/matrix-doc/pull/2946)
-  mscs: []
+  mscs:
+  #  - msc2836  # (Threading, see https://github.com/matrix-org/matrix-doc/pull/2836)
+  #  - msc2946  # (Spaces Summary, see https://github.com/matrix-org/matrix-doc/pull/2946)
   database:
-    connection_string: file:mscs.db
+    connection_string: postgresql://username@password:hostname/dendrite_mscs?sslmode=disable
     max_open_conns: 5
     max_idle_conns: 2
     conn_max_lifetime: -1
@@ -307,10 +282,10 @@ mscs:
 # Configuration for the Room Server.
 room_server:
   internal_api:
-    listen: http://localhost:7770 # Only used in polylith deployments
-    connect: http://localhost:7770 # Only used in polylith deployments
+    listen: http://[::]:7770 # The listen address for incoming API requests
+    connect: http://room_server:7770 # The connect address for other components to use
   database:
-    connection_string: file:roomserver.db
+    connection_string: postgresql://username@password:hostname/dendrite_roomserver?sslmode=disable
     max_open_conns: 10
     max_idle_conns: 2
     conn_max_lifetime: -1
@@ -318,12 +293,12 @@ room_server:
 # Configuration for the Sync API.
 sync_api:
   internal_api:
-    listen: http://localhost:7773 # Only used in polylith deployments
-    connect: http://localhost:7773 # Only used in polylith deployments
+    listen: http://[::]:7773 # The listen address for incoming API requests
+    connect: http://sync_api:7773 # The connect address for other components to use
   external_api:
     listen: http://[::]:8073
   database:
-    connection_string: file:syncapi.db
+    connection_string: postgresql://username@password:hostname/dendrite_syncapi?sslmode=disable
     max_open_conns: 10
     max_idle_conns: 2
     conn_max_lifetime: -1
@@ -335,21 +310,23 @@ sync_api:
 
 # Configuration for the User API.
 user_api:
-  # The cost when hashing passwords on registration/login. Default: 10. Min: 4, Max: 31
-  # See https://pkg.go.dev/golang.org/x/crypto/bcrypt for more information.
-  # Setting this lower makes registration/login consume less CPU resources at the cost of security
-  # should the database be compromised. Setting this higher makes registration/login consume more
-  # CPU resources but makes it harder to brute force password hashes.
-  # This value can be low if performing tests or on embedded Dendrite instances (e.g WASM builds)
-  # bcrypt_cost: 10
   internal_api:
-    listen: http://localhost:7781 # Only used in polylith deployments
-    connect: http://localhost:7781 # Only used in polylith deployments
+    listen: http://[::]:7781 # The listen address for incoming API requests
+    connect: http://user_api:7781 # The connect address for other components to use
   account_database:
-    connection_string: file:userapi_accounts.db
+    connection_string: postgresql://username@password:hostname/dendrite_userapi?sslmode=disable
     max_open_conns: 10
     max_idle_conns: 2
     conn_max_lifetime: -1
+
+  # The cost when hashing passwords on registration/login. Default: 10. Min: 4, Max: 31
+  # See https://pkg.go.dev/golang.org/x/crypto/bcrypt for more information.
+  # Setting this lower makes registration/login consume less CPU resources at the cost
+  # of security should the database be compromised. Setting this higher makes registration/login
+  # consume more CPU resources but makes it harder to brute force password hashes. This value
+  # can be lowered if performing tests or on embedded Dendrite instances (e.g WASM builds).
+  bcrypt_cost: 10
+
   # The length of time that a token issued for a relying party from
   # /_matrix/client/r0/user/{userId}/openid/request_token endpoint
   # is considered to be valid in milliseconds.
@@ -372,12 +349,13 @@ tracing:
     baggage_restrictions: null
     throttler: null
 
-# Logging configuration
+# Logging configuration. The "std" logging type controls the logs being sent to
+# stdout. The "file" logging type controls logs being written to a log folder on
+# the disk. Supported log levels are "debug", "info", "warn", "error".
 logging:
   - type: std
     level: info
   - type: file
-    # The logging level, must be one of debug, info, warn, error, fatal, panic.
     level: info
     params:
       path: ./logs

docs/CODE_STYLE.md

@@ -1,60 +0,0 @@
-# Code Style
-
-In addition to standard Go code style (`gofmt`, `goimports`), we use `golangci-lint`
-to run a number of linters, the exact list can be found under linters in [.golangci.yml](.golangci.yml).
-[Installation](https://github.com/golangci/golangci-lint#install-golangci-lint) and [Editor
-Integration](https://golangci-lint.run/usage/integrations/#editor-integration) for
-it can be found in the readme of golangci-lint.
-
-For rare cases where a linter is giving a spurious warning, it can be disabled
-for that line or statement using a [comment
-directive](https://golangci-lint.run/usage/false-positives/#nolint), e.g.  `var
-bad_name int //nolint:golint,unused`. This should be used sparingly and only
-when its clear that the lint warning is spurious.
-
-The linters can be run using [build/scripts/find-lint.sh](/build/scripts/find-lint.sh)
-(see file for docs) or as part of a build/test/lint cycle using
-[build/scripts/build-test-lint.sh](/build/scripts/build-test-lint.sh).
-
-
-## Labels
-
-In addition to `TODO` and `FIXME` we also use `NOTSPEC` to identify deviations
-from the Matrix specification.
-
-## Logging
-
-We generally prefer to log with static log messages and include any dynamic
-information in fields.
-
-```golang
-logger := util.GetLogger(ctx)
-
-// Not recommended
-logger.Infof("Finished processing keys for %s, number of keys %d", name, numKeys)
-
-// Recommended
-logger.WithFields(logrus.Fields{
-    "numberOfKeys": numKeys,
-    "entityName":   name,
-}).Info("Finished processing keys")
-```
-
-This is useful when logging to systems that natively understand log fields, as
-it allows people to search and process the fields without having to parse the
-log message.
-
-
-## Visual Studio Code
-
-If you use VSCode then the following is an example of a workspace setting that
-sets up linting correctly:
-
-```json
-{
-    "go.lintTool":"golangci-lint",
-    "go.lintFlags": [
-      "--fast"
-    ]
-}
-```

docs/CONTRIBUTING.md

@@ -1,55 +1,103 @@
+---
+title: Contributing
+parent: Development
+permalink: /development/contributing
+---
+
 # Contributing to Dendrite
 
 Everyone is welcome to contribute to Dendrite! We aim to make it as easy as
 possible to get started.
 
-Please ensure that you sign off your contributions! See [Sign Off](#sign-off)
-section below.
+## Sign off
+
+We ask that everyone who contributes to the project signs off their contributions
+in accordance with the [DCO](https://github.com/matrix-org/matrix-spec/blob/main/CONTRIBUTING.rst#sign-off).
+In effect, this means adding a statement to your pull requests or commit messages
+along the lines of:
+
+```
+Signed-off-by: Full Name <email address>
+```
+
+Unfortunately we can't accept contributions without it.
 
 ## Getting up and running
 
-See [INSTALL.md](INSTALL.md) for instructions on setting up a running dev
-instance of dendrite, and [CODE_STYLE.md](CODE_STYLE.md) for the code style
-guide.
+See the [Installation](INSTALL.md) section for information on how to build an
+instance of Dendrite. You will likely need this in order to test your changes.
 
-We use [golangci-lint](https://github.com/golangci/golangci-lint) to lint
-Dendrite which can be executed via:
+## Code style
 
+On the whole, the format as prescribed by `gofmt`, `goimports` etc. is exactly
+what we use and expect. Please make sure that you run one of these formatters before
+submitting your contribution.
+
+## Comments
+
+Please make sure that the comments adequately explain *why* your code does what it
+does. If there are statements that are not obvious, please comment what they do.
+
+We also have some special tags which we use for searchability. These are:
+
+* `// TODO:` for places where a future review, rewrite or refactor is likely required;
+* `// FIXME:` for places where we know there is an outstanding bug that needs a fix;
+* `// NOTSPEC:` for places where the behaviour specifically does not match what the
+  [Matrix Specification](https://spec.matrix.org/) prescribes, along with a description
+  of *why* that is the case.
+
+## Linting
+
+We use [golangci-lint](https://github.com/golangci/golangci-lint) to lint Dendrite
+which can be executed via:
+
+```bash
+golangci-lint run
 ```
-$ golangci-lint run
-```
+
+If you are receiving linter warnings that you are certain are spurious and want to
+silence them, you can annotate the relevant lines or methods with a `// nolint:`
+comment. Please avoid doing this if you can.
+
+## Unit tests
 
 We also have unit tests which we run via:
 
-```
-$ go test ./...
+```bash
+go test ./...
 ```
 
-## Continuous Integration
+In general, we like submissions that come with tests. Anything that proves that the
+code is functioning as intended is great, and to ensure that we will find out quickly
+in the future if any regressions happen.
 
-When a Pull Request is submitted, continuous integration jobs are run
-automatically to ensure the code builds and is relatively well-written. The jobs
-are run on [Buildkite](https://buildkite.com/matrix-dot-org/dendrite/), and the
-Buildkite pipeline configuration can be found in Matrix.org's [pipelines
-repository](https://github.com/matrix-org/pipelines).
+We use the standard [Go testing package](https://gobyexample.com/testing) for this,
+alongside some helper functions in our own [`test` package](https://pkg.go.dev/github.com/matrix-org/dendrite/test).
 
-If a job fails, click the "details" button and you should be taken to the job's
-logs.
+## Continuous integration
 
-![Click the details button on the failing build
-step](https://raw.githubusercontent.com/matrix-org/dendrite/main/docs/images/details-button-location.jpg)
+When a Pull Request is submitted, continuous integration jobs are run automatically
+by GitHub actions to ensure that the code builds and works in a number of configurations,
+such as different Go versions, using full HTTP APIs and both database engines.
+CI will automatically run the unit tests (as above) as well as both of our integration
+test suites ([Complement](https://github.com/matrix-org/complement) and
+[SyTest](https://github.com/matrix-org/sytest)).
 
-Scroll down to the failing step and you should see some log output. Scan the
-logs until you find what it's complaining about, fix it, submit a new commit,
-then rinse and repeat until CI passes.
+You can see the progress of any CI jobs at the bottom of the Pull Request page, or by
+looking at the [Actions](https://github.com/matrix-org/dendrite/actions) tab of the Dendrite
+repository.
 
-### Running CI Tests Locally
+We generally won't accept a submission unless all of the CI jobs are passing. We
+do understand though that sometimes the tests get things wrong — if that's the case,
+please also raise a pull request to fix the relevant tests!
+
+### Running CI tests locally
 
 To save waiting for CI to finish after every commit, it is ideal to run the
 checks locally before pushing, fixing errors first. This also saves other people
 time as only so many PRs can be tested at a given time.
 
-To execute what Buildkite tests, first run `./build/scripts/build-test-lint.sh`; this
+To execute what CI tests, first run `./build/scripts/build-test-lint.sh`; this
 script will build the code, lint it, and run `go test ./...` with race condition
 checking enabled. If something needs to be changed, fix it and then run the
 script again until it no longer complains. Be warned that the linting can take a
@@ -64,8 +112,7 @@ passing tests.
 If these two steps report no problems, the code should be able to pass the CI
 tests.
 
-
-## Picking Things To Do
+## Picking things to do
 
 If you're new then feel free to pick up an issue labelled [good first
 issue](https://github.com/matrix-org/dendrite/labels/good%20first%20issue).
@@ -81,17 +128,10 @@ We ask people who are familiar with Dendrite to leave the [good first
 issue](https://github.com/matrix-org/dendrite/labels/good%20first%20issue)
 issues so that there is always a way for new people to come and get involved.
 
-## Getting Help
+## Getting help
 
 For questions related to developing on Dendrite we have a dedicated room on
 Matrix [#dendrite-dev:matrix.org](https://matrix.to/#/#dendrite-dev:matrix.org)
 where we're happy to help.
 
-For more general questions please use
-[#dendrite:matrix.org](https://matrix.to/#/#dendrite:matrix.org).
-
-## Sign off
-
-We ask that everyone who contributes to the project signs off their
-contributions, in accordance with the
-[DCO](https://github.com/matrix-org/matrix-spec/blob/main/CONTRIBUTING.rst#sign-off).
+For more general questions please use [#dendrite:matrix.org](https://matrix.to/#/#dendrite:matrix.org).

docs/DESIGN.md

@@ -1,140 +0,0 @@
-# Design
-
-## Log Based Architecture
-
-### Decomposition and Decoupling
-
-A matrix homeserver can be built around append-only event logs built from the
-messages, receipts, presence, typing notifications, device messages and other
-events sent by users on the homeservers or by other homeservers.
-
-The server would then decompose into two categories: writers that add new
-entries to the logs and readers that read those entries.
-
-The event logs then serve to decouple the two components, the writers and
-readers need only agree on the format of the entries in the event log.
-This format could be largely derived from the wire format of the events used
-in the client and federation protocols:
-
-
-     C-S API   +---------+    Event Log    +---------+   C-S API
-    ---------> |         |+  (e.g. kafka)  |         |+ --------->
-               | Writers || =============> | Readers ||
-    ---------> |         ||                |         || --------->
-     S-S API   +---------+|                +---------+|   S-S API
-                +---------+                 +---------+
-
-However the way matrix handles state events in a room creates a few
-complications for this model.
-
- 1) Writers require the room state at an event to check if it is allowed.
- 2) Readers require the room state at an event to determine the users and
-    servers that are allowed to see the event.
- 3) A client can query the current state of the room from a reader.
-
-The writers and readers cannot extract the necessary information directly from
-the event logs because it would take too long to extract the information as the
-state is built up by collecting individual state events from the event history.
-
-The writers and readers therefore need access to something that stores copies
-of the event state in a form that can be efficiently queried. One possibility
-would be for the readers and writers to maintain copies of the current state
-in local databases. A second possibility would be to add a dedicated component
-that maintained the state of the room and exposed an API that the readers and
-writers could query to get the state. The second has the advantage that the
-state is calculated and stored in a single location.
-
-
-     C-S API   +---------+    Log   +--------+   Log   +---------+   C-S API
-    ---------> |         |+ ======> |        | ======> |         |+ --------->
-               | Writers ||         |  Room  |         | Readers ||
-    ---------> |         || <------ | Server | ------> |         || --------->
-     S-S API   +---------+|  Query  |        |  Query  +---------+|  S-S API
-                +---------+         +--------+          +---------+
-
-
-The room server can annotate the events it logs to the readers with room state
-so that the readers can avoid querying the room server unnecessarily.
-
-[This architecture can be extended to cover most of the APIs.](WIRING.md)
-
-## How things are supposed to work.
-
-### Local client sends an event in an existing room.
-
-  0) The client sends a PUT `/_matrix/client/r0/rooms/{roomId}/send` request
-    and an HTTP loadbalancer routes the request to a ClientAPI.
-
-  1) The ClientAPI:
-
-    * Authenticates the local user using the `access_token` sent in the HTTP
-      request.
-    * Checks if it has already processed or is processing a request with the
-      same `txnID`.
-    * Calculates which state events are needed to auth the request.
-    * Queries the necessary state events and the latest events in the room
-      from the RoomServer.
-    * Confirms that the room exists and checks whether the event is allowed by
-      the auth checks.
-    * Builds and signs the events.
-    * Writes the event to a "InputRoomEvent" kafka topic.
-    * Send a `200 OK` response to the client.
-
-  2) The RoomServer reads the event from "InputRoomEvent" kafka topic:
-
-    * Checks if it has already has a copy of the event.
-    * Checks if the event is allowed by the auth checks using the auth events
-      at the event.
-    * Calculates the room state at the event.
-    * Works out what the latest events in the room after processing this event
-      are.
-    * Calculate how the changes in the latest events affect the current state
-      of the room.
-    * TODO: Workout what events determine the visibility of this event to other
-      users
-    * Writes the event along with the changes in current state to an
-      "OutputRoomEvent" kafka topic. It writes all the events for a room to
-      the same kafka partition.
-
-  3a) The ClientSync reads the event from the "OutputRoomEvent" kafka topic:
-
-    * Updates its copy of the current state for the room.
-    * Works out which users need to be notified about the event.
-    * Wakes up any pending `/_matrix/client/r0/sync` requests for those users.
-    * Adds the event to the recent timeline events for the room.
-
-  3b) The FederationSender reads the event from the "OutputRoomEvent" kafka topic:
-
-    * Updates its copy of the current state for the room.
-    * Works out which remote servers need to be notified about the event.
-    * Sends a `/_matrix/federation/v1/send` request to those servers.
-    * Or if there is a request in progress then add the event to a queue to be
-      sent when the previous request finishes.
-
-### Remote server sends an event in an existing room.
-
-  0) The remote server sends a `PUT /_matrix/federation/v1/send` request and an
-    HTTP loadbalancer routes the request to a FederationReceiver.
-
-  1) The FederationReceiver:
-
-    * Authenticates the remote server using the "X-Matrix" authorisation header.
-    * Checks if it has already processed or is processing a request with the
-      same `txnID`.
-    * Checks the signatures for the events.
-      Fetches the ed25519 keys for the event senders if necessary.
-    * Queries the RoomServer for a copy of the state of the room at each event.
-    * If the RoomServer doesn't know the state of the room at an event then
-      query the state of the room at the event from the remote server using
-      `GET /_matrix/federation/v1/state_ids` falling back to
-      `GET /_matrix/federation/v1/state` if necessary.
-    * Once the state at each event is known check whether the events are
-      allowed by the auth checks against the state at each event.
-    * For each event that is allowed write the event to the "InputRoomEvent"
-      kafka topic.
-    * Send a 200 OK response to the remote server listing which events were
-      successfully processed and which events failed
-
-  2) The RoomServer processes the event the same as it would a local event.
-
-  3a) The ClientSync processes the event the same as it would a local event.

docs/FAQ.md

@@ -1,26 +1,34 @@
-# Frequently Asked Questions
+---
+title: FAQ
+nav_order: 1
+permalink: /faq
+---
 
-### Is Dendrite stable?
+# FAQ
+
+## Is Dendrite stable?
 
 Mostly, although there are still bugs and missing features. If you are a confident power user and you are happy to spend some time debugging things when they go wrong, then please try out Dendrite. If you are a community, organisation or business that demands stability and uptime, then Dendrite is not for you yet - please install Synapse instead.
 
-### Is Dendrite feature-complete?
+## Is Dendrite feature-complete?
 
 No, although a good portion of the Matrix specification has been implemented. Mostly missing are client features - see the readme at the root of the repository for more information.
 
-### Is there a migration path from Synapse to Dendrite?
+## Is there a migration path from Synapse to Dendrite?
 
-No, not at present. There will be in the future when Dendrite reaches version 1.0.
+No, not at present. There will be in the future when Dendrite reaches version 1.0. For now it is not
+possible to migrate an existing Synapse deployment to Dendrite.
 
-### Can I use Dendrite with an existing Synapse database?
+## Can I use Dendrite with an existing Synapse database?
 
 No, Dendrite has a very different database schema to Synapse and the two are not interchangeable.
 
-### Should I run a monolith or a polylith deployment?
+## Should I run a monolith or a polylith deployment?
 
-Monolith deployments are always preferred where possible, and at this time, are far better tested than polylith deployments are. The only reason to consider a polylith deployment is if you wish to run different Dendrite components on separate physical machines.
+Monolith deployments are always preferred where possible, and at this time, are far better tested than polylith deployments are. The only reason to consider a polylith deployment is if you wish to run different Dendrite components on separate physical machines, but this is an advanced configuration which we don't
+recommend.
 
-### I've installed Dendrite but federation isn't working
+## I've installed Dendrite but federation isn't working
 
 Check the [Federation Tester](https://federationtester.matrix.org). You need at least:
 
@@ -28,49 +36,95 @@ Check the [Federation Tester](https://federationtester.matrix.org). You need at
 * A valid TLS certificate for that DNS name
 * Either DNS SRV records or well-known files
 
-### Does Dendrite work with my favourite client?
+## Does Dendrite work with my favourite client?
 
 It should do, although we are aware of some minor issues:
 
 * **Element Android**: registration does not work, but logging in with an existing account does
 * **Hydrogen**: occasionally sync can fail due to gaps in the `since` parameter, but clearing the cache fixes this
 
-### Does Dendrite support push notifications?
+## Does Dendrite support push notifications?
 
 Yes, we have experimental support for push notifications. Configure them in the usual way in your Matrix client.
 
-### Does Dendrite support application services/bridges?
+## Does Dendrite support application services/bridges?
 
 Possibly - Dendrite does have some application service support but it is not well tested. Please let us know by raising a GitHub issue if you try it and run into problems.
 
 Bridges known to work (as of v0.5.1):
-- [Telegram](https://docs.mau.fi/bridges/python/telegram/index.html)
-- [WhatsApp](https://docs.mau.fi/bridges/go/whatsapp/index.html)
-- [Signal](https://docs.mau.fi/bridges/python/signal/index.html)
-- [probably all other mautrix bridges](https://docs.mau.fi/bridges/)
 
-Remember to add the config file(s) to the `app_service_api` [config](https://github.com/matrix-org/dendrite/blob/de38be469a23813921d01bef3e14e95faab2a59e/dendrite-config.yaml#L130-L131).
+* [Telegram](https://docs.mau.fi/bridges/python/telegram/index.html)
+* [WhatsApp](https://docs.mau.fi/bridges/go/whatsapp/index.html)
+* [Signal](https://docs.mau.fi/bridges/python/signal/index.html)
+* [probably all other mautrix bridges](https://docs.mau.fi/bridges/)
 
-### Is it possible to prevent communication with the outside world?
+Remember to add the config file(s) to the `app_service_api` section of the config file.
 
-Yes, you can do this by disabling federation - set `disable_federation` to `true` in the `global` section of the Dendrite configuration file. 
+## Is it possible to prevent communication with the outside world?
 
-### Should I use PostgreSQL or SQLite for my databases?
+Yes, you can do this by disabling federation - set `disable_federation` to `true` in the `global` section of the Dendrite configuration file.
 
-Please use PostgreSQL wherever possible, especially if you are planning to run a homeserver that caters to more than a couple of users. 
+## Should I use PostgreSQL or SQLite for my databases?
 
-### Dendrite is using a lot of CPU
+Please use PostgreSQL wherever possible, especially if you are planning to run a homeserver that caters to more than a couple of users.
 
-Generally speaking, you should expect to see some CPU spikes, particularly if you are joining or participating in large rooms. However, constant/sustained high CPU usage is not expected - if you are experiencing that, please join `#dendrite-dev:matrix.org` and let us know, or file a GitHub issue.
+## Dendrite is using a lot of CPU
 
-### Dendrite is using a lot of RAM
+Generally speaking, you should expect to see some CPU spikes, particularly if you are joining or participating in large rooms. However, constant/sustained high CPU usage is not expected - if you are experiencing that, please join `#dendrite-dev:matrix.org` and let us know what you were doing when the
+CPU usage shot up, or file a GitHub issue. If you can take a [CPU profile](PROFILING.md) then that would
+be a huge help too, as that will help us to understand where the CPU time is going.
 
-A lot of users report that Dendrite is using a lot of RAM, sometimes even gigabytes of it. This is usually due to Go's allocator behaviour, which tries to hold onto allocated memory until the operating system wants to reclaim it for something else. This can make the memory usage look significantly inflated in tools like `top`/`htop` when actually most of that memory is not really in use at all.
+## Dendrite is using a lot of RAM
 
-If you want to prevent this behaviour so that the Go runtime releases memory normally, start Dendrite using the `GODEBUG=madvdontneed=1` environment variable. It is also expected that the allocator behaviour will be changed again in Go 1.16 so that it does not hold onto memory unnecessarily in this way.
+As above with CPU usage, some memory spikes are expected if Dendrite is doing particularly heavy work
+at a given instant. However, if it is using more RAM than you expect for a long time, that's probably
+not expected. Join `#dendrite-dev:matrix.org` and let us know what you were doing when the memory usage
+ballooned, or file a GitHub issue if you can. If you can take a [memory profile](PROFILING.md) then that
+would be a huge help too, as that will help us to understand where the memory usage is happening.
 
-If you are running with `GODEBUG=madvdontneed=1` and still see hugely inflated memory usage then that's quite possibly a bug - please join `#dendrite-dev:matrix.org` and let us know, or file a GitHub issue.
-
-### Dendrite is running out of PostgreSQL database connections
+## Dendrite is running out of PostgreSQL database connections
 
 You may need to revisit the connection limit of your PostgreSQL server and/or make changes to the `max_connections` lines in your Dendrite configuration. Be aware that each Dendrite component opens its own database connections and has its own connection limit, even in monolith mode!
+
+## What is being reported when enabling anonymous stats?
+
+If anonymous stats reporting is enabled, the following data is send to the defined endpoint.
+
+```json
+{
+    "cpu_average": 0,
+    "daily_active_users": 97,
+    "daily_e2ee_messages": 0,
+    "daily_messages": 0,
+    "daily_sent_e2ee_messages": 0,
+    "daily_sent_messages": 0,
+    "daily_user_type_bridged": 2,
+    "daily_user_type_native": 97,
+    "database_engine": "Postgres",
+    "database_server_version": "11.14 (Debian 11.14-0+deb10u1)",
+    "federation_disabled": false,
+    "go_arch": "amd64",
+    "go_os": "linux",
+    "go_version": "go1.16.13",
+    "homeserver": "localhost:8800",
+    "log_level": "trace",
+    "memory_rss": 93452,
+    "monolith": true,
+    "monthly_active_users": 97,
+    "nats_embedded": true,
+    "nats_in_memory": true,
+    "num_cpu": 8,
+    "num_go_routine": 203,
+    "r30v2_users_all": 0,
+    "r30v2_users_android": 0,
+    "r30v2_users_electron": 0,
+    "r30v2_users_ios": 0,
+    "r30v2_users_web": 0,
+    "timestamp": 1651741851,
+    "total_nonbridged_users": 97,
+    "total_room_count": 0,
+    "total_users": 99,
+    "uptime_seconds": 30,
+    "version": "0.8.2"
+}
+```

docs/Gemfile

@@ -0,0 +1,5 @@
+source "https://rubygems.org"
+gem "github-pages", "~> 226", group: :jekyll_plugins
+group :jekyll_plugins do
+  gem "jekyll-feed", "~> 0.15.1"
+end

docs/Gemfile.lock

@@ -0,0 +1,283 @@
+GEM
+  remote: https://rubygems.org/
+  specs:
+    activesupport (6.0.5)
+      concurrent-ruby (~> 1.0, >= 1.0.2)
+      i18n (>= 0.7, < 2)
+      minitest (~> 5.1)
+      tzinfo (~> 1.1)
+      zeitwerk (~> 2.2, >= 2.2.2)
+    addressable (2.8.0)
+      public_suffix (>= 2.0.2, < 5.0)
+    coffee-script (2.4.1)
+      coffee-script-source
+      execjs
+    coffee-script-source (1.11.1)
+    colorator (1.1.0)
+    commonmarker (0.23.4)
+    concurrent-ruby (1.1.10)
+    dnsruby (1.61.9)
+      simpleidn (~> 0.1)
+    em-websocket (0.5.3)
+      eventmachine (>= 0.12.9)
+      http_parser.rb (~> 0)
+    ethon (0.15.0)
+      ffi (>= 1.15.0)
+    eventmachine (1.2.7)
+    execjs (2.8.1)
+    faraday (1.10.0)
+      faraday-em_http (~> 1.0)
+      faraday-em_synchrony (~> 1.0)
+      faraday-excon (~> 1.1)
+      faraday-httpclient (~> 1.0)
+      faraday-multipart (~> 1.0)
+      faraday-net_http (~> 1.0)
+      faraday-net_http_persistent (~> 1.0)
+      faraday-patron (~> 1.0)
+      faraday-rack (~> 1.0)
+      faraday-retry (~> 1.0)
+      ruby2_keywords (>= 0.0.4)
+    faraday-em_http (1.0.0)
+    faraday-em_synchrony (1.0.0)
+    faraday-excon (1.1.0)
+    faraday-httpclient (1.0.1)
+    faraday-multipart (1.0.3)
+      multipart-post (>= 1.2, < 3)
+    faraday-net_http (1.0.1)
+    faraday-net_http_persistent (1.2.0)
+    faraday-patron (1.0.0)
+    faraday-rack (1.0.0)
+    faraday-retry (1.0.3)
+    ffi (1.15.5)
+    forwardable-extended (2.6.0)
+    gemoji (3.0.1)
+    github-pages (226)
+      github-pages-health-check (= 1.17.9)
+      jekyll (= 3.9.2)
+      jekyll-avatar (= 0.7.0)
+      jekyll-coffeescript (= 1.1.1)
+      jekyll-commonmark-ghpages (= 0.2.0)
+      jekyll-default-layout (= 0.1.4)
+      jekyll-feed (= 0.15.1)
+      jekyll-gist (= 1.5.0)
+      jekyll-github-metadata (= 2.13.0)
+      jekyll-include-cache (= 0.2.1)
+      jekyll-mentions (= 1.6.0)
+      jekyll-optional-front-matter (= 0.3.2)
+      jekyll-paginate (= 1.1.0)
+      jekyll-readme-index (= 0.3.0)
+      jekyll-redirect-from (= 0.16.0)
+      jekyll-relative-links (= 0.6.1)
+      jekyll-remote-theme (= 0.4.3)
+      jekyll-sass-converter (= 1.5.2)
+      jekyll-seo-tag (= 2.8.0)
+      jekyll-sitemap (= 1.4.0)
+      jekyll-swiss (= 1.0.0)
+      jekyll-theme-architect (= 0.2.0)
+      jekyll-theme-cayman (= 0.2.0)
+      jekyll-theme-dinky (= 0.2.0)
+      jekyll-theme-hacker (= 0.2.0)
+      jekyll-theme-leap-day (= 0.2.0)
+      jekyll-theme-merlot (= 0.2.0)
+      jekyll-theme-midnight (= 0.2.0)
+      jekyll-theme-minimal (= 0.2.0)
+      jekyll-theme-modernist (= 0.2.0)
+      jekyll-theme-primer (= 0.6.0)
+      jekyll-theme-slate (= 0.2.0)
+      jekyll-theme-tactile (= 0.2.0)
+      jekyll-theme-time-machine (= 0.2.0)
+      jekyll-titles-from-headings (= 0.5.3)
+      jemoji (= 0.12.0)
+      kramdown (= 2.3.2)
+      kramdown-parser-gfm (= 1.1.0)
+      liquid (= 4.0.3)
+      mercenary (~> 0.3)
+      minima (= 2.5.1)
+      nokogiri (>= 1.13.4, < 2.0)
+      rouge (= 3.26.0)
+      terminal-table (~> 1.4)
+    github-pages-health-check (1.17.9)
+      addressable (~> 2.3)
+      dnsruby (~> 1.60)
+      octokit (~> 4.0)
+      public_suffix (>= 3.0, < 5.0)
+      typhoeus (~> 1.3)
+    html-pipeline (2.14.1)
+      activesupport (>= 2)
+      nokogiri (>= 1.4)
+    http_parser.rb (0.8.0)
+    i18n (0.9.5)
+      concurrent-ruby (~> 1.0)
+    jekyll (3.9.2)
+      addressable (~> 2.4)
+      colorator (~> 1.0)
+      em-websocket (~> 0.5)
+      i18n (~> 0.7)
+      jekyll-sass-converter (~> 1.0)
+      jekyll-watch (~> 2.0)
+      kramdown (>= 1.17, < 3)
+      liquid (~> 4.0)
+      mercenary (~> 0.3.3)
+      pathutil (~> 0.9)
+      rouge (>= 1.7, < 4)
+      safe_yaml (~> 1.0)
+    jekyll-avatar (0.7.0)
+      jekyll (>= 3.0, < 5.0)
+    jekyll-coffeescript (1.1.1)
+      coffee-script (~> 2.2)
+      coffee-script-source (~> 1.11.1)
+    jekyll-commonmark (1.4.0)
+      commonmarker (~> 0.22)
+    jekyll-commonmark-ghpages (0.2.0)
+      commonmarker (~> 0.23.4)
+      jekyll (~> 3.9.0)
+      jekyll-commonmark (~> 1.4.0)
+      rouge (>= 2.0, < 4.0)
+    jekyll-default-layout (0.1.4)
+      jekyll (~> 3.0)
+    jekyll-feed (0.15.1)
+      jekyll (>= 3.7, < 5.0)
+    jekyll-gist (1.5.0)
+      octokit (~> 4.2)
+    jekyll-github-metadata (2.13.0)
+      jekyll (>= 3.4, < 5.0)
+      octokit (~> 4.0, != 4.4.0)
+    jekyll-include-cache (0.2.1)
+      jekyll (>= 3.7, < 5.0)
+    jekyll-mentions (1.6.0)
+      html-pipeline (~> 2.3)
+      jekyll (>= 3.7, < 5.0)
+    jekyll-optional-front-matter (0.3.2)
+      jekyll (>= 3.0, < 5.0)
+    jekyll-paginate (1.1.0)
+    jekyll-readme-index (0.3.0)
+      jekyll (>= 3.0, < 5.0)
+    jekyll-redirect-from (0.16.0)
+      jekyll (>= 3.3, < 5.0)
+    jekyll-relative-links (0.6.1)
+      jekyll (>= 3.3, < 5.0)
+    jekyll-remote-theme (0.4.3)
+      addressable (~> 2.0)
+      jekyll (>= 3.5, < 5.0)
+      jekyll-sass-converter (>= 1.0, <= 3.0.0, != 2.0.0)
+      rubyzip (>= 1.3.0, < 3.0)
+    jekyll-sass-converter (1.5.2)
+      sass (~> 3.4)
+    jekyll-seo-tag (2.8.0)
+      jekyll (>= 3.8, < 5.0)
+    jekyll-sitemap (1.4.0)
+      jekyll (>= 3.7, < 5.0)
+    jekyll-swiss (1.0.0)
+    jekyll-theme-architect (0.2.0)
+      jekyll (> 3.5, < 5.0)
+      jekyll-seo-tag (~> 2.0)
+    jekyll-theme-cayman (0.2.0)
+      jekyll (> 3.5, < 5.0)
+      jekyll-seo-tag (~> 2.0)
+    jekyll-theme-dinky (0.2.0)
+      jekyll (> 3.5, < 5.0)
+      jekyll-seo-tag (~> 2.0)
+    jekyll-theme-hacker (0.2.0)
+      jekyll (> 3.5, < 5.0)
+      jekyll-seo-tag (~> 2.0)
+    jekyll-theme-leap-day (0.2.0)
+      jekyll (> 3.5, < 5.0)
+      jekyll-seo-tag (~> 2.0)
+    jekyll-theme-merlot (0.2.0)
+      jekyll (> 3.5, < 5.0)
+      jekyll-seo-tag (~> 2.0)
+    jekyll-theme-midnight (0.2.0)
+      jekyll (> 3.5, < 5.0)
+      jekyll-seo-tag (~> 2.0)
+    jekyll-theme-minimal (0.2.0)
+      jekyll (> 3.5, < 5.0)
+      jekyll-seo-tag (~> 2.0)
+    jekyll-theme-modernist (0.2.0)
+      jekyll (> 3.5, < 5.0)
+      jekyll-seo-tag (~> 2.0)
+    jekyll-theme-primer (0.6.0)
+      jekyll (> 3.5, < 5.0)
+      jekyll-github-metadata (~> 2.9)
+      jekyll-seo-tag (~> 2.0)
+    jekyll-theme-slate (0.2.0)
+      jekyll (> 3.5, < 5.0)
+      jekyll-seo-tag (~> 2.0)
+    jekyll-theme-tactile (0.2.0)
+      jekyll (> 3.5, < 5.0)
+      jekyll-seo-tag (~> 2.0)
+    jekyll-theme-time-machine (0.2.0)
+      jekyll (> 3.5, < 5.0)
+      jekyll-seo-tag (~> 2.0)
+    jekyll-titles-from-headings (0.5.3)
+      jekyll (>= 3.3, < 5.0)
+    jekyll-watch (2.2.1)
+      listen (~> 3.0)
+    jemoji (0.12.0)
+      gemoji (~> 3.0)
+      html-pipeline (~> 2.2)
+      jekyll (>= 3.0, < 5.0)
+    kramdown (2.3.2)
+      rexml
+    kramdown-parser-gfm (1.1.0)
+      kramdown (~> 2.0)
+    liquid (4.0.3)
+    listen (3.7.1)
+      rb-fsevent (~> 0.10, >= 0.10.3)
+      rb-inotify (~> 0.9, >= 0.9.10)
+    mercenary (0.3.6)
+    minima (2.5.1)
+      jekyll (>= 3.5, < 5.0)
+      jekyll-feed (~> 0.9)
+      jekyll-seo-tag (~> 2.1)
+    minitest (5.15.0)
+    multipart-post (2.1.1)
+    nokogiri (1.13.6-arm64-darwin)
+      racc (~> 1.4)
+    octokit (4.22.0)
+      faraday (>= 0.9)
+      sawyer (~> 0.8.0, >= 0.5.3)
+    pathutil (0.16.2)
+      forwardable-extended (~> 2.6)
+    public_suffix (4.0.7)
+    racc (1.6.0)
+    rb-fsevent (0.11.1)
+    rb-inotify (0.10.1)
+      ffi (~> 1.0)
+    rexml (3.2.5)
+    rouge (3.26.0)
+    ruby2_keywords (0.0.5)
+    rubyzip (2.3.2)
+    safe_yaml (1.0.5)
+    sass (3.7.4)
+      sass-listen (~> 4.0.0)
+    sass-listen (4.0.0)
+      rb-fsevent (~> 0.9, >= 0.9.4)
+      rb-inotify (~> 0.9, >= 0.9.7)
+    sawyer (0.8.2)
+      addressable (>= 2.3.5)
+      faraday (> 0.8, < 2.0)
+    simpleidn (0.2.1)
+      unf (~> 0.1.4)
+    terminal-table (1.8.0)
+      unicode-display_width (~> 1.1, >= 1.1.1)
+    thread_safe (0.3.6)
+    typhoeus (1.4.0)
+      ethon (>= 0.9.0)
+    tzinfo (1.2.9)
+      thread_safe (~> 0.1)
+    unf (0.1.4)
+      unf_ext
+    unf_ext (0.0.8.1)
+    unicode-display_width (1.8.0)
+    zeitwerk (2.5.4)
+
+PLATFORMS
+  arm64-darwin-21
+
+DEPENDENCIES
+  github-pages (~> 226)
+  jekyll-feed (~> 0.15.1)
+  minima (~> 2.5.1)
+
+BUNDLED WITH
+   2.3.7

docs/INSTALL.md

@@ -1,283 +1,15 @@
-# Installing Dendrite
-
-Dendrite can be run in one of two configurations:
-
-* **Monolith mode**: All components run in the same process. In this mode,
-   it is possible to run an in-process [NATS Server](https://github.com/nats-io/nats-server)
-   instead of running a standalone deployment. This will usually be the preferred model for
-   low-to-mid volume deployments, providing the best balance between performance and resource usage.
-
-* **Polylith mode**: A cluster of individual components running in their own processes, dealing
-  with different aspects of the Matrix protocol (see [WIRING.md](WIRING-Current.md)). Components
-  communicate with each other using internal HTTP APIs and [NATS Server](https://github.com/nats-io/nats-server).
-  This will almost certainly be the preferred model for very large deployments but scalability
-  comes with a cost. API calls are expensive and therefore a polylith deployment may end up using
-  disproportionately more resources for a smaller number of users compared to a monolith deployment.
-
-In almost all cases, it is **recommended to run in monolith mode with PostgreSQL databases**.
-
-Regardless of whether you are running in polylith or monolith mode, each Dendrite component that
-requires storage has its own database connections. Both Postgres and SQLite are supported and can
-be mixed-and-matched across components as needed in the configuration file.
-
-Be advised that Dendrite is still in development and it's not recommended for
-use in production environments just yet!
-
-## Requirements
-
-Dendrite requires:
-
-* Go 1.16 or higher
-* PostgreSQL 12 or higher (if using PostgreSQL databases, not needed for SQLite)
-
-If you want to run a polylith deployment, you also need:
-
-* A standalone [NATS Server](https://github.com/nats-io/nats-server) deployment with JetStream enabled
-
-If you want to build it on Windows, you need `gcc` in the path:
-
-* [MinGW-w64](https://www.mingw-w64.org/)
-
-## Building Dendrite
-
-Start by cloning the code:
-
-```bash
-git clone https://github.com/matrix-org/dendrite
-cd dendrite
-```
-
-Then build it:
-
-* Linux or UNIX-like systems:
-  ```bash
-  ./build.sh
-  ```
-
-* Windows:
-  ```dos
-  build.cmd
-  ```
-
-## Install NATS Server
-
-Follow the [NATS Server installation instructions](https://docs.nats.io/running-a-nats-service/introduction/installation) and then [start your NATS deployment](https://docs.nats.io/running-a-nats-service/introduction/running).
-
-JetStream must be enabled, either by passing the `-js` flag to `nats-server`,
-or by specifying the `store_dir` option in the the `jetstream` configuration.
-
-## Configuration
-
-### PostgreSQL database setup
-
-Assuming that PostgreSQL 12 (or later) is installed:
-
-* Create role, choosing a new password when prompted:
-
-  ```bash
-  sudo -u postgres createuser -P dendrite
-  ```
-
-At this point you have a choice on whether to run all of the Dendrite
-components from a single database, or for each component to have its
-own database. For most deployments, running from a single database will
-be sufficient, although you may wish to separate them if you plan to
-split out the databases across multiple machines in the future.
-
-On macOS, omit `sudo -u postgres` from the below commands.
-
-* If you want to run all Dendrite components from a single database:
-
-  ```bash
-    sudo -u postgres createdb -O dendrite dendrite
-  ```
-
-  ... in which case your connection string will look like `postgres://user:pass@database/dendrite`.
-
-* If you want to run each Dendrite component with its own database:
-
-  ```bash
-  for i in mediaapi syncapi roomserver federationapi appservice keyserver userapi_accounts; do
-      sudo -u postgres createdb -O dendrite dendrite_$i
-  done
-  ```
-
-  ... in which case your connection string will look like `postgres://user:pass@database/dendrite_componentname`.
-
-### SQLite database setup
-
-**WARNING:** SQLite is suitable for small experimental deployments only and should not be used in production - use PostgreSQL instead for any user-facing federating installation!
-
-Dendrite can use the built-in SQLite database engine for small setups.
-The SQLite databases do not need to be pre-built - Dendrite will
-create them automatically at startup.
-
-### Server key generation
-
-Each Dendrite installation requires:
-
-* A unique Matrix signing private key
-* A valid and trusted TLS certificate and private key
-
-To generate a Matrix signing private key:
-
-```bash
-./bin/generate-keys --private-key matrix_key.pem
-```
-
-**WARNING:** Make sure take a safe backup of this key! You will likely need it if you want to reinstall Dendrite, or
-any other Matrix homeserver, on the same domain name in the future. If you lose this key, you may have trouble joining
-federated rooms.
-
-For testing, you can generate a self-signed certificate and key, although this will not work for public federation:
-
-```bash
-./bin/generate-keys --tls-cert server.crt --tls-key server.key
-```
-
-If you have server keys from an older Synapse instance,
-[convert them](serverkeyformat.md#converting-synapse-keys) to Dendrite's PEM
-format and configure them as `old_private_keys` in your config.
-
-### Configuration file
-
-Create config file, based on `dendrite-config.yaml`. Call it `dendrite.yaml`. Things that will need editing include *at least*:
-
-* The `server_name` entry to reflect the hostname of your Dendrite server
-* The `database` lines with an updated connection string based on your
-  desired setup, e.g. replacing `database` with the name of the database:
-  * For Postgres: `postgres://dendrite:password@localhost/database`, e.g.
-    * `postgres://dendrite:password@localhost/dendrite_userapi_account` to connect to PostgreSQL with SSL/TLS
-    * `postgres://dendrite:password@localhost/dendrite_userapi_account?sslmode=disable` to connect to PostgreSQL without SSL/TLS
-  * For SQLite on disk: `file:component.db` or `file:///path/to/component.db`, e.g. `file:userapi_account.db`
-  * Postgres and SQLite can be mixed and matched on different components as desired.
-* Either one of the following in the `jetstream` configuration section:
-  * The `addresses` option — a list of one or more addresses of an external standalone
-    NATS Server deployment
-  * The `storage_path` — where on the filesystem the built-in NATS server should
-    store durable queues, if using the built-in NATS server
-
-There are other options which may be useful so review them all. In particular,
-if you are trying to federate from your Dendrite instance into public rooms
-then configuring `key_perspectives` (like `matrix.org` in the sample) can
-help to improve reliability considerably by allowing your homeserver to fetch
-public keys for dead homeservers from somewhere else.
-
-**WARNING:** Dendrite supports running all components from the same database in
-PostgreSQL mode, but this is **NOT** a supported configuration with SQLite. When
-using SQLite, all components **MUST** use their own database file.
-
-## Starting a monolith server
-
-The monolith server can be started as shown below. By default it listens for
-HTTP connections on port 8008, so you can configure your Matrix client to use
-`http://servername:8008` as the server:
-
-```bash
-./bin/dendrite-monolith-server
-```
-
-If you set `--tls-cert` and `--tls-key` as shown below, it will also listen
-for HTTPS connections on port 8448:
-
-```bash
-./bin/dendrite-monolith-server --tls-cert=server.crt --tls-key=server.key
-```
-
-If the `jetstream` section of the configuration contains no `addresses` but does
-contain a `store_dir`, Dendrite will start up a built-in NATS JetStream node
-automatically, eliminating the need to run a separate NATS server.
-
-## Starting a polylith deployment
-
-The following contains scripts which will run all the required processes in order to point a Matrix client at Dendrite.
-
-### nginx (or other reverse proxy)
-
-This is what your clients and federated hosts will talk to. It must forward
-requests onto the correct API server based on URL:
-
-* `/_matrix/client` to the client API server
-* `/_matrix/federation` to the federation API server
-* `/_matrix/key` to the federation API server
-* `/_matrix/media` to the media API server
-
-See `docs/nginx/polylith-sample.conf` for a sample configuration.
-
-### Client API server
-
-This is what implements CS API endpoints. Clients talk to this via the proxy in
-order to send messages, create and join rooms, etc.
-
-```bash
-./bin/dendrite-polylith-multi --config=dendrite.yaml clientapi
-```
-
-### Sync server
-
-This is what implements `/sync` requests. Clients talk to this via the proxy
-in order to receive messages.
-
-```bash
-./bin/dendrite-polylith-multi --config=dendrite.yaml syncapi
-```
-
-### Media server
-
-This implements `/media` requests. Clients talk to this via the proxy in
-order to upload and retrieve media.
-
-```bash
-./bin/dendrite-polylith-multi --config=dendrite.yaml mediaapi
-```
-
-### Federation API server
-
-This implements the federation API. Servers talk to this via the proxy in
-order to send transactions.  This is only required if you want to support
-federation.
-
-```bash
-./bin/dendrite-polylith-multi --config=dendrite.yaml federationapi
-```
-
-### Internal components
-
-This refers to components that are not directly spoken to by clients. They are only
-contacted by other components. This includes the following components.
-
-#### Room server
-
-This is what implements the room DAG. Clients do not talk to this.
-
-```bash
-./bin/dendrite-polylith-multi --config=dendrite.yaml roomserver
-```
-
-#### Appservice server
-
-This sends events from the network to [application
-services](https://matrix.org/docs/spec/application_service/unstable.html)
-running locally.  This is only required if you want to support running
-application services on your homeserver.
-
-```bash
-./bin/dendrite-polylith-multi --config=dendrite.yaml appservice
-```
-
-#### Key server
-
-This manages end-to-end encryption keys for users.
-
-```bash
-./bin/dendrite-polylith-multi --config=dendrite.yaml keyserver
-```
-
-#### User server
-
-This manages user accounts, device access tokens and user account data,
-amongst other things.
-
-```bash
-./bin/dendrite-polylith-multi --config=dendrite.yaml userapi
-```
+# Installation
+
+Please note that new installation instructions can be found
+on the [new documentation site](https://matrix-org.github.io/dendrite/),
+or alternatively, in the [installation](installation/) folder:
+
+1. [Planning your deployment](installation/1_planning.md)
+2. [Setting up the domain](installation/2_domainname.md)
+3. [Preparing database storage](installation/3_database.md)
+4. [Generating signing keys](installation/4_signingkey.md)
+5. [Installing as a monolith](installation/5_install_monolith.md)
+6. [Installing as a polylith](installation/6_install_polylith.md)
+7. [Populate the configuration](installation/7_configuration.md)
+8. [Starting the monolith](installation/8_starting_monolith.md)
+9. [Starting the polylith](installation/9_starting_polylith.md)

docs/PROFILING.md

@@ -1,8 +1,14 @@
+---
+title: Profiling
+parent: Development
+permalink: /development/profiling
+---
+
 # Profiling Dendrite
 
 If you are running into problems with Dendrite using excessive resources (e.g. CPU or RAM) then you can use the profiler to work out what is happening.
 
-Dendrite contains an embedded profiler called `pprof`, which is a part of the standard Go toolchain. 
+Dendrite contains an embedded profiler called `pprof`, which is a part of the standard Go toolchain.
 
 ## Enable the profiler
 
@@ -16,7 +22,7 @@ If pprof has been enabled successfully, a log line at startup will show that ppr
 
 ```
 WARN[2020-12-03T13:32:33.669405000Z] [/Users/neilalexander/Desktop/dendrite/internal/log.go:87] SetupPprof
-	 Starting pprof on localhost:65432
+  Starting pprof on localhost:65432
 ```
 
 All examples from this point forward assume `PPROFLISTEN=localhost:65432` but you may need to adjust as necessary for your setup.

docs/WIRING-Current.md

@@ -1,71 +0,0 @@
-This document details how various components communicate with each other. There are two kinds of components:
- - Public-facing: exposes CS/SS API endpoints and need to be routed to via client-api-proxy or equivalent.
- - Internal-only: exposes internal APIs and produces Kafka events.
-
-## Internal HTTP APIs
-
-Not everything can be done using Kafka logs. For example, requesting the latest events in a room is much better suited to
-a request/response model like HTTP or RPC. Therefore, components can expose "internal APIs" which sit outside of Kafka logs.
-Note in Monolith mode these are actually direct function calls and are not serialised HTTP requests.
-
-```
-   Tier 1            Sync                         FederationAPI      ClientAPI    MediaAPI
-Public Facing        |                             | | |    | |        | | | |
-                     2   .-------3-----------------` | |    | `--------|-|-|-|--11--------------------.
-                     |   | .--------4----------------------------------` | | |                        |
-                     |   | |         .---5-----------` |    |            | | |                        |
-                     |   | |         |  .---6----------------------------` | |                        |
-                     |   | |         |  |              |  .-----7----------` |                        |
-                     |   | |         |  |              8  | |                10                       |
-                     |   | |         |  |              |  | `---9----.       |                        |
-                     V   V V         V  V              V  V          V       V                        V
-   Tier 2           Roomserver     EDUServer         FedSender       AppService    KeyServer   ServerKeyAPI
-Internal only               |                               `------------------------12----------^   ^
-                            `------------------------------------------------------------13----------`
-
- Client ---> Server
-```
-- 2 (Sync -> Roomserver): When making backfill requests
-- 3 (FedAPI -> Roomserver): Calculating (prev/auth events) and sending new events, processing backfill/state/state_ids requests
-- 4 (ClientAPI -> Roomserver): Calculating (prev/auth events) and sending new events, processing /state requests
-- 5 (FedAPI -> EDUServer): Sending typing/send-to-device events
-- 6 (ClientAPI -> EDUServer): Sending typing/send-to-device events
-- 7 (ClientAPI -> FedSender): Handling directory lookups
-- 8 (FedAPI -> FedSender): Resetting backoffs when receiving traffic from a server. Querying joined hosts when handling alias lookup requests
-- 9 (FedAPI -> AppService): Working out if the client is an appservice user
-- 10 (ClientAPI -> AppService): Working out if the client is an appservice user
-- 11 (FedAPI -> ServerKeyAPI): Verifying incoming event signatures
-- 12 (FedSender -> ServerKeyAPI): Verifying event signatures of responses (e.g from send_join)
-- 13 (Roomserver -> ServerKeyAPI): Verifying event signatures of backfilled events
-
-In addition to this, all public facing components (Tier 1) talk to the `UserAPI` to verify access tokens and extract profile information where needed.
-
-## Kafka logs
-
-```
-                       .----1--------------------------------------------.
-                       V                                                 |
-   Tier 1            Sync                         FederationAPI      ClientAPI    MediaAPI
-Public Facing        ^   ^                                             ^  
-                     |   |                                             |
-                     2   |                                             |
-                     |   `-3------------.                              |
-                     |                  |                              |
-                     |                  |                              |
-                     |                  |                              |
-                     |   .--------4-----|------------------------------`             
-                     |   |              |           
-   Tier 2           Roomserver     EDUServer         FedSender       AppService    KeyServer   ServerKeyAPI
-Internal only              |          |                ^ ^
-                           |          `-----5----------` |
-                           `--------------------6--------`
-
-
-Producer ----> Consumer
-```
-- 1 (ClientAPI -> Sync): For tracking account data
-- 2 (Roomserver -> Sync): For all data to send to clients
-- 3 (EDUServer -> Sync): For typing/send-to-device data to send to clients
-- 4 (Roomserver -> ClientAPI): For tracking memberships for profile updates.
-- 5 (EDUServer -> FedSender): For sending EDUs over federation
-- 6 (Roomserver -> FedSender): For sending PDUs over federation, for tracking joined hosts.

docs/WIRING.md

@@ -1,229 +0,0 @@
-# Wiring
-
-The diagram is incomplete. The following things aren't shown on the diagram:
-
-* Device Messages
-* User Profiles
-* Notification Counts
-* Sending federation.
-* Querying federation.
-* Other things that aren't shown on the diagram.
-
-Diagram:
-
-
-    W -> Writer
-    S -> Server/Store/Service/Something/Stuff
-    R -> Reader
-
-               +---+                                                    +---+                              +---+
-    +----------| W |                                         +----------| S |                     +--------| R |
-    |          +---+                                         | Receipts +---+                     | Client +---+
-    | Federation |>=========================================>| Server     |>=====================>| Sync     |
-    | Receiver   |                                           |            |                       |          |
-    |            |                                 +---+     |            |                       |          |
-    |            |                        +--------| W |     |            |                       |          |
-    |            |                        | Client +---+     |            |                       |          |
-    |            |                        | Receipt  |>=====>|            |                       |          |
-    |            |                        | Updater  |       |            |                       |          |
-    |            |                        +----------+       |            |                       |          |
-    |            |                                           |            |                       |          |
-    |            |                +---+            +---+     |            |                +---+  |          |
-    |            |   +------------| W |     +------| S |     |            |       +--------| R |  |          |
-    |            |   | Federation +---+     | Room +---+     |            |       | Client +---+  |          |
-    |            |   | Backfill     |>=====>| Server |>=====>|            |>=====>| Push     |    |          |
-    |            |   +--------------+       |        |       +------------+       |          |    |          |
-    |            |                          |        |                            |          |    |          |
-    |            |                          |        |>==========================>|          |    |          |
-    |            |                          |        |                            +----------+    |          |
-    |            |                          |        |                     +---+                  |          |
-    |            |                          |        |       +-------------| R |                  |          |
-    |            |                          |        |>=====>| Application +---+                  |          |
-    |            |                          |        |       | Services     |                     |          |
-    |            |                          |        |       +--------------+                     |          |
-    |            |                          |        |                                     +---+  |          |
-    |            |                          |        |                            +--------| R |  |          |
-    |            |                          |        |                            | Client +---+  |          |
-    |            |>========================>|        |>==========================>| Search   |    |          |
-    |            |                          |        |                            |          |    |          |
-    |            |                          |        |                            +----------+    |          |
-    |            |                          |        |                                            |          |
-    |            |                          |        |>==========================================>|          |
-    |            |                          |        |                                            |          |
-    |            |                +---+     |        |                  +---+                     |          |
-    |            |       +--------| W |     |        |       +----------| S |                     |          |
-    |            |       | Client +---+     |        |       | Presence +---+                     |          |
-    |            |       | API      |>=====>|        |>=====>| Server     |>=====================>|          |
-    |            |       | /send    |       +--------+       |            |                       |          |
-    |            |       |          |                        |            |                       |          |
-    |            |       |          |>======================>|            |<=====================<|          |
-    |            |       +----------+                        |            |                       |          |
-    |            |                                           |            |                       |          |
-    |            |                                 +---+     |            |                       |          |
-    |            |                        +--------| W |     |            |                       |          |
-    |            |                        | Client +---+     |            |                       |          |
-    |            |                        | Presence |>=====>|            |                       |          |
-    |            |                        | Setter   |       |            |                       |          |
-    |            |                        +----------+       |            |                       |          |
-    |            |                                           |            |                       |          |
-    |            |                                           |            |                       |          |
-    |            |>=========================================>|            |                       |          |
-    |            |                                           +------------+                       |          |
-    |            |                                                                                |          |
-    |            |                                                      +---+                     |          |
-    |            |                                           +----------| S |                     |          |
-    |            |                                           | EDU      +---+                     |          |
-    |            |>=========================================>| Server     |>=====================>|          |
-    +------------+                                           |            |                       +----------+
-                                                   +---+     |            |
-                                          +--------| W |     |            |
-                                          | Client +---+     |            |
-                                          | Typing   |>=====>|            |
-                                          | Setter   |       |            |
-                                          +----------+       +------------+
-
-
-# Component Descriptions
-
-Many of the components are logical rather than physical. For example it is
-possible that all of the client API writers will end up being glued together
-and always deployed as a single unit.
-
-Outbound federation requests will probably need to be funnelled through a
-choke-point to implement ratelimiting and backoff correctly.
-
-## Federation Send
-
- * Handles `/federation/v1/send/` requests.
- * Fetches missing ``prev_events`` from the remote server if needed.
- * Fetches missing room state from the remote server if needed.
- * Checks signatures on remote events, downloading keys if needed.
- * Queries information needed to process events from the Room Server.
- * Writes room events to logs.
- * Writes presence updates to logs.
- * Writes receipt updates to logs.
- * Writes typing updates to logs.
- * Writes other updates to logs.
-
-## Client API /send
-
- * Handles puts to `/client/v1/rooms/` that create room events.
- * Queries information needed to process events from the Room Server.
- * Talks to remote servers if needed for joins and invites.
- * Writes room event pdus.
- * Writes presence updates to logs.
-
-## Client Presence Setter
-
- * Handles puts to the [client API presence paths](https://matrix.org/docs/spec/client_server/unstable.html#id41).
- * Writes presence updates to logs.
-
-## Client Typing Setter
-
- * Handles puts to the [client API typing paths](https://matrix.org/docs/spec/client_server/unstable.html#id32).
- * Writes typing updates to logs.
-
-## Client Receipt Updater
-
- * Handles puts to the [client API receipt paths](https://matrix.org/docs/spec/client_server/unstable.html#id36).
- * Writes receipt updates to logs.
-
-## Federation Backfill
-
- * Backfills events from other servers
- * Writes the resulting room events to logs.
- * Is a different component from the room server itself cause it'll
-   be easier if the room server component isn't making outbound HTTP requests
-   to remote servers
-
-## Room Server
-
- * Reads new and backfilled room events from the logs written by FS, FB and CRS.
- * Tracks the current state of the room and the state at each event.
- * Probably does auth checks on the incoming events.
- * Handles state resolution as part of working out the current state and the
-   state at each event.
- * Writes updates to the current state and new events to logs.
- * Shards by room ID.
-
-## Receipt Server
-
- * Reads new updates to receipts from the logs written by the FS and CRU.
- * Somehow learns enough information from the room server to workout how the
-   current receipt markers move with each update.
- * Writes the new marker positions to logs
- * Shards by room ID?
- * It may be impossible to implement without folding it into the Room Server
-   forever coupling the components together.
-
-## EDU Server
-
- * Reads new updates to typing from the logs written by the FS and CTS.
- * Updates the current list of people typing in a room.
- * Writes the current list of people typing in a room to the logs.
- * Shards by room ID?
-
-## Presence Server
-
- * Reads the current state of the rooms from the logs to track the intersection
-   of room membership between users.
- * Reads updates to presence from the logs written by the FS and the CPS.
- * Reads when clients sync from the logs from the Client Sync.
- * Tracks any timers for users.
- * Writes the changes to presence state to the logs.
- * Shards by user ID somehow?
-
-## Client Sync
-
- * Handle /client/v2/sync requests.
- * Reads new events and the current state of the rooms from logs written by the Room Server.
- * Reads new receipts positions from the logs written by the Receipts Server.
- * Reads changes to presence from the logs written by the Presence Server.
- * Reads changes to typing from the logs written by the EDU Server.
- * Writes when a client starts and stops syncing to the logs.
-
-## Client Search
-
- * Handle whatever the client API path for event search is?
- * Reads new events and the current state of the rooms from logs writeen by the Room Server.
- * Maintains a full text search index of somekind.
-
-## Client Push
-
- * Pushes unread messages to remote push servers.
- * Reads new events and the current state of the rooms from logs writeen by the Room Server.
- * Reads the position of the read marker from the Receipts Server.
- * Makes outbound HTTP hits to the push server for the client device.
-
-## Application Service
-
- * Receives events from the Room Server.
- * Filters events and sends them to each registered application service.
- * Runs a separate goroutine for each application service.
-
-# Internal Component API
-
-Some dendrite components use internal APIs to communicate information back
-and forth between each other. There are two implementations of each API, one
-that uses HTTP requests and one that does not. The HTTP implementation is
-used in multi-process mode, so processes on separate computers may still
-communicate, whereas in single-process or Monolith mode, the direct
-implementation is used. HTTP is preferred here to kafka streams as it allows
-for request responses.
-
-Running `dendrite-monolith-server` will set up direct connections between
-components, whereas running each individual component (which are only run in
-multi-process mode) will set up HTTP-based connections.
-
-The functions that make HTTP requests to internal APIs of a component are
-located in `/<component name>/api/<name>.go`, named according to what
-functionality they cover. Each of these requests are handled in `/<component
-name>/<name>/<name>.go`.
-
-As an example, the `appservices` component allows other Dendrite components
-to query external application services via its internal API. A component
-would call the desired function in `/appservices/api/query.go`. In
-multi-process mode, this would send an internal HTTP request, which would
-be handled by a function in `/appservices/query/query.go`. In single-process
-mode, no internal HTTP request occurs, instead functions are simply called
-directly, thus requiring no changes on the calling component's end.

docs/_config.yml

@@ -0,0 +1,19 @@
+title: Dendrite
+description: >-
+  Second-generation Matrix homeserver written in Go!
+baseurl: "/dendrite" # the subpath of your site, e.g. /blog
+url: ""
+twitter_username: matrixdotorg
+github_username: matrix-org
+remote_theme: just-the-docs/just-the-docs
+plugins:
+  - jekyll-feed
+aux_links:
+  "GitHub":
+    - "//github.com/matrix-org/dendrite"
+aux_links_new_tab: true
+sass:
+  sass_dir: _sass
+  style: compressed
+exclude:
+  - INSTALL.md

docs/_sass/custom/custom.scss

@@ -0,0 +1,3 @@
+footer.site-footer {
+    opacity: 10%;
+}

docs/administration.md

@@ -0,0 +1,10 @@
+---
+title: Administration
+has_children: yes
+nav_order: 4
+permalink: /administration
+---
+
+# Administration
+
+This section contains documentation on managing your existing Dendrite deployment.

docs/administration/1_createusers.md

@@ -0,0 +1,53 @@
+---
+title: Creating user accounts
+parent: Administration
+permalink: /administration/createusers
+nav_order: 1
+---
+
+# Creating user accounts
+
+User accounts can be created on a Dendrite instance in a number of ways.
+
+## From the command line
+
+The `create-account` tool is built in the `bin` folder when building Dendrite with
+the `build.sh` script.
+
+It uses the `dendrite.yaml` configuration file to connect to the Dendrite user database
+and create the account entries directly. It can therefore be used even if Dendrite is not
+running yet, as long as the database is up.
+
+An example of using `create-account` to create a **normal account**:
+
+```bash
+./bin/create-account -config /path/to/dendrite.yaml -username USERNAME
+```
+
+You will be prompted to enter a new password for the new account.
+
+To create a new **admin account**, add the `-admin` flag:
+
+```bash
+./bin/create-account -config /path/to/dendrite.yaml -username USERNAME -admin
+```
+
+## Using shared secret registration
+
+Dendrite supports the Synapse-compatible shared secret registration endpoint.
+
+To enable shared secret registration, you must first enable it in the `dendrite.yaml`
+configuration file by specifying a shared secret. In the `client_api` section of the config,
+enter a new secret into the `registration_shared_secret` field:
+
+```yaml
+client_api:
+  # ...
+  registration_shared_secret: ""
+```
+
+You can then use the `/_synapse/admin/v1/register` endpoint as per the
+[Synapse documentation](https://matrix-org.github.io/synapse/latest/admin_api/register_api.html).
+
+Shared secret registration is only enabled once a secret is configured. To disable shared
+secret registration again, remove the secret from the configuration file.

docs/administration/2_registration.md

@@ -0,0 +1,53 @@
+---
+title: Enabling registration
+parent: Administration
+permalink: /administration/registration
+nav_order: 2
+---
+
+# Enabling registration
+
+Enabling registration allows users to register their own user accounts on your
+Dendrite server using their Matrix client. They will be able to choose their own
+username and password and log in.
+
+Registration is controlled by the `registration_disabled` field in the `client_api`
+section of the configuration. By default, `registration_disabled` is set to `true`,
+disabling registration. If you want to enable registration, you should change this
+setting to `false`.
+
+Currently Dendrite supports secondary verification using [reCAPTCHA](https://www.google.com/recaptcha/about/).
+Other methods will be supported in the future.
+
+## reCAPTCHA verification
+
+Dendrite supports reCAPTCHA as a secondary verification method. If you want to enable
+registration, it is **highly recommended** to configure reCAPTCHA. This will make it
+much more difficult for automated spam systems from registering accounts on your
+homeserver automatically.
+
+You will need an API key from the [reCAPTCHA Admin Panel](https://www.google.com/recaptcha/admin).
+Then configure the relevant details in the `client_api` section of the configuration:
+
+```yaml
+client_api:
+  # ...
+  registration_disabled: false
+  recaptcha_public_key: "PUBLIC_KEY_HERE"
+  recaptcha_private_key: "PRIVATE_KEY_HERE"
+  enable_registration_captcha: true
+  captcha_bypass_secret: ""
+  recaptcha_siteverify_api: "https://www.google.com/recaptcha/api/siteverify"
+```
+
+## Open registration
+
+Dendrite does support open registration — that is, allowing users to create their own
+user accounts without any verification or secondary authentication. However, it
+is **not recommended** to enable open registration, as this leaves your homeserver
+vulnerable to abuse by spammers or attackers, who create large numbers of user
+accounts on Matrix homeservers in order to send spam or abuse into the network.
+
+It isn't possible to enable open registration in Dendrite in a single step. If you
+try to disable the `registration_disabled` option without any secondary verification
+methods enabled (such as reCAPTCHA), Dendrite will log an error and fail to start.

docs/administration/3_presence.md

@@ -0,0 +1,39 @@
+---
+title: Enabling presence
+parent: Administration
+permalink: /administration/presence
+nav_order: 3
+---
+
+# Enabling presence
+
+Dendrite supports presence, which allows you to send your online/offline status
+to other users, and to receive their statuses automatically. They will be displayed
+by supported clients.
+
+Note that enabling presence **can negatively impact** the performance of your Dendrite
+server — it will require more CPU time and will increase the "chattiness" of your server
+over federation. It is disabled by default for this reason.
+
+Dendrite has two options for controlling presence:
+
+* **Enable inbound presence**: Dendrite will handle presence updates for remote users
+  and distribute them to local users on your homeserver;
+* **Enable outbound presence**: Dendrite will generate presence notifications for your
+  local users and distribute them to remote users over the federation.
+
+This means that you can configure only one or other direction if you prefer, i.e. to
+receive presence from other servers without revealing the presence of your own users.
+
+## Configuring presence
+
+Presence is controlled by the `presence` block in the `global` section of the
+configuration file:
+
+```yaml
+global:
+  # ...
+  presence:
+    enable_inbound: false
+    enable_outbound: false
+```

docs/administration/4_adminapi.md

@@ -0,0 +1,25 @@
+---
+title: Supported admin APIs
+parent: Administration
+permalink: /administration/adminapi
+---
+
+# Supported admin APIs
+
+Dendrite supports, at present, a very small number of endpoints that allow
+admin users to perform administrative functions. Please note that there is no
+API stability guarantee on these endpoints at present — they may change shape
+without warning.
+
+More endpoints will be added in the future.
+
+## `/_dendrite/admin/evacuateRoom/{roomID}`
+
+This endpoint will instruct Dendrite to part all local users from the given `roomID`
+in the URL. It may take some time to complete. A JSON body will be returned containing
+the user IDs of all affected users.
+
+## `/_synapse/admin/v1/register`
+
+Shared secret registration — please see the [user creation page](createusers) for
+guidance on configuring and using this endpoint.

docs/development.md

@@ -0,0 +1,10 @@
+---
+title: Development
+has_children: true
+permalink: /development
+---
+
+# Development
+
+This section contains documentation that may be useful when helping to develop
+Dendrite.

docs/index.md

@@ -0,0 +1,24 @@
+---
+layout: home
+nav_exclude: true
+---
+
+# Dendrite
+
+Dendrite is a second-generation Matrix homeserver written in Go! Following the microservice
+architecture model, Dendrite is designed to be efficient, reliable and scalable. Despite being beta,
+many Matrix features are already supported.
+
+This site aims to include relevant documentation to help you to get started with and
+run Dendrite. Check out the following sections:
+
+* **[Installation](installation.md)** for building and deploying your own Dendrite homeserver
+* **[Administration](administration.md)** for managing an existing Dendrite deployment
+* **[Development](development.md)** for developing against Dendrite
+
+You can also join us in our Matrix rooms dedicated to Dendrite, but please check first that
+your question hasn't already been [answered in the FAQ](FAQ.md):
+
+* **[#dendrite:matrix.org](https://matrix.to/#/#dendrite:matrix.org)** for general project discussion and support
+* **[#dendrite-dev:matrix.org](https://matrix.to/#/#dendrite-dev:matrix.org)** for chat on Dendrite development specifically
+* **[#dendrite-alerts:matrix.org](https://matrix.to/#/#dendrite-alerts:matrix.org)** for release notifications and other important announcements

docs/installation.md

@@ -0,0 +1,10 @@
+---
+title: Installation
+has_children: true
+nav_order: 2
+permalink: /installation
+---
+
+# Installation
+
+This section contains documentation on installing a new Dendrite deployment.