dendrite/docs/installation/1_planning.md
Tak Wai Wong 82d635f237 Pull latest changes from dendrite fork (#193)
* Verify that the user ID for registration matches the spec, and the auth data (#10)

* Blacklist some sytest tests that are failing in our environment

* Commenting out test that isn't reliably passing or failing, probably a race

* refresh latest dendrite main

* pull latest from dendrite-fork subtree

* refresh latest dendrite main

* pull dendrite subtree and resolve merge conflicts

* check that userID matches the signed message

* verify that the user ID for registration is CAIP-10 compliant and MXID compliant

* removed space

Co-authored-by: Brian Meek <brian@hntlabs.com>
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>

* Fix nats.go commit (#2540)

Signed-off-by: Jean Lucas <jean@4ray.co>

* Don't return `end` if there are not more messages (#2542)

* Be more spec compliant

* Move lazyLoadMembers to own method

* Return an error if trying to invite a malformed user ID (#2543)

* Add `evacuateUser` endpoint, use it when deactivating accounts (#2545)

* Add `evacuateUser` endpoint, use it when deactivating accounts

* Populate the API

* Clean up user devices when deactivating

* Include invites, delete pushers

* Silence presence logs (#2547)

* Blacklist `Guest users can join guest_access rooms` test until it can be investigated

* Disable WebAssembly builds for now

* Try to fix backfilling (#2548)

* Try to fix backfilling

* Return start/end to not confuse clients

* Update GMSL

* Update GMSL

* Roomserver producers package (#2546)

* Give the roomserver a producers package

* Change init point

* Populate ACLs API

* Fix build issues

* `RoomEventProducer` naming

* Version 0.8.9 (#2549)

* Version 0.8.9

* Update changelog

* Takwaiw/fix concurrent registration bug (#12)

* fix concurrent registration bug. Rename decentralizedid

* remove unused module

* add regressed test to blacklist

Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>

* Test_UserStatistics Fix expected results to match observed results

* Takwaiw/dendrite publickey (#2)

* Implementation of MSC 3782 Add publickey login as a new auth type.

Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>

* Implement EIP-4361 sign in with Ethereum (#5)

* Blacklist some sytest tests that are failing in our environment

* Commenting out test that isn't reliably passing or failing, probably a race

* refresh latest dendrite main

* refresh latest dendrite main

* dendrite implementation of eip-4361

* simplify nonce generation

Co-authored-by: Brian Meek <brian@hntlabs.com>
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>

* Use rand.Seed to seed the random function generator (#6)

* Blacklist some sytest tests that are failing in our environment

* Commenting out test that isn't reliably passing or failing, probably a race

* refresh latest dendrite main

* use rand.Seed to seed the random function

Co-authored-by: Brian Meek <brian@hntlabs.com>
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>

* Create session ID during registration (#8)

* Blacklist some sytest tests that are failing in our environment

* Commenting out test that isn't reliably passing or failing, probably a race

* refresh latest dendrite main

* pull latest from dendrite-fork subtree

* refresh latest dendrite main

* Create session ID during registration

Co-authored-by: Brian Meek <brian@hntlabs.com>
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>

* Verify that the user ID for registration matches the spec, and the auth data (#10)

* Blacklist some sytest tests that are failing in our environment

* Commenting out test that isn't reliably passing or failing, probably a race

* refresh latest dendrite main

* pull latest from dendrite-fork subtree

* refresh latest dendrite main

* pull dendrite subtree and resolve merge conflicts

* check that userID matches the signed message

* verify that the user ID for registration is CAIP-10 compliant and MXID compliant

* removed space

Co-authored-by: Brian Meek <brian@hntlabs.com>
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>

* Takwaiw/fix concurrent registration bug (#12)

* fix concurrent registration bug. Rename decentralizedid

* remove unused module

* add regressed test to blacklist

Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>

* removed unused module

* feat+fix: Ignore unknown keys and verify required fields are present in appservice registration files (#2550)

* fix: ignore unknown keys in appservice configs

fixes matrix-org/dendrite#1567

* feat: verify required fields in appservice configs

* Use new testrig for key changes tests (#2552)

* Use new testrig for tests

* Log the error message

* Fix QuerySharedUsers for the SyncAPI keychange consumer (#2554)

* Make more use of base.BaseDendrite

* Fix QuerySharedUsers if no UserIDs are supplied

* Return clearer error when no state NID exists for an event (#2555)

* Wrap error from `SnapshotNIDFromEventID`

* Hopefully fix read receipts timestamps (#2557)

This should avoid coercions between signed and unsigned ints which might fix problems like `sql: converting argument $5 type: uint64 values with high bit set are not supported`.

* fix concurrency issue when checking session ID (#14)

Co-authored-by: Tak Wai Wong <tak@hntlabs.com>

* merge latest changes from dendrite main (#15)

Co-authored-by: Tak Wai Wong <tak@hntlabs.com>

* Login and Register tests for public key ethereum (#16)

* TestLoginPublicKeyNewSession

* use asserts

* setup, test, asserts

* TestLoginPublicKeyValidAuthTypeMissingSession

* invalid session id test

* create a helper newSession function

* TestLoginPublicKeyEthereumMissingUserId

* TestLoginPublicKeyEthereumAccountNotAvailable

* TestLoginPublicKeyEthereumInvalidUserId

* createEip4361TestMessage

* TestLoginPublicKeyEthereumMissingSignature

* TestLoginPublicKeyEthereum

* re-enable all publickey signin tests

* move common publickey test util to its own file

* register_public_key.go stub

* refactored common ethereum test helpers to its own folder

* refactor test helpers

* return error in test helpers

* fix regressions with ServerName

* TestRegistrationUnimplementedAlgo

* TestNewRegistration

* TestNewRegistrationSession

* verify new login session

* remove assert

* perform account creation

* TestRegisterEthereum

* Enable all tests

* move helper functions into test file

Co-authored-by: Tak Wai Wong <tak@hntlabs.com>

Co-authored-by: Brian Meek <brian@hntlabs.com>
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>
Co-authored-by: Jean Lucas <jean@4ray.co>
Co-authored-by: Till <2353100+S7evinK@users.noreply.github.com>
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
Co-authored-by: Tak Wai Wong <tak@hntlabs.com>
Co-authored-by: Kabir Kwatra <kabir@kwatra.me>
2022-07-14 15:24:37 -07:00

5.5 KiB

title parent nav_order permalink
Planning your installation Installation 1 /installation/planning

Planning your installation

Modes

Dendrite consists of several components, each responsible for a different aspect of the Matrix protocol. Users can run Dendrite in one of two modes which dictate how these components are executed and communicate.

  • Monolith mode runs all components in a single process. Components communicate through an internal NATS server with generally low overhead. This mode dramatically simplifies deployment complexity and offers the best balance between performance and resource usage for low-to-mid volume deployments.

  • Polylith mode runs all components in isolated processes. Components communicate through an external NATS server and HTTP APIs, which incur considerable overhead. While this mode allows for more granular control of resources dedicated toward individual processes, given the additional communications overhead, it is only necessary for very large deployments.

Given our current state of development, we recommend monolith mode for all deployments.

Databases

Dendrite can run with either a PostgreSQL or a SQLite backend. There are considerable tradeoffs to consider:

  • PostgreSQL: Needs to run separately to Dendrite, needs to be installed and configured separately and and will use more resources over all, but will be considerably faster than SQLite. PostgreSQL has much better write concurrency which will allow Dendrite to process more tasks in parallel. This will be necessary for federated deployments to perform adequately.

  • SQLite: Built into Dendrite, therefore no separate database engine is necessary and is quite a bit easier to set up, but will be much slower than PostgreSQL in most cases. SQLite only allows a single writer on a database at a given time, which will significantly restrict Dendrite's ability to process multiple tasks in parallel.

At this time, we recommend the PostgreSQL database engine for all production deployments.

Requirements

Dendrite will run on Linux, macOS and Windows Server. It should also run fine on variants of BSD such as FreeBSD and OpenBSD. We have not tested Dendrite on AIX, Solaris, Plan 9 or z/OS — your mileage may vary with these platforms.

It is difficult to state explicitly the amount of CPU, RAM or disk space that a Dendrite installation will need, as this varies considerably based on a number of factors. In particular:

  • The number of users using the server;
  • The number of rooms that the server is joined to — federated rooms in particular will typically use more resources than rooms with only local users;
  • The complexity of rooms that the server is joined to — rooms with more members coming and going will typically be of a much higher complexity.

Some tasks are more expensive than others, such as joining rooms over federation, running state resolution or sending messages into very large federated rooms with lots of remote users. Therefore you should plan accordingly and ensure that you have enough resources available to endure spikes in CPU or RAM usage, as these may be considerably higher than the idle resource usage.

At an absolute minimum, Dendrite will expect 1GB RAM. For a comfortable day-to-day deployment which can participate in federated rooms for a number of local users, be prepared to assign 2-4 CPU cores and 8GB RAM — more if your user count increases.

If you are running PostgreSQL on the same machine, allow extra headroom for this too, as the database engine will also have CPU and RAM requirements of its own. Running too many heavy services on the same machine may result in resource starvation and processes may end up being killed by the operating system if they try to use too much memory.

Dependencies

In order to install Dendrite, you will need to satisfy the following dependencies.

Go

At this time, Dendrite supports being built with Go 1.18 or later. We do not support building Dendrite with older versions of Go than this. If you are installing Go using a package manager, you should check (by running go version) that you are using a suitable version before you start.

PostgreSQL

If using the PostgreSQL database engine, you should install PostgreSQL 12 or later.

NATS Server

Monolith deployments come with a built-in NATS Server and therefore do not need this to be manually installed. If you are planning a monolith installation, you do not need to do anything.

Polylith deployments, however, currently need a standalone NATS Server installation with JetStream enabled.

To do so, follow the NATS Server installation instructions and then start your NATS deployment. JetStream must be enabled, either by passing the -js flag to nats-server, or by specifying the store_dir option in the the jetstream configuration.

Reverse proxy (polylith deployments)

Polylith deployments require a reverse proxy, such as NGINX or HAProxy. Configuring those is not covered in this documentation, although a sample configuration for NGINX is provided.

Windows

Finally, if you want to build Dendrite on Windows, you will need need gcc in the path. The best way to achieve this is by installing and building Dendrite under MinGW-w64.