* Verify that the user ID for registration matches the spec, and the auth data (#10)
* Blacklist some sytest tests that are failing in our environment
* Commenting out test that isn't reliably passing or failing, probably a race
* refresh latest dendrite main
* pull latest from dendrite-fork subtree
* refresh latest dendrite main
* pull dendrite subtree and resolve merge conflicts
* check that userID matches the signed message
* verify that the user ID for registration is CAIP-10 compliant and MXID compliant
* removed space
Co-authored-by: Brian Meek <brian@hntlabs.com>
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>
* Fix nats.go commit (#2540)
Signed-off-by: Jean Lucas <jean@4ray.co>
* Don't return `end` if there are not more messages (#2542)
* Be more spec compliant
* Move lazyLoadMembers to own method
* Return an error if trying to invite a malformed user ID (#2543)
* Add `evacuateUser` endpoint, use it when deactivating accounts (#2545)
* Add `evacuateUser` endpoint, use it when deactivating accounts
* Populate the API
* Clean up user devices when deactivating
* Include invites, delete pushers
* Silence presence logs (#2547)
* Blacklist `Guest users can join guest_access rooms` test until it can be investigated
* Disable WebAssembly builds for now
* Try to fix backfilling (#2548)
* Try to fix backfilling
* Return start/end to not confuse clients
* Update GMSL
* Update GMSL
* Roomserver producers package (#2546)
* Give the roomserver a producers package
* Change init point
* Populate ACLs API
* Fix build issues
* `RoomEventProducer` naming
* Version 0.8.9 (#2549)
* Version 0.8.9
* Update changelog
* Takwaiw/fix concurrent registration bug (#12)
* fix concurrent registration bug. Rename decentralizedid
* remove unused module
* add regressed test to blacklist
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>
* Test_UserStatistics Fix expected results to match observed results
* Takwaiw/dendrite publickey (#2)
* Implementation of MSC 3782 Add publickey login as a new auth type.
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>
* Implement EIP-4361 sign in with Ethereum (#5)
* Blacklist some sytest tests that are failing in our environment
* Commenting out test that isn't reliably passing or failing, probably a race
* refresh latest dendrite main
* refresh latest dendrite main
* dendrite implementation of eip-4361
* simplify nonce generation
Co-authored-by: Brian Meek <brian@hntlabs.com>
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>
* Use rand.Seed to seed the random function generator (#6)
* Blacklist some sytest tests that are failing in our environment
* Commenting out test that isn't reliably passing or failing, probably a race
* refresh latest dendrite main
* use rand.Seed to seed the random function
Co-authored-by: Brian Meek <brian@hntlabs.com>
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>
* Create session ID during registration (#8)
* Blacklist some sytest tests that are failing in our environment
* Commenting out test that isn't reliably passing or failing, probably a race
* refresh latest dendrite main
* pull latest from dendrite-fork subtree
* refresh latest dendrite main
* Create session ID during registration
Co-authored-by: Brian Meek <brian@hntlabs.com>
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>
* Verify that the user ID for registration matches the spec, and the auth data (#10)
* Blacklist some sytest tests that are failing in our environment
* Commenting out test that isn't reliably passing or failing, probably a race
* refresh latest dendrite main
* pull latest from dendrite-fork subtree
* refresh latest dendrite main
* pull dendrite subtree and resolve merge conflicts
* check that userID matches the signed message
* verify that the user ID for registration is CAIP-10 compliant and MXID compliant
* removed space
Co-authored-by: Brian Meek <brian@hntlabs.com>
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>
* Takwaiw/fix concurrent registration bug (#12)
* fix concurrent registration bug. Rename decentralizedid
* remove unused module
* add regressed test to blacklist
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>
* removed unused module
* feat+fix: Ignore unknown keys and verify required fields are present in appservice registration files (#2550)
* fix: ignore unknown keys in appservice configs
fixesmatrix-org/dendrite#1567
* feat: verify required fields in appservice configs
* Use new testrig for key changes tests (#2552)
* Use new testrig for tests
* Log the error message
* Fix QuerySharedUsers for the SyncAPI keychange consumer (#2554)
* Make more use of base.BaseDendrite
* Fix QuerySharedUsers if no UserIDs are supplied
* Return clearer error when no state NID exists for an event (#2555)
* Wrap error from `SnapshotNIDFromEventID`
* Hopefully fix read receipts timestamps (#2557)
This should avoid coercions between signed and unsigned ints which might fix problems like `sql: converting argument $5 type: uint64 values with high bit set are not supported`.
* fix concurrency issue when checking session ID (#14)
Co-authored-by: Tak Wai Wong <tak@hntlabs.com>
* merge latest changes from dendrite main (#15)
Co-authored-by: Tak Wai Wong <tak@hntlabs.com>
* Login and Register tests for public key ethereum (#16)
* TestLoginPublicKeyNewSession
* use asserts
* setup, test, asserts
* TestLoginPublicKeyValidAuthTypeMissingSession
* invalid session id test
* create a helper newSession function
* TestLoginPublicKeyEthereumMissingUserId
* TestLoginPublicKeyEthereumAccountNotAvailable
* TestLoginPublicKeyEthereumInvalidUserId
* createEip4361TestMessage
* TestLoginPublicKeyEthereumMissingSignature
* TestLoginPublicKeyEthereum
* re-enable all publickey signin tests
* move common publickey test util to its own file
* register_public_key.go stub
* refactored common ethereum test helpers to its own folder
* refactor test helpers
* return error in test helpers
* fix regressions with ServerName
* TestRegistrationUnimplementedAlgo
* TestNewRegistration
* TestNewRegistrationSession
* verify new login session
* remove assert
* perform account creation
* TestRegisterEthereum
* Enable all tests
* move helper functions into test file
Co-authored-by: Tak Wai Wong <tak@hntlabs.com>
Co-authored-by: Brian Meek <brian@hntlabs.com>
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>
Co-authored-by: Jean Lucas <jean@4ray.co>
Co-authored-by: Till <2353100+S7evinK@users.noreply.github.com>
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
Co-authored-by: Tak Wai Wong <tak@hntlabs.com>
Co-authored-by: Kabir Kwatra <kabir@kwatra.me>
Updates/adds a new multistage (build-kit) Dockerfile. (if accepted,
could make `Dockerfile.monolith` and `Dockerfile.polylith` in
`build/docker` obsolete)
There's no huge difference between the dockerfiles, except this uses a
non-root user when running the container, also doesn't copy the working
directory to the image when building.
Also adds vulnerabilities scans using
[Trivy](https://github.com/aquasecurity/trivy) for the created docker
images. (untested)
Building images is done using
```
docker build . --target image-monolith -t dendrite-monolith
docker build . --target image-polylith -t dendrite-polylith
```
As noted in the comments, only adds `dendrite-polylith-multi` to the
polylith image and all required binaries to the monolith image.
Probably needs some docs updating, if this is accepted.
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
### Pull Request Checklist
<!-- Please read docs/CONTRIBUTING.md before submitting your pull
request -->
No additional tests have been added as this simply extends the API for
the embedded pinecone demo.
Signed-off-by: `Devon Hudson <devonhudson@librem.one>`
Some systems (like nixos) don't have bash living at `/bin/bash` so using
`/usr/bin/env bash` we can make these scripts universal.
### Pull Request Checklist
<!-- Please read docs/CONTRIBUTING.md before submitting your pull
request -->
* [X] I have added added tests for PR _or_ I have justified why this PR
doesn't need tests.
* [x] Pull request includes a [sign
off](https://github.com/matrix-org/dendrite/blob/main/docs/CONTRIBUTING.md#sign-off)
Signed-off-by: `Dov Alperin <git@dov.dev>`
Signed-off-by: `Dov Alperin <git@dov.dev>`
This pull request adds the configuration and CI steps to build and
publish a container wrapping the `dendrite-demo-pinecone` command as
well as fixes a sentence structure issue in the pull request template.
As this does not touch any go source code no tests have been added
### Pull Request Checklist
<!-- Please read docs/CONTRIBUTING.md before submitting your pull
request -->
* [x] I have added tests for PR _or_ I have justified why this PR
doesn't need tests.
* [x] Pull request includes a [sign
off](https://github.com/matrix-org/dendrite/blob/main/docs/CONTRIBUTING.md#sign-off)
Signed-off-by: networkException <git@nwex.de> (by private sign-off)
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
This makes the following changes:
* The various `Defaults` functions are now responsible for setting sane defaults if `generate` is specified, rather than hiding them in `generate-config`
* Some configuration options have been marked as `omitempty` so that they don't appear in generated configs unnecessarily (monolith-specific vs. polylith-specific options)
* A new option `-polylith` has been added to `generate-config` to create a config that makes sense for polylith deployments (i.e. including the internal/external API listeners and per-component database sections)
* A new option `-normalise` has been added to `generate-config` to take an existing file and add any missing options and/or defaults
This PR refactors the app services component. It makes the following changes:
* Each appservice now gets its own NATS JetStream consumer
* The appservice database is now removed entirely, since we just use JetStream as a data source instead
* The entire component is now much simpler and we deleted lots of lines of code 💅
The result is that it should be much lighter and hopefully much more performant.
This PR does the following:
- adds a `keysize` parameter to `generate-keys`, so we can use lower sized keys when running in CI
- updates the Complement docker files to use BuildKit (requires Docker >18.09)
- uses `exec` when executing `dendrite-monotlith-server`, making it PID 1 inside docker, which results in Dendrite actually receiving the `SIGTERM` signal send by Docker. (Making it faster when running tests with Complement, as we don't take 10 seconds to timeout)
* Add race testing to tests, and fix a few small race conditions in the tests
* Enable run-sytest on MacOS
* Remove deadlock detecting mutex, per code review feedback
* Remove autoformatting related changes and a closure that is not needed
* Adjust to importing nats client as 'natsclient'
Signed-off-by: Brian Meek <brian@hntlabs.com>
* Clarify the use of gooseMutex to proect goose internal state
Signed-off-by: Brian Meek <brian@hntlabs.com>
* Remove no longer needed mutex for guarding goose
Signed-off-by: Brian Meek <brian@hntlabs.com>
* Try Ristretto cache
* Tweak
* It's beautiful
* Update GMSL
* More strict keyable interface
* Fix that some more
* Make less panicky
* Don't enforce mutability checks for now
* Determine mutability using deep equality
* Tweaks
* Namespace keys
* Make federation caches mutable
* Update cost estimation, add metric
* Update GMSL
* Estimate cost for metrics better
* Reduce counters a bit
* Try caching events
* Some guards
* Try again
* Try this
* Use separate caches for hopefully better hash distribution
* Fix bug with admitting events into cache
* Try to fix bugs
* Check nil
* Try that again
* Preserve order jeezo this is messy
* thanks VS Code for doing exactly the wrong thing
* Try this again
* Be more specific
* aaaaargh
* One more time
* That might be better
* Stronger sorting
* Cache expiries, async publishing of EDUs
* Put it back
* Use a shared cache again
* Cost estimation fixes
* Update ristretto
* Reduce counters a bit
* Clean up a bit
* Update GMSL
* 1GB
* Configurable cache sizees
* Tweaks
* Add `config.DataUnit` for specifying friendly cache sizes
* Various tweaks
* Update GMSL
* Add back some lazy loading caching
* Include key in cost
* Include key in cost
* Tweak max age handling, config key name
* Only register prometheus metrics if requested
* Review comments @S7evinK
* Don't return errors when creating caches (it is better just to crash since otherwise we'll `nil`-pointer exception everywhere)
* Review comments
* Update sample configs
* Update GHA Workflow
* Update Complement images to Go 1.18
* Remove the cache test from the federation API as we no longer guarantee immediate cache admission
* Don't check the caches in the renewal test
* Possibly fix the upgrade tests
* Update to matrix-org/gomatrixserverlib#322
* Update documentation to refer to Go 1.18
* Simplify federation API `AddPublicRoutes`
* Simplify client API `AddPublicRoutes`
* Simplify media API `AddPublicRoutes`
* Simplify sync API `AddPublicRoutes`
* Simplify `AddAllPublicRoutes`
* switch to dendrite server
* minor refactor to merge store code
* Fix issue where m.room.name is being filtered by the dendrite server
* refresh dendrite main
* refresh dendrite main
* missing merges from the last dendrite refresh
* revert unwanted changes in dendrite.yaml
* Update golangci-lint, how it's installed, and added to the PATH
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>
Co-authored-by: tak-slashtalk <64229756+tak-slashtalk@users.noreply.github.com>
* Add restrications for open registration
* Make enable open registration a parameter
* Enable registration for CI
* Update error message
* Shuffle things around a bit
* Add a warning at every startup just to be extra annoying
* Ignore shared secret when warning about open registration, since it's not strictly required when it is set if registration is otherwise enabled
* Make CI happy?
* Add missing parameter; try new parameter in upgrade-test
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
* Add new endpoint to allow admins to evacuate the local server from the room
* Guard endpoint
* Use right prefix
* Auth API
* More useful return error rather than a panic
* More useful return value again
* Update the path
* Try using inputer instead
* oh provide the config
* Try that again
* Return affected user IDs
* Don't create so many forward extremities
* Add missing `Path` to name
Co-authored-by: Till <2353100+S7evinK@users.noreply.github.com>
Go 1.18 has now been released for a while and the CI already tests Dendrite with Go 1.18
so there should be no issues. Go 1.18 brings some performance improvements for ARM via
the register calling convention so it makes sense to switch to it.
* Remove all federation_certificates occurencs
This configuration value has not been used since 2019 apparently, and indeed
it is never really used in the code base. So remove all traces of it from
the various configuration files.
Also remove the unused variable FederationCertificatePaths
Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>
* setup/config/config_test.go: remove federation_sender config snippet
The federation_sender: section was folded into the federation_api some
time ago, and this seems to be the only leftover in the code base.
So remove it.
* Move receipt sending to own JetStream producer
* Move SendToDevice to producer
* Remove most parts of the EDU server
* Fix SendToDevice & copyrights
* Move structs, cleanup EDU Server traces
* Use HeadersOnly subscription
* Missing file
* Fix linter issues
* Move consumers to own files
* Rename durable consumer; Consumer cleanup
* Docs/config cleanup
* Upload Docker images for releases to both Docker Hub and GitHub Container Registry
* Build current images on `:main` tag
* Use Dendrite flow to trigger Docker flow for `:main`
* Tweaks
* Fix references to `env.GHCR_NAMESPACE`
* Initial test
* Move CI to GHA
* Naming
* Always report all linter issues
* Remove if true
* Test complement in different variations
* Try again
* Move Complement back after initial tests and readd timeout
Make linting fail further checks
Remove CodeQL
* Update and rename tests.yml to dendrite.yml
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
