Commit graph

187 commits

Author SHA1 Message Date
Brian Meek c243677795
Merge local harmony changes, Dendrite tests passing
Signed-off-by: Brian Meek <brian@hntlabs.com>
2023-02-08 08:23:06 -08:00
Brian Meek f23e0f5e06
Logging the config in the error to debug CI
Signed-off-by: Brian Meek <brian@hntlabs.com>
2023-02-07 15:53:58 -08:00
Brian Meek 4f1c8983ad
Merge upstream latest
Signed-off-by: Brian Meek <brian@hntlabs.com>
2023-02-06 20:50:50 -08:00
Kerem Kazan dcf838c3b2 dendrite-zion.yaml added into the docker image (#965)
This PR adds the dendrite-zion.yaml into the production docker image for
dendrite
2022-11-18 18:11:48 -05:00
Tak Wai Wong 78a4a213d4 Refresh dendrite with latest contract changes (#836)
sync latest dendrite main & space manager contract changes from our
fork.

Co-authored-by: Tak Wai Wong <tak@hntlabs.com>
2022-11-04 11:23:15 -07:00
Tak Wai Wong a7ab7524da servers/dendrite Move calculation (#2856) commit 98d3f88bfb (#826) 2022-11-03 19:15:07 -07:00
Tak Wai Wong 1d633c0d5e
Force sync upstream commit 98d3f88bfb 2022-11-03 11:22:01 -07:00
Neil Alexander fb2e7d1b05
Put P2P Demos back into their own Dockerfiles 2022-11-03 13:57:44 +00:00
gitlab-nickfreeman 23a25be904
Update README.md (#2855)
There only are 2, not 3 sample `docker-compose` files.
This is only a change to the README.md, no need for Go unit tests

### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `Nick Freeman <nick@nickfreeman.de>`
2022-11-03 12:18:25 +00:00
Tak Wai Wong a9d3bdc058 subtree pull from dendrite fork for fix #2383 (#802)
Pull the fix for https://github.com/matrix-org/dendrite/issues/2838 into
the dendrite subtree. Includes latest dendrite main.

Signed-off-by: `Devon Hudson <devonhudson@librem.one>`
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Brian Meek <brian@hntlabs.com>
Signed-off-by: Austin Ellis <austin@hntlabs.com>
Signed-off-by: `Rubin Poster <rubinposter@gmail.com>`
Signed-off-by: `ash lea <example@thisismyactual.email>`
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
Co-authored-by: devonh <devon.dmytro@gmail.com>
Co-authored-by: Till <2353100+S7evinK@users.noreply.github.com>
Co-authored-by: Ashley Nelson <fant@shley.email>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Brian Meek <brian@hntlabs.com>
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>
Co-authored-by: Jean Lucas <jean@4ray.co>
Co-authored-by: Kabir Kwatra <kabir@kwatra.me>
Co-authored-by: sergekh2 <gitgitgit@khorun.com>
Co-authored-by: texuf <texuf.eth@gmail.com>
Co-authored-by: kegsay <kegan@matrix.org>
Co-authored-by: Brian Meek <brian@here.video>
Co-authored-by: John Terzis <john.c.terzis@gmail.com>
Co-authored-by: John Terzis <john@hntlabs.com>
Co-authored-by: Kerem <kerem.kazan@gmail.com>
Co-authored-by: Neboer <43609792+Neboer@users.noreply.github.com>
Co-authored-by: X. Ding <dingsm@gmail.com>
Co-authored-by: dxl <dxl@plotbridge.com>
Co-authored-by: ash lea <ashkitten@users.noreply.github.com>
Co-authored-by: 0x1a8510f2 <admin@0x1a8510f2.space>
Co-authored-by: Till Faelligen <tfaelligen@gmail.com>
Co-authored-by: Till Faelligen <davidf@element.io>
2022-11-02 16:28:06 -07:00
Tak Wai Wong 063299248b
sync fix #2838 from main 2022-11-02 13:48:55 -07:00
Neil Alexander ca8bc87380
Multi-stage Docker builds (#2850)
This builds on @S7evinK's work to make multi-stage Docker builds. Now
that we can build SQLite without Cgo this should be much simpler and
should make Docker builds in CI significantly faster.

Co-authored-by: Till Faelligen <tfaelligen@gmail.com>
Co-authored-by: Till Faelligen <davidf@element.io>
Co-authored-by: Till Faelligen <2353100+S7evinK@users.noreply.github.com>
2022-11-02 14:04:08 +00:00
Tak Wai Wong 424df14000 Sync dendrite fork changes for gating, and single chain support (#778)
* Latest dendrite main (8c7b274e4e)
* Gating implementation from John and Tak

Fixes for https://github.com/matrix-org/dendrite/issues/2838 and
https://github.com/matrix-org/dendrite/issues/2842

Co-authored-by: Tak Wai Wong <tak@hntlabs.com>
2022-11-01 11:09:34 -07:00
Tak Wai Wong ed8b5d09eb refresh fork with upstream 2022-10-28 15:49:31 -07:00
Neil Alexander 238b6ef2cd
Update Yggdrasil demo 2022-10-26 18:37:01 +01:00
devonh a74aea0714
Add network interface callback to pinecone build (#2825)
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-10-26 17:25:57 +01:00
Brian Meek 68d9300ec0
After discovering the full path using the go nm tool, properly set the ReleaseVersion in the clientapi router
Signed-off-by: Brian Meek <brian@hntlabs.com>
2022-10-24 21:20:49 -07:00
Brian Meek 93ea0f5864
Add commit ReleaseVersion to routing to return to clients in version request
Signed-off-by: Brian Meek <brian@hntlabs.com>
2022-10-24 21:18:59 -07:00
Brian Meek 4fe6a9e9da
Add commit hash to routing version API, TODO placeholder
Signed-off-by: Brian Meek <brian@hntlabs.com>
2022-10-24 21:18:59 -07:00
Brian Meek 8351e35287
Add commit hash to routing version API
Signed-off-by: Brian Meek <brian@hntlabs.com>
2022-10-24 21:18:58 -07:00
Brian Meek d641fed90b
Update golint and go versions
Signed-off-by: Brian Meek <brian@hntlabs.com>
2022-10-24 19:56:48 -07:00
Brian Meek be116212c3
Update dependencies to see if it helps make tests more reliable, easier to debug
Signed-off-by: Brian Meek <brian@hntlabs.com>
2022-10-24 19:56:48 -07:00
Tak Wai Wong 916f17ceb7
Pull latest changes from dendrite fork (#193)
* Verify that the user ID for registration matches the spec, and the auth data (#10)

* Blacklist some sytest tests that are failing in our environment

* Commenting out test that isn't reliably passing or failing, probably a race

* refresh latest dendrite main

* pull latest from dendrite-fork subtree

* refresh latest dendrite main

* pull dendrite subtree and resolve merge conflicts

* check that userID matches the signed message

* verify that the user ID for registration is CAIP-10 compliant and MXID compliant

* removed space

Co-authored-by: Brian Meek <brian@hntlabs.com>
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>

* Fix nats.go commit (#2540)

Signed-off-by: Jean Lucas <jean@4ray.co>

* Don't return `end` if there are not more messages (#2542)

* Be more spec compliant

* Move lazyLoadMembers to own method

* Return an error if trying to invite a malformed user ID (#2543)

* Add `evacuateUser` endpoint, use it when deactivating accounts (#2545)

* Add `evacuateUser` endpoint, use it when deactivating accounts

* Populate the API

* Clean up user devices when deactivating

* Include invites, delete pushers

* Silence presence logs (#2547)

* Blacklist `Guest users can join guest_access rooms` test until it can be investigated

* Disable WebAssembly builds for now

* Try to fix backfilling (#2548)

* Try to fix backfilling

* Return start/end to not confuse clients

* Update GMSL

* Update GMSL

* Roomserver producers package (#2546)

* Give the roomserver a producers package

* Change init point

* Populate ACLs API

* Fix build issues

* `RoomEventProducer` naming

* Version 0.8.9 (#2549)

* Version 0.8.9

* Update changelog

* Takwaiw/fix concurrent registration bug (#12)

* fix concurrent registration bug. Rename decentralizedid

* remove unused module

* add regressed test to blacklist

Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>

* Test_UserStatistics Fix expected results to match observed results

* Takwaiw/dendrite publickey (#2)

* Implementation of MSC 3782 Add publickey login as a new auth type.

Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>

* Implement EIP-4361 sign in with Ethereum (#5)

* Blacklist some sytest tests that are failing in our environment

* Commenting out test that isn't reliably passing or failing, probably a race

* refresh latest dendrite main

* refresh latest dendrite main

* dendrite implementation of eip-4361

* simplify nonce generation

Co-authored-by: Brian Meek <brian@hntlabs.com>
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>

* Use rand.Seed to seed the random function generator (#6)

* Blacklist some sytest tests that are failing in our environment

* Commenting out test that isn't reliably passing or failing, probably a race

* refresh latest dendrite main

* use rand.Seed to seed the random function

Co-authored-by: Brian Meek <brian@hntlabs.com>
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>

* Create session ID during registration (#8)

* Blacklist some sytest tests that are failing in our environment

* Commenting out test that isn't reliably passing or failing, probably a race

* refresh latest dendrite main

* pull latest from dendrite-fork subtree

* refresh latest dendrite main

* Create session ID during registration

Co-authored-by: Brian Meek <brian@hntlabs.com>
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>

* Verify that the user ID for registration matches the spec, and the auth data (#10)

* Blacklist some sytest tests that are failing in our environment

* Commenting out test that isn't reliably passing or failing, probably a race

* refresh latest dendrite main

* pull latest from dendrite-fork subtree

* refresh latest dendrite main

* pull dendrite subtree and resolve merge conflicts

* check that userID matches the signed message

* verify that the user ID for registration is CAIP-10 compliant and MXID compliant

* removed space

Co-authored-by: Brian Meek <brian@hntlabs.com>
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>

* Takwaiw/fix concurrent registration bug (#12)

* fix concurrent registration bug. Rename decentralizedid

* remove unused module

* add regressed test to blacklist

Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>

* removed unused module

* feat+fix: Ignore unknown keys and verify required fields are present in appservice registration files (#2550)

* fix: ignore unknown keys in appservice configs

fixes matrix-org/dendrite#1567

* feat: verify required fields in appservice configs

* Use new testrig for key changes tests (#2552)

* Use new testrig for tests

* Log the error message

* Fix QuerySharedUsers for the SyncAPI keychange consumer (#2554)

* Make more use of base.BaseDendrite

* Fix QuerySharedUsers if no UserIDs are supplied

* Return clearer error when no state NID exists for an event (#2555)

* Wrap error from `SnapshotNIDFromEventID`

* Hopefully fix read receipts timestamps (#2557)

This should avoid coercions between signed and unsigned ints which might fix problems like `sql: converting argument $5 type: uint64 values with high bit set are not supported`.

* fix concurrency issue when checking session ID (#14)

Co-authored-by: Tak Wai Wong <tak@hntlabs.com>

* merge latest changes from dendrite main (#15)

Co-authored-by: Tak Wai Wong <tak@hntlabs.com>

* Login and Register tests for public key ethereum (#16)

* TestLoginPublicKeyNewSession

* use asserts

* setup, test, asserts

* TestLoginPublicKeyValidAuthTypeMissingSession

* invalid session id test

* create a helper newSession function

* TestLoginPublicKeyEthereumMissingUserId

* TestLoginPublicKeyEthereumAccountNotAvailable

* TestLoginPublicKeyEthereumInvalidUserId

* createEip4361TestMessage

* TestLoginPublicKeyEthereumMissingSignature

* TestLoginPublicKeyEthereum

* re-enable all publickey signin tests

* move common publickey test util to its own file

* register_public_key.go stub

* refactored common ethereum test helpers to its own folder

* refactor test helpers

* return error in test helpers

* fix regressions with ServerName

* TestRegistrationUnimplementedAlgo

* TestNewRegistration

* TestNewRegistrationSession

* verify new login session

* remove assert

* perform account creation

* TestRegisterEthereum

* Enable all tests

* move helper functions into test file

Co-authored-by: Tak Wai Wong <tak@hntlabs.com>

Co-authored-by: Brian Meek <brian@hntlabs.com>
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>
Co-authored-by: Jean Lucas <jean@4ray.co>
Co-authored-by: Till <2353100+S7evinK@users.noreply.github.com>
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
Co-authored-by: Tak Wai Wong <tak@hntlabs.com>
Co-authored-by: Kabir Kwatra <kabir@kwatra.me>
2022-10-24 17:26:35 -07:00
John Terzis 478579eea1 Jterzis/update dendrite (#656)
Pulls in upstream latest changes from [dendrite-fork
](https://github.com/HereNotThere/dendrite)to subtree at
servers/dendrite here.

Co-authored-by: Tak Wai Wong <64229756+tak-hntlabs@users.noreply.github.com>
Co-authored-by: Tak Wai Wong <tak@hntlabs.com>
Co-authored-by: John Terzis <john@hntlabs.com>
2022-10-17 15:59:47 -07:00
Neil Alexander 085bf5e28b
Revert Docker changes 2022-10-04 11:33:05 +01:00
Neil Alexander 98b73652e0
Try to populate -ldflags in Docker builds 2022-10-04 11:07:54 +01:00
Till e6c992ba8b
Update Dockerfile (#2342)
Updates/adds a new multistage (build-kit) Dockerfile. (if accepted,
could make `Dockerfile.monolith` and `Dockerfile.polylith` in
`build/docker` obsolete)
There's no huge difference between the dockerfiles, except this uses a
non-root user when running the container, also doesn't copy the working
directory to the image when building.
Also adds vulnerabilities scans using
[Trivy](https://github.com/aquasecurity/trivy) for the created docker
images. (untested)

Building images is done using 
```
docker build . --target image-monolith -t dendrite-monolith
docker build . --target image-polylith -t dendrite-polylith
```

As noted in the comments, only adds `dendrite-polylith-multi` to the
polylith image and all required binaries to the monolith image.
Probably needs some docs updating, if this is accepted.

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-10-04 10:41:06 +01:00
Neil Alexander ba66b5a3b9
Allow multiple static peers in Pinecone iOS/Android demos 2022-10-03 14:43:38 +01:00
Neil Alexander 34451d21b8
P2P demo tweaks 2022-10-03 14:35:10 +01:00
devonh 90f1985bf3
Add network interface registration to pinecone demo (#2750)
### Pull Request Checklist

<!-- Please read docs/CONTRIBUTING.md before submitting your pull
request -->

No additional tests have been added as this simply extends the API for
the embedded pinecone demo.

Signed-off-by: `Devon Hudson <devonhudson@librem.one>`
2022-09-29 16:05:16 +00:00
Neil Alexander 6f3dbee5f4
Fix Go 1.18 2022-09-29 16:08:00 +01:00
Neil Alexander 68d6eb0a6f
P2P demo fixes 2022-09-29 13:06:55 +01:00
texuf 32d48002f7 Merge commit '1bf9dc0ca2c2eae4efd4b282a39d3bfe42e5e10e' into austin.ellis/dendrite
# Conflicts:
#	servers/dendrite/build/docker/Dockerfile.polylith
2022-09-28 15:12:34 -07:00
texuf 48fce3be10 Merge branch 'main' of github.com:matrix-org/dendrite into austin.ellis/merge
# Conflicts:
#	syncapi/streams/stream_notificationdata.go
#	userapi/consumers/syncapi_readupdate.go
2022-09-27 14:55:38 -07:00
Neil Alexander d531202b0e
Build Docker images using Go 1.19 (related to #2714) 2022-09-27 10:52:03 +01:00
Dov Alperin b5bfff47d2
Use /usr/bin/env bash in shebangs to make them universal (#2735)
Some systems (like nixos) don't have bash living at `/bin/bash` so using
`/usr/bin/env bash` we can make these scripts universal.

### Pull Request Checklist

<!-- Please read docs/CONTRIBUTING.md before submitting your pull
request -->

* [X] I have added added tests for PR _or_ I have justified why this PR
doesn't need tests.
* [x] Pull request includes a [sign
off](https://github.com/matrix-org/dendrite/blob/main/docs/CONTRIBUTING.md#sign-off)

Signed-off-by: `Dov Alperin <git@dov.dev>`

Signed-off-by: `Dov Alperin <git@dov.dev>`
2022-09-27 09:42:08 +01:00
networkException 40fec70d13
Add pinecone demo container image (#2710)
This pull request adds the configuration and CI steps to build and
publish a container wrapping the `dendrite-demo-pinecone` command as
well as fixes a sentence structure issue in the pull request template.

As this does not touch any go source code no tests have been added

### Pull Request Checklist

<!-- Please read docs/CONTRIBUTING.md before submitting your pull
request -->

* [x] I have added tests for PR _or_ I have justified why this PR
doesn't need tests.
* [x] Pull request includes a [sign
off](https://github.com/matrix-org/dendrite/blob/main/docs/CONTRIBUTING.md#sign-off)

Signed-off-by: networkException <git@nwex.de> (by private sign-off)
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-09-27 09:39:39 +01:00
Brian Meek 08d7e67c31 Update docker images for Dendrite to use buster
Signed-off-by: Brian Meek <brian@hntlabs.com>
2022-09-25 19:08:28 -05:00
Brian Meek 4accf677ea
Merge Matrix main
Signed-off-by: Brian Meek <brian@hntlabs.com>
2022-09-19 13:05:15 -07:00
Brian Meek 0fe5ea6c32
After discovering the full path using the go nm tool, properly set the ReleaseVersion in the clientapi router
Signed-off-by: Brian Meek <brian@hntlabs.com>
2022-09-17 14:10:36 -07:00
Brian Meek d7dc087953
Add commit ReleaseVersion to routing to return to clients in version request
Signed-off-by: Brian Meek <brian@hntlabs.com>
2022-09-16 13:28:03 -07:00
Brian Meek b95657fe94
Add commit hash to routing version API, TODO placeholder
Signed-off-by: Brian Meek <brian@hntlabs.com>
2022-09-15 15:12:56 -07:00
Brian Meek f681e54bef
Add commit hash to routing version API
Signed-off-by: Brian Meek <brian@hntlabs.com>
2022-09-15 15:00:36 -07:00
Neil Alexander 0ea948c705
Fix Pinecone demo build errors after Pinecone update 2022-09-14 14:26:24 +01:00
Neil Alexander 3a9dde28fd
P2P demo tweaks 2022-09-12 10:19:02 +01:00
Brian Meek d13085792c Update golint and go versions
Signed-off-by: Brian Meek <brian@hntlabs.com>
2022-09-03 09:04:35 -07:00
Brian Meek 973f949304
Update dependencies to see if it helps make tests more reliable, easier to debug
Signed-off-by: Brian Meek <brian@hntlabs.com>
2022-09-02 12:45:43 -07:00
Neil Alexander fea869b41f
Update P2P demos 2022-09-01 17:12:27 +01:00
Neil Alexander 51d229b025
Configuration tweaks (#2567)
This makes the following changes:

* The various `Defaults` functions are now responsible for setting sane defaults if `generate` is specified, rather than hiding them in `generate-config`
* Some configuration options have been marked as `omitempty` so that they don't appear in generated configs unnecessarily (monolith-specific vs. polylith-specific options)
* A new option `-polylith` has been added to `generate-config` to create a config that makes sense for polylith deployments (i.e. including the internal/external API listeners and per-component database sections)
* A new option `-normalise` has been added to `generate-config` to take an existing file and add any missing options and/or defaults
2022-09-01 14:15:41 +01:00
Neil Alexander ad6b902b84
Refactor appservices component (#2687)
This PR refactors the app services component. It makes the following changes:

* Each appservice now gets its own NATS JetStream consumer
* The appservice database is now removed entirely, since we just use JetStream as a data source instead
* The entire component is now much simpler and we deleted lots of lines of code 💅

The result is that it should be much lighter and hopefully much more performant.
2022-09-01 09:20:40 +01:00