Commit graph

341 commits

Author SHA1 Message Date
Kegan Dougal 3543517f24 Linting 2023-05-05 10:00:10 +01:00
Kegan Dougal e359a0b36a Linting 2023-05-04 17:48:03 +01:00
Kegan Dougal 328d3aaffd Allow self-hosted port to be selected 2023-05-04 16:55:34 +01:00
Kegan Dougal 6393b06a27 Revamp furl to be more useful
- Allow skipping TLS checks with -k (for localhost dendrites)
- Allow self-hosting server keys (for ephemeral servers)
2023-05-04 13:11:23 +01:00
kegsay f5b3144dc3
Use PDU not *Event in HeaderedEvent (#3073)
Requires https://github.com/matrix-org/gomatrixserverlib/pull/376

This has numerous upsides:
 - Less type casting to `*Event` is required.
- Making Dendrite work with `PDU` interfaces means we can swap out Event
impls more easily.
 - Tests which represent weird event shapes are easier to write.

Part of a series of refactors on GMSL.
2023-05-02 15:03:16 +01:00
kegsay 6171310307
Use PDU interface (#3070)
We only use it in a few places currently, enough to get things to
compile and run. We should be using it in much more places.

Similarly, in some places we cast []PDU back to []*Event, we need to not
do that. Likewise, in some places we cast PDU to *Event, we need to not
do that. For now though, hopefully this is a start.
2023-04-27 16:35:19 +01:00
devonh ed19efc5d7
Move fedclient interface over to gmsl (#3061)
Companion PR: https://github.com/matrix-org/gomatrixserverlib/pull/366
2023-04-24 16:23:25 +00:00
kegsay 4679098a64
Use IRoomVersion (#3064)
This is a step towards allowing arbitrary room version impls.
2023-04-24 11:50:37 +01:00
kegsay 72285b2659
refactor: update GMSL (#3058)
Sister PR to https://github.com/matrix-org/gomatrixserverlib/pull/364

Read this commit by commit to avoid going insane.
2023-04-19 15:50:33 +01:00
kegsay 0db43f13a6
refactor: use latest GMSL which splits fed client from matrix room logic (#3051)
Part of a series of refactors on GMSL.
2023-04-06 09:55:01 +01:00
Devon Hudson f4104b4b5d
Pinecone-demo: Wait on dendrite before shutting down 2023-03-27 17:19:53 -06:00
Till 5e85a00cb3
Remove BaseDendrite (#3023)
Removes `BaseDendrite` to, hopefully, make testing and composing of
components easier in the future.
2023-03-22 09:21:32 +01:00
Till 5579121c6f
Preparations for removing BaseDendrite (#3016)
Preparations to actually remove/replace `BaseDendrite`.
Quite a few changes:
- SyncAPI accepts an `fulltext.Indexer` interface (fulltext is removed
from `BaseDendrite`)
- Caches are removed from `BaseDendrite`
- Introduces a `Router` struct (likely to change)
  - also fixes #2903
- Introduces a `sqlutil.ConnectionManager`, which should remove
`base.DatabaseConnection` later on
- probably more
2023-03-17 11:09:45 +00:00
Boris Rybalkin d88f71ab71
simplify unix socket permission format (#3014)
### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `Boris Rybalkin <ribalkin@gmail.com>`
2023-03-16 08:51:21 +01:00
Robin Westerik 7cde99a7a7
Updated instructions and references to monolith to their new names (#2994)
Currently, the documentation makes use of the old names for the binary
and configuration files. This updates the documentation so that users
can follow the guide without issues again.
These changes don't require any go unit tests because it does not modify
any golang code.

Signed-off-by: `Robin Westerik <gh@westerik.me>`
2023-03-03 10:20:53 +01:00
Boris Rybalkin 6b1c9eafa9
unix socket support (#2974)
### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `Boris Rybalkin <ribalkin@gmail.com>`

I need this for Syncloud project (https://github.com/syncloud/platform)
where I run multiple apps behind an nginx on the same RPi like device so
unix socket is very convenient to not have port conflicts between apps.
Also someone opened this Issue:
https://github.com/matrix-org/dendrite/issues/2924

---------

Co-authored-by: kegsay <kegan@matrix.org>
Co-authored-by: Till <2353100+S7evinK@users.noreply.github.com>
2023-03-01 22:57:30 +01:00
Till 6c20f8f742
Refactor StoreEvent, add MaybeRedactEvent, create an EventDatabase (#2989)
This PR changes the following:
- `StoreEvent` now only stores an event (and possibly prev event),
instead of also doing redactions
- Adds a `MaybeRedactEvent` (pulled out from `StoreEvent`), which should
be called after storing events
- a few other things
2023-03-01 17:06:47 +01:00
Devon Hudson eddf31f915
Fix lint error 2023-02-24 15:49:51 -07:00
Devon Hudson b28406c7d0
Tweaks to pinecone demo to shutdown more cleanly 2023-02-24 15:41:47 -07:00
Till ad07b169b8
Refactor StoreEvent and create a new RoomDatabase interface (#2985)
This PR changes a few things:
- It pulls out the creation of several NIDs from the `StoreEvent`
function to make the functions more reusable
- Uses more caching when using those NIDs to avoid DB round trips
2023-02-24 09:40:20 +01:00
Till 4594233f89
Merge keyserver & userapi (#2972)
As discussed yesterday, a first draft of merging the keyserver and the
userapi.
2023-02-20 14:58:03 +01:00
Till 11d9b9db0e
Remove polylith/API mode (#2967)
This removes most of the code used for polylith/API mode.

This removes the `/api` internal endpoints entirely. 

Binary size change roughly 5%: 
```
51437560 Feb 13 10:15 dendrite-monolith-server # old
48759008 Feb 13 10:15 dendrite-monolith-server # new
```
2023-02-14 12:47:47 +01:00
Devon Hudson a666c06da1
Consolidate pinecone demo http server variations 2023-02-01 14:11:48 -07:00
Devon Hudson 048e35026c
Refactor common pinecone demo code to remove major duplication 2023-02-01 13:41:38 -07:00
Devon Hudson dbc2869cbd
Refactor pinecone demo to remove duplicate pinecone setup 2023-02-01 13:41:38 -07:00
Devon Hudson d4f64f91ca
Refactor pinecone demo to remove duplicate key setup 2023-02-01 13:41:37 -07:00
Devon Hudson 529feb07ee
Refactor conduit type from pinecone demo into its own package 2023-02-01 13:41:37 -07:00
Devon Hudson be43b9c0ea
Refactor common relay sync struct to remove duplication 2023-02-01 13:41:36 -07:00
Devon Hudson f98003c030
Add cmd line option to pinecone demo for enabling relaying 2023-01-29 18:13:39 -07:00
Devon Hudson 0f998e3af3
Add pinecone demo toggle for dis/enabling relaying for other nodes 2023-01-29 12:26:16 -07:00
Devon Hudson 24a865aeb7
Move relay arch into relayapi and add docs for new endpoints 2023-01-26 10:15:53 -07:00
devonh 5b73592f5a
Initial Store & Forward Implementation (#2917)
This adds store & forward relays into dendrite for p2p.
A few things have changed:
- new relay api serves new http endpoints for s&f federation
- updated outbound federation queueing which will attempt to forward
using s&f if appropriate
- database entries to track s&f relays for other nodes
2023-01-23 17:55:12 +00:00
Till 48fa869fa3
Use t.TempDir for SQLite databases, so tests don't rip out each others databases (#2950)
This should hopefully finally fix issues about `disk I/O error` as seen
[here](https://gitlab.alpinelinux.org/alpine/aports/-/jobs/955030/raw)

Hopefully this will also fix `SSL accept attempt failed` issues by
disabling HTTP keep alives when generating a config for CI.
2023-01-23 13:17:15 +01:00
Bernhard Feichtinger a2b4860912
Fix oversight in cmd/generate-config (#2946)
The -dir argument was ignored for media_api->base_path.
Signed-off-by: `Bernhard Feichtinger
<43303168+BieHDC@users.noreply.github.com>`
2023-01-20 13:13:36 +01:00
Till f762ce1050
Add clientapi tests (#2916)
This PR
- adds several tests for the clientapi, mostly around `/register` and
auth fallback.
- removes the now deprecated `homeserver` field from responses to
`/register` and `/login`
- slightly refactors auth fallback handling
2022-12-23 14:11:11 +01:00
Till 7d2344049d
Cleanup stale device lists for users we don't share a room with anymore (#2857)
The stale device lists table might contain entries for users we don't
share a room with anymore. This now asks the roomserver about left users
and removes those entries from the table.

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-12-12 08:20:59 +01:00
Till e245a26f6b
Enable/Disable internal metrics (#2899)
Basically enables us to use `test.WithAllDatabases` when testing
internal HTTP APIs, as this would otherwise result in Prometheus
complaining about already registered metric names.
2022-12-05 13:53:36 +01:00
Till 9a46d8d95c
Test and CI related changes (#2896)
In an attempt to:
- make on-boarding a bit easier (`go test ./...` should now not need
additional postgres setup)
- get code coverage faster, not only scheduled at night
- test the `create-account` binary
2022-12-02 11:44:20 +01:00
Neil Alexander 934056f21f
Fix dendrite-demo-pinecone, /_dendrite namespace setup 2022-12-01 10:45:15 +00:00
Neil Alexander 1be0afa181
Expose /_dendrite and /_synapse on the P2P demo HTTP muxes 2022-12-01 10:24:17 +00:00
Neil Alexander 6f000e9801
Make create-account more verbose 2022-12-01 10:14:26 +00:00
devonh a8e7ffc7ab
Add p2p wakeup broadcast handling to pinecone demos (#2841)
Adds wakeup broadcast handling to the pinecone demos.
This will reset their blacklist status and interrupt any ongoing
federation queue backoffs currently in progress for this peer.
The end result is that any queued events will quickly be sent to the
peer if they had disconnected while attempting to send events to them.
2022-11-18 00:29:23 +00:00
Neil Alexander 6650712a1c
Federation fixes for virtual hosting 2022-11-15 15:05:23 +00:00
Neil Alexander 72ce6acf71
Run upgrade tests for SQLite too (#2875)
This should hopefully catch problems with database migrations in SQLite
as well as PostgreSQL.
2022-11-11 11:21:16 +00:00
Neil Alexander efa50253f6
Fix lint error 2022-11-10 10:16:56 +00:00
Neil Alexander 503d9c7586
Improve logging in upgrade tests 2022-11-10 10:07:19 +00:00
devonh 4afadebd99
Add readme for dendrite-demo-pinecone (#2851) 2022-11-02 20:29:30 +00:00
Neil Alexander ca8bc87380
Multi-stage Docker builds (#2850)
This builds on @S7evinK's work to make multi-stage Docker builds. Now
that we can build SQLite without Cgo this should be much simpler and
should make Docker builds in CI significantly faster.

Co-authored-by: Till Faelligen <tfaelligen@gmail.com>
Co-authored-by: Till Faelligen <davidf@element.io>
Co-authored-by: Till Faelligen <2353100+S7evinK@users.noreply.github.com>
2022-11-02 14:04:08 +00:00
0x1a8510f2 51ab0a8ccf
Fix moderncsqlite errors and rebase onto main (#2832)
This is #2819 but rebased on latest `main`. This PR is against main too
as opposed to the `moderncsqlite` branch.

The main change here is simply:

```go
// add query parameters to the dsn
if strings.Contains(dsn, "?") {
	dsn += "&"
} else {
	dsn += "?"
}

// wait some time before erroring if the db is locked
// https://gitlab.com/cznic/sqlite/-/issues/106#note_1058094993
dsn += "_pragma=busy_timeout%3d10000"
```

### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added tests for PR _or_ I have justified why this PR
doesn't need tests.
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed off privately.

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-11-02 13:20:10 +00:00
Neil Alexander 238b6ef2cd
Update Yggdrasil demo 2022-10-26 18:37:01 +01:00