Commit graph

2758 commits

Author SHA1 Message Date
Daniel Aloni 129fc42c0b ♻️ Use origin_server_ts instead of timestamp in MR /sync responses. 2023-04-04 11:47:51 +03:00
Daniel Aloni bf0f05b8fe 🛠️ Substitutions for production CD. 2023-03-22 17:47:21 +02:00
Daniel Aloni 9177911152 🐳 Build docker image without --mount. 2023-03-19 11:10:42 +02:00
Daniel Aloni e16a7363e7 🐛 Declaring BUILDPLATFORM before usage on Dockerfile + default substitution 2023-03-15 18:20:02 +02:00
Daniel Aloni 5af369f426 🚀 CD build with subtitution variable in cloudbuild. 2023-03-15 16:33:43 +02:00
Daniel Aloni 9f7c1804b2 🚀 CD build. 2023-03-15 16:19:22 +02:00
Daniel Aloni e2d12de97d
Merge pull request #66 from globekeeper/release/upstream-v0.12.0
Release/upstream v0.12.0
2023-03-15 15:44:09 +02:00
Daniel Aloni 4da7d19fec 🐛 Return servername on completeAuth. 2023-03-15 15:04:38 +02:00
Daniel Aloni 747dcfbfdf Don't fail ci if erroring on codecov integration test. 2023-03-15 13:37:32 +02:00
Daniel Aloni a639e1e0aa Disable pinecone tests and relay api. 2023-03-15 13:13:43 +02:00
Daniel Aloni c550c2e8cb Merge remote-tracking branch 'origin' into release/upstream-v0.12.0 2023-03-15 12:30:11 +02:00
Till 74dc54684b
Version 0.12.0 (#3009) 2023-03-13 20:02:57 +01:00
Till 232aef016c
Add basic runtime tracing (#2996)
This allows us in almost all places to use regions to further trace down
long running tasks.
Also removes an unused function.
2023-03-13 16:45:14 +01:00
Till 689b5ee72f
Change default stats reporting endpoint (#3007)
It's the same instance we report to, only using the subdomain.
2023-03-10 12:27:08 +01:00
Till c7303cbf76
Update dependencies (#3006)
In preparation for a new release, let's also update a few dependencies.
2023-03-10 10:32:50 +01:00
Till 70322699ab
Unset RoomServerEvent, since we can't be sure that Set actually updates the cached entry (#3002)
This should deflake UTs and be more correct in terms of getting
`Events`.
`Events` tries to fetch the event from the cache first and may get an
unredacted event from it, while it should already be redacted.
2023-03-09 09:52:13 +01:00
Till Faelligen baef523cb0
Fix invalid roomNID returned 2023-03-07 15:35:08 +01:00
Till Faelligen 11a3fcc6cb
RoomServerEvents are mutable, given they can be redacted 2023-03-06 17:58:08 +01:00
Till Faelligen a684b850b9
Actually ignore the error if we were able to backfill events 2023-03-06 17:45:21 +01:00
Till 7d83f8b633
Add tests for UpdateRelations (#2999)
This also fixes an issue regarding updates to relations for invalid
events, which could result in us retrying said event over and over
again, if we fail to unmarshal the event to
`gomatrixserverlib.RelationContent`, this was discovered by
`@sleroq:virto.community`
2023-03-06 12:43:59 +01:00
Tim McCormack 7fc839f751
Update admin-promotion instructions; clarify ID for evacuation (#2997)
Table name has changed since instructions were written.

There's probably a better way to describe how to get the internal room
ID than I've attempted here, so feel free to adjust as needed. (It may
even be good to show an example of what an internal room ID looks like,
e.g. `!nc93825:example.com`)

### Pull Request Checklist

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
  * Doc-only change
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)

Signed-off-by: `Tim McCormack <cortex@brainonfire.net>`

---------

Co-authored-by: Till <2353100+S7evinK@users.noreply.github.com>
2023-03-05 18:42:38 +01:00
Till Faelligen 56b28b01db
Update the cache with the redacted event 2023-03-03 14:49:41 +01:00
Till 9bcd0a2105
Make redaction check easier to read (#2995)
We need to check the redaction PL in Dendrite, if we do it in GMSL, we
end up not sending the event to the output stream because it will be
rejected.

---------

Co-authored-by: kegsay <kegan@matrix.org>
2023-03-03 14:03:17 +01:00
Robin Westerik 7cde99a7a7
Updated instructions and references to monolith to their new names (#2994)
Currently, the documentation makes use of the old names for the binary
and configuration files. This updates the documentation so that users
can follow the guide without issues again.
These changes don't require any go unit tests because it does not modify
any golang code.

Signed-off-by: `Robin Westerik <gh@westerik.me>`
2023-03-03 10:20:53 +01:00
Boris Rybalkin 6b1c9eafa9
unix socket support (#2974)
### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `Boris Rybalkin <ribalkin@gmail.com>`

I need this for Syncloud project (https://github.com/syncloud/platform)
where I run multiple apps behind an nginx on the same RPi like device so
unix socket is very convenient to not have port conflicts between apps.
Also someone opened this Issue:
https://github.com/matrix-org/dendrite/issues/2924

---------

Co-authored-by: kegsay <kegan@matrix.org>
Co-authored-by: Till <2353100+S7evinK@users.noreply.github.com>
2023-03-01 22:57:30 +01:00
Till 6c20f8f742
Refactor StoreEvent, add MaybeRedactEvent, create an EventDatabase (#2989)
This PR changes the following:
- `StoreEvent` now only stores an event (and possibly prev event),
instead of also doing redactions
- Adds a `MaybeRedactEvent` (pulled out from `StoreEvent`), which should
be called after storing events
- a few other things
2023-03-01 17:06:47 +01:00
Till Faelligen 1aa70b0f56
Fix UTs 2023-03-01 15:09:10 +01:00
Till Faelligen f1ccfcf150
Only run CI if there are changes to go files or the workflow [skip ci] 2023-02-28 15:35:53 +01:00
Till Faelligen 086e205eba
Deploy on gh-pages push 2023-02-28 15:15:19 +01:00
genofire 7fff7cd2ac
feat(helm): add prometheus resources to monitor (#2958)
### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

---

I do not know, how you run helm-docs ....


otherwise i would like to add somewhere:
````markdown
* Works well with [Prometheus Operator](https://prometheus-operator.dev/) ([Helmchart](https://artifacthub.io/packages/helm/prometheus-community/kube-prometheus-stack)) and there setup of [Grafana](https://grafana.com/grafana/), by enabling following values:
```yaml
prometheus:
  servicemonitor:
    enabled: true
    labels:
      release: "kube-prometheus-stack"
  rules:
    enabled: true # will deploy alert rules
    additionalLabels:
      release: "kube-prometheus-stack"
grafana:
  dashboards:
    enabled: true # will deploy default dashboards 
```
PS: The labels `release=kube-prometheus-stack` is setup with the helmchart of the Prometheus Operator. For Grafana Dashboards it maybe need scan enable to scan in correct namespaces (or ALL), enabled by `sidecar.dashboards.searchNamespace` in [Helmchart of grafana](https://artifacthub.io/packages/helm/grafana/grafana) (which is part of PrometheusOperator, so `grafana.sidecar.dashboards.searchNamespace`) 
````

Maybe also put somewhere the Screenshot of that Grafana Dashboard:
https://grafana.com/grafana/dashboards/13916-dendrite/

---


@S7evinK do you take a look?

Signed-off-by: genofire <geno+dev@fireorbit.de>
2023-02-28 14:18:26 +01:00
Daniel Aloni d08130a375
Merge pull request #65 from globekeeper/DEV-378/forward-original-ip
🐛 Do not store proxy ips as device's ip.
2023-02-27 19:03:26 +02:00
Daniel Aloni 90e2b5cecc 🐛 Do not store proxy ips as device's ip. 2023-02-27 19:01:28 +02:00
Daniel Aloni 937afcfa2d
Merge pull request #64 from globekeeper/DEV-378/forward-original-ip
🌐 Forward IP for login/register device creation (external API)
2023-02-27 18:06:36 +02:00
Daniel Aloni c40555c836 Weird lint error. 2023-02-27 17:22:56 +02:00
Daniel Aloni 0d9f37f427 🌐 Forward IP for login/register device creation (external API) 2023-02-27 17:00:26 +02:00
Devon Hudson eddf31f915
Fix lint error 2023-02-24 15:49:51 -07:00
Devon Hudson b28406c7d0
Tweaks to pinecone demo to shutdown more cleanly 2023-02-24 15:41:47 -07:00
Till Faelligen 3d31b131fc
Cache all the things 2023-02-24 11:45:01 +01:00
Till ad07b169b8
Refactor StoreEvent and create a new RoomDatabase interface (#2985)
This PR changes a few things:
- It pulls out the creation of several NIDs from the `StoreEvent`
function to make the functions more reusable
- Uses more caching when using those NIDs to avoid DB round trips
2023-02-24 09:40:20 +01:00
David Schneider e6aa0955ff
Unify logging by using logrus for jetstream logs (#2976)
I guess tests for the logging is rather unusual so I omitted tests for
this change.

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `David Schneider <dsbrng25b@gmail.com>`

---------

Signed-off-by: David Schneider <dsbrng25b@gmail.com>
2023-02-24 08:56:53 +01:00
Yoann N d34277a6c0
Update README sample config link (#2987)
### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately
2023-02-23 09:07:20 +01:00
Till Faelligen c8ca23acdb
Fix building Element web in CI 2023-02-20 15:28:45 +01:00
Till 7f114cc538
Fix issue where device keys are removed if a device ID is reused (#2982)
Fixes https://github.com/matrix-org/dendrite/issues/2980
2023-02-20 15:26:09 +01:00
Till 4594233f89
Merge keyserver & userapi (#2972)
As discussed yesterday, a first draft of merging the keyserver and the
userapi.
2023-02-20 14:58:03 +01:00
dependabot[bot] bd6f0c14e5
Bump golang.org/x/net from 0.5.0 to 0.7.0 (#2979)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.5.0 to
0.7.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="8e2b117aee"><code>8e2b117</code></a>
http2/hpack: avoid quadratic complexity in hpack decoding</li>
<li><a
href="547e7edf38"><code>547e7ed</code></a>
http2: avoid referencing ResponseWrite.Write parameter after
returning</li>
<li><a
href="39940adcaa"><code>39940ad</code></a>
html: parse comments per HTML spec</li>
<li><a
href="87ce33ecb4"><code>87ce33e</code></a>
go.mod: update golang.org/x dependencies</li>
<li><a
href="415cb6d518"><code>415cb6d</code></a>
all: fix some comments</li>
<li><a
href="7e3c19ca52"><code>7e3c19c</code></a>
all: correct typos in comments</li>
<li><a
href="296f09aa38"><code>296f09a</code></a>
http2: case insensitive handling for 100-continue</li>
<li><a
href="f8411da775"><code>f8411da</code></a>
nettest: fix tests on dragonfly and js/wasm</li>
<li>See full diff in <a
href="https://github.com/golang/net/compare/v0.5.0...v0.7.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golang.org/x/net&package-manager=go_modules&previous-version=0.5.0&new-version=0.7.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the
default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as
the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as
the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the
default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/matrix-org/dendrite/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-17 19:33:40 +01:00
dependabot[bot] 22c4736495
Bump golang.org/x/image from 0.1.0 to 0.5.0 (#2978)
Bumps [golang.org/x/image](https://github.com/golang/image) from 0.1.0
to 0.5.0.
<details>
<summary>Commits</summary>
<ul>
<li><a
href="e6c2a4cdd5"><code>e6c2a4c</code></a>
tiff: don't pre-allocate giant slices before reading</li>
<li><a
href="3db422c472"><code>3db422c</code></a>
go.mod: update golang.org/x dependencies</li>
<li><a
href="bb712eb67b"><code>bb712eb</code></a>
go.mod: update golang.org/x dependencies</li>
<li><a
href="9b8a3be425"><code>9b8a3be</code></a>
font/plan9font: limit the font image's dimensions</li>
<li><a
href="c5235ae222"><code>c5235ae</code></a>
font/plan9font: fix byteoffset for non-zero origin</li>
<li><a
href="0888fdd524"><code>0888fdd</code></a>
font/plan9font: fix bounds overflow</li>
<li><a
href="9fdfde75ec"><code>9fdfde7</code></a>
go.mod: update golang.org/x dependencies</li>
<li>See full diff in <a
href="https://github.com/golang/image/compare/v0.1.0...v0.5.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=golang.org/x/image&package-manager=go_modules&previous-version=0.1.0&new-version=0.5.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the
default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as
the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as
the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the
default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/matrix-org/dendrite/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-02-17 16:33:49 +01:00
Till f0805071d5
Fix SQLite session_id (#2977)
This fixes an issue with device_id/session_ids.
If a `device_id` is reused, we would reuse the same `session_id`, since
we delete one device and insert a new one directly, resulting in the
query to get a new `session_id` to return the previous session_id.
(`SELECT count(access_token)`)
2023-02-17 11:39:46 +01:00
Till 11d9b9db0e
Remove polylith/API mode (#2967)
This removes most of the code used for polylith/API mode.

This removes the `/api` internal endpoints entirely. 

Binary size change roughly 5%: 
```
51437560 Feb 13 10:15 dendrite-monolith-server # old
48759008 Feb 13 10:15 dendrite-monolith-server # new
```
2023-02-14 12:47:47 +01:00
Till cc59879faa
Version 0.11.1 (#2966) 2023-02-10 18:36:59 +01:00
ShalokShalom e64ed0934d
Update coverage (#2964)
### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `Matthias Gramberg <ShalokShalom@riseup.net>`
2023-02-08 08:56:16 +01:00