Commit graph

2895 commits

Author SHA1 Message Date
Till 0d0280cf5f
/sync performance optimizations (#2927)
Since #2849 there is no limit for the current state we fetch to
calculate history visibility. In large rooms this can cause us to fetch
thousands of membership events we don't really care about.
This now only gets the state event types and senders in our timeline,
which should significantly reduce the amount of events we fetch from the
database.

Also removes `MaxTopologicalPosition`, as it is an unnecessary DB call,
given we use the result in `topological_position < $1` calls.
2023-01-17 10:08:23 +01:00
Umar Getagazov 8582c7520a
Omit state field from /messages response if empty (#2940)
The field type is `[ClientEvent]` in the
[spec](https://spec.matrix.org/v1.5/client-server-api/#get_matrixclientv3roomsroomidmessages),
but right now `null` can also be returned. Omit the field completely if
it's empty. Some clients (rightfully) assume it's either not present at
all or it's of the right type (see
https://github.com/matrix-org/matrix-react-sdk/pull/9913).

### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
    * The PR is a simple struct tag fix
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `Umar Getagazov <umar@handlerug.me>`

Signed-off-by: Umar Getagazov <umar@handlerug.me>
2023-01-17 09:07:42 +01:00
Andrew Morgan eeeb3017d6
Switch the default config option values for recaptcha_sitekey_class and recaptcha_form_field (#2939)
Attempting to use the [web auth fallback
mechanism](https://spec.matrix.org/v1.5/client-server-api/#fallback) for
Google ReCAPTCHA with the default setting for
`client_api.recaptcha_sitekey_class` of "g-recaptcha-response" results
in no captcha being rendered:


![image](https://user-images.githubusercontent.com/1342360/212482321-14980045-6e20-4d59-adaa-59a01ad88367.png)

I cross-checked the captcha code between [dendrite.matrix.org's fallback
page](https://dendrite.matrix.org/_matrix/client/r0/auth/m.login.recaptcha/fallback/web?session=asdhjaksd)
and [matrix-client.matrix.org's
one](https://matrix-client.matrix.org/_matrix/client/r0/auth/m.login.recaptcha/fallback/web?session=asdhjaksd)
(which both use the same captcha public key) and noticed a discrepancy
in the `class` attribute of the div that renders the captcha.
[ReCAPTCHA's docs
state](https://developers.google.com/recaptcha/docs/v3#automatically_bind_the_challenge_to_a_button)
to use "g-recaptcha" as the class for the submit button.

I noticed this when user `@parappanon:parappa.party` reported that they
were also seeing no captcha being rendered on their Dendrite instance.
Changing `client_api.recaptcha_sitekey_class` to "g-recaptcha" caused
their captcha to render properly as well.

There may have been a change in the class name from ReCAPTCHA v2 to v3?
The [docs for
v2](https://developers.google.com/recaptcha/docs/display#auto_render)
also request one uses "g-recaptcha" though.

Thus I propose changing the default setting to unbreak people's
recaptcha auth fallback pages. Should fix dendrite.matrix.org as well.
2023-01-16 12:52:30 +01:00
Devon Hudson 477a44faa6
Always initialize statistics server map 2023-01-12 09:23:03 -07:00
Till 0491a8e343
Fix room summary returning wrong heroes (#2930)
This should fix #2910.
Probably makes Sytest/Complement a bit upset, since this not using
`sort.Strings` anymore.
2023-01-12 10:06:03 +01:00
Devon Hudson 25dfbc6ec3
Extend cypress test timeout in ci 2023-01-11 10:47:37 -07:00
Devon Hudson 6ae1dd565c
Revert "Add cypress cloud recording"
This reverts commit b297ea7379.
2023-01-11 10:46:52 -07:00
Devon Hudson b297ea7379
Add cypress cloud recording 2023-01-11 10:40:38 -07:00
Devon Hudson 8fef692741
Edit cypress config before running tests 2023-01-11 10:10:24 -07:00
Devon Hudson 11a07d855d
Initial attempt at adding cypress tests to ci 2023-01-11 09:52:58 -07:00
Devon Hudson 97ebd72b5a
Add FAQs based on commonly asked questions from the community 2023-01-10 16:26:41 -07:00
devonh 7482cd2b47
Handle DisplayName field in admin user registration endpoint (#2935)
`/_synapse/admin/v1/register` has a `displayname` field that we were
previously ignoring.
This handles that field and adds the displayname to the new user if one
was provided.
2023-01-10 18:09:25 +00:00
Till b0c5af6674
Fix /login issue causing wrong device list updates (#2922)
Fixes https://github.com/matrix-org/dendrite/issues/2914 and possibly
https://github.com/matrix-org/dendrite/issues/2073?
2023-01-10 17:02:38 +01:00
Devon Hudson 0995dc4822
Add curl to dendrite-demo-pinecone docker container 2023-01-06 12:02:43 -07:00
Devon Hudson 54b47a98e5
Add curl to dendrite docker containers 2023-01-06 11:49:59 -07:00
Till Faelligen 3fd95e60cc
Try that again 2023-01-06 15:54:04 +01:00
Till Faelligen 002310390f
Output to docs folder, hopefully 2023-01-06 15:51:07 +01:00
Till d579ddb8e7
Add simplified helm chart (#2905)
As discussed yesterday, a simplified version of [my
helm](https://github.com/S7evinK/dendrite-helm) which deploys a monolith
with internal NATS and an optionally enabled PostgreSQL server. If the
PostgreSQL dependency is not enabled, a user specified connection string
is constructed.

Co-authored-by: kegsay <kegan@matrix.org>
2023-01-06 15:44:10 +01:00
Till 2e1fe58937
Fix backfilling (#2926)
This should fix https://github.com/matrix-org/dendrite/issues/2923
2023-01-05 09:24:00 +01:00
Till e449d174cc
Add possibility to run complement with coverage enabled (#2901)
This adds the possibility to run Complement with coverage enabled.
In combination with https://github.com/matrix-org/complement/pull/566 we
should then be able to extract the coverage logs, combine them with
https://github.com/wadey/gocovmerge (or similar) and upload them to
Codecov (with different flags, depending on SQLite, HTTP etc.)
2022-12-23 14:28:15 +01:00
Till f762ce1050
Add clientapi tests (#2916)
This PR
- adds several tests for the clientapi, mostly around `/register` and
auth fallback.
- removes the now deprecated `homeserver` field from responses to
`/register` and `/login`
- slightly refactors auth fallback handling
2022-12-23 14:11:11 +01:00
Till f47515e38b
Pushrule tweaks, make pattern non-optional on EventMatchCondition (#2918)
This should fix https://github.com/matrix-org/dendrite/issues/2882
(Tested with FluffyChat 1.7.1)
Also adds tests that the predefined push rules (as per the spec) is what
we have in Dendrite.
2022-12-23 12:52:47 +01:00
Till 5eed31fea3
Handle guest access [1/2?] (#2872)
Needs https://github.com/matrix-org/sytest/pull/1315, as otherwise the
membership events aren't persisted yet when hitting `/state` after
kicking guest users.

Makes the following tests pass:
```
Guest users denied access over federation if guest access prohibited
Guest users are kicked from guest_access rooms on revocation of guest_access
Guest users are kicked from guest_access rooms on revocation of guest_access over federation
```

Todo (in a follow up PR):
- Restrict access to CS API Endpoints as per
https://spec.matrix.org/v1.4/client-server-api/#client-behaviour-14

Co-authored-by: kegsay <kegan@matrix.org>
2022-12-22 13:05:59 +01:00
Till Faelligen 09dff951d6
More flakey tests 2022-12-22 13:04:32 +01:00
Till d1d2d16738
Fix reset password endpoint (#2921)
Fixes the admin password reset endpoint.
It was using a wrong variable, so could not detect the user.
Adds some more checks to validate we can actually change the password.
2022-12-22 11:54:03 +01:00
Till Faelligen beea2432e6
Fix flakey test 2022-12-22 11:31:54 +01:00
Till d3db542fbf
Add federation peeking table tests (#2920)
As the title says, adds tests for inbound/outbound peeking federation
table tests.

Also removes some unused code
2022-12-22 10:56:20 +01:00
Kento Okamoto 76db8e90de
Dendrite Documentation Fix (#2913)
### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->
I was reading through the Dendrite documentation on
https://matrix-org.github.io/dendrite/development/contributing and
noticed the installation link leads to a 404 error. This link works fine
if it is viewed directly from
[docs/CONTRIBUTING.md](https://github.com/matrix-org/dendrite/blob/main/docs/CONTRIBUTING.md)
but this might not be very obvious to new contributors who are reading
through the [contribution
page](https://matrix-org.github.io/dendrite/development/contributing)
directly.

This PR is mainly a small re-organization of the online documentation
mainly in the
[Development](https://matrix-org.github.io/dendrite/development) tab
along with any links throughout the doc that may be impacted by the
change. This does not contain any Go unit tests as this does not
actually touch core dendrite functionality.

* [ ] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `Kento Okamoto <kentokamoto@proton.me>`
2022-12-12 16:46:37 +00:00
Till 7d2344049d
Cleanup stale device lists for users we don't share a room with anymore (#2857)
The stale device lists table might contain entries for users we don't
share a room with anymore. This now asks the roomserver about left users
and removes those entries from the table.

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-12-12 08:20:59 +01:00
Till Faelligen aaf4e5c865
Use older sytest-dendrite image 2022-12-09 18:45:42 +01:00
dependabot[bot] 8846de7312
Bump nokogiri from 1.13.9 to 1.13.10 in /docs (#2909)
Bumps [nokogiri](https://github.com/sparklemotion/nokogiri) from 1.13.9
to 1.13.10.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/sparklemotion/nokogiri/releases">nokogiri's
releases</a>.</em></p>
<blockquote>
<h2>1.13.10 / 2022-12-07</h2>
<h3>Security</h3>
<ul>
<li>[CRuby] Address CVE-2022-23476, unchecked return value from
<code>xmlTextReaderExpand</code>. See <a
href="https://github.com/sparklemotion/nokogiri/security/advisories/GHSA-qv4q-mr5r-qprj">GHSA-qv4q-mr5r-qprj</a>
for more information.</li>
</ul>
<h3>Improvements</h3>
<ul>
<li>[CRuby] <code>XML::Reader#attribute_hash</code> now returns
<code>nil</code> on parse errors. This restores the behavior of
<code>#attributes</code> from v1.13.7 and earlier. [<a
href="https://github-redirect.dependabot.com/sparklemotion/nokogiri/issues/2715">#2715</a>]</li>
</ul>
<hr />
<p>sha256 checksums:</p>

<pre><code>777ce2e80f64772e91459b943e531dfef387e768f2255f9bc7a1655f254bbaa1
nokogiri-1.13.10-aarch64-linux.gem
b432ff47c51386e07f7e275374fe031c1349e37eaef2216759063bc5fa5624aa
nokogiri-1.13.10-arm64-darwin.gem
73ac581ddcb680a912e92da928ffdbac7b36afd3368418f2cee861b96e8c830b
nokogiri-1.13.10-java.gem
916aa17e624611dddbf2976ecce1b4a80633c6378f8465cff0efab022ebc2900
nokogiri-1.13.10-x64-mingw-ucrt.gem
0f85a1ad8c2b02c166a6637237133505b71a05f1bb41b91447005449769bced0
nokogiri-1.13.10-x64-mingw32.gem
91fa3a8724a1ce20fccbd718dafd9acbde099258183ac486992a61b00bb17020
nokogiri-1.13.10-x86-linux.gem
d6663f5900ccd8f72d43660d7f082565b7ffcaade0b9a59a74b3ef8791034168
nokogiri-1.13.10-x86-mingw32.gem
81755fc4b8130ef9678c76a2e5af3db7a0a6664b3cba7d9fe8ef75e7d979e91b
nokogiri-1.13.10-x86_64-darwin.gem
51d5246705dedad0a09b374d09cc193e7383a5dd32136a690a3cd56e95adf0a3
nokogiri-1.13.10-x86_64-linux.gem
d3ee00f26c151763da1691c7fc6871ddd03e532f74f85101f5acedc2d099e958
nokogiri-1.13.10.gem
</code></pre>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/sparklemotion/nokogiri/blob/main/CHANGELOG.md">nokogiri's
changelog</a>.</em></p>
<blockquote>
<h2>1.13.10 / 2022-12-07</h2>
<h3>Security</h3>
<ul>
<li>[CRuby] Address CVE-2022-23476, unchecked return value from
<code>xmlTextReaderExpand</code>. See <a
href="https://github.com/sparklemotion/nokogiri/security/advisories/GHSA-qv4q-mr5r-qprj">GHSA-qv4q-mr5r-qprj</a>
for more information.</li>
</ul>
<h3>Improvements</h3>
<ul>
<li>[CRuby] <code>XML::Reader#attribute_hash</code> now returns
<code>nil</code> on parse errors. This restores the behavior of
<code>#attributes</code> from v1.13.7 and earlier. [<a
href="https://github-redirect.dependabot.com/sparklemotion/nokogiri/issues/2715">#2715</a>]</li>
</ul>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="4c80121dc3"><code>4c80121</code></a>
version bump to v1.13.10</li>
<li><a
href="85410e3841"><code>85410e3</code></a>
Merge pull request <a
href="https://github-redirect.dependabot.com/sparklemotion/nokogiri/issues/2715">#2715</a>
from sparklemotion/flavorjones-fix-reader-error-hand...</li>
<li><a
href="9fe0761c47"><code>9fe0761</code></a>
fix(cruby): XML::Reader#attribute_hash returns nil on error</li>
<li><a
href="3b9c736bee"><code>3b9c736</code></a>
Merge pull request <a
href="https://github-redirect.dependabot.com/sparklemotion/nokogiri/issues/2717">#2717</a>
from sparklemotion/flavorjones-lock-psych-to-fix-bui...</li>
<li><a
href="2efa87b49a"><code>2efa87b</code></a>
test: skip large cdata test on system libxml2</li>
<li><a
href="3187d6739c"><code>3187d67</code></a>
dep(dev): pin psych to v4 until v5 builds in CI</li>
<li><a
href="a16b4bf14c"><code>a16b4bf</code></a>
style(rubocop): disable Minitest/EmptyLineBeforeAssertionMethods</li>
<li>See full diff in <a
href="https://github.com/sparklemotion/nokogiri/compare/v1.13.9...v1.13.10">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=nokogiri&package-manager=bundler&previous-version=1.13.9&new-version=1.13.10)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the
default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as
the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as
the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the
default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/matrix-org/dendrite/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-12-08 10:19:55 +00:00
Till c136a450d5
Fix newly joined users presence (#2854)
Fixes #2803 
Also refactors the presence stream to not hit the database for every
user, instead queries all users at once now.
2022-12-08 08:25:03 +01:00
Till 0351618ff4
Add UserAPI util tests (#2907)
This adds some `userapi/util` tests.
2022-12-08 08:24:24 +01:00
Till 27a1dea522
Fix issue with multiple/duplicate log entries during tests (#2906) 2022-12-08 08:24:06 +01:00
Neil Alexander ba2ffb7da9
Repeatable reads for /sync (#2783)
This puts repeatable reads into all sync streams.

Co-authored-by: kegsay <kegan@matrix.org>
2022-12-06 18:16:17 +00:00
Till Faelligen ded43e0f2d
Fix issue with sending presence events to invalid servers 2022-12-06 13:27:33 +01:00
kegsay 7583478305
Update contributing guidelines (#2904) 2022-12-05 16:54:01 +00:00
Till Faelligen b99349b18c
Use test.WithAllDatabases 2022-12-05 16:00:02 +01:00
Till Faelligen 3dc06bea81
Differentiate between project and patch 2022-12-05 15:49:11 +01:00
Till Faelligen 0e6d94757b
Enforce coverage 2022-12-05 15:24:36 +01:00
Till Faelligen 07e8ed13f6
Fix CI and test.WithAllDatabases 2022-12-05 15:09:59 +01:00
Till e245a26f6b
Enable/Disable internal metrics (#2899)
Basically enables us to use `test.WithAllDatabases` when testing
internal HTTP APIs, as this would otherwise result in Prometheus
complaining about already registered metric names.
2022-12-05 13:53:36 +01:00
Till b65f89e61e
Add tests for the AS internal API (#2898) 2022-12-02 16:42:23 +01:00
Till 9a46d8d95c
Test and CI related changes (#2896)
In an attempt to:
- make on-boarding a bit easier (`go test ./...` should now not need
additional postgres setup)
- get code coverage faster, not only scheduled at night
- test the `create-account` binary
2022-12-02 11:44:20 +01:00
Neil Alexander 934056f21f
Fix dendrite-demo-pinecone, /_dendrite namespace setup 2022-12-01 10:45:15 +00:00
Neil Alexander 1be0afa181
Expose /_dendrite and /_synapse on the P2P demo HTTP muxes 2022-12-01 10:24:17 +00:00
Neil Alexander 6f000e9801
Make create-account more verbose 2022-12-01 10:14:26 +00:00
Neil Alexander f009e54181
Push rule evaluation tweaks (#2897)
This tweaks push rule evaluation:

1. to be more strict around pattern matching and to not match empty
patterns
3. to bail if we come across a `dont_notify`, since cycles after that
are wasted
4. refactors `ActionsToTweaks` to make a bit more sense
2022-11-30 12:54:37 +00:00
Till ac5f3f025e
Calculate correct room member count for push rule evaluation (#2894)
Fixes a bug where we would return only the local member count, which
could result in wrongly calculated push rules.
2022-11-30 11:40:36 +00:00
Neil Alexander ed497aa8b2
Version 0.10.8 2022-11-29 16:26:33 +00:00