Commit graph

2467 commits

Author SHA1 Message Date
Tak Wai Wong d5050c1d61
merge latest changes from dendrite main (#15)
Co-authored-by: Tak Wai Wong <tak@hntlabs.com>
2022-10-24 17:52:01 -07:00
Tak Wai Wong b5d5f57a1c
Implement EIP-4361 sign in with Ethereum (#5)
* Blacklist some sytest tests that are failing in our environment

* Commenting out test that isn't reliably passing or failing, probably a race

* refresh latest dendrite main

* refresh latest dendrite main

* dendrite implementation of eip-4361

* simplify nonce generation

Co-authored-by: Brian Meek <brian@hntlabs.com>
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>
2022-10-24 17:43:01 -07:00
Tak Wai Wong 8ab1737e7f
Takwaiw/dendrite publickey (#2)
* Implementation of MSC 3782 Add publickey login as a new auth type.

Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>
2022-10-24 17:40:09 -07:00
Brian Meek 77e2c918fd
Test_UserStatistics Fix expected results to match observed results 2022-10-24 17:33:11 -07:00
Tak Wai Wong 916f17ceb7
Pull latest changes from dendrite fork (#193)
* Verify that the user ID for registration matches the spec, and the auth data (#10)

* Blacklist some sytest tests that are failing in our environment

* Commenting out test that isn't reliably passing or failing, probably a race

* refresh latest dendrite main

* pull latest from dendrite-fork subtree

* refresh latest dendrite main

* pull dendrite subtree and resolve merge conflicts

* check that userID matches the signed message

* verify that the user ID for registration is CAIP-10 compliant and MXID compliant

* removed space

Co-authored-by: Brian Meek <brian@hntlabs.com>
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>

* Fix nats.go commit (#2540)

Signed-off-by: Jean Lucas <jean@4ray.co>

* Don't return `end` if there are not more messages (#2542)

* Be more spec compliant

* Move lazyLoadMembers to own method

* Return an error if trying to invite a malformed user ID (#2543)

* Add `evacuateUser` endpoint, use it when deactivating accounts (#2545)

* Add `evacuateUser` endpoint, use it when deactivating accounts

* Populate the API

* Clean up user devices when deactivating

* Include invites, delete pushers

* Silence presence logs (#2547)

* Blacklist `Guest users can join guest_access rooms` test until it can be investigated

* Disable WebAssembly builds for now

* Try to fix backfilling (#2548)

* Try to fix backfilling

* Return start/end to not confuse clients

* Update GMSL

* Update GMSL

* Roomserver producers package (#2546)

* Give the roomserver a producers package

* Change init point

* Populate ACLs API

* Fix build issues

* `RoomEventProducer` naming

* Version 0.8.9 (#2549)

* Version 0.8.9

* Update changelog

* Takwaiw/fix concurrent registration bug (#12)

* fix concurrent registration bug. Rename decentralizedid

* remove unused module

* add regressed test to blacklist

Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>

* Test_UserStatistics Fix expected results to match observed results

* Takwaiw/dendrite publickey (#2)

* Implementation of MSC 3782 Add publickey login as a new auth type.

Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>

* Implement EIP-4361 sign in with Ethereum (#5)

* Blacklist some sytest tests that are failing in our environment

* Commenting out test that isn't reliably passing or failing, probably a race

* refresh latest dendrite main

* refresh latest dendrite main

* dendrite implementation of eip-4361

* simplify nonce generation

Co-authored-by: Brian Meek <brian@hntlabs.com>
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>

* Use rand.Seed to seed the random function generator (#6)

* Blacklist some sytest tests that are failing in our environment

* Commenting out test that isn't reliably passing or failing, probably a race

* refresh latest dendrite main

* use rand.Seed to seed the random function

Co-authored-by: Brian Meek <brian@hntlabs.com>
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>

* Create session ID during registration (#8)

* Blacklist some sytest tests that are failing in our environment

* Commenting out test that isn't reliably passing or failing, probably a race

* refresh latest dendrite main

* pull latest from dendrite-fork subtree

* refresh latest dendrite main

* Create session ID during registration

Co-authored-by: Brian Meek <brian@hntlabs.com>
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>

* Verify that the user ID for registration matches the spec, and the auth data (#10)

* Blacklist some sytest tests that are failing in our environment

* Commenting out test that isn't reliably passing or failing, probably a race

* refresh latest dendrite main

* pull latest from dendrite-fork subtree

* refresh latest dendrite main

* pull dendrite subtree and resolve merge conflicts

* check that userID matches the signed message

* verify that the user ID for registration is CAIP-10 compliant and MXID compliant

* removed space

Co-authored-by: Brian Meek <brian@hntlabs.com>
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>

* Takwaiw/fix concurrent registration bug (#12)

* fix concurrent registration bug. Rename decentralizedid

* remove unused module

* add regressed test to blacklist

Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>

* removed unused module

* feat+fix: Ignore unknown keys and verify required fields are present in appservice registration files (#2550)

* fix: ignore unknown keys in appservice configs

fixes matrix-org/dendrite#1567

* feat: verify required fields in appservice configs

* Use new testrig for key changes tests (#2552)

* Use new testrig for tests

* Log the error message

* Fix QuerySharedUsers for the SyncAPI keychange consumer (#2554)

* Make more use of base.BaseDendrite

* Fix QuerySharedUsers if no UserIDs are supplied

* Return clearer error when no state NID exists for an event (#2555)

* Wrap error from `SnapshotNIDFromEventID`

* Hopefully fix read receipts timestamps (#2557)

This should avoid coercions between signed and unsigned ints which might fix problems like `sql: converting argument $5 type: uint64 values with high bit set are not supported`.

* fix concurrency issue when checking session ID (#14)

Co-authored-by: Tak Wai Wong <tak@hntlabs.com>

* merge latest changes from dendrite main (#15)

Co-authored-by: Tak Wai Wong <tak@hntlabs.com>

* Login and Register tests for public key ethereum (#16)

* TestLoginPublicKeyNewSession

* use asserts

* setup, test, asserts

* TestLoginPublicKeyValidAuthTypeMissingSession

* invalid session id test

* create a helper newSession function

* TestLoginPublicKeyEthereumMissingUserId

* TestLoginPublicKeyEthereumAccountNotAvailable

* TestLoginPublicKeyEthereumInvalidUserId

* createEip4361TestMessage

* TestLoginPublicKeyEthereumMissingSignature

* TestLoginPublicKeyEthereum

* re-enable all publickey signin tests

* move common publickey test util to its own file

* register_public_key.go stub

* refactored common ethereum test helpers to its own folder

* refactor test helpers

* return error in test helpers

* fix regressions with ServerName

* TestRegistrationUnimplementedAlgo

* TestNewRegistration

* TestNewRegistrationSession

* verify new login session

* remove assert

* perform account creation

* TestRegisterEthereum

* Enable all tests

* move helper functions into test file

Co-authored-by: Tak Wai Wong <tak@hntlabs.com>

Co-authored-by: Brian Meek <brian@hntlabs.com>
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>
Co-authored-by: Jean Lucas <jean@4ray.co>
Co-authored-by: Till <2353100+S7evinK@users.noreply.github.com>
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
Co-authored-by: Tak Wai Wong <tak@hntlabs.com>
Co-authored-by: Kabir Kwatra <kabir@kwatra.me>
2022-10-24 17:26:35 -07:00
sergekh2 14d1a604ee
Config, certs and script to run local dendrite test server 2022-10-24 17:16:01 -07:00
Tak Wai Wong 1d906290cb
Takwaiw/pull latest dendrite fork changes (#137)
* Verify that the user ID for registration matches the spec, and the auth data (#10)

* Blacklist some sytest tests that are failing in our environment

* Commenting out test that isn't reliably passing or failing, probably a race

* refresh latest dendrite main

* pull latest from dendrite-fork subtree

* refresh latest dendrite main

* pull dendrite subtree and resolve merge conflicts

* check that userID matches the signed message

* verify that the user ID for registration is CAIP-10 compliant and MXID compliant

* removed space

Co-authored-by: Brian Meek <brian@hntlabs.com>
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>

* Fix nats.go commit (#2540)

Signed-off-by: Jean Lucas <jean@4ray.co>

* Don't return `end` if there are not more messages (#2542)

* Be more spec compliant

* Move lazyLoadMembers to own method

* Return an error if trying to invite a malformed user ID (#2543)

* Add `evacuateUser` endpoint, use it when deactivating accounts (#2545)

* Add `evacuateUser` endpoint, use it when deactivating accounts

* Populate the API

* Clean up user devices when deactivating

* Include invites, delete pushers

* Silence presence logs (#2547)

* Blacklist `Guest users can join guest_access rooms` test until it can be investigated

* Disable WebAssembly builds for now

* Try to fix backfilling (#2548)

* Try to fix backfilling

* Return start/end to not confuse clients

* Update GMSL

* Update GMSL

* Roomserver producers package (#2546)

* Give the roomserver a producers package

* Change init point

* Populate ACLs API

* Fix build issues

* `RoomEventProducer` naming

* Version 0.8.9 (#2549)

* Version 0.8.9

* Update changelog

* Takwaiw/fix concurrent registration bug (#12)

* fix concurrent registration bug. Rename decentralizedid

* remove unused module

* add regressed test to blacklist

Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>

* Test_UserStatistics Fix expected results to match observed results

* Takwaiw/dendrite publickey (#2)

* Implementation of MSC 3782 Add publickey login as a new auth type.

Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>

* Implement EIP-4361 sign in with Ethereum (#5)

* Blacklist some sytest tests that are failing in our environment

* Commenting out test that isn't reliably passing or failing, probably a race

* refresh latest dendrite main

* refresh latest dendrite main

* dendrite implementation of eip-4361

* simplify nonce generation

Co-authored-by: Brian Meek <brian@hntlabs.com>
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>

* Use rand.Seed to seed the random function generator (#6)

* Blacklist some sytest tests that are failing in our environment

* Commenting out test that isn't reliably passing or failing, probably a race

* refresh latest dendrite main

* use rand.Seed to seed the random function

Co-authored-by: Brian Meek <brian@hntlabs.com>
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>

* Create session ID during registration (#8)

* Blacklist some sytest tests that are failing in our environment

* Commenting out test that isn't reliably passing or failing, probably a race

* refresh latest dendrite main

* pull latest from dendrite-fork subtree

* refresh latest dendrite main

* Create session ID during registration

Co-authored-by: Brian Meek <brian@hntlabs.com>
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>

* Verify that the user ID for registration matches the spec, and the auth data (#10)

* Blacklist some sytest tests that are failing in our environment

* Commenting out test that isn't reliably passing or failing, probably a race

* refresh latest dendrite main

* pull latest from dendrite-fork subtree

* refresh latest dendrite main

* pull dendrite subtree and resolve merge conflicts

* check that userID matches the signed message

* verify that the user ID for registration is CAIP-10 compliant and MXID compliant

* removed space

Co-authored-by: Brian Meek <brian@hntlabs.com>
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>

* Takwaiw/fix concurrent registration bug (#12)

* fix concurrent registration bug. Rename decentralizedid

* remove unused module

* add regressed test to blacklist

Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>

* removed unused module

* feat+fix: Ignore unknown keys and verify required fields are present in appservice registration files (#2550)

* fix: ignore unknown keys in appservice configs

fixes matrix-org/dendrite#1567

* feat: verify required fields in appservice configs

* Use new testrig for key changes tests (#2552)

* Use new testrig for tests

* Log the error message

* Fix QuerySharedUsers for the SyncAPI keychange consumer (#2554)

* Make more use of base.BaseDendrite

* Fix QuerySharedUsers if no UserIDs are supplied

* Return clearer error when no state NID exists for an event (#2555)

* Wrap error from `SnapshotNIDFromEventID`

* Hopefully fix read receipts timestamps (#2557)

This should avoid coercions between signed and unsigned ints which might fix problems like `sql: converting argument $5 type: uint64 values with high bit set are not supported`.

* fix concurrency issue when checking session ID (#14)

Co-authored-by: Tak Wai Wong <tak@hntlabs.com>

* resolve merge conflicts

Co-authored-by: Brian Meek <brian@hntlabs.com>
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>
Co-authored-by: Jean Lucas <jean@4ray.co>
Co-authored-by: Till <2353100+S7evinK@users.noreply.github.com>
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
Co-authored-by: Tak Wai Wong <tak@hntlabs.com>
Co-authored-by: Kabir Kwatra <kabir@kwatra.me>
2022-10-24 17:15:58 -07:00
Tak Wai Wong f1d871fbea
Takwaiw/refresh dendrite fork (#132)
* Verify that the user ID for registration matches the spec, and the auth data (#10)

* Blacklist some sytest tests that are failing in our environment

* Commenting out test that isn't reliably passing or failing, probably a race

* refresh latest dendrite main

* pull latest from dendrite-fork subtree

* refresh latest dendrite main

* pull dendrite subtree and resolve merge conflicts

* check that userID matches the signed message

* verify that the user ID for registration is CAIP-10 compliant and MXID compliant

* removed space

Co-authored-by: Brian Meek <brian@hntlabs.com>
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>

* Fix nats.go commit (#2540)

Signed-off-by: Jean Lucas <jean@4ray.co>

* Don't return `end` if there are not more messages (#2542)

* Be more spec compliant

* Move lazyLoadMembers to own method

* Return an error if trying to invite a malformed user ID (#2543)

* Add `evacuateUser` endpoint, use it when deactivating accounts (#2545)

* Add `evacuateUser` endpoint, use it when deactivating accounts

* Populate the API

* Clean up user devices when deactivating

* Include invites, delete pushers

* Silence presence logs (#2547)

* Takwaiw/fix concurrent registration bug (#12)

* fix concurrent registration bug. Rename decentralizedid

* remove unused module

* add regressed test to blacklist

Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>

Co-authored-by: Brian Meek <brian@hntlabs.com>
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>
Co-authored-by: Jean Lucas <jean@4ray.co>
Co-authored-by: Till <2353100+S7evinK@users.noreply.github.com>
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-10-24 16:56:03 -07:00
Tak Wai Wong 57cdd1ff65
Latest dendrite main refresh (#129)
* Verify that the user ID for registration matches the spec, and the auth data (#10)

* Blacklist some sytest tests that are failing in our environment

* Commenting out test that isn't reliably passing or failing, probably a race

* refresh latest dendrite main

* pull latest from dendrite-fork subtree

* refresh latest dendrite main

* pull dendrite subtree and resolve merge conflicts

* check that userID matches the signed message

* verify that the user ID for registration is CAIP-10 compliant and MXID compliant

* removed space

Co-authored-by: Brian Meek <brian@hntlabs.com>
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>

* Fix nats.go commit (#2540)

Signed-off-by: Jean Lucas <jean@4ray.co>

* Don't return `end` if there are not more messages (#2542)

* Be more spec compliant

* Move lazyLoadMembers to own method

* Return an error if trying to invite a malformed user ID (#2543)

* Add `evacuateUser` endpoint, use it when deactivating accounts (#2545)

* Add `evacuateUser` endpoint, use it when deactivating accounts

* Populate the API

* Clean up user devices when deactivating

* Include invites, delete pushers

* Silence presence logs (#2547)

* update sytest blacklist

* Add new test that passed to the whitelist

* skip failed test

Co-authored-by: Brian Meek <brian@hntlabs.com>
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>
Co-authored-by: Jean Lucas <jean@4ray.co>
Co-authored-by: Till <2353100+S7evinK@users.noreply.github.com>
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-10-24 16:54:16 -07:00
Tak Wai Wong c28715c5ff
Verify that the user ID for registration matches the spec, and the auth data (#10) (#104)
* Blacklist some sytest tests that are failing in our environment

* Commenting out test that isn't reliably passing or failing, probably a race

* refresh latest dendrite main

* pull latest from dendrite-fork subtree

* refresh latest dendrite main

* pull dendrite subtree and resolve merge conflicts

* check that userID matches the signed message

* verify that the user ID for registration is CAIP-10 compliant and MXID compliant

* removed space

Co-authored-by: Brian Meek <brian@hntlabs.com>
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>

Co-authored-by: Brian Meek <brian@hntlabs.com>
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>
2022-10-24 16:42:34 -07:00
Tak Wai Wong 3e014fb70c
pull dendrite subtree and resolve merge conflicts 2022-10-24 16:39:43 -07:00
Tak Wai Wong 1683a17dbf
refresh latest dendrite main 2022-10-24 16:39:40 -07:00
Tak Wai Wong 34342934cc
pull latest from dendrite-fork subtree 2022-10-24 16:35:54 -07:00
Tak Wai Wong 88bbaed247
refresh latest dendrite main 2022-10-24 16:32:29 -07:00
Brian Meek 87851ae919
Blacklist some sytest tests that are failing in our environment 2022-10-24 16:22:06 -07:00
Tak Wai Wong a6668726ac
Takwaiw/dendrite publickey (#2)
* Implementation of MSC 3782 Add publickey login as a new auth type.

Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>
2022-10-24 16:21:40 -07:00
Brian Meek 0fa2f8ff60
Test_UserStatistics Fix expected results to match observed results 2022-10-24 16:17:55 -07:00
Till 7506e3303e
Get messages from before user left the room (#2824)
This is going to make `Can get rooms/{roomId}/messages for a departed
room (SPEC-216)` pass, since we now only grep events from before the
user left the room.
2022-10-24 17:03:04 +02:00
Neil Alexander a553fe7705
Fix slow querying of cross-signing signatures 2022-10-24 10:07:50 +01:00
Till Faelligen 0843bd776e
Fix wrong config key 2022-10-24 07:10:50 +02:00
Neil Alexander 411db6083b
Version 0.10.4 (#2822)
Changelog and version bump.
2022-10-21 15:00:51 +01:00
Till 3cf42a1d64
Add syncapi_memberships table tests (#2805) 2022-10-21 12:53:04 +02:00
Till 9e4c3171da
Optimize inserting pending PDUs/EDUs (#2821)
This optimizes the association of PDUs/EDUs to their destination by
inserting all destinations in one transaction.
2022-10-21 12:50:51 +02:00
Neil Alexander e98d75fd63
Verify room_id, type, sender and state_key field lengths using bytes rather than codepoints (update to matrix-org/gomatrixserverlib@7c772f1, reverts bbb3ade4a2) 2022-10-21 10:15:08 +01:00
Till e57b301722
Set display_name and/or avatar_url for server notices (#2820)
This should fix #2815 by making sure we actually set the `display_name`
and/or `avatar_url` and create the needed membership event.
To avoid creating a new membership event when starting Dendrite,
`SetAvatarURL` and `SetDisplayName` now return a `Changed` value, which
also makes the regular endpoints idempotent.
2022-10-21 10:48:25 +02:00
Till 40cfb9a4ea
Fix invite -> leave -> join dance when accepting invites (#2817)
As mentioned in
https://github.com/matrix-org/dendrite/issues/2361#issuecomment-1139394565
and observed by ourselves, this should fix the odd `invite -> leave ->
join` dance when accepting invites.
2022-10-21 09:26:22 +01:00
Neil Alexander 73e02463cf
Allow m.read.private to clear notifications (#2811)
Otherwise if a user switches to private read receipts, they may not be
able to clear notification counts.
2022-10-21 09:19:52 +01:00
devonh 9041491201
Mutex protect query keys response (#2812) 2022-10-20 15:54:18 +00:00
devonh b58c9bb094
Fix flakey queue test (#2818)
Ensure both events are added to the database, even if the destination is
already blacklisted.
2022-10-20 15:37:35 +00:00
Till Faelligen 539c61b3db
Remove test from blacklist 2022-10-20 12:34:53 +02:00
Till Faelligen 6a93858125
Fix race condition 2022-10-20 10:45:59 +02:00
Till e79bfd8fd5
Get state deltas without filters (#2810)
This makes the following changes:
- get state deltas without the user supplied filter, so we can actually
"calculate" state transitions
- closes `stmt` when using SQLite
- Adds presence for users who newly joined a room, even if the syncing
user already knows about the presence status (should fix
https://github.com/matrix-org/complement/pull/516)
2022-10-19 14:05:39 +02:00
Neil Alexander 8cbe14bd6d
Fix lock contention 2022-10-19 12:27:34 +01:00
Neil Alexander c1463db6c9
Fix concurrent map write in key server 2022-10-19 12:03:12 +01:00
dependabot[bot] f3dae0e749
Bump nokogiri from 1.13.6 to 1.13.9 in /docs (#2809)
Bumps [nokogiri](https://github.com/sparklemotion/nokogiri) from 1.13.6
to 1.13.9.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/sparklemotion/nokogiri/releases">nokogiri's
releases</a>.</em></p>
<blockquote>
<h2>1.13.9 / 2022-10-18</h2>
<h3>Security</h3>
<ul>
<li>[CRuby] Vendored libxml2 is updated to address <a
href="https://nvd.nist.gov/vuln/detail/CVE-2022-2309">CVE-2022-2309</a>,
<a
href="https://nvd.nist.gov/vuln/detail/CVE-2022-40304">CVE-2022-40304</a>,
and <a
href="https://nvd.nist.gov/vuln/detail/CVE-2022-40303">CVE-2022-40303</a>.
See <a
href="https://github.com/sparklemotion/nokogiri/security/advisories/GHSA-2qc6-mcvw-92cw">GHSA-2qc6-mcvw-92cw</a>
for more information.</li>
<li>[CRuby] Vendored zlib is updated to address <a
href="https://ubuntu.com/security/CVE-2022-37434">CVE-2022-37434</a>.
Nokogiri was not affected by this vulnerability, but this version of
zlib was being flagged up by some vulnerability scanners, see <a
href="https://github-redirect.dependabot.com/sparklemotion/nokogiri/issues/2626">#2626</a>
for more information.</li>
</ul>
<h3>Dependencies</h3>
<ul>
<li>[CRuby] Vendored libxml2 is updated to <a
href="https://gitlab.gnome.org/GNOME/libxml2/-/releases/v2.10.3">v2.10.3</a>
from v2.9.14.</li>
<li>[CRuby] Vendored libxslt is updated to <a
href="https://gitlab.gnome.org/GNOME/libxslt/-/releases/v1.1.37">v1.1.37</a>
from v1.1.35.</li>
<li>[CRuby] Vendored zlib is updated from 1.2.12 to 1.2.13. (See <a
href="https://github.com/sparklemotion/nokogiri/blob/v1.13.x/LICENSE-DEPENDENCIES.md#platform-releases">LICENSE-DEPENDENCIES.md</a>
for details on which packages redistribute this library.)</li>
</ul>
<h3>Fixed</h3>
<ul>
<li>[CRuby] <code>Nokogiri::XML::Namespace</code> objects, when
compacted, update their internal struct's reference to the Ruby object
wrapper. Previously, with GC compaction enabled, a segmentation fault
was possible after compaction was triggered. [<a
href="https://github-redirect.dependabot.com/sparklemotion/nokogiri/issues/2658">#2658</a>]
(Thanks, <a
href="https://github.com/eightbitraptor"><code>@​eightbitraptor</code></a>
and <a
href="https://github.com/peterzhu2118"><code>@​peterzhu2118</code></a>!)</li>
<li>[CRuby] <code>Document#remove_namespaces!</code> now defers freeing
the underlying <code>xmlNs</code> struct until the <code>Document</code>
is GCed. Previously, maintaining a reference to a <code>Namespace</code>
object that was removed in this way could lead to a segfault. [<a
href="https://github-redirect.dependabot.com/sparklemotion/nokogiri/issues/2658">#2658</a>]</li>
</ul>
<hr />
<p>sha256 checksums:</p>

<pre><code>9b69829561d30c4461ea803baeaf3460e8b145cff7a26ce397119577a4083a02
nokogiri-1.13.9-aarch64-linux.gem
e76ebb4b7b2e02c72b2d1541289f8b0679fb5984867cf199d89b8ef485764956
nokogiri-1.13.9-arm64-darwin.gem
15bae7d08bddeaa898d8e3f558723300137c26a2dc2632a1f89c8574c4467165
nokogiri-1.13.9-java.gem
f6a1dbc7229184357f3129503530af73cc59ceba4932c700a458a561edbe04b9
nokogiri-1.13.9-x64-mingw-ucrt.gem
36d935d799baa4dc488024f71881ff0bc8b172cecdfc54781169c40ec02cbdb3
nokogiri-1.13.9-x64-mingw32.gem
ebaf82aa9a11b8fafb67873d19ee48efb565040f04c898cdce8ca0cd53ff1a12
nokogiri-1.13.9-x86-linux.gem
11789a2a11b28bc028ee111f23311461104d8c4468d5b901ab7536b282504154
nokogiri-1.13.9-x86-mingw32.gem
01830e1646803ff91c0fe94bc768ff40082c6de8cfa563dafd01b3f7d5f9d795
nokogiri-1.13.9-x86_64-darwin.gem
8e93b8adec22958013799c8690d81c2cdf8a90b6f6e8150ab22e11895844d781
nokogiri-1.13.9-x86_64-linux.gem
96f37c1baf0234d3ae54c2c89aef7220d4a8a1b03d2675ff7723565b0a095531
nokogiri-1.13.9.gem
</code></pre>
<h2>1.13.8 / 2022-07-23</h2>
<h3>Deprecated</h3>
<ul>
<li><code>XML::Reader#attribute_nodes</code> is deprecated due to
incompatibility between libxml2's <code>xmlReader</code> memory
semantics and Ruby's garbage collector. Although this method continues
to exist for backwards compatibility, it is unsafe to call and may
segfault. This method will be removed in a future version of Nokogiri,
and callers should use <code>#attribute_hash</code> instead. [<a
href="https://github-redirect.dependabot.com/sparklemotion/nokogiri/issues/2598">#2598</a>]</li>
</ul>
<h3>Improvements</h3>
<ul>
<li><code>XML::Reader#attribute_hash</code> is a new method to safely
retrieve the attributes of a node from <code>XML::Reader</code>. [<a
href="https://github-redirect.dependabot.com/sparklemotion/nokogiri/issues/2598">#2598</a>,
<a
href="https://github-redirect.dependabot.com/sparklemotion/nokogiri/issues/2599">#2599</a>]</li>
</ul>
<h3>Fixed</h3>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/sparklemotion/nokogiri/blob/main/CHANGELOG.md">nokogiri's
changelog</a>.</em></p>
<blockquote>
<h2>1.13.9 / 2022-10-18</h2>
<h3>Security</h3>
<ul>
<li>[CRuby] Vendored libxml2 is updated to address <a
href="https://nvd.nist.gov/vuln/detail/CVE-2022-2309">CVE-2022-2309</a>,
<a
href="https://nvd.nist.gov/vuln/detail/CVE-2022-40304">CVE-2022-40304</a>,
and <a
href="https://nvd.nist.gov/vuln/detail/CVE-2022-40303">CVE-2022-40303</a>.
See <a
href="https://github.com/sparklemotion/nokogiri/security/advisories/GHSA-2qc6-mcvw-92cw">GHSA-2qc6-mcvw-92cw</a>
for more information.</li>
<li>[CRuby] Vendored zlib is updated to address <a
href="https://ubuntu.com/security/CVE-2022-37434">CVE-2022-37434</a>.
Nokogiri was not affected by this vulnerability, but this version of
zlib was being flagged up by some vulnerability scanners, see <a
href="https://github-redirect.dependabot.com/sparklemotion/nokogiri/issues/2626">#2626</a>
for more information.</li>
</ul>
<h3>Dependencies</h3>
<ul>
<li>[CRuby] Vendored libxml2 is updated to <a
href="https://gitlab.gnome.org/GNOME/libxml2/-/releases/v2.10.3">v2.10.3</a>
from v2.9.14.</li>
<li>[CRuby] Vendored libxslt is updated to <a
href="https://gitlab.gnome.org/GNOME/libxslt/-/releases/v1.1.37">v1.1.37</a>
from v1.1.35.</li>
<li>[CRuby] Vendored zlib is updated from 1.2.12 to 1.2.13. (See <a
href="https://github.com/sparklemotion/nokogiri/blob/v1.13.x/LICENSE-DEPENDENCIES.md#platform-releases">LICENSE-DEPENDENCIES.md</a>
for details on which packages redistribute this library.)</li>
</ul>
<h3>Fixed</h3>
<ul>
<li>[CRuby] <code>Nokogiri::XML::Namespace</code> objects, when
compacted, update their internal struct's reference to the Ruby object
wrapper. Previously, with GC compaction enabled, a segmentation fault
was possible after compaction was triggered. [<a
href="https://github-redirect.dependabot.com/sparklemotion/nokogiri/issues/2658">#2658</a>]
(Thanks, <a
href="https://github.com/eightbitraptor"><code>@​eightbitraptor</code></a>
and <a
href="https://github.com/peterzhu2118"><code>@​peterzhu2118</code></a>!)</li>
<li>[CRuby] <code>Document#remove_namespaces!</code> now defers freeing
the underlying <code>xmlNs</code> struct until the <code>Document</code>
is GCed. Previously, maintaining a reference to a <code>Namespace</code>
object that was removed in this way could lead to a segfault. [<a
href="https://github-redirect.dependabot.com/sparklemotion/nokogiri/issues/2658">#2658</a>]</li>
</ul>
<h2>1.13.8 / 2022-07-23</h2>
<h3>Deprecated</h3>
<ul>
<li><code>XML::Reader#attribute_nodes</code> is deprecated due to
incompatibility between libxml2's <code>xmlReader</code> memory
semantics and Ruby's garbage collector. Although this method continues
to exist for backwards compatibility, it is unsafe to call and may
segfault. This method will be removed in a future version of Nokogiri,
and callers should use <code>#attribute_hash</code> instead. [<a
href="https://github-redirect.dependabot.com/sparklemotion/nokogiri/issues/2598">#2598</a>]</li>
</ul>
<h3>Improvements</h3>
<ul>
<li><code>XML::Reader#attribute_hash</code> is a new method to safely
retrieve the attributes of a node from <code>XML::Reader</code>. [<a
href="https://github-redirect.dependabot.com/sparklemotion/nokogiri/issues/2598">#2598</a>,
<a
href="https://github-redirect.dependabot.com/sparklemotion/nokogiri/issues/2599">#2599</a>]</li>
</ul>
<h3>Fixed</h3>
<ul>
<li>[CRuby] Calling <code>XML::Reader#attributes</code> is now safe to
call. In Nokogiri &lt;= 1.13.7 this method may segfault. [<a
href="https://github-redirect.dependabot.com/sparklemotion/nokogiri/issues/2598">#2598</a>,
<a
href="https://github-redirect.dependabot.com/sparklemotion/nokogiri/issues/2599">#2599</a>]</li>
</ul>
<h2>1.13.7 / 2022-07-12</h2>
<h3>Fixed</h3>
<p><code>XML::Node</code> objects, when compacted, update their internal
struct's reference to the Ruby object wrapper. Previously, with GC
compaction enabled, a segmentation fault was possible after compaction
was triggered. [<a
href="https://github-redirect.dependabot.com/sparklemotion/nokogiri/issues/2578">#2578</a>]
(Thanks, <a
href="https://github.com/eightbitraptor"><code>@​eightbitraptor</code></a>!)</p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="897759cc25"><code>897759c</code></a>
version bump to v1.13.9</li>
<li><a
href="aeb1ac3283"><code>aeb1ac3</code></a>
doc: update CHANGELOG</li>
<li><a
href="c663e4905a"><code>c663e49</code></a>
Merge pull request <a
href="https://github-redirect.dependabot.com/sparklemotion/nokogiri/issues/2671">#2671</a>
from sparklemotion/flavorjones-update-zlib-1.2.13_v1...</li>
<li><a
href="212e07da28"><code>212e07d</code></a>
ext: hack to cross-compile zlib v1.2.13 on darwin</li>
<li><a
href="76dbc8c5be"><code>76dbc8c</code></a>
dep: update zlib to v1.2.13</li>
<li><a
href="24e3a9c414"><code>24e3a9c</code></a>
doc: update CHANGELOG</li>
<li><a
href="4db3b4daa9"><code>4db3b4d</code></a>
Merge pull request <a
href="https://github-redirect.dependabot.com/sparklemotion/nokogiri/issues/2668">#2668</a>
from sparklemotion/flavorjones-namespace-scopes-comp...</li>
<li><a
href="73d73d6e43"><code>73d73d6</code></a>
fix: Document#remove_namespaces! use-after-free bug</li>
<li><a
href="5f58b34724"><code>5f58b34</code></a>
fix: namespace nodes behave properly when compacted</li>
<li><a
href="b08a8586c7"><code>b08a858</code></a>
test: repro namespace_scopes compaction issue</li>
<li>Additional commits viewable in <a
href="https://github.com/sparklemotion/nokogiri/compare/v1.13.6...v1.13.9">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=nokogiri&package-manager=bundler&previous-version=1.13.6&new-version=1.13.9)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the
default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as
the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as
the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the
default for future PRs for this repo and language

You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/matrix-org/dendrite/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-10-19 11:40:38 +01:00
devonh 241d5c47df
Refactor Federation Destination Queues (#2807)
This is a refactor of the federation destination queues.
It fixes a few things, namely:
- actually retry outgoing events with backoff behaviour
- obtain enough events from the database to fill messages as much as
possible
- minimize the amount of running goroutines
  - use pure timers for backoff
  - don't restart queue unless necessary
  - close the background task when backing off
- increase max edus in a transaction to match the spec
- cleanup timers more aggresively to reduce memory usage
- add jitter to backoff timers to reduce resource spikes
- add a bunch of tests (with real and fake databases) to ensure
everything is working
2022-10-19 11:03:16 +01:00
Neil Alexander 3aa92efaa3
Namespace user API tables (#2806)
This migrates all the various user API tables, indices and sequences to
be `userapi_`-namespaced, rather than the mess they are all now.
2022-10-18 15:59:08 +01:00
Neil Alexander 9c189b1b80
Try to make AddEvent less expensive (update to matrix-org/gomatrixserverlib@a72a83f) 2022-10-18 09:51:31 +01:00
Till 07bfb791ca
Scope transactions to endpoints (#2799)
To avoid returning results from e.g. `/redact` on `/sendToDevice`
requests.
Takes the raw URL path and uses `filepath.Dir` to remove the `txnID`
(file) from it.

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-10-17 14:48:35 +02:00
Neil Alexander d72d4f8d5d
Set org.matrix.msc2285.stable in /versions 2022-10-17 10:38:22 +01:00
Till Faelligen 83c9dde219
Return error if we fail to read the response body 2022-10-17 07:27:11 +02:00
Neil Alexander 81dbad39a3
Dendrite 0.10.3
commit c6e18c18e93b54c006c6b4d0044aa53a0735a906
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Fri Oct 14 15:42:58 2022 +0100

    Changelog and version bump
2022-10-14 15:53:58 +01:00
Neil Alexander cd8f7e1251
Set inactivity threshold on durable consumers in the roomserver input API (#2795)
This prevents us from holding onto durable consumers indefinitely for
rooms that have long since turned inactive, since they do have a bit of
a processing overhead in the NATS Server. If we clear up a consumer and
then a room becomes active again, the consumer gets recreated as needed.

The threshold is set to 24 hours for now, we can tweak it later if needs
be.
2022-10-14 15:14:29 +01:00
Neil Alexander eac5678449
Update dependency now that it is fixed 2022-10-14 14:53:53 +01:00
Neil Alexander f76969831e
Update direct dependencies (#2794)
This updates a number of Dendrite's dependencies.
2022-10-14 10:59:32 +01:00
Neil Alexander 82d1d434c5
Update to NATS Server v2.9.3 and nats.go v1.18.0 2022-10-14 10:10:25 +01:00
Till a8bc558a60
Always add UnreadNotifications to joined room reponses (#2793)
Fixes a minor bug, where we failed to add `UnreadNotifications` to the
join response, if it wasn't in
`GetUserUnreadNotificationCountsForRooms`.
2022-10-14 10:38:12 +02:00
Till Faelligen fb44e33909
Relax test a bit 2022-10-14 10:37:04 +02:00
Till 088ad1dd21
Fix outliers whose auth_events are in a different room are correctly rejected (#2791)
Fixes `outliers whose auth_events are in a different room are correctly
rejected`, by validating that auth events are all from the same room and
not using rejected events for event auth.
2022-10-14 09:14:54 +02:00
Neil Alexander f3be4b3185
Revert "Federation backoff fixes and tests (#2792)"
This reverts commit dcedd1b6bf.
2022-10-13 16:06:50 +01:00