Commit graph

298 commits

Author SHA1 Message Date
Neil Alexander 9e46d5025a
Upgrade various dependencies (#2496)
* Upgrade various dependencies

* Yaml dependency too
2022-05-26 12:34:43 +01:00
Neil Alexander 5a878b6e14
Update to matrix-org/gomatrixserverlib#314 2022-05-26 11:52:39 +01:00
Neil Alexander ff53398635
Mark room versions 8 and 9 as stable 2022-05-25 16:39:51 +01:00
Neil Alexander ac4d0072cf
Fix using the remote provided event from /send_join (#2490) 2022-05-25 14:31:07 +01:00
Neil Alexander d621dd2986
Allow authing invited user in rooms with "restricted" join rule (update to matrix-org/gomatrixserverlib#313) 2022-05-24 11:08:28 +01:00
Neil Alexander 8a7567c652
Fix control flow related to joining restricted v8/v9 rooms (update to matrix-org/gomatrixserverlib#312) 2022-05-23 17:49:48 +01:00
Neil Alexander 028963af1c
Update to matrix-org/gomatrixserverlib#311 (#2479)
* Update to matrix-org/gomatrixserverlib#311

* Update to squash commit
2022-05-23 14:26:56 +01:00
Neil Alexander be9be2553f
Resolve over old and new extremities (#2457)
* Feed existing state into state res when calculating state from new extremities

* Remove duplicates

* Fix bug

* Sort and unique

* Update to matrix-org/gomatrixserverlib#308

* Trim the slice properly

* Update gomatrixserverlib again

* Update to matrix-org/gomatrixserverlib#308
2022-05-13 11:52:04 +01:00
Neil Alexander cafc2d2c10
Update NATS Server to version 2.8.2 (#2460) 2022-05-13 11:36:04 +01:00
Till f69ebc6af2
Add roomserver tests (1/?) (#2434)
* Add EventJSONTable tests

* Add eventJSON tests

* Add EventStateKeysTable tests

* Add EventTypesTable tests

* Add Events Table tests
Move variable declaration outside loops
Switch to testify/assert for tests

* Move variable declaration outside loop

* Remove random data

* Fix issue where the EventReferenceSHA256 is not set

* Add more tests

* Revert "Fix issue where the EventReferenceSHA256 is not set"

This reverts commit 8ae34c4e5f.

* Update GMSL

* Add tests for duplicate entries

* Test what happens if we select non-existing NIDs

* Add test for non-existing eventType

* Really update GMSL
2022-05-09 15:30:32 +02:00
Neil Alexander 79e2fbc663
Update to matrix-org/gomatrixserverlib#307 2022-05-09 13:53:51 +01:00
Neil Alexander 85c00208c5
Fix power level event auth bugs (update to matrix-org/gomatrixserverlib#306) 2022-05-06 15:41:16 +01:00
Neil Alexander a1a5357f79
Produce more useful event auth errors (update to matrix-org/gomatrixserverlib#305) 2022-05-06 12:46:01 +01:00
Neil Alexander e4da04e75b
Update to matrix-org/gomatrixserverlib#303 2022-05-05 14:06:05 +01:00
Neil Alexander d9e71b93b6
Use gomatrixserverlib.Client instead of http.Client (#2421)
* Update to matrix-org/gomatrixserverlib#303

* Use `gomatrixserverlib.Client` for phone-home stats

* Use `gomatrixserverlib.Client` for push notifications

* Use `gomatrixserverlib.Client` for appservices

* Use `gomatrixserverlib.Client` for three-PID invites
2022-05-05 11:33:16 +01:00
Till f023cdf8c4
Add UserAPI storage tests (#2384)
* Add tests for parts of the userapi storage

* Add tests for keybackup

* Add LoginToken tests

* Add OpenID tests

* Add profile tests

* Add pusher tests

* Add ThreePID tests

* Add notification tests

* Add more device tests, fix numeric localpart query

* Fix failing CI

* Fix numeric local part query
2022-04-27 15:05:49 +02:00
Neil Alexander d7cc187ec0
Prevent JetStream from handling OS signals, allow running as a Windows service (#2385)
* Prevent JetStream from handling OS signals, allow running as a Windows service (fixes #2374)

* Remove double import
2022-04-27 13:36:40 +01:00
Neil Alexander 144c060fa7
Remove libp2p demos (#2337)
* Remove libp2p demos

* Put it back
2022-04-21 13:31:43 +01:00
Neil Alexander a9f0a390c6
Update to NATS Server v2.8.0 and nats.go v1.14.0 (#2359) 2022-04-20 15:13:04 +01:00
Neil Alexander 711e377b9c
Update go-sqlite3-js to matrix-org/go-sqlite3-js#2 (SQLite 3.36.0) 2022-04-19 10:34:33 +01:00
Till 69f2ff7c82
Correctly use provided filters (#2339)
* Apply filters correctly

* Fix issues; Use prepareWithFilters

* Update gmsl & tests

* go.mod..

* PR comments
2022-04-11 09:05:23 +02:00
Neil Alexander 986d27a128
Update to matrix-org/pinecone@2999ea2 2022-04-08 16:39:09 +01:00
Neil Alexander 9bd9f2beba
Update to matrix-org/pinecone@9b3963248c 2022-04-08 16:23:50 +01:00
Neil Alexander 4312b87ab0
Update dependencies (#2336) 2022-04-08 12:24:49 +01:00
Neil Alexander 4d9d9cc9b1
Update to matrix-org/gomatrixserverlib#300 2022-04-05 14:43:44 +01:00
Neil Alexander 562d742240
Update to matrix-org/gomatrixserverlib#299 2022-04-04 19:04:17 +01:00
Neil Alexander c69159bda7
Update to matrix-org/pinecone@e526fa8 2022-04-04 15:16:02 +01:00
Neil Alexander 8213b2ba30
Update Pinecone P2P demo 2022-03-30 15:01:22 +01:00
Neil Alexander 6c95f9b535
Update to matrix-org/pinecone@6fb0773 2022-03-23 14:45:22 +00:00
Neil Alexander c54d88aecf
Update to matrix-org/pinecone@8687b0d 2022-03-18 15:32:14 +00:00
Neil Alexander 0fb94fc781
Update to matrix-org/gomatrixserverlib@0980b7f 2022-03-17 16:48:41 +00:00
Neil Alexander 1894f34868
Update to matrix-org/gomatrixserverlib@a62d9f3 2022-03-17 09:58:44 +00:00
Neil Alexander 047147a3af
Update Yggdrasil P2P demo (also remove QUIC dependency) 2022-03-16 16:49:10 +00:00
Neil Alexander 0ad665fabc
Update to NATS Server 2.7.4 (#2282)
* Update to NATS Server v2.7.4-beta2

* Bump to release v2.7.4

* Listen for TCP

* Revert "Listen for TCP"

This reverts commit 28dff0ab4d.
2022-03-16 10:47:18 +00:00
Neil Alexander 2696f36738
Federation client improvements (HTTP/2, reaping federation trippers, TLS client session cache, keepalive tweaks, update to matrix-org/gomatrixserverlib#297) 2022-03-16 10:27:16 +00:00
Neil Alexander e1881627d1
Implement MSC3383 (update to matrix-org/gomatrixserverlib#296) 2022-03-10 12:44:31 +00:00
Neil Alexander 030b995636
Update to matrix-org/pinecone@cfde1f8 2022-03-09 10:19:49 +00:00
Neil Alexander 0297929b76
Revert NATS server upgrade in 00b3545b14 2022-03-04 15:56:31 +00:00
Dan f05ce478f0
Implement Push Notifications (#1842)
* Add Pushserver component with Pushers API

Co-authored-by: Tommie Gannert <tommie@gannert.se>
Co-authored-by: Dan Peleg <dan@globekeeper.com>

* Wire Pushserver component

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>

* Add PushGatewayClient.

The full event format is required for Sytest.

* Add a pushrules module.

* Change user API account creation to use the new pushrules module's defaults.

Introduces "scope" as required by client API, and some small field
tweaks to make some 61push Sytests pass.

* Add push rules query/put API in Pushserver.

This manipulates account data over User API, and fires sync messages
for changes. Those sync messages should, according to an existing TODO
in clientapi, be moved to userapi.

Forks clientapi/producers/syncapi.go to pushserver/ for later extension.

* Add clientapi routes for push rules to Pushserver.

A cleanup would be to move more of the name-splitting logic into
pushrules.go, to depollute routing.go.

* Output rooms.join.unread_notifications in /sync.

This is the read-side. Pushserver will be the write-side.

* Implement pushserver/storage for notifications.

* Use PushGatewayClient and the pushrules module in Pushserver's room consumer.

* Use one goroutine per user to avoid locking up the entire server for
  one bad push gateway.
* Split pushing by format.
* Send one device per push. Sytest does not support coalescing
  multiple devices into one push. Matches Synapse. Either we change
  Sytest, or remove the group-by-url-and-format logic.
* Write OutputNotificationData from push server. Sync API is already
  the consumer.

* Implement read receipt consumers in Pushserver.

Supports m.read and m.fully_read receipts.

* Add clientapi route for /unstable/notifications.

* Rename to UpsertPusher for clarity and handle pusher update

* Fix linter errors

* Ignore body.Close() error check

* Fix push server internal http wiring

* Add 40 newly passing 61push tests to whitelist

* Add next 12 newly passing 61push tests to whitelist

* Send notification data before notifying users in EDU server consumer

* NATS JetStream

* Goodbye sarama

* Fix `NewStreamTokenFromString`

* Consume on the correct topic for the roomserver

* Don't panic, NAK instead

* Move push notifications into the User API

* Don't set null values since that apparently causes Element upsetti

* Also set omitempty on conditions

* Fix bug so that we don't override the push rules unnecessarily

* Tweak defaults

* Update defaults

* More tweaks

* Move `/notifications` onto `r0`/`v3` mux

* User API will consume events and read/fully read markers from the sync API with stream positions, instead of consuming directly

Co-authored-by: Piotr Kozimor <p1996k@gmail.com>
Co-authored-by: Tommie Gannert <tommie@gannert.se>
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-03-03 11:40:53 +00:00
Neil Alexander 00b3545b14
Update NATS Server to v2.7.3 2022-03-02 10:36:20 +00:00
Kegan Dougal 1a79060b46 Bump GMSL version 2022-03-01 14:16:47 +00:00
kegsay f1b92de017
MSC2946: Spaces Summary (round 2) (#2232)
* Initial cut at fixing up MSC2946 to work with latest spec

* bugfix: send response back correctly

* Initial working version of MSC2946

* msc2946: handle suggested_only; remove custom database

As the MSC doesn't require reverse lookups, we can just pull
the room state and inspect via the roomserver database. To
handle this, expand QueryCurrentState to support wildcards.

Use all this and handle `?suggested_only`.

* Sort child rooms

* msc2946: Make TestClientSpacesSummary pass

* msc2946: allow invited rooms to be spidered

* msc2946: support basic federation requests

* fix up go mod
2022-03-01 13:40:07 +00:00
Neil Alexander a23fda6626
Update Events call-sites which now don't return an error, update parsedRespState to sort (#2227)
* Topologically sort with `SendEventWithState`, so that earlier events should satisfy auth for later ones

* Revert "Topologically sort with `SendEventWithState`, so that earlier events should satisfy auth for later ones"

This reverts commit b0cd706012.

* Update to matrix-org/gomatrixserverlib#293

* `Events` no longer returns an error, other tweaks

* Make sure `Events` is sorted for `parsedRespState` too
2022-02-28 14:51:40 +00:00
Neil Alexander 4c07374c42
Reduce allocations significantly in state res v2, which should help to keep memory down when joining rooms too (update to matrix-org/gomatrixserverlib@f6ab9c5) 2022-02-24 17:05:49 +00:00
Neil Alexander b8a97b6ee0
Update to matrix-org/pinecone@0f0afd1a46 2022-02-23 10:45:07 +00:00
Neil Alexander 934491eda5
Update NATS Server to v2.7.2 (#2193)
* Update NATS JetStream to v2.7.2

* Remove deprecated option
2022-02-17 13:15:35 +00:00
Neil Alexander 4c8c53244e
Update prometheus 2022-02-15 16:27:22 +00:00
Neil Alexander e22e87c012
Update to matrix-org/gomatrixserverlib@20632dd 2022-02-15 11:14:43 +00:00
kegsay aa5c3b88de
Unmarshal events at the Dendrite level not GMSL level (#2164)
* Use new event json types in gmsl

* Fix EventJSON to actually unmarshal events

* Update GMSL

* Bump GMSL and improve error messages

* Send back the correct RespState

* Update GMSL
2022-02-09 20:31:24 +00:00
Neil Alexander a2b4777ae5
Update to matrix-org/gomatrixserverlib@a05e156fd8 2022-02-08 09:30:21 +00:00