Commit graph

853 commits

Author SHA1 Message Date
Neil Alexander 4664f02b4d Implement wildcard 2020-05-29 16:59:32 +01:00
Neil Alexander 43a83d375e Try to line up sync tokens again 2020-05-29 16:22:52 +01:00
Neil Alexander 2ef53ff726 nolint on getResponseWithPDUsForCompleteSync 2020-05-29 15:51:36 +01:00
Neil Alexander 0177ea04c4 Comments 2020-05-29 15:50:34 +01:00
Neil Alexander ded130f548 Break out the retrieval from the update/delete behaviour 2020-05-29 15:41:04 +01:00
Neil Alexander 3ca5d8e21b Fix tests 2020-05-29 13:50:56 +01:00
Neil Alexander fc1303bbbe Jiggle about sync a bit 2020-05-29 13:43:37 +01:00
Neil Alexander 7a073b14fc Use latest sync position 2020-05-29 12:04:32 +01:00
Neil Alexander cc456240fa Update sync position 2020-05-29 11:48:42 +01:00
Neil Alexander 5487edf1d6 Try to avoid database locks 2020-05-29 11:38:19 +01:00
Neil Alexander 005c4c0168 Fix storing and retrieving of send-to-device messages 2020-05-29 10:07:57 +01:00
Neil Alexander 4fda6c65e6 Debug logging 2020-05-29 09:39:17 +01:00
Neil Alexander 49498fb794 Fix queries 2020-05-28 18:40:55 +01:00
Neil Alexander 544f3b0a61 Fix schema 2020-05-28 18:40:26 +01:00
Neil Alexander e4eba873f5 Refactor a bit 2020-05-28 18:38:23 +01:00
Neil Alexander f4376c4582 Comments 2020-05-28 17:53:40 +01:00
Neil Alexander 5389a62952 Add send-to-device test, hopefully fix bugs 2020-05-28 17:49:56 +01:00
Neil Alexander 1e3ad88172 Fix a couple of transaction things, fix client API 2020-05-28 17:25:28 +01:00
Neil Alexander bf5df4f438 Logging 2020-05-28 16:45:54 +01:00
Neil Alexander 587108fe31 Fix sync 2020-05-28 15:52:41 +01:00
Neil Alexander 581e5929e1 pq.Array 2020-05-28 15:43:01 +01:00
Neil Alexander e3267cf12a Undo changes to test 2020-05-28 15:39:29 +01:00
Neil Alexander 4d6347b21a Handle incoming send-to-device messages, count them with EDU stream pos 2020-05-28 15:35:49 +01:00
Neil Alexander d3bf9cb31b Add some logic for send-to-device messages, add them into sync stream 2020-05-28 15:11:44 +01:00
Neil Alexander 8b7e81b423 Create send-to-device database tables, other tweaks 2020-05-28 14:34:17 +01:00
Neil Alexander 9694907ccc fix indentation in dendrite-config.yaml 2020-05-28 13:46:21 +01:00
Neil Alexander d4d4915194 Merge branch 'master' into neilalexander/sendtodevice 2020-05-28 13:45:34 +01:00
Neil Alexander 5307c499fe
Update gomatrixserverlib (#1071) 2020-05-28 13:44:16 +01:00
Neil Alexander 1b7ad3e4e0 Merge branch 'master' into neilalexander/sendtodevice 2020-05-28 12:48:02 +01:00
Neil Alexander f123d7d65b
Hopefully fix get_missing_events (#1070) 2020-05-28 12:44:34 +01:00
Kegan Dougal 2411007c4b Parse URIs correctly 2020-05-28 12:28:21 +01:00
Kegsay a6f995eb45
Merge Updater structs (#1069)
* Move Updater structs to shared and use it for postgres

* Add constructors for NewXXXUpdater and a useTxns flag

In sqlite, we set useTxns=false and comment why.

* Handle nil txn

* Handle nil in transaction

* Missed one

* Close the txn at the right time

* Don't close the transaction as we reuse it between calls
2020-05-28 11:15:21 +01:00
Neil Alexander 23a16d1a57 Merge branch 'master' into neilalexander/sendtodevice 2020-05-28 10:13:08 +01:00
Neil Alexander 02fe38e1f7
Per-user-per-device sync streams (#1068)
* Per-user-per-device sync streams

* Tweaks

* Tweaks

* Pass full device into CompleteSync

* Set user IDs and device IDs properly in tests

* Add new test, fix TestNewEventAndWasPreviouslyJoinedToRoom

* nolint a function that is not used yet

* Add test for waking up single device

* Hopefully unstick test

* Try to ensure that TestCorrectStreamWakeup doesn't block forever

* Update tests
2020-05-28 10:05:04 +01:00
Neil Alexander ba6e55ecf5 Start the send-to-device consumer 2020-05-27 15:04:06 +01:00
Neil Alexander 062325b6c6 Send to device consumer in sync API 2020-05-27 14:37:13 +01:00
Neil Alexander f91f91dcb1 Add unstable routing for now 2020-05-27 14:17:43 +01:00
Neil Alexander 74fb1daa52 Update sample config 2020-05-27 13:42:43 +01:00
Neil Alexander a90ea4c320 Groundwork for send-to-device messaging 2020-05-27 13:33:56 +01:00
Neil Alexander 57841fc35e
Read batches from incoming channels (#1067) 2020-05-27 12:16:53 +01:00
Neil Alexander 406b47267e
Return 500 when processing a transaction fails fatally (#1066) 2020-05-27 11:16:27 +01:00
Kegsay a7f12bce79
Convert remaining roomserver tables (#1065)
* Convert invites table

* Convert membership table

* Factor out remaining functions except for *Updater structs

* Remove 'implements' comments from long-forgotten interfaces. Move those comments to storage.Database
2020-05-27 11:03:47 +01:00
Neil Alexander 7d6461dd3c
Server key component (#1050)
* Server key API (works for monolith but not for polylith yet)

* Re-enable caching on server key API component

* Groundwork for HTTP APIs for server key API

* Hopefully implement HTTP for server key API

* Simplify public key request marshalling from map keys

* Update gomatrixserverlib

* go mod tidy

* Common -> internal

* remove keyring.go

* Update Docker Hub for server key API

* YAML is funny about indentation

* Wire in new server key API into hybrid monolith mode

* Create maps

* Route server key API endpoints on internal API mux

* Fix server key API URLs

* Add fetcher behaviour into server key API implementation

* Return error if we failed to fetch some keys

* Return results anyway

* Move things about a bit

* Remove unused code

* Fix comments, don't use federation sender URL in polylith mode

* Add server_key_api to sample config

* Review comments

* HTTP API to cache keys that have been requested

* Overwrite server_key_api listen in monolith hybrid mode
2020-05-27 10:19:24 +01:00
Kegan Dougal 267a4d1823 Update p2p docs 2020-05-27 09:39:26 +01:00
Kegsay c0c5d9452a Convert room_aliases previous_events state_block and state_snapshot tables (#1064)
* Convert state_snapshot and state_block tables

* Convert room_aliases and previous_events tables

* Add missing table
2020-05-27 09:36:09 +01:00
Kegsay 19aa44ecae
Convert transactions/rooms table to share more code (#1063)
* Convert rooms table

* Convert transactions table

* Convert rooms table and factor out lots of functions

* I think you'll be needing this..
2020-05-26 18:23:39 +01:00
Kegsay 803af87dc4
Convert events/event_json tables to share code (#1062)
* Convert event_json table

* Convert the events table
2020-05-26 16:45:28 +01:00
Kegsay 737c83e0ae
roomserver: start refactoring storage layer (#1061)
* Begin refactoring out roomserver/storage/*/storage.go

* Convert event_types table
2020-05-26 15:42:42 +01:00
Neil Alexander 6d50212f29
Miscellaneous fixes (#1060)
* Add missing routing for PerformDirectoryLookupRequest

* Tweak output

* Fix some bugs in devices

* Don't default to federated room joins in response to invite

* Update sytest-whitelist

* Update comments

* Return correct room ID from PerformJoin

* Fix appservice and EDU server API setup, update sytest-whitelist

* Update sytest-whitelist
2020-05-26 14:41:16 +01:00
Kegan Dougal 492af0f2ec Use Opaque in addition to Path to set naffka DB names 2020-05-25 15:29:49 +01:00