Commit graph

840 commits

Author SHA1 Message Date
Neil Alexander 544f3b0a61 Fix schema 2020-05-28 18:40:26 +01:00
Neil Alexander e4eba873f5 Refactor a bit 2020-05-28 18:38:23 +01:00
Neil Alexander f4376c4582 Comments 2020-05-28 17:53:40 +01:00
Neil Alexander 5389a62952 Add send-to-device test, hopefully fix bugs 2020-05-28 17:49:56 +01:00
Neil Alexander 1e3ad88172 Fix a couple of transaction things, fix client API 2020-05-28 17:25:28 +01:00
Neil Alexander bf5df4f438 Logging 2020-05-28 16:45:54 +01:00
Neil Alexander 587108fe31 Fix sync 2020-05-28 15:52:41 +01:00
Neil Alexander 581e5929e1 pq.Array 2020-05-28 15:43:01 +01:00
Neil Alexander e3267cf12a Undo changes to test 2020-05-28 15:39:29 +01:00
Neil Alexander 4d6347b21a Handle incoming send-to-device messages, count them with EDU stream pos 2020-05-28 15:35:49 +01:00
Neil Alexander d3bf9cb31b Add some logic for send-to-device messages, add them into sync stream 2020-05-28 15:11:44 +01:00
Neil Alexander 8b7e81b423 Create send-to-device database tables, other tweaks 2020-05-28 14:34:17 +01:00
Neil Alexander 9694907ccc fix indentation in dendrite-config.yaml 2020-05-28 13:46:21 +01:00
Neil Alexander d4d4915194 Merge branch 'master' into neilalexander/sendtodevice 2020-05-28 13:45:34 +01:00
Neil Alexander 5307c499fe
Update gomatrixserverlib (#1071) 2020-05-28 13:44:16 +01:00
Neil Alexander 1b7ad3e4e0 Merge branch 'master' into neilalexander/sendtodevice 2020-05-28 12:48:02 +01:00
Neil Alexander f123d7d65b
Hopefully fix get_missing_events (#1070) 2020-05-28 12:44:34 +01:00
Kegan Dougal 2411007c4b Parse URIs correctly 2020-05-28 12:28:21 +01:00
Kegsay a6f995eb45
Merge Updater structs (#1069)
* Move Updater structs to shared and use it for postgres

* Add constructors for NewXXXUpdater and a useTxns flag

In sqlite, we set useTxns=false and comment why.

* Handle nil txn

* Handle nil in transaction

* Missed one

* Close the txn at the right time

* Don't close the transaction as we reuse it between calls
2020-05-28 11:15:21 +01:00
Neil Alexander 23a16d1a57 Merge branch 'master' into neilalexander/sendtodevice 2020-05-28 10:13:08 +01:00
Neil Alexander 02fe38e1f7
Per-user-per-device sync streams (#1068)
* Per-user-per-device sync streams

* Tweaks

* Tweaks

* Pass full device into CompleteSync

* Set user IDs and device IDs properly in tests

* Add new test, fix TestNewEventAndWasPreviouslyJoinedToRoom

* nolint a function that is not used yet

* Add test for waking up single device

* Hopefully unstick test

* Try to ensure that TestCorrectStreamWakeup doesn't block forever

* Update tests
2020-05-28 10:05:04 +01:00
Neil Alexander ba6e55ecf5 Start the send-to-device consumer 2020-05-27 15:04:06 +01:00
Neil Alexander 062325b6c6 Send to device consumer in sync API 2020-05-27 14:37:13 +01:00
Neil Alexander f91f91dcb1 Add unstable routing for now 2020-05-27 14:17:43 +01:00
Neil Alexander 74fb1daa52 Update sample config 2020-05-27 13:42:43 +01:00
Neil Alexander a90ea4c320 Groundwork for send-to-device messaging 2020-05-27 13:33:56 +01:00
Neil Alexander 57841fc35e
Read batches from incoming channels (#1067) 2020-05-27 12:16:53 +01:00
Neil Alexander 406b47267e
Return 500 when processing a transaction fails fatally (#1066) 2020-05-27 11:16:27 +01:00
Kegsay a7f12bce79
Convert remaining roomserver tables (#1065)
* Convert invites table

* Convert membership table

* Factor out remaining functions except for *Updater structs

* Remove 'implements' comments from long-forgotten interfaces. Move those comments to storage.Database
2020-05-27 11:03:47 +01:00
Neil Alexander 7d6461dd3c
Server key component (#1050)
* Server key API (works for monolith but not for polylith yet)

* Re-enable caching on server key API component

* Groundwork for HTTP APIs for server key API

* Hopefully implement HTTP for server key API

* Simplify public key request marshalling from map keys

* Update gomatrixserverlib

* go mod tidy

* Common -> internal

* remove keyring.go

* Update Docker Hub for server key API

* YAML is funny about indentation

* Wire in new server key API into hybrid monolith mode

* Create maps

* Route server key API endpoints on internal API mux

* Fix server key API URLs

* Add fetcher behaviour into server key API implementation

* Return error if we failed to fetch some keys

* Return results anyway

* Move things about a bit

* Remove unused code

* Fix comments, don't use federation sender URL in polylith mode

* Add server_key_api to sample config

* Review comments

* HTTP API to cache keys that have been requested

* Overwrite server_key_api listen in monolith hybrid mode
2020-05-27 10:19:24 +01:00
Kegan Dougal 267a4d1823 Update p2p docs 2020-05-27 09:39:26 +01:00
Kegsay c0c5d9452a Convert room_aliases previous_events state_block and state_snapshot tables (#1064)
* Convert state_snapshot and state_block tables

* Convert room_aliases and previous_events tables

* Add missing table
2020-05-27 09:36:09 +01:00
Kegsay 19aa44ecae
Convert transactions/rooms table to share more code (#1063)
* Convert rooms table

* Convert transactions table

* Convert rooms table and factor out lots of functions

* I think you'll be needing this..
2020-05-26 18:23:39 +01:00
Kegsay 803af87dc4
Convert events/event_json tables to share code (#1062)
* Convert event_json table

* Convert the events table
2020-05-26 16:45:28 +01:00
Kegsay 737c83e0ae
roomserver: start refactoring storage layer (#1061)
* Begin refactoring out roomserver/storage/*/storage.go

* Convert event_types table
2020-05-26 15:42:42 +01:00
Neil Alexander 6d50212f29
Miscellaneous fixes (#1060)
* Add missing routing for PerformDirectoryLookupRequest

* Tweak output

* Fix some bugs in devices

* Don't default to federated room joins in response to invite

* Update sytest-whitelist

* Update comments

* Return correct room ID from PerformJoin

* Fix appservice and EDU server API setup, update sytest-whitelist

* Update sytest-whitelist
2020-05-26 14:41:16 +01:00
Kegan Dougal 492af0f2ec Use Opaque in addition to Path to set naffka DB names 2020-05-25 15:29:49 +01:00
Neil Alexander 3c3e014901 Define path prefixes in a package that doesn't create import cycles 2020-05-22 14:18:41 +01:00
Neil Alexander 06d5f1e6dc Fix API paths 2020-05-22 14:14:39 +01:00
Neil Alexander 0978630b55 Fix monolith room server-federation sender connection 2020-05-22 14:11:06 +01:00
Neil Alexander 3d06fe91f2 Fix internal HTTP API calls 2020-05-22 13:54:04 +01:00
Kegsay 3daa2327ed
dendritejs tweaks for persisting sqlite DBs (#1058)
* Use uri.path so we don't have file: in the filename

* New go-sqlite-js version
2020-05-22 12:28:48 +01:00
Neil Alexander fbdcfdd256
Use HTTP APIs when -api specified (#1057) 2020-05-22 12:28:36 +01:00
Neil Alexander fe82e1f725
Separate muxes for public and internal APIs (#1056)
* Separate muxes for public and internal APIs

* Update client-api-proxy and federation-api-proxy so they don't add /api to the path

* Tidy up

* Consistent HTTP setup

* Set up prefixes properly
2020-05-22 11:43:17 +01:00
Neil Alexander f223da2f35 Fix URL in compose file 2020-05-21 16:22:25 +01:00
Kegsay 24d8df664c
Fix #897 and shuffle directory around (#1054)
* Fix #897 and shuffle directory around

* Update find-lint

* goimports

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-05-21 14:40:13 +01:00
Neil Alexander 3fdb045116
Docker Hub (#1053)
* Initial Docker Hub test

* Change upstream from for now

* Working polylith setup and build tools

* Some readme and other bits

* Remove the readme for now

* Add readme, rename a couple of things

* COPY instead of ADD

* Simplify component builds

* Fix the docker-compose things

* Build the base image again

* Update readme, add .dockerignore

* Indentation

* Whitespace

* Update images-push.sh

* Update images-push.sh
2020-05-21 13:02:28 +01:00
Kegan Dougal 91902df95d Add tests due to https://github.com/matrix-org/sytest/pull/876 2020-05-21 11:51:15 +01:00
Kegan Dougal 7d91ef0616 This now passes on sytest/develop 2020-05-20 18:31:02 +01:00
Neil Alexander f2c07437fe
Use memberships to determine whether to reset latest events/state on room join (#1047)
* Track local/remote memberships, re-scope some input stuff

* Check if we're in the room already before resetting latest events/state

* Fix postgres, fix lint

* Review comments
2020-05-20 18:03:06 +01:00