Commit graph

27 commits

Author SHA1 Message Date
Neil Alexander 9f834726cd Constructor for TransactionWriter 2020-06-01 17:02:48 +01:00
Neil Alexander fa74ded4d6 Review comments 2020-06-01 16:51:30 +01:00
Neil Alexander 6158a8dcc5 Return errors from Do, fix dendritejs 2020-06-01 14:46:32 +01:00
Neil Alexander e323bba3fd Comments, copyrights 2020-06-01 14:14:48 +01:00
Neil Alexander 2e40e92ed1 Handle tokens properly, return immediately if waiting send-to-device messages 2020-06-01 11:19:43 +01:00
Neil Alexander 43a83d375e Try to line up sync tokens again 2020-05-29 16:22:52 +01:00
Neil Alexander 2ef53ff726 nolint on getResponseWithPDUsForCompleteSync 2020-05-29 15:51:36 +01:00
Neil Alexander ded130f548 Break out the retrieval from the update/delete behaviour 2020-05-29 15:41:04 +01:00
Neil Alexander fc1303bbbe Jiggle about sync a bit 2020-05-29 13:43:37 +01:00
Neil Alexander cc456240fa Update sync position 2020-05-29 11:48:42 +01:00
Neil Alexander 5487edf1d6 Try to avoid database locks 2020-05-29 11:38:19 +01:00
Neil Alexander 005c4c0168 Fix storing and retrieving of send-to-device messages 2020-05-29 10:07:57 +01:00
Neil Alexander 4fda6c65e6 Debug logging 2020-05-29 09:39:17 +01:00
Neil Alexander e4eba873f5 Refactor a bit 2020-05-28 18:38:23 +01:00
Neil Alexander 5389a62952 Add send-to-device test, hopefully fix bugs 2020-05-28 17:49:56 +01:00
Neil Alexander 1e3ad88172 Fix a couple of transaction things, fix client API 2020-05-28 17:25:28 +01:00
Neil Alexander 4d6347b21a Handle incoming send-to-device messages, count them with EDU stream pos 2020-05-28 15:35:49 +01:00
Neil Alexander d3bf9cb31b Add some logic for send-to-device messages, add them into sync stream 2020-05-28 15:11:44 +01:00
Neil Alexander 8b7e81b423 Create send-to-device database tables, other tweaks 2020-05-28 14:34:17 +01:00
Neil Alexander 02fe38e1f7
Per-user-per-device sync streams (#1068)
* Per-user-per-device sync streams

* Tweaks

* Tweaks

* Pass full device into CompleteSync

* Set user IDs and device IDs properly in tests

* Add new test, fix TestNewEventAndWasPreviouslyJoinedToRoom

* nolint a function that is not used yet

* Add test for waking up single device

* Hopefully unstick test

* Try to ensure that TestCorrectStreamWakeup doesn't block forever

* Update tests
2020-05-28 10:05:04 +01:00
Kegsay 24d8df664c
Fix #897 and shuffle directory around (#1054)
* Fix #897 and shuffle directory around

* Update find-lint

* goimports

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-05-21 14:40:13 +01:00
Kegsay 1414922026
sytest: Make 'Inbound federation can backfill events' pass (#1051)
* sytest: Make 'Inbound federation can backfill events' pass

This breaks 'Outbound federation can backfill events' because now
we are returning the right number of events, which the previous
test was relying on.

Previously, /messages was backfilling the membership event, causing
the test to pass. Now we are no longer backfilling the membership
event due to the change in this commit, causing the test to fail.

The test should instead be returning the membership event locally
from synacpis database, but it doesn't do it fast enough, resulting
in a no-op /sync response with a next_batch=s0_0 which will never
pick up the local membership event when it rolls in. The test
does attempt to retry, but doesn't take the new next_batch=s1_0
resulting in it missing from the /messages response.

* Linting
2020-05-20 16:04:31 +01:00
Kegsay 2b5052eccf
Add Range (#1037)
* Add Range

* Use Range
2020-05-15 09:41:12 +01:00
Kegsay 7ca230e931
Cleanup syncapi topology logic (#1035)
* Cleanup syncapi topology logic

* Variable renaming

* comments
2020-05-14 17:30:16 +01:00
Kegsay 1b34130a5b
Finish merging syncserver.go (#1033)
* Refactor all postgres tables; start work on sqlite

* wip sqlite merges; database is locked errors to investigate and failing tests

* Revert "wip sqlite merges; database is locked errors to investigate and failing tests"

This reverts commit 26cbfc5b75.

* convert current room state table

* port over sqlite topology table

* remove a few functions

* remove more functions

* Share more code

* factor out completesync and a bit more

* Remove remaining code
2020-05-14 16:11:37 +01:00
Kegsay 9ed68a3125
Factor out account data and events table (#1031)
* Factor out account data

* Factor out events table and EDU cache

* linting

* fix npe
2020-05-14 09:53:55 +01:00
Kegsay a25d477cdb
Initial syncapi storage refactor to share pq/sqlite code (#1030)
* Initial syncapi storage refactor to share pq/sqlite code

This goes down a different route than https://github.com/matrix-org/dendrite/pull/985
which tried to even reduce the boilerplate of `ExecContext` etc. The previous pattern
fails badly when there are subtle differences in parameters and hence the shared
boilerplate to read from `QueryContext` breaks. Rather than attacking it at that level,
the main place where we want to reuse code is for the `syncserver.go` itself - the
database implementation which has lots of complex logic. So instead, this commit:
 - Makes `invites_table.go` an interface.
 - Makes `SyncServerDatasource` use that interface
 - This means some functions are now identical for pq/sqlite, so factor them out
   to a temporary `shared.Database` struct which will grow until it replaces all of
   `SyncServerDatasource`.

* Missing files
2020-05-13 17:28:42 +01:00