Commit graph

115 commits

Author SHA1 Message Date
Matthew Hodgson 9561843772 stop the peek table growing by reusing rows correctly 2020-09-10 12:28:36 +01:00
Matthew Hodgson b1d1ccdbc8 allow duplicate stream ids in the peeks table 2020-09-10 09:39:06 +01:00
Matthew Hodgson 8cec7e029d
fix lots of places where we didn't rollback txns upon go errs (#1417)
Co-authored-by: Kegan Dougal <kegan@matrix.org>
2020-09-09 17:20:45 +01:00
Kegan Dougal 34b89619f2 Fix sqlite DeletePeeks API to match postgres; fix bug which incorrectly altered sp when nothing is deleted 2020-09-09 16:53:21 +01:00
Kegan Dougal 15349d8287 Return errors from SQL statements to handle rollbacks correctly 2020-09-09 16:12:46 +01:00
Matthew Hodgson 052351e0bf go fmt 2020-09-09 02:21:28 +01:00
Matthew Hodgson b45436aab0 handle exclusive writer txn for cleanliness 2020-09-09 02:15:58 +01:00
Matthew Hodgson de53608f98 fix nightmare bug where sqlite doesn't let you use out of order sub strings 2020-09-09 02:14:06 +01:00
Neil Alexander 6fe281a13d
Don't return sql.ErrNoRows on DeletePeeks since that is not an error condition 2020-09-08 14:06:08 +01:00
Matthew Hodgson b96a31db51 track transitions more clearly 2020-09-08 11:57:49 +01:00
Matthew Hodgson 9f2bc6231f (broken) rewrite to use SelectPeeksInRange rather than MarkPeeksAsOld for idempotency 2020-09-08 11:05:54 +01:00
Matthew Hodgson 7f41f39684 remove erroneous commentary 2020-09-08 09:25:05 +01:00
Matthew Hodgson 034ff3208c move peek-cancelling to consumer 2020-09-08 01:42:27 +01:00
Matthew Hodgson 55c7f2c892 add (broken) postgres; advance streampos whenever sync output changes 2020-09-08 00:56:50 +01:00
Neil Alexander 2ccd4fa069
Clear a whole bunch of lint issues 2020-09-04 16:17:02 +01:00
Neil Alexander ff65f0ec7b
Revert "Merge branch 'kegan/HACK-goid-sqlite-db-is-locked' into matthew/peeking"
This reverts commit 3cebd8dbfb, reversing
changes made to ed4b3a58a7.
2020-09-04 16:14:40 +01:00
Neil Alexander 56d772fafc
Cancel peeks on join again 2020-09-04 14:20:00 +01:00
Neil Alexander 2b8f0b8f59
Merge branch 'master' into matthew/peeking 2020-09-04 14:15:51 +01:00
Neil Alexander c63abc8660
Sync bug fixes (#1394)
* Sync bug fixes

* Remove logging
2020-09-04 11:20:47 +01:00
Matthew Hodgson b6cc4417cc re-add txn to SelectPeeks 2020-09-02 10:07:49 +01:00
Matthew Hodgson 75b91ac9e5 strip out empty roomd deltas 2020-09-01 21:59:35 +01:00
Matthew Hodgson 6424117852 use exclusive writer, and MarkPeeksAsOld more efficiently 2020-09-01 21:31:04 +01:00
Matthew Hodgson 5d7f688fa7 wrap peek storage in goid hack 2020-09-01 21:30:03 +01:00
Kegsay 82a9617659
Put redactions/filters in the writer goroutine (#1378)
* Put redactions in the writer goroutine

* Update filters on writer goroutine
2020-09-01 20:35:38 +01:00
Matthew Hodgson 3cebd8dbfb Merge branch 'kegan/HACK-goid-sqlite-db-is-locked' into matthew/peeking 2020-09-01 21:37:22 +03:00
Matthew Hodgson ed4b3a58a7 Merge branch 'kegan/redact-txn' into matthew/peeking 2020-09-01 21:34:01 +03:00
Kegan Dougal 6410b702ce Update filters on writer goroutine 2020-09-01 19:27:21 +01:00
Kegan Dougal fcdb90c91b Put redactions in the writer goroutine 2020-09-01 19:21:33 +01:00
Kegan Dougal 7bf2a27319 Track partition offsets and only log unsafe for non-selects 2020-09-01 19:17:01 +01:00
Kegan Dougal bfecc8e0e9 HACK: Track goroutine IDs to determine when we write by the wrong thread
To use: set `DENDRITE_TRACE_SQL=1` then grep for `unsafe`
2020-09-01 19:10:57 +01:00
Matthew Hodgson d0d5f70105 Merge branch 'master' into matthew/peeking 2020-09-01 19:11:51 +03:00
Neil Alexander a9f4d83d30
Fix duplicate writers (#1376)
* Fix writers

* Don't use writers in both shared and sqlite3
2020-09-01 16:58:21 +01:00
Matthew Hodgson 28219c66f5 Merge branch 'master' into matthew/peeking 2020-09-01 18:03:06 +03:00
Neil Alexander 0ab5bccd11
Storage tweaks (#1373)
* Sync API tweaks

* User API tweaks
2020-09-01 11:28:35 +01:00
Neil Alexander b0d2b39739
Remove unused SyncStreamPosition 2020-09-01 10:26:02 +01:00
Matthew Hodgson 0bb2c2c418 remove unnecessary txn for SelectPeeks 2020-09-01 00:24:23 +03:00
Matthew Hodgson e5899843ea fix SQL 2020-08-31 23:28:55 +03:00
Matthew Hodgson 6c3a896910 cancel any peeks when we join a room 2020-08-31 18:58:10 +03:00
Matthew Hodgson f006b37bf0 add peeking to getResponseWithPDUsForCompleteSync 2020-08-31 18:16:07 +03:00
Matthew Hodgson d1e4d66126 make it launch 2020-08-31 17:35:23 +03:00
Matthew Hodgson c4e5f60d71 make it build 2020-08-31 16:12:09 +03:00
Matthew Hodgson d343b8fb2c blind stab at adding a peek section to /sync 2020-08-31 15:28:31 +03:00
Matthew Hodgson cfa0be544d merge master 2020-08-31 12:26:27 +03:00
Matthew Hodgson b9342d9ee2 a very very WIP first cut of peeking via MSC2753.
doesn't yet compile or work.
needs to actually add the peeking block into the sync response.
checking in now before it gets any bigger, and to gather any initial feedback on the vague shape of it.
2020-08-30 17:46:15 +03:00
Neil Alexander c42c70597c
Add some error wrapping to sync API, use background context for sync (#1363)
* Add some error wrapping to sync API

* Don't use request context for BeginTx until mattn/go-sqlite3#764 is fixed
2020-08-28 13:28:40 +01:00
oliverpool 3205b9212d
[readability] use event.StateKeyEquals where relevant and minor for-loop refactoring (#1339)
* use event.StateKeyEquals where relevant

Signed-off-by: Olivier Charvin <git@olivier.pfad.fr>

* use userID
2020-08-25 21:04:35 +01:00
Kegsay 55498c8deb
Fix 'Invited user can reject invite over federation several times' (#1341) 2020-08-25 18:59:00 +01:00
oliverpool a4db43e096
Don't overwrite global err before return (#1293)
Signed-off-by: Olivier Charvin <git@olivier.pfad.fr>
2020-08-25 13:11:52 +01:00
Neil Alexander 9d53351dc2
Component-wide TransactionWriters (#1290)
* Offset updates take place using TransactionWriter

* Refactor TransactionWriter in current state server

* Refactor TransactionWriter in federation sender

* Refactor TransactionWriter in key server

* Refactor TransactionWriter in media API

* Refactor TransactionWriter in server key API

* Refactor TransactionWriter in sync API

* Refactor TransactionWriter in user API

* Fix deadlocking Sync API tests

* Un-deadlock device database

* Fix appservice API

* Rename TransactionWriters to Writers

* Move writers up a layer in sync API

* Document sqlutil.Writer interface

* Add note to Writer documentation
2020-08-21 10:42:08 +01:00
Neil Alexander 5aaf32bbed
Select distinct on room memberships in sync API (#1292) 2020-08-21 09:57:52 +01:00