Commit graph

1100 commits

Author SHA1 Message Date
Matthew Hodgson e5899843ea fix SQL 2020-08-31 23:28:55 +03:00
Matthew Hodgson 7b38d4857f spell out how to runoutside of docker if you want speed 2020-08-31 23:28:46 +03:00
Matthew Hodgson 6c3a896910 cancel any peeks when we join a room 2020-08-31 18:58:10 +03:00
Matthew Hodgson f006b37bf0 add peeking to getResponseWithPDUsForCompleteSync 2020-08-31 18:16:07 +03:00
Matthew Hodgson d1e4d66126 make it launch 2020-08-31 17:35:23 +03:00
Matthew Hodgson c4e5f60d71 make it build 2020-08-31 16:12:09 +03:00
Matthew Hodgson d343b8fb2c blind stab at adding a peek section to /sync 2020-08-31 15:28:31 +03:00
Matthew Hodgson 9b79f9a883 add server_name param 2020-08-31 12:27:39 +03:00
Matthew Hodgson cfa0be544d merge master 2020-08-31 12:26:27 +03:00
Matthew Hodgson d7bdf71bef make PeekingDeviceSet private 2020-08-30 17:56:49 +03:00
Matthew Hodgson b9342d9ee2 a very very WIP first cut of peeking via MSC2753.
doesn't yet compile or work.
needs to actually add the peeking block into the sync response.
checking in now before it gets any bigger, and to gather any initial feedback on the vague shape of it.
2020-08-30 17:46:15 +03:00
Neil Alexander 737802fc06
Update Complement dockerfile to not wait for apt-get input 2020-08-28 15:19:47 +01:00
Neil Alexander c42c70597c
Add some error wrapping to sync API, use background context for sync (#1363)
* Add some error wrapping to sync API

* Don't use request context for BeginTx until mattn/go-sqlite3#764 is fixed
2020-08-28 13:28:40 +01:00
Neil Alexander fee1c22790
Fix #1361 (#1362) 2020-08-28 09:46:32 +01:00
Kegsay 9af2f5f1f2
Remove device DB from clientapi (#1352)
* Remove device DB from clientapi

* Remove device DB from startup configuration

It's all an impl detail now in user API
2020-08-27 18:53:40 +01:00
Neil Alexander c0f28845f8
Try to protect GetNextTransactionPDUs (#1350) 2020-08-27 15:27:12 +01:00
Neil Alexander 034e5d5a55
Speed up accepting invites from remote users if we're already in the room (#1349) 2020-08-27 11:47:13 +01:00
Neil Alexander 7466e6b718
Fix lock errors in federation sender (#1347)
* Fix lock errors in federation sender

* Additional fix to writers
2020-08-27 11:05:41 +01:00
Kegsay 421b6b2313
Add OpenTracing documentation (#1345)
* Add OpenTracing documentation

Closes #274

* Reshuffle

* Update setup.md
2020-08-27 10:24:56 +01:00
Neil Alexander 42183bef50
Allow join to accept server_name in query parameters (#1346) 2020-08-26 18:23:08 +01:00
Kegsay 3802efe301
Port of #945 (#1343)
- Make sure we always cleanup the temp directory on error.
- Complain about it having an error prone API shape.
2020-08-26 15:38:34 +01:00
Kegsay 29d6481842
Wait for 8h between device list updates for blacklisted servers (#1344) 2020-08-26 15:38:21 +01:00
Kegsay abd16ff4a0
Modify DeviceListUpdater to retry requests according to RetryAfter (#1342)
* Modify DeviceListUpdater to retry requests according to RetryAfter

* Reduce wait time for sytest test pollution
2020-08-26 12:03:09 +01:00
oliverpool 3205b9212d
[readability] use event.StateKeyEquals where relevant and minor for-loop refactoring (#1339)
* use event.StateKeyEquals where relevant

Signed-off-by: Olivier Charvin <git@olivier.pfad.fr>

* use userID
2020-08-25 21:04:35 +01:00
Kegsay 55498c8deb
Fix 'Invited user can reject invite over federation several times' (#1341) 2020-08-25 18:59:00 +01:00
Neil Alexander c60270eea3
Enforce history visibility etc for /rooms/{roomID}/state (#1340)
* Enforce history visibility etc for /rooms/{roomID}/state

* Deduplicate OnIncomingStateRequest and OnIncomingStateTypeRequest

* Revert "Deduplicate OnIncomingStateRequest and OnIncomingStateTypeRequest"

This reverts commit 335035d66e.
2020-08-25 18:43:56 +01:00
Neil Alexander 48850d9995
Update /versions (#1338) 2020-08-25 15:44:33 +01:00
Neil Alexander 819b926820
Make redactions permanent (#1337)
* Make redactions permanent

* Update storage.go
2020-08-25 15:44:19 +01:00
Neil Alexander 287700baee
Randomly generated media IDs (#1336)
* Filenames based on hashes but media IDs randomly generated and per-upload metadata

* Lint

* Comment generateMediaID
2020-08-25 15:08:37 +01:00
oliverpool a4db43e096
Don't overwrite global err before return (#1293)
Signed-off-by: Olivier Charvin <git@olivier.pfad.fr>
2020-08-25 13:11:52 +01:00
Neil Alexander c8b873abc8
Roomserver NID caches (#1335)
* Initial work on roomserver NID caches

* Give caches to roomserver storage

* Populate caches

* Fix bugs

* Fix WASM build

* Don't hit cache twice in RoomNIDExcludingStubs

* Store reverse room ID-room NID mapping, consult caches when assigning NIDs
2020-08-25 12:32:29 +01:00
Kegsay 05242096a1
Update README.md 2020-08-25 12:09:34 +01:00
Kegsay 733abe912b
Add spec compliant path for redaction (#1334)
Possibly fixes #1194
2020-08-25 10:39:30 +01:00
Neil Alexander 720ddce0a8
Use Writer in shared package (#1296) 2020-08-25 10:29:45 +01:00
Kegsay 3b14119aff
Update README/CONTRIBUTING/sytest for AWSY issues (#1333)
* Update README/CONTRIBUTING/sytest for AWSY issues

* More docs
2020-08-25 08:11:41 +01:00
oliverpool 466d795375
Fix scripts path in documentation (#1294)
Broken by 24d8df664c

Signed-off-by: Olivier Charvin <git@olivier.pfad.fr>
2020-08-24 18:13:43 +01:00
Neil Alexander 4b2db1dff5
Update Naffka (#1295)
* Update Naffka

* Fix Naffka setup

* Update Naffka

* Update Naffka
2020-08-24 13:49:20 +01:00
Neil Alexander 10461b8870
Fix embedding Element Web into demo-libp2p and demo-yggdrasil 2020-08-21 11:01:30 +01:00
Neil Alexander 9d53351dc2
Component-wide TransactionWriters (#1290)
* Offset updates take place using TransactionWriter

* Refactor TransactionWriter in current state server

* Refactor TransactionWriter in federation sender

* Refactor TransactionWriter in key server

* Refactor TransactionWriter in media API

* Refactor TransactionWriter in server key API

* Refactor TransactionWriter in sync API

* Refactor TransactionWriter in user API

* Fix deadlocking Sync API tests

* Un-deadlock device database

* Fix appservice API

* Rename TransactionWriters to Writers

* Move writers up a layer in sync API

* Document sqlutil.Writer interface

* Add note to Writer documentation
2020-08-21 10:42:08 +01:00
Neil Alexander 5aaf32bbed
Select distinct on room memberships in sync API (#1292) 2020-08-21 09:57:52 +01:00
Kegsay ec95d331a5
Add support for running Complement on current working directories (#1291)
This will be used in the future by Buildkite to run on CI.
2020-08-20 18:35:04 +01:00
Kegsay 6d6bb75137
Add FederationClient interface to federationsender (#1284)
* Add FederationClient interface to federationsender

- Use a shim struct in HTTP mode to keep the same API as `FederationClient`.
- Use `federationsender` instead of `FederationClient` in `keyserver`.

* Pointers not values

* Review comments

* Fix unit tests

* Rejig backoff

* Unbreak test

* Remove debug logs

* Review comments and linting
2020-08-20 17:03:07 +01:00
Neil Alexander 068a3d3c9f
Roomserver per-room input parallelisation (Postgres) (#1289)
* Per-room input mutex

* GetMembership should use transaction when assigning state key NID

* Actually use writer transactions rather than ignoring them

* Limit per-room mutexes to Postgres

* Flip the check in InputRoomEvents
2020-08-20 16:24:33 +01:00
Neil Alexander 0fea056db4
Change backoff behaviour so that Failure returns planned end time (#1288) 2020-08-20 14:58:53 +01:00
Neil Alexander 5ad47d3b3d
Fix more roomserver transactions/locks (#1287)
* Fix transaction to InsertTransaction

* Remove unnecessary txn, add txns around setting up updaters
2020-08-20 09:24:52 +01:00
anandv96 f5edfb9659
#903: Client API: mutex on (user_id, room_id) (#1286)
* Client API: mutex on (user_id, room_id)

* Client API: mutex on (user_id, room_id)

Changed variable name used for the mutexes map
Changed the place where the mutex is locked
Changed unlock to a defered call instead of manually calling it
2020-08-20 08:27:43 +01:00
Neil Alexander b24747b305
Transaction writer changes, move roomserver writers (#1285)
* Updated TransactionWriters, moved locks in roomserver, various other tweaks

* Fix redaction deadlocks

* Fix lint issue

* Rename SQLiteTransactionWriter to ExclusiveTransactionWriter

* Fix us not sending transactions through in latest events updater
2020-08-19 15:38:27 +01:00
Neil Alexander 775b04d776
Roomserver updater changes (#1283)
* Take input transaction when setting up updaters

* Fix nil pointer exceptions

* Rename room recent events updater to latest events updater

* Contd rename room recent events updater to latest events updater

* Remove unnecessary interfaces for latest events and membership updaters
2020-08-19 13:24:54 +01:00
Neil Alexander 3d58417555
Roomserver database-wide TransactionWriters (#1282)
* Database-wide TransactionWriter

* Fix deadlocking Sync API tests

* Undo non-roomserver changes for now
2020-08-19 10:57:29 +01:00
Kegsay e571e196ce
Summarise key change logs (#1278) 2020-08-18 11:14:37 +01:00