Commit graph

1194 commits

Author SHA1 Message Date
Matthew Hodgson 3202c7e76f go fmt 2020-09-23 00:08:23 +01:00
Matthew Hodgson 20e2cb4b7e track latestevent on /peek 2020-09-22 23:42:19 +01:00
Matthew Hodgson 41b9b663a5 Merge branch 'master' into matthew/peeking-over-fed 2020-09-19 17:28:12 +01:00
Kegsay 18231f25b4
Implement rejected events (#1426)
* WIP Event rejection

* Still send back errors for rejected events

Instead, discard them at the federationapi /send layer rather than
re-implementing checks at the clientapi/PerformJoin layer.

* Implement rejected events

Critically, rejected events CAN cause state resolution to happen
as it can merge forks in the DAG. This is fine, _provided_ we
do not add the rejected event when performing state resolution,
which is what this PR does. It also fixes the error handling
when NotAllowed happens, as we were checking too early and needlessly
handling NotAllowed in more than one place.

* Update test to match reality

* Modify InputRoomEvents to no longer return an error

Errors do not serialise across HTTP boundaries in polylith mode,
so instead set fields on the InputRoomEventsResponse. Add `Err()`
function to make the API shape basically the same.

* Remove redundant returns; linting

* Update blacklist
2020-09-16 13:00:52 +01:00
Kegan Dougal ba6c7c4a5c Disable prometheus to unbreak tests 2020-09-15 16:15:34 +01:00
Neil Alexander 965f068d1a
Handle state with input event as new events (#1415)
* SendEventWithState events as new

* Use cumulative state IDs for final event

* Error wrapping in calculateAndSetState

* Handle overwriting same event type and state key

* Hacky way to spot historical events

* Don't exclude from sync

* Don't generate output events when rewriting forward extremities

* Update output event check

* Historical output events

* Define output room event type

* Notify key changes on state

* Don't send our membership event twice

* Deduplicate state entries

* Tweaks

* Remove unnecessary nolint

* Fix current state upsert in sync API

* Send auth events as outliers, state events as rewrite

* Sync API don't consume state events

* Process events actually

* Improve outlier check

* Fix local room check

* Remove extra room check, it seems to break the whole damn world

* Fix federated join check

* Fix nil pointer exception

* Better comments on DeduplicateStateEntries

* Reflow forced federated joins

* Don't force federated join for possibly even local invites

* Comment SendEventWithState better

* Rewrite room state in sync API storage

* Add TODO

* Clean up all room data when receiving create event

* Don't generate output events for rewrites, but instead notify that state is rewritten on the final new event

* Rename to PurgeRoom

* Exclude backfilled messages from /sync

* Split out rewriting state from updating state from state res

Co-authored-by: Kegan Dougal <kegan@matrix.org>
2020-09-15 11:17:46 +01:00
Kegsay 8dc9506210
Don't use more than 999 variables in SQLite querys. (#1425)
* Don't use more than 999 variables in SQLite querys.

Solve this problem in a more general and reusable way.
Also fix #1369
Add some unit tests.

Signed-off-by: Henrik Sölver <henrik.solver@gmail.com>

* Don't rely on testify for basic assertions

* Readability improvements and linting

Co-authored-by: Henrik Sölver <henrik.solver@gmail.com>
2020-09-14 16:39:38 +01:00
Matthew Hodgson 32f898d668 fix api.SendEventWithState with no event 2020-09-13 03:32:01 +01:00
Matthew Hodgson a160c074e8 setup inbound peeks db correctly 2020-09-13 03:31:11 +01:00
Matthew Hodgson 3caae7913b rename FSAPI's PerformPeek as PerformOutboundPeek 2020-09-13 00:14:24 +01:00
Matthew Hodgson 8f203febc1 rename ambiguous RemotePeek as InboundPeek 2020-09-13 00:04:02 +01:00
Matthew Hodgson 71732f2c28 add inbound peeks table and hook it up 2020-09-12 23:54:46 +01:00
Matthew Hodgson 0dc422cdd8 flesh out federation doc 2020-09-12 23:53:46 +01:00
Matthew Hodgson 36e32f14e6 add perform_handle_remote_peek.go 2020-09-12 23:01:43 +01:00
Matthew Hodgson 59e2be7113 rename remote_peeks_table as outbound_peeks_table 2020-09-12 23:01:06 +01:00
Matthew Hodgson 4e96e62923 rename RemotePeeks as OutboundPeeks 2020-09-12 22:59:25 +01:00
Matthew Hodgson 0ae0d11446 reimplement SS /peek to prod the RS to tell the FS about the peek 2020-09-12 22:45:00 +01:00
Matthew Hodgson 803647be56 handle SS /peek 2020-09-12 01:24:28 +01:00
Matthew Hodgson c6a2604edc support returning current state from QueryStateAndAuthChain 2020-09-12 01:23:57 +01:00
Matthew Hodgson fff18454b1 todo for authenticating state returned by /peek 2020-09-12 01:23:32 +01:00
Matthew Hodgson f8bb4487df fix sql thinko 2020-09-12 00:24:22 +01:00
Matthew Hodgson 410ac724a6 reinstate federated peeks in the roomserver (doh) 2020-09-12 00:23:57 +01:00
Matthew Hodgson df29509e7e fix build & lint 2020-09-11 22:39:50 +01:00
Matthew Hodgson a5c0521c3f fix build & lint 2020-09-11 22:31:56 +01:00
Matthew Hodgson 65e59a1af9 support SendEventWithState with optional event 2020-09-11 22:31:32 +01:00
Matthew Hodgson 4ef6a3c759 clean up bad merge 2020-09-11 21:06:51 +01:00
Matthew Hodgson baee97bff7 (almost) make it build 2020-09-11 21:04:33 +01:00
Matthew Hodgson f236e8290d Merge branch 'master' into matthew/peeking-over-fed 2020-09-11 19:44:40 +01:00
Matthew Hodgson 98cf8986a4 Revert "Merge branch 'kegan/HACK-goid-sqlite-db-is-locked' into matthew/peeking"
This reverts commit 3cebd8dbfb, reversing
changes made to ed4b3a58a7.
2020-09-11 19:44:09 +01:00
Kegsay 913020e4b7
Add hardware requirements; fixes #324 (#1420) 2020-09-10 17:37:31 +01:00
Kegsay 95d7e2336d
Add support for database migrations (#1416)
* Add support for database migrations

Closes #1246

This PR does NOT add any migrations as an example. I have
manually tested that the library works with SQL and Go based
upgrades correctly. Documentation should be sufficient for
devs to add migrations.

* Clarifications

* Linting
2020-09-10 15:10:32 +01:00
Matthew Hodgson 39507bacc3
Peeking via MSC2753 (#1370)
Initial implementation of MSC2753, as tested by https://github.com/matrix-org/sytest/pull/944.
Doesn't yet handle unpeeks, peeked EDUs, or history viz changing during a peek - these will follow.
https://github.com/matrix-org/dendrite/pull/1370 has full details.
2020-09-10 14:39:18 +01:00
Neil Alexander 35564dd73c
Process membership updates in writers (#1414) 2020-09-08 17:48:07 +01:00
Neil Alexander a0f2a4510f
Exclude deleted keys from selectBatchDeviceKeysSQL (#1412) 2020-09-08 17:47:54 +01:00
Kegsay b4bd0cc0f5
Track goids when running with tracing enabled (#1413)
* Track goids when running with tracing enabled

* Linting
2020-09-08 17:30:05 +01:00
Neil Alexander 668a722ee0
Backoff for 401s (#1410)
* Backoff for 401s

* Human-readable retry_after in logs
2020-09-08 13:41:08 +01:00
Neil Alexander 726ad6ce2e
Backoff ignore invalid signatures (#1408) 2020-09-08 10:28:13 +01:00
Neil Alexander 1602df8752
Ignore state events with invalid signatures when joining rooms (#1407)
* Use state from RespSendJoin post-check

* Don't create input events for invalid events

* Let's try this again

* Update gomatrixserverlib

* Update gomatrixserverlib to matrix-org/gomatrixserverlib@38f437f
2020-09-07 16:54:51 +01:00
Kegsay c992f4f1f4
Remove current state server (#1405)
* Remove current state server

Closes #1365 #1272 #1357

* Remove current state server from scripts/docs
2020-09-07 14:47:59 +01:00
Neil Alexander 8589f8373e
Update gomatrixserverlib to matrix-org/gomatrixserverlib@66753e2 2020-09-07 14:39:04 +01:00
Kegsay 7913759921
Remove QueryBulkStateContent from current state server (#1404)
* Remove QueryBulkStateContent from current state server

Expected fail due to db impl not existing

* Implement query bulk state content

* Fix up rejecting invites over federation

* Fix bulk content marshalling
2020-09-07 12:38:09 +01:00
Neil Alexander 895ead8048
Use background context when processing event with missing state (#1403)
* Use background context when processing event with missing state

* Five minute timeout

* Remove context from txnreq, thread through instead

* Fix unit tests
2020-09-07 12:32:40 +01:00
Neil Alexander b9caccbce8
Update GetStateEvent behaviour (#1399) 2020-09-04 19:40:21 +01:00
Kegsay 088294ee65
Remove QueryRoomsForUser from current state server (#1398) 2020-09-04 15:58:30 +01:00
Neil Alexander 5076925c18
Password changes (#1397)
* User API support for password changes

* Password changes in client API

* Update sytest-whitelist

* Remove debug logging

* Default logout_devices to true

* Fix deleting devices by local part
2020-09-04 15:16:13 +01:00
Kegsay ca8dcf46b7
Remove QuerySharedUsers from current state server (#1396)
* Remove QuerySharedUsers from current state server

* Bugfixes
2020-09-04 14:25:01 +01:00
Kegsay 81688d6bde
Remove QueryCurrentState from current-state-server (#1395) 2020-09-04 12:30:56 +01:00
Kegsay 67cc8619d3
Remove QueryKnownUsers from current state server (#1393)
* Remove QueryKnownUsers from current state server

* Fix HTTP mode
2020-09-04 11:46:01 +01:00
Neil Alexander c63abc8660
Sync bug fixes (#1394)
* Sync bug fixes

* Remove logging
2020-09-04 11:20:47 +01:00
Kegsay 2570418f42
Remove ServerACLs from the current state server (#1390)
* Remove ServerACLs from the current state server

Functionality moved to roomserver

* Nothing to see here, move along
2020-09-04 10:40:58 +01:00