Commit graph

771 commits

Author SHA1 Message Date
Neil Alexander 63775d5b1b We should tell other resident servers about the invite if we know about the room 2020-05-06 16:03:59 +01:00
Neil Alexander c744cf052f Only loopback invite event if we know the room 2020-05-06 15:59:31 +01:00
Neil Alexander 0100f65f57 Linter 2020-05-06 15:52:40 +01:00
Neil Alexander a1e80692b0
Merge branch 'master' into neilalexander/federationqueues 2020-05-06 15:45:10 +01:00
Neil Alexander e4f828101d Remodel a bit with channels 2020-05-06 15:43:38 +01:00
Neil Alexander 1be77d2a97 Loopback invite events 2020-05-06 15:10:44 +01:00
Neil Alexander 7ff2bbf13e Federation sender to ignore invites that are destined locally 2020-05-06 14:52:00 +01:00
Kegsay 1294852270
Add tests around federationapi's txnReq (#1010)
* Add necessary stubs for testing txnReq

* Add basic tests
2020-05-06 14:27:02 +01:00
Neil Alexander 7ac699bc77 Dispatch sends again 2020-05-06 14:15:33 +01:00
Neil Alexander 18cb75b0ac Dedupe destinations, fix other bug hopefully 2020-05-06 14:00:04 +01:00
Neil Alexander 9d293caec8 Don't retry sucessful invites, don't dispatch sendEvent, sendInvite etc 2020-05-06 13:15:32 +01:00
Neil Alexander 53f690f2d3 Improve logic a bit, don't block on wakeup, move idle check 2020-05-06 12:39:55 +01:00
Neil Alexander 0eee57cbea Fix comment, use atomic add 2020-05-06 10:26:27 +01:00
Neil Alexander b167f3c608 Break out statistics (tracked component-wide), report success and failures from Perform actions 2020-05-06 10:04:12 +01:00
Neil Alexander 7434715f70 Tidy up a bit 2020-05-06 09:30:12 +01:00
Neil Alexander 4ff46624ff Don't take copies again 2020-05-05 17:14:23 +01:00
Neil Alexander ade8352045 Don't accidentally drop queued messages 2020-05-05 17:02:27 +01:00
Neil Alexander 88c8e0038f
Merge branch 'master' into neilalexander/federationqueues 2020-05-05 16:46:56 +01:00
Kegsay 1db5dfe4d0
Fetch events by ID rather than use current state as this includes auth events (#1009) 2020-05-05 16:46:22 +01:00
Neil Alexander ed1bc9d2ee Take copies of events before passing to destination queues 2020-05-05 16:12:05 +01:00
Neil Alexander 10bf6fee36
Merge branch 'master' into neilalexander/federationqueues 2020-05-05 15:54:58 +01:00
Kegan Dougal a06511cae8 Remove debug line 2020-05-05 15:54:12 +01:00
Neil Alexander 13da3122e7
Merge branch 'master' into neilalexander/federationqueues 2020-05-05 15:49:14 +01:00
Kegsay 31d3b0d4a5
Prefer /state_ids when missing state across federation (#1008)
* Prefer /state_ids when missing state across federation

* Linting

* Better logging
2020-05-05 15:48:37 +01:00
Neil Alexander 74ed7140f9 Tweaks 2020-05-05 14:59:43 +01:00
Neil Alexander c71bcb2cdf Tweaks 2020-05-05 14:37:06 +01:00
Neil Alexander 93eb8163a1 Tweaks 2020-05-05 14:13:28 +01:00
Neil Alexander c4ee20c95e Improve federation sender performance and behaviour, add backoff 2020-05-05 13:20:48 +01:00
Neil Alexander 9d15312ef6
Fix RespState/RespSendJoin (#1005)
* Update gmsl

* Update gomatrixserverlib

* Add link to spec
2020-05-05 10:53:38 +01:00
Neil Alexander 9b1b095b49
Roomserver perform leave (#1004)
* First pass at PerformLeave

* Fix SQLite bulkSelectEventStateKey

* Update gomatrixserverlib

* Fix bugs

* Tidy a bit

* Satisfy King Linter

* Review comments

* Review comments

* Fix constants in SQLite event state keys table
2020-05-04 18:34:09 +01:00
Neil Alexander df80e42408 Update gomatrixserverlib 2020-05-04 15:30:36 +01:00
Andrew Morgan 65c6fbddeb
Fix newlines between white/blacklist test names in buildkite annotations (#1003) 2020-05-04 14:48:49 +01:00
Neil Alexander 5c894efd0e
Roomserver perform join (#1001)
* Add PerformJoin template

* Try roomserver perform join

* Send correct server name to FS API

* Pass through content, try to handle multiple server names

* Fix local server checks

* Don't refer to non-existent error

* Add directory lookups of aliases

* Remove unneeded parameters

* Don't repeat join events into the roomserver

* Unmarshal the content, that would help

* Check if the user is already in the room in the fedeationapi too

* Return incompatible room version error

* Use Membership, don't try more servers than needed

* Review comments, make FS API take list of servernames, dedupe them, break out of loop properly on success

* Tweaks
2020-05-04 13:53:47 +01:00
Kegsay 36bbb25561
Fix ordering when backfilling (#1000)
* Fix ordering when backfilling

The problem was that we weren't sorting the returned events
by depth when sending them back to the caller, instead we
were sorting by prev_events which is not the same thing.

* Fixup tests
2020-05-01 16:41:13 +01:00
Neil Alexander f7cfa75886
Limit database connections (#980, #564) (#998)
* Limit database connections (#564)

- Add new options to the config file database:
      max_open_conns: 100
      max_idle_conns: 2
      conn_max_lifetime: -1
- Implement connection parameter setup on the *DB (database/sql) in internal/sqlutil/trace.go:Open()
- Propagate the values in the form of DbProperties interface via all the
  Open() and NewDatabase() functions

Signed-off-by: Tomas Jirka <tomas.jirka@email.cz>

* Fix wasm builds

* Remove file accidentally added from working tree

Co-authored-by: Tomas Jirka <tomas.jirka@email.cz>
2020-05-01 13:34:53 +01:00
Neil Alexander 908108c23e
Rename FS queue package to internal (#997) 2020-05-01 13:01:50 +01:00
Kegsay 17e046f18f
Fix prev_batch tokens (#999) 2020-05-01 12:41:38 +01:00
Kegsay b28674435e
Correctly generate backpagination tokens for events which have the same depth (#996)
* Correctly generate backpagination tokens for events which have the same depth

With tests. Unfortunately the code around here is hard to understand.
There will be a subsequent PR which fixes this up now that we have a test
harness in place.

* Add postgres impl

* More linting

* Fix psql statement so it actually works
2020-05-01 11:01:34 +01:00
Neil Alexander e15f6676ac
Consolidation of roomserver APIs (#994)
* Consolidation of roomserver APIs

* Comment out alias tests for now, they are broken

* Wire AS API into roomserver again

* Roomserver didn't take asAPI param before so return to that

* Prevent roomserver asking AS API for alias info

* Rename some files

* Remove alias_test, incoherent tests and unwanted appservice integration

* Remove FS API inject on syncapi component
2020-05-01 10:48:17 +01:00
Kegsay ebbfc12592
Add tests for the storage interface (#995)
* Move docs to interface

* Add tests around syncing

* Add topology token test

* Linting
2020-04-30 17:15:29 +01:00
Neil Alexander 540f6fcd94 Update gmsl for key validity fix 2020-04-30 13:50:11 +01:00
Neil Alexander 77fe509031
Enable v5 rooms (#992)
* Enable v5 roooms

* Update sytest-whitelist

* Enable v5 rooms by default, update gomatrixserverlib
2020-04-29 19:37:00 +01:00
Kegsay 4ad52c67ca
Honour history_visibility when backfilling (#990)
* Make backfill work for shared history visibility

* fetch missing state on backfill to remember snapshots correctly

* Fix gmsl to not mux in auth events into room state

* Whoops

* Linting
2020-04-29 18:41:45 +01:00
Neil Alexander 458b364781 Update gomatrixserverlib 2020-04-29 15:33:17 +01:00
Neil Alexander 64e94e9a6f
Join room support in federation sender (#989)
* Implement PerformJoinRequest

* Rename perform functions

* Check send join response

* Temporary wiring to test federation sender room joins

* Actually pass through the config

* Make sure membership content shows join
2020-04-29 15:29:39 +01:00
Neil Alexander a308e61331
Federation sender API remodel (#988)
* Define an input API for the federationsender

* Wiring for rooomserver input API and federation sender input API

* Whoops, commit common too

* Merge input API into query API

* Rename FederationSenderQueryAPI to FederationSenderInternalAPI

* Fix dendritejs

* Rename Input to Perform

* Fix a couple of inputs -> performs

* Remove needless storage interface, add comments
2020-04-29 11:34:31 +01:00
Kegsay a4b9edb28e
Dependency inject the federation client so p2p binaries work as expected (#987) 2020-04-28 16:51:16 +01:00
Kegsay 0354836b57
Unbreak the wasm build (#986) 2020-04-28 16:22:00 +01:00
Kegsay 35b7cbd5d8
sql/backwards_extremities: Shift to table format and share code (#985)
* sql/backwards_extremities: Shift to table format and share code

This is an initial cut to reduce boilerplate at the storage layer.
It removes the need for 2x `_table.go` files, one for each DB engine,
replacing it with a single struct which has an interface which
implements the raw SQL statements.

The actual impl sits alongside the interface declaration which is
generally regarded as best practice (though no canonical sources).
Especially in this case where the impl is tiny (functions returning
strings) and relies heavily on the function signatures of the
table struct (for parameters), having the context in the same file
is useful.

* Remove _table redundancy
2020-04-28 15:50:24 +01:00
Neil Alexander 5071ecb8b3
Fix URLDecodeMapValues (#984)
* Update gomatrixserverlib

* Fix URLDecodeMapValues

* Update gomatrixserverlib
2020-04-28 12:50:49 +01:00