Commit graph

905 commits

Author SHA1 Message Date
Neil Alexander 5a7e332d1f Update comments 2020-06-16 12:21:58 +01:00
Neil Alexander 20bf0ed242 Fix server key API test 2020-06-16 12:20:33 +01:00
Neil Alexander 5e609796c9 Fix internal cache stuff again 2020-06-16 12:11:51 +01:00
Neil Alexander aae7c86fda Fix federation API test 2020-06-16 12:09:00 +01:00
Neil Alexander 3d8d9ed50f Partially revert "Move responsibility for generating local keys into server key API, don't register prom in caches unless needed, start tests"
This reverts commit d7eb54c5b3.
2020-06-16 12:06:15 +01:00
Neil Alexander 01f944218d Add URL check in test 2020-06-16 11:32:49 +01:00
Neil Alexander cd19f85175 Add comments to caching 2020-06-16 11:25:06 +01:00
Neil Alexander 2bbc43deff Break up FetchKeys 2020-06-16 11:20:38 +01:00
Neil Alexander 9fa7654beb Fix Federation API test 2020-06-15 17:28:09 +01:00
Neil Alexander c4e5c69870 Merge branch 'master' into neilalexander/keysagain 2020-06-15 17:19:49 +01:00
Neil Alexander f4e54ff4b6 Update gomatrixserverlib 2020-06-15 17:17:31 +01:00
Neil Alexander 8009093695 Tweaks 2020-06-15 17:05:03 +01:00
Kegsay 67ad661813
Unbreak HTTP mode (#1131) 2020-06-15 16:58:22 +01:00
Kegsay 7c36fb78a7
Fix rooms v3 url paths for good - with tests (#1130)
* Fix rooms v3 url paths for good - with tests

- Add a test rig around `federationapi` to test routing.
- Use `JSONVerifier` over `KeyRing` so we can stub things out more easily.
- Add `test.NopJSONVerifier` which verifies nothing.
- Add `base.BaseMux` which is the original `mux.Router` used to spawn public/internal routers.
- Listen on `base.BaseMux` and not the default serve mux as it cleans paths which we don't want.
- Factor out `ListenAndServe` to `test.ListenAndServe` and add flag for listening on TLS.

* Fix comments

* Linting
2020-06-15 16:57:59 +01:00
Neil Alexander 8afee910ef missing func 2020-06-15 16:51:59 +01:00
Neil Alexander 094ef9bd6a Fetcher timeouts 2020-06-15 16:51:30 +01:00
Neil Alexander d4e30f2838 Fetcher timeouts 2020-06-15 16:42:14 +01:00
Neil Alexander 1efab75853 oops 2020-06-15 16:36:06 +01:00
Neil Alexander 2e2cece7e3 Perform final comparison against original set of requests 2020-06-15 16:35:25 +01:00
Neil Alexander 0a92c1f5ce Debug logging 2020-06-15 16:01:19 +01:00
Neil Alexander f38ae82826 Debug logging 2020-06-15 15:53:10 +01:00
Neil Alexander 5c7a4571de Update comments, add fixes from forward-merge 2020-06-15 15:23:51 +01:00
Neil Alexander bf6faa4290 Merge branch 'master' into neilalexander/keysagain 2020-06-15 15:11:46 +01:00
Neil Alexander e630190272 Tweak caching behaviour, update tests 2020-06-15 15:03:06 +01:00
Neil Alexander aabd45995a Don't run tests for now 2020-06-15 14:26:10 +01:00
Neil Alexander 811c2d8162 Don't store our own keys in the database 2020-06-15 14:21:18 +01:00
Neil Alexander 5e01375637 Don't store our own keys in the database 2020-06-15 14:20:09 +01:00
Neil Alexander d7eb54c5b3 Move responsibility for generating local keys into server key API, don't register prom in caches unless needed, start tests 2020-06-15 14:05:57 +01:00
Kegan Dougal 1aac317341 Unbreak build 2020-06-15 10:48:43 +01:00
Kegsay 0ba1245a46
Current wiring (#1125)
* Current wiring

* Add ServerKeyAPI lines
2020-06-15 10:13:57 +01:00
Kegsay 6b5996db17
Add bare bones user API (#1127)
* Add bare bones user API

with tests!

* linting
2020-06-15 09:54:11 +01:00
Kegsay 0dc4ceaa2d
Minor perf/debugging improvements (#1121)
* Minor perf/debugging improvements

- publicroomsapi: Don't call QueryEventsByID with no event IDs
- appservice: Consume only if there are 1 or more ASes
- roomserver: don't keep a copy of the request "for debugging" - we trace now

* fedsender: return early if we have no destinations

* Unbreak tests
2020-06-12 15:11:33 +01:00
Kegsay ecd7accbad
Rehuffle where things are in the internal package (#1122)
renamed:    internal/eventcontent.go -> internal/eventutil/eventcontent.go
	renamed:    internal/events.go -> internal/eventutil/events.go
	renamed:    internal/types.go -> internal/eventutil/types.go
	renamed:    internal/http/http.go -> internal/httputil/http.go
	renamed:    internal/httpapi.go -> internal/httputil/httpapi.go
	renamed:    internal/httpapi_test.go -> internal/httputil/httpapi_test.go
	renamed:    internal/httpapis/paths.go -> internal/httputil/paths.go
	renamed:    internal/routing.go -> internal/httputil/routing.go
	renamed:    internal/basecomponent/base.go -> internal/setup/base.go
	renamed:    internal/basecomponent/flags.go -> internal/setup/flags.go
	renamed:    internal/partition_offset_table.go -> internal/sqlutil/partition_offset_table.go
	renamed:    internal/postgres.go -> internal/sqlutil/postgres.go
	renamed:    internal/postgres_wasm.go -> internal/sqlutil/postgres_wasm.go
	renamed:    internal/sql.go -> internal/sqlutil/sql.go
2020-06-12 14:55:57 +01:00
Kegsay 4675e1ddb6
Add trace logging to RoomserverInternalAPI (#1120)
This is a wrapper around whatever impl we have which then logs
the function name/request/response/error.

Also tweak when we log on kafka streams: only log on the producer
side not the consumer side: we've never had issues with comms and
having 1 message rather than N would be nice.
2020-06-12 12:10:08 +01:00
Neil Alexander 079d8fe8fb
More key tweaks (#1116) 2020-06-12 11:07:26 +01:00
Kegsay ec7718e7f8
Roomserver API changes (#1118)
* s/QueryBackfill/PerformBackfill/g

* OutputEvent now includes AddStateEvents which contain the full event of extra state events

* Only include adds not the current event

* Get adding state right
2020-06-11 19:50:40 +01:00
Kegsay 25cd2dd1c9
Remove unused internal APIs (#1117) 2020-06-11 15:07:16 +01:00
Neil Alexander 89d61c4877 Fix tiebreak in Yggdrasil demo (uses wrong key) 2020-06-11 13:08:11 +01:00
Neil Alexander 81f4ef5e09 Start testing stuff 2020-06-11 13:02:11 +01:00
Neil Alexander 687326933e More key tweaks 2020-06-11 10:50:48 +01:00
Neil Alexander ce2517b9fb Support embedding Riot Web (#1113)
* Embed

    * Support for embedding is better now

    * HTTP on localhost for Ygg demo

    * Move embed into Yggdrasil demo
2020-06-11 10:16:46 +01:00
Neil Alexander b7d0ca6855 Update Yggdrasil demo timeouts again 2020-06-11 09:50:54 +01:00
Neil Alexander 63a24e81c4 Yggdrasil demo tweaks 2020-06-10 17:18:37 +01:00
Kegsay 399b6ae334
Remove federationsender producer, which in fact was not a producer (#1115)
* Remove federationsender producer, which in fact was not a producer

* Set the signing struct
2020-06-10 16:54:43 +01:00
Neil Alexander 3b4be90000
Improvements to Yggdrasil demo (#1114)
* Improvements to Yggdrasil demo

* Fix missing copyright

* Fix tie-break
2020-06-10 16:29:02 +01:00
Kegsay 90a0aa9b3e
Remove public appservice routes (#1112) 2020-06-10 13:28:32 +01:00
Kegsay b7187a9a35
Remove clientapi producers which aren't actually producers (#1111)
* Remove clientapi producers which aren't actually producers

They are actually just convenience wrappers around the internal APIs
for roomserver/eduserver. Move their logic to their respective `api`
packages and call them directly.

* Remove TODO

* unbreak ygg
2020-06-10 12:17:54 +01:00
Neil Alexander d9d6f4568c
Use ServerKeyAPI as keyring (#1110)
* Use ServerKeyAPI as keyring

* Update fetcher name for debug logs
2020-06-10 11:02:03 +01:00
Neil Alexander 464718c3e6
Yggdrasil-based P2P demo (#1108)
* Initial work on Yggdrasil demo

* Muxing?

* Yamux

* Updates to yamux

* Updates

* Comments

* Update to use monolith stuff

* Update go.mod/go.sum

* Set defaults

* Tweaks

* Update yggdrasil

* Update config

* MarshalIndent

* Change default instance name/port

* add -peer switch

* gocyclo, for a change

* Determinate yamux roles

* Fix copyright notices

* Remove HTTP API checks as always false, remove unused topic
2020-06-10 10:54:06 +01:00
Kegsay 98cb0705ea
Remove unused UserUpdates producer (#1109) 2020-06-10 10:34:22 +01:00