Commit graph

897 commits

Author SHA1 Message Date
Neil Alexander 9fa7654beb Fix Federation API test 2020-06-15 17:28:09 +01:00
Neil Alexander c4e5c69870 Merge branch 'master' into neilalexander/keysagain 2020-06-15 17:19:49 +01:00
Neil Alexander f4e54ff4b6 Update gomatrixserverlib 2020-06-15 17:17:31 +01:00
Neil Alexander 8009093695 Tweaks 2020-06-15 17:05:03 +01:00
Kegsay 67ad661813
Unbreak HTTP mode (#1131) 2020-06-15 16:58:22 +01:00
Kegsay 7c36fb78a7
Fix rooms v3 url paths for good - with tests (#1130)
* Fix rooms v3 url paths for good - with tests

- Add a test rig around `federationapi` to test routing.
- Use `JSONVerifier` over `KeyRing` so we can stub things out more easily.
- Add `test.NopJSONVerifier` which verifies nothing.
- Add `base.BaseMux` which is the original `mux.Router` used to spawn public/internal routers.
- Listen on `base.BaseMux` and not the default serve mux as it cleans paths which we don't want.
- Factor out `ListenAndServe` to `test.ListenAndServe` and add flag for listening on TLS.

* Fix comments

* Linting
2020-06-15 16:57:59 +01:00
Neil Alexander 8afee910ef missing func 2020-06-15 16:51:59 +01:00
Neil Alexander 094ef9bd6a Fetcher timeouts 2020-06-15 16:51:30 +01:00
Neil Alexander d4e30f2838 Fetcher timeouts 2020-06-15 16:42:14 +01:00
Neil Alexander 1efab75853 oops 2020-06-15 16:36:06 +01:00
Neil Alexander 2e2cece7e3 Perform final comparison against original set of requests 2020-06-15 16:35:25 +01:00
Neil Alexander 0a92c1f5ce Debug logging 2020-06-15 16:01:19 +01:00
Neil Alexander f38ae82826 Debug logging 2020-06-15 15:53:10 +01:00
Neil Alexander 5c7a4571de Update comments, add fixes from forward-merge 2020-06-15 15:23:51 +01:00
Neil Alexander bf6faa4290 Merge branch 'master' into neilalexander/keysagain 2020-06-15 15:11:46 +01:00
Neil Alexander e630190272 Tweak caching behaviour, update tests 2020-06-15 15:03:06 +01:00
Neil Alexander aabd45995a Don't run tests for now 2020-06-15 14:26:10 +01:00
Neil Alexander 811c2d8162 Don't store our own keys in the database 2020-06-15 14:21:18 +01:00
Neil Alexander 5e01375637 Don't store our own keys in the database 2020-06-15 14:20:09 +01:00
Neil Alexander d7eb54c5b3 Move responsibility for generating local keys into server key API, don't register prom in caches unless needed, start tests 2020-06-15 14:05:57 +01:00
Kegan Dougal 1aac317341 Unbreak build 2020-06-15 10:48:43 +01:00
Kegsay 0ba1245a46
Current wiring (#1125)
* Current wiring

* Add ServerKeyAPI lines
2020-06-15 10:13:57 +01:00
Kegsay 6b5996db17
Add bare bones user API (#1127)
* Add bare bones user API

with tests!

* linting
2020-06-15 09:54:11 +01:00
Kegsay 0dc4ceaa2d
Minor perf/debugging improvements (#1121)
* Minor perf/debugging improvements

- publicroomsapi: Don't call QueryEventsByID with no event IDs
- appservice: Consume only if there are 1 or more ASes
- roomserver: don't keep a copy of the request "for debugging" - we trace now

* fedsender: return early if we have no destinations

* Unbreak tests
2020-06-12 15:11:33 +01:00
Kegsay ecd7accbad
Rehuffle where things are in the internal package (#1122)
renamed:    internal/eventcontent.go -> internal/eventutil/eventcontent.go
	renamed:    internal/events.go -> internal/eventutil/events.go
	renamed:    internal/types.go -> internal/eventutil/types.go
	renamed:    internal/http/http.go -> internal/httputil/http.go
	renamed:    internal/httpapi.go -> internal/httputil/httpapi.go
	renamed:    internal/httpapi_test.go -> internal/httputil/httpapi_test.go
	renamed:    internal/httpapis/paths.go -> internal/httputil/paths.go
	renamed:    internal/routing.go -> internal/httputil/routing.go
	renamed:    internal/basecomponent/base.go -> internal/setup/base.go
	renamed:    internal/basecomponent/flags.go -> internal/setup/flags.go
	renamed:    internal/partition_offset_table.go -> internal/sqlutil/partition_offset_table.go
	renamed:    internal/postgres.go -> internal/sqlutil/postgres.go
	renamed:    internal/postgres_wasm.go -> internal/sqlutil/postgres_wasm.go
	renamed:    internal/sql.go -> internal/sqlutil/sql.go
2020-06-12 14:55:57 +01:00
Kegsay 4675e1ddb6
Add trace logging to RoomserverInternalAPI (#1120)
This is a wrapper around whatever impl we have which then logs
the function name/request/response/error.

Also tweak when we log on kafka streams: only log on the producer
side not the consumer side: we've never had issues with comms and
having 1 message rather than N would be nice.
2020-06-12 12:10:08 +01:00
Neil Alexander 079d8fe8fb
More key tweaks (#1116) 2020-06-12 11:07:26 +01:00
Kegsay ec7718e7f8
Roomserver API changes (#1118)
* s/QueryBackfill/PerformBackfill/g

* OutputEvent now includes AddStateEvents which contain the full event of extra state events

* Only include adds not the current event

* Get adding state right
2020-06-11 19:50:40 +01:00
Kegsay 25cd2dd1c9
Remove unused internal APIs (#1117) 2020-06-11 15:07:16 +01:00
Neil Alexander 89d61c4877 Fix tiebreak in Yggdrasil demo (uses wrong key) 2020-06-11 13:08:11 +01:00
Neil Alexander 81f4ef5e09 Start testing stuff 2020-06-11 13:02:11 +01:00
Neil Alexander 687326933e More key tweaks 2020-06-11 10:50:48 +01:00
Neil Alexander ce2517b9fb Support embedding Riot Web (#1113)
* Embed

    * Support for embedding is better now

    * HTTP on localhost for Ygg demo

    * Move embed into Yggdrasil demo
2020-06-11 10:16:46 +01:00
Neil Alexander b7d0ca6855 Update Yggdrasil demo timeouts again 2020-06-11 09:50:54 +01:00
Neil Alexander 63a24e81c4 Yggdrasil demo tweaks 2020-06-10 17:18:37 +01:00
Kegsay 399b6ae334
Remove federationsender producer, which in fact was not a producer (#1115)
* Remove federationsender producer, which in fact was not a producer

* Set the signing struct
2020-06-10 16:54:43 +01:00
Neil Alexander 3b4be90000
Improvements to Yggdrasil demo (#1114)
* Improvements to Yggdrasil demo

* Fix missing copyright

* Fix tie-break
2020-06-10 16:29:02 +01:00
Kegsay 90a0aa9b3e
Remove public appservice routes (#1112) 2020-06-10 13:28:32 +01:00
Kegsay b7187a9a35
Remove clientapi producers which aren't actually producers (#1111)
* Remove clientapi producers which aren't actually producers

They are actually just convenience wrappers around the internal APIs
for roomserver/eduserver. Move their logic to their respective `api`
packages and call them directly.

* Remove TODO

* unbreak ygg
2020-06-10 12:17:54 +01:00
Neil Alexander d9d6f4568c
Use ServerKeyAPI as keyring (#1110)
* Use ServerKeyAPI as keyring

* Update fetcher name for debug logs
2020-06-10 11:02:03 +01:00
Neil Alexander 464718c3e6
Yggdrasil-based P2P demo (#1108)
* Initial work on Yggdrasil demo

* Muxing?

* Yamux

* Updates to yamux

* Updates

* Comments

* Update to use monolith stuff

* Update go.mod/go.sum

* Set defaults

* Tweaks

* Update yggdrasil

* Update config

* MarshalIndent

* Change default instance name/port

* add -peer switch

* gocyclo, for a change

* Determinate yamux roles

* Fix copyright notices

* Remove HTTP API checks as always false, remove unused topic
2020-06-10 10:54:06 +01:00
Kegsay 98cb0705ea
Remove unused UserUpdates producer (#1109) 2020-06-10 10:34:22 +01:00
Kegsay 85ac8a3f5b
Factor out how monolith routes get added (#1107)
Previously we had 3 monoliths:
 - dendrite-monolith-server
 - dendrite-demo-libp2p
 - dendritejs

which all had their own of setting up public routes. Factor this
out into a new `setup.Monolith` struct which gets all dependencies
set as fields. This is different to `basecomponent.Base` which
doesn't provide any way to set configured deps (e.g public rooms db)

Part of a larger process to clean up how we initialise Dendrite.
2020-06-09 12:07:33 +01:00
Kegsay 4f171c56a8
Split out SetupFooComponent (#1106)
* Split out adding HTTP routes from making internal APIs for clarity

* Split out more components

* Split out more things

* Finish converting

* internal mux for internal routes
2020-06-08 15:51:07 +01:00
Neil Alexander cdb9a11571 Update gomatrixserverlib 2020-06-08 13:56:03 +01:00
Neil Alexander 67784ecb56
Nuke old Docker stuff (#1104)
* Remove old Docker stuff

* Move hub stuff to upper level docker directory

* Build monolith images

* Update README.md

* Update paths

* Update README.md
2020-06-05 19:00:30 +01:00
Neil Alexander e7b19d2c70
More flexible caching (#1101) 2020-06-05 16:42:01 +01:00
Neil Alexander 76ff47c052
Use AuthChainProvider to try and speed up federated joins (#1100)
* Use MissingAuthEventHandler on performjoin to try and speed up cases where we have missing events

* Update gomatrixserverlib

* Use supplied room version

* Use AuthChainProvider

* Tweaks

* Update gomatrixserverlib

* Signature checks
2020-06-05 11:48:52 +01:00
Kegsay 29a20d1da7
General cleanup when making components (#1098)
* Remove ParseMonolith/LoadMonolith

* cleanup which components need to be made
2020-06-05 09:28:15 +01:00
Kegsay 2bd12f635c
Convert serverkeys to inthttp (#1097) 2020-06-04 16:26:35 +01:00