Commit graph

708 commits

Author SHA1 Message Date
Neil Alexander 7ef6290cf2 Fix bug in clientapi profile room version query 2020-03-26 10:05:23 +00:00
Neil Alexander eedb355b57 Strip headers on federationapi endpoints 2020-03-26 09:40:46 +00:00
Neil Alexander 50bb441215 Update gomatrixserverlib 2020-03-25 17:57:25 +00:00
Neil Alexander f319de1fbe Use NewEventFromTrustedJSON when getting events from the database 2020-03-25 16:54:55 +00:00
Neil Alexander b54d33a3e8 whoops commit federation API too 2020-03-25 16:41:18 +00:00
Neil Alexander 9e00f0e828 Sorta fix federation (kinda) 2020-03-25 16:40:55 +00:00
Neil Alexander c888ac2bd4 Handle transaction marshalling/unmarshalling within Dendrite 2020-03-25 15:13:32 +00:00
Neil Alexander 153c9924b0 Query room version in syncapi backfill 2020-03-25 14:16:21 +00:00
Neil Alexander b620099df7 Versioned SendJoin 2020-03-25 13:38:23 +00:00
Neil Alexander 35de2f444b Update gomatrixserverlib 2020-03-24 16:27:11 +00:00
Neil Alexander 6dbdfd59da Merge branch 'master' into neilalexander/stateresv2 2020-03-24 15:46:39 +00:00
Neil Alexander 0b732d6f45
Use HeaderedEvents in appservice component (#939)
* App service HeaderedEvents

* Fix database queries

* Fix lint error
2020-03-24 15:46:17 +00:00
Neil Alexander c12b493b7a Lint tweaks 2020-03-24 15:11:49 +00:00
Neil Alexander b5a53546c7 Merge branch 'master' into neilalexander/stateresv2 2020-03-24 14:47:11 +00:00
Neil Alexander 6fe6fc6704 Make room server producers use headered events 2020-03-24 14:28:38 +00:00
Kegan Dougal 951b5d5e68 Missing whitelist 2020-03-24 13:39:50 +00:00
Neil Alexander a8c23426e7 Improvements to make_join send_join dance 2020-03-24 12:36:07 +00:00
Kegsay 6bac7e5efd
Implement backfill over federation (#938)
* Implement history visibility checks for /backfill

Required for p2p to show history correctly.

* Add sytest

* Logging

* Fix two backfill bugs which prevented backfill from working correctly

- When receiving backfill requests, do not send the event that was in the original request.
- When storing backfill results, correctly update the backwards extremity for the room.

* hack: make backfill work multiple times

* add sqlite impl and remove logging

* Linting
2020-03-24 12:20:10 +00:00
Neil Alexander 8b85fa6fd2 Query room version in federation API, client API as needed 2020-03-24 09:59:48 +00:00
Neil Alexander 8624cbb0ab Revert "Remove room version cquery"
This reverts commit a170d58733.
2020-03-23 12:48:21 +00:00
Alex Chen 5a1a1ded1b
Fix dendrite config file location in docker guide (#934) 2020-03-23 11:16:17 +00:00
Neil Alexander 55a54100db Don't generate event IDs unless needed 2020-03-23 10:51:15 +00:00
Prateek Sachan c5b7a17272
Added special case for m.room.member event in appservice component (#874)
* Added special case for m.room.member event in appservice component

* fix review changes

Co-authored-by: Kegsay <kegan@matrix.org>
2020-03-20 21:40:24 +00:00
Neil Alexander 8b37191d99 Reset default back to room version 2 2020-03-20 17:39:39 +00:00
Neil Alexander 1cffb0aad0 Get room version when getting events from the roomserver database 2020-03-20 17:38:51 +00:00
Neil Alexander a170d58733 Remove room version cquery 2020-03-20 17:28:57 +00:00
Neil Alexander 63cbc614ff Send all room versions when joining, not just stable ones 2020-03-20 17:25:48 +00:00
Neil Alexander 7b7f48c589 Update gomatrixserverlib 2020-03-20 17:23:14 +00:00
Neil Alexander 8060c5c388 Fix some more nil pointer exceptions hopefully 2020-03-20 10:53:54 +00:00
Neil Alexander c1c81afbf6 Fix nil pointer exception 2020-03-20 10:42:11 +00:00
Neil Alexander 139904f604 Room version 2 by default, other wiring updates, update gomatrixserverlib 2020-03-20 09:59:32 +00:00
Neil Alexander f2030286de
Room server changes for room versions (#930)
* Rearrange state package a bit, add some code to look up the right state resolution algorithm

* Remove shared

* Add GetRoomVersionForRoomNID

* Try to use room version to get correct state resolution algorithm

* Fix room joins over federation

* nolint resolveConflictsV2 because all attempts to break it up so far just result in it being awfully less obvious how it works

* Rename Prepare to NewStateResolution

* Update comments

* Re-add missing tests
2020-03-19 18:33:04 +00:00
Neil Alexander 944d454cb0 Fix SQLite problems after sync API changes 2020-03-19 13:15:35 +00:00
Neil Alexander ad5849d222
HeaderedEvents in sync API (#922)
* Use HeaderedEvent in syncapi

* Update notifier test

* Fix persisting headered event

* Clean up unused API function

* Fix overshadowed err from linter

* Write headered JSON to invites table too

* Rename event_json to headered_event_json in syncapi database schemae

* Fix invites_table queries

* Update QueryRoomVersionCapabilitiesResponse comment

* Fix syncapi SQLite
2020-03-19 12:07:01 +00:00
Kegsay bfbf96eec9
p2p: Implement published rooms (#923)
* Create and glue ExternalPublicRoomsProvider into the public rooms component

This is how we will link p2p stuff to dendrite proper.

* Use gmsl structs rather than our own

* Implement federated public rooms

- Make thirdparty endpoint r0 so riot-web loads the public room list

* Typo

* Missing callsites
2020-03-19 11:04:08 +00:00
Prateek Sachan dc06c69887
Implement /joined_rooms (#911)
* Implemented /joined_rooms

* Removed account endpoint added by mistake

* trigger ci
2020-03-19 10:25:36 +00:00
Abhinav Krishna C K ec38783192
maintenance: Fix matrix-org#896 use %w format verb to wrap errors (#916)
* maintenance: Fix matrix-org#896 use %w format verb to wrap errors

* In Go version 1.13 a new formatting verb introduced for fmt.Errorf
  %w https://blog.golang.org/go1.13-errors

* update %s to %w to wrap errors.

* Update all instances of error type to use %w

Signed-off-by: Abhinav Krishna C K <me@abhy.me>

Co-authored-by: Kegsay <kegan@matrix.org>
2020-03-18 12:48:51 +00:00
Prateek Sachan c019ad7086
Log errors from rows.Close (#920)
* Log errors from rows.Close

* fixed imports

* Added contextual messages

* fixed review changes
2020-03-18 10:17:18 +00:00
Neil Alexander c2bd0b97b3
Get room versions from database (#918)
* Retrieve room version where known in roomserver

* Get room versions in alias code

* Increase gocyclothreshold to 13, since we hit that number a lot

* Remove gocyclo nolint from StoreEvent

* Update interface to get room version from room ID instead of NID

* Remove new API

* Fixed this query for SQLite but not for Postgres
2020-03-17 18:00:10 +00:00
Kegsay 1467cc10d8
bugfix: Fix a bug which caused failures to join rooms over federation (#917)
* bugfix: Fix a bug which caused failures to join rooms over federation

The cause of this was the semantics of `/send_join`'s `auth_chain` response.

Previously, we would only send back the auth chain *for the join event* and
not the entire room state. However, we would then try to check that the
room state is valid, and then be missing auth events.

Now, we send back the entire auth chain for all room state in `/send_join`.
The spec needs to be clarified that this is what the chain should be.

* refactor: split out grabbing state to reduce cyclo complexity
2020-03-17 17:18:48 +00:00
Kegsay 4af8323df3
bugfix: Fix a bug which caused prev_content not to be sent to clients (#919)
I don't know how this ever passed QA... also fix a missing rows.Close()
2020-03-17 16:45:40 +00:00
Neil Alexander a66c701b29
Add room version to room create request, persist in storage (#915)
* Add room version into createRoomReq

* Extract room version from m.room.create event when persisting

* Reduce cyclomatic complexity

* Update whitelist, gomatrixserverlib, tweaks to roomserver

* Update sytest-whitelist again
2020-03-17 15:12:01 +00:00
Neil Alexander aebf347a79
Implement gomatrixserverlib.HeaderedEvent in roomserver Kafka output (#914)
* Use Event.Headered

* Use HeaderedEvent in roomserver kafka output

* Fix syncserver-integration-tests

* Update producers to roomserver inputs

* Update gomatrixserverlib

* Update gomatrixserverlib

* Update gomatrixserverlib

* Update gomatrixserverlib

* Update gomatrixserverlib

* Update gomatrixserverlib
2020-03-17 11:01:25 +00:00
Kegsay 9f74a8798e
bugfix: Fix #908 by setting the correct state after the event (#913)
* bugfix: Fix #908 by setting the correct state after the event

Previously, this would only happen if the state already existed
previously!

* Structured logging
2020-03-16 17:51:58 +00:00
Neil Alexander acb505b717
Implement gomatrixserverlib.HeaderedEvent in roomserver query API (#912)
* Implement gomatrixserverlib.HeaderedEvent, which should allow us to store room version headers along with the event across API boundaries and consumers/producers, and intercept unmarshalling to get the event structure right

* Add federationsender to previous
2020-03-16 17:29:52 +00:00
Neil Alexander 452f393dd7
Update room version descriptors, add error handling (#906)
* Update room version descriptors, add error handling

* Fix database queries

* Drop Get from version package

* Fix database wrapping, add comments for version descriptions

* Don't set default room_version value in SQL
2020-03-16 16:05:29 +00:00
Kegsay dfd8b93d93
Federation: Add event_auth endpoint (#905)
* Federation: Add event_auth endpoint

Signed-off-by: Sudhanshu Jaiswal <sudsjaiswal@gmail.com>

* Update metric names

Signed-off-by: Sudhanshu Jaiswal <sudsjaiswal@gmail.com>

* Rebase

* Add comment from @APwhitehat

Co-authored-by: Sudhanshu Jaiswal <sudsjaiswal@gmail.com>
2020-03-14 20:08:54 +00:00
Kegan Dougal 7353de7dbb Make P2P use JSServer 2020-03-13 10:15:01 +00:00
Kegsay 8bc5084d8d
p2p: Use JSServer for comms rather than GoJsConn (#888)
* p2p: Use JSServer for comms rather than GoJsConn

This has several benefits:
 - it fixes a bug whereby you could not transmit >4k bytes to/from JS/Go land.
 - it more clearly exposes the interface point between Go and JS: a single
   global function call.
 - it presents a nicer API shape than the previous `net.Conn`.
 - it doesn't needlessly 'stream' data which is already sitting in-memory.

This is currently only active for local CS API traffic, another PR will
add Federation P2P support.

* Typo
2020-03-11 12:18:37 +00:00
Kegsay d71b72816d
p2p: use separate topics! (#886)
Without it, setting a display name fails and /sync wedges forever.
2020-03-10 18:10:10 +00:00