Commit graph

177 commits

Author SHA1 Message Date
Neil Alexander 3c2e6f967b
Federation fixes and error handling (#970)
* Improve error handling in federation /send endpoint a bit

* Remove unknownRoomError, use unmarshalError when unable to get room ID

* Swap out a couple more internal server errors

* Update gomatrixserverlib

* Update gomatrixserverlib

* Update gomatrixserverlib

* Update gomatrixserverlib

* Update gomatrixserverlib

* Update gomatrixserverlib

* Return bad limit in error

* Same with domain/userid
2020-04-16 17:59:55 +01:00
Neil Alexander 7b3edf4622
Tweaks to backfill (#964) 2020-04-15 14:39:27 +01:00
Neil Alexander 895a72b6ee
Move /room/{roomID}/state endpoints into client API (#606) (#962)
* Move /room/{roomID}/state endpoints into client API (#606)

* Update sytest-whitelist

* Blacklist tests which rely on endpoints we don't implement
2020-04-14 18:36:08 +01:00
Neil Alexander dacee648f7
Federation for v3/v4 rooms (#954)
* Update gomatrixserverlib

* Default to room version 4

* Update gomatrixserverlib

* Limit prev_events and auth_events

* Fix auth_events, prev_events

* Fix linter issues

* Update gomatrixserverlib

* Fix getState

* Update sytest-whitelist

* Squashed commit of the following:

commit 067b875063
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Fri Apr 3 14:29:06 2020 +0100

    Invites v2 endpoint (#952)

    * Start converting v1 invite endpoint to v2

    * Update gomatrixserverlib

    * Early federationsender code for sending invites

    * Sending invites sorta happens now

    * Populate invite request with stripped state

    * Remodel a bit, don't reflect received invites

    * Handle invite_room_state

    * Handle room versions a bit better

    * Update gomatrixserverlib

    * Tweak order in destinationQueue.next

    * Revert check in processMessage

    * Tweak federation sender destination queue code a bit

    * Add comments

commit 955244c092
Author: Ben B <benne@klimlive.de>
Date:   Fri Apr 3 12:40:50 2020 +0200

    use custom http client instead of the http DefaultClient (#823)

    This commit replaces the default client from the http lib with a custom one.
    The previously used default client doesn't come with a timeout. This could cause
    unwanted locks.
    That solution chosen here creates a http client in the base component dendrite
    with a constant timeout of 30 seconds. If it should be necessary to overwrite
    this, we could include the timeout in the dendrite configuration.
    Here it would be a good idea to extend the type "Address" by a timeout and
    create an http client for each service.

    Closes #820

    Signed-off-by: Benedikt Bongartz <benne@klimlive.de>

    Co-authored-by: Kegsay <kegan@matrix.org>

* Update sytest-whitelist, sytest-blacklist

* Update go.mod/go.sum

* Add some error wrapping for debug

* Add a NOTSPEC to common/events.go

* Perform state resolution at send_join

* Set default room version to v2 again

* Tweak GetCapabilities

* Add comments to ResolveConflictsAdhoc

* Update sytest-blacklist

* go mod tidy

* Update sytest-whitelist, sytest-blacklist

* Update versions

* Updates from review comments

* Update sytest-blacklist, sytest-whitelist

* Check room versions compatible at make_join, add some comments, update gomatrixserverlib, other tweaks

* Set default room version back to v2

* Update gomatrixserverlib, sytest-whitelist
2020-04-09 15:46:06 +01:00
Neil Alexander 067b875063
Invites v2 endpoint (#952)
* Start converting v1 invite endpoint to v2

* Update gomatrixserverlib

* Early federationsender code for sending invites

* Sending invites sorta happens now

* Populate invite request with stripped state

* Remodel a bit, don't reflect received invites

* Handle invite_room_state

* Handle room versions a bit better

* Update gomatrixserverlib

* Tweak order in destinationQueue.next

* Revert check in processMessage

* Tweak federation sender destination queue code a bit

* Add comments
2020-04-03 14:29:06 +01:00
Kegsay 8fbe9f4078
Implement typing over federation (#949)
Also fix a pet peeve of mine: not putting units on things!!!

Manually tested on p2p and works well, with some fudge factor delay.
2020-03-30 16:40:28 +01:00
Neil Alexander 05e1ae8745
Further room version wiring (#936)
* Room version 2 by default, other wiring updates, update gomatrixserverlib

* Fix nil pointer exception

* Fix some more nil pointer exceptions hopefully

* Update gomatrixserverlib

* Send all room versions when joining, not just stable ones

* Remove room version cquery

* Get room version when getting events from the roomserver database

* Reset default back to room version 2

* Don't generate event IDs unless needed

* Revert "Remove room version cquery"

This reverts commit a170d58733.

* Query room version in federation API, client API as needed

* Improvements to make_join send_join dance

* Make room server producers use headered events

* Lint tweaks

* Update gomatrixserverlib

* Versioned SendJoin

* Query room version in syncapi backfill

* Handle transaction marshalling/unmarshalling within Dendrite

* Sorta fix federation (kinda)

* whoops commit federation API too

* Use NewEventFromTrustedJSON when getting events from the database

* Update gomatrixserverlib

* Strip headers on federationapi endpoints

* Fix bug in clientapi profile room version query

* Update gomatrixserverlib

* Return more useful error if room version query doesn't find the room

* Update gomatrixserverlib

* Update gomatrixserverlib

* Maybe fix federation

* Fix formatting directive

* Update sytest whitelist and blacklist

* Temporarily disable room versions 3 and 4 until gmsl is fixed

* Fix count of EDUs in logging

* Update gomatrixserverlib

* Update gomatrixserverlib

* Update gomatrixserverlib

* Rely on EventBuilder in gmsl to generate the event IDs for us

* Some review comments fixed

* Move function out of common and into gmsl

* Comment in federationsender destinationqueue

* Update gomatrixserverlib
2020-03-27 16:28:22 +00:00
Kegsay 9f74a8798e
bugfix: Fix #908 by setting the correct state after the event (#913)
* bugfix: Fix #908 by setting the correct state after the event

Previously, this would only happen if the state already existed
previously!

* Structured logging
2020-03-16 17:51:58 +00:00
Neil Alexander acb505b717
Implement gomatrixserverlib.HeaderedEvent in roomserver query API (#912)
* Implement gomatrixserverlib.HeaderedEvent, which should allow us to store room version headers along with the event across API boundaries and consumers/producers, and intercept unmarshalling to get the event structure right

* Add federationsender to previous
2020-03-16 17:29:52 +00:00
Kegsay dfd8b93d93
Federation: Add event_auth endpoint (#905)
* Federation: Add event_auth endpoint

Signed-off-by: Sudhanshu Jaiswal <sudsjaiswal@gmail.com>

* Update metric names

Signed-off-by: Sudhanshu Jaiswal <sudsjaiswal@gmail.com>

* Rebase

* Add comment from @APwhitehat

Co-authored-by: Sudhanshu Jaiswal <sudsjaiswal@gmail.com>
2020-03-14 20:08:54 +00:00
Neil Alexander 6a1111c3d4
Try to recursively find auth events (to a point) if they are missing (#881)
* Try to recursively find auth events (to a point) if they are missing

* Remove recursion limit for now and other review fixes

* Simplify error handling for recursion

* Pass room version 1 only to MakeJoin until room version support comes later
2020-03-06 16:58:10 +00:00
Neil Alexander 59a1f4b8ed
Remove httputil.LogThenError so that the line numbers are reported properly - make error reporting slightly more useful (#879) 2020-03-02 16:20:44 +00:00
Kegsay b6ea1bc67a
Support sqlite in addition to postgres (#869)
* Move current work into single branch

* Initial massaging of clientapi etc (not working yet)

* Interfaces for accounts/devices databases

* Duplicate postgres package for sqlite3 (no changes made to it yet)

* Some keydb, accountdb, devicedb, common partition fixes, some more syncapi tweaking

* Fix accounts DB, device DB

* Update naffka dependency for SQLite

* Naffka SQLite

* Update naffka to latest master

* SQLite support for federationsender

* Mostly not-bad support for SQLite in syncapi (although there are problems where lots of events get classed incorrectly as backward extremities, probably because of IN/ANY clauses that are badly supported)

* Update Dockerfile -> Go 1.13.7, add build-base (as gcc and friends are needed for SQLite)

* Implement GET endpoints for account_data in clientapi

* Nuke filtering for now...

* Revert "Implement GET endpoints for account_data in clientapi"

This reverts commit 4d80dff458.

* Implement GET endpoints for account_data in clientapi (#861)

* Implement GET endpoints for account_data in clientapi

* Fix accountDB parameter

* Remove fmt.Println

* Fix insertAccountData SQLite query

* Fix accountDB storage interfaces

* Add empty push rules into account data on account creation (#862)

* Put SaveAccountData into the right function this time

* Not sure if roomserver is better or worse now

* sqlite work

* Allow empty last sent ID for the first event

* sqlite: room creation works

* Support sending messages

* Nuke fmt.println

* Move QueryVariadic etc into common, other device fixes

* Fix some linter issues

* Fix bugs

* Fix some linting errors

* Fix errcheck lint errors

* Make naffka use postgres as fallback, fix couple of compile errors

* What on earth happened to the /rooms/{roomID}/send/{eventType} routing

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-02-13 17:27:33 +00:00
Behouba Manassé 9937c05bea
Pass cfg by reference around the codebase (#819)
* Pass cfg by reference around the codebase

* Merge branch 'master' into pass-cfg-by-ref

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-02-11 11:18:12 +00:00
S7evinK b34fce0d85 Use gomatrixserverlib.Transaction instead of local type (#590) (#811) 2019-12-20 15:02:09 +00:00
Abhishek Kumar e2d73855eb Refuse /send_join without m.room.create (#824)
Signed-off-by: Abhishek Kumar <abhishekkumar2718@gmail.com>
2019-12-20 14:42:57 +00:00
Brendan Abolivier af9568ba44 Fix /send_join and /send_leave (#821)
Fix the /send_join and /send_leave endpoints, so that they use the v2 endpoints as mandated by MSC1802. Also comment out the SyTest tests that are failing because of lack of support for the v1 endpoints.
2019-12-20 14:41:32 +00:00
Neil Alexander b4d638cd04
Return server names (#833)
* Remove unnecessary map->array processing

* Return server names in room federation directory query

* Knock off a TODO
2019-12-20 13:24:57 +00:00
Alex Chen a81917c3e7
Make trailing slash on server key request optional (#788)
Cherry-picked from 7e861b60fbd721b374ec929926b14e57dc60ec41
2019-08-20 01:01:53 +08:00
Alex Chen d63d2a50cd
Replace event content types with ones in gomatrixserverlib (#785)
The types that are not in gomatrixserverlib are not replaced.

This also updates the gomatrixserverlib dependency.
2019-08-16 01:45:11 +08:00
Alex Chen f8d2860765 Replace membership and visibility values with constants (#774)
Signed-off-by: Alex Chen <minecnly@gmail.com>
2019-08-06 15:07:36 +01:00
Andrew Morgan e56d6e41fd Remove the trailing slash from /send/{txnID} and /backfill/{roomID} (#746)
In conjunction with matrix-org/sytest#651, /send/{txnID} and /backfill/{roomID} should not have trailing slashes according to the spec.
2019-07-18 15:41:29 +08:00
Andrew Morgan e2251199a4
Lots of small typo fixes (#737) 2019-07-12 16:43:01 +01:00
Andrew Morgan 33a1392541
Encode URLs properly (#728)
We were escaping the URL before performing any pattern matching on it.
This meant that if you sent data that URLdecoded to a "/", it would count as
a "/" in the URL, potentially causing a 404. This was causing some flaky tests
with some randomly-generated query parameters.

Now, we keep URLs encoded while doing the pattern matching, and only afterwards
do we URL decode each query parameter individually before passing them to their
respective handler functions.

github.com/gorilla/mux was also updated to v1.7.3 to fix a bug with URL encoding and subrouters.
2019-07-03 16:38:50 +01:00
Alex Chen b88112b05d Make federation state request 404 when event not in the room - fixes #625 (#716)
Signed-off-by: Alex Chen <minecnly@gmail.com>
2019-06-25 11:32:15 +01:00
Andrew Morgan bc382bba46
Fix pipeline, emoji and syntax (#713)
Fixes #697

Switched to golangci-lint, fixes issues with buildkite and does some linting fixes to appease the new linters.
2019-06-19 14:05:03 +01:00
ruben 74827428bd use go module for dependencies (#594) 2019-05-21 21:56:55 +01:00