Commit graph

79 commits

Author SHA1 Message Date
kegsay c9409078ac
Merge branch 'main' into s7evink/consent-tracking 2022-05-09 16:04:36 +01:00
Till 6493c0c0f2
Move LL cache (#2429) 2022-05-06 15:33:34 +02:00
Till Faelligen 4d285fff60 Merge branch 'main' of github.com:matrix-org/dendrite into s7evink/consent-tracking 2022-05-06 09:43:50 +02:00
kegsay d86dcbef66
syncapi: define specific interfaces for internal HTTP communications (#2416)
* syncapi: use finer-grained interfaces when making the syncapi

* Use specific interfaces for syncapi-roomserver interactions

* Define query access token api for shared http auth code
2022-05-05 09:56:03 +01:00
Till Faelligen ef62255685 Merge branch 'main' of github.com:matrix-org/dendrite into s7evink/consent-tracking 2022-05-03 09:54:25 +02:00
Neil Alexander b527e33c16
Send all account data on complete sync by default
Squashed commit of the following:

commit 0ec8de57261d573a5f88577aa9d7a1174d3999b9
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Tue Apr 26 16:56:30 2022 +0100

    Select filter onto provided target filter

commit da40b6fffbf5737864b223f49900048f557941f9
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Tue Apr 26 16:48:00 2022 +0100

    Specify other field too

commit ffc0b0801f63bb4d3061b6813e3ce5f3b4c8fbcb
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Tue Apr 26 16:45:44 2022 +0100

    Send as much account data as possible during complete sync
2022-04-26 16:58:20 +01:00
Till c07f347f00
Reuse the existing lazyload cache on /context and /messages (#2367) 2022-04-22 11:38:29 +02:00
Till Faelligen 2b496be2c3 Merge branch 'main' of github.com:matrix-org/dendrite into s7evink/consent-tracking 2022-04-20 17:26:27 +02:00
Till 29f2168789
Make /messages filterable (#2347)
* Make /messages filterable
Fix bug when determining if an event contains an URL

* Add newly passing test

* Fix test
2022-04-13 13:16:02 +02:00
Till 69f2ff7c82
Correctly use provided filters (#2339)
* Apply filters correctly

* Fix issues; Use prepareWithFilters

* Update gmsl & tests

* go.mod..

* PR comments
2022-04-11 09:05:23 +02:00
Neil Alexander 475d3c1af9
Better mapping of stream positions to topological positions in /messages (#2263)
* Convert stream positions into topological positions for both `from` and `to` in `/messages`

* Hopefully it works now

* Remove unnecessary logging

* Return sane values if `StreamToTopologicalPosition` can't work out the right thing to do

* Revert logging change

* tweaks

* Fix `selectEventIDsInRangeASCSQL`

* Test `Getting messages going forward is limited for a departed room (SPEC-216)` was passing incorrectly so un-whitelist it
2022-03-18 10:40:01 +00:00
Till Faelligen b9479a6f18 Merge branch 'main' of github.com:matrix-org/dendrite into s7evink/consent-tracking 2022-03-16 08:43:19 +01:00
S7evinK a2cf1aaf48
Fix /context with lazy_load_members (#2277)
* Add membership events to the end of the list, to ensure Sytest sees them

* Move tests to allowlist

* Append to correct list, fix logging message

* Add flakey tests to blacklist

* Remove flakey tests from whitelist
2022-03-14 20:04:24 +01:00
Till Faelligen 519ea13510 Add AuthAPICheck and optional functional checks
Rename several variables
2022-03-04 17:01:18 +01:00
Till Faelligen fa26aa9138 Merge branch 'main' of github.com:matrix-org/dendrite into s7evink/consent-tracking 2022-03-04 15:10:23 +01:00
Neil Alexander 7fc62d8178
Fix a panic in OnIncomingMessagesRequest (#2250)
It's possible for `GetStateEvent` to return `nil` if there was no error but the state event wasn't found. Therefore we need to be prepared for that case.

This should fix #2247.
2022-03-04 10:24:26 +00:00
Till Faelligen e6e62497c9 Merge branch 'main' of github.com:matrix-org/dendrite into s7evink/consent-tracking 2022-03-04 09:18:34 +01:00
Neil Alexander 72022a6ecf
Return 404 if event given to /context was not found (#2245) 2022-03-03 17:58:24 +00:00
S7evinK af610df85a
Return state on calls to /message and lazy load members (#2218)
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-03-01 14:39:56 +00:00
Till Faelligen ce658ab8f2 Fix missing params 2022-02-24 08:54:54 +01:00
Till Faelligen 2042303c6c Merge branch 'main' of github.com:matrix-org/dendrite into s7evink/consent-tracking 2022-02-22 07:48:36 +01:00
Neil Alexander bbe7d37928
Fix logic error on context history visibility (#2211) 2022-02-21 16:38:53 +00:00
S7evinK cf525d1f61
Implement /context (#2207)
* Add QueryEventsAfter

* Add /context

* Make all tests pass on sqlite

* Add queries to get the events for /context requests

* Move /context to the syncapi

* Revert "Add QueryEventsAfter"

This reverts commit 440a771d10.

* Simplify getting the required events

* Apply RoomEventFilter when getting events

* Add passing tests

* Remove logging

* Remove unused SQL statements
Update comments & add TODO
2022-02-21 17:12:22 +01:00
Till Faelligen 61cdb714df Use typed values for Consent 2022-02-21 16:23:28 +01:00
Till Faelligen 9c3a1cfd47 Merge branch 'main' of github.com:matrix-org/dendrite into s7evink/consent-tracking 2022-02-21 12:08:03 +01:00
Neil Alexander dbded87525
Expose sync endpoints via /v3 (#2203) 2022-02-18 14:14:16 +00:00
Till Faelligen d19518fca5 Add ConsentNotGiven error
Verify consent on desired endpoints
Store consent on POST requests
2022-02-15 11:07:24 +01:00
Till Faelligen 89340cfc52 Verify the user has given their consent, otherwise block access 2022-02-14 18:11:56 +01:00
S7evinK da2023ba04
Rename Riot to Element (#1874)
* s/riot/element/g

Signed-off-by: Till Faelligen <tfaelligen@gmail.com>

* fix formatting

Co-authored-by: kegsay <kegan@matrix.org>
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2021-07-20 09:45:40 +01:00
Neil Alexander d15836e260
Increase gocyclo complexity to 25 (and remove all but 2 golint directives related to it) (#1783) 2021-03-03 14:35:57 +00:00
Neil Alexander b70238f2d5
Basic sync filtering (#1721)
* Add some filtering (postgres only for now)

* Fix build error

* Try to use request filter

* Use default filter as a template when retrieving from the database

* Remove unused strut

* Update sytest-whitelist

* Add filtering to SelectEarlyEvents

* Fix Postgres selectEarlyEvents query

* Attempt filtering on SQLite

* Test limit, set field for limit/order in prepareWithFilters

* Remove debug logging, add comments

* Tweaks, debug logging

* Separate SQLite stream IDs

* Fix filtering in current state table

* Fix lock issues

* More tweaks

* Current state requires room ID

* Review comments
2021-01-19 18:00:42 +00:00
Neil Alexander 55cfe391f7
Update /messages pagination token behaviour (#1708)
* Tweak pagination tokens

* start should be the specified from

* Don't reverse start and end

* Tweak getStartEnd again

* Update sytest-whitelist

* NOTSPEC: Re-add iOS end of topology
2021-01-13 12:59:29 +00:00
Kegsay 76becac003
Add start_stream to /messages (#1648) 2020-12-16 18:10:39 +00:00
Kegsay 42e9cbf342
NOTSPEC: Make ?from= optional in /messages (#1647) 2020-12-16 17:31:03 +00:00
Neil Alexander 9c03b0a4fa
Refactor sync tokens (#1628)
* Refactor sync tokens

* Comment out broken notifier test

* Update types, sytest-whitelist

* More robust token checking

* Remove New functions for streaming tokens

* Export Logs in StreamingToken

* Fix tests
2020-12-10 18:57:10 +00:00
Neil Alexander b5aa7ca3ab
Top-level setup package (#1605)
* Move config, setup, mscs into "setup" top-level folder

* oops, forgot the EDU server

* Add setup

* goimports
2020-12-02 17:41:00 +00:00
Neil Alexander 20a01bceb2
Pass pointers to events — reloaded (#1583)
* Pass events as pointers

* Fix lint errors

* Update gomatrixserverlib

* Update gomatrixserverlib

* Update to matrix-org/gomatrixserverlib#240
2020-11-16 15:44:53 +00:00
S7evinK eccd0d2c1b
Implement forgetting about rooms (#1572)
* Add basic storage methods

* Add internal api handler

* Add check for forgotten room

* Add /rooms/{roomID}/forget endpoint

* Add missing rsAPI method

* Remove unused parameters

* Add passing tests

Signed-off-by: Till Faelligen <tfaelligen@gmail.com>

* Add missing file

* Add postgres migration

* Add sqlite migration

* Use Forgetter to forget room

* Remove empty line

* Update HTTP status codes

It looks like the spec calls for these to be 400, rather than 403: https://matrix.org/docs/spec/client_server/r0.6.1#post-matrix-client-r0-rooms-roomid-forget

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-11-05 10:19:23 +00:00
Neil Alexander c4756eee0e
Don't store backfilled events using request context (#1478) 2020-10-05 11:06:31 +01:00
Kegsay 279044cd90
Add history visibility guards (#1470)
* Add history visibility guards

Default to 'joined' visibility to avoid leaking events, until we get
around to implementing history visibility completely. Related #617

* Don't apply his vis checks on shared rooms

* Fix order of checks

* Linting and remove another misleading check

* Update whitelist
2020-10-02 17:08:13 +01:00
Matthew Hodgson 5bbe73747c unbreak CORS on /capabilities.
fixes https://github.com/vector-im/element-web/issues/15297
2020-09-27 22:23:42 +01:00
Neil Alexander 9677a95afc
API setup refactoring (#1266)
* Start HTTP endpoint refactoring

* Update SetupAndServeHTTP

* Fix builds

* Don't set up external listener if no address configured

* TLS HTTP setup

* Break apart client/federation/key/media muxes

* Tweaks

* Fix P2P demos

* Fix media API routing

* Review comments @Kegsay

* Update sample config

* Fix gobind build

* Fix External -> Public in federation API test
2020-08-13 12:16:37 +01:00
Neil Alexander 4b09f445c9
Configuration format v1 (#1230)
* Initial pass at refactoring config (not finished)

* Don't forget current state and EDU servers

* More shifting around

* Update server key API tests

* Fix roomserver test

* Fix more tests

* Further tweaks

* Fix current state server test (sort of)

* Maybe fix appservices

* Fix client API test

* Include database connection string in database options

* Fix sync API build

* Update config test

* Fix unit tests

* Fix federation sender build

* Fix gobind build

* Set Listen address for all services in HTTP monolith mode

* Validate config, reinstate appservice derived in directory, tweaks

* Tweak federation API test

* Set MaxOpenConnections/MaxIdleConnections to previous values

* Update generate-config
2020-08-10 14:18:04 +01:00
Kegsay a2174d3294
Implement /keys/changes (#1232)
* Implement /keys/changes

And refactor QueryKeyChanges to accept a `to` offset.

* Unbreak tests

* Sort keys when serialising log tokens
2020-07-30 14:52:21 +01:00
Kegsay 1ad7219e4b
Implement /sync limited and read timeline limit from stored filters (#1168)
* Move filter table to syncapi where it is used

* Implement /sync `limited` and read timeline limit from stored filters

We now fully handle `room.timeline.limit` filters (in-line + stored) and
return the right value for `limited` syncs.

* Update whitelist

* Default to the default timeline limit if it's unset, also strip the extra event correctly

* Update whitelist
2020-06-26 15:34:41 +01:00
Neil Alexander ddf1c8adf1
Hacks for supporting Riot iOS (#1148)
* Join room body is optional

* Support deprecated login by user/password

* Implement dummy key upload endpoint

* Make a very determinate end to /messages if we hit the create event in back-pagination

* Linting
2020-06-17 17:41:45 +01:00
Kegsay 9c77022513
Make userapi responsible for checking access tokens (#1133)
* Make userapi responsible for checking access tokens

There's still plenty of dependencies on account/device DBs, but this
is a start. This is a breaking change as it adds a required config
value `listen.user_api`.

* Cleanup

* Review comments and test fix
2020-06-16 14:10:55 +01:00
Kegsay ecd7accbad
Rehuffle where things are in the internal package (#1122)
renamed:    internal/eventcontent.go -> internal/eventutil/eventcontent.go
	renamed:    internal/events.go -> internal/eventutil/events.go
	renamed:    internal/types.go -> internal/eventutil/types.go
	renamed:    internal/http/http.go -> internal/httputil/http.go
	renamed:    internal/httpapi.go -> internal/httputil/httpapi.go
	renamed:    internal/httpapi_test.go -> internal/httputil/httpapi_test.go
	renamed:    internal/httpapis/paths.go -> internal/httputil/paths.go
	renamed:    internal/routing.go -> internal/httputil/routing.go
	renamed:    internal/basecomponent/base.go -> internal/setup/base.go
	renamed:    internal/basecomponent/flags.go -> internal/setup/flags.go
	renamed:    internal/partition_offset_table.go -> internal/sqlutil/partition_offset_table.go
	renamed:    internal/postgres.go -> internal/sqlutil/postgres.go
	renamed:    internal/postgres_wasm.go -> internal/sqlutil/postgres_wasm.go
	renamed:    internal/sql.go -> internal/sqlutil/sql.go
2020-06-12 14:55:57 +01:00
Kegsay ec7718e7f8
Roomserver API changes (#1118)
* s/QueryBackfill/PerformBackfill/g

* OutputEvent now includes AddStateEvents which contain the full event of extra state events

* Only include adds not the current event

* Get adding state right
2020-06-11 19:50:40 +01:00
Neil Alexander fe82e1f725
Separate muxes for public and internal APIs (#1056)
* Separate muxes for public and internal APIs

* Update client-api-proxy and federation-api-proxy so they don't add /api to the path

* Tidy up

* Consistent HTTP setup

* Set up prefixes properly
2020-05-22 11:43:17 +01:00