Commit graph

2121 commits

Author SHA1 Message Date
Till Faelligen 7df7a966b8 Merge branch 'main' of github.com:matrix-org/dendrite into s7evink/consent-tracking 2022-05-09 17:21:37 +02:00
Till Faelligen 75ca5490bc Fix build 2022-05-09 17:21:27 +02:00
Neil Alexander 79da75d483
Federation consumer adds_state_event_ids tweak (#2441)
* Don't ask roomserver for events we already have in federation API

* Check number of events returned is as expected

* Preallocate array

* Improve shape a bit
2022-05-09 16:19:35 +01:00
kegsay c9409078ac
Merge branch 'main' into s7evink/consent-tracking 2022-05-09 16:04:36 +01:00
Neil Alexander 1a7f4c8aa9
Don't try to re-fetch the event if it is listed in adds_state_event_ids (#2437)
* Don't try to re-fetch the event in the output message

* Try that again

* Add the initial event into the set
2022-05-09 15:22:33 +01:00
Till f69ebc6af2
Add roomserver tests (1/?) (#2434)
* Add EventJSONTable tests

* Add eventJSON tests

* Add EventStateKeysTable tests

* Add EventTypesTable tests

* Add Events Table tests
Move variable declaration outside loops
Switch to testify/assert for tests

* Move variable declaration outside loop

* Remove random data

* Fix issue where the EventReferenceSHA256 is not set

* Add more tests

* Revert "Fix issue where the EventReferenceSHA256 is not set"

This reverts commit 8ae34c4e5f.

* Update GMSL

* Add tests for duplicate entries

* Test what happens if we select non-existing NIDs

* Add test for non-existing eventType

* Really update GMSL
2022-05-09 15:30:32 +02:00
Neil Alexander 09d754cfbf
One NATS instance per BaseDendrite (#2438)
* One NATS instance per `BaseDendrite`

* Fix roomserver
2022-05-09 14:15:24 +01:00
Neil Alexander 79e2fbc663
Update to matrix-org/gomatrixserverlib#307 2022-05-09 13:53:51 +01:00
Neil Alexander 4c15c73b3a
Add (user_id, device_id) index on OTK table (#2435) 2022-05-09 11:13:04 +01:00
Neil Alexander 633ca06eb9
Version 0.8.3rc1 2022-05-06 16:34:52 +01:00
Neil Alexander 6bc6184d70
Simplify calculateLatest (#2430)
* Simplify `calculateLatest`

* Comments
2022-05-06 15:52:44 +01:00
Neil Alexander 85c00208c5
Fix power level event auth bugs (update to matrix-org/gomatrixserverlib#306) 2022-05-06 15:41:16 +01:00
Till 6493c0c0f2
Move LL cache (#2429) 2022-05-06 15:33:34 +02:00
Neil Alexander 507f63d0fc
Add PolylithMode base config option (#2428)
* Add `PolylithMode` base config option

* Polylith mode always uses HTTP APIs
2022-05-06 13:51:48 +01:00
Neil Alexander a1a5357f79
Produce more useful event auth errors (update to matrix-org/gomatrixserverlib#305) 2022-05-06 12:46:01 +01:00
kegsay 85704eff20
Clean up interface definitions (#2427)
* tidy up interfaces

* remove unused GetCreatorIDForAlias

* Add RoomserverUserAPI interface

* Define more interfaces

* Use AppServiceInternalAPI for consistent naming

* clean up federationapi constructor a bit

* Fix monolith in -http mode
2022-05-06 12:39:26 +01:00
Till Faelligen 3d3773d3d4 Rename migrations so they are executed 2022-05-06 10:15:42 +02:00
Till Faelligen 4bd9a73c13 Fix database locked 2022-05-06 09:50:20 +02:00
Till Faelligen 4d285fff60 Merge branch 'main' of github.com:matrix-org/dendrite into s7evink/consent-tracking 2022-05-06 09:43:50 +02:00
Till 4705f5761e
Add FAQ entry for anonymous stats (#2419) 2022-05-06 08:26:24 +02:00
kegsay 9957752a9d
Define component interfaces based on consumers (2/2) (#2425)
* convert remaining interfaces

* Tidy up the userapi interfaces
2022-05-05 19:30:38 +01:00
Neil Alexander e4da04e75b
Update to matrix-org/gomatrixserverlib#303 2022-05-05 14:06:05 +01:00
Neil Alexander 42f35a57ac
Update table names for user API stats table 2022-05-05 13:42:12 +01:00
Neil Alexander 530fd488a9
Don't log consumer errors on shutdown 2022-05-05 13:29:39 +01:00
kegsay 506de4bb3d
Define component interfaces based on consumers (1/2) (#2423)
* Specify interfaces used by appservice, do half of clientapi

* convert more deps of clientapi to finer-grained interfaces

* Convert mediaapi and rest of clientapi

* Somehow this got missed
2022-05-05 13:17:38 +01:00
Neil Alexander d9e71b93b6
Use gomatrixserverlib.Client instead of http.Client (#2421)
* Update to matrix-org/gomatrixserverlib#303

* Use `gomatrixserverlib.Client` for phone-home stats

* Use `gomatrixserverlib.Client` for push notifications

* Use `gomatrixserverlib.Client` for appservices

* Use `gomatrixserverlib.Client` for three-PID invites
2022-05-05 11:33:16 +01:00
Till 1bfe87aa56
Fix user already joined when using server notices (#2364) 2022-05-05 12:01:28 +02:00
Till 658e82f8bc
Don't use in-memory db for userapi tests (#2417)
* Don't use in-memory db

* Use WithAllDatabases where possible
2022-05-05 12:00:18 +02:00
kegsay 5c37f165ae
Errors from createdb are non-fatal (#2420)
As they are expected if the database already exists.
2022-05-05 10:53:52 +01:00
kegsay d86dcbef66
syncapi: define specific interfaces for internal HTTP communications (#2416)
* syncapi: use finer-grained interfaces when making the syncapi

* Use specific interfaces for syncapi-roomserver interactions

* Define query access token api for shared http auth code
2022-05-05 09:56:03 +01:00
Till 3c940c428d
Add opt-in anonymous stats reporting (#2249)
* Initial phone home stats queries

* Add userAgent to UpdateDeviceLastSeen
Add new Table for tracking daily user vists

* Add user_daily_visits table

* Fix queries

* userapi stats tables & queries

* userapi interface and internal api

* sycnapi stats queries

* testing phone home stats

* Add complete config to syncapi

* add missing files

* Fix queries

* Send empty request

* Add version & monolith stats

* Add configuration for phone home stats

* Move WASM to its own file, add config and comments

* Add tracing methods

* Add total rooms

* Add more fields, actually send data somewhere

* Move stats to the userapi

* Move phone home stats to util package

* Cleanup

* Linter & parts of GH comments

* More GH comments changes
- Move comments to SQL statements
- Shrink interface, add struct for stats
- No fatal errors, use defaults

* Be more explicit when querying

* Fix wrong calculation & wrong query params
Add tests

* Add Windows stats

* ADd build constraint

* Use new testing structure
Fix issues with getting values when using SQLite
Fix wrong AddDate value
Export UpdateUserDailyVisits

* Fix query params

* Fix test

* Add comment about countR30UsersSQL and countR30UsersV2SQL; fix test

* Update config

* Also update example config file

* Use OS level proxy, update logging

Co-authored-by: kegsay <kegan@matrix.org>
2022-05-04 19:04:28 +02:00
Till Faelligen 10dc02f1ea Debugging unit tests.. 2022-05-04 18:38:43 +02:00
Till Faelligen 94ed2d3689 Add tests, use simple http.HandlerFunc 2022-05-04 17:57:46 +02:00
Till Faelligen 964e1cef85 Remove noise 2022-05-04 17:57:29 +02:00
Till Faelligen 2eb3aab07e Split out UserConsentPolicyAPI for easier testing 2022-05-04 17:56:46 +02:00
Till Faelligen 60ba4b5612 Fix stupid mistake.. and just return the NullString 2022-05-04 17:53:49 +02:00
Till Faelligen 88612ddd0c Deduplicate constructing consent URL 2022-05-04 14:40:25 +02:00
Till Faelligen bddf8ed3ac Merge branch 'main' of github.com:matrix-org/dendrite into s7evink/consent-tracking 2022-05-04 14:34:16 +02:00
Till Faelligen 4d5feb2544 Fix userapi issues 2022-05-04 14:33:51 +02:00
Till Faelligen cd7a7606a1 Remove noise 2022-05-04 14:12:52 +02:00
Till Faelligen dc8cea6d57 PR comments config 2022-05-04 13:47:08 +02:00
Neil Alexander b0a9e85c4a
Fix bug in database global setup 2022-05-03 17:40:56 +01:00
Neil Alexander e01d1e1f5b
Skip tests that require a database if we can't connect to one (#2413)
* Skip tests that require a database if we can't connect to one

* Add `DENDRITE_SKIP_DB_TESTS` environment variable to bring @kegsay joy

* Call it `DENDRITE_TEST_SKIP_NODB` intead
2022-05-03 17:38:54 +01:00
kegsay e0cdf64c33
Merge branch 'main' into s7evink/consent-tracking 2022-05-03 17:31:36 +01:00
Neil Alexander dd061a172e
Tidy up AddPublicRoutes (#2412)
* Simplify federation API `AddPublicRoutes`

* Simplify client API `AddPublicRoutes`

* Simplify media API `AddPublicRoutes`

* Simplify sync API `AddPublicRoutes`

* Simplify `AddAllPublicRoutes`
2022-05-03 17:17:02 +01:00
Neil Alexander 4ad5f9c982
Global database connection pool (for monolith mode) (#2411)
* Allow monolith components to share a single database pool

* Don't yell about missing connection strings

* Rename field

* Setup tweaks

* Fix panic

* Improve configuration checks

* Update config

* Fix lint errors

* Update comments
2022-05-03 16:35:06 +01:00
Till Faelligen ef62255685 Merge branch 'main' of github.com:matrix-org/dendrite into s7evink/consent-tracking 2022-05-03 09:54:25 +02:00
Till 979a551f1e
Return null if MaxFileSizeBytes is 0 (#2409)
* Return "null" if MaxFileSizeBytes is 0

* Add comment and nil check (better save than sorry)

* Simplify config
2022-05-02 10:47:16 +02:00
Brian Meek bfa344e831
Test_Devices, sqlite may return devices in different order, test should still pass (#2406) 2022-04-30 00:23:11 +02:00
Till 987d7adc5d
Return "to", if we didn't return any presence events (#2407)
Return correct stream position, if we didn't return any presence events
2022-04-30 00:07:50 +02:00