Commit graph

98 commits

Author SHA1 Message Date
Neil Alexander 594ab15007
don't commit 2022-11-10 10:55:05 +00:00
Neil Alexander 8cac948d7e
Drop constraint if exists 2022-11-10 10:54:49 +00:00
Neil Alexander 88dbe945f6
Try to find old indices if necessary 2022-11-10 10:49:25 +00:00
Neil Alexander f980ff267d
Drop primary keys in favour of new indices 2022-11-10 10:06:37 +00:00
Neil Alexander d9ab350ecf
Hmm 2022-11-08 15:01:55 +00:00
Neil Alexander 40122b5423
Separate migrations 2022-11-08 14:27:23 +00:00
Neil Alexander d27bcca3e9
Tweak migrations again 2022-11-08 13:37:47 +00:00
Neil Alexander 32127f4f1c
Tweak migrations 2022-11-08 13:11:37 +00:00
Neil Alexander 786f008607
Try that again 2022-11-08 10:27:01 +00:00
Neil Alexander 196839ebf4
Fix wrong server name reference 2022-11-08 09:24:08 +00:00
Neil Alexander a32982cb7f
Fix some more things 2022-11-07 16:52:28 +00:00
Neil Alexander a46bf9878b
Fix some bugs 2022-11-07 16:06:37 +00:00
Neil Alexander be2f90f0b8
OpenID 2022-11-07 16:01:12 +00:00
Neil Alexander 62dd0afc0b
Three PID 2022-11-07 15:29:51 +00:00
Neil Alexander e36decf025
Fix another bug 2022-11-07 15:11:46 +00:00
Neil Alexander 4998e0757d
Notifications 2022-11-07 15:02:42 +00:00
Neil Alexander 6ea580d9c9
Fix some bugs 2022-11-07 14:15:43 +00:00
Neil Alexander c72c58dab9
Pushers 2022-11-07 13:22:35 +00:00
Neil Alexander 76ac6dbdf1
Profiles tables 2022-11-07 13:04:55 +00:00
Neil Alexander 2e74be1bf2
Clean up some logging 2022-11-07 12:07:30 +00:00
Neil Alexander 03fc5fd0ea
what is going on 2022-11-07 11:11:18 +00:00
Neil Alexander 3b816f306e
Migration tweaks 2022-11-07 10:22:13 +00:00
Neil Alexander d058e052fc
Devices table 2022-11-07 09:30:26 +00:00
Neil Alexander da7f2a7047
More stuff 2022-11-04 15:11:28 +00:00
Neil Alexander a0cc4c806c
Account tables 2022-11-04 12:56:33 +00:00
Neil Alexander d1c61f5f95
Account data 2022-11-04 12:09:49 +00:00
Neil Alexander 7fd2c10975
Add server_name columns 2022-11-04 11:09:43 +00:00
Till 86b25a6337
Add message stats to reporting (#2748)
Since we're now listening on the `OutputRoomEvent` stream, we are able
to store messages stats.
2022-11-02 10:18:11 +00:00
Till Faelligen 869bf4d0ac
Fix flakey stats tests 2022-11-01 08:39:16 +01:00
Till e57b301722
Set display_name and/or avatar_url for server notices (#2820)
This should fix #2815 by making sure we actually set the `display_name`
and/or `avatar_url` and create the needed membership event.
To avoid creating a new membership event when starting Dendrite,
`SetAvatarURL` and `SetDisplayName` now return a `Changed` value, which
also makes the regular endpoints idempotent.
2022-10-21 10:48:25 +02:00
Neil Alexander 3aa92efaa3
Namespace user API tables (#2806)
This migrates all the various user API tables, indices and sequences to
be `userapi_`-namespaced, rather than the mess they are all now.
2022-10-18 15:59:08 +01:00
Till Faelligen fb44e33909
Relax test a bit 2022-10-14 10:37:04 +02:00
Neil Alexander fb6cb2dbcb
Tweak GetAccountByPassword more 2022-10-10 11:14:16 +01:00
Neil Alexander 980fa55846
Stronger passwordless account checks (fixes #2780) 2022-10-10 10:39:29 +01:00
Till 249b32c4f3
Refactor notifications (#2688)
This PR changes the handling of notifications
- removes the `StreamEvent` and `ReadUpdate` stream
- listens on the `OutputRoomEvent` stream in the UserAPI to inform the
SyncAPI about unread notifications
- listens on the `OutputReceiptEvent` stream in the UserAPI to set
receipts/update notifications
- sets the `read_markers` directly from within the internal UserAPI

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-09-27 15:01:34 +02:00
Neil Alexander 3e87096a21
Use TxStmt in SQLite pusher table 2022-09-26 09:54:54 +01:00
Till Faelligen 3c416517b0
Fix possible "Database is locked" issue 2022-09-26 10:45:35 +02:00
Neil Alexander 34e1dc210b
Use database writer in user API DeleteOldNotifications 2022-09-09 12:59:28 +01:00
Till 64472d9aab
Update getting pushrules, add tests, tweak pushrules (#2705)
This PR
- adds tests for `evaluatePushrules`
- removes the need for the UserAPI on the `OutputStreamEventConsumer`
(for easier testing)
- adds a method to get the pushrules from the database
- adds a new default pushrule for `m.reaction` events (and some other
tweaks)
2022-09-09 13:56:33 +02:00
Till 1b7f84250a
Fix linter issues (#2624)
* Try that again

* All hail the mighty linter?

* And once again

* goimport all the things
2022-08-05 11:12:41 +02:00
Till 081f5e7226
Update database migrations, remove goose (#2264)
* Add new db migration

* Update migrations
Remove goose

* Add possibility to test direct upgrades

* Try to fix WASM test

* Add checks for specific migrations

* Remove AddMigration
Use WithTransaction
Add Dendrite version to table

* Fix linter issues

* Update tests

* Update comments, outdent if

* Namespace migrations

* Add direct upgrade tests, skipping over one version

* Split migrations

* Update go version in CI

* Fix copy&paste mistake

* Use contexts in migrations

Co-authored-by: kegsay <kegan@matrix.org>
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-07-25 10:39:22 +01:00
Till ae7b6dd516
Fix #2498 (#2506) 2022-05-31 14:36:51 +02:00
Till beed39a8f4
Fix pq: invalid input syntax for integer when trying to get a numeric localpart (#2505)
* Match at least once

* Add the test
2022-05-31 11:27:04 +02:00
kegsay 6de29c1cd2
bugfix: E2EE device keys could sometimes not be sent to remote servers (#2466)
* Fix flakey sytest 'Local device key changes get to remote servers'

* Debug logs

* Remove internal/test and use /test only

Remove a lot of ancient code too.

* Use FederationRoomserverAPI in more places

* Use more interfaces in federationapi; begin adding regression test

* Linting

* Add regression test

* Unbreak tests

* ALL THE LOGS

* Fix a race condition which could cause events to not be sent to servers

If a new room event which rewrites state arrives, we remove all joined hosts
then re-calculate them. This wasn't done in a transaction so for a brief period
we would have no joined hosts. During this interim, key change events which arrive
would not be sent to destination servers. This would sporadically fail on sytest.

* Unbreak new tests

* Linting
2022-05-17 13:23:35 +01:00
Neil Alexander 42f35a57ac
Update table names for user API stats table 2022-05-05 13:42:12 +01:00
Till 3c940c428d
Add opt-in anonymous stats reporting (#2249)
* Initial phone home stats queries

* Add userAgent to UpdateDeviceLastSeen
Add new Table for tracking daily user vists

* Add user_daily_visits table

* Fix queries

* userapi stats tables & queries

* userapi interface and internal api

* sycnapi stats queries

* testing phone home stats

* Add complete config to syncapi

* add missing files

* Fix queries

* Send empty request

* Add version & monolith stats

* Add configuration for phone home stats

* Move WASM to its own file, add config and comments

* Add tracing methods

* Add total rooms

* Add more fields, actually send data somewhere

* Move stats to the userapi

* Move phone home stats to util package

* Cleanup

* Linter & parts of GH comments

* More GH comments changes
- Move comments to SQL statements
- Shrink interface, add struct for stats
- No fatal errors, use defaults

* Be more explicit when querying

* Fix wrong calculation & wrong query params
Add tests

* Add Windows stats

* ADd build constraint

* Use new testing structure
Fix issues with getting values when using SQLite
Fix wrong AddDate value
Export UpdateUserDailyVisits

* Fix query params

* Fix test

* Add comment about countR30UsersSQL and countR30UsersV2SQL; fix test

* Update config

* Also update example config file

* Use OS level proxy, update logging

Co-authored-by: kegsay <kegan@matrix.org>
2022-05-04 19:04:28 +02:00
Neil Alexander 4ad5f9c982
Global database connection pool (for monolith mode) (#2411)
* Allow monolith components to share a single database pool

* Don't yell about missing connection strings

* Rename field

* Setup tweaks

* Fix panic

* Improve configuration checks

* Update config

* Fix lint errors

* Update comments
2022-05-03 16:35:06 +01:00
Brian Meek bfa344e831
Test_Devices, sqlite may return devices in different order, test should still pass (#2406) 2022-04-30 00:23:11 +02:00
Till Faelligen 8683ff78b1 Make tests more reliable 2022-04-28 15:06:34 +02:00
Till f023cdf8c4
Add UserAPI storage tests (#2384)
* Add tests for parts of the userapi storage

* Add tests for keybackup

* Add LoginToken tests

* Add OpenID tests

* Add profile tests

* Add pusher tests

* Add ThreePID tests

* Add notification tests

* Add more device tests, fix numeric localpart query

* Fix failing CI

* Fix numeric local part query
2022-04-27 15:05:49 +02:00