Commit graph

2255 commits

Author SHA1 Message Date
Till a7e92f8cb9
History visibility database changes (#2533)
* Add new history_visibility column

* Update SQL queries to include history_visibility

* Store the history visibilty calculated by the roomserver

* Update GMSL

* Update migrations

* Fix migration

* Update GMSL

* Fix `go.sum`

* Update GMSL to use sql.Scanner & sql.Valuer

* Re-order migration/table creation

* Update gomatrixserverlib

* Add history_visibility column to current_room_state

* Fix migrations

* Return error instead of Fatal log

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-07-18 14:46:15 +02:00
Maximilian Gaedig c0c909d306
Fix connection_string format in dendrite-sample.polylith.yaml (#2574) 2022-07-17 23:01:52 +01:00
andreever 4ff57993ab
Update 1_createusers.md (#2571)
* Update 1_createusers.md

Added description on how to create user accounts when running in docker.

* Update 1_createusers.md

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-07-15 16:33:05 +01:00
Neil Alexander 90bf01d8b1
Use sync API database in filterSharedUsers (#2572)
* Add function to the sync API storage package for filtering shared users

* Use the database instead of asking the RS API

* Fix unit tests

* Fix map handling in `filterSharedUsers`
2022-07-15 16:25:26 +01:00
Neil Alexander 69c86295f7
Update sytest-whitelist 2022-07-14 13:51:00 +01:00
Neil Alexander 950659320e
Add a Troubleshooting page 2022-07-14 10:09:26 +01:00
Neil Alexander a1f9b02edf
Pointerise types.RoomInfo in the cache so we can update it in-place in the latest events updater 2022-07-13 10:13:34 +01:00
Neil Alexander 9cd8e9d4b9
Use /v3 to request media from remote servers (update to matrix-org/gomatrixserverlib#324) 2022-07-13 09:32:03 +01:00
Till 09f0ff14c8
Minor SendToDevice fix (#2565)
* Avoid unnecessary marshalling if sending to the local server

* Fix ordering of ToDevice messages

* Revive SendToDevice test
2022-07-12 08:23:58 +02:00
Neil Alexander 3ea21273bc
Ristretto cache (#2563)
* Try Ristretto cache

* Tweak

* It's beautiful

* Update GMSL

* More strict keyable interface

* Fix that some more

* Make less panicky

* Don't enforce mutability checks for now

* Determine mutability using deep equality

* Tweaks

* Namespace keys

* Make federation caches mutable

* Update cost estimation, add metric

* Update GMSL

* Estimate cost for metrics better

* Reduce counters a bit

* Try caching events

* Some guards

* Try again

* Try this

* Use separate caches for hopefully better hash distribution

* Fix bug with admitting events into cache

* Try to fix bugs

* Check nil

* Try that again

* Preserve order jeezo this is messy

* thanks VS Code for doing exactly the wrong thing

* Try this again

* Be more specific

* aaaaargh

* One more time

* That might be better

* Stronger sorting

* Cache expiries, async publishing of EDUs

* Put it back

* Use a shared cache again

* Cost estimation fixes

* Update ristretto

* Reduce counters a bit

* Clean up a bit

* Update GMSL

* 1GB

* Configurable cache sizees

* Tweaks

* Add `config.DataUnit` for specifying friendly cache sizes

* Various tweaks

* Update GMSL

* Add back some lazy loading caching

* Include key in cost

* Include key in cost

* Tweak max age handling, config key name

* Only register prometheus metrics if requested

* Review comments @S7evinK

* Don't return errors when creating caches (it is better just to crash since otherwise we'll `nil`-pointer exception everywhere)

* Review comments

* Update sample configs

* Update GHA Workflow

* Update Complement images to Go 1.18

* Remove the cache test from the federation API as we no longer guarantee immediate cache admission

* Don't check the caches in the renewal test

* Possibly fix the upgrade tests

* Update to matrix-org/gomatrixserverlib#322

* Update documentation to refer to Go 1.18
2022-07-11 14:31:31 +01:00
Neil Alexander eb8dc50a97
Update Pinecone to matrix-org/pinecone@1ce778f 2022-07-08 14:54:08 +01:00
Till f76f28e6db
Fix issue uint64 values with high bit are not supported in presence (#2562)
* Fix issue #2528

* Use gomatrixserverlib.Timestamp

* Use ParseUint instead of ParseInt
2022-07-07 16:29:25 +02:00
Till f3e8a9a4cb
Fix nil pointer access when redacting events (#2560) 2022-07-07 11:40:53 +02:00
Neil Alexander 460dccf93d
Hopefully fix read receipts timestamps (#2557)
This should avoid coercions between signed and unsigned ints which might fix problems like `sql: converting argument $5 type: uint64 values with high bit set are not supported`.
2022-07-05 17:13:26 +01:00
Neil Alexander c0f824d437
Wrap error from SnapshotNIDFromEventID 2022-07-05 15:06:10 +01:00
Neil Alexander d4341a2d97
Return clearer error when no state NID exists for an event (#2555) 2022-07-05 15:01:34 +01:00
Till 5087b36af0
Fix QuerySharedUsers for the SyncAPI keychange consumer (#2554)
* Make more use of base.BaseDendrite

* Fix QuerySharedUsers if no UserIDs are supplied
2022-07-05 14:50:56 +02:00
Till f29cdb26f6
Use new testrig for key changes tests (#2552)
* Use new testrig for tests

* Log the error message
2022-07-05 14:50:24 +02:00
Kabir Kwatra 43147bd654
feat+fix: Ignore unknown keys and verify required fields are present in appservice registration files (#2550)
* fix: ignore unknown keys in appservice configs

fixes matrix-org/dendrite#1567

* feat: verify required fields in appservice configs
2022-07-05 12:53:51 +01:00
Neil Alexander b5c55faf98
Version 0.8.9 (#2549)
* Version 0.8.9

* Update changelog
2022-07-01 12:00:32 +01:00
Neil Alexander b50a24c666
Roomserver producers package (#2546)
* Give the roomserver a producers package

* Change init point

* Populate ACLs API

* Fix build issues

* `RoomEventProducer` naming
2022-07-01 10:54:07 +01:00
Till 89cd0e8fc1
Try to fix backfilling (#2548)
* Try to fix backfilling

* Return start/end to not confuse clients

* Update GMSL

* Update GMSL
2022-07-01 11:49:26 +02:00
Neil Alexander 086f182e24
Disable WebAssembly builds for now 2022-07-01 09:50:06 +01:00
Neil Alexander 54bed4c593
Blacklist Guest users can join guest_access rooms test until it can be investigated 2022-07-01 09:37:54 +01:00
Till 561c159ad7
Silence presence logs (#2547) 2022-06-30 12:34:37 +02:00
Neil Alexander 519bc1124b
Add evacuateUser endpoint, use it when deactivating accounts (#2545)
* Add `evacuateUser` endpoint, use it when deactivating accounts

* Populate the API

* Clean up user devices when deactivating

* Include invites, delete pushers
2022-06-29 15:29:39 +01:00
Neil Alexander 2dea466685
Return an error if trying to invite a malformed user ID (#2543) 2022-06-29 12:32:24 +01:00
Till 2086992caf
Don't return end if there are not more messages (#2542)
* Be more spec compliant

* Move lazyLoadMembers to own method
2022-06-29 10:49:12 +02:00
Jean Lucas 920a20821b
Fix nats.go commit (#2540)
Signed-off-by: Jean Lucas <jean@4ray.co>
2022-06-27 09:15:19 +01:00
Neil Alexander 7120eb6bc9
Add InputDeviceListUpdate to the keyserver, remove old input API (#2536)
* Add `InputDeviceListUpdate` to the keyserver, remove old input API

* Fix copyright

* Log more information when a device list update fails
2022-06-15 14:27:07 +01:00
Till 1b90cc9536
Fix rare panic when returning user devices over federation (#2534) 2022-06-15 12:50:02 +02:00
Neil Alexander 4c2a10f1a6
Handle state before, send history visibility in output (#2532)
* Check state before event

* Tweaks

* Refactor a bit, include in output events

* Don't waste time if soft failed either

* Tweak control flow, comments, use GMSL history visibility type
2022-06-13 15:11:10 +01:00
Emanuele Aliberti c500958583
generic CaddyFile in front of Dendrite (monolith) (#2531)
for Caddy 2.5.x

Co-authored-by: emanuele.aliberti <emanuele.aliberti@mtka.eu>
2022-06-13 12:08:46 +01:00
Till Faelligen e1136f4d3e Make the linter happy again 2022-06-13 11:46:59 +02:00
Neil Alexander 0a7f7dc716
Add --difference to resolve-state tool 2022-06-13 10:16:30 +01:00
Neil Alexander 89d2adadbd
Attempt to raise the file descriptor limit at startup (#2527) 2022-06-10 10:58:04 +01:00
Neil Alexander 1030072285
Rename the page to "Optimise your installation" 2022-06-10 10:18:32 +01:00
Neil Alexander 16ed1633b6
Highlighting in docs 2022-06-10 10:15:14 +01:00
Neil Alexander e2a64773ce
Add new next steps page to the documentation 2022-06-10 10:14:15 +01:00
Till 660f7839f5
Correctly redact events over federation (#2526)
* Ensure we check powerlevel/origin before redacting an event

* Add passing test

* Use pl.UserLevel

* Make check more readable, also check for the sender
2022-06-09 18:38:07 +02:00
Neil Alexander 83797573be
Version 0.8.8 (#2525) 2022-06-09 12:18:50 +01:00
Till 289b3c5608
Allow "registration is idempotent, with username specified" to pass (#2488)
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-06-09 11:26:48 +01:00
Neil Alexander 3cdefcf765
Update sample configs for exempt_user_ids 2022-06-08 09:13:25 +01:00
Neil Alexander c4df6d7723
Optimise state res v2 by only updating the allower when necessary (update to matrix-org/gomatrixserverlib@e55d796) 2022-06-07 15:34:58 +01:00
Neil Alexander b21a2223ef
Optimise state res v2 by parsing power level content less often (update to matrix-org/gomatrixserverlib@e938c07) 2022-06-07 15:04:00 +01:00
Neil Alexander 6d4bd5d890
Rate limiting changes (#2519)
* Rate limiting changes

This makes the following changes:

* For logged in users, the rate limiting now applies to the device session rather than the remote IP address;
* For non-logged in users, the rate limiting continues to apply to remote address as it does today;
* It is now possible to add user IDs to the `exempt_user_ids` option under `rate_limiting` to exclude bots from rate limiting;
* Admin and appservice users are now exempt from rate limiting by default.

* Fix build with media API
2022-06-07 14:24:04 +01:00
Neil Alexander 27948fb304
Optimise loadAuthEvents, add roomserver tracing 2022-06-07 14:23:26 +01:00
Neil Alexander aafb7bf120
Avoid panicking unnecessarily when shutting down the gobind P2P demos (#2520) 2022-06-07 10:46:21 +01:00
Neil Alexander 0d7020fbaf
Send tombstone to other servers when upgrading rooms 2022-06-06 17:27:50 +01:00
Neil Alexander 2cb609c428
Room upgrade tweaks
Squashed commit of the following:

commit 7a1568c716866594af6d0b1d561c58c96de29b20
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Mon Jun 6 15:17:49 2022 +0100

    Make errors more useful

commit 64befe7c9a901b00650442171660c2dc4ea575fa
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Mon Jun 6 15:02:40 2022 +0100

    Tweak ordering a bit
2022-06-06 15:18:02 +01:00