Commit graph

2210 commits

Author SHA1 Message Date
kegsay a69066ed7c
Merge branch 'main' into s7evink/hisvismessages 2022-07-29 10:53:32 +01:00
Neil Alexander 962b76da44
Make the User API responsible for sending account data output events (#2592)
* Make the User API responsible for sending account data output events

* Clean up producer

* Review comments
2022-07-25 17:30:07 +01:00
Neil Alexander 7490a50a4e
Fix imports 2022-07-25 15:19:42 +01:00
Neil Alexander ab3f8a5cd1
Merge branch 'main' into s7evink/hisvismessages 2022-07-25 15:17:54 +01:00
Neil Alexander 497ab4e1b7
Remove goose from Dockerfiles 2022-07-25 14:46:35 +01:00
Neil Alexander ccea23cd40
Remove room_id field from MSC2946 stripped events (closes #2588) 2022-07-25 11:42:22 +01:00
Jonathan Bartlett b836243a24
Add .well-known/matrix/client to clientapi (#2551)
Signed-off-by: Jonathan Bartlett <jonathan@jonnobrow.co.uk>

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-07-25 10:39:57 +01:00
Till 081f5e7226
Update database migrations, remove goose (#2264)
* Add new db migration

* Update migrations
Remove goose

* Add possibility to test direct upgrades

* Try to fix WASM test

* Add checks for specific migrations

* Remove AddMigration
Use WithTransaction
Add Dendrite version to table

* Fix linter issues

* Update tests

* Update comments, outdent if

* Namespace migrations

* Add direct upgrade tests, skipping over one version

* Split migrations

* Update go version in CI

* Fix copy&paste mistake

* Use contexts in migrations

Co-authored-by: kegsay <kegan@matrix.org>
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-07-25 10:39:22 +01:00
Neil Alexander c7d978274d
Try to fix HTTP 500s on /members (#2581) 2022-07-22 19:43:48 +01:00
Neil Alexander f0c8a03649
Membership updater refactoring (#2541)
* Membership updater refactoring

* Pass in membership state

* Use membership check rather than referring to state directly

* Delete irrelevant membership states

* We don't need the leave event after all

* Tweaks

* Put a log entry in that I might stand a chance of finding

* Be less panicky

* Tweak invite handling

* Don't freak if we can't find the event NID

* Use event NID from `types.Event`

* Clean up

* Better invite handling

* Placate the almighty linter

* Blacklist a Sytest which is otherwise fine under Complement for reasons I don't understand

* Fix the sytest after all (thanks @S7evinK for the spot)
2022-07-22 14:44:04 +01:00
Till Faelligen e30565c029
Try to optimize database requests 2022-07-22 12:27:36 +02:00
dependabot[bot] a201b4400d
Bump tzinfo from 1.2.9 to 1.2.10 in /docs (#2584)
Bumps [tzinfo](https://github.com/tzinfo/tzinfo) from 1.2.9 to 1.2.10.
- [Release notes](https://github.com/tzinfo/tzinfo/releases)
- [Changelog](https://github.com/tzinfo/tzinfo/blob/master/CHANGES.md)
- [Commits](https://github.com/tzinfo/tzinfo/compare/v1.2.9...v1.2.10)

---
updated-dependencies:
- dependency-name: tzinfo
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2022-07-22 10:36:10 +01:00
Till Faelligen 53885048a8
More cleanup 2022-07-22 09:12:01 +02:00
Till Faelligen 17fee30c0e
Remove noise/unneeded code 2022-07-22 09:06:24 +02:00
Till Faelligen 7cd8f40b48
Revert "Move queries to topology table"
This reverts commit 4f0d41be9c.
2022-07-22 08:59:35 +02:00
Till Faelligen bbbf1036b7
Use actual internal rsAPI, default to shared visibility in tests 2022-07-22 08:32:12 +02:00
Till Faelligen 2502e66592
Tweaks and logging 2022-07-22 07:32:29 +02:00
Till Faelligen e24dcaa205
Add method to fetch the membership at a given event 2022-07-22 07:15:34 +02:00
Till Faelligen bde6f67932
Merge branch 'main' of github.com:matrix-org/dendrite into s7evink/hisvismessages 2022-07-21 09:04:06 +02:00
Matt Holt 35ce551c8f
docs: Add build page; correct proxy info; fix Caddy example (#2579)
* Add build page; correct proxy info; fix Caddy example

* Improve Caddyfile example

* Apply review comments; add polylith Caddyfile
2022-07-21 07:08:17 +02:00
Till Faelligen 6d7fd13c82
Merge branch 'main' of github.com:matrix-org/dendrite into s7evink/hisvismessages 2022-07-20 13:18:50 +02:00
Till 9507966ebd
Fix issue with membership event_nid being 0 (#2580) 2022-07-20 12:39:06 +02:00
Tulir Asokan 84a7797883
Explain how SRV works in Matrix and discourage using it (#2577)
* Explain how SRV works in Matrix and discourage using it

* Minor tweaks to formatting

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-07-19 15:19:03 +01:00
Till Faelligen 503afceccb Update migrations to set "correct" history visibility 2022-07-19 13:30:43 +02:00
Neil Alexander 5c01306bb5
Add event state key cache (#2576) 2022-07-19 12:15:48 +01:00
Neil Alexander 583b8ea273
Update FAQ 2022-07-19 11:51:46 +01:00
Till Faelligen 56d8a2371f Remove unused statement 2022-07-19 12:41:54 +02:00
Till Faelligen 910bd9b4a8 Update history visibility checks to use gmsl
Update tests
2022-07-19 12:37:16 +02:00
Till Faelligen 1c43dbcc12 Remove unused query 2022-07-19 12:36:23 +02:00
Till Faelligen 874489544b Default to shared history visibility 2022-07-19 12:28:47 +02:00
Till Faelligen b540dfa48d Set the visibility when creating events 2022-07-19 12:28:23 +02:00
Till Faelligen ab9e4353ee Update gomatrixserverlib 2022-07-19 12:28:05 +02:00
Till Faelligen 9b6b6585c2 Merge branch 'main' of github.com:matrix-org/dendrite into s7evink/hisvismessages 2022-07-18 18:23:02 +02:00
Till Faelligen bcff14adea Set historyVisibility in rowsToStreamEvents 2022-07-18 18:19:44 +02:00
Neil Alexander 99b696e775
Tweak cache counters (#2575)
* Tweak cache counters

This makes the number of counters relative to the
maximum cache size. Since the counters
effectively manage the size of the bloom filter,
larger caches need more counters and smaller
caches need less.

10 counters per 1KB data means that the default
cache size of 1GB should result in a bloom filter
and TinyLRU admission set of about 16MB
estimated.

* Remove line left by accident
2022-07-18 14:49:41 +01:00
Till Faelligen 993496db9e Merge branch 'main' of github.com:matrix-org/dendrite into s7evink/hisvismessages 2022-07-18 14:47:16 +02:00
Till a7e92f8cb9
History visibility database changes (#2533)
* Add new history_visibility column

* Update SQL queries to include history_visibility

* Store the history visibilty calculated by the roomserver

* Update GMSL

* Update migrations

* Fix migration

* Update GMSL

* Fix `go.sum`

* Update GMSL to use sql.Scanner & sql.Valuer

* Re-order migration/table creation

* Update gomatrixserverlib

* Add history_visibility column to current_room_state

* Fix migrations

* Return error instead of Fatal log

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-07-18 14:46:15 +02:00
Maximilian Gaedig c0c909d306
Fix connection_string format in dendrite-sample.polylith.yaml (#2574) 2022-07-17 23:01:52 +01:00
andreever 4ff57993ab
Update 1_createusers.md (#2571)
* Update 1_createusers.md

Added description on how to create user accounts when running in docker.

* Update 1_createusers.md

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-07-15 16:33:05 +01:00
Neil Alexander 90bf01d8b1
Use sync API database in filterSharedUsers (#2572)
* Add function to the sync API storage package for filtering shared users

* Use the database instead of asking the RS API

* Fix unit tests

* Fix map handling in `filterSharedUsers`
2022-07-15 16:25:26 +01:00
Neil Alexander 69c86295f7
Update sytest-whitelist 2022-07-14 13:51:00 +01:00
Neil Alexander 950659320e
Add a Troubleshooting page 2022-07-14 10:09:26 +01:00
Neil Alexander a1f9b02edf
Pointerise types.RoomInfo in the cache so we can update it in-place in the latest events updater 2022-07-13 10:13:34 +01:00
Neil Alexander 9cd8e9d4b9
Use /v3 to request media from remote servers (update to matrix-org/gomatrixserverlib#324) 2022-07-13 09:32:03 +01:00
Till 09f0ff14c8
Minor SendToDevice fix (#2565)
* Avoid unnecessary marshalling if sending to the local server

* Fix ordering of ToDevice messages

* Revive SendToDevice test
2022-07-12 08:23:58 +02:00
Neil Alexander 3ea21273bc
Ristretto cache (#2563)
* Try Ristretto cache

* Tweak

* It's beautiful

* Update GMSL

* More strict keyable interface

* Fix that some more

* Make less panicky

* Don't enforce mutability checks for now

* Determine mutability using deep equality

* Tweaks

* Namespace keys

* Make federation caches mutable

* Update cost estimation, add metric

* Update GMSL

* Estimate cost for metrics better

* Reduce counters a bit

* Try caching events

* Some guards

* Try again

* Try this

* Use separate caches for hopefully better hash distribution

* Fix bug with admitting events into cache

* Try to fix bugs

* Check nil

* Try that again

* Preserve order jeezo this is messy

* thanks VS Code for doing exactly the wrong thing

* Try this again

* Be more specific

* aaaaargh

* One more time

* That might be better

* Stronger sorting

* Cache expiries, async publishing of EDUs

* Put it back

* Use a shared cache again

* Cost estimation fixes

* Update ristretto

* Reduce counters a bit

* Clean up a bit

* Update GMSL

* 1GB

* Configurable cache sizees

* Tweaks

* Add `config.DataUnit` for specifying friendly cache sizes

* Various tweaks

* Update GMSL

* Add back some lazy loading caching

* Include key in cost

* Include key in cost

* Tweak max age handling, config key name

* Only register prometheus metrics if requested

* Review comments @S7evinK

* Don't return errors when creating caches (it is better just to crash since otherwise we'll `nil`-pointer exception everywhere)

* Review comments

* Update sample configs

* Update GHA Workflow

* Update Complement images to Go 1.18

* Remove the cache test from the federation API as we no longer guarantee immediate cache admission

* Don't check the caches in the renewal test

* Possibly fix the upgrade tests

* Update to matrix-org/gomatrixserverlib#322

* Update documentation to refer to Go 1.18
2022-07-11 14:31:31 +01:00
Neil Alexander eb8dc50a97
Update Pinecone to matrix-org/pinecone@1ce778f 2022-07-08 14:54:08 +01:00
Till f76f28e6db
Fix issue uint64 values with high bit are not supported in presence (#2562)
* Fix issue #2528

* Use gomatrixserverlib.Timestamp

* Use ParseUint instead of ParseInt
2022-07-07 16:29:25 +02:00
Till f3e8a9a4cb
Fix nil pointer access when redacting events (#2560) 2022-07-07 11:40:53 +02:00
Neil Alexander 460dccf93d
Hopefully fix read receipts timestamps (#2557)
This should avoid coercions between signed and unsigned ints which might fix problems like `sql: converting argument $5 type: uint64 values with high bit set are not supported`.
2022-07-05 17:13:26 +01:00