Commit graph

2337 commits

Author SHA1 Message Date
kegsay 7499147550
Add test infrastructure code for dendrite unit/integ tests (#2331)
* Add test infrastructure code for dendrite unit/integ tests

Start re-enabling some syncapi storage tests in the process.

* Linting

* Add postgres service to unit tests

* dendrite not syncv3

* Skip test which doesn't work

* Linting

* Add `jetstream.PrepareForTests`

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-04-08 10:12:30 +01:00
Neil Alexander 955e6eb307
Dendrite 0.8.1 2022-04-07 17:13:14 +01:00
Neil Alexander 7902859bc1
Fix lock contention in sync notifier 2022-04-07 17:08:15 +01:00
Neil Alexander 87b1ed1338
Version 0.8.0 (#2330)
* Version 0.8.0

* Update changelog again
2022-04-07 16:09:40 +01:00
Neil Alexander 2dc35403d1
Send avatar updates asynchronously, same as display name updates 2022-04-07 15:50:42 +01:00
Till 60ee7eef4c
Add possibility to ignore users (#2329)
* Add ignore users

* Ignore users in pushrules
Add passing tests

* Update sytest lists

* Store ignore knowledge in the sync API

* Fix copyrights

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-04-07 15:08:19 +01:00
Neil Alexander 99ef547295
Simplify presence stringification (should help with vector-im/element-android#5712) 2022-04-07 10:10:28 +01:00
Neil Alexander 99f2007eb6
Fix bug where state events could be requested twice in appservices (#2328)
* Fix bug where state events could be requested twice (should fix #2305)

* Avoid a pointless HTTP request if possible
2022-04-06 16:10:31 +01:00
Neil Alexander 850932c7a7
Update README.md 2022-04-06 15:23:09 +01:00
Neil Alexander c28f00bf96
Sync notifier tweaks (#2327)
* Micro-optimisations, lock fixes

* Refactor `SharedUsers`

* Reuse map to reduce allocations/GC pressure

* oh yeah, initialise it

* Leave room for the user ID we'll no doubt append afterward
2022-04-06 15:04:41 +01:00
Neil Alexander 602818460d
Reduce allocations in /sync presence stream (#2326)
* Reduce allocations on presence

* Try to reduce allocations further

* Tweak `IsSharedUser` some more

* Take map lock
2022-04-06 13:31:44 +01:00
Till e5e3350ce1
Add presence module V2 (#2312)
* Syncapi presence

* Clientapi http presence handler

* Why is this here?

* Missing files

* FederationAPI presence implementation

* Add new presence stream

* Pinecone update

* Pinecone update

* Add passing tests

* Make linter happy

* Add presence producer

* Add presence config option

* Set user to unavailable after x minutes

* Only set currently_active if online
Avoid unneeded presence updates when syncing

* Tweaks

* Query devices for last_active_ts
Fixes & tweaks

* Export SharedUsers/SharedUsers

* Presence stream in MemoryStorage

* Remove status_msg_nil

* Fix sytest crashes

* Make presence types const and use stringer for it

* Change options to allow inbound/outbound presence

* Fix option & typo

* Update configs

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-04-06 13:11:19 +02:00
Neil Alexander 16e2d243fc
Reduce the number of allocations made by localRoomMembers when consuming stream events for push notifications (#2324) 2022-04-06 10:43:54 +01:00
Neil Alexander f7109de500
Nuke a couple of tests for now 2022-04-05 16:45:01 +01:00
Neil Alexander b49a6757c8
Update the "Are We Synapse Yet?" list 2022-04-05 16:43:29 +01:00
Neil Alexander c874d0b2ce
Merge branch 'main' of github.com:matrix-org/dendrite 2022-04-05 14:43:58 +01:00
Neil Alexander 4d9d9cc9b1
Update to matrix-org/gomatrixserverlib#300 2022-04-05 14:43:44 +01:00
Till c84937b852
Add Are We Synapse Yet to GHA (#2321)
* Add Are We Synapse Yet to GHA

* Better output & add comments
2022-04-05 15:32:30 +02:00
Neil Alexander 47be39c18e
Update sytest-whitelist again 2022-04-05 11:49:29 +01:00
Emanuele Aliberti fb10633a80
build.sh to build.cmd (#2319)
* build.sh to build.cmd

convert and adapt for Windows users

* remove hardwired GO ARCH

* silence cmd.exe

* update directions for Windows

* Update INSTALL.md

Co-authored-by: emanuele.aliberti <emanuele.aliberti@mtka.eu>
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-04-05 11:27:29 +01:00
David Spenler 2defc4249d
Added /upgrade endpoint (#2307)
* Added /upgrade endpoint

* fix

* Fix lints

* More lint lifex

* Move room upgrading to the roomserver

* Remove extraneous arg

* Fix HTTP API for `PerformUpgrade`

* Reduce number of API calls in `generateInitialEvents`, preserve membership fields

* Refactor `generateInitialEvents` to preserve old state events for all but the essential room setup events

* Handle ban events in the state transfer

* Refactor and comment `createTemporaryPowerLevels`

* Only send two power levels if we needed to override the levels, preserve miscellaneous fields in the create event

* Fix copyrights

* Review comments @S7evinK

* Update sytest whitelist

* Specify empty state keys, use `EventLevel`, remove unnecessary check on state copy

* Add comment to `restrictOldRoomPowerLevels`

* Ensure canonical aliases exist before clearing

* Copy invites as well as bans

* Fix return error on `m.room.tombstone` handling in client API

* Relax checks for well-formedness of join rules, membership event etc

Co-authored-by: Alex Kursell <alex@awk.run>
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
Co-authored-by: kegsay <kegan@matrix.org>
2022-04-05 10:04:08 +01:00
Neil Alexander 562d742240
Update to matrix-org/gomatrixserverlib#299 2022-04-04 19:04:17 +01:00
Sebastian Spaeth 1783496423
remove obsolete config entry "federation_certificates" (#2318)
* Remove all federation_certificates occurencs

This configuration value has not been used since 2019 apparently, and indeed
it is never really used in the code base. So remove all traces of it from
the various configuration files.

Also remove the unused variable FederationCertificatePaths

Signed-off-by: Sebastian Spaeth <Sebastian@SSpaeth.de>

* setup/config/config_test.go: remove federation_sender config snippet

The federation_sender: section was folded into the federation_api some
time ago, and this seems to be the only leftover in the code base.
So remove it.
2022-04-04 17:31:41 +01:00
Neil Alexander f2f9d2916b
Fix dendritejs-pinecone build 2022-04-04 17:26:01 +01:00
Neil Alexander cee12a7ab0
Enhanced calm at startup of Pinecone demos 2022-04-04 15:35:47 +01:00
Neil Alexander c69159bda7
Update to matrix-org/pinecone@e526fa8 2022-04-04 15:16:02 +01:00
Neil Alexander 9b316ac64c
Slower federation warm-up (#2320)
* Wake destination queues gradually, rather than all at once

* Delay device list updates too

* Maximum two minute warmup period
2022-04-04 15:14:10 +01:00
Neil Alexander 6748a2a823
Fix sign-off link in CONTRIBUTING.md 2022-04-04 11:02:16 +01:00
S7evinK a0f5d8e1a5
Fix unlimitedSize uploads (#2317) 2022-04-04 10:32:53 +02:00
Neil Alexander cd8fac152e
Include joined and invite member counts in room summary (#2315)
* Include joined and invite member counts in room summary

This should fix #2314 and also fix the problem where some clients like Element Android, Fluffychat etc would display the wrong member count for a given room.

* Improve SQLite query precision

* Check existence of state key for membership events
2022-04-01 16:14:38 +01:00
Neil Alexander 8213b2ba30
Update Pinecone P2P demo 2022-03-30 15:01:22 +01:00
S7evinK 49dc49b232
Remove eduserver (#2306)
* Move receipt sending to own JetStream producer

* Move SendToDevice to producer

* Remove most parts of the EDU server

* Fix SendToDevice & copyrights

* Move structs, cleanup EDU Server traces

* Use HeadersOnly subscription

* Missing file

* Fix linter issues

* Move consumers to own files

* Rename durable consumer; Consumer cleanup

* Docs/config cleanup
2022-03-29 14:14:35 +02:00
Neil Alexander 7972915806
User directory for nearby Pinecone peers (P2P demo) (#2311)
* User directory for nearby Pinecone peers

* Fix mux routing

* Use config to determine which server notices user to exclude
2022-03-28 16:25:26 +01:00
Neil Alexander 0692be44d9
Fix account availability on register 2022-03-28 13:31:17 +01:00
Neil Alexander 8099bcbc8b
P2P demo tweaks 2022-03-28 11:44:25 +01:00
Neil Alexander 34b9c8c670
Ensure Dendrite has stopped in Pinecone demo Stop() 2022-03-28 10:55:53 +01:00
Neil Alexander ceb3874469
Allow stored session parameters to be overwritten in the registration request (#2309)
* Allow stored session parameters to be overwritten in the registration request

* Remove logging

* Close request body

* Use `httputil.UnmarshalJSON` as that should enforce UTF-8 correctness

* Return `M_NOT_JSON` on read error

* Whoops, return the value of `httputil.UnmarshalJSON`

* Remove redundant comment
2022-03-28 10:47:35 +01:00
Neil Alexander 08d995d809
Version 0.7.0 (#2304) 2022-03-25 14:53:06 +00:00
Neil Alexander 1554d51b37
Pass DOCKER_TOKEN secret into Docker workflow 2022-03-25 14:27:41 +00:00
Neil Alexander 565b5423ea
One final tweak to the GHA pipeline 2022-03-25 13:41:28 +00:00
S7evinK 873c4d7e2c
Fixes for create-account (#2285)
* Check user existence
Fallback to asking for the password if non is defined

* Add missing tests

* Update to not use pointers, verify username length

* Re-add possibilty to create passwordless account

* Fix config issue

* Fix test again

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-03-25 14:38:24 +01:00
Neil Alexander 5b5e6a59b6
Give packages permission to update-docker-images 2022-03-25 13:31:41 +00:00
Neil Alexander 7f3d42bb46
Use correct path name 2022-03-25 13:29:09 +00:00
Neil Alexander 62bd559275
Factor Docker step into own job 2022-03-25 13:28:20 +00:00
Neil Alexander c8e1ad5997
Specify branch name in Docker flow 2022-03-25 13:26:18 +00:00
Neil Alexander f25afa1e19
Healthcheck endpoints (#2303)
* Health monitoring endpoints

* Rename endpoints
2022-03-25 13:25:15 +00:00
Neil Alexander 28642683fc
Fix Docker flow 2022-03-25 13:25:02 +00:00
Neil Alexander b113217a6d
Use most recent event in response to get latest stream position in incremental sync (#2302)
* Use latest event position in response for advancing the stream position in an incremental sync

* Create some calm

* Use To in worst case

* Don't waste CPU cycles on an empty response after all

* Bug fixes

* Fix another bug
2022-03-25 12:38:16 +00:00
Neil Alexander e6d4bdeed5
Try to recover from corrupted NATS streams in memory temporarily (#2301) 2022-03-25 12:24:21 +00:00
Neil Alexander 5e780d3ca2
Chain Docker update onto main 2022-03-25 10:08:13 +00:00