Commit graph

2361 commits

Author SHA1 Message Date
Neil Alexander 4417f24678
Protect processEventWithMissingState with per-room mutex, to prevent mass CPU burn/RAM usage
Squashed commit of the following:

commit 7fad77c10e3c1c78feddb37351812b209d9c0f25
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Mon Jun 28 15:06:52 2021 +0100

    Fix processEventWithMissingStateMutexes

commit 138cddcac7b8373a8e1816a232f84a7bda6adcdf
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Mon Jun 28 13:59:44 2021 +0100

    Use internal.MutexByRoom

commit 6e6f026cfad31da391ad261cfec16d41dff1b15b
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Mon Jun 28 13:50:18 2021 +0100

    Try to slow things down per room

commit b97d406dff2e11769a9202fbf58b138a541ca449
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Mon Jun 28 13:41:27 2021 +0100

    Try to slow things down

commit 8866120ebf880b4fd8a456937f69903e233c19a2
Merge: 9f2de8a2 4a37b19a
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Mon Jun 28 13:40:33 2021 +0100

    Merge branch 'neilalexander/rsinputfifo' into neilalexander/rsinputfifo2

commit 4a37b19a8f
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Mon Jun 28 13:34:54 2021 +0100

    Add comments

commit f9ab3f4b81
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Mon Jun 28 13:31:21 2021 +0100

    Tweaks

commit 9f2de8a29cadec4c785d9c2e4e74c1138305f759
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Mon Jun 28 13:15:59 2021 +0100

    Ask origin only for missing things for now

commit 8fd878c75a
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Mon Jun 28 11:18:11 2021 +0100

    Make sure someone wakes up

commit b63f699f1b
Author: Neil Alexander <neilalexander@users.noreply.github.com>
Date:   Mon Jun 28 11:12:58 2021 +0100

    Use a FIFO queue instead of a channel to reduce backpressure
2021-06-28 15:11:59 +01:00
Neil Alexander 7c3991ee2f
Use a custom FIFO queue for the RS input API (#1888)
* Use a FIFO queue instead of a channel to reduce backpressure

* Make sure someone wakes up

* Tweaks

* Add comments
2021-06-28 15:11:36 +01:00
Neil Alexander a6f7e83596
Fix bugs in P2P demos 2021-06-25 15:59:59 +01:00
Neil Alexander bd7b0b1787
Update go.mod/go.sum 2021-06-25 15:13:21 +01:00
Neil Alexander 097a5833c8
dendritejs-pinecone 2021-06-25 14:20:36 +01:00
Neil Alexander 6ae976a55a
dendrite-demo-pinecone: Accept any origin 2021-06-22 15:39:29 +01:00
database64128 79181f2348
✂️ Media API: Handle unlimited file size (#1881) 2021-06-22 10:01:44 +01:00
Neil Alexander 5357df36c9
Fix panic in roomserver 2021-06-21 09:41:12 +01:00
Neil Alexander 3a77bca28b
Revert "Ensure user IDs match the spec (matrix-org/gomatrixserverlib#261)"
This reverts commit 30e9353fdd.
2021-06-16 09:07:41 +01:00
Neil Alexander 30e9353fdd
Ensure user IDs match the spec (matrix-org/gomatrixserverlib#261) 2021-06-15 15:18:56 +01:00
Neil Alexander 5ec7d3a3ec
Update go.mod/go.sum 2021-06-14 15:04:05 +01:00
S7evinK 9ed0440421
Set MaxFileSizeBytes <= 0 to "unlimited" (#1875)
* Set MaxFileSizeBytes < 0 to "unlimited"

Signed-off-by: Till Faelligen <tfaelligen@gmail.com>

* int64 overflows later in mediaapi/routing/upload.go[doUpload]

* Prevent int overflow when uploading
2021-06-14 14:12:03 +01:00
Arnaud Venturi 9633ed747d
Remove mention of enabling naffka in install instructions, as it is the default value (#1853)
Signed-off-by: Arnaud Venturi <git@toadjaune.eu>
2021-06-14 14:09:35 +01:00
Ben Yanke ce574f5e29
Fix Key Generation Docs - resolves #1759 (#1865)
This fixes the issue found in #1759 which broke due to go changes. The new command allows you to generate keys with docker, and drop them in the current working directory.
2021-06-14 14:08:59 +01:00
S7evinK 183f43d38d
Add parameters to specify password (#1868)
* Add parameters to specify password

Signed-off-by: Till Faelligen <tfaelligen@gmail.com>

* Fix typo

* Add testdata

* Use go1.13 compatible way to read files
2021-06-14 14:08:29 +01:00
Adam Greig eae1c46310
Use NotFound instead of Forbidden for missing account data (#1872)
Signed-off-by: Adam Greig <adam@adamgreig.com>
2021-06-14 14:06:14 +01:00
Neil Alexander 051cd63f1e
Update go.mod/go.sum 2021-06-14 13:26:02 +01:00
Neil Alexander bd9dec8e06
Pinecone demo updates 2021-06-14 13:13:07 +01:00
Rasmus Thomsen 2c9a390fa6
Add CORS to nginx config (#1791)
Without this entry, setups where users have the homeserver on the URL
matrix.myurl.com but want the servername to be myurl.com don't work by default
since clients like element.io can't connect to the homeserver
2021-06-07 11:06:57 +01:00
Ben Langfeld caa5c89ecb
Link to Synapse (#1863)
Not everyone is very familiar.
2021-06-07 10:10:29 +01:00
Arnaud Venturi 6fa5743ea0
Add missing IPv6 "ssl" keyword in nginx config examples (#1854)
Signed-off-by: Arnaud Venturi <git@toadjaune.eu>
2021-06-07 09:29:54 +01:00
S7evinK 89a6787fdb
Try to optimize SelectOneTimeKeys (#1851)
* Try to optimize SelectOneTimeKeys

Signed-off-by: Till Faelligen <tfaelligen@gmail.com>

* Use pg.Array when using ANY...

Co-authored-by: Kegsay <kegan@matrix.org>
2021-06-07 09:17:46 +01:00
S7evinK 8b22c4270d
Use LimitReader to prevent DoS risk (#1843)
* Use LimitReader to prevent DoS risk

Signed-off-by: Till Faelligen <tfaelligen@gmail.com>

* Check if bytesWritten is equal to the maxFileSize
Add tests

Signed-off-by: Till Faelligen <tfaelligen@gmail.com>

* Use oldschool defer to cleanup after the tests

* Let LimitReader read MaxFileSizeBytes + 1

Co-authored-by: Kegsay <kegan@matrix.org>
2021-06-07 09:17:20 +01:00
bodqhrohro c488d3db75
Fix SIGSEGV in IsInterestedInRoomID (#1846)
* Avoid crash on non-compiled room regex

Roughly fixes #1845 (actual compiling still needed)

Signed-off-by: Bohdan Horbeshko <bodqhrohro@gmail.com>

* Compile regexes for all namespaces

Deadheres the regex compiling from building larger regexes for possibly
exclusive namespaces only. A complete fix for #1845, so regexes for
rooms namespaces and other non-whitelisted namespaces can be used
more safely.

Signed-off-by: Bohdan Horbeshko <bodqhrohro@gmail.com>

* Appservice config: handle regexp parsing errors

Signed-off-by: diamondburned <datutbrus@gmail.com>
Signed-off-by: Bohdan Horbeshko <bodqhrohro@gmail.com>

Co-authored-by: Kegsay <kegan@matrix.org>
2021-06-07 09:13:40 +01:00
bodqhrohro f18001ecbb
Split the select+update query for txn_id counter (#1855)
The update part wasn't executed actually for SQLite, so it is moved to
a separate statement.

Fixes #1852.

Signed-off-by: Bohdan Horbeshko <bodqhrohro@gmail.com>
2021-06-07 08:51:19 +01:00
Neil Alexander b0aa101dcd
Update go.mod/go.sum for matrix-org/pinecone (Build 79) 2021-06-02 12:23:01 +01:00
Neil Alexander 3797d818c0
Update gomatrixserverlib to matrix-org/gomatrixserverlib#259 2021-05-25 12:01:07 +01:00
Neil Alexander 3f84cd6cd0
Update go.mod/go.sum 2021-05-25 09:52:55 +01:00
Neil Alexander 30f021700a
Update go.mod/go.sum 2021-05-25 09:49:02 +01:00
Neil Alexander a7f2845a6a
Demo tweaks 2021-05-24 13:12:05 +01:00
Neil Alexander 79c5485c8d
Allow clearing federation blacklist at startup for P2P demos 2021-05-24 11:43:24 +01:00
Neil Alexander d2d2164025
Update pinecone demo 2021-05-24 11:17:32 +01:00
Neil Alexander 2948ffd782
Update go.mod/go.sum 2021-05-18 16:20:04 +01:00
Neil Alexander 08a4370c94
Update go.mod/go.sum 2021-05-18 15:58:50 +01:00
Neil Alexander e9e0309089
Merge branch 'master' of github.com:matrix-org/dendrite 2021-05-18 15:47:23 +01:00
Neil Alexander 140cae81cc
Update dendrite-demo-pinecone 2021-05-18 15:47:15 +01:00
Michael Telatynski 4691adc8f8
Update MSC2946 implementation for stable spaces (#1859)
Now that MSC1772 passed FCP its identifiers have stabilised
This outright drops support for experimental spaces but that's what you get for being on the bleeding edge
2021-05-18 15:17:37 +01:00
Neil Alexander 1d89c4a3cd
Update go.mod/go.sum 2021-05-10 17:04:06 +01:00
Neil Alexander c76f820c86
Update go.mod/go.sum 2021-05-10 17:03:02 +01:00
Neil Alexander a49d06138e
Updates to dendrite-demo-pinecone 2021-05-10 16:59:03 +01:00
Neil Alexander 81d60d5448
Update room directory in Pinecone demo some more 2021-05-07 12:48:30 +01:00
Neil Alexander 603bf590f0
Fix public room directory in Pinecone demo 2021-05-07 12:17:14 +01:00
Neil Alexander aa672068ab
Update pinecone in go.mod/go.sum 2021-05-07 11:49:44 +01:00
Neil Alexander 1002e87b60
Pinecone P2P demo (#1856)
* Pinecone demo

* Enable multicast, fix HTTP routing

* Fix multicast import

* Fix build

* Update Pinecone demo

* Fix the keys

* Tweaks

* Pinecone room directory support (early)

* Fix gobind-pinecone

* Add pinecone listener

* Fix public key value

* Use AuthenticatedConnect for dial

* Fix gobind-pinecone

* Stop panics

* Give fsAPI to keyserver

* Pinecone demo fixes

* Update gobind build scripts

* Account creation

* Tweaks

* Setup tweaks

* API tweaks

* API tweaks

* API tweaks

* Port mutex

* Re-enable multicast

* Add ReadCopy

* Update quic-go, fixes

* Shutdowns fixed for iOS

* Update build script

* Add WebSocket support

* Bug fixes

* Netconn context

* Fix WebSocket connectivity

* Fixes to gobind API

* Strip frameworks

* Configurability updates

* Update go.mod

* Update go.mod/go.sum

* Update go.mod/go.sum

* Update go.mod/go.sum

* Try to stay connected tto static peer

* Update gobind-pinecone

* Update go.mod/go.sum

* Test uTP+TLS

* Use HTTP/2

* Don't use HTTP/2

* Update go.mod/go.sum

* Attempt to reconnect to the static peer if it drops

* Stay connected to static peers more stickily

* Retry room directory lookups if they fail

* NewQUIC -> NewSessions

* Storage updates

* Don't return immediately when there's nothing to sync

* Updates

* Try to reconnect to static peer more

* Update go.mod/go.sum

* Require Go 1.14

* Update go.mod/go.sum

* Update go.mod/go.sum
2021-05-06 12:00:42 +01:00
Neil Alexander 464b908bd0
Don't return immediately when there's nothing to sync 2021-04-26 16:33:42 +01:00
Neil Alexander c67d8da3eb
Fix bug in SQLite migration 2021-04-26 13:45:47 +01:00
Neil Alexander 5ce1fe80de
State storage refactor (#1839)
* Hash-deduplicated state storage (and migrations) for PostgreSQL and SQLite

* Refactor droomserver database setup for migrations

* Fix conflict statements

* Update migration names

* Set a boundary for old to new block/snapshot IDs so we don't rewrite them more than once accidentally

* Create sequence if not exists

* Fix boundary queries

* Fix boundary queries

* Use Query

* Break out queries a bit

* More sequence tweaks

* Query parameters are not playing the game

* Injection escaping may not work for CREATE SEQUENCE after all

* Fix snapshot sequence name

* Use boundaried IDs in SQLite too

* Use IFNULL for SQLite

* Use COALESCE in PostgreSQL

* Review comments @Kegsay
2021-04-26 13:25:57 +01:00
Fero d6e9b7b307
Remove the 'Content-Type' request header requirement (#1834) 2021-04-19 16:29:51 +01:00
Neil Alexander a9faa1bc44
Fix registration error when disabled 2021-04-15 09:58:26 +01:00
Kegsay 656d11ec90
fedsender: tolerate dupe membership events (#1824)
* fedsender: tolerate dupe membership events

Previously if the fedsender got a duplicate membership event it would cause
the entire process to crash. Now it doesn't. This masks an issue with the
roomserver where it can emit duplicate membership events.

* Update joined_hosts_table.go
2021-04-14 11:11:54 +01:00