Commit graph

2082 commits

Author SHA1 Message Date
Tommie Gannert 46b4abd4cf golangci-lint fixes. 2022-05-25 19:05:16 +02:00
Tommie Gannert 6de730b2ee Fixes for SSO.
* Verbose logging.
* Cookie needs a path.
* Configurable callback URL.
* Various sanity checks.
2022-05-25 18:33:11 +02:00
Tommie Gannert 73e83c2b51 Fix SSO table initialization. 2022-05-25 18:32:03 +02:00
Tommie Gannert 58233c790a Split SSO into OAuth2 and OIDC. Add OIDC discovery.
GitHub implements OAuth2, but not OpenID Connect.

This means it needs more magic constants than those that can do OIDC
discovery (and where Userinfo is in OIDC-compatible.)

Fixes the HTTP client to have a timeout.
2022-05-25 01:17:08 +02:00
Tommie Gannert a6b13a703d Add automatic registration of SSO accounts. 2022-05-23 18:16:54 +02:00
Tommie Gannert 03cf5a5c08 Replace ThreePID lookups with the new SSO lookups. 2022-05-23 17:55:38 +02:00
Tommie Gannert c3f7945284 Add SSO UserAPI endpoints.
This is mostly copied from the ThirdPID, but with a primary key that
matches OpenID Connect nomenclature. There's a namspace to ensure
other SSO solutions can be supported, but there's only one namespace
defined for now.
2022-05-23 17:42:36 +02:00
Tommie Gannert c9ad7206c8 Enable SyTest "login types include SSO" 2022-05-23 16:25:29 +02:00
Tommie Gannert c1c2a0448c Disable m.login.token if there are no enabled login methods to use it. 2022-05-23 16:25:29 +02:00
Tommie Gannert 43989aa017 Support for m.login.sso.
This is forked from @anandv96's #1374. Closes #1297.
2022-05-23 16:25:29 +02:00
Tommie Gannert 1d6501ae30 Ensure every HTTP server context has a timeout.
Code that uses http.NewRequestWithContext will see the same deadline.
2022-05-23 08:37:39 +02:00
Till a53c9300aa
Fix /deactivate (#2474)
* Fix /deactivate

* Update test to correctly check the expected response
2022-05-20 13:27:11 +02:00
kegsay 21dd5a7176
syncapi: don't return early for no-op incremental syncs (#2473)
* syncapi: don't return early for no-op incremental syncs

Comments explain why, but basically it's an inefficient use
of bandwidth and some sytests rely on /sync to block.

* Honour timeouts

* Actually return a response with timeout=0
2022-05-19 09:00:56 +01:00
Till f321a7d55e
Really SKIP_NODB (#2472)
* Really SKIP_NODB

* Use fatalError in createLocalDB

* Check if createdb exists

* Revert change

* Remove !Quiet
2022-05-18 15:17:23 +02:00
kegsay b3162755a9
bugfix: fix race condition when updating presence via /sync (#2470)
* bugfix: fix race condition when updating presence via /sync

Previously when presence is updated via /sync, we would send the presence update
asyncly via NATS. This created a race condition:
 - If the presence update is processed quickly, the /sync which triggered the presence
   update would see an online presence.
 - If the presence update was processed slowly, the /sync which triggered the presence
   update would see an offline presence.

This is the root cause behind the flakey sytest: 'User sees their own presence in a sync'.

The fix is to ensure we update the database/advance the stream position synchronously
for local users.

* Bugfix for test
2022-05-17 15:53:08 +01:00
Kegan Dougal ac92e04772 Remove debug logging 2022-05-17 13:31:48 +01:00
kegsay 6de29c1cd2
bugfix: E2EE device keys could sometimes not be sent to remote servers (#2466)
* Fix flakey sytest 'Local device key changes get to remote servers'

* Debug logs

* Remove internal/test and use /test only

Remove a lot of ancient code too.

* Use FederationRoomserverAPI in more places

* Use more interfaces in federationapi; begin adding regression test

* Linting

* Add regression test

* Unbreak tests

* ALL THE LOGS

* Fix a race condition which could cause events to not be sent to servers

If a new room event which rewrites state arrives, we remove all joined hosts
then re-calculate them. This wasn't done in a transaction so for a brief period
we would have no joined hosts. During this interim, key change events which arrive
would not be sent to destination servers. This would sporadically fail on sytest.

* Unbreak new tests

* Linting
2022-05-17 13:23:35 +01:00
kegsay cd82460513
Add docs which explain how to calculate coverage (#2468) 2022-05-17 10:45:50 +01:00
Till 05607d6b87
Add roomserver tests (3/4) (#2447)
* Add Room Aliases tests

* Add Rooms table test

* Move StateKeyTuplerSorter to the types package

* Add StateBlock tests
Some optimizations

* Add State Snapshot tests
Some optimization

* Return []int64 and convert to pq.Int64Array for postgres

* Move []types.EventNID back to rows.Next()

* Update tests, rename SelectRoomIDs
2022-05-16 19:33:16 +02:00
Neil Alexander 6af35385ba
Version 0.8.5 (#2461)
* Version 0.8.5

* Update changelog

* Update changelog
2022-05-13 13:17:15 +01:00
Neil Alexander b40b548432
The Pinecone gobind demo must listen on localhost for baseURL to be correct 2022-05-13 12:06:47 +01:00
Neil Alexander 1698c39579
Update README.md 2022-05-13 11:52:42 +01:00
Neil Alexander be9be2553f
Resolve over old and new extremities (#2457)
* Feed existing state into state res when calculating state from new extremities

* Remove duplicates

* Fix bug

* Sort and unique

* Update to matrix-org/gomatrixserverlib#308

* Trim the slice properly

* Update gomatrixserverlib again

* Update to matrix-org/gomatrixserverlib#308
2022-05-13 11:52:04 +01:00
Neil Alexander cafc2d2c10
Update NATS Server to version 2.8.2 (#2460) 2022-05-13 11:36:04 +01:00
Till Faelligen b57fdcc82d Only try to get OTKs if the context isn't done yet 2022-05-13 10:28:00 +02:00
Till 870f9b0c3f
Shuffle config Verify/Defaults a bit around (#2459) 2022-05-13 09:33:55 +02:00
Neil Alexander fc670f03a2
Separate sample configs for monolith and polylith (#2456)
* Update sample configs

* Update references

* Remove sections that are dead in the monolith sample
2022-05-12 12:05:55 +01:00
Till 0d1505a4c1
Fix create-account with global database settings (#2455)
* Fix create-account with global database settings

* Avoid warning about open registration
2022-05-12 11:35:35 +02:00
Kegan Dougal 3437adf597 Wait 100ms for events to be processed by syncapi 2022-05-12 10:11:46 +01:00
Till 58af7f61b6
Fix OTK upload spam (#2448)
* Fix OTK spam

* Update comment

* Optimize selectKeysCountSQL to only return max 100 keys

* Return CurrentPosition if the request timed out

* Revert "Return CurrentPosition if the request timed out"

This reverts commit 7dbdda9641.

Co-authored-by: kegsay <kegan@matrix.org>
2022-05-11 17:15:18 +01:00
Neil Alexander 24f7be968d
Fix link 2022-05-11 15:46:45 +01:00
Neil Alexander 19a9166eb0
New documentation: https://matrix-org.github.io/dendrite/ 2022-05-11 15:39:36 +01:00
kegsay 9599b3686e
More syncapi tests (#2451)
* WIP tests for flakey create event

* Uncomment all database test
2022-05-11 13:44:32 +01:00
kegsay c15bfefd0d
Add RoomExists flag to QueryMembershipForUser (#2450)
Fixes https://github.com/matrix-org/complement/pull/369
2022-05-11 11:29:23 +01:00
Till 6db08b2874
Add roomserver tests (2/?) (#2445)
* Add invite table tests; move variable declarations

* Add Membership table tests

* Move variable declarations

* Add PrevEvents table tests

* Add Published table test

* Add Redactions tests
Fix bug in SQLite markRedactionValidatedSQL

* PR comments, better readability for invite tests
2022-05-10 14:41:12 +02:00
Neil Alexander 1897e2f1c0
Version 0.8.4 2022-05-10 12:44:29 +01:00
Neil Alexander e2a932ec0b
Add indexes to syncapi_output_room_events table that satisfy the filters (#2446) 2022-05-10 11:23:36 +01:00
Neil Alexander 77722c5a4f
Back out matrix-org/dendrite#2421 by restoring http.Clients
This creates problems with non-HTTPS endpoints and should fix #2444.
2022-05-10 11:08:10 +01:00
database64128 1b3fa9689c
🏗️ mediaapi/thumbnailer: fix build with bimg (#2440)
Co-authored-by: kegsay <kegan@matrix.org>
2022-05-09 17:51:30 +01:00
Neil Alexander 6b3c183396
Version 0.8.3 (#2431)
* Version 0.8.3

* Update changelog
2022-05-09 17:31:14 +01:00
kegsay 236b16aa6c
Begin adding syncapi component tests (#2442)
* Add very basic syncapi tests

* Add a way to inject jetstream messages

* implement add_state_ids

* bugfixes

* Unbreak tests

* Remove now un-needed API call

* Linting
2022-05-09 17:23:02 +01:00
Neil Alexander a443d1e5f3
Don't store invites in sync API that aren't relevant to local users (#2439) 2022-05-09 16:25:22 +01:00
Neil Alexander 79da75d483
Federation consumer adds_state_event_ids tweak (#2441)
* Don't ask roomserver for events we already have in federation API

* Check number of events returned is as expected

* Preallocate array

* Improve shape a bit
2022-05-09 16:19:35 +01:00
Neil Alexander 1a7f4c8aa9
Don't try to re-fetch the event if it is listed in adds_state_event_ids (#2437)
* Don't try to re-fetch the event in the output message

* Try that again

* Add the initial event into the set
2022-05-09 15:22:33 +01:00
Till f69ebc6af2
Add roomserver tests (1/?) (#2434)
* Add EventJSONTable tests

* Add eventJSON tests

* Add EventStateKeysTable tests

* Add EventTypesTable tests

* Add Events Table tests
Move variable declaration outside loops
Switch to testify/assert for tests

* Move variable declaration outside loop

* Remove random data

* Fix issue where the EventReferenceSHA256 is not set

* Add more tests

* Revert "Fix issue where the EventReferenceSHA256 is not set"

This reverts commit 8ae34c4e5f.

* Update GMSL

* Add tests for duplicate entries

* Test what happens if we select non-existing NIDs

* Add test for non-existing eventType

* Really update GMSL
2022-05-09 15:30:32 +02:00
Neil Alexander 09d754cfbf
One NATS instance per BaseDendrite (#2438)
* One NATS instance per `BaseDendrite`

* Fix roomserver
2022-05-09 14:15:24 +01:00
Neil Alexander 79e2fbc663
Update to matrix-org/gomatrixserverlib#307 2022-05-09 13:53:51 +01:00
Neil Alexander 4c15c73b3a
Add (user_id, device_id) index on OTK table (#2435) 2022-05-09 11:13:04 +01:00
Neil Alexander 633ca06eb9
Version 0.8.3rc1 2022-05-06 16:34:52 +01:00
Neil Alexander 6bc6184d70
Simplify calculateLatest (#2430)
* Simplify `calculateLatest`

* Comments
2022-05-06 15:52:44 +01:00