Neil Alexander
534f9a9eb6
Refactor forward extremities ( #1556 )
...
* Add resolve-state helper
* Tweaks
* Refactor forward extremities, again
* Tweaks
* Minor optimisation
* Make path a bit clearer
* Only process state/membership if forward extremities have changed
* Usage comments in resolve-state
2020-10-21 15:37:07 +01:00
Vladislav Povalyaev
e4f3f38f35
Update development percentages ( #1553 )
2020-10-20 20:49:03 +01:00
Kegsay
6b8791b868
Always call overridden form of GetLatestEventsForUpdate ( #1554 )
...
This ensures we don't make txns on sqlite still, which can cause
'database is locked' errors.
2020-10-20 19:32:33 +01:00
Neil Alexander
7612f64e3c
Update gomatrixserverlib to matrix-org/gomatrixserverlib#220
2020-10-20 17:23:04 +01:00
Neil Alexander
24e38c4135
Internal HTTP APIs over H2C ( #1541 )
...
* H2C on internal HTTP because SCIENCE
* Update comments
2020-10-20 17:13:12 +01:00
Neil Alexander
7ca89ef511
Update gomatrixserverlib
2020-10-20 17:10:37 +01:00
Neil Alexander
6a16d46fba
Version 0.2.0 ( #1551 )
...
* v0.2.0-rc2
* Update CHANGES.md (also with some markdown lint suggestions)
* Update version number
* Update CHANGES.md
* Update CHANGES.md
* Update CHANGES.md
* Add known issue
* Update CHANGES.md
2020-10-20 16:46:53 +01:00
Neil Alexander
39c7a8915c
Multi-personality polylith binary ( #1552 )
...
* Initial work oon multipersonality binary
* Remove old binaries
* Monolith and polylith binaries
* Better logging
* dendrite-poly-multi
* Fix path
* Copyright notices etc
* Tweaks
* Update Docker, INSTALL.md
* Take first argument if flags package doesn't find any args
* Postgres 9.6 or later, fix some more Docker stuff
* Don't create unnecessary e2ekey DB
* Run go mod tidy
2020-10-20 16:11:24 +01:00
Neil Alexander
6c3c621de0
Remove invalid state delta check ( #1550 )
2020-10-20 12:36:16 +01:00
Kegan Dougal
837c295c26
Linting
2020-10-20 12:29:53 +01:00
Pika
53a745f333
fix create-account ( #1546 )
2020-10-20 11:47:37 +01:00
Kegsay
eb86e2b336
Fix sqlite locking bugs present on sytest ( #1543 )
...
* Fix sqite locking bugs present on sytest
Comments do the explaining.
* Fix deadlock in sqlite mode
Caused by starting a writer whilst within a writer
* Only complain about invalid state deltas for non-overwrite events
* Do not re-process outlier unnecessarily
2020-10-20 11:42:54 +01:00
Neil Alexander
92982a402f
Update Docker ( #1542 )
...
* Separate Docker images, rather than tags
* Allow specifying tag to build/push/pull
* Include goose in Docker builds
2020-10-20 11:34:22 +01:00
Neil Alexander
a71360d099
Update INSTALL.md ( #1549 )
2020-10-20 11:22:46 +01:00
Neil Alexander
c7bf122a26
Update sytest lists
2020-10-19 15:38:42 +01:00
Neil Alexander
5d80ff11a0
Update sytest-whitelist
2020-10-19 15:17:28 +01:00
Devon Johnson
45abdcaeb9
Send state after event, not current
...
Signed-off-by: Devon Johnson <djohnson1865@gmail.com>
2020-10-19 15:12:37 +01:00
Neil Alexander
6e63df1d9a
KindOld ( #1531 )
...
* Add KindOld
* Don't process latest events/memberships for old events
* Allow federationsender to ignore duplicate key entries when LatestEventIDs is duplicated by RS output events
* Signal to downstream components if an event has become a forward extremity
* Don't exclude from sync
* Soft-fail checks on KindNew
* Don't run the latest events updater at all for KindOld
* Don't make federation sender change after all
* Kind in federation sender join
* Don't send isForwardExtremity
* Fix syncapi
* Update comments
* Fix SendEventWithState
* Update sytest-whitelist
* Generate old output events
* Sync API consumes old room events
* Update comments
2020-10-19 14:59:13 +01:00
S7evinK
0974f6e2c0
Fix internal http api ( #1535 )
...
Signed-off-by: Till Faelligen <tfaelligen@gmail.com>
2020-10-19 09:38:10 +01:00
Neil Alexander
640e8c50ec
Take write lock for rate limit map ( #1532 )
...
* Take write lock for rate limit map
* Fix potential race condition
2020-10-16 15:44:39 +01:00
Kegsay
4a7fb9c045
Automatically upgrade databases on startup ( #1529 )
...
* Support auto-upgrading accounts DB
* Auto-upgrade device DB deltas
* Support up/downgrading from cmd/goose
* Linting
* Create tables then do migrations then prepare statements
To avoid failing due to some things not existing
* Linting
2020-10-15 18:09:41 +01:00
Neil Alexander
3e5d38e284
Improve state resolution v2 performance ( matrix-org/gomatrixserverlib#237 )
2020-10-15 16:20:19 +01:00
Neil Alexander
e154c45b51
Better logging around db.StoreEvent
2020-10-15 14:14:17 +01:00
Neil Alexander
49abe359e6
Start Kafka connections for each component that needs them ( #1527 )
...
* Start Kafka connection for each component that needs one
* Fix roomserver unit tests
* Rename to naffkaInstance (@Kegsay review comment)
* Fix import cycle
2020-10-15 13:27:13 +01:00
Neil Alexander
10f1beb0de
Don't re-run state resolution on a single trusted state snapshot ( #1526 )
...
* Don't re-run state resolution on a single trusted state snapshot
* Lint
* Check if backward extremity is create event before checking missing state
2020-10-15 12:08:49 +01:00
Kegsay
e3c2b081c7
txn nil guard when rolling back LatestEventsUpdater ( #1524 )
...
* txn nil guard when rolling back LatestEventsUpdater
* Spell lint correctly
2020-10-14 17:05:09 +01:00
Neil Alexander
6f12b8f85c
Ignore typing events where sender doesn't match origin ( #1523 )
...
* Ignore typing notifications where the sender doesn't match the origin
* Update sytest-whitelist
* Fix formatting directives
2020-10-14 16:49:25 +01:00
Neil Alexander
e3a3908654
Update sytest-whitelist ( closes #1315 )
2020-10-14 16:07:15 +01:00
Neil Alexander
8d9ecb3996
Ignore duplicate redaction entries ( #1522 )
2020-10-14 15:24:43 +01:00
Neil Alexander
286dd408ae
Better semver compliance for version reporting
2020-10-14 12:51:06 +01:00
Neil Alexander
7a1fd123de
Improved state handling in /send ( #1521 )
...
* Capture errors
* Don't request only state key tuples needed for auth (we end up discarding room state this way)
* QueryStateAfterEvent returns all state when no tuples supplied
* Resolve state
* Comments
2020-10-14 12:39:37 +01:00
Neil Alexander
20aec70ead
Send cumulative state when creating room ( #1519 )
...
* Send state with new room events
* lookupEvent sends outliers
* Revert "lookupEvent sends outliers"
This reverts commit 3e16556441
2020-10-14 09:57:27 +01:00
Neil Alexander
9d6b77c58a
Try to retrieve missing auth events from multiple servers ( #1516 )
...
* Recursively fetch auth events if needed
* Fix processEvent call
* Ask more servers in lookupEvent
* Don't panic!
* Panic at the Disco
* Find servers more aggressively
* Add getServers
* Fix number of servers to 5, don't bail making RespState if auth events missing
* Fix panic
* Ignore missing state events too
* Report number of servers correctly
* Don't reuse request context for /send_join
* Update federation API tests
* Don't recurse processEvents
* Implement getEvents differently
2020-10-13 11:53:20 +01:00
Kegsay
d7ea814fa8
Wrap NewMembershipUpdater in a db writer ( #1515 )
2020-10-13 10:20:27 +01:00
Neil Alexander
73bc28b11f
Update docker-compose.deps.yml
2020-10-13 09:54:07 +01:00
Neil Alexander
2f578531ba
Update Docker samples
2020-10-13 09:50:11 +01:00
Neil Alexander
8001627cfc
Get missing event tweaks ( #1514 )
...
* Adjust backfill to send backward extremity with state before other backfilled events, include prev_events with no state amongst missing events
* Not finished refactor
* Fix test
* Remove isInboundTxn
* Remove debug logging
2020-10-12 15:56:15 +01:00
Pieter
0804594a61
Add systemd example service to docs ( #1512 )
...
Signed-off-by: Pieter Hollander <git@pieterhollander.nl>
2020-10-12 10:06:56 +01:00
Neil Alexander
6b579e6aba
Don't ignore local keys from fetchers ( #1513 )
2020-10-12 10:02:20 +01:00
Kegsay
9096bfcee8
Validate m.room.create events in send_join responses ( #1505 )
...
* Validate m.room.create events in send_join responses
For sytest compliance, refs #1315 and #1317
Fixes #1317
* Linting
2020-10-10 00:21:15 +01:00
Neil Alexander
fe5d1400bf
Update federation timeouts ( #1504 )
...
* Update to matrix-org/gomatrixserverlib#234
* Update gomatrixserverlib
* Update federation timeouts
* Fix dendritejs
* Increase /send context time in destination queue
2020-10-09 17:08:32 +01:00
Neil Alexander
4df7e345bb
Only return 500 on /send if a database error occurs ( #1503 )
2020-10-09 15:06:43 +01:00
S7evinK
2bd0449c5b
Use default value when adding column which is not null ( #1501 )
...
Signed-off-by: Till Faelligen <tfaelligen@gmail.com>
2020-10-09 12:41:53 +01:00
S7evinK
1cd525ef0d
Extend device_devices table ( #1471 )
...
* Add last_used_ts and IP to database
* Add migrations
* Rename column
Prepare statements
* Add interface method and implement it
Signed-off-by: Till Faelligen <tfaelligen@gmail.com>
* Rename struct fields
* Add user_agent to database
* Add userAgent to registration calls
* Add missing "IF NOT EXISTS"
* use txn writer
* Add UserAgent to Device
Co-authored-by: Kegsay <kegan@matrix.org>
2020-10-09 09:17:23 +01:00
Pestdoktor
c4c8bfd027
reject invalid UTF-8 ( #1472 )
...
* reject invalid UTF-8
Signed-off-by: Jonas Fentker <jonas@fentker.eu>
* update sytest-whitelist
Signed-off-by: Jonas Fentker <jonas@fentker.eu>
Co-authored-by: Kegsay <kegan@matrix.org>
2020-10-09 09:15:51 +01:00
Loïck Bonniot
f3e8ae01ef
Implement fully read markers ( #1475 )
...
See #653
Signed-off-by: Loïck Bonniot <git@lesterpig.com>
Co-authored-by: Kegsay <kegan@matrix.org>
2020-10-09 09:15:35 +01:00
Kegsay
009401ad4d
Version 0.1.0
...
Beta README (#1466 )
* Beta docs
* More tweaks
* More docs
* Update README.md (#1497 )
* Call out missing features
* Add CHANGES
* Call out CHANGES
* Update INSTALL.md
* Update README.md
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-10-08 17:47:43 +01:00
Kegan Dougal
b12b7abcc0
v0.1.0
...
Squashed commit of the following:
commit 570528e5f19d76fe2fd7ea646246eea9650b1b60
Author: Kegan Dougal <kegan@matrix.org>
Date: Thu Oct 8 14:52:10 2020 +0100
v0.1.0
commit 02c020bced3be6d1a63e1001ab9c0ef6231ba087
Merge: db840f02 8035c50c78f6e1a3c8fc68553e12f6e9c8fc68555c15bf385141e7d9eea48bca7a83e7d9eea4a03fa76370f2f7cecdd952ddded7f7cecdd9a8
2020-10-08 14:53:46 +01:00
Neil Alexander
8035c50c06
Don't get into situations where we have no forward extremities
2020-10-08 14:46:25 +01:00
Neil Alexander
78f6e1a31e
Don't set new state NID if state regression
2020-10-08 13:30:30 +01:00
