Commit graph

1286 commits

Author SHA1 Message Date
Bohdan Horbeshko be5c532301 Assign CREATEDB permission to the test user
During the tests databases get recreated, and this fails despite of the
user being the owner of a dropped database. Maybe related to certain
PostgreSQL version.

Signed-off-by: Bohdan Horbeshko <bodqhrohro@gmail.com>
2020-11-19 00:28:52 +02:00
Neil Alexander e65a3443a4
Fix paths in quickstart 2020-11-18 15:11:48 +00:00
Neil Alexander 265cf5e835
Protect txnReq.newEvents with mutex (#1587)
* Protect txnReq.newEvents and txnReq.haveEvents with mutex

* Missing defer

* Remove t.haveEventsMutex
2020-11-18 11:31:58 +00:00
TR_SLimey 3bcb003248
Add IPv6 setting to nginx conf (#1582)
* Added IPv6 setting to nginx conf

https://matrix.to/#/!yomrOFwgFXzmeMAbzX:matrix.org/$sIkSxL002BKottY1Fop-bkusuyCJ1Y0zIFzqSWDYRrc?via=matrix.org&via=privacytools.io&via=dendrite.link

* Add IPv6 support to polylith nginx conf also
2020-11-18 10:33:17 +00:00
David Spenler 35ea55e70b
Implemented whois endpoint (#1573)
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-11-17 10:07:03 +00:00
Aaron Raimist d3b3371856
Update README to remove mention that read receipts are not implemented (#1586)
Signed-off-by: Aaron Raimist <aaron@raim.ist>
2020-11-17 08:43:22 +00:00
Neil Alexander 20a01bceb2
Pass pointers to events — reloaded (#1583)
* Pass events as pointers

* Fix lint errors

* Update gomatrixserverlib

* Update gomatrixserverlib

* Update to matrix-org/gomatrixserverlib#240
2020-11-16 15:44:53 +00:00
Neil Alexander d8b526b603
Update docker readme 2020-11-16 11:27:59 +00:00
Neil Alexander bfa9df4a5a
Acknowledgements in changelog 2020-11-16 11:25:24 +00:00
Neil Alexander 3856e547de
Version 0.3.0 2020-11-16 11:13:05 +00:00
Neil Alexander 3034d8e805
Update sytest lists to blacklist flaky 'Forgotten room messages cannot be paginated' test 2020-11-16 10:51:55 +00:00
Mayeul Cantan af41fcadc4
Fix Dendrite not backfilling on world_readable rooms (#1575)
The previous implementation was only checking if room history was
"shared", which it wasn't for rooms where a user was invited, or world
readable rooms.
This implementation leverages the IsServerAllowed method, which already
implements the complete verification algorithm.

Signed-off-by: `Mayeul Cantan <oss+matrix@mayeul.net>`

Co-authored-by: Kegsay <kegan@matrix.org>
2020-11-16 10:47:16 +00:00
Felix Hanley 41c0b5acfb
Update build script to run on any POSIX (#1577)
Signed-off-by: Felix Hanley <felix@userspace.com.au>
2020-11-16 10:09:53 +00:00
S7evinK cb41c992f7
Add -version flag (#1581)
Signed-off-by: Till Faelligen <tfaelligen@gmail.com>
2020-11-16 09:54:58 +00:00
Neil Alexander b7e288084c
Update README.md 2020-11-15 12:39:11 +00:00
Neil Alexander 8ce740d949
Fix username regex to allow = character (#1578) 2020-11-12 10:36:54 +00:00
S7evinK bcb89ada5e
Implement read receipts (#1528)
* fix conversion from int to string yields a string of one rune, not a string of digits

* Add receipts table to syncapi

* Use StreamingToken as the since value

* Add required method to testEDUProducer

* Make receipt json creation "easier" to read

* Add receipts api to the eduserver

* Add receipts endpoint

* Add eduserver kafka consumer

* Add missing kafka config

* Add passing tests to whitelist

Signed-off-by: Till Faelligen <tfaelligen@gmail.com>

* Fix copy & paste error

* Fix column count error

* Make outbound federation receipts pass

* Make "Inbound federation rejects receipts from wrong remote" pass

* Don't use errors package

* - Add TODO for batching requests
- Rename variable

* Return a better error message

* - Use OutputReceiptEvent instead of InputReceiptEvent as result
- Don't use the errors package for errors
- Defer CloseAndLogIfError to close rows
- Fix Copyright

* Better creation/usage of JoinResponse

* Query all joined rooms instead of just one

* Update gomatrixserverlib

* Add sqlite3 migration

* Add postgres migration

* Ensure required sequence exists before running migrations

* Clarification on comment

* - Fix a bug when creating client receipts
- Use concrete types instead of interface{}

* Remove dead code
Use key for timestamp

* Fix postgres query...

* Remove single purpose struct

* Use key/value directly

* Only apply receipts on initial sync or if edu positions differ,
otherwise we'll be sending the same receipts over and over again.

* Actually update the id, so it is correctly send in syncs

* Set receipt on request to /read_markers

* Fix issue with receipts getting overwritten

* Use fmt.Errorf instead of pkg/errors

* Revert "Add postgres migration"

This reverts commit 722fe5a046.

* Revert "Add sqlite3 migration"

This reverts commit d113b03f64.

* Fix selectRoomReceipts query

* Make golangci-lint happy

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-11-09 18:46:11 +00:00
S7evinK eccd0d2c1b
Implement forgetting about rooms (#1572)
* Add basic storage methods

* Add internal api handler

* Add check for forgotten room

* Add /rooms/{roomID}/forget endpoint

* Add missing rsAPI method

* Remove unused parameters

* Add passing tests

Signed-off-by: Till Faelligen <tfaelligen@gmail.com>

* Add missing file

* Add postgres migration

* Add sqlite migration

* Use Forgetter to forget room

* Remove empty line

* Update HTTP status codes

It looks like the spec calls for these to be 400, rather than 403: https://matrix.org/docs/spec/client_server/r0.6.1#post-matrix-client-r0-rooms-roomid-forget

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2020-11-05 10:19:23 +00:00
Neil Alexander 2ce2112ddb
Update gjson for safe integer ranges 2020-11-05 09:42:28 +00:00
Petter Jakub Økland fe339c8b98
Created polylith reverse proxy sample for Hiawatha (#1571) 2020-10-28 16:37:56 +00:00
S7evinK d5675feb96
Add possibilty to configure MaxMessageBytes for sarama (#1563)
* Add configuration for max_message_bytes for sarama

* Log all errors when sending multiple messages

Signed-off-by: Till Faelligen <tfaelligen@gmail.com>

* Add missing config

* - Better comments on what MaxMessageBytes is used for
- Also sets the size the consumer may use
2020-10-27 14:11:37 +00:00
Neil Alexander c5888bb64c
Update Docker sample configs 2020-10-27 13:58:58 +00:00
Neil Alexander 0af35bec1a
Update documentation (#1569)
* Update dendrite-config.yaml

* Update README.md

* Update INSTALL.md

* Update INSTALL.md
2020-10-26 22:09:13 +00:00
Neil Alexander 59428cdde3
Version 0.2.1 (#1558)
* Version 0.2.1

* Credit where credit due

* Update CHANGES.md

* Include PRs
2020-10-22 12:38:12 +01:00
Adit Sachde b66b2eab32
Document dendrite server key format (#1520)
Co-authored-by: Kegsay <kegan@matrix.org>
2020-10-22 12:21:31 +01:00
Neil Alexander f32320a235
Exclude old state events from sync when consumed from roomserver (#1548)
* Exclude old room events from sync when consumed from roomserver

* Fix comment

* Experimental hack
2020-10-22 11:50:48 +01:00
Neil Alexander 3afc623098
Fix RewritesState bug (#1557)
* Set RewritesState once

* Check if any new state provided

* Obey rewritesState

* Don't nuke everything the sync API knows when purging state

* Fix panic from duplicate insert

* Consistency

* Use HasState

* Remove nolint

* Clean up joined rooms on state rewrite
2020-10-22 10:39:16 +01:00
Neil Alexander 04dc019e5e
Don't set empty state snapshots 2020-10-21 16:21:36 +01:00
Neil Alexander 534f9a9eb6
Refactor forward extremities (#1556)
* Add resolve-state helper

* Tweaks

* Refactor forward extremities, again

* Tweaks

* Minor optimisation

* Make path a bit clearer

* Only process state/membership if forward extremities have changed

* Usage comments in resolve-state
2020-10-21 15:37:07 +01:00
Vladislav Povalyaev e4f3f38f35
Update development percentages (#1553) 2020-10-20 20:49:03 +01:00
Kegsay 6b8791b868
Always call overridden form of GetLatestEventsForUpdate (#1554)
This ensures we don't make txns on sqlite still, which can cause
'database is locked' errors.
2020-10-20 19:32:33 +01:00
Neil Alexander 7612f64e3c
Update gomatrixserverlib to matrix-org/gomatrixserverlib#220 2020-10-20 17:23:04 +01:00
Neil Alexander 24e38c4135
Internal HTTP APIs over H2C (#1541)
* H2C on internal HTTP because SCIENCE

* Update comments
2020-10-20 17:13:12 +01:00
Neil Alexander 7ca89ef511
Update gomatrixserverlib 2020-10-20 17:10:37 +01:00
Neil Alexander 6a16d46fba
Version 0.2.0 (#1551)
* v0.2.0-rc2

* Update CHANGES.md (also with some markdown lint suggestions)

* Update version number

* Update CHANGES.md

* Update CHANGES.md

* Update CHANGES.md

* Add known issue

* Update CHANGES.md
2020-10-20 16:46:53 +01:00
Neil Alexander 39c7a8915c
Multi-personality polylith binary (#1552)
* Initial work oon multipersonality binary

* Remove old binaries

* Monolith and polylith binaries

* Better logging

* dendrite-poly-multi

* Fix path

* Copyright notices etc

* Tweaks

* Update Docker, INSTALL.md

* Take first argument if flags package doesn't find any args

* Postgres 9.6 or later, fix some more Docker stuff

* Don't create unnecessary e2ekey DB

* Run go mod tidy
2020-10-20 16:11:24 +01:00
Neil Alexander 6c3c621de0
Remove invalid state delta check (#1550) 2020-10-20 12:36:16 +01:00
Kegan Dougal 837c295c26 Linting 2020-10-20 12:29:53 +01:00
Pika 53a745f333
fix create-account (#1546) 2020-10-20 11:47:37 +01:00
Kegsay eb86e2b336
Fix sqlite locking bugs present on sytest (#1543)
* Fix sqite locking bugs present on sytest

Comments do the explaining.

* Fix deadlock in sqlite mode

Caused by starting a writer whilst within a writer

* Only complain about invalid state deltas for non-overwrite events

* Do not re-process outlier unnecessarily
2020-10-20 11:42:54 +01:00
Neil Alexander 92982a402f
Update Docker (#1542)
* Separate Docker images, rather than tags

* Allow specifying tag to build/push/pull

* Include goose in Docker builds
2020-10-20 11:34:22 +01:00
Neil Alexander a71360d099
Update INSTALL.md (#1549) 2020-10-20 11:22:46 +01:00
Neil Alexander c7bf122a26
Update sytest lists 2020-10-19 15:38:42 +01:00
Neil Alexander 5d80ff11a0
Update sytest-whitelist 2020-10-19 15:17:28 +01:00
Devon Johnson 45abdcaeb9
Send state after event, not current
Signed-off-by: Devon Johnson <djohnson1865@gmail.com>
2020-10-19 15:12:37 +01:00
Neil Alexander 6e63df1d9a
KindOld (#1531)
* Add KindOld

* Don't process latest events/memberships for old events

* Allow federationsender to ignore duplicate key entries when LatestEventIDs is duplicated by RS output events

* Signal to downstream components if an event has become a forward extremity

* Don't exclude from sync

* Soft-fail checks on KindNew

* Don't run the latest events updater at all for KindOld

* Don't make federation sender change after all

* Kind in federation sender join

* Don't send isForwardExtremity

* Fix syncapi

* Update comments

* Fix SendEventWithState

* Update sytest-whitelist

* Generate old output events

* Sync API consumes old room events

* Update comments
2020-10-19 14:59:13 +01:00
S7evinK 0974f6e2c0
Fix internal http api (#1535)
Signed-off-by: Till Faelligen <tfaelligen@gmail.com>
2020-10-19 09:38:10 +01:00
Neil Alexander 640e8c50ec
Take write lock for rate limit map (#1532)
* Take write lock for rate limit map

* Fix potential race condition
2020-10-16 15:44:39 +01:00
Kegsay 4a7fb9c045
Automatically upgrade databases on startup (#1529)
* Support auto-upgrading accounts DB

* Auto-upgrade device DB deltas

* Support up/downgrading from cmd/goose

* Linting

* Create tables then do migrations then prepare statements

To avoid failing due to some things not existing

* Linting
2020-10-15 18:09:41 +01:00
Neil Alexander 3e5d38e284
Improve state resolution v2 performance (matrix-org/gomatrixserverlib#237) 2020-10-15 16:20:19 +01:00