Commit graph

2956 commits

Author SHA1 Message Date
Neil Alexander 503d9c7586
Improve logging in upgrade tests 2022-11-10 10:07:19 +00:00
John Terzis 40830b8a37 Space,Channel soft deletion with dendrite gating, tests (#889)
Closes HNT-244.

The following PR implements Space,Channel soft deletion using on-chain
`disabled` flag scope to space, channel respectively. On message sync,
dendrite will now gate disabled rooms by performing a leave on the user
attempting to sync unless the user is the owner (more on this later). To
re-join, given rooms (spaces,channels) are created by default using
`invite` membership state, the owner will need to undo the on-chain
`disabled` flag, setting it false then re-invite users that left the
room as a side effect of it becoming disabled previously.

The owner does not leave the space, channel because if they did then
there would be no one left to invite users let alone themselves back in
if the action is ever undone.

What is not implemented in this PR:
1. **Transitive leaves on channels in a space** - If a space is
disabled, users will leave the space but not the channels within the
space. To allow for fully disabling a space and all its' channels, the
client can offer a view to the owner that iterates over the channels and
space to disable all on-chain. Furthermore, we could implement a batch
on-chain method that fully disables all channels within a space (plus
the space) in one on-chain call to save the owner gas.
2. **Data deletion** - No data is remove from the DAGs or on-chain.
Therefore deletion is soft and reversible.
3. **New hook to check if a room is disabled** - the client can leverage
existing on-chain public read only methods `getSpaceInfoBySpaceId`,
`getChannelInfoByChannelId` to read the state of each in order to remove
spaces, channels from a member's view that are disabled.
2022-11-09 17:07:51 -07:00
Tak Wai Wong df41f84bfa generate localhost and goerli types (#895) 2022-11-09 14:23:54 -08:00
Neil Alexander bdaae060cc
Update Ristretto 2022-11-09 14:07:29 +00:00
Neil Alexander a5cabdbac5
Remove unspecced fields from Transaction (update to matrix-org/gomatrixserverlib@715dc88) 2022-11-09 09:24:29 +00:00
Tak Wai Wong 29dabec35e fix copy path of the deploy script (#856) 2022-11-07 11:53:26 -08:00
Till Faelligen 205a15621a
Add custom build flag to satisfy Sytest 2022-11-07 15:07:47 +01:00
Till c125203eb6
Handle m.room.tombstone events in the UserAPI (#2864)
Fixes #2863 and makes 
```
/upgrade preserves direct room state
local user has tags copied to the new room
remote user has tags copied to the new room
```
pass.
2022-11-07 09:47:18 +01:00
Neil Alexander a7b74176e3
Revert Docker user change 2022-11-04 21:49:18 +00:00
Tak Wai Wong 147b17eccf write out the rolemanager address in the json (#837)
Need the address to implement the ZionRoleManagerShim

I ran this in  zion-governance/ and it worked :
forge script scripts/Local.s.sol:DeployLocal --rpc-url
http://localhost:8545/ --private-key … --broadcast

space-manager.json has the rolemanager address
2022-11-04 14:04:03 -07:00
Till Faelligen b2712cd2b1
Fix GHA release script 2022-11-04 20:58:24 +01:00
Tak Wai Wong 78a4a213d4 Refresh dendrite with latest contract changes (#836)
sync latest dendrite main & space manager contract changes from our
fork.

Co-authored-by: Tak Wai Wong <tak@hntlabs.com>
2022-11-04 11:23:15 -07:00
Tak Wai Wong 15dd2011f1
Merge pull request #61 from HereNotThere/pr/dendrite-fork
sync upstream changes
2022-11-04 10:37:17 -07:00
Tak Wai Wong a0df5f69fb
refresh contracts and addresses (#60) 2022-11-04 10:36:49 -07:00
Tak Wai Wong 341bfe4fdd
sync upstream changes 2022-11-04 09:40:48 -07:00
Neil Alexander 7c73b131f4
Version 0.10.7 (#2861)
Changelog and version bump.
2022-11-04 15:33:20 +00:00
Till efe28db631
Update latestPosition when getting reversed room delta (#2860)
Regression test added in
https://github.com/matrix-org/complement/pull/551
Should fix https://github.com/matrix-org/dendrite/issues/2514?
2022-11-04 15:39:09 +01:00
Till b13cb43785
Send presence to joined hosts only (#2858)
Send presence events only to rooms the user is participating, not all
servers we know about.
Should fix #2752
2022-11-04 13:23:00 +01:00
Till Faelligen eeabe892a9
Cache go mod directory 2022-11-04 11:54:53 +01:00
Giuseppe Rodriguez 686450bdad HNT-320-321 Updates create channel to take role ids and makes contract types script simpler (#807)
Closes HNT-320

Co-authored-by: Tak Wai Wong <64229756+tak-hntlabs@users.noreply.github.com>
2022-11-03 21:35:44 -07:00
Tak Wai Wong a7ab7524da servers/dendrite Move calculation (#2856) commit 98d3f88bfb (#826) 2022-11-03 19:15:07 -07:00
Tak Wai Wong 2d6cd9148b
Merge pull request #59 from HereNotThere/tak/sync-upstream
sync upstream commit f5b11e30a4
2022-11-03 11:42:01 -07:00
Tak Wai Wong 1d633c0d5e
Force sync upstream commit 98d3f88bfb 2022-11-03 11:22:01 -07:00
Neil Alexander 98d3f88bfb
Move prev_batch calculation (#2856)
This might help #2847.
2022-11-03 16:56:21 +00:00
Neil Alexander fb2e7d1b05
Put P2P Demos back into their own Dockerfiles 2022-11-03 13:57:44 +00:00
Neil Alexander 9625a79926
Update to matrix-org/pinecone@37f2e9b 2022-11-03 13:06:21 +00:00
gitlab-nickfreeman 23a25be904
Update README.md (#2855)
There only are 2, not 3 sample `docker-compose` files.
This is only a change to the README.md, no need for Go unit tests

### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added Go unit tests or [Complement integration
tests](https://github.com/matrix-org/complement) for this PR _or_ I have
justified why this PR doesn't need tests
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `Nick Freeman <nick@nickfreeman.de>`
2022-11-03 12:18:25 +00:00
Neil Alexander 8704e84898
Tweak removeDuplicates calls to use events instead of recentEvents (#2853)
... since `events` is *after* history visibility filtering, not before
it.
2022-11-03 10:19:37 +00:00
0x1a8510f2 1fcbb9b5e5
Fix workdir in Dockerfile (and make it a volume) (#2852) 2022-11-03 08:37:58 +00:00
Till Faelligen 85d740ea1b
Add GHA permission to upload security events 2022-11-03 08:26:46 +01:00
Till Faelligen f5b11e30a4
Hopefully fix GHA sarif upload 2022-11-03 08:20:51 +01:00
Tak Wai Wong a9d3bdc058 subtree pull from dendrite fork for fix #2383 (#802)
Pull the fix for https://github.com/matrix-org/dendrite/issues/2838 into
the dendrite subtree. Includes latest dendrite main.

Signed-off-by: `Devon Hudson <devonhudson@librem.one>`
Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Brian Meek <brian@hntlabs.com>
Signed-off-by: Austin Ellis <austin@hntlabs.com>
Signed-off-by: `Rubin Poster <rubinposter@gmail.com>`
Signed-off-by: `ash lea <example@thisismyactual.email>`
Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
Co-authored-by: devonh <devon.dmytro@gmail.com>
Co-authored-by: Till <2353100+S7evinK@users.noreply.github.com>
Co-authored-by: Ashley Nelson <fant@shley.email>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Brian Meek <brian@hntlabs.com>
Co-authored-by: Tak Wai Wong <takwaiw@gmail.com>
Co-authored-by: Jean Lucas <jean@4ray.co>
Co-authored-by: Kabir Kwatra <kabir@kwatra.me>
Co-authored-by: sergekh2 <gitgitgit@khorun.com>
Co-authored-by: texuf <texuf.eth@gmail.com>
Co-authored-by: kegsay <kegan@matrix.org>
Co-authored-by: Brian Meek <brian@here.video>
Co-authored-by: John Terzis <john.c.terzis@gmail.com>
Co-authored-by: John Terzis <john@hntlabs.com>
Co-authored-by: Kerem <kerem.kazan@gmail.com>
Co-authored-by: Neboer <43609792+Neboer@users.noreply.github.com>
Co-authored-by: X. Ding <dingsm@gmail.com>
Co-authored-by: dxl <dxl@plotbridge.com>
Co-authored-by: ash lea <ashkitten@users.noreply.github.com>
Co-authored-by: 0x1a8510f2 <admin@0x1a8510f2.space>
Co-authored-by: Till Faelligen <tfaelligen@gmail.com>
Co-authored-by: Till Faelligen <davidf@element.io>
2022-11-02 16:28:06 -07:00
Tak Wai Wong a70e1de91a
Merge branch 'matrix-org:main' into main 2022-11-02 15:14:53 -07:00
Tak Wai Wong 5c64593dcb
Merge pull request #58 from HereNotThere/sync-upstream
Sync upstream changes to pick up fix for #2838
2022-11-02 15:13:46 -07:00
Tak Wai Wong 063299248b
sync fix #2838 from main 2022-11-02 13:48:55 -07:00
devonh 4afadebd99
Add readme for dendrite-demo-pinecone (#2851) 2022-11-02 20:29:30 +00:00
Tak Wai Wong 86f9b5ca89
Fix issue where a member is forced to leave a room when the invite is marked deleted #2838 (#55)
* fix #2838 by querying db directly for membership info

* fix lint error

* revert changes and use SelectMembershipForUser directly

* Remove extra membership checks

* Add a comment to explain why the membership check is necessary
2022-11-02 12:34:07 -07:00
Neil Alexander ef52731e9f
Tweak FLAGS in GHA Docker builds 2022-11-02 14:41:38 +00:00
Neil Alexander 9c0725feac
Maybe fix GHA 2022-11-02 14:09:19 +00:00
Neil Alexander ca8bc87380
Multi-stage Docker builds (#2850)
This builds on @S7evinK's work to make multi-stage Docker builds. Now
that we can build SQLite without Cgo this should be much simpler and
should make Docker builds in CI significantly faster.

Co-authored-by: Till Faelligen <tfaelligen@gmail.com>
Co-authored-by: Till Faelligen <davidf@element.io>
Co-authored-by: Till Faelligen <2353100+S7evinK@users.noreply.github.com>
2022-11-02 14:04:08 +00:00
0x1a8510f2 51ab0a8ccf
Fix moderncsqlite errors and rebase onto main (#2832)
This is #2819 but rebased on latest `main`. This PR is against main too
as opposed to the `moderncsqlite` branch.

The main change here is simply:

```go
// add query parameters to the dsn
if strings.Contains(dsn, "?") {
	dsn += "&"
} else {
	dsn += "?"
}

// wait some time before erroring if the db is locked
// https://gitlab.com/cznic/sqlite/-/issues/106#note_1058094993
dsn += "_pragma=busy_timeout%3d10000"
```

### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [x] I have added tests for PR _or_ I have justified why this PR
doesn't need tests.
* [x] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed off privately.

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-11-02 13:20:10 +00:00
Neil Alexander 16c2a95900
Improve logging for processEventWithMissingState 2022-11-02 11:30:49 +00:00
Till 86b25a6337
Add message stats to reporting (#2748)
Since we're now listening on the `OutputRoomEvent` stream, we are able
to store messages stats.
2022-11-02 10:18:11 +00:00
Till b367cfeddf
Implement /thirdparty endpoints (#2831)
Implements the following endpoints 
```
GET /_matrix/client/v3/thirdparty/protocols
GET /_matrix/client/v3/thirdparty/protocols/{protocol}
GET /_matrix/client/v3/thirdparty/location
GET /_matrix/client/v3/thirdparty/location/{protocol}
GET /_matrix/client/v3/thirdparty/user
GET /_matrix/client/v3/thirdparty/user/{protocol}
```
2022-11-02 10:17:53 +00:00
Tak Wai Wong 75a508cc27
Fix issue where a member is forced to leave a room when the invite is marked deleted (#2839)
Proposed fix for issue:
https://github.com/matrix-org/dendrite/issues/2838

Suppose bob received invites to spaceA and spaceB.
When Bob joins spaceA, we add an OutputEvent event to retire the invite.
This sets the invite to "deleted" in the database. This makes sense.

The bug is in stream_invites.go. Triggered when bob received a new
invite for spaceB, and does a client sync.

In the block (line 76)
`for roomID := range retiredInvites

   if _, ok := req.Response.Rooms.Invite[roomID]; ok {
	continue
   }

   if _, ok := req.Response.Rooms.Join[roomID]; ok {
	continue
  }
...
` 

Bob is not in either maps even though he had just accepted the invite
for spaceA. Consequently, the spaceA invite is treated as a retired
invite, and a membership Leave event is generated. What bob sees is that
after accepting the invite to spaceB, he lose access to spaceA.


### Pull Request Checklist

<!-- Please read
https://matrix-org.github.io/dendrite/development/contributing before
submitting your pull request -->

* [ ] I have added tests for PR _or_ I have justified why this PR
doesn't need tests.
* [x ] Pull request includes a [sign off below using a legally
identifiable
name](https://matrix-org.github.io/dendrite/development/contributing#sign-off)
_or_ I have already signed off privately

Signed-off-by: `Tak Wai Wong <tak@hntlabs.com>`

Co-authored-by: Neil Alexander <neilalexander@users.noreply.github.com>
2022-11-02 10:02:23 +00:00
Neil Alexander 3db9e98456
Don't limit "state" (#2849)
This is apparently some incorrect behaviour that we built as a result of
a spec bug (matrix-org/matrix-spec#1314) where we were applying a filter
to the `"state"` section of the `/sync` response incorrectly. The client
then has no way to know that the state was limited.

This PR removes the state limiting, which probably also helps #2842.
2022-11-02 09:34:19 +00:00
Neil Alexander 8a1904ffe5
Update pull request template 2022-11-02 09:33:00 +00:00
John Terzis b4223a1a18 Jterzis/sync dendrite (#793) 2022-11-01 23:17:48 -07:00
John Terzis f2d6888452
Merge pull request #56 from HereNotThere/jterzis/hnt-272-gate-msg-sync
message sync gating
2022-11-01 19:14:44 -07:00
John Terzis 6059650464 message sync gating 2022-11-01 18:07:15 -07:00