Commit graph

1674 commits

Author SHA1 Message Date
Brendan Abolivier 28346b39e8 3PID invite exchange over federation (#222)
* Use federation to auth the event if the server isn't in the room

* Use MakeAPI for 3pid onbind handler as it isn't a standard federation request

* Error check

* Temporarily disable tests

* Fix return on 3PID invite

* Re-enable tests

* Remove useless else

* gb vendor update github.com/matrix-org/gomatrixserverlib

* gb vendor update github.com/matrix-org/gomatrixserverlib

* Implement same behaviour as synapse

* Fix condition and array initialisation

* Log errors on iteration and throw one if no server could be reached

* Fix err not being initialised

* Fix lint

* Fix import path
2017-09-11 19:08:24 +01:00
Mark Haines 6cb9d900b9 Fix response to federation /invite to match the format expected by synapse (#221)
* Fix response to /invite to match the format expected by synapse

* gb vendor update github.com/matrix-org/gomatrixserverlib

* Use gomatrixserverlib.RespInvite

* gb vendor update github.com/matrix-org/gomatrixserverlib
2017-09-11 18:07:12 +01:00
Mark Haines 5740cb3e58 Redact the event before verifying its signature
... in the federation invite API.
2017-09-11 15:47:50 +01:00
Mark Haines 31f6c21d98 Enable GC for the pre-commit hook, and tune it for throughput (#219) 2017-09-08 15:56:52 +01:00
Brendan Abolivier 4d1d503d43 Processing of pending invites on 3PID binding (#218)
* Add missing file headers

* Move the ID server's signatures verification to common

* Allow verification without specifying a server name

* Add third-party structs to membership events content

* Add processing of 3PID onbind requests

* Use reference for third party invite data

* Fix return arguments order

* Revert "Move the ID server's signatures verification to common"

This reverts commit 9344201031.

* Revert "Allow verification without specifying a server name"

This reverts commit fd27afbf82.

* Remove checks that are already occurring in gomatrixserverlib

* Change return type of createInviteFrom3PIDInvite

* Add doc, add checks in fillDisplayName

* Use MakeFedAPI

* Invert condition

* Use AuthEvents to retrieve the 3PID invite

* Update comment

* Remove unused parameter

* gb vendor update github.com/matrix-org/gomatrixserverlib
2017-09-08 15:17:12 +01:00
Erik Johnston fad997303b Turn on 'unparam' linting (#216)
* Turn on 'unparam' linting

* Increase deadline

* Fix mediaapi integeration test
2017-09-07 12:50:39 +01:00
Erik Johnston ea80ffb3af Add get event federation API (#209)
* Add get event federation API

* Handle err correctly

* Add missing readers

* Ignore gocyclo lint for routing function

* Add todo

* Fix styling a bit
2017-09-06 12:38:22 +01:00
Erik Johnston 2dcb3a11a5 Use gometalinter (#210)
* Remove unused struct field

* Ignore unused test data

* Remove unused variables

* Remove deadcode

* Fix up vetshadow warnings

* Convert to using gometalinter

* Update travis

* Use vendored versions of gometalinter

* Make gometalinter install its stuff

* Vendor misspell
2017-09-05 17:40:46 +01:00
Erik Johnston a26d7c2899 gb vendor fetch github.com/alecthomas/gometalinter 2017-09-05 17:09:50 +01:00
Mark Haines 643d05b157 Add function for making authed federation APIs (#208) 2017-09-04 13:14:01 +01:00
Richard van der Hoff d4cce7369f Add scripts to run sytest on jenkins (#207) 2017-09-01 15:28:17 +01:00
Brendan Abolivier 960af3d628 Implement client API routes for 3PID handling (#205)
* Create package for handling 3pid processes and move invite processing there

* Add database table and functions for tracking 3PIDs

* Add structures and functions to interact with an ID server

* Add handlers for 3PIDs management

* Fix 3PIDs retrieval sending null if no 3PID known for a user

* Include medium in database requests and function calls

* Publish an association if it has been validated and requested

* Add TODO markers for tursted ID server check

* Use a structure instead of a map to represent a 3PID
2017-09-01 10:13:10 +01:00
Richard van der Hoff 8c2e6273e3 Add command to generate test keys/certs (#204)
Add `generate-keys` command, which can be used to generate TLS keys/certs, and,
more usefully, matrix signing keys.
2017-08-31 12:28:58 +01:00
Richard van der Hoff fc51f72bf9 Avoid swallowing error from kafka (#203) 2017-08-31 12:27:53 +01:00
Brendan Abolivier 2183712e04 Process requests for 3PID invites (#201)
* Structure for 3PID invite

* Generate invite from 3PID known by ID server

* Load user profile in a separate function

* Generate m.room.third_party_invite if the ID server doesn't know the 3PID

* Fix URLs to the spec in comments

* Move third-party invites to a separate package and doc' it

* Handle non-OK status codes on lookup

* Send display name to identity server when asking to store an invite

* Remove join response structure

* Change the way some variables are declared or passed as argument

* Use gomatrixserverlib.Base64String instead of the builtin base64 package

* Don't copy the public keys array
2017-08-29 15:17:26 +01:00
Brendan Abolivier 685e056ab3 Membership viewing API when user left the room (#194)
* Implement case where user left the room

* Filter by membership event

* Move the logic from the storage to the query API

* Fix check on state entries iteration

* Remove aliases methods from query API

* Use structure for response to match with the spec

* Remove filtering on /members and implement /joined_members
2017-08-24 16:00:14 +01:00
Andi fceb027ecc Add publicroomsapi to the database list (#199)
Signed-off-by: Andreas Brönnimann <foss@exul.org>
2017-08-23 22:06:25 +01:00
Mark Haines 9c954501a2 Implement /federation/v1/invite/{roomID}/{eventID} (#197)
* Implement /federation/v1/invite/{roomID}/{eventID}

* Use NotJSON instead of BadJSON when the JSON couldn't be decoded
2017-08-23 15:13:47 +01:00
Mark Haines 17c60759c4 Add query API for listing active invites (#196)
* Add query API for listing active invites

This lists the invites for a user in a room that could be used to
join the room over federation.

* s/Lookup/Look up/

* Fix implements comments
2017-08-23 15:08:48 +01:00
Brendan Abolivier 166ac9d092 Fix sync not returning on room join (#195)
* Use BuildEvent method on room join

* Fix building the list of room members in the sync notifier

* Fix building the list of room members in the sync notifier

* Rephrase comment
2017-08-22 14:14:37 +01:00
Brendan Abolivier b15ce900ab Implement public rooms APIs (#185)
* Move events contents to common

* Basic database structure

* Complete database update

* Support visibility update and retrieval

* Add HTTP methods for visibility update and retrieval

* Add the database for the new component

* Add a listener for the new component

* Fix attribute update statements

* Create public rooms component

* Fix failing test

* Add roomserver consumer

* Fix a bug in aliases creation

* Add a check on type

* Implement public rooms directory

* Use auth API for visibility update

* Support filtering

* Add component to monolith

* Various fixes

* Fix computation of next public rooms batch

* Retrieve state events from the roomserver query API + avoid dupes on join

* Split update of string or boolean attribute in two separate functions

* Use event type to detect duplicate joins

* Improve the joined members counter computation

* Use event.RoomID()
2017-08-22 11:12:51 +01:00
Richard van der Hoff fc86821a90 notes on running monolith server (#193)
* notes on running monolith server

* Clarify default ports for monolith server
2017-08-22 11:01:14 +01:00
Richard van der Hoff 46877b6baa Add installation instructions to repo (#192) 2017-08-21 17:34:05 +01:00
Mark Haines 808c2e09f6 Make txn *sql.Tx arguments optional everywhere using a utility function (#191)
* Make txn *sql.Tx arguments optional everywhere using a utility function

* Clarify that if the txn is nil the stmt will run outside a transaction
2017-08-21 17:20:23 +01:00
Mark Haines 57b7097368 Add input API for adding invites to the roomserver. (#187)
* Add input API for adding invites to the roomserver.

This API handles invites received over federation that occur outside of a room.

* Add some docstring for withTransaction

* Use a nicer pattern for wrapping transactions

* Fix MembershipUpdater method to not commit the transaction before returning it

* Use the Transaction interface from common
2017-08-21 16:37:11 +01:00
Brendan Abolivier 5950293e79 Membership viewing API (#174)
* Basic memberships retrieval

* Change the way the memberships are saved in the client API database

* Retrieve single membership

* Get memberships only if the user is or has been in the room

* Check server name on room ID instead of user ID

* Save the join membership event and updates it when necessary

* Membership events retrieval + update on leave

* Implement the API on the roomserver and client API server

* Fix comments

* Remove the functions and attributes used before the new query API

* Explicitely state what we return in query

* Remove tab
2017-08-21 16:34:26 +01:00
Mark Haines 81179a0595 Fix integration test proxy certificate arguments (#190) 2017-08-21 16:23:59 +01:00
Brendan Abolivier 35df4767c6 Remove defaults for TLS so the proxy can more easily serve over HTTP (#189) 2017-08-21 14:25:47 +01:00
Mark Haines efbc14f7b9 vars["txnID"] is always empty for state events (#188) 2017-08-18 16:10:28 +01:00
Brendan Abolivier f607ef29c2 Add a route matching the trailing slash on the state event sending route (#182)
* Add a route matching the trailing slash on the state event sending route

* Use single route for both cases

* Use synapse regexp
2017-08-18 15:33:40 +01:00
Mark Haines 877ea5cb62 Remove StopProcessingAfter from the roomserver consumer as it is unused (#186) 2017-08-18 11:33:10 +01:00
Mark Haines ba8b5d8bf9 Fix kafka consumer setup in monolith. (#184)
We can't consume the same topic on a single kafka consumer more than
once. So when using kafka we have to create a new consumer for each
component in the monolith.
2017-08-16 16:55:34 +01:00
Mark Haines c27d1fdfb4 Optionally use naffka in the monolithic server (#183)
* dependency injection for the kafka consumers/producers

* Optionally use naffka in the monolithic server

* remember to call setupKafka()

* tweak imports

* fix integration tests

* Add use_naffka to the example config

* Update comment on the listen APIs
2017-08-16 13:36:41 +01:00
Mark Haines 0d894e3da5 gb vendor fetch github.com/matrix-org/naffka 2017-08-15 16:06:09 +01:00
Mark Haines 2071387f3c Add tables for tracking the state of invites to the room server. (#165)
* Storage functions for invite events

* Add table for tracking membership state

* More stuff

* More stuff

* Use utility methods from gomatrixserverlib, rather than reimplementing them

* More stuff

* Return string rather than pointer to string

* Update gomatrixserverlib

* Use HTTP API for roomserver input.

* Use synchronous HTTP API for writing events to the roomserver

* Remove unused config for kafka topic

* Add new output types to roomserver for invites

* Write membership updates

* Separate filtering from pairing up changes in membershipChanges

* Fix SQL

* Fix SQL

* Namespace the tables

* Fix SQL

* Use clearer names for some of the variables

* Rename senderID for consistency

* Restructure update membership

* Comments

* More comment

* Fix SQL

* More comments

* Assign state keys inside the transaction

* Comment on the purpose of the latestEventsUpdater

* Comment on the purpose of updateMembership

* Remove duplicate fields from stateChange

* Attempt to rewrite comment in 'english'

* More comments

* Fix comment

* Comment

* more comments
2017-08-08 16:38:03 +01:00
Brendan Abolivier c35803c9d8 Add TLS (https) support to the monolithic server (#180)
* Add HTTPS listen and serve, and move both listen and serve to goroutines

* Add some log so we know the initialisation went well
2017-08-07 13:39:53 +01:00
Mark Haines 4d1504ee93 Add prefixes to namespace the SQL tables. (#177)
* Add prefixes to namespace the SQL tables.

This means that multiple components can share a single database schema
without colliding with each other.

Once this lands it will be possible to run a single monolithic dendrite
against a single postgresql schema.

Hopefully this will make trivial deployments and development easier.

* Comment
2017-08-07 11:51:46 +01:00
Jonas Platte 21a8c24d5f Add .editorconfig (#179) 2017-08-05 02:25:40 +01:00
Jonas Platte c15e4572ff Fix typo: ed2519 -> ed25519 (#178) 2017-08-05 02:25:03 +01:00
Brendan Abolivier 8ccc5d108b Implement membership APIs (#171)
* Implement membership endpoints

* Use FillBuilder when possible

* Fix typo in membership event content

* Fix state key invite membership events not being correctly set

* Set membership content to match the profile of the user in state_key

* Move event building and rename common function

* Doc getMembershipStateKey

* Check if user is local before lookin up their profile
2017-08-04 16:32:10 +01:00
Brendan Abolivier 03dd456b47 Fix bad initialisation of sync API notifier in monolith (#176) 2017-08-04 14:54:08 +01:00
Mark Haines 1a28cf9767 Add a monolithic server that combines all the components into one (#175)
* Add a monolithic server that combines all the components into one

* Review comments

* Expose the matrix APIs directly
2017-08-04 13:12:36 +01:00
Mark Haines 4975eb9074 Move setting up the api mux to outside the routing.Setup functions. (#173)
This makes it possible to setup all the component APIs on a single http
listener which is necessary if we want to combine all the components
into a single monolith.
2017-08-03 15:10:39 +01:00
Mark Haines 3b07633326 Simplify preparing the roomserver SQL (#172) 2017-08-02 16:50:34 +01:00
Brendan Abolivier 0fbb8b7824 Make account data sync incremental (#170)
* Clean roomserver consumer

* Make account data sync incremental

* Use a different name for the sync AD table

* Improved error logging

* Created missing topic in tests

* Add client API topic to tests

* Add client API topic to common

* Move data batch retrieval

* Add database index for data retrieval

* Fix typo in table name

* Fix indentation
2017-08-02 16:21:35 +01:00
Brendan Abolivier 7d17df6f51 Add local room join (#169)
* Add local room join

* Update TODO
2017-07-28 15:29:12 +01:00
Brendan Abolivier 34165c7761 Add default config file name for media API and room servers (#168) 2017-07-28 11:32:16 +01:00
Brendan Abolivier c7e36ba2a1 Implement room aliases (#167)
* Add database functions to interact with aliases

* Save room alias

* Check if alias already exists

* Implement local alias lookup

* Implement alias removal

* Move alias API

* Handle case with no alias set for a room

* Apply requested changes
2017-07-28 11:31:43 +01:00
Brendan Abolivier 3e394e9e21 Save and retrieve account data (#166)
* Save function for account data

* Fix upsert + add empty routes and function

* Save account data

* Retrieval functions

* Implement retrieval in /sync

* Fix arrays not correctly initialised

* Merge account data retrieval functions

* Request DB only once per request

* Initialise array

* Fix comment
2017-07-26 14:53:11 +01:00
Brendan Abolivier 6d073dcf9f Propagate profile update through rooms (#163)
* Use gomatrixserverlib function to split user ID

* Propagate profile update via m.room.member events

* Send profile data on room join

* Send profile data on room creation

* Rename variable

* Move membership update to roomserver consumer

* Improve iteration

* Move event update from client API server to sync API server

* Change the way buildMembershipEvents is called

* Forbid update of someone else's profile

* Use gomatrixserverlib method

* Fix depth and previous events not being set

* Fix wrong removal in latest commit

* Update all events instead of only memberships

* Handle case where there is no state key

* Fix test
2017-07-25 16:10:59 +01:00