Commit graph

2252 commits

Author SHA1 Message Date
Brendan Abolivier 83c3c7e1db
Fix the ordering of events in a response to /sync (#588) 2018-11-07 19:12:23 +00:00
Brendan Abolivier 56058b9469
Implement the /backfill federation endpoint (#585)
* Implement the /backfill federation endpoint

* Make the BFS loop block common between QueryMissingEvents and QueryPreviousEvents

* Improve comments on the BFS loop block

* Optimisation: prevent unnecessary redefinitions/reallocations

* Add trailing slash at the end of the route for parity with synapse

* Replace QueryPreviousEvents with QueryBackfill

* Change the backfill response to comply with the specs and synapse's behaviour
2018-11-07 11:38:01 +00:00
Matthew Hodgson daf57b19b7 make docker work on current codebase (#584) 2018-11-06 18:14:39 +00:00
Brendan Abolivier 8ff136e595
Fix interactive registration failing (#583)
* Fix interactive registration failing because of being confused with AS registration

* Fix AS registration tests

* Move AS registration handling to dedicated function and split the switch/case to avoid unnecessary condition

* Ignore handleRegistrationFlow() for gocyclo and add some doc/comments on the code
2018-11-06 14:40:37 +00:00
Zero King 1a82e6bc58 Add Go 1.11 to .travis.yml (#580) 2018-09-04 18:20:30 +01:00
Zero King a0b38a6f2b Fix typo (#579) 2018-09-01 09:31:47 +01:00
krombel b71d922a72 Propagate error with wrong ?ts= param back to client (#576)
* make MatrixError implement error interface

* let ParseTSParam return error when no int transmitted

* fix to high cyclo for SendEvent

* Move generateSendEvent below SendEvent

* Drop ParseIntParam() as it is used only in one place

* Parse ts param at the beginning of JoinRoom

to be able to abort right in the beginning and
to not parse the MatrixError to get an error response

* make ParseTSParam() return error instead of JSONResponse
2018-08-22 13:40:25 +01:00
Andrew Morgan 0b5ae4692e Make use of /users/{userID} in relevant APIs (#522)
* Query whether a room alias exists on app services

Signed-off-by: Andrew Morgan <andrewm@matrix.org>

* Query AS /alias/ API at a lower level

* Add support for querying /users/ on appservices

* Have endpoints query AS users if not found locally
2018-08-20 10:45:17 +01:00
Andrew Morgan 2382d363ab Include appservice namespace in username available check (#504)
Signed-off-by: Andrew Morgan <andrewm@matrix.org>
2018-08-20 10:23:01 +01:00
mohit kumar singh d07a652d8e return same device as sent from client if it exists in db (#414)
Signed-off-by: mohit kumar singh <mohitkumarsingh907@gmail.com>
2018-08-20 10:22:06 +01:00
Anant Prakash 5d52863b9f
[Federation] Send typing events (#572)
* GetJoinedHosts from federation server db

* Add dummy api.OutputTypingEvent

* Add a typing server consumer to federation sender

* Update queue to support EDU events

* Update OutputTypingEvent format

* Use SendEDU in federation server, remove dummy/api

* Add helpful comments

* fix typo

* remove origin field

* Count EDUs in sendCounter
2018-08-10 20:56:57 +05:30
Andrew Morgan bab4d1401f
AppServices: Implement /users/{userID} (#521)
* Add support for querying /users/ on appservices

* Fix copy/paste error
2018-08-10 07:47:14 -07:00
Andrew Morgan 609646c19b
Implement GET /rooms/{roomAlias} (#494)
* Query whether a room alias exists on app services

Signed-off-by: Andrew Morgan <andrewm@matrix.org>

* URL encode room alias before sending to AS

* Add /room/ to path

* Query AS /alias/ API at a lower level

* Don't verify self-signed AS certificates

* Don't skip cert validation on appservices, fix logging

* Separate req.WithContext

* Linting

* Do not warn when an AS room alias does not exist
2018-08-08 08:17:10 -07:00
Andrew Morgan e05a31f4c2
Federation: Improve /query/directory endpoint (#525) 2018-08-07 06:41:50 -07:00
Anant Prakash b11591fdd1
Fix up timestamp messaging (#570)
* Revert "implement AS timestamp massaging (#542)"

This reverts commit a56752f3f6.

* Update APIs to support ts

* refactor MakeJoin, make code consistent

* Keep one parameter per line
2018-08-06 18:39:25 +05:30
Anant Prakash 2c2200718a
Cleanup code in federationapi/routing (#571)
Signed-off-by: Anant Prakash <anantprakashjsr@gmail.com>
2018-08-06 18:37:29 +05:30
Anant Prakash dc89e04e7d
Add whoami endpoint support (#434)
* Add whoami endpoint support

Signed-off-by: Anant Prakash <anantprakashjsr@gmail.com>

* Trim code, update against merged changes
2018-08-04 15:02:02 +05:30
Andrew Morgan 1176c1d574 Add detail to room alias name error message (#565) 2018-08-03 15:56:48 +01:00
Anant Prakash 1165b49da7
Implement Typing server (#567)
* update gomatrixserverlib

* Make removeUser public

* Implement api.TypingServerInputAPI

* Integrate the typing server component, create kafka topic

* Add typing server cmd for multiprocess dendrite
2018-08-02 22:52:44 +05:30
Anant Prakash 9cdd3a66e4
Add TypingCache to maintain a list of users typing (#559)
* Add typing cache

Signed-off-by: Anant Prakash <anantprakashjsr@gmail.com>

* Add tests for typingCache

* Make test stricter

* Handle cases where expireTime is updated

* Make the slice comparisons sturdy

* Use timers to call removeUser after timeout

* Add test for TypingCache.removeUser

Signed-Off-By: Matthias Kesler <krombel@krombel.de>

* Write deterministic test
2018-07-31 16:22:57 +05:30
Derek f8ca2561cc removed outdated "flag" comments on clientapi membership storage (#566)
Signed-off-by: Derek Meer <derekmeer@protonmail.com>
2018-07-31 10:06:04 +01:00
Anant Prakash 2a2f42990a
Refactor string slice helper to common/test (#562)
* Refactor string slice helper to common/test

Signed-off-by: Anant Prakash <anantprakashjsr@gmail.com>

* Do not modify the given slice

* Add Unsorted in the function name
2018-07-26 22:04:39 +05:30
Anant Prakash 53a726fab1
Correct user/devices path (#557) 2018-07-25 18:28:10 +05:30
Anant Prakash 38965ef5e2
Support PUTing typing status in clientapi (#550)
* Add handler for typing events

* Add typing events producer

* Setup typing server component

* Send one event per API call
2018-07-24 20:19:49 +05:30
krombel 68131ca7a3 Handle AS with auth header (#548)
* Handle AS with auth header

* fix lint (gocyclo)
2018-07-23 06:40:35 -07:00
Andrew Morgan d2ae425752
Generate SenderLocalpart AS user (#505)
* Generate sender_localpart user for each AS on startup

Signed-off-by: Andrew Morgan <andrewm@matrix.org>

* Clean up diff
2018-07-17 09:31:40 -07:00
Andrew Morgan dbbfd26be7
Support inhibit_login registration option (#532)
* Support inhibit_login registration option

* Fix completeRegistration func definition

* Change InhibitLogin to a WeakBoolean
2018-07-17 08:57:20 -07:00
Andrew Morgan a56752f3f6
implement AS timestamp massaging (#542) 2018-07-17 08:45:30 -07:00
Andrew Morgan 99005d6a91
Allow appservices to register without a login type (#529)
* Allow appservices to register without a logintype

Signed-off-by: Andrew Morgan <andrewm@matrix.org>

* Reduce cyclomatic complexity on Register function

* Fix some grammar
2018-07-17 08:39:49 -07:00
Andrew Morgan d2b349a75a
Check userID against AS namespace instead of localpart (#533) 2018-07-17 08:36:23 -07:00
Andrew Morgan 40b1dd6f12
Have TravisCI run with go 1.10 as well (#527) 2018-07-17 07:58:32 -07:00
Andrew Morgan e3b9754d59
Internal API for Appservices (#541) 2018-07-17 07:36:04 -07:00
Anant Prakash 2fb2f7ca7b
Add AS support to common.MakeAuthAPI (#427)
* Add AS support to MakeAuthAPI
Make clientapi utilize the same

Signed-off-by: Anant Prakash <anantprakashjsr@gmail.com>

* Add user parameter support to MakeAuthAPI

* Make VerifyAccessToken private, let VerifyUserFromRequest return the
device if present

* Make a dummy device for AS users

* Refactor arguments into auth.Data

* Update routing of all components

* Update code comment

* Use const AppServiceDeviceID

* Handle cases when AS is not masquerading
2018-07-17 03:47:03 +05:30
Andrew Morgan 3ac23f9063
Remove trailing slashes from appservice url (#546)
* Remove trailing slashes from appservice url

* Use TrimRight instead
2018-07-16 08:16:01 -07:00
Andrew Morgan 49b63089f5
Consolidate AS interest checking (#539)
* Methods for checking if an AS is interested in events

* Look through rooms namespace for matching room IDs
2018-07-16 05:30:04 -07:00
Anant Prakash d4b24462d1
return err when err != nil (#545) 2018-07-11 17:46:38 +05:30
Anant Prakash 730c4f74db
[federation] Fix state API endpoints (#518)
* [federation] Fix state API endpoints

Signed-off-by: Anant Prakash <anantprakashjsr@gmail.com>

* Use parseEventIDParam instead
2018-07-11 15:43:04 +05:30
Anant Prakash c8feee7354
[roomserver] Add internal API QueryMembershipForUser (#544)
* Add internal API QueryMembershipForUser

Signed-off-by: Anant Prakash <anantprakashjsr@gmail.com>

* Change sender to user_id
2018-07-11 15:40:37 +05:30
Anant Prakash 74b319c42a
[federation] Add make_leave & send_leave (#535)
* [federation] Add make_leave & send_leave

* Remove redundant parameters

* Check membership is set to leave

Signed-off-by: Anant Prakash <anantprakashjsr@gmail.com>
2018-07-11 15:37:44 +05:30
Anant Prakash c87fb4d894
Remove unnecessary req.Close() (#537)
Signed-off-by: Anant Prakash <anantprakashjsr@gmail.com>
2018-07-06 17:06:19 +05:30
Andrew Morgan 385cab8dc9
Actually check if alias is reserved (#534) 2018-07-06 03:29:15 -07:00
Andrew Morgan ae19db60e3
Check userID instead of username from application services (#500)
* Check UserID instead of username from AS's. Tests.

* add tests to validateApplicationService

* Use some literals, organize URLs & checks

* Fix error messages and incorrect test
2018-07-06 03:28:49 -07:00
Andrew Morgan 20af8a6786
Allow multiple regexes per as namespace (#528)
Signed-off-by: Andrew Morgan <andrewm@matrix.org>
2018-07-06 02:46:12 -07:00
Andrew Morgan 8da54352e5
Document internal APIs and Application Services component (#499)
* Document internal APIs, app services in WIRING.md

* Add application services component to WIRING diagram

* Fix typo and clarify what mode internal APIs use HTTP
2018-07-06 02:33:53 -07:00
Andrew Morgan 83b3df762b
Apply the same username checks to AS's, without '_' check (#501) 2018-07-06 02:27:11 -07:00
Andrew Morgan 7f7ae65098
Don't require preset to create room (#496)
* Don't require preset to create room

* Set defaults if no preset. Use switch instead of if

* Add default case and comment to r.Preset switch
2018-07-06 02:20:17 -07:00
Andrew Morgan 7736e247b7
Send Application Service Events (#477)
* Prevent sql scanning into nil value in accounts_table

Signed-off-by: Andrew Morgan <andrewm@matrix.org>

* Remove uneccessary logging, null checking

* Don't forget to set the localpart

* Simplify error checking

* Store And Send Application Service Events

* Modify INSTALL.md and dendrite-config.yaml for the new appservice database
* Correct all instances of casing on 'application service' to align with
spec
* Store incoming events that an app service is interested in in the
database to be later read by transaction workers.
* Retrieve these events from transaction workers, one per AS.
* Minimal transaction ID data is stored as well to recover after
server failure.
* Send events to AS and exponentially backoff on failure.

Signed-off-by: Andrew Morgan <andrewm@matrix.org>

* Finish my own sentences.

* Fix up database interaction

* Change to event-based AS sending

* Reduce cyclomatic complexity

* Appease the errcheck gods

* Delete by int ID instead of string.

This was causing some events to not be deleted, as < an eventID doesn't
really make much sense.

* Check if there are more events to send before sleeping

* Send same transaction if last send attempt failed

* Don't backoff on non-200s, tight send loop, 1 event query

* Remove tight send loop. Fix events not being deleted

* Additionally order by event id, track main.go

* Return the last txnID, which our events are using

* Remove old main.go file

* Prevent duplicate events from being sent...

* Strip event content if it doesn't contain anything

Signed-off-by: Andrew Morgan <andrewm@matrix.org>

* Update gomatrixserverlib and use Unsigned AS event prop

* Fixes

* Fix sync server comment
* Remove unnecessary printlns
* Use logrus Fields
* Worker state methods
* Remove sillyness

* Fix up event filtering

* Handle transaction event limit in loop

* Switch to using a sequence for transaction IDs

* Don't verify self-signed AS certificates

* Fix logging

* Use gmsl.Event instead of AS-only event in transactions

Also clear up the logic on lookupStateEvents a little bit.

* Change invalid_txn_id to global (for efficiency)

* Use a bool for EventsReady instead of an int
2018-07-05 09:34:59 -07:00
Anant Prakash 8f5526763c Remove unused params (#538)
Signed-off-by: Anant Prakash <anantprakashjsr@gmail.com>
2018-07-04 03:05:20 -07:00
Anant Prakash 142fbcde21
Return 404 when profile is not found (#524)
* Return 404 when profile is not found

* Handler sql.ErrNoRows and avoid returning nil

update federation GetProfile as well

Signed-off-by: Anant Prakash <anantprakashjsr@gmail.com>
2018-07-03 19:27:52 +05:30
Andrew Morgan 1cc21d6bcd
Prevent alias registration in AS exclusive namespace (#480)
* Prevent alias registration in AS exclusive namespace

* Fix M_Exclusive errors so they return status 400 instead of 403.

Signed-off-by: Andrew Morgan <andrewm@matrix.org>

* Directly access namespace map instead of searching for key
2018-06-29 04:09:00 -07:00