Commit graph

226 commits

Author SHA1 Message Date
Robert Swain cc6595f848 common/test/server: Clarify postgresContainerName 2017-06-08 14:53:45 +02:00
Robert Swain f68ef67cb6 travis: Explain openssl key+cert generation 2017-06-08 14:50:31 +02:00
Robert Swain 19e0c274de cmd/mediaapi-integration-tests: Test remote thumbnail before download
This ordering also exercises the cold cache immediate generation of a
size configured for pregeneration.
2017-06-08 14:45:11 +02:00
Robert Swain c41cdd6174 cmd/mediaapi-integration-tests: Document state of tests 2017-06-08 14:44:35 +02:00
Robert Swain 37d3a1b305 cmd/mediaapi-integration-tests: Add valid dynamic thumbnail test 2017-06-08 13:18:56 +02:00
Robert Swain 7b002ace4c travis: Install openssl and generate server.{crt,key} 2017-06-08 13:11:38 +02:00
Robert Swain f7e16450a2 mediaapi/thumbnailer: Check if request is larger than original 2017-06-08 13:07:41 +02:00
Robert Swain f2ac8d442a mediaapi/thumbnailer: Factor out isThumbnailExists
Appease gocyclo^w^w simplify
2017-06-08 13:06:32 +02:00
Robert Swain dc5d7f78e5 cmd/mediaapi-integration-tests: Log size and method for thumbnails 2017-06-08 13:03:38 +02:00
Robert Swain 9169bd2acd cmd/mediaapi-integration-tests: Make HTTPS requests 2017-06-08 12:12:22 +02:00
Robert Swain 8b9205062b common/test/client: Do not verify TLS certs for testing
We will commonly use self-signed certs.
2017-06-08 12:11:37 +02:00
Robert Swain 536e7402aa cmd/client-api-proxy: Optionally listen for HTTPS 2017-06-08 12:10:52 +02:00
Robert Swain 097373a1d6 cmd/mediaapi-integration-tests: Add totem.jpg test file 2017-06-08 10:50:38 +02:00
Robert Swain 6b66cb06fc Merge branch 'master' into rob/media-tests 2017-06-08 10:49:42 +02:00
Robert Swain 0e0fe28b22 mediaapi/thumbnailer: Store thumbnail according to requested size 2017-06-08 10:41:22 +02:00
Robert Swain d6c3b171db cmd/mediaapi-integration-tests: Add upload/download/thumbnail tests 2017-06-08 10:40:39 +02:00
Robert Swain 9673792b48 common/test/client: Don't panic on EOF in CanonicalJSONInput 2017-06-08 10:39:26 +02:00
Robert Swain 2b166c0ca8 travis: Build the client-api-proxy for media tests 2017-06-07 18:38:28 +02:00
Robert Swain 5ce0c11d94 travis: Enable media API integration tests 2017-06-07 18:35:06 +02:00
Robert Swain ec60077a42 cmd/mediaapi-integration-tests: Add proxies in front of servers
This is needed so that origins can be correctly configured and used for
remote media.
2017-06-07 18:30:08 +02:00
Robert Swain 9a48de7db3 common/test/server: Add StartProxy to create a reverse proxy 2017-06-07 18:28:04 +02:00
Robert Swain 39ec5fdf63 common/test/server: Remove redundant argument 2017-06-07 18:27:26 +02:00
Robert Swain 9b887aa696 common/test/client: Fix uninitialised LastRequestErr 2017-06-07 18:26:24 +02:00
Mark Haines b184a48897 Fetching missing state from the roomserver. (#135)
* Fetching missing state from the roomserver.

Whenever the syncserver receives an event from the room server that adds
state that isn't in the syncserver's local database it should fetch
those state events from the roomserver.

* Fix append

* Put comment back

* Comments

* s/addsStateEvents/lookupStateEvents/

* Fix spelling

* Include the stream position that a state event was added at in the current state tables

* Fix comment

* Review comments
2017-06-07 16:35:41 +01:00
Robert Swain 935f3275b9 cmd/mediaapi-integration-tests: Make startMediaAPI return listen address 2017-06-07 17:18:46 +02:00
Robert Swain 83e54d9d74 cmd/mediaapi-integration-tests: Simplify getMediaURL argument 2017-06-07 17:15:25 +02:00
Robert Swain 82f9ac5cc9 cmd/mediaapi-integration-tests: Simplify slice initialisation 2017-06-07 16:38:04 +02:00
Robert Swain dac32b386d common/test/client: Remove stray debug print 2017-06-07 16:08:57 +02:00
Robert Swain 6650246ee8 cmd/syncserver-integration-tests: Port to new common/test infra 2017-06-07 16:04:40 +02:00
Robert Swain e3b18f9138 common/test/client: Handle timed out requests for tests that passed 2017-06-07 16:03:17 +02:00
Mark Haines 515cce1a45 Add support for receiving room events over federation. (#130)
* Add API for querying events by ID.

* Fix tense

* Start implementing federation ingress

* More stuff

* Hook up federation event receiving

* Handle the case where we are missing state

* Fix docstring and comments

* Fix infinite loop when printing unknownRoomError
2017-06-07 14:32:53 +01:00
Robert Swain f48c293cb0 common/test: Add some server init and client request utilities 2017-06-07 14:56:44 +02:00
Robert Swain 85fcef2968 cmd/mediaapi-integration-tests: Add foundation for testing 2017-06-07 14:47:31 +02:00
Robert Swain 2d202cec07 mediaapi: Add thumbnail support (#132)
* vendor: Add bimg image processing library

bimg is MIT licensed. It depends on the C library libvips which is LGPL
v2.1+ licensed. libvips must be installed separately.

* mediaapi: Add YAML config file support

* mediaapi: Add thumbnail support

* mediaapi: Add missing thumbnail files

* travis: Add ppa and install libvips-dev

* travis: Another ppa and install libvips-dev attempt

* travis: Add sudo: required for sudo apt* usage

* mediaapi/thumbnailer: Make comparison code more readable

* mediaapi: Simplify logging of thumbnail properties

* mediaapi/thumbnailer: Rename metrics to fitness

Metrics is used in the context of monitoring with Prometheus so renaming
to avoid confusion.

* mediaapi/thumbnailer: Use math.Inf() for max aspect and size

* mediaapi/thumbnailer: Limit number of parallel generators

Fall back to selecting from already-/pre-generated thumbnails or serving
the original.

* mediaapi/thumbnailer: Split bimg code into separate file

* vendor: Add github.com/nfnt/resize pure go image scaler

* mediaapi/thumbnailer: Add nfnt/resize thumbnailer

* travis: Don't install libvips-dev via ppa

* mediaapi: Add notes to README about resizers

* mediaapi: Elaborate on scaling libs in README
2017-06-07 01:12:49 +02:00
Robert Swain def49400bc cmd/client-api-proxy: Add proxy for media API (#133) 2017-06-05 15:55:04 +02:00
Mark Haines 721c2df484 Clean up syncapi storage to match the coding style set by roomserver storage (#131) 2017-06-05 10:37:04 +01:00
Mark Haines ef7b934d51 Add Query API for querying events by ID (#129) 2017-06-02 14:32:36 +01:00
Mark Haines 1b6e06aa59 Revert "Add API for querying events by ID. (#127)" (#128)
This reverts commit ce82158abb.
2017-06-02 14:24:13 +01:00
Mark Haines ce82158abb Add API for querying events by ID. (#127)
* Add API for querying events by ID.

* Fix tense

* Start implementing federation ingress

* More stuff

* Hook up federation event receiving

* Fix comments

* Comment on the order of the arrays
2017-06-02 11:19:34 +01:00
Robert Swain 3b498c8074 Merge pull request #126 from matrix-org/rob/media-download-remote
Add support for downloading of remote files
2017-06-01 18:01:42 +02:00
Robert Swain 7244fc15b3 media/writers/download: Make use of CreateMediaDownloadRequest from lib 2017-06-01 17:57:05 +02:00
Robert Swain 0fff33928e vendor: Bump gomatrixserverlib to pull in CreateMediaDownloadRequest 2017-06-01 17:56:30 +02:00
Robert Swain a398cdd193 mediaapi/writers/download: Use mime.ParseMediaType to parse Content-Disposition 2017-06-01 16:04:41 +02:00
Robert Swain 2d822c57f9 mediaapi/writers/download: Use DNS address as not all HSes have SRV 2017-06-01 15:30:36 +02:00
Robert Swain cae309fa26 mediaapi/writers/download: Handle panic to ensure waking of goroutines
If the active request were to panic, we need to ensure all the waiting
goroutines get woken up.
2017-06-01 14:54:59 +02:00
Robert Swain ce21b32d38 mediaapi/writers/download: Wrap broadcast call in closure to re-evaluate args 2017-06-01 14:44:00 +02:00
Robert Swain 4457ebddca mediaapi/writers/download: Rework remote file download synchronisation
Avoid locking around db requests by only locking around active requests
and always creating an active request if there is none. A nice side
effect of this is that if many parallel requests for remote media come
in (a common case) then only one database query is made for the duration
of the query.
2017-06-01 12:32:15 +02:00
Robert Swain a3b1c7535a mediaapi/writers/download: Remove unnecesary Unlock, Lock after Cond.Wait 2017-06-01 08:39:35 +02:00
Robert Swain 923013aa55 mediaapi: Add support for downloading of remote files 2017-05-31 17:56:11 +02:00
Robert Swain 07c821daec mediaapi/writers: Fix InternalServerError responses 2017-05-31 17:41:42 +02:00