Commit graph

194 commits

Author SHA1 Message Date
Robert Swain 9fc5abdb3f mediaapi/writers: Rename utils.go to fileutils.go
Better reflects the content of the file.
2017-05-18 16:01:06 +02:00
Robert Swain 2e795ed8aa mediaapi/storage: Improve GetMediaMetadata description 2017-05-18 15:57:07 +02:00
Robert Swain 1f2ac60bee mediaapi/routing: Sync make() to makeAPI() as in clientapi 2017-05-18 15:53:48 +02:00
Robert Swain 04c4a2d05a cmd/dendrite-media-api-server: Move os.Getenv() for consistency 2017-05-18 15:50:09 +02:00
Robert Swain 1057e2e117 Merge branch 'master' into rob/media-upload 2017-05-18 12:47:41 +02:00
Robert Swain e4a97d13b3 Merge pull request #108 from matrix-org/rob/golang-1.8
.travis.yml: Bump golang to 1.8
2017-05-18 12:47:08 +02:00
Robert Swain 4df470eab5 .travis.yml: Bump golang to 1.8 2017-05-18 12:38:09 +02:00
Robert Swain ec0d584fe7 cmd/dendrite-media-api-server: Log format string with Infof not Info 2017-05-18 12:23:17 +02:00
Robert Swain 8085c1f863 mediaapi/types: Clarify what is ActiveRemoteRequests.Set's key 2017-05-18 12:19:03 +02:00
Robert Swain 28ef35d36a mediaapi/storage: Rework GetMediaMetadata API to return new MediaMetadata 2017-05-18 12:09:33 +02:00
Robert Swain 2fca4bbd65 mediaapi/config: Fix max_file_size_bytes YAML tag 2017-05-18 11:58:41 +02:00
Robert Swain c5cd5a93b9 mediaapi: Use ServerName type from gomatrixserverlib 2017-05-18 11:57:44 +02:00
Robert Swain bd9db7557a mediaapi/README: Add link to spec section 2017-05-18 11:50:24 +02:00
Robert Swain 7727a8c61e cmd/dendrite-media-api-server: Add MAX_FILE_SIZE_BYTES configuration 2017-05-18 11:44:48 +02:00
Robert Swain 846aece163 mediaapi: MaxFileSize -> MaxFileSizeBytes 2017-05-18 11:36:26 +02:00
Robert Swain 35a0b5d2e9 cmd/dendrite-media-api-server: Add BASE_PATH configuration 2017-05-18 11:34:01 +02:00
Robert Swain ff3009ffdd cmd/dendrite-media-api-server: Add SERVER_NAME configuration 2017-05-18 11:32:30 +02:00
Robert Swain deee6f84c7 mediaapi/writers/upload: Move file first as db is source of truth
The database is the source of truth. If we add the metadata to the
database and it succeeds, and then the file fails to be moved, we think
we have a file when we actually don't.
2017-05-18 11:10:41 +02:00
Robert Swain 3f904e1cdb mediaapi/writer/upload: Remove unnecessary logic 2017-05-18 11:09:09 +02:00
Robert Swain f28235c05d mediaapi/writers/upload: Factor out removeDir
Reduces complexity of Upload. Note that we never care about the error
from os.RemoveAll() beyond logging as we are already in an error case.
2017-05-18 11:07:03 +02:00
Robert Swain 5348b64edc mediaapi/writers/download: Reduce complexity of copyToActiveAndPassive 2017-05-18 10:17:11 +02:00
Robert Swain b80d5ab919 cmd/mediaapi-integration-tests: Test downloading same file 100 times
Spawns a GET request for the same file in 100 parallel go routines and
prints the body (which is some error JSON) in case of not 200 OK. Also
prints the number of successful requests.

This of course should take command line arguments for the URL and number
of requests but that can be done as soon as needed.
2017-05-18 09:12:01 +02:00
Robert Swain 8cf507f85f mediaapi/writers: Never return server errors to user but log them 2017-05-18 09:04:36 +02:00
Robert Swain eddc3e0789 mediaapi/writers/download: Log warnings about why we return 404 2017-05-18 08:57:29 +02:00
Robert Swain c43aa20adc mediaapi/writers/download: Remove duplicate log message 2017-05-18 08:56:54 +02:00
Kegsay 917c433fd2 Minor refactoring (#106)
- `s/Server/OutputRoomEvent/` in `consumers` to accurately reflect what is being consumed.
- `s/set/userIDSet/` in `notifier.go` for clarity.
- Removed lying comments.
2017-05-17 17:29:26 +01:00
Kegsay c8c5a40ebd Update README.md 2017-05-17 16:37:58 +01:00
Kegsay 0457a3bb69 State of the union update 2017-05-17 16:36:17 +01:00
Kegsay ccd0eb2851 /sync bugfix: Check transitions to 'leave' do not leak events afterwards (#105) 2017-05-17 16:21:27 +01:00
Robert Swain a8b7130745 mediaapi/writers/download: Clean up copyToActiveAndPassive 2017-05-17 16:39:01 +02:00
Kegsay d5a44fd3e8 Only wake up /sync requests which the event is for (#101) 2017-05-17 15:38:24 +01:00
Robert Swain 9f8458716d mediaapi/writers/download: Pass config members to respondFromRemoteFile 2017-05-17 16:37:11 +02:00
Robert Swain 19ffa4d75a mediaapi/writers/download: respondFromLocalFile only needs basePath 2017-05-17 16:30:01 +02:00
Robert Swain 13b1051a3e mediaapi/writers/download: Add logger to downloadRequest and use it 2017-05-17 16:24:30 +02:00
Robert Swain 6e24fb86cb mediaapi/writer/download: Make functions into methods and use MediaMetadata 2017-05-17 16:13:54 +02:00
Robert Swain 937162a722 mediaapi/routing: Pass pointer to config struct instead of copying 2017-05-17 13:29:35 +02:00
Kegsay 0a3d44a80a Handle room transitions to 'leave' in incremental /sync requests (#104)
* Add test for transition to leave

* Add test for join/leave in the same /sync response
2017-05-17 10:25:59 +01:00
Kegsay 675759c192 Add invites to /sync responses (#103) 2017-05-15 17:41:54 +01:00
Kegsay 94e1c62745 Remove intermediary /sync response struct (#102)
The logic required to populate the right bits of `RoomData` tends towards
the complete `/sync` response struct, so just use the actual response struct
and save the hassle of mapping between the two. It may not make much difference
in its current form, but the next PR will make use of this.

This PR has no functional changes.
2017-05-15 15:18:08 +01:00
Kegsay cc7117392f Send the entire room state down when transitioning to 'join' on a /sync response (#100)
This is only 'mostly' correct currently, because what should be no-op dupe
joins will actually trigger the entire room state to be re-sent.

Bizarrely, it's significantly easier to just do that than work out if we should,
and there are no client-visible effects to doing so, so we just do it for now.
2017-05-12 16:56:17 +01:00
Kegsay 2a9abefd92 Add more syncserver tests (#99)
Fixed exactly the same bug as before, but this time for incremental
sync. Yay tests!
2017-05-11 15:51:35 +01:00
Robert Swain a821b9155f mediaapi/writers/utils: Reorder createFileWriter return parameters for consistency 2017-05-11 14:40:35 +02:00
Robert Swain 074e899000 mediaapi/writers/utils: Move and reuse createTempFileWriter 2017-05-11 14:39:40 +02:00
Robert Swain abcbb57aa1 mediaapi/writers/download: A bunch more refactoring 2017-05-11 14:31:24 +02:00
Robert Swain 6c3dbb9b48 mediaapi/writers/download: Factor out createRemoteRequest 2017-05-11 10:36:32 +02:00
Robert Swain ef7dc8c8c5 mediaapi/writers/download: Clarify log message that file is remote 2017-05-11 09:31:40 +02:00
Robert Swain 5e66ad6d39 mediaapi/writers/download: Comment the flow of respondFromRemoteFile 2017-05-11 09:30:10 +02:00
Robert Swain da28e4a3e2 mediaapi/writers/download: Update TODO comment that is done 2017-05-11 09:19:56 +02:00
Robert Swain 9c29a31e7e mediaapi/writers/download: Factor out respondFromRemoteFile 2017-05-11 09:19:34 +02:00
Robert Swain b6af021858 mediaapi/writers/download: Unlock activeRemoteRequests after too many tries 2017-05-11 09:18:02 +02:00