Robert Swain
7727a8c61e
cmd/dendrite-media-api-server: Add MAX_FILE_SIZE_BYTES configuration
2017-05-18 11:44:48 +02:00
Robert Swain
846aece163
mediaapi: MaxFileSize -> MaxFileSizeBytes
2017-05-18 11:36:26 +02:00
Robert Swain
35a0b5d2e9
cmd/dendrite-media-api-server: Add BASE_PATH configuration
2017-05-18 11:34:01 +02:00
Robert Swain
ff3009ffdd
cmd/dendrite-media-api-server: Add SERVER_NAME configuration
2017-05-18 11:32:30 +02:00
Robert Swain
deee6f84c7
mediaapi/writers/upload: Move file first as db is source of truth
...
The database is the source of truth. If we add the metadata to the
database and it succeeds, and then the file fails to be moved, we think
we have a file when we actually don't.
2017-05-18 11:10:41 +02:00
Robert Swain
3f904e1cdb
mediaapi/writer/upload: Remove unnecessary logic
2017-05-18 11:09:09 +02:00
Robert Swain
f28235c05d
mediaapi/writers/upload: Factor out removeDir
...
Reduces complexity of Upload. Note that we never care about the error
from os.RemoveAll() beyond logging as we are already in an error case.
2017-05-18 11:07:03 +02:00
Robert Swain
5348b64edc
mediaapi/writers/download: Reduce complexity of copyToActiveAndPassive
2017-05-18 10:17:11 +02:00
Robert Swain
b80d5ab919
cmd/mediaapi-integration-tests: Test downloading same file 100 times
...
Spawns a GET request for the same file in 100 parallel go routines and
prints the body (which is some error JSON) in case of not 200 OK. Also
prints the number of successful requests.
This of course should take command line arguments for the URL and number
of requests but that can be done as soon as needed.
2017-05-18 09:12:01 +02:00
Robert Swain
8cf507f85f
mediaapi/writers: Never return server errors to user but log them
2017-05-18 09:04:36 +02:00
Robert Swain
eddc3e0789
mediaapi/writers/download: Log warnings about why we return 404
2017-05-18 08:57:29 +02:00
Robert Swain
c43aa20adc
mediaapi/writers/download: Remove duplicate log message
2017-05-18 08:56:54 +02:00
Kegsay
917c433fd2
Minor refactoring ( #106 )
...
- `s/Server/OutputRoomEvent/` in `consumers` to accurately reflect what is being consumed.
- `s/set/userIDSet/` in `notifier.go` for clarity.
- Removed lying comments.
2017-05-17 17:29:26 +01:00
Kegsay
c8c5a40ebd
Update README.md
2017-05-17 16:37:58 +01:00
Kegsay
0457a3bb69
State of the union update
2017-05-17 16:36:17 +01:00
Kegsay
ccd0eb2851
/sync bugfix: Check transitions to 'leave' do not leak events afterwards ( #105 )
2017-05-17 16:21:27 +01:00
Robert Swain
a8b7130745
mediaapi/writers/download: Clean up copyToActiveAndPassive
2017-05-17 16:39:01 +02:00
Kegsay
d5a44fd3e8
Only wake up /sync requests which the event is for ( #101 )
2017-05-17 15:38:24 +01:00
Robert Swain
9f8458716d
mediaapi/writers/download: Pass config members to respondFromRemoteFile
2017-05-17 16:37:11 +02:00
Robert Swain
19ffa4d75a
mediaapi/writers/download: respondFromLocalFile only needs basePath
2017-05-17 16:30:01 +02:00
Robert Swain
13b1051a3e
mediaapi/writers/download: Add logger to downloadRequest and use it
2017-05-17 16:24:30 +02:00
Robert Swain
6e24fb86cb
mediaapi/writer/download: Make functions into methods and use MediaMetadata
2017-05-17 16:13:54 +02:00
Robert Swain
937162a722
mediaapi/routing: Pass pointer to config struct instead of copying
2017-05-17 13:29:35 +02:00
Kegsay
0a3d44a80a
Handle room transitions to 'leave' in incremental /sync requests ( #104 )
...
* Add test for transition to leave
* Add test for join/leave in the same /sync response
2017-05-17 10:25:59 +01:00
Kegsay
675759c192
Add invites to /sync responses ( #103 )
2017-05-15 17:41:54 +01:00
Kegsay
94e1c62745
Remove intermediary /sync response struct ( #102 )
...
The logic required to populate the right bits of `RoomData` tends towards
the complete `/sync` response struct, so just use the actual response struct
and save the hassle of mapping between the two. It may not make much difference
in its current form, but the next PR will make use of this.
This PR has no functional changes.
2017-05-15 15:18:08 +01:00
Kegsay
cc7117392f
Send the entire room state down when transitioning to 'join' on a /sync response ( #100 )
...
This is only 'mostly' correct currently, because what should be no-op dupe
joins will actually trigger the entire room state to be re-sent.
Bizarrely, it's significantly easier to just do that than work out if we should,
and there are no client-visible effects to doing so, so we just do it for now.
2017-05-12 16:56:17 +01:00
Kegsay
2a9abefd92
Add more syncserver tests ( #99 )
...
Fixed exactly the same bug as before, but this time for incremental
sync. Yay tests!
2017-05-11 15:51:35 +01:00
Robert Swain
a821b9155f
mediaapi/writers/utils: Reorder createFileWriter return parameters for consistency
2017-05-11 14:40:35 +02:00
Robert Swain
074e899000
mediaapi/writers/utils: Move and reuse createTempFileWriter
2017-05-11 14:39:40 +02:00
Robert Swain
abcbb57aa1
mediaapi/writers/download: A bunch more refactoring
2017-05-11 14:31:24 +02:00
Robert Swain
6c3dbb9b48
mediaapi/writers/download: Factor out createRemoteRequest
2017-05-11 10:36:32 +02:00
Robert Swain
ef7dc8c8c5
mediaapi/writers/download: Clarify log message that file is remote
2017-05-11 09:31:40 +02:00
Robert Swain
5e66ad6d39
mediaapi/writers/download: Comment the flow of respondFromRemoteFile
2017-05-11 09:30:10 +02:00
Robert Swain
da28e4a3e2
mediaapi/writers/download: Update TODO comment that is done
2017-05-11 09:19:56 +02:00
Robert Swain
9c29a31e7e
mediaapi/writers/download: Factor out respondFromRemoteFile
2017-05-11 09:19:34 +02:00
Robert Swain
b6af021858
mediaapi/writers/download: Unlock activeRemoteRequests after too many tries
2017-05-11 09:18:02 +02:00
Robert Swain
f0c6ec23c9
mediaapi/writers/download: Move log messages to correct code paths
2017-05-11 09:17:37 +02:00
Robert Swain
563330b82a
mediaapi/writers/download: Rename attempts to tries
...
Tries is more usual language in programming.
2017-05-11 09:16:20 +02:00
Kegsay
a506985a66
Allow multiple sync server test requests ( #98 )
...
- Test data for the sync server is now in its own file.
- Rejig the sync server tests to support multiple /sync requests and corresponding
assertions.
- Fixed a minor bug which resulted in state events to appear twice in /sync
responses when syncing without a `since` parameter.
2017-05-10 17:48:35 +01:00
Robert Swain
304a275e1b
mediaapi/writers/download: Give remote files a placeholder UserID
2017-05-10 16:09:17 +02:00
Robert Swain
df52b1aef2
mediaapi/writers/download: Only try to write as much as was read
2017-05-10 16:08:51 +02:00
Robert Swain
8f68f61117
mediaapi/writers/download: Remove done TODOs
2017-05-10 16:08:32 +02:00
Robert Swain
5d67787c46
mediaapi/writers/download: Use consistent temporary file name
2017-05-10 16:07:18 +02:00
Robert Swain
49ec095b59
mediaapi/writers/download: Try to only request remote files once
...
If multiple requests arrive for the same remote file, we want to
download them once and then serve to all the remaining incoming requests
from the cache.
The main thing missing from the code at this point is a mechanism to
time out database queries. They are made across a network and so we
should be robust to network connectivity issues. This is a general
problem across dendrite and not limited to just this code.
2017-05-10 16:04:39 +02:00
Robert Swain
b280722591
mediaapi/writers/download: Factor out respondFromLocalFile()
2017-05-10 16:02:56 +02:00
Robert Swain
412f408ea0
mediaapi/writers/utils: Improve debug logging of moveFile()
2017-05-10 16:01:27 +02:00
Robert Swain
da7feaf741
Merge branch 'master' into rob/media-upload
2017-05-10 16:00:08 +02:00
Robert Swain
cdd2617f8e
Merge pull request #97 from matrix-org/rob/util-logging-context-fix
...
Update github.com/matrix-org/util for request context fix for logging
2017-05-10 15:56:49 +02:00
Robert Swain
042d636e34
Update github.com/matrix-org/util for request context fix for logging
2017-05-10 15:43:58 +02:00
