Robert Swain
cd7e04ec0d
mediaapi/writers/download: Move comment next to code to which it refers
2017-05-22 21:06:14 +02:00
Robert Swain
f88c537f43
mediaapi/writers/download: Separate 404 and 500 due to db query failure
2017-05-22 21:03:57 +02:00
Robert Swain
3d8aec28b1
mediaapi/writers/download: Remove unnecessary retry loop
2017-05-22 20:54:03 +02:00
Robert Swain
5e338cc805
mediaapi/writers/download: Fix reference to spec for /download
2017-05-22 17:08:54 +02:00
Robert Swain
48c3a505cd
mediaapi/writers/upload: Use Logger and MediaMetadata from uploadRequest
...
Also make storeFileAndMetadata a method on uploadRequest.
2017-05-22 17:05:45 +02:00
Robert Swain
8626308dac
mediaapi/writers/download: getMatrixUrls -> getMatrixURLs
2017-05-22 16:41:46 +02:00
Robert Swain
0fcf74014d
mediaapi/fileutils: Improve function name and API docs
2017-05-22 16:41:07 +02:00
Robert Swain
f6f33e8068
mediaapi/fileutils: Fix missing error check lost-in-refactoring
2017-05-22 16:24:41 +02:00
Robert Swain
ac356721e6
mediaapi/fileutils: Rename and simplify documentation of WriteTempFile
2017-05-22 16:24:08 +02:00
Robert Swain
227fe64252
mediaapi/writers: Pass used member instead of entire struct
2017-05-22 15:38:51 +02:00
Robert Swain
0ecb645f27
mediaapi/fileutils: Move logic from function to call site
...
Makes for better readability. The only reason it was inside the function
was to avoid being too complex for gocyclo checks.
2017-05-22 15:21:56 +02:00
Robert Swain
6d000794ec
mediaapi/fileutils: Fix and simplify API doc
2017-05-22 15:16:53 +02:00
Robert Swain
703685f0dd
mediaapi/storage: Correct comment about Database
2017-05-22 15:13:35 +02:00
Robert Swain
2f6db5d317
mediaapi/storage: StoreMediaMetadata error if origin and id not unique in db
2017-05-22 15:11:28 +02:00
Robert Swain
2004dfcfcf
mediaapi: ContentLength -> FileSizeBytes
2017-05-22 15:07:45 +02:00
Robert Swain
1e639b0868
mediaapi/config: Document that MaxFileSizeBytes 0 means unlimited
2017-05-22 14:23:46 +02:00
Robert Swain
6fe6f45093
mediaapi/writers: Remove unnecessary comparison to true
2017-05-22 14:13:57 +02:00
Robert Swain
8684f80ebd
mediaapi/fileutils: Move fileutils to its own package
2017-05-22 14:12:11 +02:00
Robert Swain
7bdcece102
mediaapi/writers/fileutils: Group error variables
2017-05-22 14:02:08 +02:00
Robert Swain
73b419a126
mediaapi/writers: Factor out common moveFileWithHashCheck functionality
2017-05-22 12:20:07 +02:00
Robert Swain
5d4b5ef1bc
mediaapi/writers/download: Add Base64Hash to useful log messages
2017-05-22 12:19:06 +02:00
Robert Swain
01c565ddfb
mediaapi/writers/fileutils: Store files based on hash, not media ID
...
This avoids having to sanitize the origin and media ID for files from
remote servers. It also allows us to deduplicate files across all files
uploaded to this homeserver or downloaded from remote homeservers.
2017-05-22 10:34:56 +02:00
Robert Swain
979ce964ab
Merge branch 'master' into rob/media-upload
2017-05-22 10:28:37 +02:00
Robert Swain
8f7ce9adc0
mediaapi/writers/upload: Add note about Content-Disposition override
2017-05-22 10:27:48 +02:00
Robert Swain
3cea54db0b
mediaapi/writers/download: Simplify user error message
...
They already known the origin and media ID so it is redundant.
2017-05-22 10:27:02 +02:00
Robert Swain
5f604cc41f
mediaapi/writers/upload: Infof -> Info as no formatting in string
2017-05-22 10:26:30 +02:00
Robert Swain
5536fec902
mediaapi/writers: Add base64hash to media_repository table
...
A SHA-256 hash sum in golang base64 URLEncoding format (contains only
URL-safe characters) is now calculated and stored for every file
transferred to this server.
Uploads to the server use this hash as the MediaID. Downloads from
remote servers retain their MediaID from the remote server, but can use
the hash for local deduplication and integrity checking purposes.
2017-05-22 10:24:03 +02:00
Robert Swain
370cb74d2d
mediaapi/writers: Reuse same writer code for upload and download
...
This now calculates a hash for downloads from remote servers as well as
uploads to this server.
2017-05-22 10:19:52 +02:00
Robert Swain
9af66a1963
mediaapi/writers: Reuse removeDir throughout the package
2017-05-22 10:13:37 +02:00
Mark Haines
6605333f6f
Start implementing the federation server keys API. ( #112 )
...
* Start implementing the federation server keys API.
* Fix copyright
* Fix comments
* Comment on the key format
* Better explain what the ValidityPeriod is
* Return a 200 status code
2017-05-19 16:06:41 +01:00
Robert Swain
318531d011
mediaapi/writers/upload: Make assign in-line in if
2017-05-19 12:27:55 +02:00
Robert Swain
86cb8e32f7
mediaapi/writers/upload: Clarify order of moving file and storing metadata
2017-05-19 12:26:27 +02:00
Robert Swain
1242fdba22
mediaapi: Improve logging throughout, leveraging logrus features
2017-05-19 12:21:10 +02:00
Mark Haines
aa179d451c
Update version of gomatrixserverlib ( #111 )
2017-05-19 10:46:17 +01:00
Robert Swain
5d5f156500
mediaapi/writers/fileutils: Rework file path layout
...
From experience with synapse, splitting the files into subdirectories
based on the beginnings of the filenames helps with browsability. As we
are using MediaIDs that are base64-encoded, each character has 64
possibilities, which is a nice upper limit on the number of
subdirectories in a directory in terms of browsing. We have two levels
of single character directories for added convenience, creating up to
4096 buckets.
2017-05-19 11:34:40 +02:00
Kegsay
9d4d18ae7f
Add AccountDatabase for storing user accounts ( #110 )
...
Including the ability to add new accounts with a user/password and
select accounts using a user/password. Uses bcrypt to hash passwords.
2017-05-19 10:27:03 +01:00
Robert Swain
12b0cdde06
mediaapi/writers/upload: Explain the use of TeeReader
2017-05-19 11:01:44 +02:00
Robert Swain
f7d11f87c1
mediaapi/writers/upload: Add comment about why we hash the file data
2017-05-19 10:59:12 +02:00
Robert Swain
3e5ac85ce1
mediaapi/writers/upload: Clarify TODO comment
2017-05-19 10:53:47 +02:00
Robert Swain
cdd4222e45
mediaapi/writers/fileutils: Return errors to log using request context
2017-05-19 10:46:50 +02:00
Robert Swain
5dd90fbff3
mediaapi/writers/fileutils: Make note of further file path validation todo
2017-05-18 18:00:56 +02:00
Robert Swain
7af45e4664
mediaapi/writers/upload: Refactor Upload() into three new functions
2017-05-18 17:56:19 +02:00
Robert Swain
00e8fed3a7
mediaapi/writers: Add validation and error handling to getPathFromMediaMetadata
2017-05-18 17:39:30 +02:00
Robert Swain
10a2b2f8e6
mediaapi: Also rename all basePath variables to absBasePath for clarity
2017-05-18 17:37:32 +02:00
Robert Swain
995e1f2c99
cmd/dendrite-media-api-server: Make base path absolute
2017-05-18 17:25:12 +02:00
Robert Swain
f5422787a1
mediaapi/writers: Move single-value error return assignment into if
2017-05-18 16:02:43 +02:00
Robert Swain
9fc5abdb3f
mediaapi/writers: Rename utils.go to fileutils.go
...
Better reflects the content of the file.
2017-05-18 16:01:06 +02:00
Robert Swain
2e795ed8aa
mediaapi/storage: Improve GetMediaMetadata description
2017-05-18 15:57:07 +02:00
Robert Swain
1f2ac60bee
mediaapi/routing: Sync make() to makeAPI() as in clientapi
2017-05-18 15:53:48 +02:00
Robert Swain
04c4a2d05a
cmd/dendrite-media-api-server: Move os.Getenv() for consistency
2017-05-18 15:50:09 +02:00
