Commit graph

253 commits

Author SHA1 Message Date
Robert Swain 1d3d9689df Merge cd7e04ec0d into db23640f69 2017-05-26 07:32:09 +00:00
Kegsay db23640f69 cmd: Add create test accounts command (#119) 2017-05-25 17:41:45 +01:00
Mark Haines 84ad4ff9f6 Start implementing /join for room aliases for rooms the server is not in. (#115)
* Start implementing the join room API

* Hacks to get join room working

* Make the TLS fingerprint configurable

* Fix the client API proxy to handle '#' correctly

* Return a 200 OK response

* Write the join event along with current state to the room server

* Comment on the error handling

* Fix typos

* Fix tab

* Add TODO for moving authEventIDs to gomatrixserverlib

* Comment on why we ignore the key ID argument for local keys

* Avoid 'preceeded'

* Neaten the control flow

* Neaten the control flow even more

* Return pointers

* Rename err to lastErr for the loop
2017-05-25 16:08:28 +01:00
Kegsay 445dce14ae Add in devices_table to store device information (#118) 2017-05-25 13:33:50 +01:00
Mark Haines e6835660b0 Update gomatrixserverlib 2017-05-24 15:05:42 +01:00
Kegsay 3b9222e8f7 Glue together devices and auth with the current HTTP code (#117)
- Renamed `clientapi/auth/types` to `clientapi/auth/authtypes` for the same
  horrible namespace clashing reasons as `storage`.
- Factored out `makeAPI` to `common`.
- Added in `makeAuthAPI`.
2017-05-23 17:43:05 +01:00
Mark Haines 309300a744 Call 'go build' from the pre-commit hook to check that everything builds (#116) 2017-05-23 09:12:59 +01:00
Robert Swain cd7e04ec0d mediaapi/writers/download: Move comment next to code to which it refers 2017-05-22 21:06:14 +02:00
Robert Swain f88c537f43 mediaapi/writers/download: Separate 404 and 500 due to db query failure 2017-05-22 21:03:57 +02:00
Robert Swain 3d8aec28b1 mediaapi/writers/download: Remove unnecessary retry loop 2017-05-22 20:54:03 +02:00
Mark Haines a7acfa5546 Fix build after #114 2017-05-22 19:28:26 +01:00
Mark Haines e756b3b0eb Update gomatrixserverlib 2017-05-22 18:11:30 +01:00
Kegsay d63a1ddc7c Rename packages under /auth (#114)
Previously, all database stuff was under the helpfully named
package 'storage'. However, this convention is used throughout all
of dendrite, which will clash the moment we want to add auth to all
the CS API endpoints. To prevent the package name clash, add
sub-directories which represent what is being stored so the final
usage ends up being:

```
func doThing(db *storage.SyncServerDatabase, authDB *accounts.Database)
{
    // ...
}
```
2017-05-22 16:49:32 +01:00
Robert Swain 5e338cc805 mediaapi/writers/download: Fix reference to spec for /download 2017-05-22 17:08:54 +02:00
Robert Swain 48c3a505cd mediaapi/writers/upload: Use Logger and MediaMetadata from uploadRequest
Also make storeFileAndMetadata a method on uploadRequest.
2017-05-22 17:05:45 +02:00
Kegsay 0325459e7f Implement dummy registration and hook it up to AccountDatabase (#113) 2017-05-22 15:55:39 +01:00
Robert Swain 8626308dac mediaapi/writers/download: getMatrixUrls -> getMatrixURLs 2017-05-22 16:41:46 +02:00
Robert Swain 0fcf74014d mediaapi/fileutils: Improve function name and API docs 2017-05-22 16:41:07 +02:00
Robert Swain f6f33e8068 mediaapi/fileutils: Fix missing error check lost-in-refactoring 2017-05-22 16:24:41 +02:00
Robert Swain ac356721e6 mediaapi/fileutils: Rename and simplify documentation of WriteTempFile 2017-05-22 16:24:08 +02:00
Robert Swain 227fe64252 mediaapi/writers: Pass used member instead of entire struct 2017-05-22 15:38:51 +02:00
Robert Swain 0ecb645f27 mediaapi/fileutils: Move logic from function to call site
Makes for better readability. The only reason it was inside the function
was to avoid being too complex for gocyclo checks.
2017-05-22 15:21:56 +02:00
Robert Swain 6d000794ec mediaapi/fileutils: Fix and simplify API doc 2017-05-22 15:16:53 +02:00
Robert Swain 703685f0dd mediaapi/storage: Correct comment about Database 2017-05-22 15:13:35 +02:00
Robert Swain 2f6db5d317 mediaapi/storage: StoreMediaMetadata error if origin and id not unique in db 2017-05-22 15:11:28 +02:00
Robert Swain 2004dfcfcf mediaapi: ContentLength -> FileSizeBytes 2017-05-22 15:07:45 +02:00
Robert Swain 1e639b0868 mediaapi/config: Document that MaxFileSizeBytes 0 means unlimited 2017-05-22 14:23:46 +02:00
Robert Swain 6fe6f45093 mediaapi/writers: Remove unnecessary comparison to true 2017-05-22 14:13:57 +02:00
Robert Swain 8684f80ebd mediaapi/fileutils: Move fileutils to its own package 2017-05-22 14:12:11 +02:00
Robert Swain 7bdcece102 mediaapi/writers/fileutils: Group error variables 2017-05-22 14:02:08 +02:00
Robert Swain 73b419a126 mediaapi/writers: Factor out common moveFileWithHashCheck functionality 2017-05-22 12:20:07 +02:00
Robert Swain 5d4b5ef1bc mediaapi/writers/download: Add Base64Hash to useful log messages 2017-05-22 12:19:06 +02:00
Robert Swain 01c565ddfb mediaapi/writers/fileutils: Store files based on hash, not media ID
This avoids having to sanitize the origin and media ID for files from
remote servers. It also allows us to deduplicate files across all files
uploaded to this homeserver or downloaded from remote homeservers.
2017-05-22 10:34:56 +02:00
Robert Swain 979ce964ab Merge branch 'master' into rob/media-upload 2017-05-22 10:28:37 +02:00
Robert Swain 8f7ce9adc0 mediaapi/writers/upload: Add note about Content-Disposition override 2017-05-22 10:27:48 +02:00
Robert Swain 3cea54db0b mediaapi/writers/download: Simplify user error message
They already known the origin and media ID so it is redundant.
2017-05-22 10:27:02 +02:00
Robert Swain 5f604cc41f mediaapi/writers/upload: Infof -> Info as no formatting in string 2017-05-22 10:26:30 +02:00
Robert Swain 5536fec902 mediaapi/writers: Add base64hash to media_repository table
A SHA-256 hash sum in golang base64 URLEncoding format (contains only
URL-safe characters) is now calculated and stored for every file
transferred to this server.

Uploads to the server use this hash as the MediaID. Downloads from
remote servers retain their MediaID from the remote server, but can use
the hash for local deduplication and integrity checking purposes.
2017-05-22 10:24:03 +02:00
Robert Swain 370cb74d2d mediaapi/writers: Reuse same writer code for upload and download
This now calculates a hash for downloads from remote servers as well as
uploads to this server.
2017-05-22 10:19:52 +02:00
Robert Swain 9af66a1963 mediaapi/writers: Reuse removeDir throughout the package 2017-05-22 10:13:37 +02:00
Mark Haines 6605333f6f Start implementing the federation server keys API. (#112)
* Start implementing the federation server keys API.

* Fix copyright

* Fix comments

* Comment on the key format

* Better explain what the ValidityPeriod is

* Return a 200 status code
2017-05-19 16:06:41 +01:00
Robert Swain 318531d011 mediaapi/writers/upload: Make assign in-line in if 2017-05-19 12:27:55 +02:00
Robert Swain 86cb8e32f7 mediaapi/writers/upload: Clarify order of moving file and storing metadata 2017-05-19 12:26:27 +02:00
Robert Swain 1242fdba22 mediaapi: Improve logging throughout, leveraging logrus features 2017-05-19 12:21:10 +02:00
Mark Haines aa179d451c Update version of gomatrixserverlib (#111) 2017-05-19 10:46:17 +01:00
Robert Swain 5d5f156500 mediaapi/writers/fileutils: Rework file path layout
From experience with synapse, splitting the files into subdirectories
based on the beginnings of the filenames helps with browsability. As we
are using MediaIDs that are base64-encoded, each character has 64
possibilities, which is a nice upper limit on the number of
subdirectories in a directory in terms of browsing. We have two levels
of single character directories for added convenience, creating up to
4096 buckets.
2017-05-19 11:34:40 +02:00
Kegsay 9d4d18ae7f Add AccountDatabase for storing user accounts (#110)
Including the ability to add new accounts with a user/password and
select accounts using a user/password. Uses bcrypt to hash passwords.
2017-05-19 10:27:03 +01:00
Robert Swain 12b0cdde06 mediaapi/writers/upload: Explain the use of TeeReader 2017-05-19 11:01:44 +02:00
Robert Swain f7d11f87c1 mediaapi/writers/upload: Add comment about why we hash the file data 2017-05-19 10:59:12 +02:00
Robert Swain 3e5ac85ce1 mediaapi/writers/upload: Clarify TODO comment 2017-05-19 10:53:47 +02:00