sigb.us/dendrite
1
0
Fork 0
Code Issues Pull requests Actions Packages Projects Releases 1 Wiki Activity

Update guides for running sytest (#958)

Browse source 
* Update guides for running sytest

Bring the details for running sytest against postgres up to date; prefer docker over manual setup

Signed-off-by: Alex Chen <minecnly@gmail.com>

* Better flags

Signed-off-by: Alex Chen <minecnly@gmail.com>
This commit is contained in:
Alex Chen 2020-04-12 20:48:24 +08:00 committed by GitHub
parent 317658acea
commit d359851708
No known key found for this signature in database
GPG key ID: 4AEE18F83AFDEB23
1 changed files with 47 additions and 22 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

69
docs/sytest.md
View file

@ -10,9 +10,9 @@ passes.
## Finding out which tests to add ## Finding out which tests to add
We recommend you run the tests locally by manually setting up SyTest or using a We recommend you run the tests locally by using the SyTest docker image or
SyTest docker image. After running the tests, a script will print the tests you manually setting up SyTest. After running the tests, a script will print the
need to add to `sytest-whitelist`. tests you need to add to `sytest-whitelist`.
You should proceed after you see no build problems for dendrite after running: You should proceed after you see no build problems for dendrite after running:
@ -20,9 +20,32 @@ You should proceed after you see no build problems for dendrite after running:
./build.sh ./build.sh
``` ```
### Using the SyTest Docker image
Use the following commands to pull the latest SyTest image and run the tests:
```sh
docker pull matrixdotorg/sytest-dendrite
docker run --rm -v /path/to/dendrite/:/src/ -v /path/to/log/output/:/logs/ matrixdotorg/sytest-dendrite
```
`/path/to/dendrite/` should be replaced with the actual path to your dendrite
source code. The test results TAP file and homeserver logging output will go to
`/path/to/log/output`. The output of the command should tell you if you need to
add any tests to `sytest-whitelist`.
When debugging, the following Docker `run` options may also be useful:
* `-v /path/to/sytest/:/sytest/`: Use your local SyTest repository at
`/path/to/sytest` instead of pulling from GitHub. This is useful when you want
to speed things up or make modifications to SyTest.
* `--entrypoint bash`: Prevent the container from automatically starting the
tests. When used, you need to manually run `/bootstrap.sh dendrite` inside
the container to start them.
### Manually Setting up SyTest ### Manually Setting up SyTest
Make sure you have Perl v5+ installed, and get SyTest with: If you don't want to use the Docker image, you can also run SyTest by hand. Make
sure you have Perl 5 or above, and get SyTest with:
(Note that this guide assumes your SyTest checkout is next to your (Note that this guide assumes your SyTest checkout is next to your
`dendrite` checkout.) `dendrite` checkout.)
@ -37,12 +60,23 @@ Set up the database:
```sh ```sh
sudo -u postgres psql -c "CREATE USER dendrite PASSWORD 'itsasecret'" sudo -u postgres psql -c "CREATE USER dendrite PASSWORD 'itsasecret'"
sudo -u postgres psql -c "CREATE DATABASE sytest_template OWNER dendrite" for i in dendrite0 dendrite1 sytest_template; do sudo -u postgres psql -c "CREATE DATABASE $i OWNER dendrite;"; done
mkdir -p "server-0" mkdir -p "server-0"
cat > "server-0/database.yaml" << EOF cat > "server-0/database.yaml" << EOF
args: args:
user: dendrite user: dendrite
database: dendrite password: itsasecret
database: dendrite0
host: 127.0.0.1
sslmode: disable
type: pg
EOF
mkdir -p "server-1"
cat > "server-1/database.yaml" << EOF
args:
user: dendrite
password: itsasecret
database: dendrite1
host: 127.0.0.1 host: 127.0.0.1
sslmode: disable sslmode: disable
type: pg type: pg
@ -52,29 +86,20 @@ EOF
Run the tests: Run the tests:
```sh ```sh
./run-tests.pl -I Dendrite::Monolith -d ../dendrite/bin -W ../dendrite/sytest-whitelist -O tap --all | tee results.tap POSTGRES=1 ./run-tests.pl -I Dendrite::Monolith -d ../dendrite/bin -W ../dendrite/sytest-whitelist -O tap --all | tee results.tap
``` ```
where `tee` lets you see the results while they're being piped to the file. where `tee` lets you see the results while they're being piped to the file, and
`POSTGRES=1` enables testing with PostgeSQL. If the `POSTGRES` environment
variable is not set or is set to 0, SyTest will fall back to SQLite 3. For more
flags and options, see https://github.com/matrix-org/sytest#running.
Once the tests are complete, run the helper script to see if you need to add Once the tests are complete, run the helper script to see if you need to add
any newly passing test names to `sytest-whitelist` in the project's root directory: any newly passing test names to `sytest-whitelist` in the project's root
directory:
```sh ```sh
../dendrite/show-expected-fail-tests.sh results.tap ../dendrite/sytest-whitelist ../dendrite/sytest-blacklist ../dendrite/show-expected-fail-tests.sh results.tap ../dendrite/sytest-whitelist ../dendrite/sytest-blacklist
``` ```
If the script prints nothing/exits with 0, then you're good to go. If the script prints nothing/exits with 0, then you're good to go.
### Using a SyTest Docker image
Ensure you have the latest image for SyTest, then run the tests:
```sh
docker pull matrixdotorg/sytest-dendrite
docker run --rm -v /path/to/dendrite/:/src/ matrixdotorg/sytest-dendrite
```
where `/path/to/dendrite/` should be replaced with the actual path to your
dendrite source code. The output should tell you if you need to add any tests to
`sytest-whitelist`.