synapse/docker
kegsay 11a8ae0632
complement: enable dirty runs (#16520)
* complement: enable dirty runs

* Add changelog

* Set a low connpool limit when running in Complement

Dirty runs can cause many containers to be running concurrently,
which seems to easily exhaust resources on the host. The increased
speedup from dirty runs also seems to use more db connections on
workers, which are misconfigured currently to have
`SUM(workers * cp_max) > max_connections`, causing
```
FATAL:  sorry, too many clients already
```
which results in tests failing.

* Try p=2 concurrency to restrict slowness of servers which causes partial state join tests to flake

* Debug logging

* Only run flakey tests

* Only adjust connection pool limits in worker mode

* Move cp vars to somewhere where they get executed in CI

* Move cp values back to where they actually work

* Debug logging

* Try p=1 to see if this makes worker mode happier

* Remove debug logging
2023-10-27 18:29:20 +01:00
..
complement complement: enable dirty runs (#16520) 2023-10-27 18:29:20 +01:00
conf complement: enable dirty runs (#16520) 2023-10-27 18:29:20 +01:00
conf-workers Unix Sockets for HTTP Replication (#15708) 2023-07-11 13:08:06 -05:00
Dockerfile Use bookwork as docker base image (#16324) 2023-09-15 13:14:10 +01:00
Dockerfile-dhvirtualenv Remove support for Python 3.7 (#15851) 2023-07-05 18:45:42 -05:00
Dockerfile-workers Use bookwork as docker base image (#16324) 2023-09-15 13:14:10 +01:00
README-testing.md Call out buildkit is required when building test docker images (#13338) 2022-07-21 14:29:58 +02:00
README.md Log when events are (unexpectedly) filtered out of responses in tests (#14213) 2023-06-01 21:27:18 -05:00
build_debian.sh Fix Shellcheck SC2006: Use $(...) notation 2021-10-22 23:08:55 +01:00
configure_workers_and_start.py Add `/notifications` endpoint to workers (#16265) 2023-09-07 09:26:07 +00:00
editable.Dockerfile Use bookwork as docker base image (#16324) 2023-09-15 13:14:10 +01:00
prefix-log Use supervisord to supervise Postgres and Caddy in the Complement image. (#12480) 2022-04-27 14:39:41 +01:00
start.py Update ruff config (#16283) 2023-09-08 11:24:36 -04:00

README.md

Synapse Docker

This Docker image will run Synapse as a single process. By default it uses a sqlite database; for production use you should connect it to a separate postgres database. The image also does not provide a TURN server.

This image should work on all platforms that are supported by Docker upstream. Note that Docker's WS1-backend Linux Containers on Windows platform is experimental and is not supported by this image.

Volumes

By default, the image expects a single volume, located at /data, that will hold:

  • configuration files;
  • uploaded media and thumbnails;
  • the SQLite database if you do not configure postgres;
  • the appservices configuration.

You are free to use separate volumes depending on storage endpoints at your disposal. For instance, /data/media could be stored on a large but low performance hdd storage while other files could be stored on high performance endpoints.

In order to setup an application service, simply create an appservices directory in the data volume and write the application service Yaml configuration file there. Multiple application services are supported.

Generating a configuration file

The first step is to generate a valid config file. To do this, you can run the image with the generate command line option.

You will need to specify values for the SYNAPSE_SERVER_NAME and SYNAPSE_REPORT_STATS environment variable, and mount a docker volume to store the configuration on. For example:

docker run -it --rm \
    --mount type=volume,src=synapse-data,dst=/data \
    -e SYNAPSE_SERVER_NAME=my.matrix.host \
    -e SYNAPSE_REPORT_STATS=yes \
    matrixdotorg/synapse:latest generate

For information on picking a suitable server name, see https://matrix-org.github.io/synapse/latest/setup/installation.html.

The above command will generate a homeserver.yaml in (typically) /var/lib/docker/volumes/synapse-data/_data. You should check this file, and customise it to your needs.

The following environment variables are supported in generate mode:

  • SYNAPSE_SERVER_NAME (mandatory): the server public hostname.
  • SYNAPSE_REPORT_STATS (mandatory, yes or no): whether to enable anonymous statistics reporting.
  • SYNAPSE_HTTP_PORT: the port Synapse should listen on for http traffic. Defaults to 8008.
  • SYNAPSE_CONFIG_DIR: where additional config files (such as the log config and event signing key) will be stored. Defaults to /data.
  • SYNAPSE_CONFIG_PATH: path to the file to be generated. Defaults to <SYNAPSE_CONFIG_DIR>/homeserver.yaml.
  • SYNAPSE_DATA_DIR: where the generated config will put persistent data such as the database and media store. Defaults to /data.
  • UID, GID: the user id and group id to use for creating the data directories. If unset, and no user is set via docker run --user, defaults to 991, 991.
  • SYNAPSE_LOG_LEVEL: the log level to use (one of DEBUG, INFO, WARNING or ERROR). Defaults to INFO.
  • SYNAPSE_LOG_SENSITIVE: if set and the log level is set to DEBUG, Synapse will log sensitive information such as access tokens. This should not be needed unless you are a developer attempting to debug something particularly tricky.
  • SYNAPSE_LOG_TESTING: if set, Synapse will log additional information useful for testing.

Postgres

By default the config will use SQLite. See the docs on using Postgres for more info on how to use Postgres. Until this section is improved this issue may provide useful information.

Running synapse

Once you have a valid configuration file, you can start synapse as follows:

docker run -d --name synapse \
    --mount type=volume,src=synapse-data,dst=/data \
    -p 8008:8008 \
    matrixdotorg/synapse:latest

(assuming 8008 is the port Synapse is configured to listen on for http traffic.)

You can then check that it has started correctly with:

docker logs synapse

If all is well, you should now be able to connect to http://localhost:8008 and see a confirmation message.

The following environment variables are supported in run mode:

  • SYNAPSE_CONFIG_DIR: where additional config files are stored. Defaults to /data.
  • SYNAPSE_CONFIG_PATH: path to the config file. Defaults to <SYNAPSE_CONFIG_DIR>/homeserver.yaml.
  • SYNAPSE_WORKER: module to execute, used when running synapse with workers. Defaults to synapse.app.homeserver, which is suitable for non-worker mode.
  • UID, GID: the user and group id to run Synapse as. If unset, and no user is set via docker run --user, defaults to 991, 991. Note that this user must have permission to read the config files, and write to the data directories.
  • TZ: the timezone the container will run with. Defaults to UTC.

For more complex setups (e.g. for workers) you can also pass your args directly to synapse using run mode. For example like this:

docker run -d --name synapse \
    --mount type=volume,src=synapse-data,dst=/data \
    -p 8008:8008 \
    matrixdotorg/synapse:latest run \
    -m synapse.app.generic_worker \
    --config-path=/data/homeserver.yaml \
    --config-path=/data/generic_worker.yaml

If you do not provide -m, the value of the SYNAPSE_WORKER environment variable is used. If you do not provide at least one --config-path or -c, the value of the SYNAPSE_CONFIG_PATH environment variable is used instead.

Generating an (admin) user

After synapse is running, you may wish to create a user via register_new_matrix_user.

This requires a registration_shared_secret to be set in your config file. Synapse must be restarted to pick up this change.

You can then call the script:

docker exec -it synapse register_new_matrix_user http://localhost:8008 -c /data/homeserver.yaml --help

Remember to remove the registration_shared_secret and restart if you no-longer need it.

TLS support

The default configuration exposes a single HTTP port: http://localhost:8008. It is suitable for local testing, but for any practical use, you will either need to use a reverse proxy, or configure Synapse to expose an HTTPS port.

For documentation on using a reverse proxy, see https://github.com/matrix-org/synapse/blob/master/docs/reverse_proxy.md.

For more information on enabling TLS support in synapse itself, see https://matrix-org.github.io/synapse/latest/setup/installation.html#tls-certificates. Of course, you will need to expose the TLS port from the container with a -p argument to docker run.

Legacy dynamic configuration file support

The docker image used to support creating a dynamic configuration file based on environment variables. This is no longer supported, and an error will be raised if you try to run synapse without a config file.

It is, however, possible to generate a static configuration file based on the environment variables that were previously used. To do this, run the docker container once with the environment variables set, and migrate_config command line option. For example:

docker run -it --rm \
    --mount type=volume,src=synapse-data,dst=/data \
    -e SYNAPSE_SERVER_NAME=my.matrix.host \
    -e SYNAPSE_REPORT_STATS=yes \
    matrixdotorg/synapse:latest migrate_config

This will generate the same configuration file as the legacy mode used, and will store it in /data/homeserver.yaml. You can then use it as shown above at Running synapse.

Note that the defaults used in this configuration file may be different to those when generating a new config file with generate: for example, TLS is enabled by default in this mode. You are encouraged to inspect the generated configuration file and edit it to ensure it meets your needs.

Building the image

If you need to build the image from a Synapse checkout, use the following docker build command from the repo's root:

DOCKER_BUILDKIT=1 docker build -t matrixdotorg/synapse -f docker/Dockerfile .

You can choose to build a different docker image by changing the value of the -f flag to point to another Dockerfile.

Disabling the healthcheck

If you are using a non-standard port or tls inside docker you can disable the healthcheck whilst running the above docker run commands.

   --no-healthcheck

Disabling the healthcheck in docker-compose file

If you wish to disable the healthcheck via docker-compose, append the following to your service configuration.

  healthcheck:
    disable: true

Setting custom healthcheck on docker run

If you wish to point the healthcheck at a different port with docker command, add the following

  --health-cmd 'curl -fSs http://localhost:1234/health'

Setting the healthcheck in docker-compose file

You can add the following to set a custom healthcheck in a docker compose file. You will need docker-compose version >2.1 for this to work.

healthcheck:
  test: ["CMD", "curl", "-fSs", "http://localhost:8008/health"]
  interval: 15s
  timeout: 5s
  retries: 3
  start_period: 5s

Using jemalloc

Jemalloc is embedded in the image and will be used instead of the default allocator. You can read about jemalloc by reading the Synapse Admin FAQ.