Skip to content

Commit

Permalink
Add the ability to set the log level using the `SYNAPSE_TEST_LOG_LEVE…
Browse files Browse the repository at this point in the history
…L` environment when using `complement.sh`. (matrix-org#13152)
  • Loading branch information
reivilibre authored Jul 5, 2022
1 parent e514495 commit 65e6755
Show file tree
Hide file tree
Showing 6 changed files with 44 additions and 6 deletions.
1 change: 1 addition & 0 deletions changelog.d/13152.misc
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
Add the ability to set the log level using the `SYNAPSE_TEST_LOG_LEVEL` environment when using `complement.sh`.
7 changes: 7 additions & 0 deletions docker/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,13 @@ The following environment variables are supported in `generate` mode:
* `UID`, `GID`: the user id and group id to use for creating the data
directories. If unset, and no user is set via `docker run --user`, defaults
to `991`, `991`.
* `SYNAPSE_LOG_LEVEL`: the log level to use (one of `DEBUG`, `INFO`, `WARNING` or `ERROR`).
Defaults to `INFO`.
* `SYNAPSE_LOG_SENSITIVE`: if set and the log level is set to `DEBUG`, Synapse
will log sensitive information such as access tokens.
This should not be needed unless you are a developer attempting to debug something
particularly tricky.


## Postgres

Expand Down
6 changes: 6 additions & 0 deletions docker/conf/log.config
Original file line number Diff line number Diff line change
Expand Up @@ -49,11 +49,17 @@ handlers:
class: logging.StreamHandler
formatter: precise

{% if not SYNAPSE_LOG_SENSITIVE %}
{#
If SYNAPSE_LOG_SENSITIVE is unset, then override synapse.storage.SQL to INFO
so that DEBUG entries (containing sensitive information) are not emitted.
#}
loggers:
synapse.storage.SQL:
# beware: increasing this to DEBUG will make synapse log sensitive
# information such as access tokens.
level: INFO
{% endif %}

root:
level: {{ SYNAPSE_LOG_LEVEL or "INFO" }}
Expand Down
20 changes: 14 additions & 6 deletions docker/configure_workers_and_start.py
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,10 @@
# * SYNAPSE_USE_EXPERIMENTAL_FORKING_LAUNCHER: Whether to use the forking launcher,
# only intended for usage in Complement at the moment.
# No stability guarantees are provided.
# * SYNAPSE_LOG_LEVEL: Set this to DEBUG, INFO, WARNING or ERROR to change the
# log level. INFO is the default.
# * SYNAPSE_LOG_SENSITIVE: If unset, SQL and SQL values won't be logged,
# regardless of the SYNAPSE_LOG_LEVEL setting.
#
# NOTE: According to Complement's ENTRYPOINT expectations for a homeserver image (as defined
# in the project's README), this script may be run multiple times, and functionality should
Expand All @@ -38,7 +42,7 @@
import subprocess
import sys
from pathlib import Path
from typing import Any, Dict, List, Mapping, MutableMapping, NoReturn, Set
from typing import Any, Dict, List, Mapping, MutableMapping, NoReturn, Optional, Set

import yaml
from jinja2 import Environment, FileSystemLoader
Expand Down Expand Up @@ -552,13 +556,17 @@ def generate_worker_log_config(
Returns: the path to the generated file
"""
# Check whether we should write worker logs to disk, in addition to the console
extra_log_template_args = {}
extra_log_template_args: Dict[str, Optional[str]] = {}
if environ.get("SYNAPSE_WORKERS_WRITE_LOGS_TO_DISK"):
extra_log_template_args["LOG_FILE_PATH"] = "{dir}/logs/{name}.log".format(
dir=data_dir, name=worker_name
)
extra_log_template_args["LOG_FILE_PATH"] = f"{data_dir}/logs/{worker_name}.log"

extra_log_template_args["SYNAPSE_LOG_LEVEL"] = environ.get("SYNAPSE_LOG_LEVEL")
extra_log_template_args["SYNAPSE_LOG_SENSITIVE"] = environ.get(
"SYNAPSE_LOG_SENSITIVE"
)

# Render and write the file
log_config_filepath = "/conf/workers/{name}.log.config".format(name=worker_name)
log_config_filepath = f"/conf/workers/{worker_name}.log.config"
convert(
"/conf/log.config",
log_config_filepath,
Expand Down
4 changes: 4 additions & 0 deletions docs/development/contributing_guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -309,6 +309,10 @@ The above will run a monolithic (single-process) Synapse with SQLite as the data
- Passing `POSTGRES=1` as an environment variable to use the Postgres database instead.
- Passing `WORKERS=1` as an environment variable to use a workerised setup instead. This option implies the use of Postgres.
To increase the log level for the tests, set `SYNAPSE_TEST_LOG_LEVEL`, e.g:
```sh
SYNAPSE_TEST_LOG_LEVEL=DEBUG COMPLEMENT_DIR=../complement ./scripts-dev/complement.sh -run TestImportHistoricalMessages
```
### Prettier formatting with `gotestfmt`
Expand Down
12 changes: 12 additions & 0 deletions scripts-dev/complement.sh
Original file line number Diff line number Diff line change
Expand Up @@ -145,6 +145,18 @@ else
test_tags="$test_tags,faster_joins"
fi


if [[ -n "$SYNAPSE_TEST_LOG_LEVEL" ]]; then
# Set the log level to what is desired
export PASS_SYNAPSE_LOG_LEVEL="$SYNAPSE_TEST_LOG_LEVEL"

# Allow logging sensitive things (currently SQL queries & parameters).
# (This won't have any effect if we're not logging at DEBUG level overall.)
# Since this is just a test suite, this is fine and won't reveal anyone's
# personal information
export PASS_SYNAPSE_LOG_SENSITIVE=1
fi

# Run the tests!
echo "Images built; running complement"
cd "$COMPLEMENT_DIR"
Expand Down

0 comments on commit 65e6755

Please sign in to comment.