Skip to content

Commit

Permalink
Merge tag 'v1.37.0rc1' into develop
Browse files Browse the repository at this point in the history
Synapse 1.37.0rc1 (2021-06-24)
==============================

This release deprecates the current spam checker interface. See the [upgrade notes](https://matrix-org.github.io/synapse/develop/upgrade#deprecation-of-the-current-spam-checker-interface) for more information on how to update to the new generic module interface.

This release also removes support for fetching and renewing TLS certificates using the ACME v1 protocol, which has been fully decommissioned by Let's Encrypt on June 1st 2021. Admins previously using this feature should use a [reverse proxy](https://matrix-org.github.io/synapse/develop/reverse_proxy.html) to handle TLS termination, or use an external ACME client (such as [certbot](https://certbot.eff.org/)) to retrieve a certificate and key and provide them to Synapse using the `tls_certificate_path` and `tls_private_key_path` configuration settings.

Features
--------

- Implement "room knocking" as per [MSC2403](matrix-org/matrix-spec-proposals#2403). Contributed by @Sorunome and anoa. ([\matrix-org#6739](matrix-org#6739), [\matrix-org#9359](matrix-org#9359), [\matrix-org#10167](matrix-org#10167), [\matrix-org#10212](matrix-org#10212), [\matrix-org#10227](matrix-org#10227))
- Add experimental support for backfilling history into rooms ([MSC2716](matrix-org/matrix-spec-proposals#2716)). ([\matrix-org#9247](matrix-org#9247))
- Implement a generic interface for third-party plugin modules. ([\matrix-org#10062](matrix-org#10062), [\matrix-org#10206](matrix-org#10206))
- Implement config option `sso.update_profile_information` to sync SSO users' profile information with the identity provider each time they login. Currently only displayname is supported. ([\matrix-org#10108](matrix-org#10108))
- Ensure that errors during startup are written to the logs and the console. ([\matrix-org#10191](matrix-org#10191))

Bugfixes
--------

- Fix a bug introduced in Synapse v1.25.0 that prevented the `ip_range_whitelist` configuration option from working for federation and identity servers. Contributed by @mikure. ([\matrix-org#10115](matrix-org#10115))
- Remove a broken import line in Synapse's `admin_cmd` worker. Broke in Synapse v1.33.0. ([\matrix-org#10154](matrix-org#10154))
- Fix a bug introduced in Synapse v1.21.0 which could cause `/sync` to return immediately with an empty response. ([\matrix-org#10157](matrix-org#10157), [\matrix-org#10158](matrix-org#10158))
- Fix a minor bug in the response to `/_matrix/client/r0/user/{user}/openid/request_token` causing `expires_in` to be a float instead of an integer. Contributed by @lukaslihotzki. ([\matrix-org#10175](matrix-org#10175))
- Always require users to re-authenticate for dangerous operations: deactivating an account, modifying an account password, and adding 3PIDs. ([\matrix-org#10184](matrix-org#10184))
- Fix a bug introduced in Synpase v1.7.2 where remote server count metrics collection would be incorrectly delayed on startup. Found by @heftig. ([\matrix-org#10195](matrix-org#10195))
- Fix a bug introduced in Synapse v1.35.1 where an `allow` key of a `m.room.join_rules` event could be applied for incorrect room versions and configurations. ([\matrix-org#10208](matrix-org#10208))
- Fix performance regression in responding to user key requests over federation. Introduced in Synapse v1.34.0rc1. ([\matrix-org#10221](matrix-org#10221))

Improved Documentation
----------------------

- Add a new guide to decoding request logs. ([\matrix-org#8436](matrix-org#8436))
- Mention in the sample homeserver config that you may need to configure max upload size in your reverse proxy. Contributed by @aaronraimist. ([\matrix-org#10122](matrix-org#10122))
- Fix broken links in documentation. ([\matrix-org#10180](matrix-org#10180))
- Deploy a snapshot of the documentation website upon each new Synapse release. ([\matrix-org#10198](matrix-org#10198))

Deprecations and Removals
-------------------------

- The current spam checker interface is deprecated in favour of a new generic modules system. See the [upgrade notes](https://matrix-org.github.io/synapse/develop/upgrade#deprecation-of-the-current-spam-checker-interface) for more information on how to update to the new system. ([\matrix-org#10062](matrix-org#10062), [\matrix-org#10210](matrix-org#10210), [\matrix-org#10238](matrix-org#10238))
- Stop supporting the unstable spaces prefixes from MSC1772. ([\matrix-org#10161](matrix-org#10161))
- Remove Synapse's support for automatically fetching and renewing certificates using the ACME v1 protocol. This protocol has been fully turned off by Let's Encrypt for existing installations on June 1st 2021. Admins previously using this feature should use a [reverse proxy](https://matrix-org.github.io/synapse/develop/reverse_proxy.html) to handle TLS termination, or use an external ACME client (such as [certbot](https://certbot.eff.org/)) to retrieve a certificate and key and provide them to Synapse using the `tls_certificate_path` and `tls_private_key_path` configuration settings. ([\matrix-org#10194](matrix-org#10194))

Internal Changes
----------------

- Update the database schema versioning to support gradual migration away from legacy tables. ([\matrix-org#9933](matrix-org#9933))
- Add type hints to the federation servlets. ([\matrix-org#10080](matrix-org#10080))
- Improve OpenTracing for event persistence. ([\matrix-org#10134](matrix-org#10134), [\matrix-org#10193](matrix-org#10193))
- Clean up the interface for injecting OpenTracing over HTTP. ([\matrix-org#10143](matrix-org#10143))
- Limit the number of in-flight `/keys/query` requests from a single device. ([\matrix-org#10144](matrix-org#10144))
- Refactor EventPersistenceQueue. ([\matrix-org#10145](matrix-org#10145))
- Document `SYNAPSE_TEST_LOG_LEVEL` to see the logger output when running tests. ([\matrix-org#10148](matrix-org#10148))
- Update the Complement build tags in GitHub Actions to test currently experimental features. ([\matrix-org#10155](matrix-org#10155))
- Add a `synapse_federation_soft_failed_events_total` metric to track how often events are soft failed. ([\matrix-org#10156](matrix-org#10156))
- Fetch the corresponding complement branch when performing CI. ([\matrix-org#10160](matrix-org#10160))
- Add some developer documentation about boolean columns in database schemas. ([\matrix-org#10164](matrix-org#10164))
- Add extra logging fields to better debug where events are being soft failed. ([\matrix-org#10168](matrix-org#10168))
- Add debug logging for when we enter and exit `Measure` blocks. ([\matrix-org#10183](matrix-org#10183))
- Improve comments in structured logging code. ([\matrix-org#10188](matrix-org#10188))
- Update [MSC3083](matrix-org/matrix-spec-proposals#3083) support with modifications from the MSC. ([\matrix-org#10189](matrix-org#10189))
- Remove redundant DNS lookup limiter. ([\matrix-org#10190](matrix-org#10190))
- Upgrade `black` linting tool to 21.6b0. ([\matrix-org#10197](matrix-org#10197))
- Expose OpenTracing trace id in response headers. ([\matrix-org#10199](matrix-org#10199))
  • Loading branch information
babolivier committed Jun 24, 2021
2 parents 7f25d73 + bb472f3 commit 763dba7
Show file tree
Hide file tree
Showing 50 changed files with 155 additions and 53 deletions.
71 changes: 71 additions & 0 deletions CHANGES.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,74 @@
Synapse 1.37.0rc1 (2021-06-24)
==============================

This release deprecates the current spam checker interface. See the [upgrade notes](https://matrix-org.github.io/synapse/develop/upgrade#deprecation-of-the-current-spam-checker-interface) for more information on how to update to the new generic module interface.

This release also removes support for fetching and renewing TLS certificates using the ACME v1 protocol, which has been fully decommissioned by Let's Encrypt on June 1st 2021. Admins previously using this feature should use a [reverse proxy](https://matrix-org.github.io/synapse/develop/reverse_proxy.html) to handle TLS termination, or use an external ACME client (such as [certbot](https://certbot.eff.org/)) to retrieve a certificate and key and provide them to Synapse using the `tls_certificate_path` and `tls_private_key_path` configuration settings.


Features
--------

- Implement "room knocking" as per [MSC2403](https://github.com/matrix-org/matrix-doc/pull/2403). Contributed by @Sorunome and anoa. ([\#6739](https://github.com/matrix-org/synapse/issues/6739), [\#9359](https://github.com/matrix-org/synapse/issues/9359), [\#10167](https://github.com/matrix-org/synapse/issues/10167), [\#10212](https://github.com/matrix-org/synapse/issues/10212), [\#10227](https://github.com/matrix-org/synapse/issues/10227))
- Add experimental support for backfilling history into rooms ([MSC2716](https://github.com/matrix-org/matrix-doc/pull/2716)). ([\#9247](https://github.com/matrix-org/synapse/issues/9247))
- Implement a generic interface for third-party plugin modules. ([\#10062](https://github.com/matrix-org/synapse/issues/10062), [\#10206](https://github.com/matrix-org/synapse/issues/10206))
- Implement config option `sso.update_profile_information` to sync SSO users' profile information with the identity provider each time they login. Currently only displayname is supported. ([\#10108](https://github.com/matrix-org/synapse/issues/10108))
- Ensure that errors during startup are written to the logs and the console. ([\#10191](https://github.com/matrix-org/synapse/issues/10191))


Bugfixes
--------

- Fix a bug introduced in Synapse v1.25.0 that prevented the `ip_range_whitelist` configuration option from working for federation and identity servers. Contributed by @mikure. ([\#10115](https://github.com/matrix-org/synapse/issues/10115))
- Remove a broken import line in Synapse's `admin_cmd` worker. Broke in Synapse v1.33.0. ([\#10154](https://github.com/matrix-org/synapse/issues/10154))
- Fix a bug introduced in Synapse v1.21.0 which could cause `/sync` to return immediately with an empty response. ([\#10157](https://github.com/matrix-org/synapse/issues/10157), [\#10158](https://github.com/matrix-org/synapse/issues/10158))
- Fix a minor bug in the response to `/_matrix/client/r0/user/{user}/openid/request_token` causing `expires_in` to be a float instead of an integer. Contributed by @lukaslihotzki. ([\#10175](https://github.com/matrix-org/synapse/issues/10175))
- Always require users to re-authenticate for dangerous operations: deactivating an account, modifying an account password, and adding 3PIDs. ([\#10184](https://github.com/matrix-org/synapse/issues/10184))
- Fix a bug introduced in Synpase v1.7.2 where remote server count metrics collection would be incorrectly delayed on startup. Found by @heftig. ([\#10195](https://github.com/matrix-org/synapse/issues/10195))
- Fix a bug introduced in Synapse v1.35.1 where an `allow` key of a `m.room.join_rules` event could be applied for incorrect room versions and configurations. ([\#10208](https://github.com/matrix-org/synapse/issues/10208))
- Fix performance regression in responding to user key requests over federation. Introduced in Synapse v1.34.0rc1. ([\#10221](https://github.com/matrix-org/synapse/issues/10221))


Improved Documentation
----------------------

- Add a new guide to decoding request logs. ([\#8436](https://github.com/matrix-org/synapse/issues/8436))
- Mention in the sample homeserver config that you may need to configure max upload size in your reverse proxy. Contributed by @aaronraimist. ([\#10122](https://github.com/matrix-org/synapse/issues/10122))
- Fix broken links in documentation. ([\#10180](https://github.com/matrix-org/synapse/issues/10180))
- Deploy a snapshot of the documentation website upon each new Synapse release. ([\#10198](https://github.com/matrix-org/synapse/issues/10198))


Deprecations and Removals
-------------------------

- The current spam checker interface is deprecated in favour of a new generic modules system. See the [upgrade notes](https://matrix-org.github.io/synapse/develop/upgrade#deprecation-of-the-current-spam-checker-interface) for more information on how to update to the new system. ([\#10062](https://github.com/matrix-org/synapse/issues/10062), [\#10210](https://github.com/matrix-org/synapse/issues/10210), [\#10238](https://github.com/matrix-org/synapse/issues/10238))
- Stop supporting the unstable spaces prefixes from MSC1772. ([\#10161](https://github.com/matrix-org/synapse/issues/10161))
- Remove Synapse's support for automatically fetching and renewing certificates using the ACME v1 protocol. This protocol has been fully turned off by Let's Encrypt for existing installations on June 1st 2021. Admins previously using this feature should use a [reverse proxy](https://matrix-org.github.io/synapse/develop/reverse_proxy.html) to handle TLS termination, or use an external ACME client (such as [certbot](https://certbot.eff.org/)) to retrieve a certificate and key and provide them to Synapse using the `tls_certificate_path` and `tls_private_key_path` configuration settings. ([\#10194](https://github.com/matrix-org/synapse/issues/10194))


Internal Changes
----------------

- Update the database schema versioning to support gradual migration away from legacy tables. ([\#9933](https://github.com/matrix-org/synapse/issues/9933))
- Add type hints to the federation servlets. ([\#10080](https://github.com/matrix-org/synapse/issues/10080))
- Improve OpenTracing for event persistence. ([\#10134](https://github.com/matrix-org/synapse/issues/10134), [\#10193](https://github.com/matrix-org/synapse/issues/10193))
- Clean up the interface for injecting OpenTracing over HTTP. ([\#10143](https://github.com/matrix-org/synapse/issues/10143))
- Limit the number of in-flight `/keys/query` requests from a single device. ([\#10144](https://github.com/matrix-org/synapse/issues/10144))
- Refactor EventPersistenceQueue. ([\#10145](https://github.com/matrix-org/synapse/issues/10145))
- Document `SYNAPSE_TEST_LOG_LEVEL` to see the logger output when running tests. ([\#10148](https://github.com/matrix-org/synapse/issues/10148))
- Update the Complement build tags in GitHub Actions to test currently experimental features. ([\#10155](https://github.com/matrix-org/synapse/issues/10155))
- Add a `synapse_federation_soft_failed_events_total` metric to track how often events are soft failed. ([\#10156](https://github.com/matrix-org/synapse/issues/10156))
- Fetch the corresponding complement branch when performing CI. ([\#10160](https://github.com/matrix-org/synapse/issues/10160))
- Add some developer documentation about boolean columns in database schemas. ([\#10164](https://github.com/matrix-org/synapse/issues/10164))
- Add extra logging fields to better debug where events are being soft failed. ([\#10168](https://github.com/matrix-org/synapse/issues/10168))
- Add debug logging for when we enter and exit `Measure` blocks. ([\#10183](https://github.com/matrix-org/synapse/issues/10183))
- Improve comments in structured logging code. ([\#10188](https://github.com/matrix-org/synapse/issues/10188))
- Update [MSC3083](https://github.com/matrix-org/matrix-doc/pull/3083) support with modifications from the MSC. ([\#10189](https://github.com/matrix-org/synapse/issues/10189))
- Remove redundant DNS lookup limiter. ([\#10190](https://github.com/matrix-org/synapse/issues/10190))
- Upgrade `black` linting tool to 21.6b0. ([\#10197](https://github.com/matrix-org/synapse/issues/10197))
- Expose OpenTracing trace id in response headers. ([\#10199](https://github.com/matrix-org/synapse/issues/10199))


Synapse 1.36.0 (2021-06-15)
===========================

Expand Down
1 change: 0 additions & 1 deletion changelog.d/10062.feature

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10062.removal

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10080.misc

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10108.feature

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10115.bugfix

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10122.doc

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10134.misc

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10143.misc

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10144.misc

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10145.misc

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10148.misc

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10154.bugfix

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10155.misc

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10156.misc

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10157.bugfix

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10158.bugfix

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10160.misc

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10161.removal

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10164.misc

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10167.feature

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10168.misc

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10175.bugfix

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10180.doc

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10183.misc

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10184.bugfix

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10188.misc

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10189.misc

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10190.misc

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10191.feature

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10193.misc

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10194.removal

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10195.bugfix

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10197.misc

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10198.doc

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10199.misc

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10206.feature

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10208.bugfix

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10210.removal

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10212.feature

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10221.bugfix

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/10227.feature

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/6739.feature

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/8436.doc

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/9247.feature

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/9359.feature

This file was deleted.

1 change: 0 additions & 1 deletion changelog.d/9933.misc

This file was deleted.

2 changes: 1 addition & 1 deletion synapse/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@
except ImportError:
pass

__version__ = "1.36.0"
__version__ = "1.37.0rc1"

if bool(os.environ.get("SYNAPSE_TEST_PATCH_LOG_CONTEXTS", False)):
# We import here so that we don't have to install a bunch of deps when
Expand Down
13 changes: 7 additions & 6 deletions synapse/events/spamcheck.py
Original file line number Diff line number Diff line change
Expand Up @@ -109,6 +109,8 @@ def async_wrapper(f: Optional[Callable]) -> Optional[Callable[..., Awaitable]]:
if f is None:
return None

wrapped_func = f

if f.__name__ == "check_registration_for_spam":
checker_args = inspect.signature(f)
if len(checker_args.parameters) == 3:
Expand All @@ -133,19 +135,18 @@ def wrapper(
request_info,
)

f = wrapper
wrapped_func = wrapper
elif len(checker_args.parameters) != 4:
raise RuntimeError(
"Bad signature for callback check_registration_for_spam",
)

def run(*args, **kwargs):
# We've already made sure f is not None above, but mypy doesn't do well
# across function boundaries so we need to tell it f is definitely not
# None.
assert f is not None
# mypy doesn't do well across function boundaries so we need to tell it
# wrapped_func is definitely not None.
assert wrapped_func is not None

return maybe_awaitable(f(*args, **kwargs))
return maybe_awaitable(wrapped_func(*args, **kwargs))

return run

Expand Down
76 changes: 76 additions & 0 deletions tests/handlers/test_register.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
from synapse.api.auth import Auth
from synapse.api.constants import UserTypes
from synapse.api.errors import Codes, ResourceLimitError, SynapseError
from synapse.events.spamcheck import load_legacy_spam_checkers
from synapse.spam_checker_api import RegistrationBehaviour
from synapse.types import RoomAlias, RoomID, UserID, create_requester

Expand Down Expand Up @@ -79,6 +80,39 @@ async def check_registration_for_spam(
return RegistrationBehaviour.ALLOW


class TestLegacyRegistrationSpamChecker:
def __init__(self, config, api):
pass

async def check_registration_for_spam(
self,
email_threepid,
username,
request_info,
):
pass


class LegacyAllowAll(TestLegacyRegistrationSpamChecker):
async def check_registration_for_spam(
self,
email_threepid,
username,
request_info,
):
return RegistrationBehaviour.ALLOW


class LegacyDenyAll(TestLegacyRegistrationSpamChecker):
async def check_registration_for_spam(
self,
email_threepid,
username,
request_info,
):
return RegistrationBehaviour.DENY


class RegistrationTestCase(unittest.HomeserverTestCase):
"""Tests the RegistrationHandler."""

Expand All @@ -95,6 +129,8 @@ def make_homeserver(self, reactor, clock):

hs = self.setup_test_homeserver(config=hs_config)

load_legacy_spam_checkers(hs)

module_api = hs.get_module_api()
for module, config in hs.config.modules.loaded_modules:
module(config=config, api=module_api)
Expand Down Expand Up @@ -535,6 +571,46 @@ def test_spam_checker_deny(self):
"""A spam checker can deny registration, which results in an error."""
self.get_failure(self.handler.register_user(localpart="user"), SynapseError)

@override_config(
{
"spam_checker": [
{
"module": TestSpamChecker.__module__ + ".LegacyAllowAll",
}
]
}
)
def test_spam_checker_legacy_allow(self):
"""Tests that a legacy spam checker implementing the legacy 3-arg version of the
check_registration_for_spam callback is correctly called.
In this test and the following one we test both success and failure to make sure
any failure comes from the spam checker (and not something else failing in the
call stack) and any success comes from the spam checker (and not because a
misconfiguration prevented it from being loaded).
"""
self.get_success(self.handler.register_user(localpart="user"))

@override_config(
{
"spam_checker": [
{
"module": TestSpamChecker.__module__ + ".LegacyDenyAll",
}
]
}
)
def test_spam_checker_legacy_deny(self):
"""Tests that a legacy spam checker implementing the legacy 3-arg version of the
check_registration_for_spam callback is correctly called.
In this test and the previous one we test both success and failure to make sure
any failure comes from the spam checker (and not something else failing in the
call stack) and any success comes from the spam checker (and not because a
misconfiguration prevented it from being loaded).
"""
self.get_failure(self.handler.register_user(localpart="user"), SynapseError)

@override_config(
{
"modules": [
Expand Down

0 comments on commit 763dba7

Please sign in to comment.