Protocol and Reference Implementation
Mercure is a protocol allowing to push data updates to web browsers and other HTTP clients in a convenient, fast, reliable and battery-efficient way. It is especially useful to publish real-time updates of resources served through web APIs, to reactive web and mobile apps.
The protocol has been published as an Internet Draft that is maintained in this repository.
A reference, production-grade, implementation of a Mercure hub (the server) is also available here. It's a free software (AGPL) written in Go. It is provided along with a library that can be used in any Go application to implement the Mercure protocol directly (without a hub) and an official Docker image.
In addition, a managed and high-scalability version of Mercure is available in private beta.
- native browser support, no lib nor SDK required (built on top of HTTP and server-sent events)
- compatible with all existing servers, even those who don't support persistent connections (serverless architecture, PHP, FastCGI...)
- built-in connection re-establishment and state reconciliation
- JWT-based authorization mechanism (securely dispatch an update to some selected subscribers)
- performant, leverages HTTP/2 multiplexing
- designed with hypermedia in mind, also supports GraphQL
- auto-discoverable through web linking
- message encryption support
- can work with old browsers (IE7+) using an
EventSourcepolyfill - connection-less push in controlled environments (e.g. browsers on mobile handsets tied to specific carriers)
The reference hub implementation:
- Fast, written in Go
- Works everywhere: static binaries and Docker images available
- Automatic HTTP/2 and HTTPS (using Let's Encrypt) support
- CORS support, CSRF protection mechanism
- Cloud Native, follows the Twelve-Factor App methodology
- Open source (AGPL)
Example implementation of a client (the subscriber), in JavaScript:
// The subscriber subscribes to updates for the https://example.com/users/dunglas topic
// and to any topic matching https://example.com/books/{id}
const url = new URL('https://example.com/hub');
url.searchParams.append('topic', 'https://example.com/books/{id}');
url.searchParams.append('topic', 'https://example.com/users/dunglas');
const eventSource = new EventSource(url);
// The callback will be called every time an update is published
eventSource.onmessage = e => console.log(e); // do something with the payloadOptionaly, the authorization mechanism can be used to subscribe to private updates. Also optionaly, the hub URL can be automatically discovered:
fetch('https://example.com/books/1') // Has this header `Link: <https://example.com/hub>; rel="mercure"`
.then(response => {
// Extract the hub URL from the Link header
const hubUrl = response.headers.get('Link').match(/<([^>]+)>;\s+rel=(?:mercure|"[^"]*mercure[^"]*")/)[1];
// Subscribe to updates using the first snippet, do something with response's body...
});
To dispatch an update, the publisher (an application server, a web browser...) just need to send a POST HTTP request to the hub.
Example using Node.js / Serverless:
// Handle a POST, PUT, PATCH or DELETE request or finish an async job...
// and notify the hub
const https = require('https');
const querystring = require('querystring');
const postData = querystring.stringify({
'topic': 'https://example.com/books/1',
'data': JSON.stringify({ foo: 'updated value' }),
});
const req = https.request({
hostname: 'example.com',
port: '443',
path: '/hub',
method: 'POST',
headers: {
Authorization: 'Bearer <valid-jwt-token>',
// the JWT must have a mercure.publish key containing an array of targets (can be empty for public updates)
// the JWT key must be shared between the hub and the server
'Content-Type': 'application/x-www-form-urlencoded',
'Content-Length': Buffer.byteLength(postData),
}
}, /* optional response handler */);
req.write(postData);
req.end();
// You'll probably prefer use the request library or the node-fetch polyfill in real projects,
// but any HTTP client, written in any language, will be just fine.The JWT must contain a publish property containing an array of targets. This array can be empty to allow publishing anonymous updates only. Example publisher JWT (demo key: !UnsecureChangeMe!).
Example of usage: the Mercure integration in API Platform:
- a webapp retrieves the availability status of a product from a REST API and displays it: only one is still available
- 3 minutes later, the last product is bought by another customer
- the webapp's view instantly shows that this product isn't available anymore
- a webapp tells the server to compute a report, this task is costly and will take some time to complete
- the server delegates the computation of the report to an asynchronous worker (using message queue), and closes the connection with the webapp
- the worker sends the report to the webapp when it is computed
- a webapp allows several users to edit the same document concurrently
- changes made are immediately broadcasted to all connected users
Mercure gets you covered!
The full protocol specification can be found in spec/mercure.md.
It is also available as an IETF's Internet Draft,
and is designed to be published as a RFC.
An OpenAPI specification of the hub API is also available in spec/openapi.yaml.
A managed, high-scalability version of Mercure is available in private beta. Drop us a mail for details and pricing.
Grab a binary from the release page and run:
JWT_KEY='myJWTKey' ADDR=':3000' DEMO=1 ALLOW_ANONYMOUS=1 CORS_ALLOWED_ORIGINS=* PUBLISH_ALLOWED_ORIGINS='http://localhost:3000' ./mercure
The server is now available on http://localhost:3000, with the demo mode enabled. Because ALLOW_ANONYMOUS is set to 1, anonymous subscribers are allowed.
To run it in production mode, and generate automatically a Let's Encrypt TLS certificate, just run the following command as root:
JWT_KEY='myJWTKey' ACME_HOSTS='example.com' ./mercure
The value of the ACME_HOSTS environment variable must be updated to match your domain name(s).
A Let's Enctypt TLS certificate will be automatically generated.
If you omit this variable, the server will be exposed on an (unsecure) HTTP connection.
When the server is up and running, the following endpoints are available:
POST https://example.com/hub: to publish updatesGET https://example.com/hub: to subscribe to updates
See the protocol for further informations.
To compile the development version and register the demo page, see CONTRIBUTING.md.
A Docker image is available on Docker Hub. The following command is enough to get a working server in demo mode:
docker run \
-e JWT_KEY='myJWTKey' -e DEMO=1 -e ALLOW_ANONYMOUS=1 -e PUBLISH_ALLOWED_ORIGINS='http://localhost' \
-p 80:80 \
dunglas/mercure
The server, in demo mode, is available on http://localhost. Anonymous subscribers are allowed.
In production, run:
docker run \
-e JWT_KEY='myJWTKey' -e ACME_HOSTS='example.com' \
-p 80:80 -p 443:443 \
dunglas/mercure
Be sure to update the value of ACME_HOSTS to match your domain name(s), a Let's Encrypt TLS certificate will be automatically generated.
To install Mercure in a Kubernetes cluster, use the official Helm Chart:
helm install stable/mercure
ACME_CERT_DIR: the directory where to store Let's Encrypt certificatesACME_HOSTS: a comma separated list of hosts for which Let's Encrypt certificates must be issuedADDR: the address to listen on (example:127.0.0.1:3000, default to:httpor:httpsdepending if HTTPS is enabled or not)ALLOW_ANONYMOUS: set to1to allow subscribers with no valid JWT to connectCERT_FILE: a cert file (to use a custom certificate)KEY_FILE: a key file (to use a custom certificate)COMPRESS: set to0to disable HTTP compression support (default to enabled)CORS_ALLOWED_ORIGINS: a comma separated list of allowed CORS origins, can be*for allDB_PATH: the path of the bbolt database (default toupdates.dbin the current directory)DEBUG: set to1to enable the debug mode (prints recovery stack traces)DEMO: set to1to enable the demo mode (automatically enabled whenDEBUG=1)HEARTBEAT_INTERVAL: interval between heartbeats (useful with some proxies, and old browsers), set to0sto disable (default), example15sHISTORY_SIZE: size of the history (to retrieve lost messages using theLast-Event-IDheader), set to0to never remove old events (default)HISTORY_CLEANUP_FREQUENCY: chances to trigger history cleanup when an update occurs, must be a number between0(never cleanup) and1(cleanup after every publication), default to0.3JWT_KEY: the JWT key to use for both publishers and subscribersLOG_FORMAT: the log format, can beJSON,FLUENTDorTEXT(default)PUBLISH_ALLOWED_ORIGINS: a comma separated list of origins allowed to publish (only applicable when using cookie-based auth)PUBLISHER_JWT_KEY: must contain the secret key to valid publishers' JWT, can be omited ifJWT_KEYis setREAD_TIMEOUT: maximum duration for reading the entire request, including the body, set to0sto disable (default), example:2mSUBSCRIBER_JWT_KEY: must contain the secret key to valid subscribers' JWT, can be omited ifJWT_KEYis setWRITE_TIMEOUT: maximum duration before timing out writes of the response, set to0sto disable (default), example:2m
If ACME_HOSTS or both CERT_FILE and KEY_FILE are provided, an HTTPS server supporting HTTP/2 connection will be started.
If not, an HTTP server will be started (not secure).
- Be sure to set a secret key (and not a JWT) in
JWT_KEY(or inSUBSCRIBER_JWT_KEYandPUBLISHER_JWT_KEY) - If the secret key contains special characters, be sure to escape them properly, especially if you set the environment variable in a shell, or in a YAML file (Kubernetes...)
- The publisher always needs a valid JWT, even if
ALLOW_ANONYMOUSis set to1, this JWT must have a property namedpublishand containing an array of targets (example) - The subscriber needs a valid JWT only if
ALLOW_ANONYMOUSis set to0(default), or to subscribe to private updates, in this case the JWT must have a property namedsubscribeand containing an array of targets (example)
For both the publish and subscribe properties, the array can be empty to publish only public updates, or set it to ["*"] to allow accessing to all targets.
If subscribing to the EventSource in the browser doesn't work (the browser instantly disconnects from the stream or complains about CORS policy on receiving an event), check that you've set a proper value for CORS_ALLOWED_ORIGINS on running Mercure. It's fine to use CORS_ALLOWED_ORIGINS=* for your local development.
Try our URI template tester to ensure that the template matches the topic.
Because they are delivery agnostic, Mercure plays particulary well with GraphQL's subscriptions.
In response to the subscription query, the GraphQL server may return a corresponding topic URL.
The client can then subscribe to the Mercure's event stream corresponding to this subscription by creating a new EventSource with an URL like https://example.com/hub?topic=https://example.com/subscriptions/<subscription-id> as parameter.
Updates for the given subscription can then be sent from the GraphQL server to the clients through the Mercure hub (in the data property of the server-sent event).
To unsubscribe, the client just calls EventSource.close().
Mercure can easily be integrated with Apollo GraphQL by creating a dedicated transport.
Use the following file as a template to run the Mercure hub with Supervisor:
[program:mercure]
command=/path/to/mercure
process_name=%(program_name)s_%(process_num)s
numprocs=1
environment=JWT_KEY="myJWTKey"
directory=/tmp
autostart=true
autorestart=true
startsecs=5
startretries=10
user=www-data
redirect_stderr=false
stdout_capture_maxbytes=1MB
stderr_capture_maxbytes=1MB
stdout_logfile=/path/to/logs/mercure.out.log
stderr_logfile=/path/to/logs/mercure.error.logSave this file to /etc/supervisor/conf.d/mercure.conf.
Run supervisorctl reread and supervisorctl update to activate and start the Mercure hub.
NGINX is supported out of the box. Use the following proxy configuration:
server {
listen 80 ssl http2;
listen [::]:80 ssl http2;
ssl_certificate /path/to/ssl/cert.crt;
ssl_certificate_key /path/to/ssl/cert.key;
location / {
proxy_pass http://url-of-your-mercure-hub;
proxy_read_timeout 24h;
proxy_http_version 1.1;
proxy_set_header Connection "";
}
}WebSocket is a low level protocol, Mercure is a high level one. Mercure provides convenient built-in features such as authorization, re-connection and state reconciliation ; while with WebSocket, you need to implement them yourself. Also, unlike Mercure (which is built on top of HTTP and Server-Sent Events), WebSocket is not designed to leverage HTTP/2.
HTTP/2 connections are multiplexed and bidirectional by default (it was not the case of HTTP/1).
When using Mercure over a h2 connection (recommended), your app can receive data through Server-Sent Events, and send data to the server with regular POST (or PUT/PATCH/DELETE) requests, with no overhead.
Basically, in most cases Mercure can be used as a modern and easier to use replacement for WebSocket.
WebSub is a server-to-server only protocol, while Mercure is also a server-to-client and client-to-client protocol.
Mercure has been heavily inspired by WebSub, and we tried to make the protocol as close as possible from the WebSub one.
Mercure uses Server-Sent Events to dispatch the updates, while WebSub use POST requests. Also, Mercure has an advanced authorization mechanism, and allows to subscribe to several topics with only one connection using URI templates.
The Push API is a simplex protocol mainly designed to send notifications to devices currently not connected to the application. In most implementations, the size of the payload to dispatch is very limited, and the messages are sent through the proprietary APIs and servers of the browsers' and operating systems' vendors.
On the other hand, Mercure is a duplex protocol designed to send live updates to devices currently connected to the web or mobile app. The payload is not limited, and the message goes directly from your servers to the clients.
In summary, use the Push API to send notifications to offline users (that will be available in Chrome, Android and iOS's notification centers), and use Mercure to receive and publish live updates when the user is using the app.
- PHP library to publish Mercure updates
- Official Mercure support for the Symfony framework
- Official Mercure support for the API Platform framework
EventSourcepolyfill for Edge/IE and old browsersEventSourcepolyfill for React NativeEventSourceimplementation for Node- Server-Sent Events client for Go
- JavaScript library to parse
Linkheaders - JavaScript library to decrypt JWE using the WebCrypto API
- Official Push and Real-Time Capabilities for Symfony and API Platform using Mercure (Symfony blog)
- Tech Workshop: Mercure by Kevin Dunglas at SensioLabs (SensioLabs)
- Tutoriel vidéo : Notifications instantanées avec Mercure (Grafikart)
- Mercure, un protocole pour pousser des mises à jour vers des navigateurs et app mobiles en temps réel (Les-Tilleuls.coop)
A Gatling-based load test is provided in this repository. It allows to test any implementation of the protocol, including the reference implementation.
See LoadTest.scala to learn how to use it.
See CONTRIBUTING.md.
Created by Kévin Dunglas. Graphic design by Laury Sorriaux. Sponsored by Les-Tilleuls.coop.