Skip to content

Commit

Permalink
!per akka#18463 Make Persistence Query API explorable
Browse files Browse the repository at this point in the history
* make the standard queries "single method interfaces" that may be implemented
  by a query journal plugin
* remove hints (major problems with varargs anyway), the hints for standard
  queries  should be given in configuration instead, e.g. refresh-interval
  • Loading branch information
patriknw committed Sep 17, 2015
1 parent a45f31c commit 5bd245f
Show file tree
Hide file tree
Showing 46 changed files with 1,492 additions and 953 deletions.
310 changes: 201 additions & 109 deletions akka-docs/rst/java/code/docs/persistence/PersistenceQueryDocTest.java

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -10,15 +10,10 @@

import akka.actor.ActorSystem;
import akka.persistence.journal.WriteEventAdapter;
import akka.persistence.journal.EventSeq;
import akka.persistence.journal.Tagged;
import akka.persistence.query.AllPersistenceIds;
import akka.persistence.query.EventEnvelope;
import akka.persistence.query.EventsByPersistenceId;
import akka.persistence.query.EventsByTag;
import akka.persistence.query.PersistenceQuery;
import akka.persistence.query.javadsl.ReadJournal;
import akka.persistence.query.journal.leveldb.LeveldbReadJournal;
import akka.persistence.query.journal.leveldb.javadsl.LeveldbReadJournal;
import akka.stream.ActorMaterializer;
import akka.stream.javadsl.Source;

Expand All @@ -30,38 +25,41 @@ public void demonstrateReadJournal() {
//#get-read-journal
final ActorMaterializer mat = ActorMaterializer.create(system);

ReadJournal queries =
PersistenceQuery.get(system).getReadJournalFor(LeveldbReadJournal.Identifier());
LeveldbReadJournal queries =
PersistenceQuery.get(system).getReadJournalFor(LeveldbReadJournal.class,
LeveldbReadJournal.Identifier());
//#get-read-journal
}

public void demonstrateEventsByPersistenceId() {
//#EventsByPersistenceId
ReadJournal queries =
PersistenceQuery.get(system).getReadJournalFor(LeveldbReadJournal.Identifier());
LeveldbReadJournal queries =
PersistenceQuery.get(system).getReadJournalFor(LeveldbReadJournal.class,
LeveldbReadJournal.Identifier());

Source<EventEnvelope, BoxedUnit> source =
queries.query(EventsByPersistenceId.create("some-persistence-id", 0, Long.MAX_VALUE));
queries.eventsByPersistenceId("some-persistence-id", 0, Long.MAX_VALUE);
//#EventsByPersistenceId
}

public void demonstrateAllPersistenceIds() {
//#AllPersistenceIds
ReadJournal queries =
PersistenceQuery.get(system).getReadJournalFor(LeveldbReadJournal.Identifier());
LeveldbReadJournal queries =
PersistenceQuery.get(system).getReadJournalFor(LeveldbReadJournal.class,
LeveldbReadJournal.Identifier());

Source<String, BoxedUnit> source =
queries.query(AllPersistenceIds.getInstance());
Source<String, BoxedUnit> source = queries.allPersistenceIds();
//#AllPersistenceIds
}

public void demonstrateEventsByTag() {
//#EventsByTag
ReadJournal queries =
PersistenceQuery.get(system).getReadJournalFor(LeveldbReadJournal.Identifier());
LeveldbReadJournal queries =
PersistenceQuery.get(system).getReadJournalFor(LeveldbReadJournal.class,
LeveldbReadJournal.Identifier());

Source<EventEnvelope, BoxedUnit> source =
queries.query(EventsByTag.create("green", 0));
queries.eventsByTag("green", 0);
//#EventsByTag
}

Expand Down
57 changes: 21 additions & 36 deletions akka-docs/rst/java/persistence-query-leveldb.rst
Original file line number Diff line number Diff line change
Expand Up @@ -38,10 +38,10 @@ extension:
Supported Queries
=================

EventsByPersistenceId
---------------------
EventsByPersistenceIdQuery and CurrentEventsByPersistenceIdQuery
----------------------------------------------------------------

``EventsByPersistenceId`` is used for retrieving events for a specific ``PersistentActor``
``eventsByPersistenceId`` is used for retrieving events for a specific ``PersistentActor``
identified by ``persistenceId``.

.. includecode:: code/docs/persistence/query/LeveldbPersistenceQueryDocTest.java#EventsByPersistenceId
Expand All @@ -55,15 +55,10 @@ The returned event stream is ordered by sequence number, i.e. the same order as
``PersistentActor`` persisted the events. The same prefix of stream elements (in same order)
are returned for multiple executions of the query, except for when events have been deleted.

The query supports two different completion modes:

* The stream is not completed when it reaches the end of the currently stored events,
but it continues to push new events when new events are persisted. This is the
default mode that is used when no hints are given. It can also be specified with
hint ``RefreshInterval``.

* The stream is completed when it reaches the end of the currently stored events.
This mode is specified with hint ``NoRefresh``.
The stream is not completed when it reaches the end of the currently stored events,
but it continues to push new events when new events are persisted.
Corresponding query that is completed when it reaches the end of the currently
stored events is provided by ``currentEventsByPersistenceId``.

The LevelDB write journal is notifying the query side as soon as events are persisted, but for
efficiency reasons the query side retrieves the events in batches that sometimes can
Expand All @@ -73,36 +68,31 @@ hint.
The stream is completed with failure if there is a failure in executing the query in the
backend journal.

AllPersistenceIds
-----------------
AllPersistenceIdsQuery and CurrentPersistenceIdsQuery
-----------------------------------------------------

``AllPersistenceIds`` is used for retrieving all ``persistenceIds`` of all persistent actors.
``allPersistenceIds`` is used for retrieving all ``persistenceIds`` of all persistent actors.

.. includecode:: code/docs/persistence/query/LeveldbPersistenceQueryDocTest.java#AllPersistenceIds

The returned event stream is unordered and you can expect different order for multiple
executions of the query.

The query supports two different completion modes:

* The stream is not completed when it reaches the end of the currently used ``persistenceIds``,
but it continues to push new ``persistenceIds`` when new persistent actors are created.
This is the default mode that is used when no hints are given. It can also be specified with
hint ``RefreshInterval``.

* The stream is completed when it reaches the end of the currently used ``persistenceIds``.
This mode is specified with hint ``NoRefresh``.
The stream is not completed when it reaches the end of the currently used `persistenceIds`,
but it continues to push new `persistenceIds` when new persistent actors are created.
Corresponding query that is completed when it reaches the end of the currently
currently used `persistenceIds` is provided by ``currentPersistenceIds``.

The LevelDB write journal is notifying the query side as soon as new ``persistenceIds`` are
created and there is no periodic polling or batching involved in this query.

The stream is completed with failure if there is a failure in executing the query in the
backend journal.

EventsByTag
-----------
EventsByTag and CurrentEventsByTag
----------------------------------

``EventsByTag`` is used for retrieving events that were marked with a given tag, e.g.
``eventsByTag`` is used for retrieving events that were marked with a given tag, e.g.
all domain events of an Aggregate Root type.

.. includecode:: code/docs/persistence/query/LeveldbPersistenceQueryDocTest.java#EventsByTag
Expand Down Expand Up @@ -131,15 +121,10 @@ tagged event stream.

Events deleted using ``deleteMessages(toSequenceNr)`` are not deleted from the "tagged stream".

The query supports two different completion modes:

* The stream is not completed when it reaches the end of the currently stored events,
but it continues to push new events when new events are persisted. This is the
default mode that is used when no hints are given. It can also be specified with
hint ``RefreshInterval``.

* The stream is completed when it reaches the end of the currently stored events.
This mode is specified with hint ``NoRefresh``.
The stream is not completed when it reaches the end of the currently stored events,
but it continues to push new events when new events are persisted.
Corresponding query that is completed when it reaches the end of the currently
stored events is provided by ``currentEventsByTag``.

The LevelDB write journal is notifying the query side as soon as tagged events are persisted, but for
efficiency reasons the query side retrieves the events in batches that sometimes can
Expand Down
64 changes: 41 additions & 23 deletions akka-docs/rst/java/persistence-query.rst
Original file line number Diff line number Diff line change
Expand Up @@ -51,20 +51,20 @@ Read Journals

In order to issue queries one has to first obtain an instance of a ``ReadJournal``.
Read journals are implemented as `Community plugins`_, each targeting a specific datastore (for example Cassandra or JDBC
databases). For example, given a library that provides a ``akka.persistence.query.noop-read-journal`` obtaining the related
databases). For example, given a library that provides a ``akka.persistence.query.my-read-journal`` obtaining the related
journal is as simple as:

.. includecode:: code/docs/persistence/PersistenceQueryDocTest.java#basic-usage

Journal implementers are encouraged to put this identifier in a variable known to the user, such that one can access it via
``getJournalFor(NoopJournal.identifier)``, however this is not enforced.
``getJournalFor(NoopJournal.class, NoopJournal.identifier)``, however this is not enforced.

Read journal implementations are available as `Community plugins`_.


Predefined queries
------------------
Akka persistence query comes with a number of ``Query`` objects built in and suggests Journal implementors to implement
Akka persistence query comes with a number of query interfaces built in and suggests Journal implementors to implement
them according to the semantics described below. It is important to notice that while these query types are very common
a journal is not obliged to implement all of them - for example because in a given journal such query would be
significantly inefficient.
Expand All @@ -75,34 +75,37 @@ significantly inefficient.

The predefined queries are:

AllPersistenceIds
^^^^^^^^^^^^^^^^^
AllPersistenceIdsQuery and CurrentPersistenceIdsQuery
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

``AllPersistenceIds`` which is designed to allow users to subscribe to a stream of all persistent ids in the system.
``allPersistenceIds`` which is designed to allow users to subscribe to a stream of all persistent ids in the system.
By default this stream should be assumed to be a "live" stream, which means that the journal should keep emitting new
persistence ids as they come into the system:

.. includecode:: code/docs/persistence/PersistenceQueryDocTest.java#all-persistence-ids-live

If your usage does not require a live stream, you can disable refreshing by using *hints*, providing the built-in
``NoRefresh`` hint to the query:
If your usage does not require a live stream, you can use the ``currentPersistenceIds`` query:

.. includecode:: code/docs/persistence/PersistenceQueryDocTest.java#all-persistence-ids-snap

EventsByPersistenceId
^^^^^^^^^^^^^^^^^^^^^
EventsByPersistenceIdQuery and CurrentEventsByPersistenceIdQuery
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

``EventsByPersistenceId`` is a query equivalent to replaying a :ref:`PersistentActor <event-sourcing-scala>`,
``eventsByPersistenceId`` is a query equivalent to replaying a :ref:`PersistentActor <event-sourcing-java>`,
however, since it is a stream it is possible to keep it alive and watch for additional incoming events persisted by the
persistent actor identified by the given ``persistenceId``. Most journals will have to revert to polling in order to achieve
this, which can be configured using the ``RefreshInterval`` query hint:
persistent actor identified by the given ``persistenceId``.

.. includecode:: code/docs/persistence/PersistenceQueryDocTest.java#events-by-persistent-id-refresh
.. includecode:: code/docs/persistence/PersistenceQueryDocTest.java#events-by-persistent-id

EventsByTag
^^^^^^^^^^^
Most journals will have to revert to polling in order to achieve this,
which can typically be configured with a ``refresh-interval`` configuration property.

``EventsByTag`` allows querying events regardless of which ``persistenceId`` they are associated with. This query is hard to
If your usage does not require a live stream, you can use the ``currentEventsByPersistenceId`` query.

EventsByTag and CurrentEventsByTag
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

``eventsByTag`` allows querying events regardless of which ``persistenceId`` they are associated with. This query is hard to
implement in some journals or may need some additional preparation of the used data store to be executed efficiently.
The goal of this query is to allow querying for all events which are "tagged" with a specific tag.
That includes the use case to query all domain events of an Aggregate Root type.
Expand Down Expand Up @@ -134,6 +137,7 @@ query has an optionally supported offset parameter (of type ``Long``) which the
For example a journal may be able to use a WHERE clause to begin the read starting from a specific row, or in a datastore
that is able to order events by insertion time it could treat the Long as a timestamp and select only older events.

If your usage does not require a live stream, you can use the ``currentEventsByTag`` query.

Materialized values of queries
------------------------------
Expand All @@ -142,19 +146,22 @@ which are a feature of `Akka Streams`_ that allows to expose additional values a

More advanced query journals may use this technique to expose information about the character of the materialized
stream, for example if it's finite or infinite, strictly ordered or not ordered at all. The materialized value type
is defined as the ``M`` type parameter of a query (``Query[T,M]``), which allows journals to provide users with their
is defined as the second type parameter of the returned ``Source``, which allows journals to provide users with their
specialised query object, as demonstrated in the sample below:

.. includecode:: code/docs/persistence/PersistenceQueryDocTest.java#materialized-query-metadata-classes
.. includecode:: code/docs/persistence/PersistenceQueryDocTest.java#materialized-query-metadata
.. includecode:: code/docs/persistence/PersistenceQueryDocTest.java#advanced-journal-query-types

.. includecode:: code/docs/persistence/PersistenceQueryDocTest.java#advanced-journal-query-definition

.. includecode:: code/docs/persistence/PersistenceQueryDocTest.java#advanced-journal-query-usage

.. _materialized values: http://doc.akka.io/docs/akka-stream-and-http-experimental/1.0/java/stream-quickstart.html#Materialized_values
.. _Akka Streams: http://doc.akka.io/docs/akka-stream-and-http-experimental/1.0/java.html
.. _Community plugins: http://akka.io/community/#plugins-to-akka-persistence-query

Performance and denormalization
===============================
When building systems using :ref:`event-sourcing-scala` and CQRS (`Command & Query Responsibility Segragation`_) techniques
When building systems using :ref:`event-sourcing-java` and CQRS (`Command & Query Responsibility Segragation`_) techniques
it is tremendously important to realise that the write-side has completely different needs from the read-side,
and separating those concerns into datastores that are optimised for either side makes it possible to offer the best
expirience for the write and read sides independently.
Expand Down Expand Up @@ -232,8 +239,13 @@ Most users will not need to implement journals themselves, except if targeting a
ReadJournal plugin API
----------------------

Journals *MUST* return a *failed* ``Source`` if they are unable to execute the passed in query.
For example if the user accidentally passed in an ``SqlQuery()`` to a key-value journal.
A read journal plugin must implement ``akka.persistence.query.ReadJournalProvider`` which
creates instances of ``akka.persistence.query.scaladsl.ReadJournal`` and
``akka.persistence.query.javaadsl.ReadJournal``. The plugin must implement both the ``scaladsl``
and the ``javadsl`` interfaces because the ``akka.stream.scaladsl.Source`` and
``akka.stream.javadsl.Source`` are different types and even though those types can easily be converted
to each other it is most convenient for the end user to get access to the Java or Scala ``Source`` directly.
As illustrated below one of the implementations can delegate to the other.

Below is a simple journal implementation:

Expand All @@ -243,6 +255,12 @@ And the ``EventsByTag`` could be backed by such an Actor for example:

.. includecode:: code/docs/persistence/query/MyEventsByTagJavaPublisher.java#events-by-tag-publisher

If the underlying datastore only supports queries that are completed when they reach the
end of the "result set", the journal has to submit new queries after a while in order
to support "infinite" event streams that include events stored after the initial query
has completed. It is recommended that the plugin use a configuration property named
``refresh-interval`` for defining such a refresh interval.

Plugin TCK
----------

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,7 @@ trait DomainEvent
case class Person(name: String, age: Int) extends DomainEvent
case class Box(length: Int) extends DomainEvent

case class MyTaggingJournalModel(payload: Any, tags: immutable.Set[String])
case class MyTaggingJournalModel(payload: Any, tags: Set[String])

//#identity-event-adapter
class MyEventAdapter(system: ExtendedActorSystem) extends EventAdapter {
Expand Down Expand Up @@ -230,4 +230,4 @@ object v1 {
trait Event
trait UserEvent extends v1.Event
trait ItemEvent extends v1.Event
}
}
Loading

0 comments on commit 5bd245f

Please sign in to comment.