The Internals of Spark Structured Streaming

The project contains the sources of The Internals of Spark Structured Streaming online book.

Tools

The project is based on or uses the following tools:

• Apache Spark

• MkDocs which strives for being a fast, simple and downright gorgeous static site generator that's geared towards building project documentation

• Material for MkDocs theme

• Markdown

• Visual Studio Code as a Markdown editor

• Docker to run the Material for MkDocs (with plugins and extensions)

Previewing Book

Simply use the official Docker image to get up and running with the above tools.

Start mkdocs serve in the project root (the folder with mkdocs.yml) as follows:

$ docker run --rm -it -p 8000:8000 -v ${PWD}:/docs squidfunk/mkdocs-material

Consult the MkDocs documentation to get started and learn how to build the project.

$ docker run --rm -it -p 8000:8000 -v ${PWD}:/docs squidfunk/mkdocs-material --help
Usage: mkdocs [OPTIONS] COMMAND [ARGS]...

  MkDocs - Project documentation with Markdown.

Options:
  -V, --version  Show the version and exit.
  -q, --quiet    Silence warnings
  -v, --verbose  Enable verbose output
  -h, --help     Show this message and exit.

Commands:
  build      Build the MkDocs documentation
  gh-deploy  Deploy your documentation to GitHub Pages
  new        Create a new MkDocs project
  serve      Run the builtin development server

Use mkdocs build --clean to remove any stale files.

Consider --dirtyreload for a faster reloads.

$ docker run --rm -it \
    -p 8000:8000 \
    -v ${PWD}:/docs \
    squidfunk/mkdocs-material \
    serve --dirtyreload --verbose --dev-addr 0.0.0.0:8000

No Sphinx?! Why?

Read Giving up on Read the Docs, reStructuredText and Sphinx.