Skip to content

⛱ Broadly scoped ecosystem & mono-repository of 198 TypeScript projects (and ~175 examples) for general purpose, functional, data driven development

License

Notifications You must be signed in to change notification settings

aprenderia/umbrella

Repository files navigation

thi.ng/umbrella

GitHub Workflow Status Code Climate Become a patron Mastodon Follow

About

"A collection of functional programming libraries that can be composed together. Unlike a framework, thi.ng is a suite of instruments and you (the user) must be the composer of. Geared towards versatility, not any specific type of music." — @loganpowell via Twitter

Please visit thi.ng for additional information & topic based search of packages relevant to your use cases...

This project is NOT a framework! This mono-repository is home to currently 178 individual TypeScript/ES6 projects (and ~120 example projects), a wide and jointly developed collection of packages, tools and general purpose building blocks for (non-exhaustive list of topics, see package overview below):

  • Functional programming (ES6 iterators/generators, composition, memoization, transducers, multi-methods)
  • Reactive programming, stream / transducer based dataflow graphs / pipelines / DOM
  • Data structures & data transformations for wide range of use cases (maps, sets, heaps, queues, graphs etc.)
  • WebAssembly bridge APIs & data structure bindings code generators for multiple target languages
  • PEG-style functional parser combinators w/ (optional) custom grammar definition language
  • 2D geometry generation, shape primitives, math, manipulation, intersections, conversions & visualizations
  • Canvas abstractions, pixel buffers & SVG serialization/conversion
  • Vector, matrix (dense & sparse), ECS implementations with optional support for strided memory layouts
  • Semi-declarative WebGL 1/2 abstraction layer
  • DSL for shader functions defined in TypeScript and cross-compilation to GLSL, JS, VEX etc.
  • Value-based equivalence (vs. native object identity)
  • DSP building blocks: oscillators, noise generators, filters, 1D FFT/IFFT, muxers, rate converters
  • Immutable data handling, state containers, transacted state updates, Undo-Redo history
  • Data driven UI component toolkits (DOM-based, canvas-based, immediate-mode, multiple approaches...)
  • Multi-format, multi-channel pixel buffers (int & float based), conversions, dithering, Porter-Duff alpha-blending operators
  • Color space/format conversions, matrix based color manipulation, gradient generators, color palettes
  • Date-time abstraction, relative dates, iterators, formatters, math
  • WebWorker workflow abstractions
  • Forth-style pointfree DSL for functional composition and DSL development/extension
  • S-expression parser & runtime (interpreter) infrastructure for custom DSL creation
  • SIMD batch-processing of vector data
  • Pen-plotter (AxiDraw) toolchain & geometry conversions
  • Various interpolations, math helpers, automatic differentiation (Dual numbers)
  • etc.

Once more, this project is NOT a framework, provides no turn-key, one-size-fits-all approach and instead encourages a mix & match philosophy for various key aspects of application design (in & outside the browser). Customization points are provided wherever possible and usually only expect certain interfaces/type signatures rather than concrete implementations.

All packages:

  • are versioned independently
  • released via thi.ng/monopub
  • distributed as ESM modules (ES2020 syntax) with export maps, TypeScript typings & change logs
  • highly modular with largely only a single function / class (only closely related functions) per file to help w/ selective imports and tree shaking
  • provide re-exports of all their publics for full library imports
  • have either none or only @thi.ng internal runtime dependencies (w/ very few exceptions!)
  • declare public interfaces, enums & types in an api.ts file
  • have auto-generated online documentation at docs.thi.ng
  • licensed under Apache Software License 2.0

Getting started

The sheer number and varied nature & purpose of these packages makes it impossible to provide traditional "Getting started" tutorials. To compensate, this repo provides ~120 example projects, detailed readmes (at the very least for all the larger and/or more important packages) and hundreds of small usage examples/snippets in various docstrings.

If unsure about something, please reach out!

Project templates

You might find one of the following template repos an useful starting point:

  • tpl-umbrella-basic: Bare-bones template repo for browser-based projects
  • tpl-umbrella-fxhash: Project template repo for generative art projects on the fx(hash) platform
  • tpl-umbrella-zig: Minimal browser project template for hybrid TypeScript & Zig (WebAssembly) apps

Examples & Showcase

There's a steadily growing number (~120) of standalone examples of different complexities (often combining functionality from several packages) in the examples directory.

Example screenshots (small selection)

awesome.thi.ng

Due to other priorities still very much in its infancy & planning stage, but please help to document your own usage of these packages by contributing project information to the awesome.thi.ng repo, which will be used to build a showcase site... Thank you!

Generative art projects

Several generative art projects by Karsten Schmidt on fx(hash) have been created exclusively with libraries from this collection.

De/Frag series
Quasiflock C-SCAPE ASCII-SCAPE

Blog posts

Videos

Community, contributing, getting help

Join the (still new) discussions here on Github, get in touch via Mastodon or use the issue tracker. If you'd like to contribute in other ways, please first read this document.

In general, we welcome contributions of all kinds (docs, examples, bug fixes, feature requests, financial contributions etc.). You can find a fairly detailed overview for contributors here: CONTRIBUTING.md.

Note: The default branch for this repo is develop and all PRs should be based on this branch. This too means, the README files on this branch might refer to yet-unreleased features or packages. Please use the main branch for viewing the most recently released version(s)!.

Wiki

So far the wiki has only been updated sporadically, but please be sure to check it out for project-wide glossary, information, cookbooks, useful snippets etc.

Projects

Latest updates

As of: 2023-03-02

Status Package Version Changelog
@thi.ng/color version changelog
@thi.ng/emoji version changelog
@thi.ng/hdom version changelog
@thi.ng/hiccup-markdown version changelog
@thi.ng/math version changelog
@thi.ng/strings version changelog

Fundamentals

Project Version Changelog Description
@thi.ng/args version changelog Declarative & functional CLI arg parsing & coercions
@thi.ng/api version changelog Common types, decorators, mixins
@thi.ng/bench version changelog Basic benchmarking helpers
@thi.ng/checks version changelog Type & value checks
@thi.ng/compare version changelog Comparators
@thi.ng/compose version changelog Functional composition helpers
@thi.ng/date version changelog Date/time iterators, formatters, rounding
@thi.ng/defmulti version changelog Dynamic multiple dispatch
@thi.ng/distance version changelog n-D distance metrics & K-nearest neighborhoods
@thi.ng/equiv version changelog Deep value equivalence checking
@thi.ng/errors version changelog Custom error types
@thi.ng/expose version changelog Conditional global variable exposition
@thi.ng/hex version changelog Hex value formatters for U4-64 words
@thi.ng/logger version changelog Basis infrastructure for arbitrary logging
@thi.ng/memoize version changelog Function memoization w/ customizable caching
@thi.ng/oquery version changelog Pattern based query engine for JS objects
@thi.ng/parse version changelog Parser combinators & AST generator/transformer
@thi.ng/paths version changelog Immutable nested object accessors
@thi.ng/strings version changelog Higher-order string formatting utils
@thi.ng/system version changelog Minimal life cycle container for stateful app components
@thi.ng/testament version changelog Minimal test runner
@thi.ng/transclude version changelog Template engine for text document generation

Maths

Project Version Changelog Description
@thi.ng/dual-algebra version changelog Dual number algebra / automatic differentiation
@thi.ng/dsp version changelog DSP utils, composable signal gens/processors
@thi.ng/fuzzy version changelog Fuzzy logic primitives & rule inference engine
@thi.ng/intervals version changelog Open/closed intervals, queries, set ops
@thi.ng/math version changelog Assorted common math functions & utilities
@thi.ng/matrices version changelog Matrix operations
@thi.ng/sparse version changelog Sparse matrix & vector impls
@thi.ng/vectors version changelog Fixed & arbitrary-length vector ops

Randomness

Project Version Changelog Description
@thi.ng/colored-noise version changelog 1D colored noise generators
@thi.ng/ksuid version changelog K-sortable unique identifiers, binary & base-N encoded
@thi.ng/lowdisc version changelog n-D Low discrepancy sequence generators
@thi.ng/random version changelog Seedable PRNG implementations, distributions & utilities
@thi.ng/random-fxhash version changelog PRNG impl & utilities for fxhash projects

File / file format / hardware support

Project Version Changelog Description
@thi.ng/axidraw version changelog Minimal, declarative AxiDraw plotter controller
@thi.ng/bencode version changelog Bencode binary format encoding
@thi.ng/csv version changelog Customizable CSV parser/object mapper
@thi.ng/dot version changelog Graphviz DOM & export
@thi.ng/dsp-io-wav version changelog WAV file format exporter
@thi.ng/file-io version changelog Assorted file I/O utils for NodeJS
@thi.ng/geom-io-obj version changelog Wavefront OBJ model parser
@thi.ng/hiccup-css version changelog CSS from nested JS data structures
@thi.ng/hiccup-html version changelog Type-checked HTML5 element wrappers for hiccup
@thi.ng/hiccup-markdown version changelog Hiccup-to-Markdown serialization
@thi.ng/hiccup-svg version changelog hiccup based SVG vocab
@thi.ng/iges version changelog IGES format geometry serialization
@thi.ng/markdown-table version changelog Markdown table generator / formatter
@thi.ng/mime version changelog File extension to MIME type mappings
@thi.ng/pixel-io-geotiff version changelog GeoTIFF reader support for thi.ng/pixel
@thi.ng/pixel-io-netpbm version changelog 1/8/16/24bit NetPBM image format reader/writer
@thi.ng/prefixes version changelog Linked Data, RDF & xmlns prefixes/URLs
@thi.ng/sax version changelog SAX-like XML parser / transducer
@thi.ng/tangle version changelog Literate programming utilities

Iterator, stream & sequence processing

Project Version Changelog Description
@thi.ng/csp version changelog Channel based async ops
@thi.ng/fsm version changelog FSM / parser primitives
@thi.ng/grid-iterators version changelog 2D grid iterator strategies
@thi.ng/iterators version changelog ES6 generators / iterators
@thi.ng/seq version changelog Lisp/Clojure-style sequence abstraction
@thi.ng/transducers version changelog Composable data transformations
@thi.ng/transducers-binary version changelog Binary data related transducers
@thi.ng/transducers-fsm version changelog Finite state transducer
@thi.ng/transducers-hdom version changelog Transducer based hdom UI updates
@thi.ng/transducers-patch version changelog Patch-based, array & object editing
@thi.ng/transducers-stats version changelog Technical / statistical analysis

Reactive programming

Project Version Changelog Description
@thi.ng/rstream version changelog Push-based, reactive event stream primitves
@thi.ng/rstream-csp version changelog Adapter bridge CSP -> rstream
@thi.ng/rstream-dot version changelog Graphviz visualization of rstream topologies
@thi.ng/rstream-gestures version changelog Mouse & touch event stream abstraction
@thi.ng/rstream-graph version changelog Declarative dataflow graph construction
@thi.ng/rstream-log version changelog Hierarchical structured data logging
@thi.ng/rstream-log-file version changelog Log-file output handler
@thi.ng/rstream-query version changelog Triple store & query engine

Algorithms & data structures

Project Version Changelog Description
@thi.ng/adjacency version changelog Adjacency matrices & graph algorithms
@thi.ng/arrays version changelog Array utilities
@thi.ng/associative version changelog Alt Set & Map implementations
@thi.ng/atom version changelog Immutable value wrappers, views, history
@thi.ng/bitfield version changelog 1D/2D bit field implementations
@thi.ng/cache version changelog In-memory caches / strategies
@thi.ng/cellular version changelog Highly configurable 1D Cellular automata
@thi.ng/dcons version changelog Doubly-linked list
@thi.ng/diff version changelog Array & object diffing
@thi.ng/dgraph version changelog Dependency graph
@thi.ng/ecs version changelog Entity-Component System
@thi.ng/egf version changelog Extensible Graph Format
@thi.ng/gp version changelog Genetic programming helpers / AST gen
@thi.ng/heaps version changelog Binary & d-ary heap impls
@thi.ng/idgen version changelog Versioned ID generation / free-list
@thi.ng/k-means version changelog K-means clustering of n-D data
@thi.ng/ramp version changelog Parametric, interpolated lookup tables
@thi.ng/quad-edge version changelog Quad-edge, dual-graph data structure
@thi.ng/resolve-map version changelog DAG computations & value resolution
@thi.ng/vclock version changelog Vector clock functions / comparators
@thi.ng/zipper version changelog Immutable tree editing / navigation

Frontend / UI

Project Version Changelog Description
@thi.ng/adapt-dpi version changelog HDPI canvas adaptation / styling util
@thi.ng/dl-asset version changelog Asset/canvas/file download helpers
@thi.ng/emoji version changelog Bi-directional emoji lookup tables (names/chars)
@thi.ng/hdiff version changelog String diffing w/ hiccup output (includes CLI)
@thi.ng/hdom version changelog Hiccup based VDOM & diffing
@thi.ng/hdom-canvas version changelog hdom adapter for hiccup-canvas
@thi.ng/hdom-components version changelog hdom based UI components
@thi.ng/hdom-mock version changelog hdom mock implementation (testing / prototyping)
@thi.ng/hiccup version changelog S-expression based HTML/XML serialization
@thi.ng/hiccup-canvas version changelog hiccup interpreter for canvas api
@thi.ng/hiccup-carbon-icons version changelog IBM Carbon icons in hiccup format
@thi.ng/imgui version changelog Immediate mode GUI
@thi.ng/interceptors version changelog Composable event handlers & processor
@thi.ng/rdom version changelog Reactive, diff-less, async UI components
@thi.ng/rdom-canvas version changelog rdom component wrapper for thi.ng/hiccup-canvas
@thi.ng/rdom-components version changelog Unstyled, customizable component collection
@thi.ng/router version changelog Customizable browser & non-browser router
@thi.ng/text-canvas version changelog Text-mode canvas, drawing, tables, charts
@thi.ng/text-format version changelog Color text formatting w/ ANSI & HTML presets

Geometry, image & visualization

Project Version Changelog Description
@thi.ng/color version changelog Color conversions, gradients
@thi.ng/color-palettes version changelog Collection of image-based color palettes
@thi.ng/dgraph-dot version changelog Dependency graph -> Graphviz
@thi.ng/distance-transform version changelog Image based distance field generation
@thi.ng/fuzzy-viz version changelog Visualization, instrumentation for @thi.ng/fuzzy
@thi.ng/geom version changelog 2D only geometry types & ops
@thi.ng/geom-accel version changelog Spatial indexing data structures
@thi.ng/geom-api version changelog Shared types & interfaces
@thi.ng/geom-axidraw version changelog Shape conversions for AxiDraw pen plotter
@thi.ng/geom-arc version changelog 2D elliptic arc utils
@thi.ng/geom-clip-line version changelog 2D line clipping
@thi.ng/geom-clip-poly version changelog 2D convex polygon clipping
@thi.ng/geom-closest-point version changelog Closest point helpers
@thi.ng/geom-fuzz version changelog Fuzzy 2D shape drawing / filling
@thi.ng/geom-hull version changelog 2D convex hull (Graham scan)
@thi.ng/geom-isec version changelog Point & shape intersection tests
@thi.ng/geom-isoline version changelog 2D contour line extraction
@thi.ng/geom-poly-utils version changelog 2D polygon helpers
@thi.ng/geom-resample version changelog nD polyline / curve resampling
@thi.ng/geom-sdf version changelog 2D SDF creation, conversions, operators, utilities
@thi.ng/geom-splines version changelog nD cubic / quadratic spline ops
@thi.ng/geom-subdiv-curve version changelog nD iterative subdivision curves
@thi.ng/geom-tessellate version changelog nD convex polygon tessellators
@thi.ng/geom-trace-bitmap version changelog bitmap image to vector conversion
@thi.ng/geom-voronoi version changelog 2D iterative delaunay/voronoi
@thi.ng/lsys version changelog Extensible L-System architecture
@thi.ng/pixel version changelog Multi-format pixel buffers
@thi.ng/pixel-dither version changelog Image dithering w/ various algorithm presets
@thi.ng/poisson version changelog nD Poisson disk sampling
@thi.ng/porter-duff version changelog Alpha blending / compositing ops
@thi.ng/rasterize version changelog Shape drawing, filling & rasterization
@thi.ng/scenegraph version changelog Extensible 2D/3D scenegraph
@thi.ng/simd version changelog WebAssembly SIMD vector batch processing
@thi.ng/viz version changelog Declarative & functional data visualization toolkit

WebGL / GPGPU

Project Version Changelog Description
@thi.ng/shader-ast version changelog AST DSL for x-platform shader code
@thi.ng/shader-ast-glsl version changelog GLSL code generator
@thi.ng/shader-ast-js version changelog JS code generator
@thi.ng/shader-ast-optimize version changelog AST code optimization strategies
@thi.ng/shader-ast-stdlib version changelog 100+ useful AST shader functions
@thi.ng/webgl version changelog WebGL 1/2 / GPGPU facilities
@thi.ng/webgl-msdf version changelog MSDF font rendering
@thi.ng/webgl-shadertoy version changelog Shadertoy-like WebGL setup

Low-level, binary, memory management, interop

Project Version Changelog Description
@thi.ng/base-n version changelog Arbitrary base-n encoding/decoding with presets
@thi.ng/binary version changelog Assorted binary / bitwise ops, utilities
@thi.ng/bitstream version changelog Bitwise input / output streams
@thi.ng/dlogic version changelog Digital logic ops / constructs
@thi.ng/leb128 version changelog WASM based LEB128 varint encoder / decoder
@thi.ng/malloc version changelog Raw & typed array memory pool & allocator
@thi.ng/morton version changelog Z-order-curve / Morton coding
@thi.ng/range-coder version changelog Binary data Range encoder / decoder
@thi.ng/rle-pack version changelog Run-length encoding data compression
@thi.ng/soa version changelog Memory mapped data structures & serialization
@thi.ng/unionstruct version changelog Wrapper for C-like structs / unions
@thi.ng/vector-pools version changelog Data structures for memory mapped vectors
@thi.ng/wasm-api version changelog Modular, extensible JS/WebAssembly bridge API
@thi.ng/wasm-api-bindgen version changelog Polyglot bindings code generator for C/Zig/TS
@thi.ng/wasm-api-canvas version changelog WASM-side Canvas2D API bindings
@thi.ng/wasm-api-dom version changelog WASM-side DOM manipulation
@thi.ng/wasm-api-schedule version changelog WASM-side delayed code execution/scheduling

DSLs

Project Version Changelog Description
@thi.ng/pointfree version changelog Stack-based DSL & functional composition
@thi.ng/pointfree-lang version changelog Forth-like syntax layer for @thi.ng/pointfree
@thi.ng/sexpr version changelog S-Expression parser & runtime infrastructure

Building

git clone https://github.com/thi-ng/umbrella.git
cd umbrella

yarn install
yarn build

Once the entire mono-repo has been fully built at least once before, individual packages can then be (re)built like so:

yarn workspace @thi.ng/transducers run build

# or

(cd packages/transducers && yarn build)

Building example projects

Please see the example build instructions in the wiki for further details.

Testing

(Most, but not all packages have tests)

Due to various build/config issues/complexities, we're now using our own minimal test runner @thi.ng/testament

yarn test

# or individually
yarn workspace @thi.ng/rstream run test

Documentation

Autogenerated documentation (using TypeDoc) will be saved under /packages/*/doc/ and is also available at docs.thi.ng.

yarn doc

Furthermore, an experimental tsdoc-based documentation repo provides an alternative markdown output including cross-references between all packages, but currently isn't frequently updated (only every few months):

https://github.com/thi-ng/umbrella-docs-temp

Acknowledgements

Maintaining a large monorepo like this requires a lot of infrastructure and I'm grateful for the tooling provided by the following projects to simplify those tasks:

License

© 2015 - 2023 Karsten Schmidt // Apache Software License 2.0

Contributors ✨

Thanks goes to these wonderful people (emoji key):

Karsten Schmidt
Karsten Schmidt

💻 📖 🚧 💵
Alberto
Alberto

💻 💡 🐛 🤔 💵
Arthur Carabott
Arthur Carabott

💻 🤔 💡 📝 💵
André Wachter
André Wachter

💻 🤔 🐛
Gavin Cannizzaro
Gavin Cannizzaro

💻 🐛 🤔
Logan Powell
Logan Powell

📖 🐛 🤔 💵
Marcin Ignac
Marcin Ignac

🐛
arcticnoah
arcticnoah

💻
allforabit
allforabit

🐛 💻 🤔 💵
Yifan Wu
Yifan Wu

🐛 📖
stwind
stwind

💻 🐛
evilive
evilive

💻
Bnaya Peretz
Bnaya Peretz

💻 🐛 🤔
oljeger
oljeger

🐛
Nik Shevchenko
Nik Shevchenko

🐛 💻
Matei Adriel
Matei Adriel

💻 🐛 🤔
Pierre Grimaud
Pierre Grimaud

📖
Matt Huebert
Matt Huebert

💵
Raphael Saunier
Raphael Saunier

💵
Eric Ihli
Eric Ihli

💵
David Pham
David Pham

💵
TBD
TBD

🐛 🤔
Pedro Henriques dos Santos Teixeira
Pedro Henriques dos Santos Teixeira

💵
Jamie Owen
Jamie Owen

💻 🐛
Robert Kesteson
Robert Kesteson

🐛 💻
Chancy Kennedy
Chancy Kennedy

💵
Jarred Sumner
Jarred Sumner

🐛
Jamie Slome
Jamie Slome

🐛 🛡️
d3v53c
d3v53c

🐛 🛡️
Jannis Pohlmann
Jannis Pohlmann

🐛
Shakthi Prasad G S
Shakthi Prasad G S

🐛 💻
Robin Gower
Robin Gower

🐛
Michael Latzoni
Michael Latzoni

🐛
Z Yin
Z Yin

🐛
Damien Seguin
Damien Seguin

🐛 💻
Rui Gil
Rui Gil

🐛
Ja|nz
Ja|nz

💻 🚇 🚧
Tyler Freeman
Tyler Freeman

🐛 💻
blackhuman
blackhuman

🐛
David Negstad
David Negstad

💻
Muhammad Ridho
Muhammad Ridho

🐛 💻
MarcusWagberg
MarcusWagberg

💻
Masashi Yoshikawa
Masashi Yoshikawa

🐛
Guido Schmidt
Guido Schmidt

🐛 💵
tsukamotohideki
tsukamotohideki

💵
Dave Meehan
Dave Meehan

💻 🤔
Aurélien Bottazini
Aurélien Bottazini

🐛
Alex
Alex

🐛 💵
Ross Cairns
Ross Cairns

💵
Ruslan Prakapchuk
Ruslan Prakapchuk

💵
Jarrod Davis
Jarrod Davis

💵

This project follows the all-contributors specification. Contributions of any kind welcome!

About

⛱ Broadly scoped ecosystem & mono-repository of 198 TypeScript projects (and ~175 examples) for general purpose, functional, data driven development

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • TypeScript 98.5%
  • Zig 1.2%
  • Shell 0.2%
  • C 0.1%
  • PEG.js 0.0%
  • JavaScript 0.0%