Update Roadmap and contribution guidelines

docs/_data/sidebars/gtt_sidebar.yml

@@ -18,7 +18,7 @@ entries:
       output: pdf
       type: frontmatter
 
-  - title: Getting Started
+  - title: General information
     output: web, pdf
     folderitems:
     - title: About
@@ -34,7 +34,7 @@ entries:
       url: /gtt_roadmap.html
       output: web, pdf
 
-  - title: Basics
+  - title: Tutorials
     output: web, pdf
     folderitems:
     - title: The market data feed
@@ -55,10 +55,3 @@ entries:
     - title: Introduction to Triggers
       url: /gtt_tutorials_triggers.html
       output: web, pdf
-
-  - title: Advanced topics
-    output: web, pdf
-    folderitems:
-    - title: Build a Market Maker bot
-      url: /gtt_advanced_marketmaking.html
-      output: web, pdf

docs/_data/sidebars/home_sidebar.yml

@@ -14,9 +14,3 @@ entries:
     - title: GTT API reference
       url: /apiref
       output: web
-    - title: GTT UI Users' Guide
-      url: /gttui_about.html
-      output: web
-    - title: GTT UI API reference
-      url: /apiref-ui\/index.html
-      output: web

docs/images/platform_layers.xml

@@ -0,0 +1 @@
+<mxfile userAgent="Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36" version="8.3.5" editor="www.draw.io" type="device"><diagram id="8d704758-e39d-e9a4-76e4-a5be341853c0" name="Page-1">7Zvdc6M2EMD/Gj/WAwhw/HhJfO1DMs1crr3rk0cGGasByRVyPu6v7wokDAgnTkLOxJd4MoZF3/tjtbvgETrL7n8XeL265DFJR54T34/Q+cjzXBRM4UtJHkrJyVQLEkFjXWgruKY/iBY6WrqhMckbBSXnqaTrpjDijJFINmRYCH7XLLbkabPXNU6IJbiOcGpLv9FYrvQsvHAr/4PQZGV6dkM9vwWObhLBN0z3N/LQsvgrL2fYtKUnmq9wzO9qIjQboTPBuSyPsvszkqq1NctW1vu842o1bkGY3KeCV1a4xemGmBGHKVQ9XcBBog4u8AMRUAbadn6D/78YXVISq9mQPFcLCc0xScQSR8TUhh6rBvRM5YNZ3WJ9iBqBA5fvVlSS67WqjM7vgCeQrWSWwpkLh9UKqZMUL0h6Wq3xGU+5gEuMM6KKSsFviBHC0k9mn8KZ6mNJ07QmP/dn7ucA5InAMYWlajW05ExqKF1fn9eqO8UfyHFKEwayiKj5V1O9JUKS+50acSs9w/1DeEakeIAiugI6KWvoO8cNNSl3Ww4nhp5VjUFfy7BGP6la3qofDjQB3TSgDhpauqvUoeYf43xV6bGus4YizKI2dRA46mN0esVzKilvLKZaRgr35EWrwIJLybNagU9aDZIrdvhGppRB78Y0OCVFazWD7D5RFmvMl0sakXFOBDSSj+/IYl4ez7nATBmHJ1g7JCJG1Q/N0xoh/okNSDB5PSD+ByDzRBDCBs1Htbv8fECCD0Dmi2LuA+YjQAfjI/zg4x0YkIl7MECmFiB/Y1G6m19m119V60ydfLuuu525RRFMXj4TGS2yFrQNQUbjWHXT6bk2fdsGy/3ou4/9YdpUr4nWauqtIrG6fr0eXEx3jx2CsPiTCuKUElKc5zR6TJP1xSmvmJDNfcaywtKJh+9KU2MvMOf/aM2RGKJDPT4u5IonnOF0tpWeRhu4teNqfFhIewblPFVbj+sN1oJvRKRLaZ8cmkyI0UO3dmvqCzq0Z2SCpFjS2+YgulSqe7jicJvtdD6R24KiHLuuVI88W+20GPTCVjvlhK12Cr6qSe+H3B6bzoGRQ1O/wdzYKSF8HndF/SsiKCyOslylCeoTRn9gMLY9XfelNKKJ38TRRW+G42TwOIZtGieDpDEYGI1+i8bpC2H0p60c0NR/MxhPBgrjC1gr2Z1Yu3evzIUDYy6cBE3mWj74vsyFLXY9580MIOpyAXcknWEQRdL5q0rUsgSOTrnMu7PMugkx3LzzafEZDTLv3E4q7pt3nvSRd97DQ1Mrv945T/0gCC9MceeV869aeGr+vURFnYnVHbeEp2+Jcyyx6jZJBElwkbkwwXERF68FkVr8Pu+Wwz6lKZl81Ky/IQ+2ibxQe4TncBETseD8xk55HFfibJOrtFmU385hdIfOp+6CwSRUp+PmLtxhPEySpfHMDvWAim07/1SMjNTTZoYTkhG9Yx89K5wtCegsAkdhjqUEvxUrBQ6ZnNBpBbCeDQ7qcB6nPYBjx6GXWNwQWZACO0cOWjl2KxPDJrrAOcnHGWdUcqHYyeF74BYndFsWx0UdJqcrR3/SAzl20HhB/9vQmEo1RFBItEkL1+PY8VFPdOAbbI+KOHE635rcub42bI6Q51sguTZIbgdIfg8g2U97ZvfRqngBAyJjLMkvipN5WqhOhg1QYN7se8QOhV3pj/D1+KA9Mqk/OWwMOp6lvV2YYG7VfcJGX4eNl6QIG9eCA3pZViRV3md8eNhsSgnfgeJD5FmKv2IXZZ4suil0etSGcuu2maM56EYWPc3RoS3mDjJ2BovI3nAnb+PzI/sNT51GWlHl9VLMfj1ywAbSoUMTOu1tNrCgMaK+X8lBdnLyUtka1QDsJ/rlf76WNKM/dMbxqBkCryEia5mPF5jFhQLmxklV4xw0R3bcaDssfkeqKughVYXsuPELWXNQx9FvWBHPsg2DNoowZvwv3wgGCmbJXGwOn918yva4T+5XXgcxz880wOn2py/lk8Xt74vQ7H8=</diagram></mxfile>

docs/pages/gtt/gtt_contributing.md

@@ -3,35 +3,38 @@ title: Contributing
 keywords: GTT, overview, contributing
 last_updated: July 14, 2017
 tags: [getting_started]
-summary: "Please contribute to this project. We accept PRs for bugfixes, new features and new exchanges. Please read our contribution guide
-lines first though."
+summary: "How to contribute to this project"
 sidebar: gtt_sidebar
 permalink: gtt_contributing.html
 folder: gtt
 toc: true
 ---
 
 The GTT is an open-source project. While it was conceived and is maintained by Coinbase, our vision is to have the community actively
-contribute to the project.
+contribute to the project, but please note the following:
 
-To contribute to the GTT, [fork the project on GitHub](https://github.com/coinbase/gdax-tt/), make your changes and [submit them as a PR](https://github.com/coinbase/gdax-tt/pulls). The community will review and
-then hopefully merge your changes in, and it will form part of the next release of the GTT.
+# Before submitting any PRs, please take note
 
-We want this to be a community project, so the GTT should evolve to become what you want it to be. So we will accept pull requests for
-any and all of the following:
-* Bug fixes
-* New components and filters
-* New exchange support
-* Documentation
-* Tutorials
-* Working trading bot engines (that make use of the GTT, obviously)
-* Algorithm components
-* UI components (see the [gtt-ui](https://github.com/coinbase/gdax-tt/issues/52) project.
+This is an open-source project. This means that there are as many different ideas and thoughts on how to code, test and architect as there are contributors. This also means that getting code into production will often take longer than your average closed-source or corporate project and _way_ longer than any project that you run yourself.
 
-Please read and follow the contribution guidelines before you submit. This will smooth the flow for getting your code included in
-the project.
+All contributions must adhere to the [General Coinbase Contribution Guidelines](https://github.com/coinbase/gdax-tt/blob/master/CONTRIBUTING.md) and [Code of Conduct](https://github.com/coinbase/code-of-conduct)
 
-## Contribution Guidelines
+To contribute to the GTT, [fork the project on GitHub](https://github.com/coinbase/gdax-tt/), make your changes and [submit them as a PR](https://github.com/coinbase/gdax-tt/pulls).
+
+In addition to the general Coinbase contribution guidelines, there are some additional guidelines for the GTT specifically.
+
+## Product Roadmap
+
+Check the [Product Roadmap](./gtt_roadmap.html) and see whether your contribution is within the scope of the project. If it is, great! If not, you still have a few options
+
+1. Open a new issue, outlining your proposal / feature-request, and _indicate that you are willing to code it up yourself_.
+1. If your idea sits in Layer 3 or 4 (as defined in the Roadmap), the best approach is actually to _start a new project_, use the GTT as a library, and then [open a new issue][1] so that we can link to your project or bot from our documentation.
+
+The following PR types are the most welcome and will almost certainly be merged:
+* Bug fixes!
+* Unit tests (see notes on tests below)
+
+## Coding language
 
 Pull requests must be written in [TypeScript](www.typescriptlang.org). If you're a Javascript programmer, don't fret, the learning curve is very flat.
 TypeScript is a superset of Javascript, but because it is statically typed, whole classes of bugs are eliminated from your
@@ -53,19 +56,27 @@ You can auto-fix many of the more typographic errors, assuming you've installed
     tslint -p . --fix
 
 You can also make your life much easier by adding TSlint integration into your editor. WebStorm / IntelliJ for example,
-autofix many errors by running the Reformat-Code command.
+auto-fix many errors by running the Reformat-Code command.
 
 ### Unit tests
 
-Our goal is to reach >80% for Beta and ultimately 100% test coverage.
+We also welcome new tests to help move our test coverage towards 100%.
 
-For this reason we _highly encourage_ including unit tests in your PRs.
+As with the coding guidelines, we have standardised on some libraries and practices for testing, so here are some testing PR dos and don'ts.
 
-Our standard test runner is Mocha, and we use the standard `assert` module for assertions.
+To run the full battery of unit tests run `yarn test`
 
-To run the full battery of unit tests run
+To get coverage statistics, run `yarn run coverage`
 
-    yarn test
+#### Do
+* Include tests with your general PRs
+* Submit PRs that add new tests to improve test coverage (This is a great way to learn the API and uncover edge cases, BTW)
+* Use [Mocha](https://mochajs.org/) BDD syntax (`describe`, `it`) for defining tests
+* Use the standard `assert` module for assertions.
+
+#### Don't
+* Add new testing or assertion libraries as dependencies (e.g. chai, cucumber, etc.)
+* Don't let your unit tests make network calls (Use [nock](https://github.com/node-nock/nock) for HTTP request mocking)
 
 ### Documentation
 
@@ -76,3 +87,19 @@ directly from the code.
 
 {% include warning.html content="If you run TypeDoc locally to view your generated documentation, *do not check those changes into your PR*!
 For one, it adds too much noise to the PR and secondly, we will automatically update the API docs anyway." %}
+
+### PR hygiene
+
+* Try and keep your PRs contained to a single feature or bugfix. It makes them easier to review.
+
+* **Do not** commit IDE-specific files and metadata. Preferably, don't add those files to `.gitignore` either. You can locally ignore files by adding them to the `.git/info/exclude` file in your local working tree. This file follows the same rules as `.gitignore`. So for example, to exclude Webstorm metadata without changing the `.gitignore` file, I could run
+
+```
+$ echo ".idea" >> .git/info/exclude
+```
+
+* If you want to embark on a large-scale refactoring effort, please open an [issue][1] first to discuss this. In particular, breaking changes or wide-ranging changes to interfaces are likely to get rejected if they haven't gone through a preview and discussion first.
+
+
+
+[1]: https://github.com/coinbase/gdax-tt/issues

docs/pages/gtt/gtt_roadmap.md

@@ -9,41 +9,101 @@ permalink: gtt_roadmap.html
 folder: gtt
 ---
 
-The GTT is currently in alpha mode. Right now the code will have (possibly money-losing) bugs,
-and not all of the pieces of the toolkit are there yet. But we'll get there, and [with your help](#gtt_contributing),
-we'll get there.
+*Note*: The GTT is currently in alpha mode. Right now the code will have (possibly money-losing) bugs
 
-# Roadmap to GTT Beta
+## A modular approach to building out trading platforms
 
-These major features are missing ahead of a beta release:
-* Balance monitor - a tool to monitor balances across accounts (and/or exchanges) and automatically transfer funds if required
-* Profit monitor - track individual trades, deposits and withdrawals and provide profitability reports
+We've seen the GTT gain a fair amount of traction since its first release, which is great news. We've also
+had a chance to look at other projects that have been doing great things in this space. It's become clear that
+trying to be "everything to everybody" is not a sustainable goal for an open-source project like this; especially
+given the limited amount of resources dedicated to the project. A better approach is to use something akin to the Unix
+philosophy of building out a set of interoperable tools, each of which does one thing and does it well.
 
-Minor features:
+With this in mind, one can roughly separate the entire "Trading Platform" space into layers, where each layer builds on the
+previous one.
 
-* Complete console support
-* Complete Bitfinex support
-* Add Poloniex support
-* Full FX & crypto exchange rate coverage
-* Improved documentation
+{% include image.html file="platform_layers.svg" alt="Trading platform layers" caption="Each layer builds upon the previous, adding functionality and complexity" %}
 
-Other:
+The GTT's zone of influence is primarily Layer 2, and to some extent Layer 1.
 
-* \>80% test coverage
+### Layer 1 - A unified messaging interface
 
+The goal of Layer 1 functionality is to bring external data from a variety of sources and with minimal manipulation, present it to GTT users in a uniform way.
 
-# Roadmap to GTT 1.0
+There are dozens of crypto-currency exchanges (at least 70 and counting) running at present. Most of these exchanges offer a REST API for pulling orderbook data, querying balances, placing orders etc. Some also provide a live data stream (usually as a websocket interface) for live data monitoring. This means that there are essentially 70 different implementations of roughly the same set of user requirements.
 
-These major features are missing ahead of a 1.0 release:
+The purpose of Layer 1 is to provide a standard communication interface so that a single set of instructions can be translated to any supported exchange and _vice versa_.
 
-* Basic arbitrage bot - watch multiple markets and trade on arbitrage opportunities
-* Portfolio tracker bot - Set a portfolio target and have the bot trade to rebalance your portfolio when prices shift
-* WS feed to UI components
-* Basic set of UI components - multiple orderbook demo
-* Trading analysis tools
-* Full Documentation
+Initially, one of the goals of the GTT was to provide this functionality. However, we soon discovered the [CCXT project](https://github.com/ccxt/ccxt), which does a fabulous job of this already for over 70 exchanges. Therefore, the GTT delegates the majority of Level 1 functionality to CCXT. Specifically, almost all non-GDAX REST requests will be serviced by CCXT.
+
+The GTT currently implements WS support for some of the larger exchanges (GDAX, Poloniex, Bitfinex, Bittrex). As CCXT begins to support WS data, we may consider phasing out WS support in the GTT. Until that happens, the GTT intends to provide Layer 1 functionality for live WS message tooling for GDAX and a few of the major other exchanges.
+
+A unified APi for communicating back to the exchange also resides in Layer 1. This includes order placements and management, balance queries and deposits and withdrawals. For the most part this functionality is also delegated to CCXT where appropriate.
+
+Other services that the GTT also supports that fall under Layer 1 include:
+* Fiat exchange rates
+
+Layer 1 services that are NOT considered in the GTT scope:
+* Qualitative data streams (e.g. Twitter) for sentiment analysis
+* Traditional stock tickers (e.g. S&P-500)
+
+### Layer 2 - Data aggregation and interpretation
+
+The Layer 2 tools take the data from Layer 1, which is now in a consistent format, regardless of source, and starts to make sense of it. The types of things one could expect to find here:
+
+* Live orderbooks - the digestion of live (or simulated) websocket messages to derive and maintain a live orderbook.
+* Aggregated orderbooks - combining several live orderbooks into an aggregated orderbook
+* Order statistics - For example, [market order slippage](https://www.investopedia.com/terms/s/slippage.asp), the total immediate cost for _n_ units and spread
+* High-Level Order management - This tool sends order messages to exchanges via Layer 1, but also tracks metadata such as total orders made and current position.
+* Exchange rate services - Using fiat price data from Layer 1, messages can be transformed to reflect prices in any desired fiat currency equivalent.
+* Market statistics - For example, market / trading history (candles), TA indicators like moving average, Bollinger bands and the like.
+* Logging and reporting - includes Logging interfaces and conditional alerts and triggers.
+
+### Layer 3 - Trading Bots
+
+Layer 3 is where the big impact lies. More or less sophisticated trading bots can be built up using the tools in Layer 2 to actually implement and execute trading strategies. This layer is generally out-of-scope og the GTT, but we hope to provide some sample projects that illustrate what can be done. Some examples we plan on demonstrating:
+
+* A simple market-maker bot
+* A portfolio manager
+* A utility to easily query the state of any exchange or currency with a simple command line interface
+
+### Layer 4 - Support tools
+
+Layer 4 covers support tools. For expedience, these tools have been included as an additional layer, but in reality many of them are stand-alone tools that add key functionality to any project employing the GTT
+
+* Profit monitor - track individual trades, deposits and withdrawals and provide profitability reports. See [ALE](https://github.com/CjS77/ale) as an example of a simple implementation. It provides features to track every trade, calculate your current position, realized and unrealized profits/losses. It's currently lacking a REST interface to make it a truly standalone utility.
+* Data storage and replay tools (useful for backtesting). For example, InfluxDB is a very good time-series focused database, which can ingest thousands of messages per second. It is fairly straightforward to pipe unified Layer 1 websocket data into InfluxDB for permanent storage and later retrieval. GTT Layer 2 objects will be able to ingest those messages as if they're coming from a live source.
+* Machine learning - Optimizing and tuning trading algorithms is typically done using historical data. If you're
+capturing GTT data in a persistent store, like InfluxDB, then you're freed up to carry out model exploration and optimization in any suitable environment, independent of the GTT. Arguably, as of early 2018, the leading environment for ML, whether it's classical optimization or neural nets, is (Tensorflow)[https://www.tensorflow.org/]. If C/C++ isn't your cup of tea, there are very good bindings for Python as well. There are no node.js or Typescript bindings as of yet, so if you use Tensorflow and the GTT for your trading bot tuning, you will likely have two separate implementations of your models.
+
+
+## Roadmap
+
+With all that said, we can map out the project roadmap. Ideally, the roadmap would be paired with target dates and a regular release schedule, but the GDAX team is incredibly resource constrained and alas, the GTT has a lower priority than say, keeping  our customers' money safe, keeping the site up running and adding awesome new features.
+
+This roadmap is also not cast in stone. We might move things around, add new things, or remove others. But it should serve as a fair indicator of the near-, medium- and long-term goals of the project.
+
+
+
+### Priority 1 (0.5.0 release)
+* Bug fixes
+* \>75% test coverage
+* Complete authenticated action support (trades, account management, withdrawals & transfers)
+* Console utility
+
+### Priority 2 (0.7.0 release)
+* 80% test coverage
+* Working market maker bot
+* Basic TA indicators (RSI, MA, EMA)
+
+### Priority 3 (1.0.0 release)
+* 90% test coverage
+* Additional TA indicators
+* Advanced trading algorithms (VWAP, TWAP etc.)
+
+### No Priority
+Ideas that we're thinking about but have no plans to implement any time soon:
+* Portfolio balancing application
+* gRPC service for interfacing with external services (e.g. UIs, applications in other languages, databases)
 
-Minor features
 
-* Additional exchange support
-* 100% test coverage