Merge branch 'develop' into align_userdata

.dependabot/config.yml

@@ -0,0 +1,17 @@
+version: 1
+
+update_configs:
+  - package_manager: "python"
+    directory: "/"
+    update_schedule: "weekly"
+    allowed_updates:
+    - match:
+        update_type: "all"
+    target_branch: "develop"
+
+  - package_manager: "docker"
+    directory: "/"
+    update_schedule: "daily"
+    allowed_updates:
+    - match:
+        update_type: "all"

Dockerfile

@@ -1,4 +1,4 @@
-FROM python:3.7.3-slim-stretch
+FROM python:3.7.4-slim-stretch
 
 RUN apt-get update \
     && apt-get -y install curl build-essential libssl-dev \

docs/bot-usage.md

@@ -2,7 +2,7 @@
 
 This page explains the different parameters of the bot and how to run it.
 
-!Note:
+!!! Note:
     If you've used `setup.sh`, don't forget to activate your virtual environment (`source .env/bin/activate`) before running freqtrade commands.
 
 
@@ -48,21 +48,24 @@ optional arguments:
 
 ```
 
-### How to use a different configuration file?
+### How to specify which configuration file be used?
 
-The bot allows you to select which configuration file you want to use. Per
-default, the bot will load the file `./config.json`
+The bot allows you to select which configuration file you want to use by means of
+the `-c/--config` command line option:
 
 ```bash
 freqtrade -c path/far/far/away/config.json
 ```
 
+Per default, the bot loads the `config.json` configuration file from the current
+working directory.
+
 ### How to use multiple configuration files?
 
 The bot allows you to use multiple configuration files by specifying multiple
-`-c/--config` configuration options in the command line. Configuration parameters
-defined in the last configuration file override parameters with the same name
-defined in the previous configuration file specified in the command line.
+`-c/--config` options in the command line. Configuration parameters
+defined in the latter configuration files override parameters with the same name
+defined in the previous configuration files specified in the command line earlier.
 
 For example, you can make a separate configuration file with your key and secrete
 for the Exchange you use for trading, specify default configuration file with
@@ -235,7 +238,7 @@ usage: freqtrade hyperopt [-h] [-i TICKER_INTERVAL] [--timerange TIMERANGE]
                           [--customhyperopt NAME] [--hyperopt-path PATH]
                           [--eps] [-e INT]
                           [-s {all,buy,sell,roi,stoploss} [{all,buy,sell,roi,stoploss} ...]]
-                          [--dmmp] [--print-all] [-j JOBS]
+                          [--dmmp] [--print-all] [--no-color] [-j JOBS]
                           [--random-state INT] [--min-trades INT] [--continue]
                           [--hyperopt-loss NAME]
 
@@ -271,6 +274,8 @@ optional arguments:
                         (same as setting `max_open_trades` to a very high
                         number).
   --print-all           Print all results, not only the best ones.
+  --no-color            Disable colorization of hyperopt results. May be
+                        useful if you are redirecting output to a file.
   -j JOBS, --job-workers JOBS
                         The number of concurrently running jobs for
                         hyperoptimization (hyperopt worker processes). If -1
@@ -284,17 +289,18 @@ optional arguments:
   --continue            Continue hyperopt from previous runs. By default,
                         temporary files will be removed and hyperopt will
                         start from scratch.
-  --hyperopt-loss       NAME
-                        Specify the class name of the hyperopt loss function
+  --hyperopt-loss NAME  Specify the class name of the hyperopt loss function
                         class (IHyperOptLoss). Different functions can
                         generate completely different results, since the
-                        target for optimization is different. (default:
-                        `DefaultHyperOptLoss`).
+                        target for optimization is different. Built-in
+                        Hyperopt-loss-functions are: DefaultHyperOptLoss,
+                        OnlyProfitHyperOptLoss, SharpeHyperOptLoss.
+                        (default: `DefaultHyperOptLoss`).
 ```
 
 ## Edge commands
 
-To know your trade expectacny and winrate against historical data, you can use Edge.
+To know your trade expectancy and winrate against historical data, you can use Edge.
 
 ```
 usage: freqtrade edge [-h] [-i TICKER_INTERVAL] [--timerange TIMERANGE]

docs/configuration.md

@@ -1,15 +1,34 @@
 # Configure the bot
 
-This page explains how to configure your `config.json` file.
+This page explains how to configure the bot.
 
-## Setup config.json
+## The Freqtrade configuration file
 
-We recommend to copy and use the `config.json.example` as a template
+The bot uses a set of configuration parameters during its operation that all together conform the bot configuration. It normally reads its configuration from a file (Freqtrade configuration file).
+
+Per default, the bot loads configuration from the `config.json` file located in the current working directory.
+
+You can change the name of the configuration file used by the bot with the `-c/--config` command line option.
+
+In some advanced use cases, multiple configuration files can be specified and used by the bot or the bot can read its configuration parameters from the process standard input stream.
+
+If you used the [Quick start](installation.md/#quick-start) method for installing 
+the bot, the installation script should have already created the default configuration file (`config.json`) for you.
+
+If default configuration file is not created we recommend you to copy and use the `config.json.example` as a template
 for your bot configuration.
 
-The table below will list all configuration parameters.
+The Freqtrade configuration file is to be written in the JSON format.
+
+Additionally to the standard JSON syntax, you may use one-line `// ...` and multi-line `/* ... */` comments in your configuration files and trailing commas in the lists of parameters.
+
+Do not worry if you are not familiar with JSON format -- simply open the configuration file with an editor of your choice, make some changes to the parameters you need, save your changes and, finally, restart the bot or, if it was previously stopped, run it again with the changes you made to the configuration. The bot validates syntax of the configuration file at startup and will warn you if you made any errors editing it.
+
+## Configuration parameters
+
+The table below will list all configuration parameters available.
 
-Mandatory Parameters are marked as **Required**.
+Mandatory parameters are marked as **Required**.
 
 |  Command | Default | Description |
 |----------|---------|-------------|
@@ -53,6 +72,7 @@ Mandatory Parameters are marked as **Required**.
 | `experimental.use_sell_signal` | false | Use your sell strategy in addition of the `minimal_roi`. [Strategy Override](#parameters-in-the-strategy).
 | `experimental.sell_profit_only` | false | Waits until you have made a positive profit before taking a sell decision. [Strategy Override](#parameters-in-the-strategy).
 | `experimental.ignore_roi_if_buy_signal` | false | Does not sell if the buy-signal is still active. Takes preference over `minimal_roi` and `use_sell_signal`. [Strategy Override](#parameters-in-the-strategy).
+| `experimental.block_bad_exchanges` | true | Block exchanges known to not work with freqtrade. Leave on default unless you want to test if that exchange works now.
 | `pairlist.method` | StaticPairList | Use static or dynamic volume-based pairlist. [More information below](#dynamic-pairlists).
 | `pairlist.config` | None | Additional configuration for dynamic pairlists. [More information below](#dynamic-pairlists).
 | `telegram.enabled` | true | **Required.** Enable or not the usage of Telegram.

docs/data-analysis.md

@@ -31,6 +31,16 @@ df = load_trades_from_db("sqlite:///tradesv3.sqlite")
 df.groupby("pair")["sell_reason"].value_counts()
 ```
 
+### Load multiple configuration files
+
+This option can be usefull to inspect the results of passing in multiple configs in case of problems
+
+``` python
+from freqtrade.configuration import Configuration
+config = Configuration.from_files(["config1.json", "config2.json"])
+print(config)
+```
+
 ## Strategy debugging example
 
 Debugging a strategy can be time-consuming. FreqTrade offers helper functions to visualize raw data.

docs/developer.md

@@ -17,6 +17,29 @@ Alternatively (if your system is not supported by the setup.sh script), follow t
 
 This will install all required tools for development, including `pytest`, `flake8`, `mypy`, and `coveralls`.
 
+### Tests
+
+New code should be covered by basic unittests. Depending on the complexity of the feature, Reviewers may request more in-depth unittests.
+If necessary, the Freqtrade team can assist and give guidance with writing good tests (however please don't expect anyone to write the tests for you).
+
+#### Checking log content in tests
+
+Freqtrade uses 2 main methods to check log content in tests, `log_has()` and `log_has_re()` (to check using regex, in case of dynamic log-messages).
+These are available from `conftest.py` and can be imported in any test module.
+
+A sample check looks as follows:
+
+``` python
+from freqtrade.tests.conftest import log_has, log_has_re
+
+def test_method_to_test(caplog):
+    method_to_test()
+
+    assert log_has("This event happened", caplog)
+    # Check regex with trailing number ...
+    assert log_has_re(r"This dynamic event happened and produced \d+", caplog)
+```
+
 ## Modules
 
 ### Dynamic Pairlist

docs/docker.md

@@ -26,6 +26,10 @@ To update the image, simply run the above commands again and restart your runnin
 
 Should you require additional libraries, please [build the image yourself](#build-your-own-docker-image).
 
+!!! Note Docker image update frequency
+    The official docker images with tags `master`, `develop` and `latest` are automatically rebuild once a week to keep the base image uptodate.
+    In addition to that, every merge to `develop` will trigger a rebuild for `develop` and `latest`.
+
 ### Prepare the configuration files
 
 Even though you will use docker, you'll still need some files from the github repository.

docs/hyperopt.md

@@ -164,7 +164,11 @@ By default, FreqTrade uses a loss function, which has been with freqtrade since
 A different loss function can be specified by using the `--hyperopt-loss <Class-name>` argument.
 This class should be in its own file within the `user_data/hyperopts/` directory.
 
-Currently, the following loss functions are builtin: `DefaultHyperOptLoss` (default legacy Freqtrade hyperoptimization loss function), `SharpeHyperOptLoss` (optimizes Sharpe Ratio calculated on the trade returns) and `OnlyProfitHyperOptLoss` (which takes only amount of profit into consideration).
+Currently, the following loss functions are builtin:
+
+* `DefaultHyperOptLoss` (default legacy Freqtrade hyperoptimization loss function)
+* `OnlyProfitHyperOptLoss` (which takes only amount of profit into consideration)
+* `SharpeHyperOptLoss` (optimizes Sharpe Ratio calculated on the trade returns)
 
 ### Creating and using a custom loss function
 
@@ -348,6 +352,10 @@ def populate_buy_trend(self, dataframe: DataFrame) -> DataFrame:
     return dataframe
 ```
 
+By default, hyperopt prints colorized results -- epochs with positive profit are printed in the green color. This highlighting helps you find epochs that can be interesting for later analysis. Epochs with zero total profit or with negative profits (losses) are printed in the normal color. If you do not need colorization of results (for instance, when you are redirecting hyperopt output to a file) you can switch colorization off by specifying the `--no-color` option in the command line.
+
+You can use the `--print-all` command line option if you would like to see all results in the hyperopt output, not only the best ones. When `--print-all` is used, current best results are also colorized by default -- they are printed in bold (bright) style. This can also be switched off with the `--no-color` command line option.
+
 ### Understand Hyperopt ROI results
 
 If you are optimizing ROI (i.e. if optimization search-space contains 'all' or 'roi'), your result will look as follows and include a ROI table:

docs/strategy-customization.md

@@ -309,8 +309,10 @@ if self.dp:
         dataframe['best_bid'] = ob['bids'][0][0]
         dataframe['best_ask'] = ob['asks'][0][0]
 ```
-!Warning The order book is not part of the historic data which means backtesting and hyperopt will not work if this
- method is used.
+
+!!! Warning
+    The order book is not part of the historic data which means backtesting and hyperopt will not work if this
+    method is used.
 
 #### Available Pairs
 

freqtrade/configuration/__init__.py

@@ -1,2 +1,3 @@
-from freqtrade.configuration.arguments import Arguments, TimeRange  # noqa: F401
+from freqtrade.configuration.arguments import Arguments  # noqa: F401
+from freqtrade.configuration.timerange import TimeRange  # noqa: F401
 from freqtrade.configuration.configuration import Configuration  # noqa: F401

freqtrade/configuration/arguments.py

@@ -2,10 +2,8 @@
 This module contains the argument manager class
 """
 import argparse
-import re
-from typing import List, NamedTuple, Optional
+from typing import List, Optional
 
-import arrow
 from freqtrade.configuration.cli_options import AVAILABLE_CLI_OPTIONS
 from freqtrade import constants
 
@@ -23,7 +21,8 @@
 
 ARGS_HYPEROPT = ARGS_COMMON_OPTIMIZE + ["hyperopt", "hyperopt_path",
                                         "position_stacking", "epochs", "spaces",
-                                        "use_max_market_positions", "print_all", "hyperopt_jobs",
+                                        "use_max_market_positions", "print_all",
+                                        "print_colorized", "print_json", "hyperopt_jobs",
                                         "hyperopt_random_state", "hyperopt_min_trades",
                                         "hyperopt_continue", "hyperopt_loss"]
 
@@ -44,18 +43,6 @@
                     ["pairs", "timerange", "export", "exportfilename", "db_url", "trade_source"])
 
 
-class TimeRange(NamedTuple):
-    """
-    NamedTuple defining timerange inputs.
-    [start/stop]type defines if [start/stop]ts shall be used.
-    if *type is None, don't use corresponding startvalue.
-    """
-    starttype: Optional[str] = None
-    stoptype: Optional[str] = None
-    startts: int = 0
-    stopts: int = 0
-
-
 class Arguments(object):
     """
     Arguments Class. Manage the arguments received by the cli
@@ -138,45 +125,3 @@ def _build_subcommands(self) -> None:
         )
         list_exchanges_cmd.set_defaults(func=start_list_exchanges)
         self._build_args(optionlist=ARGS_LIST_EXCHANGES, parser=list_exchanges_cmd)
-
-    @staticmethod
-    def parse_timerange(text: Optional[str]) -> TimeRange:
-        """
-        Parse the value of the argument --timerange to determine what is the range desired
-        :param text: value from --timerange
-        :return: Start and End range period
-        """
-        if text is None:
-            return TimeRange(None, None, 0, 0)
-        syntax = [(r'^-(\d{8})$', (None, 'date')),
-                  (r'^(\d{8})-$', ('date', None)),
-                  (r'^(\d{8})-(\d{8})$', ('date', 'date')),
-                  (r'^-(\d{10})$', (None, 'date')),
-                  (r'^(\d{10})-$', ('date', None)),
-                  (r'^(\d{10})-(\d{10})$', ('date', 'date')),
-                  (r'^(-\d+)$', (None, 'line')),
-                  (r'^(\d+)-$', ('line', None)),
-                  (r'^(\d+)-(\d+)$', ('index', 'index'))]
-        for rex, stype in syntax:
-            # Apply the regular expression to text
-            match = re.match(rex, text)
-            if match:  # Regex has matched
-                rvals = match.groups()
-                index = 0
-                start: int = 0
-                stop: int = 0
-                if stype[0]:
-                    starts = rvals[index]
-                    if stype[0] == 'date' and len(starts) == 8:
-                        start = arrow.get(starts, 'YYYYMMDD').timestamp
-                    else:
-                        start = int(starts)
-                    index += 1
-                if stype[1]:
-                    stops = rvals[index]
-                    if stype[1] == 'date' and len(stops) == 8:
-                        stop = arrow.get(stops, 'YYYYMMDD').timestamp
-                    else:
-                        stop = int(stops)
-                return TimeRange(stype[0], stype[1], start, stop)
-        raise Exception('Incorrect syntax for timerange "%s"' % text)