Currency: Add additional functionality, refactor and improvements (th…

…rasher-corp#881)

* currency: Add method to derive pair

* currency: Add method to lower entire charset but used the slice copy and returned that. This will change the original, just gotta see if this is an issue, but the slice usually goes out of scope anyway.

* currency/pairs: add filter method

* currency: add function to derive select currencies from currency pairs

* currency/engine: slight adjustments

* currency: fix linter issue also shift burden of proof to caller instead of repair, more performant.

* currency: more linter

* pairs: optimize; reduce allocs/op and B/op

* currency: Add in function 'NewPairsFromString' for testing purposes

* currency: don't suppress error

* currency: stop panic on empty currency code

* currency: Add helper method to match currencies between exchanges

* currency: fixed my bad spelling

* currency: Implement stable coin checks, refactored base code methods, optimized upper and lower case strings for currency code/pairs

* currency: add pairs method to derive stable coins from internal list.

* Currency: Cleanup, fix tests.

* engine/exchanges/currency: fix whoops

* Currency: force govet no copy on Item datatype

* Currency: fix naughty linter issues

* exchange: revert change

* currency/config: fix config upgrade mistake

* currency: re-implement currency sub-systems

* *RetrieveConfigCurrencyPairs removed
*CheckCurrencyConfigValues to only provide warnings, add additional support when, disable when support is lost or not available and set default values.
*Drop Cryptocurrencies from configuration as this is not needed.
*Drop REST Poll delay field as this was unused.
*Update default values for currencyFileUpdateDuration & foreignExchangeUpdateDuration.
*Allow Role to be marshalled for file type.
*Refactor RunUpdater to verify and check config values and set default running foreign exchange provider.

* currency: cleanup

* currency: change match -> equal for comparison which is more of a standard and little easier to find

* currency: address nits

* currency: fix whoops

* currency: Add some more pairs methods

* currency: linter issues

* currency: RM unused field

* currency: rm verbose

* currency: fix word

* currency: gocritic

* currency: fix another whoopsie

* example_config: default to show log system name

* Currency: Force all support packages to use Equal method for comparison as there is a small comparison bug when checking upper and lower casing, this has a more of a pronounced impact between exchanges and client instances of currency generation

* currency: fix log name

* ordermanager: fix potential panic

* currency: small optim.

* engine: display correct bool and force shutdown

* currency: add function and fix regression
* Change ConvertCurrency -> ConvertFiat to be more precise
* ADD GetForeignExchangeRate to get specific exchange rate for fiat pair
* Fix currency display and formatting regression and tied in with config.Currency fields

* engine: fix tests

* currency: return the amount when no conversion needs to take place

* currency: reduce method name

* currency: Address nits glorious nits

* currency: fix linter

* currency: addr nits

* currency: check underlying role in test

* gct: change to EMPTYCODE and EMPTYPAIR across codebase

* currency: fix nits

* currency: this fixes test race but this issue has not been resolved. Please see: https://trello.com/c/54eizOIo/143-currency-package-upgrades

* currency: Add temp dir for testing

* Update engine/engine.go

Co-authored-by: Adrian Gallagher <[email protected]>

* documentation: update and regen

* currency: Address niterinos

* currency: Add test case for config upgrade when falling over to exchange rate host as default from exchangeRates provider

* currency: addr nits

* currency: fix whoops

Co-authored-by: Ryan O'Hara-Reid <[email protected]>
Co-authored-by: Adrian Gallagher <[email protected]>

README.md

@@ -68,7 +68,7 @@ However, we welcome pull requests for any exchange which does not match this cri
 + Connection monitor package.
 + gRPC service and JSON RPC proxy. See [gRPC service](/gctrpc/README.md).
 + gRPC client. See [gctcli](/cmd/gctcli/README.md).
-+ Forex currency converter packages (CurrencyConverterAPI, CurrencyLayer, Fixer.io, OpenExchangeRates).
++ Forex currency converter packages (CurrencyConverterAPI, CurrencyLayer, Exchange Rates, Fixer.io, OpenExchangeRates, Exchange Rate Host).
 + Packages for handling currency pairs, tickers and orderbooks.
 + Portfolio management tool; fetches balances from supported exchanges and allows for custom address tracking.
 + Basic event trigger system.
@@ -143,13 +143,13 @@ Binaries will be published once the codebase reaches a stable condition.
 
 |User|Contribution Amount|
 |--|--|
-| [thrasher-](https://github.com/thrasher-) | 662 |
-| [shazbert](https://github.com/shazbert) | 231 |
+| [thrasher-](https://github.com/thrasher-) | 664 |
+| [shazbert](https://github.com/shazbert) | 232 |
 | [gloriousCode](https://github.com/gloriousCode) | 194 |
 | [dependabot-preview[bot]](https://github.com/apps/dependabot-preview) | 88 |
-| [dependabot[bot]](https://github.com/apps/dependabot) | 50 |
+| [dependabot[bot]](https://github.com/apps/dependabot) | 57 |
 | [xtda](https://github.com/xtda) | 47 |
-| [lrascao](https://github.com/lrascao) | 21 |
+| [lrascao](https://github.com/lrascao) | 27 |
 | [Rots](https://github.com/Rots) | 15 |
 | [vazha](https://github.com/vazha) | 15 |
 | [ydm](https://github.com/ydm) | 15 |

backtester/backtest/backtest.go

@@ -525,19 +525,19 @@ func (bt *BackTest) setupExchangeSettings(cfg *config.Config) (exchange.Exchange
 func (bt *BackTest) loadExchangePairAssetBase(exch, base, quote, ass string) (gctexchange.IBotExchange, currency.Pair, asset.Item, error) {
 	e, err := bt.exchangeManager.GetExchangeByName(exch)
 	if err != nil {
-		return nil, currency.Pair{}, "", err
+		return nil, currency.EMPTYPAIR, "", err
 	}
 
 	var cp, fPair currency.Pair
 	cp, err = currency.NewPairFromStrings(base, quote)
 	if err != nil {
-		return nil, currency.Pair{}, "", err
+		return nil, currency.EMPTYPAIR, "", err
 	}
 
 	var a asset.Item
 	a, err = asset.New(ass)
 	if err != nil {
-		return nil, currency.Pair{}, "", err
+		return nil, currency.EMPTYPAIR, "", err
 	}
 
 	exchangeBase := e.GetBase()
@@ -547,7 +547,7 @@ func (bt *BackTest) loadExchangePairAssetBase(exch, base, quote, ass string) (gc
 
 	fPair, err = exchangeBase.FormatExchangeCurrency(cp, a)
 	if err != nil {
-		return nil, currency.Pair{}, "", err
+		return nil, currency.EMPTYPAIR, "", err
 	}
 	return e, fPair, a, nil
 }

backtester/eventhandlers/exchange/exchange.go

@@ -320,7 +320,7 @@ func (e *Exchange) SetExchangeAssetCurrencySettings(exch string, a asset.Item, c
 	}
 
 	for i := range e.CurrencySettings {
-		if e.CurrencySettings[i].Pair == cp &&
+		if e.CurrencySettings[i].Pair.Equal(cp) &&
 			e.CurrencySettings[i].Asset == a &&
 			exch == e.CurrencySettings[i].Exchange {
 			e.CurrencySettings[i] = *c

backtester/eventhandlers/exchange/exchange_test.go

@@ -44,7 +44,7 @@ func TestReset(t *testing.T) {
 func TestSetCurrency(t *testing.T) {
 	t.Parallel()
 	e := Exchange{}
-	e.SetExchangeAssetCurrencySettings("", "", currency.Pair{}, &Settings{})
+	e.SetExchangeAssetCurrencySettings("", "", currency.EMPTYPAIR, &Settings{})
 	if len(e.CurrencySettings) != 0 {
 		t.Error("expected 0")
 	}
@@ -264,7 +264,7 @@ func TestExecuteOrder(t *testing.T) {
 	d := &kline.DataFromKline{
 		Item: gctkline.Item{
 			Exchange: "",
-			Pair:     currency.Pair{},
+			Pair:     currency.EMPTYPAIR,
 			Asset:    "",
 			Interval: 0,
 			Candles: []gctkline.Candle{
@@ -385,7 +385,7 @@ func TestExecuteOrderBuySellSizeLimit(t *testing.T) {
 	d := &kline.DataFromKline{
 		Item: gctkline.Item{
 			Exchange: "",
-			Pair:     currency.Pair{},
+			Pair:     currency.EMPTYPAIR,
 			Asset:    "",
 			Interval: 0,
 			Candles: []gctkline.Candle{

backtester/eventhandlers/portfolio/portfolio_test.go

@@ -303,7 +303,7 @@ func TestUpdate(t *testing.T) {
 func TestGetFee(t *testing.T) {
 	t.Parallel()
 	p := Portfolio{}
-	f := p.GetFee("", "", currency.Pair{})
+	f := p.GetFee("", "", currency.EMPTYPAIR)
 	if !f.IsZero() {
 		t.Error("expected 0")
 	}
@@ -323,7 +323,7 @@ func TestGetFee(t *testing.T) {
 func TestGetComplianceManager(t *testing.T) {
 	t.Parallel()
 	p := Portfolio{}
-	_, err := p.GetComplianceManager("", "", currency.Pair{})
+	_, err := p.GetComplianceManager("", "", currency.EMPTYPAIR)
 	if !errors.Is(err, errNoPortfolioSettings) {
 		t.Errorf("received: %v, expected: %v", err, errNoPortfolioSettings)
 	}

backtester/eventhandlers/statistics/fundingstatistics.go

@@ -35,11 +35,11 @@ func CalculateFundingStatistics(funds funding.IFundingManager, currStats map[str
 		}
 		var relevantStats []relatedCurrencyPairStatistics
 		for k, v := range exchangeAssetStats {
-			if k.Base == report.Items[i].Currency {
+			if k.Base.Equal(report.Items[i].Currency) {
 				relevantStats = append(relevantStats, relatedCurrencyPairStatistics{isBaseCurrency: true, stat: v})
 				continue
 			}
-			if k.Quote == report.Items[i].Currency {
+			if k.Quote.Equal(report.Items[i].Currency) {
 				relevantStats = append(relevantStats, relatedCurrencyPairStatistics{stat: v})
 			}
 		}

backtester/funding/funding.go

@@ -112,7 +112,7 @@ func (f *FundManager) AddUSDTrackingData(k *kline.DataFromKline) error {
 		}
 		if strings.EqualFold(f.items[i].exchange, k.Item.Exchange) &&
 			f.items[i].asset == k.Item.Asset {
-			if f.items[i].currency == k.Item.Pair.Base {
+			if f.items[i].currency.Equal(k.Item.Pair.Base) {
 				if f.items[i].usdTrackingCandles == nil &&
 					trackingcurrencies.CurrencyIsUSDTracked(k.Item.Pair.Quote) {
 					f.items[i].usdTrackingCandles = k
@@ -123,7 +123,7 @@ func (f *FundManager) AddUSDTrackingData(k *kline.DataFromKline) error {
 				baseSet = true
 			}
 			if trackingcurrencies.CurrencyIsUSDTracked(f.items[i].currency) {
-				if f.items[i].pairedWith != nil && f.items[i].currency != basePairedWith {
+				if f.items[i].pairedWith != nil && !f.items[i].currency.Equal(basePairedWith) {
 					continue
 				}
 				if f.items[i].usdTrackingCandles == nil {
@@ -267,10 +267,10 @@ func (f *FundManager) Transfer(amount decimal.Decimal, sender, receiver *Item, i
 		}
 	}
 
-	if sender.currency != receiver.currency {
+	if !sender.currency.Equal(receiver.currency) {
 		return errTransferMustBeSameCurrency
 	}
-	if sender.currency == receiver.currency &&
+	if sender.currency.Equal(receiver.currency) &&
 		sender.exchange == receiver.exchange &&
 		sender.asset == receiver.asset {
 		return fmt.Errorf("%v %v %v %w", sender.exchange, sender.asset, sender.currency, errCannotTransferToSameFunds)
@@ -336,7 +336,7 @@ func (f *FundManager) GetFundingForEvent(ev common.EventHandler) (*Pair, error)
 // GetFundingForEAC This will construct a funding based on the exchange, asset, currency code
 func (f *FundManager) GetFundingForEAC(exch string, a asset.Item, c currency.Code) (*Item, error) {
 	for i := range f.items {
-		if f.items[i].BasicEqual(exch, a, c, currency.Code{}) {
+		if f.items[i].BasicEqual(exch, a, c, currency.EMPTYCODE) {
 			return f.items[i], nil
 		}
 	}
@@ -514,7 +514,7 @@ func (i *Item) Equal(item *Item) bool {
 	if item == nil || i == nil {
 		return false
 	}
-	if i.currency == item.currency &&
+	if i.currency.Equal(item.currency) &&
 		i.asset == item.asset &&
 		i.exchange == item.exchange {
 		if i.pairedWith == nil && item.pairedWith == nil {
@@ -523,7 +523,7 @@ func (i *Item) Equal(item *Item) bool {
 		if i.pairedWith == nil || item.pairedWith == nil {
 			return false
 		}
-		if i.pairedWith.currency == item.pairedWith.currency &&
+		if i.pairedWith.currency.Equal(item.pairedWith.currency) &&
 			i.pairedWith.asset == item.pairedWith.asset &&
 			i.pairedWith.exchange == item.pairedWith.exchange {
 			return true
@@ -537,19 +537,19 @@ func (i *Item) BasicEqual(exch string, a asset.Item, currency, pairedCurrency cu
 	return i != nil &&
 		i.exchange == exch &&
 		i.asset == a &&
-		i.currency == currency &&
+		i.currency.Equal(currency) &&
 		(i.pairedWith == nil ||
-			(i.pairedWith != nil && i.pairedWith.currency == pairedCurrency))
+			(i.pairedWith != nil && i.pairedWith.currency.Equal(pairedCurrency)))
 }
 
 // MatchesCurrency checks that an item's currency is equal
 func (i *Item) MatchesCurrency(c currency.Code) bool {
-	return i != nil && i.currency == c
+	return i != nil && i.currency.Equal(c)
 }
 
 // MatchesItemCurrency checks that an item's currency is equal
 func (i *Item) MatchesItemCurrency(item *Item) bool {
-	return i != nil && item != nil && i.currency == item.currency
+	return i != nil && item != nil && i.currency.Equal(item.currency)
 }
 
 // MatchesExchange checks that an item's exchange is equal

backtester/funding/funding_test.go

@@ -247,12 +247,12 @@ func TestAddPair(t *testing.T) {
 	}
 	if resp.Base.exchange != exch ||
 		resp.Base.asset != a ||
-		resp.Base.currency != pair.Base {
+		!resp.Base.currency.Equal(pair.Base) {
 		t.Error("woah nelly")
 	}
 	if resp.Quote.exchange != exch ||
 		resp.Quote.asset != a ||
-		resp.Quote.currency != pair.Quote {
+		!resp.Quote.currency.Equal(pair.Quote) {
 		t.Error("woah nelly")
 	}
 	if resp.Quote.pairedWith != resp.Base {
@@ -841,7 +841,7 @@ func TestMatchesCurrency(t *testing.T) {
 	if !i.MatchesCurrency(currency.BTC) {
 		t.Error("expected true")
 	}
-	if i.MatchesCurrency(currency.Code{}) {
+	if i.MatchesCurrency(currency.EMPTYCODE) {
 		t.Error("expected false")
 	}
 	if i.MatchesCurrency(currency.NewCode("")) {
@@ -855,7 +855,7 @@ func TestCreateSnapshot(t *testing.T) {
 	f.items = append(f.items, &Item{
 		exchange:           "",
 		asset:              "",
-		currency:           currency.Code{},
+		currency:           currency.EMPTYCODE,
 		initialFunds:       decimal.Decimal{},
 		available:          decimal.Decimal{},
 		reserved:           decimal.Decimal{},

backtester/funding/trackingcurrencies/trackingcurrencies.go

@@ -121,13 +121,13 @@ func pairContainsUSD(pair currency.Pair) bool {
 // this will allow for data retrieval and total tracking on backtesting runs
 func findMatchingUSDPairs(pair currency.Pair, pairs *currency.PairStore) (basePair, quotePair currency.Pair, err error) {
 	if pairs == nil {
-		return currency.Pair{}, currency.Pair{}, errNilPairs
+		return currency.EMPTYPAIR, currency.EMPTYPAIR, errNilPairs
 	}
 	if pairContainsUSD(pair) {
-		return currency.Pair{}, currency.Pair{}, ErrCurrencyContainsUSD
+		return currency.EMPTYPAIR, currency.EMPTYPAIR, ErrCurrencyContainsUSD
 	}
 	if !pairs.Available.Contains(pair, true) {
-		return currency.Pair{}, currency.Pair{}, fmt.Errorf("%v %w", pair, errCurrencyNotFoundInPairs)
+		return currency.EMPTYPAIR, currency.EMPTYPAIR, fmt.Errorf("%v %w", pair, errCurrencyNotFoundInPairs)
 	}
 	var baseFound, quoteFound bool
 

backtester/funding/trackingcurrencies/trackingcurrencies_test.go

@@ -82,8 +82,8 @@ func TestFindMatchingUSDPairs(t *testing.T) {
 			description:    "already has USD",
 			initialPair:    currency.NewPair(currency.BTC, currency.USDT),
 			availablePairs: &currency.PairStore{Available: currency.Pairs{currency.NewPair(currency.BTC, currency.USDT)}},
-			basePair:       currency.Pair{},
-			quotePair:      currency.Pair{},
+			basePair:       currency.EMPTYPAIR,
+			quotePair:      currency.EMPTYPAIR,
 			expectedErr:    ErrCurrencyContainsUSD,
 		},
 		{
@@ -99,31 +99,31 @@ func TestFindMatchingUSDPairs(t *testing.T) {
 			initialPair:    currency.NewPair(currency.BTC, currency.LTC),
 			availablePairs: &currency.PairStore{Available: currency.Pairs{currency.NewPair(currency.BTC, currency.LTC), currency.NewPair(currency.BTC, currency.DAI)}},
 			basePair:       currency.NewPair(currency.BTC, currency.DAI),
-			quotePair:      currency.Pair{},
+			quotePair:      currency.EMPTYPAIR,
 			expectedErr:    errNoMatchingQuoteUSDFound,
 		},
 		{
 			description:    "base currency has no matching USD pair",
 			initialPair:    currency.NewPair(currency.BTC, currency.LTC),
 			availablePairs: &currency.PairStore{Available: currency.Pairs{currency.NewPair(currency.BTC, currency.LTC), currency.NewPair(currency.LTC, currency.USDT)}},
-			basePair:       currency.Pair{},
+			basePair:       currency.EMPTYPAIR,
 			quotePair:      currency.NewPair(currency.LTC, currency.USDT),
 			expectedErr:    errNoMatchingBaseUSDFound,
 		},
 		{
 			description:    "both base and quote don't have USD pairs",
 			initialPair:    currency.NewPair(currency.BTC, currency.LTC),
 			availablePairs: &currency.PairStore{Available: currency.Pairs{currency.NewPair(currency.BTC, currency.LTC)}},
-			basePair:       currency.Pair{},
-			quotePair:      currency.Pair{},
+			basePair:       currency.EMPTYPAIR,
+			quotePair:      currency.EMPTYPAIR,
 			expectedErr:    errNoMatchingPairUSDFound,
 		},
 		{
 			description:    "currency doesnt exist in available pairs",
 			initialPair:    currency.NewPair(currency.BTC, currency.LTC),
 			availablePairs: &currency.PairStore{Available: currency.Pairs{currency.NewPair(currency.BTC, currency.DOGE)}},
-			basePair:       currency.Pair{},
-			quotePair:      currency.Pair{},
+			basePair:       currency.EMPTYPAIR,
+			quotePair:      currency.EMPTYPAIR,
 			expectedErr:    errCurrencyNotFoundInPairs,
 		},
 	}

cmd/documentation/currency_templates/fx.tmpl

@@ -4,6 +4,7 @@
 
 + Currency Converter API support
 + Currency Layer support
++ Exchange Rates support
 + Fixer.io support
 + Open Exchange Rates support
 + ExchangeRate.host support

cmd/documentation/root_templates/root_readme.tmpl

@@ -69,7 +69,7 @@ However, we welcome pull requests for any exchange which does not match this cri
 + Connection monitor package.
 + gRPC service and JSON RPC proxy. See [gRPC service](/gctrpc/README.md).
 + gRPC client. See [gctcli](/cmd/gctcli/README.md).
-+ Forex currency converter packages (CurrencyConverterAPI, CurrencyLayer, Fixer.io, OpenExchangeRates).
++ Forex currency converter packages (CurrencyConverterAPI, CurrencyLayer, Exchange Rates, Fixer.io, OpenExchangeRates, Exchange Rate Host).
 + Packages for handling currency pairs, tickers and orderbooks.
 + Portfolio management tool; fetches balances from supported exchanges and allows for custom address tracking.
 + Basic event trigger system.

cmd/exchange_wrapper_issues/exchange_wrapper_issues_test.go

@@ -7,7 +7,7 @@ import (
 )
 
 func TestDisruptFormatting(t *testing.T) {
-	_, err := disruptFormatting(currency.Pair{})
+	_, err := disruptFormatting(currency.EMPTYPAIR)
 	if err == nil {
 		t.Fatal("error cannot be nil")
 	}

cmd/exchange_wrapper_issues/main.go

@@ -953,10 +953,10 @@ func outputToConsole(exchangeResponses []ExchangeResponses) {
 // ensure format currency pair is used throughout the code base.
 func disruptFormatting(p currency.Pair) (currency.Pair, error) {
 	if p.Base.IsEmpty() {
-		return currency.Pair{}, errors.New("cannot disrupt formatting as base is not populated")
+		return currency.EMPTYPAIR, errors.New("cannot disrupt formatting as base is not populated")
 	}
 	if p.Quote.IsEmpty() {
-		return currency.Pair{}, errors.New("cannot disrupt formatting as quote is not populated")
+		return currency.EMPTYPAIR, errors.New("cannot disrupt formatting as quote is not populated")
 	}
 
 	return currency.Pair{