atr (and others) indicators not working with resample #1243
Comments
|
Just to add:
On the left is atr applied to daily data, on the right is the data on the left converted back to minute data, i.e. the following command that I found in the docs: atr = atr.reindex(df.index).ffill()As you see the daily atr starts on 18 Jan while the one sampled back to 1m data starts on 20 Jan... tough the values are the same. |
|
According to docs:
data2 = data.resample('D',label='right').agg({'Open': 'first',
'High': 'max',
'Low': 'min',
'Close': 'last'})#.dropna()
print(data2)Gives as outputIndeed if I check: print(data[data.index.date == pd.to_datetime('2020-01-04').date()])Instead if I don't specify the label ```python
data2 = data.resample('D').agg({'Open': 'first',
'High': 'max',
'Low': 'min',
'Close': 'last'}).dropna()Everything works good, and also:
now the sma indicator and atr strart on the same day, which is the intended result. Hence the final resampled atr function would be: def average_daily_range(df, period):
# df_resampled = df.resample('D', label='right').agg({'High': 'max', 'Low': 'min', 'Close': 'last'})
df_resampled = df.resample('D').agg({'High': 'max', 'Low': 'min', 'Close': 'last'})
df_resampled.dropna(inplace=True)
atr = ta.atr(df_resampled['High'], df_resampled['Low'], df_resampled['Close'], period)
atr = atr.reindex(data.index).ffill()
return atrSo there are two problems that emerged from this issue:
|
Seems to break here: backtesting.py/backtesting/lib.py Lines 319 to 328 in b1a869c Following the logic, I think the inner branch is missing a trailing
Use of |
|
Yes but on my data there should be data in Sunday. In my minute data (forex) there is data from Sunday 22:00 to Friday 23:00. So the fact that the resampled version using Thanks for your kind response and for this amazing library. |
What its shape, ndim? It looks like this condition holds:
Note, the lib actually does: backtesting.py/backtesting/lib.py Lines 329 to 330 in b1a869c Does this help? |
|
Hello, so: atr = ta.atr(data.High, data.Low, data.Close)
print(type(atr))
print(atr.shape)Produces: So why doesn't it work if I do: Also the problem about |
|
Doing more tests... Resampling candlestick data from 1min to 5min using suggested data5m = data.resample('5min', label='right').agg({
"Open": "first",
"High": "max",
"Low": "min",
"Close": "last",
})Produces:
As you see the end result is a candlestick that starts at
Indeed, if I don't use data5m = data.resample('5min').agg({
"Open": "first",
"High": "max",
"Low": "min",
"Close": "last",
})I obtain the intended outcome:
So the resample_apply function:
Edit: I double checked with TradingView charts and they are identical, proving even further we don't need `label='right'`Resampled data chart:
Tradingview:
|
Thanks for the illustrative example! You don't learn that the supposed 22:00:00 bar closed at 1.20032 until 22:04:00 bar closes! I don't know what labeling your data source uses, but plotting that info anytime before the complete end of bar 22:04:00 would introduce look-ahead bias. Likewise, applying a simple passthrough function: from backtesting import Strategy, Backtest
from backtesting.test import EURUSD
class S(Strategy):
def init(self):
resample_apply('1d', lambda x: x, self.data.Close, color='blue')
def next(self):
pass
bt = Backtest(EURUSD, S)
_ = bt.run()
bt.plot()
you can see it uses previous complete bar's value as the current value. Had it used the current bar's (potentially incomplete) value, this would introduce look-ahead bias and would redraw / repaint / mislead, like TradingView does. This part of the issue is "works-as-planned" / wontfix.
Please provide the following output: >>> atr = ta.atr(h, l, c)
>>> atr.__class__.__mro__
>>> atr
>>> np.ndim(atr) |
|
Dear kernc, yes I see your point of view. I always used the TradingView and MT4/5 way of labeling data, always been aware of repainting and learned to account for it on my testings.
Yes, the idea is that the Open price is fixed and High, Low, Close changes every new tick until the candle closes. I see that other data providers, like Bloomberg, label the data on the close.
That's perfectly fine, I'll account for this. Thank's for taking time to explain :)
atr = ta.atr(data.High, data.Low, data.Close)
print(atr.__class__.__mro__)
print(atr)
print(np.ndim(atr)) |
|
Well, that's confusing. If the result object is already a Series, there's no way I see for it to crash with |
|
Well... If you manage to find some time you can manage to try for yourself... import pandas_ta as ta
from backtesting import Strategy
import backtesting as bt
from backtesting.lib import resample_apply
# Import some minute-level data
class myStrat(Strategy):
def init(self):
self.atr = resample_apply('D', ta.atr, self.data.High, self.data.Low, self.data.Close)
...Sorry if the snippet is not 100% accurate, I'm on my phone. Anyway I'm sure you get what I'm trying to say. |
Hi, I'm using minute level data:
Dataframe sample
I need the average daily range, so I thought I could just resample the atr to daily frequency. So I followed the documentation:
Error output:
However I don't get any error with:
So, again following the docs, I tried doing it myself:
Resampled df:
Now I have a working ATR resampled to daily... But there is a problem, as you may have noticed both sma and atr are resampled daily with a period of 14:
As you see the ATR start on 20 jan 2020 at 12:00, while the SMA start on 17 jan 2020 at 12:00. So am I doing something wrong or is the library that should be updated?
Packages version:
The text was updated successfully, but these errors were encountered: