Skip to content

Commit

Permalink
docs: fix and improve content in tutorial/TimesNet_tutorial.ipynb
Browse files Browse the repository at this point in the history
  • Loading branch information
@shresthasurav
shresthasurav committed Oct 21, 2023
1 parent 8e3f7dc commit dd15a3e
Showing 1 changed file with 17 additions and 17 deletions.
34 changes: 17 additions & 17 deletions tutorial/TimesNet_tutorial.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@
"metadata": {},
"source": [
"### 3. TimesBlock Construction\n",
" The core idea of `TimesNet` lies in the construction of `TimesBlock`, which generally gets the base frequencies by implementing FFT on the data, and then reshapes the times series to 2D variation respectivelly from the main base frequencies, followed by a 2D convolution whose outputs are reshaped back and added with weight to form the final output.\n",
" The core idea of `TimesNet` lies in the construction of `TimesBlock`, which generally gets the base frequencies by implementing FFT on the data, and then reshapes the times series to 2D variation respectively from the main base frequencies, followed by a 2D convolution whose outputs are reshaped back and added with weight to form the final output.\n",
"\n",
" In the following section, we will have a detailed view on `TimesBlock`.\n",
"\n",
Expand Down Expand Up @@ -125,14 +125,14 @@
" #B: batch size T: length of time series N:number of features\n",
" period_list, period_weight = FFT_for_Period(x, self.k)\n",
" #FFT_for_Period() will be shown later. Here, period_list([top_k]) denotes \n",
" #the top_k-significant period and peroid_weight([B, top_k]) denotes its weight(amplitude)\n",
" #the top_k-significant period and period_weight([B, top_k]) denotes its weight(amplitude)\n",
"\n",
" res = []\n",
" for i in range(self.k):\n",
" period = period_list[i]\n",
"\n",
" # padding : to form a 2D map, we need total length of the sequence, plus the part \n",
" # to be predicted, to be devisible by the peroid, so padding is needed\n",
" # to be predicted, to be divisible by the period, so padding is needed\n",
" if (self.seq_len + self.pred_len) % period != 0:\n",
" length = (\n",
" ((self.seq_len + self.pred_len) // period) + 1) * period\n",
Expand All @@ -149,7 +149,7 @@
" out = out.reshape(B, length // period, period,\n",
" N).permute(0, 3, 1, 2).contiguous()\n",
" \n",
" #2D convolution to grap the intra- and inter- peroid information\n",
" #2D convolution to grap the intra- and inter- period information\n",
" out = self.conv(out)\n",
"\n",
" # reshape back, similar to reshape\n",
Expand All @@ -167,7 +167,7 @@
" period_weight = period_weight.unsqueeze(\n",
" 1).unsqueeze(1).repeat(1, T, N, 1)\n",
" \n",
" #add by weight the top_k peroids' result, getting the result of this TimesBlock\n",
" #add by weight the top_k periods' result, getting the result of this TimesBlock\n",
" res = torch.sum(res * period_weight, -1)\n",
"\n",
" # residual connection\n",
Expand All @@ -192,7 +192,7 @@
" # xf shape [B, T, C], denoting the amplitude of frequency(T) given the datapiece at B,N\n",
" xf = torch.fft.rfft(x, dim=1) \n",
"\n",
" # find period by amplitudes: here we assume that the peroidic features are basically constant\n",
" # find period by amplitudes: here we assume that the periodic features are basically constant\n",
" # in different batch and channel, so we mean out these two dimensions, getting a list frequency_list with shape[T] \n",
" # each element at pos t of frequency_list denotes the overall amplitude at frequency (t)\n",
" frequency_list = abs(xf).mean(0).mean(-1) \n",
Expand All @@ -205,7 +205,7 @@
" #The result will never require gradient.Convert to a numpy instance\n",
" top_list = top_list.detach().cpu().numpy()\n",
" \n",
" #peroid:a list of shape [top_k], recording the peroids of mean frequencies respectively\n",
" #period:a list of shape [top_k], recording the periods of mean frequencies respectively\n",
" period = x.shape[1] // top_list\n",
"\n",
" #Here,the 2nd item returned has a shape of [B, top_k],representing the biggest top_k amplitudes \n",
Expand Down Expand Up @@ -238,7 +238,7 @@
"source": [
"### 4. TimesNet\n",
"\n",
"So far we've got `TimesBlock`, which is excel at retreiving intra- and inter- peroid temporal information. We become capable of building a `TimesNet`. `TimesNet` is proficient in multitasks including short- and long-term forecasting, imputation, classification, and anomaly detection.\n",
"So far we've got `TimesBlock`, which is excel at retrieving intra- and inter- period temporal information. We become capable of building a `TimesNet`. `TimesNet` is proficient in multitasks including short- and long-term forecasting, imputation, classification, and anomaly detection.\n",
"\n",
"In this section, we'll have a detailed overview on how `TimesNet` gains its power in these tasks."
]
Expand Down Expand Up @@ -325,7 +325,7 @@
"source": [
"#### 4.1 Forecast\n",
"\n",
"The basic idea of forecasting is to lengthen the known sequence to (seq_len+pred_len), which is the total length after forecasting. Then by several TimesBlock layers together with layer normalization, some underlying intra- and inter- peroid information is represented. With these information, we can project it to the output space. Whereafter by denorm ( if Non-stationary Transformer) we get the final output."
"The basic idea of forecasting is to lengthen the known sequence to (seq_len+pred_len), which is the total length after forecasting. Then by several TimesBlock layers together with layer normalization, some underlying intra- and inter- period information is represented. With these information, we can project it to the output space. Whereafter by denorm ( if Non-stationary Transformer) we get the final output."
]
},
{
Expand Down Expand Up @@ -395,7 +395,7 @@
" # TimesNet\n",
" for i in range(self.layer):\n",
" enc_out = self.layer_norm(self.model[i](enc_out))\n",
" # porject back\n",
" # project back\n",
" dec_out = self.projection(enc_out)\n",
"\n",
" # De-Normalization from Non-stationary Transformer\n",
Expand Down Expand Up @@ -435,7 +435,7 @@
" # TimesNet\n",
" for i in range(self.layer):\n",
" enc_out = self.layer_norm(self.model[i](enc_out))\n",
" # porject back\n",
" # project back\n",
" dec_out = self.projection(enc_out)\n",
" # De-Normalization from Non-stationary Transformer\n",
" dec_out = dec_out * \\\n",
Expand Down Expand Up @@ -520,7 +520,7 @@
"source": [
"### 5. Training and Settings\n",
"\n",
"By now we've succesfully build up `TimesNet`. We are now facing the problem how to train and test this neural network. The action of training, validating as well as testing is implemented at __*exp*__ part, in which codes for different tasks are gathered. These experiments are not only for `TimesNet` training, but also feasible for any other time series representation model. But here, we simply use `TimesNet` to analyse.\n",
"By now we've successfully build up `TimesNet`. We are now facing the problem how to train and test this neural network. The action of training, validating as well as testing is implemented at __*exp*__ part, in which codes for different tasks are gathered. These experiments are not only for `TimesNet` training, but also feasible for any other time series representation model. But here, we simply use `TimesNet` to analyse.\n",
"\n",
"`TimesNet` is a state-of-art in multiple tasks, while here we would only introduce its training for long-term forecast task, since the backbone of the training process for other tasks is similar to this one. Again, test and validation code can be easily understood once you've aware how the training process works. So first of all, we are going to focus on the training of `TimesNet` on task long-term forecasting.\n",
"\n",
Expand All @@ -531,9 +531,9 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"#### 5.1 Training for Long-term Forcast Task\n",
"#### 5.1 Training for Long-term Forecast Task\n",
"\n",
"The following codes represents the process of training model for long-term forcasting task. We'll have a detailed look at it. To make it brief, the training part can be briefly divided into several parts, including Data Preparation, Creating Save Path, Initialization, Optimizer and Loss Function Selection, Using Mixed Precision Training, Training Loop, Validation and Early Stopping, Learning Rate Adjustment, Loading the Best Model.\n",
"The following codes represents the process of training model for long-term forecasting task. We'll have a detailed look at it. To make it brief, the training part can be briefly divided into several parts, including Data Preparation, Creating Save Path, Initialization, Optimizer and Loss Function Selection, Using Mixed Precision Training, Training Loop, Validation and Early Stopping, Learning Rate Adjustment, Loading the Best Model.\n",
"\n",
"For more details, please see the code below. 'train' process is defined in the experiment <font color=orange>__class Exp_Long_Term_Forecast__</font>."
]
Expand Down Expand Up @@ -692,7 +692,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"#### 5.2 Early Stoping Mechanism\n",
"#### 5.2 Early Stopping Mechanism\n",
"\n",
"<font color=purple>__EarlyStopping__</font> is typically a custom class or function that monitors the performance of a model during training, usually by tracking a certain metric (commonly validation loss or accuracy).It's a common technique used in deep learning to prevent overfitting during the training.\n",
"\n",
Expand Down Expand Up @@ -1040,7 +1040,7 @@
"source": [
"### 7. Dataloader and DataProvider\n",
"\n",
"In the process of training, we simply take the dataloader for granted, by the function `self._get_data(flag='train')`. So how does this line work? Have a look at the defination(in <font color=orange>__class Exp_Long_Term_Forecast__</font>):"
"In the process of training, we simply take the dataloader for granted, by the function `self._get_data(flag='train')`. So how does this line work? Have a look at the definition(in <font color=orange>__class Exp_Long_Term_Forecast__</font>):"
]
},
{
Expand Down Expand Up @@ -1459,7 +1459,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Here, the bash command may not succussfully implemented for lack of proper packages in the environment.If so, just follow the error info to install the missed package step by step, until reach a success. The sign of the experiment succussfully being running is that the information about this experiment be printed out, like:"
"Here, the bash command may not be successfully implemented due to a lack of proper packages in the environment. If that's the case, simply follow the error information to install the missing package step by step until you achieve success. The sign of a successful experiment running is that information about the experiment is printed out, such as:"
]
},
{
Expand Down

0 comments on commit dd15a3e

Please sign in to comment.