Are there any examples of using Transformers for Time Series?

73 次查看（过去 30 天）
Yeray
Yeray 2022-12-22
回答： Philip Brown 2024-5-30
Good afternoon All,
I'm researching how to develop/train/use Transformers (those similar to GPT) for S&P 500 historical data as an example. It seems like Transformers are mainly used for NLP but there are articles out there that utilize them for time series. Can any fellow lead the way in how could I get started in doing these in MATLAB? I have done general things like convolutions, LSTMs, etc. but not on a transformers caliber with the DL Toolbox.
I found this MATLAB package (pasted below) to be somewhat useful but it seems to lean more into language rather than historical data points.
https://www.mathworks.com/matlabcentral/fileexchange/107375-transformer-models
Another side question: It's possibe to use the Deep Network Designer app to layout a transformers architecture?
Any tip is appreciated.
Thank you,
Yeray

请先登录，再回答此问题。

采纳的回答

Philip Brown
Philip Brown 2024-5-30
Update in 2024: there's a GitHub repo/FileExchange submission and blog post covering using transformers for time series data which I think closely matches your request.
Deep Network Designer supports transformer layers; more details in this answer.

更多回答（1 个）

Sandeep
Sandeep 2023-8-31
编辑：Sandeep 2023-8-31
Hi Yeray,
Developing and training Transformers for time series data, such as S&P 500 historical data, is indeed possible. While Transformers are widely used in NLP tasks, they can also be applied to other domains, including time series analysis.
To get started, you can take a look at the following tips:
  1. Familiarize yourself with the basics of Transformers and their architecture. Understanding concepts like self-attention, multi-head attention, and positional encoding will be helpful.
  2. Explore the MATLAB package you found Transformer Models. Although it may be more focused on language tasks, you can still adapt it for time series analysis.
  3. Preprocess your S&P 500 historical data into a suitable format for training the Transformer model. You may need to consider factors such as input window size, target prediction horizon, and normalization.
  4. Train the Transformer model using your preprocessed data by adjusting the hyperparameters such as learning rate, batch size, and number of layers, to achieve optimal performance.
  5. Evaluate the trained model's performance on validation or test data using appropriate metrics for time series analysis, such as mean squared error (MSE) or mean absolute error (MAE).
  6. Remember that developing and training Transformers for time series analysis may require some experimentation and fine-tuning to achieve the desired results.
Hope you find it helpful. Good luck with your research!

类别

在 Help Center 和 File Exchange 中查找有关 Sequence and Numeric Feature Data Workflows 的更多信息

标签

产品


版本

R2022b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by