A Transformer Architecture for Time-Series Data Applied to Stock-Market Closing-Price Prediction

Abstract

The transformer architecture revolutionized natural language processing (NLP) tasks and gavebirth to powerful models such as the generative pre-trained (GPT) model, bidirectional encoderrepresentation from the transformer (BERT), text-to-text transformer. This thesis dives into thisunexplored landscape, investigating the potential of transformer-based models for accuratelypredicting future closing prices of stocks. We propose a novel architecture, derived from theoriginal transformer, specifically crafted for this task. Our approach involves not only building andoptimizing this model but also tuning its hyperparameters for each of the four major stock marketsectors: technology, finance, pharmaceutical, and FMCG (Fast Moving Consumer Goods). Bycarefully tailoring the model to each sector's unique characteristics, we aim to maximize itseffectiveness and capture nuanced market dynamics. Finally, we put our model to the test,evaluating its performance on unseen data against established time series models such as the long-short-term memory (LSTM) network. This comparative analysis will not only reveal the efficacyof our transformer-based approach but also highlight its potential advantages in terms of accuracyand interpretability compared to traditional methods. Through this exploration, we hope toilluminate a promising new avenue for stock price prediction, offering valuable insights toresearchers and investors alike.