Getting Started with AI in Trading: A Beginner's Guide

The forex (FX) marketplace is one of the most popular monetary markets for buying and selling currencies because it is the largest financial marketplace inside the global in terms of trading volume [1].
Foreign exchange market refers to the market in which currencies from around the world are traded. It allows investors to buy or sell a currency of their choice [2].

The traditional trading methods can be further divided into the technical analysis and fundamental analysis [3]. Similarly, the modern trading methods can be further divided into algorithmic trading and machine learning (ML) approaches [3].

The traditional financial trading methods are often implemented manually by humans, while the modern methods are executed automatically by computers [3].

Algorithmic trading can be performed by either rule-based methods or ML-based approaches [3].

Algorithmic trading can be performed by either rule-based methods or ML-based approaches [3].

In recent years, financial trading has seen an increase in participation from automated trading agents. This growth has been driven by financial trading firms using automated strategies that they apply to all suitable markets [4].

Automated trading, which is also known as algorithmic trading, is a method of using a predesigned computer program to submit a large number of trading orders to an exchange [5].

One approach to automated trading is the rule-based algorithmic approaches, such as the technical analysis of the price time series aiming to detect plausible signals that entail certain market movements, thus triggering trading actions that will yield a profit if such movements occur. One step further is the machine learning techniques that automatically determine the patterns that lead to predictable market movement. Such techniques require the construction of supervised labels, from the price time series, which describe the direction of the future price movement [6].

a few authors differentiate the trading decisions (quantitative trading) from the particular trading execution (algorithmic trading) [7].

Quantitative trading strategies are designed to look for relationships between data about an underlying security and its future price and then to generate alpha on a trading desk [8].

Quantitative trading consists of strategies based on quantitative analysis which rely on mathematical computations and number-crunching to identify trading opportunities, in other words, it is a kind of investment management system designed to conduct transactions based on mathematical models and algorithmic technology [8].

For a long time, quantitative human traders have been getting “phased out” due to their inconsistent behavior and, consequently, performance [6].

As a new kind of investment analysis idea which combines modem mathematics and financial data, quantitative trading is playing an active and significant role in financial markets all over the world [8].

Algorithmic trading is a branch of quantitative investment, refers to the use of quantitative analysis method to execute actions automatically according to the algorithm model and achieve asset trading without manual intervention [9].

High-frequency trading refers to a type of algorithmic trade in financial instruments with a very high speed. To capture fleeting liquidity imbalances and pricing inefficiencies, HFT strategies are generally designed based on statistical arbitrage, liquidity provisions or liquidity detection. HFT strategies often present surprisingly concise and simple programming, which is usually developed based on linear reversion [5].

High-frequency trading is a method of intervention on the financial markets that guided by mathematical algorithms and HFT strategies have reached considerable volumes of commercial traffic and are responsible for most of the transaction traffic of some stock exchanges [10].

Financial markets have a complex, uncertain, and dynamic nature, making them challenging for trading [3]. Algorithmic trading based on machine learning is a developing and promising field of research [3].

In recent years, using the ML as an intelligent agent has risen in popularity over the alternative of the traditional approaches in which a human being makes a decision [11].

The ML and DL have enhanced the performance in algorithmic trading. Firstly, they can extract complex patterns from data that are difficult for humans to accomplish. Secondly, emotion does not affect their performance, which is a disadvantage for humans [11].

In recent years, machine learning and deep learning algorithms have been widely applied to build prediction and classification models for the financial market [1]. the disadvantage of these algorithms are not trained to model positions [1].

In rule-based algorithmic trading, computers trade in financial markets through predetermined rules specified by humans [3]. These rules can be defined based on the traditional trading methods, mathematical models, or human-made strategies [3]. On the other hand, in ML-based algorithmic trading, computers are trained on historical data to trade in markets without any direct human interventions [3].

ML-based algorithms can extract patterns, relationships, and knowledge from the historical data without the need for predetermined guidelines or strategies by domain experts, and ML-based algorithmic trading approaches can discover profitable insights unknown to humans [3].

Humans might make wrong trading decisions regarding their emotional and psychological factors and Manual trading can be prone to human errors, while computers are not negatively affected by these factors in making the trading decisions and are more precise in executing trading decisions than humans [3].

Computers are very fast in executing trading decisions and are far more capable than human minds of processing a tremendous amount of information in real-time, monitor the markets permanently, do not get tired or sleep and have immediate reactions when unexpected events occur [3].

Learning in trading systems is not merely limited to generating trading signals and predicting future price. The optimization of order execution is another important research field and especially vital for HFT strategies. As for machine learning, a reinforcement learning and supervised learning method were proposed [5].

The machine learning approach for algorithmic trading can be further divided into the supervised learning approach and the reinforcement learning approach [9].

The supervised learning method has attempted to predict the stock price or price trend of the next time point [9].

Price prediction aims to build a model that can precisely predict future prices, whereas algorithmic trading is not limited to the price prediction and attempts to participate in the financial market (e.g. choosing a position and the number of trading shares) to maximize profit [11].

It is claimed that a more precise prediction does not necessarily result in a higher profit. In other words, a trader’s overall loss due to incorrect actions may be greater than the gain due to correct ones [11].

Nevertheless, the predicted price or trends of supervised learning method cannot be directly mapped to the trading action. Therefore, after predict the price, prior knowledge of financial field is required to choose trading actions and The accuracy of predict will affect the decision-making process, which is based on the predict result, and resulting in the second error propagation [9].

In addition, prediction-based supervised method does not take into account environmental factors such as transaction cost (TC) and cannot solve the temporal credit assignment problem, resulting in its poor performance in algorithmic trading that has the characteristic of delayed return [9].

Supervised agents are trained using handcrafted labels, which require for their creation an arbitrary profit margin compared to commission and other costs. However, the actual profit is not always directly related to the aforementioned handcrafted labels, since various other parameters, such as the volatility of the market and the confidence of the agent can affect the resulting profit or loss [4].

Financial data are noisy, and it may be the reason why using supervised learning methods have not been very successful in the past [12].

the supervised ML methods have shown a potential ability to predict financial markets such as the foreign exchange (Forex) market, their prediction accuracy might not be enough to make them applicable for algorithmic trading in real markets due to the noisy, volatile, and uncertain nature of financial time series [3].

The research efforts in applying ML-based methods for financial trading, attempt to predict the future prices by training supervised learning models on financial time series historical data. However, due to the noise, uncertainty, and volatility in financial time series, the accuracy of their predictive models might not be satisfactory for trading in markets [3].

Noise-free labels unfortunately can be difficult to construct since the extreme and unpredictable nature of financial markets does not allow for calculating a single “hard” threshold to determine whether a price movement is significant or not [6].

There are two compelling reasons why the ML and DL in a supervised learning approach are unsuitable for algorithmic trading. Firstly, supervised learning is improper for learning problems with long-term and delayed rewards, such as trading in financial markets, which is why Reinforcement Learning (RL), a subfield of ML, is required to solve a decision-making problem (trading) in an uncertain environment (financial market) using the Markov Decision Process (MDP). Secondly, in supervised learning, labeling is a critical issue affecting the performance of the final model. To illustrate, classification and regression approaches with defined labels may not be appropriate, leading to the selection of RL, which does not require labels and instead uses a goal (reward function) to determine its policy [11].

ML algorithms such as deep reinforcement learning (DRL) can autonomously make optimal decisions in complex environments [3].

The need for supervised labels can be alleviated by the use of reinforcement learning (RL). In RL there is no need for supervised labels since RL can consider the magnitude of the rewards instead of solely considering the direction of each price movement. In RL, an agent is allowed to interact with the environment and receives rewards or punishments. In the financial trading setting, the agent decides what trading action to take and is rewarded or punished according to its trading performance [6].

In contrast, when DRL is used, the trading profits can be tightly integrated in the agent’s reward, along with every other market cost, since simulated trading environments can be developed and used for training the DRL agents which removes the complexity of handcrafted label selection and allows the DRL optimization to determine which position is worth taking and has predictable outcomes based on the received reward [4].

In this way, DRL agents are able to directly optimize the metrics related to the task at hand, i.e., the obtained profit, in simulated trading environments [4].

In financial trading utilizing direct price forecasts or handcrafted labels by experts to train supervised models is argued to be suboptimal while RL is more likely to yield better results by directly optimizing the performance [4].

This is attributed to the large difference of transaction costs incurred by the labels used for the supervised learning compared to the transaction costs of the predictions by the models [4].

optimizing deep RL agents is notoriously difficult and unstable, especially in noisy financial environments, significantly hindering the performance of trading agents [4].

The disadvantage of value-based method is that the state value is expressed in tabular form and can only deal with discrete state and action space problems. The sensitivity of the strategy derived from the value function is also high, minor changes of the value function will lead to major changes in the strategy. Since algorithmic trading is a continuous state scenario, value-based RL approach has no advantage. Compared with value function based model, the policy-based RL model is more stable [9].

RL agents perform better than the benchmark strategies, which demonstrates the potential advantage of the RL-based trading strategies over the rule-based ones [3].

There is no need for supervised labels since RL can consider the magnitude of the rewards instead of solely considering the direction of each price movement. This benefit over the supervised learning methods has led to an increasing number of works that attempt to exploit RL for various financial trading tasks [6].

• [1] T. Chau, M.-T. Nguyen, D.-V. Ngo, A.-D. T. Nguyen, and T.-H. Do, “Deep Reinforcement Learning methods for Automation Forex Trading,” in 2022 RIVF International Conference on Computing and Communication Technologies (RIVF), Dec. 2022, pp. 671–676, doi: 10.1109/RIVF55975.2022.10013861.
  
• [2] H. Jamali, Y. Chihab, I. García-Magariño, and O. Bencharef, “Hybrid Forex prediction model using multiple regression, simulated annealing, reinforcement learning and technical analysis,” IAES Int. J. Artif. Intell., vol. 12, no. 2, pp. 892–911, 2023, doi: 10.11591/ijai.v12.i2.pp892-911.
  
• [3] A. Shavandi and M. Khedmati, “A multi-agent deep reinforcement learning framework for algorithmic trading in financial markets,” Expert Syst. Appl., vol. 208, no. December 2021, p. 118124, Dec. 2022, doi: 10.1016/j.eswa.2022.118124.
  
• [4] A. Tsantekidis, N. Passalis, and A. Tefas, “Diversity-driven knowledge distillation for financial trading using Deep Reinforcement Learning.,” Neural Netw., vol. 140, pp. 193–202, Aug. 2021, doi: 10.1016/j.neunet.2021.02.026.
  
• [5] B. Huang, Y. Huan, L. Da Xu, L. Zheng, and Z. Zou, “Automated trading systems statistical and machine learning methods and hardware implementation: a survey,” Enterp. Inf. Syst., vol. 13, no. 1, pp. 132–144, Jan. 2019, doi: 10.1080/17517575.2018.1493145.
  
• [6] A. Tsantekidis, N. Passalis, A.-S. Toufa, K. Saitas-Zarkias, S. Chairistanidis, and A. Tefas, “Price Trailing for Financial Trading Using Deep Reinforcement Learning.,” IEEE Trans. neural networks Learn. Syst., vol. 32, no. 7, pp. 2837–2846, Jul. 2021, doi: 10.1109/TNNLS.2020.2997523.
  
• [7] M. H. Sazu, “How Machine Learning Can Drive High Frequency Algorithmic Trading for Technology Stocks,” Int. J. Data Sci. Adv. Anal. (ISSN 2563-4429), vol. 4, no. 4, pp. 84–93, 2022, [Online]. Available: http://ijdsaa.com/index.php/welcome/article/view/97.
  
• [8] Yelin Li, Junjie Wu, and Hui Bu, “When quantitative trading meets machine learning: A pilot survey,” in 2016 13th International Conference on Service Systems and Service Management (ICSSSM), Jun. 2016, pp. 1–6, doi: 10.1109/ICSSSM.2016.7538632.
  
• [9] K. Lei, B. Zhang, Y. Li, M. Yang, and Y. Shen, “Time-driven feature-aware jointly deep reinforcement learning for financial signal representation and algorithmic trading,” Expert Syst. Appl., vol. 140, p. 112872, Feb. 2020, doi: 10.1016/j.eswa.2019.112872.
  
• [10] F. Rundo, “Deep LSTM with Reinforcement Learning Layer for Financial Trend Prediction in FX High Frequency Trading Systems,” Appl. Sci., vol. 9, no. 20, p. 4460, Oct. 2019, doi: 10.3390/app9204460.
  
• [11] naseh majidi, M. Shamsi, and F. Marvasti, “Algorithmic Trading Using Continuous Action Space Deep Reinforcement Learning,” SSRN Electron. J., no. 98, Oct. 2022, doi: 10.2139/ssrn.4276310.
  
• [12] T. Huotari, J. Savolainen, and M. Collan, “Deep Reinforcement Learning Agent for S&P 500 Stock Selection,” Axioms, vol. 9, no. 4, p. 130, Nov. 2020, doi: 10.3390/axioms9040130.
  
• [13] K. S. Zarkias, N. Passalis, A. Tsantekidis, and A. Tefas, “Deep Reinforcement Learning for Financial Trading Using Price Trailing,” in ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), May 2019, vol. 2019-May, pp. 3067–3071, doi: 10.1109/ICASSP.2019.8683161.
  
• [14] J. Korczak and M. Hernes, “Deep Learning for Financial Time Series Forecasting in A-Trader System,” in Proceedings of the 2017 Federated Conference on Computer Science and Information Systems, FedCSIS 2017, Sep. 2017, vol. 11, pp. 905–912, doi: 10.15439/2017F449.
  
• [15] E. S. Ponomarev, I. V. Oseledets, and A. S. Cichocki, “Using Reinforcement Learning in the Algorithmic Trading Problem,” J. Commun. Technol. Electron., vol. 64, no. 12, pp. 1450–1457, Dec. 2019, doi: 10.1134/S1064226919120131.
  
• [16] V. Mnih et al., “Asynchronous Methods for Deep Reinforcement Learning,” 33rd Int. Conf. Mach. Learn. ICML 2016, vol. 4, pp. 2850–2869, Feb. 2016, [Online]. Available: http://arxiv.org/abs/1602.01783.