skip to navigation skip to content

Can robots beat the market?

by Dr Mehrshad Motahari, Research Associate, Cambridge Centre for Finance and Cambridge Endowment for Research in Finance

Robot analysing the stockmarket.
Mehrshad Motahari.
Dr Mehrshad Motahari

The growing trend of replacing active investment managers with computer algorithms (The Economist, 2019) has led to a surge in the use of artificial intelligence (AI)[1] in investing. This means that more AI-based algorithms (alpha algos) are being used to devise investment strategies. In most cases, the algorithm itself tests the viability of these strategies and even executes trades while keeping transaction costs to a minimum. A common question, however, is whether these algorithms can generate profitable investments. The following is a summary of findings from a number of recent studies on the issue.

AI-based investment strategies often use forecasts of future asset performance metrics, the most popular being returns. AI models utilise a range of data inputs, including technical and fundamental indicators, economic measures, and texts (like online posts and news articles), to predict future returns (Bartram, Branke, and Motahari, 2020). These predictions then form the basis of an investment strategy by rebalancing portfolio weights to favour stocks that will outperform and to move away from those that will underperform.

In a recent hallmark study, Gu, Kelly, and Xiu (2020) investigate a variety of AI models that can be used to forecast future stock returns. The study looks at 30,000 US stocks, from 1957 to 2016 and includes a set of predictor variables, including 94 stock characteristics, interactions of each characteristic with eight aggregate time-series variables, and 74 industry sector dummy variables.

According to the results, the best-performing investment strategy is based on the return predictions of the neural network model; a value-weighted long-short decile spread strategy using neural network predictions generates an annualised out-of-sample Sharpe ratio of 1.35. This is more than double the Sharpe ratio of a regression-based strategy from the literature. The out of sample performance of AI approaches is robust across a range of specifications.

Why do AI approaches perform better in predicting returns than classic tools such as ordinary linear regressions? Gu, Kelly, and Xiu (2020) argue that this is due to the ability of most AI techniques to capture nonlinear relationships between dependant and independent variables. Such relationships are often missed by linear regressions. Moreover, many AI techniques are able to select the most relevant variables from a large set of predictors. This allows the model inputs to shrink while keeping the most important variables. In another recent paper, Freyberger, Neuhierl, and Weber (2020) shows an investment strategy based on a model with this feature selection property can generate a Sharpe ratio that is 2.5 times larger than that of an ordinary linear regression model.

Despite the remarkable success of AI models in predicting returns, some doubt the feasibility of investment strategies that are based on these predictions. Avramov, Cheng, and Metzker (2020) looks at the neural network methodology used in Gu, Kelly, and Xiu (2020) and show that the investment strategy return based on this approach is largely driven by subsamples of microcaps, firms with no credit rating coverage, and distressed stocks. In addition, the strategy tends to be profitable mostly during periods of high limits to arbitrage, including high market volatility and low liquidity. It appears that AI models have improved upon return forecasting, due to their flexible structure and ability to capture complex relationships from vast amounts of data. However, the jury is still out on whether the predictions do, in fact, lead to investments that outperform conventional benchmarks in practice. What is clear for now is that AI provides us with the best tools for forecasting returns empirically.

[1] In finance, many studies refer to ‘machine learning’ or ‘ML’ (which is a subcategory of AI) instead of ‘AI’. The argument for using ‘ML’ is that it is a more precise term, considering that the AI techniques that are applied in finance are mostly ML approaches.


Avramov, D., Cheng, S. and Metzker, L. (2020) “Machine learning versus economic restrictions: evidence from stock return predictability.” Social Science Research Network No.3450322

Bartram, S. M., Branke, J. and Motahari, M. (2020) “Artificial intelligence in asset management.” CEPR Discussion Paper No.14525 / Social Science Research Network No.35603330.

The Economist (2019) “The stockmarket is now run by computers, algorithms and passive managers.”

Freyberger, J., Neuhierl, A. and Weber, M. (2020) “Dissecting characteristics nonparametrically.” The Review of Financial Studies, 33(5): 2326-2377

Gu, S., Kelly, B. and Xui, D. (2020) “Empirical asset pricing via machine learning.” The Review of Financial Studies, 33(5): 2223-2273