Evaluation of Reinforcement Learning Techniques for Trading on a Diverse PortfolioKhare, Ishan S.;Martheswaran, Tarun K.;Dassanaike-Perera, Akshana
doi: 10.48550/arxiv.2309.03202pmid: N/A
Abstract:This work seeks to answer key research questions regarding the viability of reinforcement learning over the S&P 500 index. The on-policy techniques of Value Iteration (VI) and State-action-reward-state-action (SARSA) are implemented along with the off-policy technique of Q-Learning. The models are trained and tested on a dataset comprising multiple years of stock market data from 2000-2023. The analysis presents the results and findings from training and testing the models using two different time periods: one including the COVID-19 pandemic years and one excluding them. The results indicate that including market data from the COVID-19 period in the training dataset leads to superior performance compared to the baseline strategies. During testing, the on-policy approaches (VI and SARSA) outperform Q-learning, highlighting the influence of bias-variance tradeoff and the generalization capabilities of simpler policies. However, it is noted that the performance of Q-learning may vary depending on the stability of future market conditions. Future work is suggested, including experiments with updated Q-learning policies during testing and trading diverse individual stocks. Additionally, the exploration of alternative economic indicators for training the models is proposed.
Long-Term Mean-Variance Optimization Under Mean-Reverting Equity ReturnsPreisel, Michael
doi: 10.48550/arxiv.2309.07488pmid: N/A
Abstract:This paper studies the mean-variance optimal portfolio choice of an investor pre-committed to a deterministic investment policy in continuous time in a market with mean-reversion in the risk-free rate and the equity risk-premium. In the tradition of Markowitz, optimal policies are restricted to a subclass of factor exposures in which losses cannot exceed initial capital and it is shown that the optimal policy is characterized by an Euler-Lagrange equation derived by the method of Calculus of Variations. It is a main result, that the Euler-Lagrange equation can be recast into a matrix differential equation by an integral transformation of the factor exposure and that the solution to the characteristic equation can be parametrized by the eigenvalues of the associated lambda-matrix, hence, the optimization problem is equivalent to a spectral problem. Finally, explicit solutions to the optimal policy are provided by application of suitable boundary conditions and it is demonstrated that - if in fact the equity risk-premium is slowly mean-reverting - then investors committing to long investment horizons realize better risk-return trade-offs than investors with shorter investment horizons.
Decentralised Finance and Automated Market Making: Predictable Loss and Optimal Liquidity ProvisionCartea, Álvaro;Drissi, Fayçal;Monga, Marcello
doi: 10.48550/arxiv.2309.08431pmid: N/A
Abstract:Constant product markets with concentrated liquidity (CL) are the most popular type of automated market makers. In this paper, we characterise the continuous-time wealth dynamics of strategic LPs who dynamically adjust their range of liquidity provision in CL pools. Their wealth results from fee income, the value of their holdings in the pool, and rebalancing costs. Next, we derive a self-financing and closed-form optimal liquidity provision strategy where the width of the LP's liquidity range is determined by the profitability of the pool (provision fees minus gas fees), the predictable losses (PL) of the LP's position, and concentration risk. Concentration risk refers to the decrease in fee revenue if the marginal exchange rate (akin to the midprice in a limit order book) in the pool exits the LP's range of liquidity. When the drift in the marginal rate is stochastic, we show how to optimally skew the range of liquidity to increase fee revenue and profit from the expected changes in the marginal rate. Finally, we use Uniswap v3 data to show that, on average, LPs have traded at a significant loss, and to show that the out-of-sample performance of our strategy is superior to the historical performance of LPs in the pool we consider.
Computer says 'no': Exploring systemic bias in ChatGPT using an audit approachLippens, Louis
doi: 10.48550/arxiv.2309.07664pmid: N/A
Abstract:Large language models offer significant potential for increasing labour productivity, such as streamlining personnel selection, but raise concerns about perpetuating systemic biases embedded into their pre-training data. This study explores the potential ethnic and gender bias of ChatGPT, a chatbot producing human-like responses to language tasks, in assessing job applicants. Using the correspondence audit approach from the social sciences, I simulated a CV screening task with 34,560 vacancy-CV combinations where the chatbot had to rate fictitious applicant profiles. Comparing ChatGPT's ratings of Arab, Asian, Black American, Central African, Dutch, Eastern European, Hispanic, Turkish, and White American male and female applicants, I show that ethnic and gender identity influence the chatbot's evaluations. Ethnic discrimination is more pronounced than gender discrimination and mainly occurs in jobs with favourable labour conditions or requiring greater language proficiency. In contrast, gender discrimination emerges in gender-atypical roles. These findings suggest that ChatGPT's discriminatory output reflects a statistical mechanism echoing societal stereotypes. Policymakers and developers should address systemic bias in language model-driven applications to ensure equitable treatment across demographic groups. Practitioners should practice caution, given the adverse impact these tools can (re)produce, especially in selection decisions involving humans.
Decentralized Token Economy Theory (DeTEcT)Sadykhov, Rem;Goodell, Geoffrey;de Montigny, Denis;Schoernig, Martin;Treleaven, Philip
doi: 10.3389/fbloc.2023.1298330pmid: N/A
Abstract:This paper presents a pioneering approach for simulation of economic activity, policy implementation, and pricing of goods in token economies. The paper proposes a formal analysis framework for wealth distribution analysis and simulation of interactions between economic participants in an economy. Using this framework, we define a mechanism for identifying prices that achieve the desired wealth distribution according to some metric, and stability of economic dynamics. The motivation to study tokenomics theory is the increasing use of tokenization, specifically in financial infrastructures, where designing token economies is in the forefront. Tokenomics theory establishes a quantitative framework for wealth distribution amongst economic participants and implements the algorithmic regulatory controls mechanism that reacts to changes in economic conditions. In our framework, we introduce a concept of tokenomic taxonomy where agents in the economy are categorized into agent types and interactions between them. This novel approach is motivated by having a generalized model of the macroeconomy with controls being implemented through interactions and policies. The existence of such controls allows us to measure and readjust the wealth dynamics in the economy to suit the desired objectives.
Beyond Gut Feel: Using Time Series Transformers to Find Investment GemsCao, Lele;Halvardsson, Gustaf;McCornack, Andrew;von Ehrenheim, Vilhelm;Herman, Pawel
doi: 10.48550/arxiv.2309.16888pmid: N/A
Abstract:This paper addresses the growing application of data-driven approaches within the Private Equity (PE) industry, particularly in sourcing investment targets (i.e., companies) for Venture Capital (VC) and Growth Capital (GC). We present a comprehensive review of the relevant approaches and propose a novel approach leveraging a Transformer-based Multivariate Time Series Classifier (TMTSC) for predicting the success likelihood of any candidate company. The objective of our research is to optimize sourcing performance for VC and GC investments by formally defining the sourcing problem as a multivariate time series classification task. We consecutively introduce the key components of our implementation which collectively contribute to the successful application of TMTSC in VC/GC sourcing: input features, model architecture, optimization target, and investor-centric data processing. Our extensive experiments on two real-world investment tasks, benchmarked towards three popular baselines, demonstrate the effectiveness of our approach in improving decision making within the VC and GC industry.
Monte Carlo Simulation for Trading Under a L\'evy-Driven Mean-Reverting FrameworkLeung, Tim;Lu, Kevin W.
doi: 10.48550/arxiv.2309.05512pmid: N/A
Abstract:We present a Monte Carlo approach to pairs trading on mean-reverting spreads modeled by Lévy-driven Ornstein-Uhlenbeck processes. Specifically, we focus on using a variance gamma driving process, an infinite activity pure jump process to allow for more flexible models of the price spread than is available in the classical model. However, this generalization comes at the cost of not having analytic formulas, so we apply Monte Carlo methods to determine optimal trading levels and develop a variance reduction technique using control variates. Within this framework, we numerically examine how the optimal trading strategies are affected by the parameters of the model. In addition, we extend our method to bivariate spreads modeled using a weak variance alpha-gamma driving process, and explore the effect of correlation on these trades.
Liquidity Dynamics in RFQ Markets and Impact on PricingBergault, Philippe;Guéant, Olivier
doi: 10.48550/arxiv.2309.04216pmid: N/A
Abstract:To assign a value to a portfolio, it is common to use Mark-to-Market prices. However, how should one proceed when the securities are illiquid? When transaction prices are scarce, how can one use all the available real-time information? In this article, we address these questions for over-the-counter (OTC) markets based on requests for quotes (RFQs). We extend the concept of micro-price, which was recently introduced for assets exchanged through limit order books in the market microstructure literature, and incorporate ideas from the recent literature on OTC market making. To account for liquidity imbalances in RFQ markets, we use an approach based on bidimensional Markov-modulated Poisson processes. Beyond extending the concept of micro-price to RFQ markets, we introduce the new concept of Fair Transfer Price. Our concepts of price can be used to value securities fairly, even when the market is relatively illiquid and/or tends to be one-sided.
Aggregation of financial marketsMenz, Georg;Voß, Moritz
doi: 10.48550/arxiv.2309.04116pmid: N/A
Abstract:We present a formal framework for the aggregation of financial markets mediated by arbitrage. Our main tool is to characterize markets via utility functions and to employ a one-to-one correspondence to limit order book states. Inspired by the theory of thermodynamics, we argue that the arbitrage-mediated aggregation mechanism gives rise to a market-dynamical entropy, which quantifies the loss of liquidity caused by aggregation. As a concrete guiding example, we illustrate our general approach with the Uniswap v2 automated market maker protocol used in decentralized cryptocurrency exchanges, which we characterize as a so-called ideal market. We derive its equivalent limit order book representation and explicitly compute the arbitrage-mediated aggregation of two liquidity pools of the same asset pair with different marginal prices. We also discuss future directions of research in this emerging theory of market dynamics.
Don't Let MEV Slip: The Costs of Swapping on the Uniswap ProtocolAdams, Austin;Chan, Benjamin Y;Markovich, Sarit;Wan, Xin
doi: 10.48550/arxiv.2309.13648pmid: N/A
Abstract:We present the first in-depth empirical characterization of the costs of trading on a decentralized exchange (DEX). Using quoted prices from the Uniswap Labs interface for two pools -- USDC-ETH (5bps) and PEPE-ETH (30bps) -- we evaluate the efficiency of trading on DEXs. Our main tool is slippage -- the difference between the realized execution price of a trade, and its quoted price -- which we breakdown into its benign and adversarial components. We also present an alternative way to quantify and identify slippage due to adversarial reordering of transactions, which we call reordering slippage, that does not require quoted prices or mempool data to calculate. We find that the composition of transaction costs varies tremendously with the trade's characteristics. Specifically, while for small swaps, gas costs dominate costs, for large swaps price-impact and slippage account for the majority of it. Moreover, when trading PEPE, a popular 'memecoin', the probability of adversarial slippage is about 80% higher than when trading a mature asset like USDC. Overall, our results provide preliminary evidence that DEXs offer a compelling trust-less alternative to centralized exchanges for trading digital assets.