Algorithmic Trading Notes

Reading notes for the book Quantitative Trading: How to Build Your Own Algorithmic Trading Business.

The book is a great introduction to algorithmic trading, especially for people who wants to start as individual traders, but has no prior experience.

The Whats, Whos, and Whys of Quantitative Trading

Quantitative trading, also known as algorithmic trading, is the trading of securities based strictly on the buy/sell decisions of computer algorithms. The computer algorithms are designed and perhaps programmed by the traders themselves, based on the historical performance of the encoded strategy tested against historical financial data.

Not all technical analysis can be regarded as quantitative trading. For example, a head and shoulder formation is not quantitative. As long as information is converted into quantitative information, it can be regarded as part of quantitative trading.

The book focuses specifically on statistical arbitrage trading. One does not need an advanced degree to become a statistical arbitrage trader.

The ideal independent quantitative trader is therefore someone who has some prior experience with finance or computer programming, who has enough savings to withstand the inevitable losses and periods without income, and whose emotion has found the right balance between fear and greed.

Fishing for Ideas

HOW TO IDENTIFY A STRATEGY THAT SUITS YOU

It is not difficult to come up with ideas for strategies. There are thousands of trading ideas online for free. The challenge is finding a strategy that suits you. Some factors to consider include

• Working hours - how much time do you have to trade? This will decide if you need fully automated strategies or semi automated ones.
• Your programming skills
• Trading Capital - the author suggested an initial capital of $100,000. Low capital means you might need leverage, which is risky unless you have a consistently profitable strategy. It also limits the strategy you can use. Dollar neutral or market neutral strategies (hedged or pair trades) might require twice the captial or leverage compared to a directional trading strategy (long or short only).

Your Goal

Most people who choose to become traders want to earn a steady (hopefully increasing) monthly, or at least quarterly, income. The strategies to pursue for short-term income versus long-term capital gain are distinguished mainly by their holding periods. Holding too long means you won’t be generating much montly income.

How Does It Compare With a Benchmark and How Consistent Are Its Returns?

Another issue to consider is the consistency of the returns generated by a strategy. Though a strategy may have the same average return as the benchmark, perhaps it delivered positive returns every month while the benchmark occasionally suffered some very bad months. In this case, we would still deem the strategy superior. This leads us to consider the Sharpe ratio, rather than returns, as the proper performance measurement of a quantitative trading strategy.

$$
Sharpe Ratio = \frac{Average of Excess Returns}{Standard Deviation of Excess Returns}
$$
where
$$
Excess Returns = Portfolio Returns - Benchmark Returns
$$
Now the benchmark is usually the market index to which the securities you are trading belong.

Sometimes, however, you can still make an educated guess based on the flimsiest of information:

• If a strategy trades only a few times a year, chances are its Sharpe ratio won’t be high. This does not prevent it from being part of your multistrategy trading business, but it does disqualify the strategy from being your main profit center.
• If a strategy has deep (e.g., more than 10 percent) or lengthy (e.g., four or more months) drawdowns, it is unlikely that it will have a high Sharpe ratio.

As a rule of thumb, any strategy that has a Sharpe ratio of less than 1 is not suitable as a stand-alone strategy. For a strategy that achieves profitability almost every month, its (annualized) Sharpe ratio is typically greater than 2. For a strategy that is profitable almost every day, its Sharpe ratio is usually greater than 3.

How Will Transaction Costs Affect the Strategy?

Every time a strategy buys and sells a security, it incurs a transaction cost. The more frequent it trades, the larger the impact of transaction costs will be on the profitability of the strategy. These transaction costs are not just due to commission fees charged by the broker. There will also be the cost of liquidity—when you buy and sell securities at their market prices, you are paying the bid-ask spread. If you buy and sell securities using limit orders, however, you avoid the liquidity costs but incur opportunity costs. This is because your limit orders may not be executed, and therefore you may miss out on the potential profits of your trade.

Does the Data Suffer from Survivorship Bias?

A historical database of stock prices that does not include stocks that have disappeared due to bankruptcies, delistings, mergers, or acquisitions suffer from the so-called survivorship bias, because only “survivors” of those often unpleasant events remain in the database.

Does the Strategy Suffer from Data-Snooping Bias?

In general, the more rules the strategy has, and the more parameters the model has, the more likely it is going to suffer data-snooping bias.

This is basically similar to the concept of overfitting in ML.

Does the Strategy “Fly under the Radar” of Institutional Money Managers?

You should look for those strategies that fly under the radar of most institutional investors. Those niches are the ones that are likely still to be profitable because they have not yet been completely arbitraged away by the gigantic hedge funds.

Backtesting

A key difference between a traditional investment management process and a quantitative investment process is the possibility of backtesting a quantitative investment strategy to see how it would have performed in the past.

FINDING AND USING HISTORICAL DATABASES

• Are the Data Split and Dividend Adjusted?
• Are the Data Survivorship Bias Free?
• Does Your Strategy Use High and Low Data?
  For almost all daily stock data, the high and low prices are far noisier than the open and close prices. What this means is that even when you had placed a buy limit order below the recorded high of a day, it might not have been filled, and vice versa for a sell limit order.

PERFORMANCE MEASUREMENT

The Sharpe ratio and drawdowns are the two most important. If you use average annualized returns, you have to tell people a number of things about what denominator you use to calculate returns. For example, is the return a leveraged one (the denominator is based on account equity), or is it unleveraged (the denominator is based on market value of the portfolio)? If the equity or market value changes daily, do you use a moving average as the denominator, or just the value at the end of each day or each month?

To further facilitate comparison across strategies, most traders annualize the Sharpe ratio.

In general, if you calculate your average and standard deviation of returns based on a certain trading period T, whether T is a month, a day, or an hour, and you want to annualize these quantities, you have to first find out how many such trading periods there are in a year (call it $N_T$). Then
$$
\text{Annualized Sharpe Ratio} = \sqrt{N_T} \times \text{Sharpe Ratio Based On T}
$$

COMMON BACKTESTING PITFALLS TO AVOID

• Look-Ahead Bias - refers to the situation when you are using information that was available only at a time ahead of the instant the trade was made. Basically data leakage.
• Data-Snooping Bias - The most basic safeguard against data-snooping bias is to ensure that you have a sufficient amount of backtest data relative to the number of free parameters you want to optimize.
• Out-of-Sample Testing - Basically having a hold out set of data for testing. The ultimate out-of-sample testing is familiar to many traders, and it is called paper trading.
• Sensitivity Analysis - Once you have optimized your parameters as well as various features of your model and have verified that its performance on a test set is still reasonable, vary these parameters or make some small qualitative changes in the features of the model and see how the performance changes on both the training and the test sets. If the drop is so drastic that any parameter set other than the optimal one is unacceptable, the model most likely suffers from data-snooping bias.
• Transaction Costs - No backtest performance is realistic without incorporating transaction costs.

STRATEGY REFINEMENT

If a strategy does not deliver superb backtest performance on first trial, there are some common ways to improve it. For example, traders often prefer to exclude pharmaceutical stocks from their technical trading program because of the dramatic impact of news on their prices, or else they may exclude stocks that have pending merger or acquisition deals. Other traders change the entry and exit timing or frequency of the trades. Yet another variation concerns the selection of the stock universe.

When introducing these refinements to your strategy, it is preferable that the refinement has some basis in fundamental economics or a well-studied market phenomenon, rather than some arbitrary rule based on trial and error. Otherwise, data-snooping bias looms.

Execution Systems

WHAT AN AUTOMATED TRADING SYSTEM CAN DO FOR YOU

An automated trading system will retrieve up-to-date market data from your brokerage or other data vendors, run a trading algorithm to generate orders, and submit those orders to your brokerage for execution.

A fully automated system has the advantage that it minimizes human errors and delays.

For lower-frequency quantitative trading strategies, there is a semiautomated alternative: One can generate the orders using programs such as Excel or MATLAB, then submit those orders using built-in tools such as a basket trader or spread trader offered by your brokerage.

WHY DOES ACTUAL PERFORMANCE DIVERGE FROM EXPECTATIONS?

• Do you have bugs in your Automated Trading System (ATS) software?
• Do the trades generated by your ATS match the ones generated by your backtest program?
• Are the execution costs much higher than what you expected?
• Are you trading illiquid stocks that caused a lot of market impact?

To see if data-snooping bias is causing the underperformance of your live trading, try to eliminate as many rules and as many parameters in your strategy as possible.

Regime shifts refer to the situation when the financial market structure or the macroeconomic environment undergoes a drastic change so much so that trading strategies that were profitable before may not be profitable now.

Money and Risk Management

OPTIMAL CAPITAL ALLOCATION AND LEVERAGE

Suppose you plan to trade several strategies, each with their own expected returns and standard deviations. How should you allocate capital among them in an optimal way?

One approximation that I will make is that the probability distribution of the returns of each of the trading strategy $i$ is Gaussian, with a fixed mean $m_i$ and standard deviation $s_i$. Let’s denote the optimal fractions of your equity that you should allocate to each of your $n$ strategies by a column vector
$$
F^\star = (f^\star_1, f^\star_2, …, f^\star_n)^T
$$
Given our optimization objective and the Gaussian assumption,
Dr. Thorp has shown that the optimal allocation is given by
$$
F^\star = C^{-1}M
$$
where $C$ is the covariance matrix and $C_{ij}$ is the covariance of the returns of the $i^{th}$ and $j^{th}$ strategies, and
$$
M = (m_1, m_2, …, m_n)^T
$$
is the column vector of mean returns of the strategies.

If we assume that the strategies are all statistically independent, the covariance matrix becomes a diagonal matrix, with the diagonal elements equal to the variance of the individual strategies. This leads to an especially simple formula:
$$
f_i = m_i/s^2_i
$$
This is the famous Kelly formula as applied to continuous finance as opposed to gambling with discrete outcomes, and it gives the optimal leverage one should employ for a particular trading strategy.

Risk management always dictates that you should reduce your position size whenever there is a loss, even when it means realizing those losses.

Beyond position risk (which is comprised of both market risk and specific risk), there are other forms of risks to consider: model risk, software risk, and natural disaster risk, in decreasing order of likelihood.

• Model risk simply refers to the possibility that trading losses are not due to the statistical vagaries of the market, but to the fact that the trading model is wrong.
• Model risk can also come from increased competition from other institutional traders all running the same strategy as you; or it could be a result of some fundamental change in market structure that eliminated the edge of your trading model.

PSYCHOLOGICAL PREPAREDNESS

The first behavioral bias is known variously as the endowment effect, status quo bias, or loss aversion.

Another common bias that I have personally experienced is the “representativeness bias”—people tend to put too much weight on recent experience and underweight long-term average.

Greed is the more usual emotion when the model is having a good run and is generating a lot of profits. The temptation now is to increase its leverage quickly in order to get rich quickly.

Both despair and greed can lead to overleveraging.

Special Topics in Quantitative Trading

MEAN-REVERTING VERSUS MOMENTUM STRATEGIES

Trading strategies can be profitable only if securities prices are either mean-reverting or trending. Otherwise, they are random-walking, and trading will be futile.

Academic research has indicated that stock prices are on average very close to random walking. However, this does not mean that under certain special conditions, they cannot exhibit some degree of mean reversion or trending behavior. Furthermore, at any given time, stock prices can be both mean reverting and trending depending on the time horizon you are interested in. Constructing a trading strategy is essentially a matter of determining if the prices under certain conditions and for a certain time horizon will be mean reverting or trending, and what the initial reference price should be at any given time.

One must make sure the data is thoroughly cleansed of such fictitious quotes before one can completely trust your backtesting performance on a mean-reverting strategy.
Survivorship bias also affects the backtesting of mean-reverting strategies disproportionately.

Momentum can be generated by the slow diffusion of information—as more people become aware of certain news, more people decide to buy or sell a stock, thereby driving the price in the same direction.

Besides the slow diffusion of information, momentum can be caused by the incremental execution of a large order due to the liquidity needs or private investment decisions of a large investor.

Momentum can also be generated by the herdlike behavior of investors: investors interpret the (possibly random and meaningless) buying or selling decisions of others as the sole justifications of their own trading decisions.

There is one last contrast between mean-reverting and momentum strategies that is worth pondering. What are the effects of increasing competition from traders with the same strategies? For mean-reverting strategies, the effect typically is the gradual elimination of any arbitrage opportunity, and thus gradually diminishing returns down to zero. When the number of arbitrage opportunities has been reduced to almost zero, the mean-reverting strategy is subject to the risk that an increasing percentage of trading signals are actually due to fundamental changes in stocks’ valuation and thus is not going to mean revert. For momentum strategies, the effect of competition is often the diminishing of the time horizon over which the trend will continue. As news disseminates at a faster rate and as more traders take advantage of this trend earlier on, the equilibrium price will be reached sooner. Any trade entered after this equilibrium price is reached will be unprofitable.

REGIME SWITCHING

The concept of regimes is most basic to financial markets. What else are “bull” and “bear” markets if not regimes?

Some of the other most common financial or economic regimes studied are inflationary vs. recessionary regimes, high- vs. low- volatility regimes, and mean-reverting vs. trending regimes.

STATIONARITY AND COINTEGRATION

It is obvious that if the price series of a security is stationary, it would be a great candidate for a mean-reversion strategy. Unfortunately, most stock price series are not stationary—they exhibit a geometric random walk that gets them farther and farther away from their starting values. However, you can often find a pair of stocks such that if you long one and short the other, the market value of the pair is stationary. If this is the case, then the two individual time series are said to be cointegrated.

If a price series (of a stock, a pair of stocks, or, in general, a portfolio of stocks) is stationary, then a mean-reverting strategy is guaranteed to be profitable, as long as the stationarity persists into the future (which is by no means guaranteed). However, the converse is not true. You don’t necessarily need a stationary price series in order to have a successful mean-reverting strategy. Even a nonstationary price series can have many short-term reversal opportunities that one can exploit, as many traders have discovered.

Many pair traders are unfamiliar with the concepts of stationarity and cointegration. But most of them are familiar with correlation, which superficially seems to mean the same thing as cointegration. Actually, they are quite different.

FACTOR MODELS

There is a well-known framework in quantitative finance called factor models (also known as arbitrage pricing theory) that attempts to capture the different drivers of returns such as earnings growth rates, interest rate, or the market capitalization of a company. These drivers are called factors.

Mathematically, we can write the excess returns (returns minus risk-free rate) $R$ of $N$ stocks as
$$
R = Xb + u
$$
where $X$ is an $N\times N$ matrix of factor of exposures, $b$ is an $N$ vector of factor returns, and $u$ an $N$ vector of specific returns.

The terms factor exposure, factor return, and specific return are commonly used in quantitative finance, and it is well worth our effort to understand their meanings. Factor returns are the common drivers of stock returns, and are therefore independent of a particular stock. Factor exposures are the sensitivities to each of these common drivers. Any part of a stock’s return that cannot be explained by these common factor returns is deemed a specific return (i.e., specific to a stock and essentially regarded as just random noise within the APT framework). Each stock’s specific return is assumed to be uncorrelated to another stock’s.

WHAT IS YOUR EXIT STRATEGY?

• A fixed holding period
• A target price or profit cap
• The latest entry signals
• A stop price

SEASONAL TRADING STRATEGIES

This type of trading strategy is also called the calendar effect. Generally, these strategies recommend that you buy or sell certain securities at a fixed date of every year, and close the position at another fixed date.

However, from my own experience, much of the seasonality in equity markets has weakened or even disappeared in recent years, perhaps due to the widespread knowledge of this trading opportunity, whereas some seasonal trades in commodity futures are still profitable.

HIGH-FREQUENCY TRADING STRATEGIES

The reason why these strategies have Sharpe ratio is simple: Based on the “law of large numbers,” the more bets you can place, the smaller the percent deviation from the mean return you will experience.

With this high Sharpe ratio, one can increase the leverage to a much higher level than longer-term strategies can, and this high leverage in turn boosts the return-on-equity of the strategy to often stratospheric levels.

In general, though, these strategies aim to exploit tiny inefficiencies in the market or to provide temporary liquidity needs for a small fee. Unlike betting on macroeconomic trends or company fundamentals where the market environment can experience upheavals during the lifetime of a trade, such inefficiencies and need for liquidity persist day to day, allowing consistent daily profits to be made. Furthermore, high-frequency strategies typically trade securities in modest sizes. Without large positions to unwind, risk management for high-frequency portfolios is fairly easy: “Deleveraging” can be done very quickly in the face of losses, and certainly one can stop trading and be completely in cash when the going gets truly rough.

Though successful high-frequency strategies have such numerous merits, it is not easy to backtest such strategies when the average holding period decreases to minutes or even seconds. Transaction costs are of paramount importance in testing such strategies. Without incorporating transactions, the simplest strategies may seem to work at high frequencies. As a consequence, just having high-frequency data with last prices is not sufficient—data with bid, ask, and last quotes is needed to find out the profitability of executing on the bid versus the ask. Sometimes, we may even need historical order book information for backtesting. Quite often, the only true test for such strategies is to run it in real-time unless one has an extremely sophisticated simulator.

Backtesting is only a small part of the game in high-frequency trading. High-speed execution may account for a large part of the actual profits or losses. Professional high-frequency trading firms have been writing their strategies in C instead of other, more user-friendly languages, and locating their servers next to the exchange or a major Internet backbone to reduce the microsecond delays.

Somehow, the market is chronically underpricing high-beta stocks. Hence, given a choice between a portfolio of high-beta stocks and a portfolio of low-beta stocks, we should prefer the low-beta one, which we can then leverage up to achieve the maximum compounded growth rate.
There is one usual caveat, however. All this is based on the Gaussian assumption of return distributions. (See discussions in Chapter 6 on this issue.) Since the actual returns distributions have fat tails, one should be quite wary of using too much leverage on normally low-beta stocks.

I have described a large part of the statistical arbitrageur’s standard arsenal: mean reversion and momentum, regime switching, stationarity and cointegration, arbitrage pricing theory or factor model, seasonal trading models, and, finally, high-frequency trading.

• Mean-reverting regimes are more prevalent than trending regimes.
• There are some tricky data issues involved with backtesting mean-reversion strategies: Outlier quotes and survivorship bias are among them.
• Trending regimes are usually triggered by the diffusion of new information, the execution of a large institutional order, or “herding” behavior.
• Competition between traders tends to reduce the number of mean-reverting trading opportunities.
• Competition between traders tends to reduce the optimal holding period of a momentum trade.
• Regime switching can sometimes be detected using a data-mining approach with numerous input features.
• A stationary price series is ideal for a mean-reversion trade.
• Two or more nonstationary price series can be combined to form a stationary one if they are “cointegrating.”
• Cointegration and correlation are different things: Cointegration is about the long-term behavior of the prices of two or more stocks, while correlation is about the short-term behavior of
  their returns.
• Factor models, or arbitrage pricing theory, are commonly used
  for modeling how fundamental factors affect stock returns linearly.
• One of the most well-known factor models is the Fama-French Three-Factor model, which postulates that stock returns are proportional to their beta and book-to-price ratio, and negatively to their market capitalizations.
• Factor models typically have a relatively long holding period and long drawdowns due to regime switches.
• Exit signals should be created differently for mean-reversion versus momentum strategies.
• Estimation of the optimal holding period of a mean-reverting strategy can be quite robust, due to the Ornstein-Uhlenbeck formula.
• Estimation of the optimal holding period of a momentum strategy can be error prone due to the small number of signals.
• Stop loss can be suitable for momentum strategies but not reversal strategies.
• Seasonal trading strategies for stocks (i.e., calendar effect) have become unprofitable in recent years.
• Seasonal trading strategies for commodity futures continue to be profitable.
• High-frequency trading strategies rely on the “law of large numbers” for their high Sharpe ratios.
• High-frequency trading strategies typically generate the highest long-term compounded growth due to their high Sharpe ratios.
• High-frequency trading strategies are very difficult to backtest and very technology reliant for their execution.
• Holding a highly leveraged portfolio of low-beta stocks should generate higher long-term compounded growth than holding unleveraged portfolio of high-beta stocks.

Most statistical arbitrage trading strategies are some combination of these effects or models: Whether they are profitable or not is more of an issue of where and when to apply them than whether they are theoretically correct or not.

Conclusions

The key, it turns out, is capacity, a concept I introduced at the end of Chapter 2. (To recap: Capacity is the amount of equity a strategy can generate good returns on.) It is far, far easier to generate a high Sharpe ratio trading a $100,000 account than a $100 million account. There are many simple and profitable strategies that can work at the low capacity end that would be totally unsuitable to hedge funds. This is the niche for independent traders like us.

Let me elaborate on this capacity issue. Most profitable strategies that have low capacities are acting as market makers: providing short-term liquidity when it is needed and taking quick profits when the liquidity need disappears. If, however, you have billions of dollars to manage, you now become the party in need of liquidity, and you have to pay for it. To minimize the cost of this liquidity demand, you necessarily have to hold your positions over long periods of time. When you hold for long periods, your portfolio will be subject to macroeconomic changes (i.e., regime shifts) that can cause great damage to your portfolio. Though you may still be profitable in the long run if your models are sound, you cannot avoid the occasional sharp drawdowns that attract newspaper headlines.

The intense competition among hedge funds means the strategies become less profitable. The lowered returns in turn pressure the fund manager to overleverage. To beat out the competition, traders need to resort to more and more complicated models, which in turn invite data-snooping bias. But despite the increasing complexity of the models, the fundamental market inefficiency that they are trying to exploit may remain the same, and thus their portfolios may still end up holding very similar positions.

Another reason that independent traders can often succeed when large funds fail is the myriad constraints imposed by management in an institutional setting.

When your strategy shows initial profits, these managers may impose enormous pressure for you to scale up quickly, and when your strategy starts to lose, they may force you to liquidate the portfolios and abandon the strategy immediately. None of these interferences in the quantitative investment process is mathematically optimal.