The Contrasting Worlds of Vectorized and Event-Based Backtesting
Backtesting is a cornerstone of any successful trading strategy. It allows us to evaluate the performance of a trading system using historical data, providing insights into its potential profitability
and risk profile. However, not all backtesting methodologies are created equal. Two primary approaches exist: vectorized and event-based backtesting. Understanding the strengths and weaknesses of each is crucial for building robust and reliable trading systems.
Vectorized backtesting leverages the power of array operations, typically using libraries like NumPy and pandas in Python. This approach excels in its efficiency and conciseness, especially when dealing with simple trading rules. It operates by performing calculations on entire datasets at once, treating the historical data as a single, continuous stream. This can lead to remarkably fast execution times, making it attractive for initial strategy exploration and optimization.
Consider a straightforward moving average crossover strategy. In a vectorized approach, you would calculate the moving averages for the entire historical dataset and then identify crossover points by comparing the two moving average series. The calculations are performed on arrays, allowing for optimized operations that are significantly faster than iterative methods.
import pandas as pd
import numpy as np
# Assume historical data is loaded into a pandas DataFrame called 'data'
# with a 'Close' column representing closing prices.
def calculate_moving_averages(data, short_window, long_window):
"""Calculates short and long moving averages."""
data['SMA_short'] = data['Close'].rolling(window=short_window).mean()
data['SMA_long'] = data['Close'].rolling(window=long_window).mean()
return data
def generate_signals(data):
"""Generates buy/sell signals based on moving average crossover."""
data['Signal'] = 0.0 # Initialize signal column
data['Signal'][data['SMA_short'] > data['SMA_long']] = 1.0 # Buy signal
data['Signal'][data['SMA_short'] < data['SMA_long']] = -1.0 # Sell signal
# Generate trade positions based on signal changes
data['Position'] = data['Signal'].diff()
return data
# Example usage (assuming 'data' DataFrame is loaded)
# Set short and long window periods
short_window = 20
long_window = 50
# Calculate moving averages
data = calculate_moving_averages(data.copy(), short_window, long_window) #Using copy to avoid SettingWithCopyWarning
# Generate trading signals
data = generate_signals(data)
# Display the first few rows with signals
print(data[['Close', 'SMA_short', 'SMA_long', 'Signal', 'Position']].head())
The example above showcases the conciseness of vectorized backtesting. The rolling() function in pandas efficiently computes moving averages across the entire dataset. The trading signals are then generated based on the comparison of these moving averages. This approach works well for this simple example. However, the power and speed of vectorized backtesting come at a cost.
The Limitations of Vectorized Backtesting: A Reality Check
While vectorized backtesting is often a good starting point, it suffers from several critical limitations when applied to more complex strategies and real-world market dynamics. These limitations can lead to overly optimistic performance estimates and ultimately, flawed trading decisions.
One of the most significant issues is look-ahead bias. Vectorized backtesting typically processes all available data simultaneously. This means that the strategy has access to future information when making trading decisions. For example, in a moving average crossover strategy, the backtester might know the future price movements when determining the crossover points. This is obviously unrealistic. In a real-world scenario, traders only have access to historical data up to the current point in time. This look-ahead bias artificially inflates performance metrics.
Consider a strategy that attempts to trade on a specific news event. Vectorized backtesting might use the entire dataset, including data after the news event, to determine the optimal entry and exit points. This would lead to inflated profitability, as the strategy appears to react perfectly to the news. This is not replicable in live trading.
Another critical drawback of vectorized backtesting lies in its simplified market mechanics. It often fails to accurately model the complexities of real-world trading. For instance, it might ignore transaction costs, such as brokerage fees and slippage (the difference between the expected price and the actual price at which a trade is executed). It may also assume perfect order execution, which is rarely the case, especially in volatile markets. Vectorized backtesting can also struggle with indivisible assets, such as stocks, where you cannot buy a fraction of a share.
# Example of a simplified vectorized backtest that does not account for transaction costs.
# Using the data generated in the previous vectorized example.
def calculate_returns(data, transaction_cost_pct=0.0):
"""Calculates returns, ignoring transaction costs."""
data['Return'] = data['Close'].pct_change()
data['Strategy_Return'] = data['Signal'].shift(1) * data['Return'] # Shift to align with trading signal.
data['Strategy_Return'] = data['Strategy_Return'] - np.abs(data['Position']) * transaction_cost_pct # Subtract transaction costs.
data['Cumulative_Return'] = (1 + data['Strategy_Return']).cumprod()
return data
# Example Usage
data = calculate_returns(data.copy(), transaction_cost_pct=0.001) # Example transaction cost of 0.1%
print(data[['Close', 'Return', 'Strategy_Return', 'Cumulative_Return']].tail())
The code above highlights how easy it is to omit realistic trading costs in a vectorized approach. The transaction_cost_pct parameter is set to 0.0 by default, which will significantly skew the results. Adding transaction costs is a simple modification, but the assumption of instantaneous execution at the ‘Close’ price is still an important simplification.
Finally, vectorized backtesting struggles with path-dependent calculations and recursive logic. Many trading strategies involve calculations that depend on the history of past trades, such as calculating cumulative profit and loss (P&L), drawdown, or the number of consecutive winning trades. Vectorized approaches can become cumbersome and computationally inefficient when dealing with these types of calculations. Furthermore, strategies that require iterative calculations based on prior results, such as strategies that dynamically adjust position sizes based on past performance, are difficult to implement efficiently using vectorized methods.
Embracing Realism: The Event-Based Approach
To overcome the limitations of vectorized backtesting and build more robust trading systems, we turn to event-based backtesting. This approach simulates the arrival of new data in a more realistic, step-by-step manner, mirroring the way a live trading system operates.
In event-based backtesting, an “event” represents the arrival of new information. This could be a new closing price, a new tick (the smallest increment of price change), a change in interest rates, a news release, or any other relevant data point. The backtesting engine processes these events sequentially, updating the state of the trading system and generating trading decisions based on the current available information.
The core advantage of event-based backtesting is its incremental approach. It simulates the arrival of data in real-time, allowing for a more accurate representation of the trading environment. The system reacts to each event as it occurs, just as a live trading system would. This avoids the look-ahead bias inherent in vectorized approaches.
import pandas as pd
class Event:
"""Base class for all events."""
def __init__(self, timestamp, type, data=None):
self.timestamp = timestamp
self.type = type
self.data = data
class MarketEvent(Event):
"""Represents a market data event (e.g., price update)."""
def __init__(self, timestamp, symbol, price):
super().__init__(timestamp, 'MARKET_DATA')
self.symbol = symbol
self.price = price
class SignalEvent(Event):
"""Represents a signal generated by a strategy (e.g., buy/sell)."""
def __init__(self, timestamp, symbol, signal_type, quantity=1):
super().__init__(timestamp, 'SIGNAL')
self.symbol = symbol
self.signal_type = signal_type # 'BUY', 'SELL', 'HOLD'
self.quantity = quantity
class OrderEvent(Event):
"""Represents an order to be sent to the broker."""
def __init__(self, timestamp, symbol, order_type, quantity, side):
super().__init__(timestamp, 'ORDER')
self.symbol = symbol
self.order_type = order_type # e.g., 'MKT' for market order
self.quantity = quantity
self.side = side # 'BUY' or 'SELL'
class FillEvent(Event):
"""Represents the execution of an order."""
def __init__(self, timestamp, symbol, quantity, side, fill_price, commission):
super().__init__(timestamp, 'FILL')
self.symbol = symbol
self.quantity = quantity
self.side = side
self.fill_price = fill_price
self.commission = commission
The code above introduces several event classes. The Event base class provides a common structure for all events, including a timestamp and type. The MarketEvent class represents new market data, such as a price update. The SignalEvent, OrderEvent, and FillEvent classes represent signals, orders, and executions, respectively. This event-driven architecture forms the foundation for simulating the trading process.
The event-based approach also allows for realistic modeling capabilities. It enables a detailed simulation of market processes, including transaction costs, slippage, order book dynamics, and various order types (market orders, limit orders, stop orders, etc.). The backtesting engine can model the impact of these factors on the trading strategy’s performance.
# Example of modeling transaction costs and slippage in an event-driven context
class ExecutionHandler:
"""Handles order execution, including slippage and transaction costs."""
def execute_order(self, order_event, current_price, slippage_pct=0.001, transaction_cost_pct=0.001):
"""Simulates order execution, including slippage and transaction costs."""
fill_price = current_price # Base case: assuming immediate execution at current market price
if order_event.side == 'BUY':
fill_price *= (1 + slippage_pct) # Simulate slippage (higher price for buy)
elif order_event.side == 'SELL':
fill_price *= (1 - slippage_pct) # Simulate slippage (lower price for sell)
commission = transaction_cost_pct * order_event.quantity * fill_price
fill_event = FillEvent(
timestamp=order_event.timestamp,
symbol=order_event.symbol,
quantity=order_event.quantity,
side=order_event.side,
fill_price=fill_price,
commission=commission
)
return fill_event
This ExecutionHandler class simulates order execution, including slippage and transaction costs. The execute_order method adjusts the fill price based on the order side (‘BUY’ or ‘SELL’) to account for slippage and calculates the commission. This allows us to assess the realistic impact of these factors on the trading strategy’s profitability.
Event-based backtesting inherently supports path dependency. The system maintains a record of all past events, enabling it to track historical data and perform calculations that rely on the complete trading history. This is critical for accurately modeling strategies that use cumulative P&L, drawdown, or other path-dependent metrics.
Furthermore, event-based backtesting promotes reusability. The code can be structured into modular components that can be easily reused across different strategies. For example, the event classes, execution handlers, and data feed components can be reused in multiple backtesting projects, reducing development time and promoting code maintainability.
Finally, event-based backtesting systems are typically structured in a way that closely mirrors a live trading system. The event-driven architecture, with its focus on receiving and processing events, closely aligns with the real-time operation of a live trading system. This close proximity to live trading makes event-based backtesting a valuable tool for testing and deploying trading strategies.
Data Representation and Chapter Structure: Building the Framework
In the context of event-based backtesting, data is typically represented in discrete units or “bars”. A “bar” represents a specific period of time, containing data such as the open, high, low, and close prices (OHLC), and the volume traded during that period. The duration of a bar can vary depending on the trading strategy. For intraday strategies, we might use one-minute bars, five-minute bars, or even tick-by-tick data. For longer-term strategies, we might use daily, weekly, or monthly bars.
The following sections will delve into the practical implementation of event-based backtesting. We will begin by constructing a foundational Backtesting Base Class, which will provide the core functionality for managing events, data feeds, and order execution. This base class will serve as the foundation for building more specialized backtesting classes.
Following the Backtesting Base Class, we will explore two specific trading strategies: a Long-Only Backtesting Class and a Long-Short Backtesting Class. These classes will inherit from the base class and implement specific trading logic, demonstrating how to build and test different types of trading strategies.
The goals of this article are:
To provide a comprehensive understanding of event-based modeling.
To guide the creation of reusable and extensible classes for more realistic backtesting.
To provide a foundational backtesting infrastructure upon which to build and test a variety of trading strategies.
Building the Foundation: The Backtesting Base Class
This section introduces the BacktestingBase class, the cornerstone of our event-based backtesting framework. The primary objective is to establish a robust and reusable infrastructure. This class will serve as the foundation upon which we’ll build more sophisticated trading strategies in subsequent sections. The design prioritizes flexibility and a realistic simulation of market behavior, allowing us to rigorously evaluate trading ideas before deploying them in live markets. To achieve these goals, we’ll define the core requirements that the BacktestingBase class must fulfill. These requirements are critical for creating a backtesting environment that is both effective and adaptable.
Core Requirements for the Base Class
The BacktestingBase class must meet several key requirements to provide a solid foundation for our backtesting endeavors. These requirements dictate the class’s functionality and influence its design choices.
Data Handling: The class must efficiently retrieve and prepare historical market data. This includes reading data from external sources, cleaning and formatting the data, and potentially calculating initial indicators to inform trading decisions.
Order Execution: The class should handle the simulation of basic buy and sell orders, reflecting the core mechanics of trading. This involves tracking positions, managing cash balances, and simulating trade executions based on market prices.
Position Management: The class must effectively manage trading positions, accounting for entries, exits, and the overall position size. This includes handling both long and short positions (although the base class will focus on long-only to start).
Performance Calculation: The class should provide the necessary tools to calculate and report key performance metrics, such as net profit, returns, and the number of trades executed.
Flexibility and Extensibility: The class should be designed to allow for easy extension and customization. This means incorporating features like transaction costs and more complex order types in future iterations.
Helper Functions: The inclusion of helper functions to handle tasks such as data plotting, printing state variables, and date/price retrieval enhances readability, facilitates debugging, and provides the user with immediate insights into the backtesting process.
By adhering to these requirements, we can create a BacktestingBase class that is both powerful and adaptable, ready to support the development and evaluation of a wide range of trading strategies.
Data Retrieval and Preparation: The Cornerstone of Backtesting
The accurate and reliable handling of historical data is paramount for any backtesting framework. The BacktestingBase class is designed to encapsulate this critical functionality. Specifically, it will be responsible for retrieving and preparing end-of-day (EOD) data, typically sourced from a CSV file. The choice of EOD data simplifies the initial discussion, allowing us to focus on the core backtesting logic while still providing a practical foundation. This simplified approach is useful for educational purposes.
The data preparation stage is equally crucial. This involves a series of steps to ensure the data is clean, properly formatted, and ready for use in the backtesting process. This preparation may include cleaning the data to handle missing values, formatting the data to ensure consistency, and calculating initial indicators or returns that will inform trading decisions. For example, the class might calculate the daily log returns, which are essential for evaluating the performance of a trading strategy. We will see how this is implemented in the get_data() method.
Streamlining the Process: Helper and Convenience Functions
To enhance the usability and readability of the BacktestingBase class, we incorporate helper and convenience functions. These functions streamline the backtesting process and provide users with valuable insights into the simulation’s progress and results. These functions encapsulate common tasks, making the code cleaner, more modular, and easier to understand.
Some examples of these helpful functions include:
plot_data(): This function visualizes the historical price data, providing a clear picture of the market conditions during the backtesting period.print_balance(): This function displays the current cash balance, allowing users to monitor the financial state of the simulated trading account.print_net_wealth(): This function calculates and displays the net wealth (cash plus the value of any open positions), offering a comprehensive view of the portfolio’s performance.get_date_price(): This function retrieves the date and price for a given bar, which is essential for order execution and performance calculations.
These functions are not just for convenience; they enhance readability and simplify debugging. They allow users to easily understand the backtesting results and identify potential issues.
Simulating Market Orders: Order Placement in Action
At the heart of any backtesting framework is the ability to simulate the execution of trading orders. For simplicity, the BacktestingBase class will focus on market buy and sell orders. Market orders are the most basic type of order, executed immediately at the best available price. This simplification keeps the focus on the core backtesting logic, allowing us to build a solid foundation before introducing more complex order types.
The place_buy_order() and place_sell_order() methods within the class will handle order placement. These methods will simulate the execution of a buy or sell order and update the relevant state variables, such as the cash balance, the number of units held, and the trade counter. For example, when a buy order is placed, the class will:
Retrieve the current date and price.
Calculate the number of units to purchase (if not provided).
Update the cash balance by subtracting the cost of the purchase.
Increase the number of units held.
Increment the trade counter.
Print trade execution information for monitoring.
The place_sell_order() will follow a similar logic, but it will update the cash balance by adding the proceeds from the sale and decrease the number of units held.
Ensuring Accurate Performance: Position Closing
To accurately calculate the performance of a trading strategy, it’s essential to close out all open positions at the end of the backtesting period. The BacktestingBase class will include a close_out() method for this purpose. This method simulates the final trade, ensuring that all positions are liquidated at the last available price. This allows for an accurate calculation of the final profit or loss.
The close_out() method will perform the following steps:
Retrieve the final date and price.
Update the cash balance based on the value of the position.
Set the number of units held to zero.
Increment the trade counter.
Print the final balance, net performance, and the number of trades executed.
This approach ensures that all gains or losses from the trading strategy are realized and accounted for in the final performance calculations. No transaction costs are subtracted in this step. The final balance, net performance, and number of trades executed are then printed.
Building Upon the Foundation: Connecting to Subsequent Sections
The BacktestingBase class is designed to serve as the foundation for the subsequent sections of this article. Its design allows us to build more complex trading strategies upon this base, promoting reusability and maintainability. By creating a robust base class that meets the core requirements outlined above, we can then create specialized classes for different trading strategies.
For example, we will build classes for long-only and long-short backtesting. These classes will inherit from the BacktestingBase class and extend its functionality to implement specific trading strategies. This approach promotes code reuse and simplifies the development process.
Code Walkthrough and Implementation
Now, let’s dive into the implementation details of the BacktestingBase class. We’ll examine the code in detail, explaining each method and its role in the backtesting process.
The __init__ Method: Setting the Stage
The __init__ method is the constructor of the BacktestingBase class. It initializes the class’s attributes and sets up the environment for the backtesting process.
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
class BacktestingBase:
"""
Base class for event-based backtesting.
"""
def __init__(self, symbol, start, end, amount, ftc=0.0, ptc=0.0, verbose=True):
"""
Initializes the backtesting environment.
Args:
symbol (str): The financial instrument symbol.
start (str): The start date for backtesting (YYYY-MM-DD).
end (str): The end date for backtesting (YYYY-MM-DD).
amount (float): The initial amount of capital available.
ftc (float): Fixed transaction costs per trade (default: 0.0).
ptc (float): Percentage transaction costs per trade (default: 0.0).
verbose (bool): If True, prints trade execution details (default: True).
"""
self.symbol = symbol
self.start = start
self.end = end
self._amount = amount # Private attribute for initial amount
self.amount = amount # Public attribute for current amount
self.ftc = ftc # Fixed transaction costs
self.ptc = ptc # Percentage transaction costs
self.verbose = verbose
self.units = 0 # Number of units held
self.trades = 0 # Trade counter
self.position = 0 # Current position (0: no position, 1: long, -1: short)
self.get_data() # Load and prepare the data
First, we import the necessary libraries: numpy for numerical operations, pandas for data manipulation, and matplotlib.pyplot for plotting.
The initial amount available is stored in two attributes: _amount and amount. The _amount attribute is a private attribute, conventionally indicated by the leading underscore. This means that it’s intended for internal use within the class, and the user should not directly modify it. The amount attribute, on the other hand, represents the current available capital and is publicly accessible.
The __init__ method takes several parameters:
symbol: The financial instrument symbol (e.g., ‘AAPL’).start: The start date for backtesting (YYYY-MM-DD).end: The end date for backtesting (YYYY-MM-DD).amount: The initial amount of capital available.ftc: Fixed transaction costs per trade (default: 0.0).ptc: Percentage transaction costs per trade (default: 0.0).verbose: A boolean flag to control the output of trade execution details (default: True).
We assume no transaction costs by default, although the parameters for fixed and percentage transaction costs are provided. Finally, the get_data() method is called to load and prepare the historical market data.
The get_data Method: Data Acquisition and Preparation
The get_data method is responsible for retrieving and preparing the historical market data from a CSV file. The method is called in the __init__ method.
def get_data(self):
"""
Retrieves and prepares EOD data from a CSV file.
"""
try:
raw = pd.read_csv('datas/' + self.symbol + '_EOD_data.csv',
index_col=0, parse_dates=True)
except FileNotFoundError:
print(f"Error: Data file for {self.symbol} not found.")
return
raw = raw[self.start:self.end] # Select data for the specified time interval
self.data = raw.copy() # Create a copy to avoid modifying the original data
self.data.rename(columns={'Adj Close': 'price'}, inplace=True)
self.data['returns'] = np.log(self.data['price'] / self.data['price'].shift(1))
self.data.dropna(inplace=True) # Remove rows with missing values
The get_data() method first attempts to read data from a CSV file with the filename convention of symbol_EOD_data.csv. The CSV file is expected to be located in a directory named datas. The index_col=0 argument specifies that the first column of the CSV file should be used as the index (date). The parse_dates=True argument converts the index column to datetime objects.
If the data file is not found, a FileNotFoundError exception is caught, and an error message is printed, and the function returns.
Next, the data is filtered to include only the data within the specified start and end dates. A copy of the filtered data is created to avoid modifying the original data. The rename() method is used to rename the ‘Adj Close’ column to ‘price’, making the data more accessible. The log returns are calculated using the formula np.log(price / price.shift(1)). The shift(1) function is used to calculate the price from the previous day.
Finally, the dropna() method is used to remove any rows containing missing data. This is essential to prevent errors during subsequent calculations. This code has been used in previous articles.
Helper Methods: Enhancing Usability
The BacktestingBase class includes several helper methods to enhance usability and provide insights into the backtesting process.
def plot_data(self, cols=None):
"""
Plots the closing prices.
Args:
cols (list, optional): Columns to plot (default: None, plots 'price').
"""
if cols is None:
cols = ['price']
self.data[cols].plot(figsize=(10, 6))
plt.title(f'{self.symbol} Price Data')
plt.ylabel('Price')
plt.xlabel('Date')
plt.show()
def get_date_price(self, bar):
"""
Returns the date and price for a given bar.
Args:
bar (int): The index of the bar.
Returns:
tuple: A tuple containing the date and price.
"""
date = self.data.index[bar]
price = self.data['price'][bar]
return date, price
def print_balance(self, bar):
"""
Prints the current cash balance.
Args:
bar (int): The index of the bar.
"""
print(f' | {self.get_date_price(bar)[0].strftime("%Y-%m-%d")} | Balance: {self.amount:6.2f}')
def print_net_wealth(self, bar):
"""
Prints the current net wealth.
Args:
bar (int): The index of the bar.
"""
nw = self.amount + self.units * self.data['price'][bar]
print(f' | {self.get_date_price(bar)[0].strftime("%Y-%m-%d")} | Net Wealth: {nw:6.2f}')
plot_data(): This method plots the closing prices of the security. It takes an optionalcolsargument, which specifies the columns to plot. If no columns are specified, it plots the ‘price’ column. Theplt.title(),plt.ylabel(), andplt.xlabel()functions are used to add a title and axis labels. The plot is displayed usingplt.show().get_date_price(bar): This method returns the date and price for a given bar index. It retrieves the date and price from theself.dataDataFrame and returns them as a tuple.print_balance(bar): This method prints the current cash balance at a given bar index. It uses theget_date_price()method to get the date and formats the output for readability.print_net_wealth(bar): This method prints the current net wealth at a given bar index. Net wealth is calculated as the sum of the current cash balance and the value of any open positions. It also uses theget_date_price()method to get the date and formats the output for readability.
These helper methods provide valuable insights into the backtesting process, making it easier to monitor performance and identify any potential issues.
Order Placement Methods: place_buy_order() and place_sell_order()
The core of the backtesting logic lies in the order placement methods. These methods simulate the execution of buy and sell orders and update the relevant state variables.
def place_buy_order(self, bar, units=None, amount=None):
"""
Places a buy order.
Args:
bar (int): The index of the bar.
units (int, optional): The number of units to buy (default: None).
amount (float, optional): The amount to spend on buying (default: None).
"""
date, price = self.get_date_price(bar) # Get the date and price
if amount is not None:
units = int(amount / (price * (1 + self.ptc)) ) # Calculate units if amount is provided
elif units is None:
raise ValueError('units or amount has to be specified')
self.amount -= (units * price * (1 + self.ptc) + self.ftc) # Update cash balance
self.units += units # Increase the number of units
self.trades += 1 # Increment the trade counter
if self.verbose: # Print trade execution information if verbose is True
print(f' | {date.strftime("%Y-%m-%d")} | Buy | {units:6d} | @ {price:7.2f} | Balance: {self.amount:7.2f}')
self.position = 1 # set position to long (1)
The place_buy_order() method simulates the execution of a buy order. It takes three arguments:
bar: The index of the bar.units: The number of units to buy (optional).amount: The amount to spend on buying (optional).
The method first calls get_date_price() to retrieve the date and price for the given bar. Then, it checks if the amount is specified; if so, the number of units is calculated. If neither units nor amount is specified, a ValueError is raised.
The cash balance is updated by subtracting the cost of the purchase, including any transaction costs (both fixed and percentage). The number of units held is increased, and the trade counter is incremented. If the verbose flag is set to True, the method prints trade execution information, including the date, order type, number of units, price, and current balance. Finally, the position is set to 1, indicating a long position.
def place_sell_order(self, bar, units=None, amount=None):
"""
Places a sell order.
Args:
bar (int): The index of the bar.
units (int, optional): The number of units to sell (default: None).
amount (float, optional): The amount to receive from selling (default: None).
"""
date, price = self.get_date_price(bar)
if units is None:
units = int(amount / price)
self.amount += units * price * (1 - self.ptc) - self.ftc
self.units -= units
self.trades += 1
if self.verbose:
print(f' | {date.strftime("%Y-%m-%d")} | Sell | {units:6d} | @ {price:7.2f} | Balance: {self.amount:7.2f}')
self.position = 0
The place_sell_order() method simulates the execution of a sell order. It is similar to place_buy_order(), but the cash balance is updated by adding the proceeds from the sale, less any transaction costs. The number of units held is decreased, and the trade counter is incremented. If the verbose flag is set to True, trade execution information is printed. The position is set to 0, indicating no open position.
Closing Out Positions: The close_out() Method
The close_out() method is crucial for ensuring accurate performance calculations. It closes out any open positions at the end of the backtesting period.
def close_out(self, bar):
"""
Closes out any open positions at the end of the backtesting period.
Args:
bar (int): The index of the bar.
"""
date, price = self.get_date_price(bar)
self.amount += self.units * price
self.trades += 1
if self.verbose:
print(f' | {date.strftime("%Y-%m-%d")} | CLOSE | {self.units:6d} | @ {price:7.2f} | Balance: {self.amount:7.2f}')
print(75 * '-')
print(f' | Net Performance | {self.amount - self._amount:7.2f} |')
print(f' | Number of Trades | {self.trades:7d} |')
print(75 * '-')
The close_out() method takes the bar index as an argument. It first retrieves the date and price using get_date_price(). The cash balance is updated by adding the value of the open position (units * price). The trade counter is incremented. If the verbose flag is set to True, trade execution information is printed. Finally, it prints the net performance (profit or loss) and the number of trades executed.
The __main__ Section: Putting It All Together
To illustrate the functionality of the BacktestingBase class, let’s examine a simple example of how it can be used. The following code snippet demonstrates how to instantiate the class and use it to retrieve and plot the data.
if __name__ == '__main__':
bt = BacktestingBase('AAPL', '2023-01-01', '2023-12-31', 10000)
print(bt.data.info())
bt.plot_data()
This code first instantiates an object of the BacktestingBase class, passing in the symbol (‘AAPL’), the start date (‘2023-01-01’), the end date (‘2023-12-31’), and the initial amount (10000). This instantiation automatically leads to data retrieval and preparation through the __init__ method. The print(bt.data.info()) line prints information about the data, including the data type, memory usage, and any missing values. Finally, the plot_data() method is called to visualize the data.
The output would include information about the DataFrame and a plot of the data, showing the closing prices of Apple stock (AAPL) over the specified period.
From Base to Strategy: Transition to Next Sections
The BacktestingBase class provides a solid foundation for building more complex trading strategies. The classes in the next sections will build upon this base class to implement various trading strategies. This object-oriented approach promotes code reusability and maintainability.
We will explore the development of both long-only and long-short backtesting classes. These classes will inherit from the BacktestingBase class and extend its functionality to implement specific trading strategies. This will allow us to test and evaluate different trading ideas rigorously.
Object-Oriented Programming Benefits
The use of object-oriented programming (OOP) provides significant advantages for building our backtesting infrastructure. OOP principles such as encapsulation, inheritance, and polymorphism promote reusability, maintainability, and extensibility. Encapsulation allows us to bundle data and methods within a single class, creating a modular and organized codebase. Inheritance allows us to create specialized classes (e.g., long-only, long-short) that inherit from a base class, reducing code duplication and promoting code reuse. Polymorphism allows us to treat objects of different classes in a uniform manner, making it easier to build complex trading strategies.
The BacktestingBase class is designed to be easily enhanced to add more features, such as different order types, transaction costs, and risk management tools.
Visualizing the Data: Figure 6-1
The plot of the data, as retrieved for the symbol by the BacktestingBase class, provides a crucial visual representation of the market data. This plot shows the closing prices of the asset over the backtesting period. This allows us to understand the data, confirm that the data has been retrieved and prepared correctly, and provides a visual context for evaluating the performance of any trading strategy that we implement. The plot helps to understand the price fluctuations, trends, and volatility of the asset. We see the data plotted from the start date until the end date, which we have defined in the __init__ method.
Keeping It Concise: Importance of Simplifications
To keep the code concise and focused on the core functionality, we have made certain simplifications in the Python classes. For instance, we have not implemented checks for liquidity, which would add complexity. We also expect that at least one of the parameters (units or amount) will be specified for order placement. These are technical and economic simplifications that allow us to concentrate on the essential elements of backtesting while avoiding unnecessary complexity. These simplifications make the code easier to understand and maintain.
Having previously established the fundamental building blocks of backtesting, we now turn to a critical real-world constraint: long-only investing. Many institutional investors, such as pension funds and mutual funds, are either legally prohibited from short selling or operate under policies that restrict it. Even individual investors may choose a long-only approach for various reasons, including risk aversion or regulatory limitations. This section focuses on adapting our backtesting framework to accommodate such constraints. We will explore the implementation of a BacktestLongOnly class that builds upon the foundational BacktestBase class introduced earlier. This class will allow us to rigorously evaluate the performance of trading strategies, specifically those like Simple Moving Average (SMA), momentum, and mean reversion, within the confines of a long-only portfolio. The goal is to understand how these strategies perform when we are only allowed to buy and hold assets, or hold cash equivalents.
The BacktestLongOnly Class: Structure and Initialization
The BacktestLongOnly class inherits from BacktestBase, inheriting all the base functionality, and then extending it to handle long-only constraints. This approach allows for code reuse and ensures a consistent framework for backtesting. Let’s begin by outlining the structure of the BacktestLongOnly class, focusing on the core modifications needed to enforce the long-only restriction.
import pandas as pd
class BacktestBase: # Assuming BacktestBase is defined elsewhere
def __init__(self, data, initial_capital=100000, commission=0.001):
self.data = data
self.initial_capital = initial_capital
self.capital = initial_capital
self.commission = commission
self.positions = 0 # Current position (0: neutral, 1: long)
self.holdings = 0
self.trades = pd.DataFrame(columns=['date', 'action', 'price', 'shares', 'commission'])
self.cash_history = [initial_capital]
self.position_history = [0] # Track positions over time
self.date_history = [data.index[0]] # Track the dates
self.price_history = [data['Close'][0]] # Track prices over time
def place_buy_order(self, date, price, shares):
cost = shares * price * (1 + self.commission)
if cost <= self.capital:
self.capital -= cost
self.holdings += shares
self.trades = pd.concat([self.trades, pd.DataFrame({'date': [date], 'action': ['buy'], 'price': [price], 'shares': [shares], 'commission': [shares * price * self.commission]})], ignore_index=True)
self.cash_history.append(self.capital)
self.position_history.append(1)
self.date_history.append(date)
self.price_history.append(price)
return True # Order successful
else:
print(f"Insufficient capital to buy {shares} shares at {price} on {date}")
return False # Order failed
def place_sell_order(self, date, price, shares):
if self.holdings >= shares:
proceeds = shares * price * (1 - self.commission)
self.capital += proceeds
self.holdings -= shares
self.trades = pd.concat([self.trades, pd.DataFrame({'date': [date], 'action': ['sell'], 'price': [price], 'shares': [shares], 'commission': [shares * price * self.commission]})], ignore_index=True)
self.cash_history.append(self.capital)
self.position_history.append(0)
self.date_history.append(date)
self.price_history.append(price)
return True # Order successful
else:
print(f"Insufficient shares to sell {shares} shares on {date}")
return False # Order failed
def close_out(self, date, price):
if self.holdings > 0:
self.place_sell_order(date, price, self.holdings)
def calculate_performance(self):
# Simplified performance calculation. More sophisticated calculations
# like Sharpe ratio, Sortino ratio, and max drawdown are useful, but
# omitted here for brevity.
final_capital = self.capital + self.holdings * self.data['Close'].iloc[-1] # Assume last closing price
return final_capital / self.initial_capital - 1
def get_trades(self):
return self.trades
def get_cash_history(self):
return pd.Series(self.cash_history, index=self.date_history)
def get_position_history(self):
return pd.Series(self.position_history, index=self.date_history)
class BacktestLongOnly(BacktestBase):
def __init__(self, data, initial_capital=100000, commission=0.001):
super().__init__(data, initial_capital, commission)
self.position = 0 # 0 for neutral, 1 for long. Short positions are not allowed.
def place_sell_order(self, date, price, shares):
# Override the sell order to prevent short selling. We only sell if
# we currently have a long position.
if self.position == 1 and self.holdings >= shares:
proceeds = shares * price * (1 - self.commission)
self.capital += proceeds
self.holdings -= shares
self.trades = pd.concat([self.trades, pd.DataFrame({'date': [date], 'action': ['sell'], 'price': [price], 'shares': [shares], 'commission': [shares * price * self.commission]})], ignore_index=True)
self.cash_history.append(self.capital)
self.position_history.append(0)
self.date_history.append(date)
self.price_history.append(price)
self.position = 0 # Reset position to neutral after selling
return True
else:
print(f"Cannot sell. Either not in a long position or insufficient shares on {date}")
return False
The key change here is in the overridden place_sell_order method. Within the BacktestLongOnly class, we’ve added a check: if self.position == 1 and self.holdings >= shares:. This ensures that a sell order is only executed if the investor is currently holding a long position (self.position == 1) and has sufficient shares to sell. If the conditions are not met, the sell order is rejected, effectively preventing short selling. The position attribute is also updated in this method, which is crucial for tracking the portfolio’s state. The constructor (__init__) initializes the position to 0 (neutral), reflecting the starting state of the portfolio. This design ensures that the long-only constraint is strictly enforced throughout the backtesting period. Any attempt to short sell will be blocked.
The run_mean_reversion_strategy() Method: A Detailed Example
To illustrate the practical application of the BacktestLongOnly class, we will now examine the run_mean_reversion_strategy() method. This method provides a clear, step-by-step procedure for backtesting a mean reversion trading strategy within the constraints of our long-only framework. The mean reversion strategy, as the name suggests, capitalizes on the tendency of asset prices to revert to their mean or average value over time. When the price deviates significantly from its moving average, a mean reversion strategy would typically bet on the price returning to the mean.
We will define the method with two parameters: SMA (the Simple Moving Average in days) and threshold (a deviation-based signal relative to the SMA). The SMA parameter determines the lookback period for calculating the moving average. The threshold parameter sets the percentage deviation from the SMA that triggers a buy or sell signal.
Before diving into the code, let’s outline the key steps involved in this method:
Initialization: Set the initial parameters for the strategy, including the
SMAperiod and thethreshold.Looping Through Data: Iterate through the historical price data, starting from the period required to calculate the SMA.
Buy Signal: If the current price is below the SMA minus the threshold, and the portfolio is not already long (i.e.,
self.position == 0), place a buy order.Sell Signal: If the current price is above the SMA plus the threshold, and the portfolio is long (i.e.,
self.position == 1), place a sell order.Closing Positions: After the loop, ensure that any open positions are closed out at the end of the backtesting period.
Now, let’s examine the code implementation:
import pandas as pd
import numpy as np
class BacktestLongOnly(BacktestBase):
# ... (Previous code from BacktestLongOnly class) ...
def run_mean_reversion_strategy(self, SMA, threshold):
print(f"Running Mean Reversion Strategy with SMA: {SMA} and Threshold: {threshold}")
# Calculate the Simple Moving Average
self.data['SMA'] = self.data['Close'].rolling(window=SMA).mean()
self.data.dropna(inplace=True) # Drop NaN values after calculating SMA
# Iterate through the data, starting from the SMA period
for i in range(SMA, len(self.data)):
date = self.data.index[i]
price = self.data['Close'][i]
sma = self.data['SMA'][i]
# Buy signal: price below SMA - threshold and no current position
if price < (sma * (1 - threshold)) and self.position == 0:
shares_to_buy = int(self.capital / price) # Buy as many shares as possible
if shares_to_buy > 0:
if self.place_buy_order(date, price, shares_to_buy):
self.position = 1 # Update the position to long
# Sell signal: price above SMA + threshold and current long position
elif price > (sma * (1 + threshold)) and self.position == 1:
if self.holdings > 0:
self.place_sell_order(date, price, self.holdings) # Sell all holdings
self.position = 0 # Update the position to neutral
# Close out any remaining positions at the end
self.close_out(self.data.index[-1], self.data['Close'].iloc[-1])
print("Mean Reversion Strategy Finished")
Let’s break down this code. First, the method takes SMA and threshold as inputs, which represent the length of the moving average and the deviation from the moving average that triggers a trade, respectively.
The code starts by calculating the Simple Moving Average (SMA) using the .rolling() function in pandas, applied to the ‘Close’ price data, with a window equal to the SMA parameter. The .mean() function then calculates the average over that window. The dropna(inplace=True) method removes any rows containing NaN (Not a Number) values. These are generated at the beginning of the data due to the rolling window calculation. We must remove these to prevent errors in the subsequent trading logic.
Next, a for loop iterates through the dataset, starting from the SMA period. This is because we need sufficient historical data to calculate the SMA. Inside the loop, the current date, price, and SMA value are retrieved.
The core trading logic is encapsulated within if and elif statements. The first if statement checks for a buy signal: if price < (sma * (1 - threshold)) and self.position == 0:. This condition triggers a buy order if the current price is below the SMA minus the threshold and the portfolio is currently not holding any shares (i.e., self.position is 0). The shares_to_buy calculation determines how many shares can be purchased given the available capital. The place_buy_order method, inherited from the base class, is then called to execute the trade. The self.position = 1 line then sets the portfolio’s position to long.
The elif statement handles sell signals. elif price > (sma * (1 + threshold)) and self.position == 1: checks if the current price is above the SMA plus the threshold and the portfolio is already holding shares (i.e., self.position is 1). If these conditions are met, the place_sell_order method is called to sell all holdings. Finally, self.position = 0 resets the portfolio to a neutral position after selling.
Finally, after the loop completes, the close_out method is called to liquidate any remaining positions at the end of the backtesting period, ensuring a clean exit from the market. The close_out method takes the last date and the closing price on that date as arguments.
Strategy Execution and Results
To illustrate how the run_mean_reversion_strategy() method works in practice, let’s assume we have a dataset of historical stock prices stored in a pandas DataFrame called data. This DataFrame has an index of dates and a ‘Close’ column representing the closing prices. The following code snippet provides a simple example of how to instantiate the BacktestLongOnly class, run the mean reversion strategy, and analyze the results.
# Sample data (replace with your actual data)
import pandas as pd
import numpy as np
# Generate sample data (replace with your actual data)
np.random.seed(42) # for reproducibility
dates = pd.date_range(start='2023-01-01', end='2023-12-31', freq='B') # Business days
prices = np.cumsum(np.random.randn(len(dates)) + 0.01) + 100 # Simulate prices
data = pd.DataFrame({'Close': prices}, index=dates)
# Instantiate the BacktestLongOnly class
backtest = BacktestLongOnly(data=data)
# Run the mean reversion strategy
SMA_period = 20 # Example SMA period
threshold_deviation = 0.02 # Example threshold
backtest.run_mean_reversion_strategy(SMA_period, threshold_deviation)
# Calculate and print the performance
performance = backtest.calculate_performance()
print(f"Strategy Performance: {performance:.2%}")
# Access the trades
trades_df = backtest.get_trades()
print("\nTrades:")
print(trades_df)
# Access the cash history
cash_history = backtest.get_cash_history()
print("\nCash History:")
print(cash_history)
# Access the position history
position_history = backtest.get_position_history()
print("\nPosition History:")
print(position_history)
In this example, we begin by creating a pandas DataFrame data with sample price data, simulating the behavior of a stock price over a year. This is a crucial step, as the data represents the “market” that our strategy will interact with. Remember to replace this with your actual historical price data.
Next, we instantiate the BacktestLongOnly class, passing the data as an argument. This initializes the backtesting environment, including the initial capital and commission rates.
Then, we call the run_mean_reversion_strategy() method, passing in the SMA_period (20 days) and threshold_deviation (2%) parameters. These parameters dictate how the strategy will trade.
After the strategy has been executed, we calculate and print the performance using backtest.calculate_performance(). This provides a simple measure of the strategy’s success (or failure). Other performance metrics such as Sharpe ratio or maximum drawdown could be added for a more detailed analysis.
Finally, we access and display the trades, cash history, and position history using the get_trades(), get_cash_history(), and get_position_history() methods. These methods provide valuable insight into the strategy’s behavior, allowing us to see when trades were executed, how the capital fluctuated over time, and the portfolio’s position (long or neutral) at each point in time. The trades DataFrame provides a record of all buy and sell transactions, including the date, action (buy or sell), price, number of shares traded, and commission paid. The cash history tracks the portfolio’s cash balance over time, and the position history tracks the portfolio’s position (long or neutral) over time.
Impact of Transaction Costs
One of the most critical factors to consider when evaluating a trading strategy is the impact of transaction costs. These costs, which include brokerage commissions, bid-ask spreads, and market impact, can significantly erode profits, particularly for strategies that involve frequent trading. The BacktestBase class and, by extension, the BacktestLongOnly class, includes a commission parameter, which allows us to model these costs.
To illustrate the impact of transaction costs, we can modify the example above by running the same strategy with different commission rates. Consider the following code to simulate this:
# Re-run the backtest with varying commission rates
commission_rates = [0.000, 0.001, 0.002, 0.005] # Example rates
results = {}
for commission_rate in commission_rates:
backtest = BacktestLongOnly(data=data, commission=commission_rate)
backtest.run_mean_reversion_strategy(SMA_period, threshold_deviation)
performance = backtest.calculate_performance()
results[commission_rate] = performance
print("\nPerformance with different commission rates:")
for rate, performance in results.items():
print(f"Commission Rate: {rate:.3f}, Performance: {performance:.2%}")
This code re-runs the mean reversion strategy with a range of commission rates (0%, 0.1%, 0.2%, and 0.5%). This allows us to directly compare the impact of transaction costs on the strategy’s performance. The results would demonstrate how performance decreases as the commission rate increases. This is expected, as higher commissions lead to lower net profits after each trade.
The number of trades a strategy generates is another important factor influencing performance. Strategies that trade frequently will incur higher transaction costs, which can significantly impact their profitability. The trades DataFrame, obtained via the get_trades() method, allows us to easily analyze the frequency of trades. By counting the number of rows in this DataFrame, we can determine how many trades the strategy executed during the backtesting period.
Comparative Analysis of Strategies
Beyond the mean reversion strategy, the BacktestLongOnly class can be used to backtest other trading strategies, such as SMA-based strategies and momentum strategies. These strategies can be implemented by creating new methods within the BacktestLongOnly class, similar to run_mean_reversion_strategy(), each tailored to a specific trading approach.
For instance, an SMA-based strategy might generate buy signals when the price crosses above the SMA and sell signals when the price crosses below. A momentum strategy could identify stocks exhibiting strong price increases over a defined period and buy them, selling when the momentum wanes. The core backtesting logic (buying and selling based on specific conditions) would remain similar, but the signal generation logic would differ.
By backtesting multiple strategies within the same framework, we can perform a comparative analysis of their performance. This involves comparing key performance metrics, such as the final portfolio value, Sharpe ratio, maximum drawdown, and the number of trades executed. We can then assess the trade-offs between different strategies and evaluate which ones are most suitable for a long-only portfolio.
Let’s consider some hypothetical results. Imagine we backtest the SMA, momentum, and mean reversion strategies, both without and with transaction costs (e.g., a commission of 0.1%). We might find that the momentum strategy outperforms the SMA-based strategy before considering transaction costs. However, after incorporating transaction costs, the SMA-based strategy, which may generate fewer trades, could outperform the momentum strategy. This highlights the crucial role of transaction costs in determining the overall profitability of a trading strategy.
# Example: Backtesting SMA, Momentum, and Mean Reversion strategies
class BacktestLongOnly(BacktestBase):
# ... (previous code) ...
def run_sma_strategy(self, short_window, long_window):
# Implement an SMA crossover strategy
print(f"Running SMA Strategy with Short Window: {short_window} and Long Window: {long_window}")
self.data['SMA_short'] = self.data['Close'].rolling(window=short_window).mean()
self.data['SMA_long'] = self.data['Close'].rolling(window=long_window).mean()
self.data.dropna(inplace=True)
for i in range(max(short_window, long_window), len(self.data)):
date = self.data.index[i]
price = self.data['Close'][i]
sma_short = self.data['SMA_short'][i]
sma_long = self.data['SMA_long'][i]
# Buy signal: short SMA crosses above long SMA
if sma_short > sma_long and self.data['SMA_short'][i-1] <= self.data['SMA_long'][i-1] and self.position == 0:
shares_to_buy = int(self.capital / price)
if shares_to_buy > 0:
if self.place_buy_order(date, price, shares_to_buy):
self.position = 1
# Sell signal: short SMA crosses below long SMA
elif sma_short < sma_long and self.data['SMA_short'][i-1] >= self.data['SMA_long'][i-1] and self.position == 1:
if self.holdings > 0:
self.place_sell_order(date, price, self.holdings)
self.position = 0
self.close_out(self.data.index[-1], self.data['Close'].iloc[-1])
def run_momentum_strategy(self, lookback_period, threshold):
# Implement a momentum strategy
print(f"Running Momentum Strategy with Lookback: {lookback_period} and Threshold: {threshold}")
self.data['Momentum'] = (self.data['Close'] / self.data['Close'].shift(lookback_period)) - 1
self.data.dropna(inplace=True)
for i in range(lookback_period, len(self.data)):
date = self.data.index[i]
price = self.data['Close'][i]
momentum = self.data['Momentum'][i]
# Buy signal: momentum exceeds threshold and no current position
if momentum > threshold and self.position == 0:
shares_to_buy = int(self.capital / price)
if shares_to_buy > 0:
if self.place_buy_order(date, price, shares_to_buy):
self.position = 1
# Sell signal: momentum drops below threshold and current long position
elif momentum < -threshold and self.position == 1:
if self.holdings > 0:
self.place_sell_order(date, price, self.holdings)
self.position = 0
self.close_out(self.data.index[-1], self.data['Close'].iloc[-1])
# Example Usage (assuming data is already loaded)
short_window = 20
long_window = 50
momentum_lookback = 10
momentum_threshold = 0.05
sma_result = BacktestLongOnly(data=data, commission=0.001)
sma_result.run_sma_strategy(short_window, long_window)
sma_performance = sma_result.calculate_performance()
sma_trades = sma_result.get_trades()
momentum_result = BacktestLongOnly(data=data, commission=0.001)
momentum_result.run_momentum_strategy(momentum_lookback, momentum_threshold)
momentum_performance = momentum_result.calculate_performance()
momentum_trades = momentum_result.get_trades()
mean_reversion_result = BacktestLongOnly(data=data, commission=0.001)
mean_reversion_result.run_mean_reversion_strategy(SMA_period, threshold_deviation)
mean_reversion_performance = mean_reversion_result.calculate_performance()
mean_reversion_trades = mean_reversion_result.get_trades()
print(f"SMA Performance: {sma_performance:.2%}, Trades: {len(sma_trades)}")
print(f"Momentum Performance: {momentum_performance:.2%}, Trades: {len(momentum_trades)}")
print(f"Mean Reversion Performance: {mean_reversion_performance:.2%}, Trades: {len(mean_reversion_trades)}")
The code first implements the run_sma_strategy() and run_momentum_strategy() methods within the BacktestLongOnly class. The run_sma_strategy() method implements a simple moving average crossover strategy, buying when the short-term SMA crosses above the long-term SMA and selling when the opposite occurs. The run_momentum_strategy() method implements a momentum strategy, buying when the momentum exceeds a defined threshold and selling when the momentum falls below a threshold. Then the code uses these methods to execute the strategies. The example then prints the performance of each strategy, along with the number of trades executed, allowing for a direct comparison of the impact of transaction costs.
The Frequency of Trades: A Third Dimension of Performance
The backtesting results provide a multi-faceted view of a strategy’s performance. Beyond the traditional metrics like returns and risk, the frequency of trades emerges as a crucial third dimension. As demonstrated, a high trading frequency can lead to higher transaction costs, potentially eroding any gains from the strategy.
This insight has important implications for investment strategy selection. Strategies that exhibit high trading frequency may not be suitable for long-term investment goals due to the associated transaction costs. In contrast, strategies with lower trading frequency, or those involving buy-and-hold approaches, may be more cost-effective over the long run.
This understanding highlights the importance of considering low-cost, passive investment strategies as an alternative to strategies with high trading frequency. Exchange-Traded Funds (ETFs), for example, offer a convenient and cost-effective way to gain exposure to a diversified portfolio of assets. ETFs typically have low expense ratios and can be traded like stocks, making them a viable option for investors seeking to minimize transaction costs. The backtesting results, therefore, provide a practical takeaway, linking strategy performance to real-world investment choices. By carefully analyzing the frequency of trades and the associated transaction costs, investors can make more informed decisions about the most appropriate investment strategies for their portfolios.
Expanding Backtesting Capabilities: Introducing the BacktestLongShort Class
Building on the foundation of the BacktestBase class, which provided a framework for simulating long-only trading strategies, we now introduce the BacktestLongShort class. This class significantly extends the backtesting capabilities by incorporating the ability to take both long and short positions. This addition is crucial for exploring a wider range of trading strategies, especially those that seek to profit from both rising and falling markets, such as market-neutral or arbitrage strategies.
The core of this expanded functionality lies in two new methods: go_long() and go_short(). These methods allow the backtesting engine to simulate entering and exiting short positions alongside long positions, thus providing a comprehensive environment for testing a broader spectrum of investment ideas. We will focus our explanation on the go_long() method, as go_short() mirrors its behavior with the necessary adjustments for short selling.
Deconstructing the go_long() Method
The go_long() method, the cornerstone of our long/short capability, is designed to handle the intricacies of entering a long position within a backtesting environment. It takes the current bar (typically representing a time period’s data, such as a day or hour) as a parameter, allowing the simulation to interact with price and volume data. Additionally, it accepts either the number of units to trade or a currency amount to invest. This flexibility allows for strategy implementations that either target a specific number of shares or use a percentage of available capital.
The logic within go_long() is structured to manage various scenarios, ensuring the correct execution of trading actions. Let’s dissect the key steps. First, it checks the current position. If self.position == -1, indicating a short position is active, we must first close the short position. This is achieved by calling self.place_buy_order(), effectively covering the short position. Only after the short position is closed can we initiate a long position. The place_buy_order() method, which we inherited from the BacktestBase class, handles the mechanics of placing a buy order at the current market price (or a simulated price based on the current bar’s data).
Following the handling of a potential short position, the code determines how to execute the long trade based on the provided parameters:
Units Specified: If the
unitsparameter is provided, the method places a buy order for the specified number of units. This is the most straightforward scenario, as it directly dictates the size of the long position.Amount Specified: If the
amountparameter is provided, the behavior depends on the value ofamount.amountis ‘all’: In this case, the method uses the entire available cash balance to purchase the asset. This strategy is useful when a trader wants to invest all available capital in a particular asset.amountis a numerical value: The method places a buy order for the specified currency amount. This allows for the purchase of assets based on a specific investment size.
For conciseness and to maintain focus on the core backtesting logic, we intentionally omit detailed liquidity checks. In a real-world trading environment, it is crucial to ensure that sufficient liquidity exists to execute a trade at the desired price. However, within the scope of this backtesting class, we prioritize simplifying the simulation process. Similarly, parameter validation (e.g., ensuring that either ‘units’ or ‘amount’ is provided, but not both) is omitted for brevity.
Here’s a basic code representation of the go_long() method:
class BacktestLongShort(BacktestBase): # Inherit from the base class
def __init__(self, symbol, data, initial_capital=100000, commission=0.0):
super().__init__(symbol, data, initial_capital, commission) # Initialize base class
self.position = 0 # 0: Neutral, 1: Long, -1: Short
def go_long(self, bar, units=None, amount=None):
"""
Places a buy order to go long.
Args:
bar (pd.Series): Current bar data.
units (int, optional): Number of units to buy. Defaults to None.
amount (float, optional): Amount of currency to invest. Defaults to None.
"""
if self.position == -1: # If currently short, close the short position
self.place_buy_order(bar, self.position_size) # place_buy_order uses self.position_size
if units:
self.place_buy_order(bar, units)
elif amount:
if amount == 'all':
units_to_buy = self.cash / bar['close'] # Estimate units to buy
self.place_buy_order(bar, units_to_buy)
else:
units_to_buy = amount / bar['close'] # Calculate units to buy
self.place_buy_order(bar, units_to_buy)
self.position = 1 # Set position to long
# ... other methods ...
In this example, the BacktestLongShort class inherits from the BacktestBase class, which encapsulates the basic trading functionalities. The go_long() method is then defined to handle the core logic of entering a long position as explained above. The self.position variable keeps track of our current position (neutral, long, or short). Also, note the use of the self.place_buy_order() method inherited from the BacktestBase class, which handles the actual order placement and transaction processing.
Mirroring the Functionality: The go_short() Method
The go_short() method, mirroring the functionality of go_long(), is designed to facilitate short selling within the backtesting framework. It allows for the simulation of selling an asset you do not own, with the expectation of buying it back at a lower price.
The logic within go_short() is symmetrical to go_long(). If a long position exists (self.position == 1), it must be closed using self.place_sell_order() before a short position can be initiated. This ensures that the simulation correctly accounts for the closing of existing positions before opening a new one.
Similar to go_long(), go_short() accepts either the number of units to sell short or a currency amount. If units are provided, a sell order is placed for the specified number of units. If amount is given, the behavior is as follows: if amount is set to ‘all’, the entire available cash balance is used to short sell the asset. Otherwise, a sell order is placed for the specified currency amount.
Here’s a simplified example of the go_short() method, using the same inheritance pattern as the go_long() method to illustrate the symmetry:
def go_short(self, bar, units=None, amount=None):
"""
Places a sell order to go short.
Args:
bar (pd.Series): Current bar data.
units (int, optional): Number of units to sell short. Defaults to None.
amount (float, optional): Amount of currency to sell short. Defaults to None.
"""
if self.position == 1: # If currently long, close the long position
self.place_sell_order(bar, self.position_size) # place_sell_order uses self.position_size
if units:
self.place_sell_order(bar, units)
elif amount:
if amount == 'all':
units_to_sell = self.cash / bar['close'] # Estimate units to sell
self.place_sell_order(bar, units_to_sell)
else:
units_to_sell = amount / bar['close'] # Calculate units to sell
self.place_sell_order(bar, units_to_sell)
self.position = -1 # Set position to short
Design Choices and Simplifications
The design of BacktestLongShort and its methods reflects a conscious effort to balance functionality with simplicity. We’ve made several design choices to streamline the backtesting process and to keep the code concise and focused on the core logic of trading strategy simulation.
As mentioned earlier, we intentionally omit checks for sufficient liquidity and validation of input parameters (e.g., ensuring that either ‘units’ or ‘amount’ is provided). These omissions represent both economic and technical simplifications.
Economic Simplifications: Liquidity checks, while critical in real-world trading, can add significant complexity to the backtesting simulation. They require simulating order book dynamics, which can slow down the backtesting process. By omitting these checks, we prioritize speed and ease of implementation, allowing us to test a wider range of strategies more efficiently.
Technical Simplifications: Parameter validation, while crucial for robust code, can introduce additional complexity. By focusing on the core functionality, we reduce the risk of introducing bugs and improve readability. We assume that the strategies implemented using this class will be carefully designed, and that the user will be responsible for providing valid inputs.
These simplifications allow us to concentrate on simulating the fundamental trading logic and to evaluate the performance of various strategies without being bogged down by the nuances of order book dynamics or input validation.
Implementing a Mean Reversion Strategy: The Core Loop
To demonstrate the practical application of the BacktestLongShort class, let’s examine the core loop from the run_mean_reversion_strategy() method. This strategy is chosen because it inherently requires the ability to handle both long and short positions and to transition between them.
A mean reversion strategy capitalizes on the tendency of prices to revert to their mean or average value. In this context, we use a simple moving average (SMA) as our mean. The strategy’s core logic involves identifying periods when the price deviates significantly from the SMA. When the price falls below a certain threshold relative to the SMA, a long position is initiated, anticipating a price increase back towards the mean. Conversely, when the price rises above a certain threshold, a short position is initiated, anticipating a price decrease.
The run_mean_reversion_strategy() method iterates through the historical price data and makes trading decisions based on the defined rules. The heart of this strategy lies in the conditional logic within the loop.
Here’s the implementation of the run_mean_reversion_strategy() method:
def run_mean_reversion_strategy(self, sma_period=20, threshold=0.01):
"""
Runs a mean reversion strategy.
Args:
sma_period (int): Period for calculating the Simple Moving Average. Defaults to 20.
threshold (float): Percentage threshold from SMA to trigger trades. Defaults to 0.01.
"""
for i, bar in enumerate(self.data.iterrows()):
# Ensure we have enough data for SMA calculation
if i < sma_period:
continue
# Calculate SMA
sma = self.data['close'].iloc[i - sma_period:i].mean()
price = bar[1]['close']
# Conditional logic for trading decisions
if self.position == 0: # Neutral position
if price < (sma * (1 - threshold)):
self.go_long(bar[1], amount='all') # Go long if price is below SMA - threshold
print(f"{bar[1]['date']}: Go Long at {price:.2f}, SMA: {sma:.2f}")
elif price > (sma * (1 + threshold)):
self.go_short(bar[1], amount='all') # Go short if price is above SMA + threshold
print(f"{bar[1]['date']}: Go Short at {price:.2f}, SMA: {sma:.2f}")
elif self.position == 1: # Long position
if price >= sma: # Close long position if price reaches or exceeds SMA
self.place_sell_order(bar[1], self.position_size) # place_sell_order uses self.position_size
self.position = 0
print(f"{bar[1]['date']}: Close Long at {price:.2f}, SMA: {sma:.2f}")
elif self.position == -1: # Short position
if price <= sma: # Close short position if price reaches or falls below SMA
self.place_buy_order(bar[1], self.position_size) # place_buy_order uses self.position_size
self.position = 0
print(f"{bar[1]['date']}: Close Short at {price:.2f}, SMA: {sma:.2f}")
self.close_out() # Close any open positions at the end
Let’s break down the conditional logic:
Neutral Position Check (
self.position == 0): This is the initial state. If the position is neutral (neither long nor short), the code checks the current price relative to the SMA and the threshold.If
price < (sma * (1 - threshold)): The price is significantly below the SMA. The strategy triggers ago_long()call, initiating a long position. The output also prints a message indicating the action.If
price > (sma * (1 + threshold)): The price is significantly above the SMA. The strategy triggers ago_short()call, initiating a short position. The output also prints a message indicating the action.
Long Position Check (
self.position == 1): If a long position is active, the code checks if the price is at or above the SMA.If
price >= sma: The long position is closed usingself.place_sell_order(). The position is set back to neutral. The output also prints a message indicating the action.
Short Position Check (
self.position == -1): If a short position is active, the code checks if the price is at or below the SMA.If
price <= sma: The short position is closed usingself.place_buy_order(). The position is set back to neutral. The output also prints a message indicating the action.
Finally, the self.close_out() method, which is called at the end of the loop, ensures that any open positions are closed at the last available price, providing a clean end to the backtesting simulation. This helps to accurately calculate the final performance metrics.
Analyzing the Performance Results
Once the Python script is executed, and the mean reversion strategy is backtested, it’s crucial to analyze the performance results. It’s common to find that, contrary to the initial expectation that adding short-selling capabilities would improve performance, the results show that strategies, both with and without transaction costs, perform worse. It’s not uncommon for some configurations to even result in net losses or the accumulation of debt.
For instance, a typical outcome might be the following: the equity curve, which tracks the portfolio’s value over time, shows a downward trend, indicating consistent losses. The Sharpe ratio, a measure of risk-adjusted return, might be negative, signaling poor risk-adjusted performance. The maximum drawdown, which represents the largest peak-to-trough decline during a specific period, may be significant, indicating substantial risk exposure.
These results highlight the importance of not jumping to conclusions based on initial backtesting results. Several factors can contribute to the observed performance:
Market Conditions: The mean reversion strategy, like any strategy, is sensitive to market conditions. It performs well in range-bound or trending markets, but it can struggle in highly volatile or strongly trending markets.
Parameter Optimization: The performance of the strategy is highly dependent on the parameters, such as the SMA period and the threshold. Poorly chosen parameters can lead to sub-optimal performance.
Transaction Costs: The inclusion of transaction costs can significantly impact the profitability of a strategy, especially for high-frequency trading.
Short Squeeze/Covering: Short positions may be exposed to high risk when the price of the asset unexpectedly rises. This can lead to significant losses if the short position is not quickly closed.