tensortrade
An open source reinforcement learning framework for training, evaluating, and deploying robust trading agents.
Top Related Projects
A toolkit for developing and comparing reinforcement learning algorithms.
Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
A fork of OpenAI Baselines, implementations of reinforcement learning algorithms
A standard API for single-agent reinforcement learning environments, with popular reference environments and related utilities (formerly Gym)
Lean Algorithmic Trading Engine by QuantConnect (Python, C#)
Quick Overview
TensorTrade is an open-source Python framework for building, training, evaluating, and deploying robust trading algorithms using reinforcement learning. It provides a modular architecture for composing trading environments, allowing users to define custom instruments, action schemes, reward functions, and trading logic.
Pros
- Flexible and modular design, allowing for easy customization of trading environments
- Integration with popular machine learning libraries like TensorFlow and PyTorch
- Supports multiple asset classes and exchange types
- Provides built-in performance metrics and visualization tools
Cons
- Steep learning curve for users new to reinforcement learning or algorithmic trading
- Limited documentation and examples for advanced use cases
- May require significant computational resources for complex trading strategies
- Still in active development, which may lead to breaking changes in future versions
Code Examples
- Creating a simple trading environment:
from tensortrade.env import TradingEnvironment
from tensortrade.feed.core import Stream, DataFeed
from tensortrade.data import CSVDataset
from tensortrade.oms.instruments import ExchangePair
from tensortrade.oms.exchanges import SimulatedExchange
from tensortrade.oms.services.execution.simulated import execute_order
dataset = CSVDataset("path/to/data.csv")
feed = DataFeed([
Stream.source(dataset.data, ['open', 'high', 'low', 'close']).rename("USD-BTC")
])
exchange = SimulatedExchange("simulated", service=execute_order)(
[ExchangePair("USD", "BTC")]
)
env = TradingEnvironment(
feed=feed,
portfolio=Portfolio("USD", [
ExchangePair("USD", "BTC")
]),
action_scheme="managed-risk",
reward_scheme="risk-adjusted",
window_size=20
)
- Training a trading agent using Stable Baselines3:
from stable_baselines3 import PPO
model = PPO("MlpPolicy", env, verbose=1)
model.learn(total_timesteps=10000)
- Evaluating the trained model:
from tensortrade.env.default.renderers import PlotlyTradingChart
obs = env.reset()
done = False
while not done:
action, _states = model.predict(obs)
obs, rewards, done, info = env.step(action)
env.render(renderer="plotly")
Getting Started
To get started with TensorTrade, follow these steps:
- Install TensorTrade:
pip install tensortrade
- Import necessary modules:
from tensortrade.env import TradingEnvironment
from tensortrade.feed.core import DataFeed, Stream
from tensortrade.oms.exchanges import SimulatedExchange
from tensortrade.oms.services.execution.simulated import execute_order
from tensortrade.oms.instruments import ExchangePair
- Create a simple trading environment and run a random agent:
import numpy as np
# Create a simple environment (assuming you have data)
env = TradingEnvironment(
feed=your_data_feed,
portfolio=your_portfolio,
action_scheme="managed-risk",
reward_scheme="simple",
window_size=10
)
# Run a random agent
for _ in range(100):
action = env.action_space.sample()
state, reward, done, info = env.step(action)
if done:
env.reset()
For more detailed examples and documentation, refer to the official TensorTrade documentation.
Competitor Comparisons
A toolkit for developing and comparing reinforcement learning algorithms.
Pros of Gym
- Broader scope, supporting a wide range of reinforcement learning environments
- Larger community and more extensive documentation
- Well-established standard in the RL research community
Cons of Gym
- Not specifically designed for financial trading environments
- Requires more setup and customization for trading-specific tasks
- Less focus on features tailored to algorithmic trading
Code Comparison
Gym:
import gym
env = gym.make('CartPole-v1')
observation = env.reset()
for _ in range(1000):
action = env.action_space.sample()
observation, reward, done, info = env.step(action)
TensorTrade:
from tensortrade.env import TradingEnvironment
from tensortrade.feed import DataFeed
env = TradingEnvironment(feed=DataFeed(), window_size=20)
state = env.reset()
for _ in range(1000):
action = env.action_space.sample()
state, reward, done, info = env.step(action)
TensorTrade is specifically designed for financial trading environments, offering built-in support for common trading operations and data structures. Gym, on the other hand, provides a more general-purpose framework for reinforcement learning tasks across various domains. While Gym requires more customization for trading-specific applications, it benefits from a larger ecosystem and broader applicability in RL research.
Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
Pros of Ray
- More versatile and general-purpose distributed computing framework
- Larger community and ecosystem with extensive documentation
- Better scalability for large-scale machine learning and AI applications
Cons of Ray
- Steeper learning curve due to its broader scope
- May be overkill for simpler trading applications
- Less focused on financial trading compared to TensorTrade
Code Comparison
Ray example:
import ray
@ray.remote
def f(x):
return x * x
futures = [f.remote(i) for i in range(4)]
print(ray.get(futures))
TensorTrade example:
from tensortrade.env import TradingEnvironment
from tensortrade.feed import DataFeed
from tensortrade.oms import OrderManagementSystem
env = TradingEnvironment(
feed=DataFeed(),
oms=OrderManagementSystem()
)
Ray is a distributed computing framework that can be used for various applications, including machine learning and AI. TensorTrade is specifically designed for building and training trading agents. While Ray offers more flexibility and scalability, TensorTrade provides a more focused approach for financial trading applications. The code examples demonstrate Ray's distributed computing capabilities and TensorTrade's trading-specific environment setup.
A fork of OpenAI Baselines, implementations of reinforcement learning algorithms
Pros of Stable-baselines
- Broader scope, supporting various reinforcement learning algorithms and environments
- More mature project with extensive documentation and examples
- Larger community and more frequent updates
Cons of Stable-baselines
- Not specifically tailored for trading environments
- May require more setup and customization for trading-specific tasks
- Steeper learning curve for users focused solely on trading applications
Code Comparison
TensorTrade example:
from tensortrade.env import TradingEnvironment
from tensortrade.features import TAIndicator
from tensortrade.rewards import RiskAdjustedReturns
env = TradingEnvironment(
instruments=['BTC-USD'],
features=[TAIndicator('close', 'rsi')],
reward_scheme=RiskAdjustedReturns()
)
Stable-baselines example:
from stable_baselines3 import PPO
from stable_baselines3.common.vec_env import DummyVecEnv
from custom_trading_env import TradingEnvironment
env = DummyVecEnv([lambda: TradingEnvironment()])
model = PPO("MlpPolicy", env, verbose=1)
model.learn(total_timesteps=10000)
The code comparison shows that TensorTrade provides a more streamlined setup for trading environments, while Stable-baselines requires custom environment creation but offers more flexibility in algorithm choice and training process.
A standard API for single-agent reinforcement learning environments, with popular reference environments and related utilities (formerly Gym)
Pros of Gymnasium
- Broader scope, supporting a wide range of reinforcement learning environments
- More active development and larger community support
- Better documentation and examples for various use cases
Cons of Gymnasium
- Not specifically tailored for financial trading environments
- May require additional customization for trading-specific tasks
- Steeper learning curve for users focused solely on trading applications
Code Comparison
TensorTrade:
from tensortrade.env import TradingEnvironment
from tensortrade.feed import DataFeed
from tensortrade.oms import OrderManagementSystem
env = TradingEnvironment(
feed=DataFeed(),
oms=OrderManagementSystem()
)
Gymnasium:
import gymnasium as gym
env = gym.make('CartPole-v1')
observation, info = env.reset(seed=42)
for _ in range(1000):
action = env.action_space.sample()
observation, reward, terminated, truncated, info = env.step(action)
TensorTrade is more focused on trading environments, while Gymnasium provides a general-purpose framework for reinforcement learning tasks. TensorTrade offers built-in components for trading systems, whereas Gymnasium requires custom implementation for trading-specific environments.
Lean Algorithmic Trading Engine by QuantConnect (Python, C#)
Pros of Lean
- More mature and actively maintained project with a larger community
- Supports multiple asset classes (stocks, forex, crypto, options)
- Provides a comprehensive backtesting and live trading framework
Cons of Lean
- Steeper learning curve due to its complexity
- Requires C# knowledge for advanced customization
- Less focused on reinforcement learning compared to TensorTrade
Code Comparison
Lean (C#):
public class MyAlgorithm : QCAlgorithm
{
public override void Initialize()
{
SetStartDate(2020, 1, 1);
SetCash(100000);
AddEquity("AAPL");
}
}
TensorTrade (Python):
from tensortrade.env import TradingEnvironment
env = TradingEnvironment(
instruments=['AAPL'],
start_date='2020-01-01',
base_instrument='USD'
)
Both repositories aim to provide algorithmic trading frameworks, but they differ in their approach and target audience. Lean offers a more comprehensive solution for professional traders and quants, supporting multiple asset classes and providing a robust backtesting engine. TensorTrade, on the other hand, focuses on reinforcement learning applications in trading, making it more accessible for researchers and AI enthusiasts. Lean's maturity and active community support give it an edge in terms of stability and features, while TensorTrade's simplicity and focus on machine learning make it attractive for those exploring AI-driven trading strategies.
Convert 
designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual CopilotREADME
TensorTrade
Train RL agents to trade. Can they beat Buy-and-Hold?
TensorTrade is an open-source Python framework for building, training, and evaluating reinforcement learning agents for algorithmic trading. The framework provides composable components for environments, action schemes, reward functions, and data feeds that can be combined to create custom trading systems.
Quick Start
# Requires Python 3.12+
python3.12 -m venv tensortrade-env && source tensortrade-env/bin/activate
pip install -e .
# For training with Ray/RLlib (recommended)
pip install -r examples/requirements.txt
# Run training
python examples/training/train_simple.py
Documentation & Tutorials
ð Tutorial Index â Start here for the complete learning curriculum.
Foundations
- The Three Pillars â RL + Trading + Data concepts
- Architecture â How components work together
- Your First Run â Run and understand output
Domain Knowledge
- Trading for RL Practitioners
- RL for Traders
- Common Failures â Critical pitfalls to avoid
- Full Introduction â New to both domains
Core Components
- Action Schemes â BSH and order execution
- Reward Schemes â Why PBR works
- Observers & Feeds â Feature engineering
Training
- First Training â Train with Ray RLlib
- Ray RLlib Deep Dive â Configuration options
- Optuna Optimization â Hyperparameter tuning
Advanced Topics
- Overfitting â Detection and prevention
- Commission Analysis â Key research findings
- Walk-Forward Validation â Proper evaluation
Additional Resources
- Experiments Log â Full research documentation
- Environment Setup â Detailed installation guide
- API Reference
Research Findings
We conducted extensive experiments training PPO agents on BTC/USD. Key results:
| Configuration | Test P&L | vs Buy-and-Hold |
|---|---|---|
| Agent (0% commission) | +$239 | +$594 |
| Agent (0.1% commission) | -$650 | -$295 |
| Buy-and-Hold | -$355 | â |
The agent demonstrates directional prediction capability at zero commission. The primary challenge is trading frequencyâcommission costs currently exceed prediction profits. See EXPERIMENTS.md for methodology and detailed analysis.
Architecture
âââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââ
â TradingEnv â
â â
â Observer ââââââ> Agent ââââââ> ActionScheme ââââââ> Portfolio â
â (features) (policy) (BSH/Orders) (wallets) â
â ^ â â
â âââââââââââââ RewardScheme <ââââââââââââââââââââââââ â
â (PBR) â
â â
â DataFeed ââââââ> Exchange ââââââ> Broker ââââââ> Trades â
âââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââââ
| Component | Purpose | Default |
|---|---|---|
| ActionScheme | Converts agent output to orders | BSH (Buy/Sell/Hold) |
| RewardScheme | Computes learning signal | PBR (Position-Based Returns) |
| Observer | Generates observations | Windowed features |
| Portfolio | Manages wallets and positions | USD + BTC |
| Exchange | Simulates execution | Configurable commission |
Training Scripts
| Script | Description |
|---|---|
examples/training/train_simple.py | Basic demo with wallet tracking |
examples/training/train_ray_long.py | Distributed training with Ray RLlib |
examples/training/train_optuna.py | Hyperparameter optimization |
examples/training/train_best.py | Best configuration from experiments |
Installation
Requirements: Python 3.11 or 3.12
# Create environment
python3.12 -m venv tensortrade-env
source tensortrade-env/bin/activate # Windows: tensortrade-env\Scripts\activate
# Install
pip install --upgrade pip
pip install -r requirements.txt
pip install -e .
# Verify
pytest tests/tensortrade/unit -v
# Training dependencies (optional)
pip install -r examples/requirements.txt
See ENVIRONMENT_SETUP.md for platform-specific instructions and troubleshooting.
Docker
make run-notebook # Jupyter
make run-docs # Documentation
make run-tests # Test suite
Project Structure
tensortrade/
âââ tensortrade/ # Core library
â âââ env/ # Trading environments
â âââ feed/ # Data pipeline
â âââ oms/ # Order management
â âââ data/ # Data fetching
âââ examples/
â âââ training/ # Training scripts
â âââ notebooks/ # Jupyter tutorials
âââ docs/
â âââ tutorials/ # Learning curriculum
â âââ EXPERIMENTS.md # Research log
âââ tests/
Troubleshooting
| Issue | Solution |
|---|---|
| "No stream satisfies selector" | Update to v1.0.4-dev1+ |
| Ray installation fails | Run pip install --upgrade pip first |
| NumPy version conflict | pip install "numpy>=1.26.4,<2.0" |
| TensorFlow CUDA issues | pip install tensorflow[and-cuda]>=2.15.1 |
Contributing
See CONTRIBUTING.md for guidelines.
Priority areas:
- Trading frequency reduction (position sizing, holding periods)
- Commission-aware reward schemes
- Alternative action spaces
Community
License
Top Related Projects
A toolkit for developing and comparing reinforcement learning algorithms.
Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
A fork of OpenAI Baselines, implementations of reinforcement learning algorithms
A standard API for single-agent reinforcement learning environments, with popular reference environments and related utilities (formerly Gym)
Lean Algorithmic Trading Engine by QuantConnect (Python, C#)
Convert designs to code with AI
Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.
Try Visual Copilot