Convert Figma logo to code with AI

AI4Finance-Foundation logoFinRL

FinRLĀ®: Financial Reinforcement Learning. šŸ”„

15,109
3,331
15,109
302

Top Related Projects

46,477

Qlib is an AI-oriented Quant investment platform that aims to use AI tech to empower Quant Research, from exploring ideas to implementing productions. Qlib supports diverse ML modeling paradigms, including supervised learning, market dynamics modeling, and RL, and is now equipped with https://github.com/microsoft/RD-Agent to automate R&D process.

An open source reinforcement learning framework for training, evaluating, and deploying robust trading agents.

Python Backtesting library for trading strategies

Python library for backtesting trading strategies & analyzing financial markets (formerly pythalesians)

MlFinLab helps portfolio managers and traders who want to leverage the power of machine learning by providing reproducible, interpretable, and easy to use tools.

20,638

Lean Algorithmic Trading Engine by QuantConnect (Python, C#)

Quick Overview

FinRL is an open-source library for financial reinforcement learning. It provides a unified framework for various financial tasks, including portfolio management, algorithmic trading, and risk management. The library aims to bridge the gap between reinforcement learning and quantitative finance.

Pros

  • Comprehensive framework for applying RL to finance
  • Supports multiple market environments and RL algorithms
  • Includes pre-built datasets and data processing tools
  • Active community and regular updates

Cons

  • Steep learning curve for those new to RL or finance
  • Limited documentation for some advanced features
  • Performance can be slow for large-scale simulations
  • Requires careful hyperparameter tuning for optimal results

Code Examples

  1. Creating an environment:
from finrl.apps import stocks_trading
from finrl.config import TRAINED_MODEL_DIR, RESULTS_DIR

env = stocks_trading.StockTradingEnv(df=processed_df, 
                                     stock_dim=len(stock_list),
                                     hmax=100,
                                     initial_amount=1000000,
                                     transaction_cost_pct=0.001)
  1. Training an agent:
from finrl.agents.stablebaselines3 import DRLAgent

agent = DRLAgent(env=env)
model = agent.get_model("ppo")
trained_model = agent.train_model(model=model, 
                                  tb_log_name='ppo',
                                  total_timesteps=50000)
  1. Backtesting:
from finrl.plot import backtest_stats, backtest_plot

df_account_value, df_actions = DRLAgent.DRL_prediction(
    model=trained_model,
    environment=env)

perf_stats_all = backtest_stats(account_value=df_account_value)
backtest_plot(df_account_value, 
              baseline_ticker='SPY', 
              baseline_start=df_account_value.index[0], 
              baseline_end=df_account_value.index[-1])

Getting Started

  1. Install FinRL:
pip install finrl
  1. Import necessary modules:
from finrl.apps import stocks_trading
from finrl.agents.stablebaselines3 import DRLAgent
from finrl.config import TRAINED_MODEL_DIR, RESULTS_DIR
import pandas as pd
  1. Prepare data and create environment:
# Load and preprocess your data
df = pd.read_csv('your_data.csv')
processed_df = stocks_trading.preprocess_data(df)

# Create environment
env = stocks_trading.StockTradingEnv(df=processed_df, 
                                     stock_dim=len(stock_list),
                                     hmax=100,
                                     initial_amount=1000000,
                                     transaction_cost_pct=0.001)
  1. Train and evaluate a model:
agent = DRLAgent(env=env)
model = agent.get_model("ppo")
trained_model = agent.train_model(model=model, 
                                  tb_log_name='ppo',
                                  total_timesteps=50000)

# Backtest the trained model
df_account_value, df_actions = DRLAgent.DRL_prediction(
    model=trained_model,
    environment=env)

Competitor Comparisons

46,477

Qlib is an AI-oriented Quant investment platform that aims to use AI tech to empower Quant Research, from exploring ideas to implementing productions. Qlib supports diverse ML modeling paradigms, including supervised learning, market dynamics modeling, and RL, and is now equipped with https://github.com/microsoft/RD-Agent to automate R&D process.

Pros of qlib

  • More comprehensive and feature-rich, offering a wider range of tools and models
  • Better documentation and extensive examples for various use cases
  • Stronger industry backing and support from Microsoft

Cons of qlib

  • Steeper learning curve due to its complexity and extensive features
  • Less focus on reinforcement learning compared to FinRL
  • May be overkill for simpler financial modeling tasks

Code Comparison

FinRL example:

from finrl.apps import config
from finrl.finrl_meta.preprocessor.yahoodownloader import YahooDownloader
from finrl.finrl_meta.preprocessor.preprocessors import FeatureEngineer

df = YahooDownloader(start_date="2009-01-01", 
                     end_date="2021-01-01", 
                     ticker_list=config.DOW_30_TICKER).fetch_data()

qlib example:

import qlib
from qlib.data import D
from qlib.config import REG_CN

qlib.init(provider_uri='~/.qlib/qlib_data/cn_data', region=REG_CN)
instruments = ['SH000300']
fields = ['$close', '$volume', '$factor', '$paused']
df = D.features(instruments, fields, start_time='2010-01-01', end_time='2017-12-31')

Both libraries provide data fetching and preprocessing capabilities, but qlib offers more flexibility in data sources and feature engineering.

An open source reinforcement learning framework for training, evaluating, and deploying robust trading agents.

Pros of TensorTrade

  • More flexible and customizable trading environment
  • Supports a wider range of financial instruments and markets
  • Better documentation and examples for getting started

Cons of TensorTrade

  • Less focus on deep reinforcement learning algorithms
  • Smaller community and fewer contributions
  • Limited built-in data sources and preprocessing tools

Code Comparison

FinRL example:

from finrl.agents.stablebaselines3.models import DRLAgent
from finrl.finrl_meta.preprocessor.yahoodownloader import YahooDownloader

df = YahooDownloader(start_date="2009-01-01", end_date="2021-01-01", ticker_list=["AAPL"]).fetch_data()
agent = DRLAgent(env=env)
model = agent.get_model("ppo")

TensorTrade example:

from tensortrade.env import default
from tensortrade.feed.core import Stream, DataFeed
from tensortrade.oms.instruments import USD, BTC

env = default.create(
    instrument=USD/BTC,
    features=DataFeed([Stream.source(exchange.streams["USD-BTC"], dtype="float")]),
    action_scheme="managed-risk",
    reward_scheme="risk-adjusted"
)

Both libraries provide tools for reinforcement learning in financial markets, but TensorTrade offers more flexibility in environment creation and supports a broader range of financial instruments. FinRL, on the other hand, focuses more on deep reinforcement learning algorithms and provides better integration with popular data sources.

Python Backtesting library for trading strategies

Pros of backtrader

  • More mature and established project with a larger community
  • Comprehensive documentation and extensive examples
  • Flexible and customizable framework for backtesting trading strategies

Cons of backtrader

  • Primarily focused on backtesting, with limited support for live trading
  • Steeper learning curve for beginners due to its extensive features
  • Less emphasis on machine learning and AI-driven strategies

Code Comparison

backtrader:

class MyStrategy(bt.Strategy):
    def __init__(self):
        self.sma = bt.indicators.SimpleMovingAverage(self.data.close, period=20)

    def next(self):
        if self.data.close[0] > self.sma[0]:
            self.buy()
        elif self.data.close[0] < self.sma[0]:
            self.sell()

FinRL:

class DRLAgent(object):
    def __init__(self, env, price_array, tech_array, turbulence_array):
        self.env = env
        self.price_array = price_array
        self.tech_array = tech_array
        self.turbulence_array = turbulence_array

    def train(self):
        # Training logic here

Python library for backtesting trading strategies & analyzing financial markets (formerly pythalesians)

Pros of finmarketpy

  • More comprehensive financial market analysis tools, including backtesting and market data handling
  • Better documentation and examples for various financial market scenarios
  • Wider range of financial instruments and asset classes supported

Cons of finmarketpy

  • Less focus on reinforcement learning for financial applications
  • Not as actively maintained or updated as FinRL
  • Steeper learning curve for users new to financial market analysis

Code Comparison

FinRL example (portfolio optimization):

env = StockPortfolioEnv(df=df, 
                        stock_dim=stock_dimension,
                        hmax=100, 
                        initial_amount=1000000,
                        transaction_cost_pct=0.001)

model = PPO("MlpPolicy", env, verbose=1)
model.learn(total_timesteps=50000)

finmarketpy example (backtesting):

backtest = Backtest(market_data, 
                    trading_model,
                    parameters,
                    date_start=start_date,
                    date_end=end_date)

backtest.run_backtest()
backtest.plot_results()

MlFinLab helps portfolio managers and traders who want to leverage the power of machine learning by providing reproducible, interpretable, and easy to use tools.

Pros of mlfinlab

  • More comprehensive library for financial machine learning tasks
  • Implements advanced techniques from academic research papers
  • Larger community and more frequent updates

Cons of mlfinlab

  • Steeper learning curve due to more complex implementations
  • Less focus on reinforcement learning for trading strategies
  • Requires more domain knowledge to use effectively

Code Comparison

mlfinlab:

from mlfinlab.data_structures import imbalance_data_structures as ds

tick_data = ds.get_tick_data(file_path, num_prev_ticks=10)
dollar_bars = ds.get_dollar_bars(tick_data, threshold=70000000)

FinRL:

from finrl.meta.preprocessor.yahoodownloader import YahooDownloader
from finrl.meta.env_stock_trading.env_stocktrading import StockTradingEnv

df = YahooDownloader(start_date, end_date, ticker_list).fetch_data()
env = StockTradingEnv(df, initial_amount=100000, hmax=1000, transaction_cost_pct=0.001)

Both libraries offer valuable tools for financial machine learning, but mlfinlab focuses more on data structures and advanced techniques, while FinRL emphasizes reinforcement learning for trading strategies. The choice between them depends on the specific requirements of your project and your level of expertise in financial machine learning.

20,638

Lean Algorithmic Trading Engine by QuantConnect (Python, C#)

Pros of Lean

  • More comprehensive and production-ready algorithmic trading platform
  • Larger community and extensive documentation
  • Supports multiple asset classes and brokers

Cons of Lean

  • Steeper learning curve for beginners
  • Less focus on reinforcement learning techniques
  • Requires more setup and configuration

Code Comparison

FinRL example (reinforcement learning approach):

env = StockTradingEnv(df=df, ...)
model = PPO("MlpPolicy", env, ...)
model.learn(total_timesteps=50000)

Lean example (traditional algorithmic approach):

public class MyAlgorithm : QCAlgorithm
{
    public override void Initialize()
    {
        SetStartDate(2020, 1, 1);
        SetCash(100000);
        AddEquity("SPY", Resolution.Daily);
    }
}

FinRL focuses on applying reinforcement learning to financial trading, making it easier to implement RL-based strategies. Lean, on the other hand, provides a more traditional algorithmic trading framework with a wider range of features and asset classes. FinRL is more suitable for researchers and those interested in RL applications in finance, while Lean is better suited for production-ready algorithmic trading systems.

Convert Figma logo designs to code with AI

Visual Copilot

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

image

FinRL: Financial Reinforcement Learning → FinRL-X

Downloads Downloads Join Discord Python 3.6 PyPI Documentation Status License X LinkedIn

[!IMPORTANT] FinRL-X is the next-generation evolution of FinRL, designed for AI-native, modular, and production-oriented quantitative trading.

  • This repository (FinRL) preserves the original end-to-end educational and research framework.
  • For the latest architecture, live trading deployment, and production-focused development, please use FinRL-X / FinRL-Trading.

FinRL® is widely recognized as the first open-source framework for financial reinforcement learning. This repository contains the original FinRL library for education, benchmarking, and research prototyping.

For the next-generation AI-native and production-oriented trading stack, please visit FinRL-X / FinRL-Trading.

FinRL Ecosystem Roadmap

GenerationPositioningTarget UsersRepositoryDescription
FinRL-MetaMarket EnvironmentsPractitionersFinRL-MetaGym-style financial market environments and benchmarks
FinRLClassic End-to-End FrameworkLearners, Developers, ResearchersFinRLOriginal train-test-trade pipeline for financial reinforcement learning
ElegantRLAlgorithm LayerResearchers and ExpertsElegantRLLightweight and elegant DRL algorithms
FinRL-XNext Generation / ProductionProfessional traders, institutions, hedge fundsFinRL-TradingAI-native modular infrastructure for deployment-aware quantitative trading

Recommended for new users: Start with FinRL-X / FinRL-Trading if you are building modern or production-oriented trading systems.

Ć°ĀŸĀ”Ā„ FinRL-X vs. FinRL: What Changed

CapabilityFinRL (Stage 1.0)FinRL-X (Stage 3.0)
ParadigmDeep Reinforcement LearningAI-Native (ML + DRL + LLM-ready)
ArchitectureThree-layer coupled monolithFully decoupled modular layers
StrategiesDRL agents (A2C, DDPG, PPO, SAC, TD3)ML selection + DRL timing + extensible base
Data Layer14 manually-wired processorsAuto-select: Yahoo Finance → FMP → WRDS
BacktestingCustom hand-rolled evaluation loopsProfessional bt library engine
Live TradingBasic Alpaca supportFull multi-account integration + risk controls
Configurationconfig.py + config_tickers.pyType-safe Pydantic + .env multi-env
Risk ManagementGym environment constraints onlyOrder · portfolio · strategy-level controls
Target UsersResearchers & studentsQuants, institutions, production deployments
PaperarXiv:2011.09607arXiv:2603.21330

FinGPT: an open-source project for financial large language models, designed for research and real-world FinTech applications.

Visitors Discord

Outline

Project Contributors

FinRL® is an open-source financial reinforcement learning framework developed by contributors from the AI4Finance community and maintained by the AI4Finance Foundation.

Key contributors include:

  • Hongyang (Bruce) Yang – research and development on financial reinforcement learning frameworks, market environments, and quantitative trading applications
  • [other contributors…]

Overview

FinRL is the original open-source framework for financial reinforcement learning, organized around three core layers:

  • Market Environments
  • DRL Agents
  • Financial Applications

For a trading task, an agent interacts with a market environment and learns sequential decision-making policies.

This repository focuses on the classic FinRL workflow for education, experimentation, and research prototyping.

For the next-generation production-oriented stack, including modular deployment and AI-native trading infrastructure, please visit FinRL-X / FinRL-Trading.

Videos FinRL at AI4Finance Youtube Channel.

FinRL Stock Trading 2026 Tutorial

This tutorial demonstrates the original FinRL workflow for educational and research purposes. For the latest production-oriented pipeline, please use FinRL-X / FinRL-Trading.

Step 1: Clone the Repository

git clone https://github.com/AI4Finance-Foundation/FinRL.git
cd FinRL

Step 2: Create and Activate Virtual Environment

python3 -m venv venv
source venv/bin/activate

Step 3: Install FinRL

pip install -e .

Step 4: Run the Scripts

1. Data Download & Preprocessing

python examples/FinRL_StockTrading_2026_1_data.py

This script downloads DOW 30 stock data from Yahoo Finance, adds technical indicators (MACD, RSI, etc.), VIX, and turbulence index, then splits the data into training set (2014–2025) and trading set (2026-01-01 to 2026-03-20), saving them as train_data.csv and trade_data.csv.

2. Train DRL Agents

python examples/FinRL_StockTrading_2026_2_train.py

This script trains 5 DRL agents (A2C, DDPG, PPO, TD3, SAC) using Stable Baselines 3 on the training data. Trained models are saved to the trained_models/ directory.

3. Backtest

python examples/FinRL_StockTrading_2026_3_Backtest.py

This script loads the trained agents, runs them on the trading data, and compares their performance against two baselines: Mean Variance Optimization (MVO) and the DJIA index. Results are printed to the console and a plot is saved as backtest_result.png.

File Structure

The main folder finrl has three subfolders applications, agents, meta. We employ a train-test-trade pipeline with three files: train.py, test.py, and trade.py.

FinRL
Ć¢Ā”ĀœĆ¢Ā”Ā€Ć¢Ā”Ā€ finrl (main folder)
│   Ć¢Ā”ĀœĆ¢Ā”Ā€Ć¢Ā”Ā€ applications
│   	Ć¢Ā”ĀœĆ¢Ā”Ā€Ć¢Ā”Ā€ Stock_NeurIPS2018
│   	Ć¢Ā”ĀœĆ¢Ā”Ā€Ć¢Ā”Ā€ imitation_learning
│   	Ć¢Ā”ĀœĆ¢Ā”Ā€Ć¢Ā”Ā€ cryptocurrency_trading
│   	Ć¢Ā”ĀœĆ¢Ā”Ā€Ć¢Ā”Ā€ high_frequency_trading
│   	Ć¢Ā”ĀœĆ¢Ā”Ā€Ć¢Ā”Ā€ portfolio_allocation
│   	└── stock_trading
│   Ć¢Ā”ĀœĆ¢Ā”Ā€Ć¢Ā”Ā€ agents
│   	Ć¢Ā”ĀœĆ¢Ā”Ā€Ć¢Ā”Ā€ elegantrl
│   	Ć¢Ā”ĀœĆ¢Ā”Ā€Ć¢Ā”Ā€ rllib
│   	└── stablebaseline3
│   Ć¢Ā”ĀœĆ¢Ā”Ā€Ć¢Ā”Ā€ meta
│   	Ć¢Ā”ĀœĆ¢Ā”Ā€Ć¢Ā”Ā€ data_processors
│   	Ć¢Ā”ĀœĆ¢Ā”Ā€Ć¢Ā”Ā€ env_cryptocurrency_trading
│   	Ć¢Ā”ĀœĆ¢Ā”Ā€Ć¢Ā”Ā€ env_portfolio_allocation
│   	Ć¢Ā”ĀœĆ¢Ā”Ā€Ć¢Ā”Ā€ env_stock_trading
│   	Ć¢Ā”ĀœĆ¢Ā”Ā€Ć¢Ā”Ā€ preprocessor
│   	Ć¢Ā”ĀœĆ¢Ā”Ā€Ć¢Ā”Ā€ data_processor.py
│       Ć¢Ā”ĀœĆ¢Ā”Ā€Ć¢Ā”Ā€ meta_config_tickers.py
│   	└── meta_config.py
│   Ć¢Ā”ĀœĆ¢Ā”Ā€Ć¢Ā”Ā€ config.py
│   Ć¢Ā”ĀœĆ¢Ā”Ā€Ć¢Ā”Ā€ config_tickers.py
│   Ć¢Ā”ĀœĆ¢Ā”Ā€Ć¢Ā”Ā€ main.py
│   Ć¢Ā”ĀœĆ¢Ā”Ā€Ć¢Ā”Ā€ plot.py
│   Ć¢Ā”ĀœĆ¢Ā”Ā€Ć¢Ā”Ā€ train.py
│   Ć¢Ā”ĀœĆ¢Ā”Ā€Ć¢Ā”Ā€ test.py
│   └── trade.py
│
Ć¢Ā”ĀœĆ¢Ā”Ā€Ć¢Ā”Ā€ examples
Ć¢Ā”ĀœĆ¢Ā”Ā€Ć¢Ā”Ā€ unit_tests (unit tests to verify codes on env & data)
│   Ć¢Ā”ĀœĆ¢Ā”Ā€Ć¢Ā”Ā€ environments
│   	└── test_env_cashpenalty.py
│   └── downloaders
│   	Ć¢Ā”ĀœĆ¢Ā”Ā€Ć¢Ā”Ā€ test_yahoodownload.py
│   	└── test_alpaca_downloader.py
Ć¢Ā”ĀœĆ¢Ā”Ā€Ć¢Ā”Ā€ setup.py
Ć¢Ā”ĀœĆ¢Ā”Ā€Ć¢Ā”Ā€ requirements.txt
└── README.md

Supported Data Sources

Data SourceTypeRange and FrequencyRequest LimitsRaw DataPreprocessed Data
AkshareCN Securities2015-now, 1dayAccount-specificOHLCVPrices&Indicators
AlpacaUS Stocks, ETFs2015-now, 1minAccount-specificOHLCVPrices&Indicators
BaostockCN Securities1990-12-19-now, 5minAccount-specificOHLCVPrices&Indicators
BinanceCryptocurrencyAPI-specific, 1s, 1minAPI-specificTick-level daily aggregated trades, OHLCVPrices&Indicators
CCXTCryptocurrencyAPI-specific, 1minAPI-specificOHLCVPrices&Indicators
EODhistoricaldataUS SecuritiesFrequency-specific, 1minAPI-specificOHLCVPrices&Indicators
IEXCloudNMS US securities1970-now, 1 day100 per second per IPOHLCVPrices&Indicators
JoinQuantCN Securities2005-now, 1min3 requests each timeOHLCVPrices&Indicators
QuantConnectUS Securities1998-now, 1sNAOHLCVPrices&Indicators
RiceQuantCN Securities2005-now, 1msAccount-specificOHLCVPrices&Indicators
SinopacTaiwan securities2023-04-13~now, 1minAccount-specificOHLCVPrices&Indicators
TushareCN Securities, A-share-now, 1 minAccount-specificOHLCVPrices&Indicators
WRDSUS Securities2003-now, 1ms5 requests each timeIntraday TradesPrices&Indicators
YahooFinanceUS SecuritiesFrequency-specific, 1min2,000/hourOHLCVPrices&Indicators

OHLCV: open, high, low, and close prices; volume. adjusted_close: adjusted close price

Technical indicators: 'macd', 'boll_ub', 'boll_lb', 'rsi_30', 'dx_30', 'close_30_sma', 'close_60_sma'. Users also can add new features.

Installation

Status Update

Version History [click to expand]
  • 2022-06-25 0.3.5: Formal release of FinRL, neo_finrl is changed to FinRL-Meta with related files in directory: meta.
  • 2021-08-25 0.3.1: pytorch version with a three-layer architecture, apps (financial tasks), drl_agents (drl algorithms), neo_finrl (gym env)
  • 2020-12-14 Upgraded to Pytorch with stable-baselines3; Removed TensorFlow 1.x support; TensorFlow 2.0 support was under development at the time.
  • 2020-11-27 0.1: Beta version with tensorflow 1.5

Tutorials

Publications

TitleConference/JournalLinkCitationsYear
Dynamic Datasets and Market Environments for Financial Reinforcement LearningMachine Learning - Springer Naturepaper code512024
FinRL-Meta: FinRL-Meta: Market Environments and Benchmarks for Data-Driven Financial Reinforcement LearningNeurIPS 2022paper code1362022
FinRL: Deep reinforcement learning framework to automate trading in quantitative financeACM International Conference on AI in Finance (ICAIF)paper2122021
FinRL: A deep reinforcement learning library for automated stock trading in quantitative financeNeurIPS 2020 Deep RL Workshoppaper2752020
Deep reinforcement learning for automated stock trading: An ensemble strategyACM International Conference on AI in Finance (ICAIF)paper code4262020
Practical deep reinforcement learning approach for stock tradingNeurIPS 2018 Workshop on Challenges and Opportunities for AI in Financial Servicespaper code3032018

News

Citing FinRL

For the next-generation AI-native modular trading infrastructure, see FinRL-X / FinRL-Trading.

@inproceedings{yang2026finrlx,
  title     = {FinRL-X: An AI-Native Modular Infrastructure for Quantitative Trading},
  author    = {Yang, Hongyang and Zhang, Boyu and She, Yang and Liao, Xinyu and Zhang, Xiaoli},
  booktitle = {Proceedings of the 2nd International Workshop on Decision Making and Optimization in Financial Technologies (DMO-FinTech)},
  year      = {2026},
  note      = {Workshop at PAKDD 2026}
}

If you use the original FinRL framework, please cite the FinRL papers:

@article{finrl2020,
    author  = {Liu, Xiao-Yang and Yang, Hongyang and Chen, Qian and Zhang, Runjia and Yang, Liuqing and Xiao, Bowen and Wang, Christina Dan},
    title   = {{FinRL}: A deep reinforcement learning library for automated stock trading in quantitative finance},
    journal = {Deep RL Workshop, NeurIPS 2020},
    year    = {2020}
}

Join and Contribute

Welcome to AI4Finance community!

Please check Contributing Guidelines.

Contributors

Thanks to all contributors who have helped build FinRL.

FinRL contributors

LICENSE

MIT License

Trademark Notice

FinRL and the FinRL logo are trademarks of FinRL LLC. Use of these marks by the AI4Finance Foundation is permitted under license. The open-source license for this repository does not grant any right to use the FinRL name, logo, or related trademarks without prior written permission from FinRL LLC, except as permitted by applicable law.

Disclaimer: We are sharing codes for academic purposes under the MIT license. Nothing herein constitutes financial advice or a recommendation to trade real money. Users are solely responsible for any financial decisions made using this software. Consult a qualified professional before deploying capital.