Five Years After His Trading-AI Paper, a Bay Area Engineer Put It on Trial

SAN FRANCISCO, Oct. 28, 2026 — In 2021, Bicheng (Kenneth) Wang and co-author Xinyi Zhang wrote a paper with a striking result. A deep reinforcement learning agent called DDPG, trained to trade 20 large US stocks, earned a 55.5 percent annual return over 2019 and 2020, with a Sharpe ratio of 1.38. Two others, TD3 and SAC, reported Sharpe ratios of 1.37 and 1.33.

Results like that are the currency of financial AI. They appear in papers, slide decks and social media posts.

This fall, Wang gave his own result a second look, in public. He posted the paper, originally titled "Applying Deep Learning to Stock Trading: LSTM Forecasting and Deep Reinforcement Learning," to arXiv on Sept. 25, 2026 (arxiv.org/abs/2609.31870) and made a re-test public in the same repository, under stricter rules. It is still a draft awaiting his co-author's reply. The verdict on the headline number was blunt: it does not hold.

A lab for claims, not a strategy for sale

The repository is called RL-pricing, and it describes itself as a reproducibility lab for trading AI. Its purpose is to take published claims about deep reinforcement learning traders, large language model trading agents and price forecasters, and test them all under one pre-registered protocol. Each verdict is meant to run beside the authors' reply, or a note that none arrived.

"The leaderboard and the live track are the instruments that produce the evidence; the reports are the product," the lab's documentation states.

RL-pricing does not sell a trading strategy; its code and evidence are free under the MIT license. Since Oct. 23, 2026, the lab has been installable from PyPI as the rl-pricing package, and as of Oct. 28, 2026, the repository had 267 stars and 22 forks on GitHub. It publishes evidence about whether other people's strategies, and Wang's own, do what their authors said.

How a claim goes on trial

Every case in the lab follows the same steps, and the order matters.

First, the claim is written down verbatim, with the table and page it came from, along with the rule that will decide it. For new cases, that plan is committed to the repository before the first experiment runs, so the original branch commit's timestamp shows the test was designed before the results were known. R0's plan was registered after the fact.

Second, the claim is tested at up to three levels. Level one replicates the result with the authors' own code and setup. Level two re-tests it under the lab's own engine: trading signals computed at the market close, orders filled at the next day's open, trading fees charged, at least five random seeds, and data free of survivorship bias, or a universe clearly labeled as biased. Level three tries new time windows, new sets of stocks, and for anything that can trade daily, a live ledger.

Third, a verdict is assigned by rule, not by opinion. A claim holds if the claimed value lies inside the lab's 95 percent interval and, for performance claims, beats the stated baseline with a bootstrap p-value below 0.05. It partly holds if the direction is right but the size shrinks by more than half or significance is lost. It does not hold if the direction is opposite or no seed reaches the claimed value. It is inconclusive if the test cannot detect the effect, and the report says what would decide it.

Fourth, the authors receive the draft seven days before publication, and their reply is printed unedited beside the report. The language rules are strict. "Wording stays neutral," the documentation says. A claim "does not hold under X"; the lab does "not say anyone was wrong."

The defendant: his own paper

The lab's first report, labeled R0, is the re-test of Wang and Zhang's 2021 paper, now on arXiv as Deep Reinforcement Learning for Equity Trading: Benchmarking Actor-Critic Methods with Forward Retraining (arXiv:2609.31870).

The report opens with two bold words: "Read first." It then states that the reported 55.51 percent annual return for DDPG does not hold, and that no run of any agent in any setup reaches it. Across 75 runs, the best was 46.0 percent.

Wang tested the paper three ways, with five seeds for each of five agents on the same 20 stocks and the same 2019 to 2020 window, charging 0.1 percent on every buy and sell.

The first setup used the paper's own trading environment, upgraded with realistic next-day fills, and kept its built-in safety rule, which sells everything when a market turbulence index spikes. The second setup was identical but switched the safety rule off. The third used the lab's own benchmark task, in which agents choose portfolio weights and have no hand-coded exits.

For comparison, the report used the simplest strategy there is. Buy all 20 stocks in equal amounts and hold them, known as 1/N. Over the same two years, 1/N earned 29.7 percent a year with a Sharpe ratio of 1.10.

What the evidence showed

The findings, published with every per-seed result, read like a careful audit of a younger researcher's work.

The stock list did most of the work. The 20 stocks were chosen in 2021 from the largest US companies, a selection that looks back on companies already known to have done well, and 2019 to 2020 was a strong market. The median of the 50 runs in the paper's own environment earned 26.1 percent a year, below simply holding the same stocks. The lab attributes much of the headline to the hindsight-chosen universe and the bull market.

The reported Sharpe ratios looked like best-case seeds. For DDPG, TD3 and SAC, the reported Sharpe ratios of 1.33 to 1.38 sat near the best of five seeds, not the average. In the main setup, all five Sharpe claims were rated "partly holds."

The safety rule was doing the trading. With the turbulence rule on, all 25 runs moved to cash on the same five trading sessions of March 2020, the weeks of the pandemic crash. That move was made by the environment, not by anything the agents learned. With the rule off, every agent except one fell below 1/N on average.

One agent survived, but not convincingly. The SAC agent beat 1/N with an alpha of about 7.5 percent a year. Over a window of only two years, that was not statistically significant, with a best bootstrap p-value of 0.133.

The benchmark reversed the picture. In the lab's portfolio-weights environment, the same agents trailed 1/N. The report leaves the cause open: share counts versus weights, the input features, or the original environment's per-trade limits.

The report is just as open about its own shortcuts. It states that its pre-registration for R0 was written after the fact, that no level-one replication was run because the original paper did not specify fill timing or fees, and that an equal-weight portfolio replaced the paper's S&P 500 baseline because no index series was in the evidence. R0 ships per-seed results, hashes of the input data and one command that recomputes its numbers from the committed evidence.

Wang published a verdict against his own headline result as the first entry in his own lab, and set the protocol by which every later case will be judged.

Next on the docket

The lab has three more cases scheduled, each with a public due date.

R1, due December 1, 2026, re-tests the FinRL ensemble strategy presented at the ICAIF conference in 2020, which reported a Sharpe ratio of 1.30 against 0.47 for the Dow Jones Industrial Average over 2016 to 2020.

R2, due January 5, 2027, examines TradingAgents, a widely discussed framework that uses teams of large language models to trade. Its backtest reported Sharpe ratios of 5.6 to 8.2 on Apple, Alphabet and Amazon in the first quarter of 2024. The lab will replicate it and check whether the models remember that period from their training data.

R3, due February 15, 2027, is planned as a three-month live test of TradingAgents on the same three stocks. Each decision is to be committed before the US market opens, and scoring will check each commit against GitHub's run records, so late or edited entries are rejected.

That last design reflects a rule the lab applies to every language model agent: LLM agents only count live, unless the model's training cutoff predates the test window. A model trained on internet text may have seen the very price history it is being tested on, so a historical backtest cannot prove it predicted anything. Live decisions, time-stamped before the market opens, can.

Anyone can bring a case. The lab accepts nominations as GitHub issues that quote the claim verbatim, name the metric, window and baseline, and link to code if any exists.

From the lab to a trading platform with brakes

Wang applies the same standards to software meant to trade real money, though no real broker session has run yet. His companion project, financial-agent (github.com/BichengWang/financial-agent), is a Python platform for systematic ETF research, cost-aware backtesting, statistical certification and paper trading, with tightly gated execution through Alpaca and Interactive Brokers.

Before a strategy can be considered for live trading, it has to earn a certificate. That requires passing out-of-sample tests, a deflated Sharpe ratio that accounts for how many variants were tried, combinatorial cross-validation, bootstrap intervals and stress tests. Live trading then requires a certificate no more than 90 days old for the exact strategy and parameters, at least 20 clean paper-trading sessions with the same broker, mean absolute slippage within twice the modeled spread plus slippage, and an explicit human opt-in. Runtime risk limits cannot be loosened by configuration, and a persistent kill switch stops new orders.

The project publishes its evidence the same way the lab does, including results that do not flatter it. In a cost-adjusted historical backtest over data from February 2006 to October 2026, its default trend strategy returned 5.6 percent a year with a maximum drawdown of 16.5 percent, compared with 11.1 percent a year and a 54.3 percent drawdown for a modeled buy-and-hold of the S&P 500 ETF. The strategy passed all eight historical checks, while a probability of backtest overfitting (PBO) of 0.65 warned that tuning its parameters added no reliable value. A research signal driven by a language model failed its gate outright because there was not yet enough history to test it, and the project publishes that result as well. The current version, 1.2.0, is labeled beta, and the documentation states plainly that live performance is unproven.

Why it matters

AI trading claims now reach far beyond research labs.

What is missing is a public, neutral way to check those numbers. Wang's lab offers one: write down the claim, fix the rule before running the test, test it with realistic costs and many seeds, publish everything, and let the authors respond. It is free, released under the MIT license and open to anyone who wants to nominate a claim or reproduce a verdict.

The lab's first case may be its most persuasive argument. By putting his own five-year-old result through the process and publishing the outcome, Wang showed that the method works even when the researcher being tested is the one running the test.

About Bicheng (Kenneth) Wang

Bicheng (Kenneth) Wang is a Bay Area engineer who builds machine-learning decision systems for digital commerce, and the methods used to evaluate and control them. His independent open-source work includes RL-pricing, a reproducibility lab for trading AI; financial-agent, a research and paper-trading platform with gated execution; ai-decision-system, replayable records for AI-assisted decisions; and multi-agents, a runner and benchmark for teams of coding agents. He is the co-author, with Xinyi Zhang, of "Deep Reinforcement Learning for Equity Trading: Benchmarking Actor-Critic Methods with Forward Retraining" (arXiv:2609.31870).

Learn more: github.com/BichengWang