feat(backtest): 稳健性指标 Sortino + 蒙特卡罗回撤分位 + per-trade 明细 (#67)

* feat(backtest): 新增稳健性指标 Sortino + 蒙特卡罗回撤分位 + per-trade 明细回测原本只有 Sharpe/Calmar/最大回撤, 缺防过拟合视角。本 PR 补三类: 1. Sortino 比率: 用下行偏差 (MAR=0 目标半方差) 替代总标准差, 只惩罚负收益波动。无亏损时约定返回 None (不虚报 0/inf)。各 stats 函数用与自身 Sharpe 相同的收益基准 (逐笔 or 日频)。 2. 蒙特卡罗最大回撤分位: 对逐笔收益有放回重抽样 1000 次, 估计回撤分布, 报 P50(中位) 与 P95(95% 置信最坏)。回答"仅因成交顺序运气回撤能有多坏", 单次样本内回撤看不到这个。固定种子 (42) 保证可复现/可测。 3. per-trade 明细: best/worst/median_pnl/avg_holding_days, 补 _calc_stats 与 _calc_portfolio_stats 原本缺失的逐笔视角。三个 stats 函数 (_calc_stats / _calc_independent_candidate_result / _calc_portfolio_stats) 全部接入, 用共享静态 helper (_sortino_ratio / _mc_drawdown_percentiles / _per_trade_block) 避免重复。纯 additive, 既有字段不变, 空交易安全。前端 StrategyBacktest 结果区新增索提诺 / 蒙卡回撤(中位) / 蒙卡回撤(95%最坏) 三个指标卡 (stats 为 Record<string,any>, 无需改类型)。新增 tests/backtest/test_robustness_metrics.py 11 用例: Sortino 手算校验/ 无下行 None/优于 Sharpe/样本不足; 蒙卡确定性/P95≤P50≤0/全正零回撤/样本不足; _calc_stats 与 portfolio 字段集成 + 空交易安全。注: upstream main 既有 test_trailing_take_profit_exits_after_activation 失败与本 PR 无关 (改动前后一致复现)。 * test+fix(backtest): 子代理审查修复 — 补 full 主路径覆盖 + finite 护栏 + 内存上限子代理审查发现的缺口, 本次补全: 覆盖 (测试): - [关键] full 模式主路径 _calc_independent_candidate_result 原无集成测试, 漏拼 sortino/mc 字典展开会导致前端指标卡空值却测试全绿。新增该分支断言。 - MC 确定性测试从"两次相等"升级为钉死快照值 (p50=-0.0976/p95=-0.2108), 一旦有人把种子改成系统熵立即红。 - p95 断言从恒真的 p95<=p50 (percentile 单调性数学恒成立, 无信息) 改为 p50<0 且 p95<p95 严格更差, 真正验证数值。 - 补全盈利交易 → sortino 集成层为 None 的用例 (原仅 helper 层覆盖)。健壮性 (实现): - 三个 helper 入口 finite 过滤: inf/nan 收益会让 cumprod 传播 nan → 分位 nan → asdict 直出非法 JSON (项目 _safe 只洗 rows 不洗 stats)。剔除后与纯净输入结果一致 (加测试守护)。 - MC 内存护栏: full 模式可数千笔, n_sims×n×8B×4 数组瞬时可达数百 MB。按 2M 单元上限压降模拟次数 (小样本 n_sims=1000 不变, 快照稳定)。注: 逐笔年化口径 (_calc_stats 对 pnls ×√252) 落在死路径 simulate() (无 app 调用方) 且复刻既有 Sharpe, 不在本 PR 范围。两条活路径 (portfolio/independent) 均用日频基准, ×√252 正确。
2026-09-12 16:44:15 +08:00 · 2026-07-08 12:10:05 +08:00
parent 0121a5583f
commit 3a5f3e05b6
3 changed files with 244 additions and 0 deletions
@@ -1280,6 +1280,66 @@ class BacktestEngine:

    # ── 统计计算 ──────────────────────────────────────

+    @staticmethod
+    def _sortino_ratio(returns: np.ndarray, periods_per_year: int = 252) -> float | None:
+        """Sortino 比率: 用下行偏差 (仅惩罚负收益) 替代总标准差, 年化。
+
+        下行偏差 = sqrt(mean(min(r, 0)^2)), MAR=0 的目标半方差 (对全部样本求均, 非仅负样本)。
+        无下行波动 (无亏损) 时 Sortino 未定义, 返回 None (与 profit_factor 的 None 约定一致,
+        不虚报 0 或 inf)。样本不足 (<2) 返回 0.0 (与 sharpe 的退化约定一致)。
+        """
+        returns = returns[np.isfinite(returns)]  # 剔除 inf/nan, 防止污染均值/序列化出非法 JSON
+        if len(returns) < 2:
+            return 0.0
+        mean = float(np.mean(returns))
+        downside = np.minimum(returns, 0.0)
+        downside_dev = float(np.sqrt(np.mean(downside ** 2)))
+        if downside_dev <= 0:
+            return None
+        return mean / downside_dev * float(np.sqrt(periods_per_year))
+
+    @staticmethod
+    def _mc_drawdown_percentiles(pnls: np.ndarray, n_sims: int = 1000) -> dict:
+        """自助重抽样交易序列, 估计最大回撤的分布 — 回答"仅因成交顺序运气, 回撤能有多坏"。
+
+        对每笔收益有放回重抽样 n_sims 次, 各自算最大回撤, 取分位:
+        - mc_maxdd_p50: 中位场景最大回撤
+        - mc_maxdd_p95: 95% 置信最坏场景 (= 分布 5 分位, 更负)
+
+        固定种子保证可复现/可测。样本 <3 无统计意义, 返回 None。
+        大样本 (如 full 模式数千笔) 时按 2M 单元上限压降模拟次数, 防止瞬时数组 OOM。
+        """
+        pnls = pnls[np.isfinite(pnls)]  # 剔除 inf/nan, 否则 cumprod 传播 nan 导致分位为 nan
+        n = len(pnls)
+        if n < 3:
+            return {"mc_maxdd_p50": None, "mc_maxdd_p95": None}
+        # 内存护栏: samples/equity/peak/dd 各占 eff_sims*n*8B, 控总单元 <= 2M (~64MB 峰值)
+        eff_sims = min(n_sims, max(200, 2_000_000 // n))
+        rng = np.random.default_rng(42)
+        samples = rng.choice(pnls, size=(eff_sims, n), replace=True)
+        equity = np.cumprod(1.0 + samples, axis=1)
+        peak = np.maximum.accumulate(equity, axis=1)
+        dd = (equity - peak) / peak
+        maxdds = dd.min(axis=1)
+        return {
+            "mc_maxdd_p50": round(float(np.percentile(maxdds, 50)), 4),
+            "mc_maxdd_p95": round(float(np.percentile(maxdds, 5)), 4),
+        }
+
+    @staticmethod
+    def _per_trade_block(pnls: np.ndarray, durations: np.ndarray) -> dict:
+        """per-trade 明细字段: best/worst/median_pnl/avg_holding_days。"""
+        pnls = pnls[np.isfinite(pnls)]  # 剔除 inf/nan, 防 best/worst 出非法值
+        durations = durations[np.isfinite(durations)] if len(durations) else durations
+        if not len(pnls):
+            return {"best": 0.0, "worst": 0.0, "median_pnl": 0.0, "avg_holding_days": 0.0}
+        return {
+            "best": round(float(np.max(pnls)), 4),
+            "worst": round(float(np.min(pnls)), 4),
+            "median_pnl": round(float(np.median(pnls)), 4),
+            "avg_holding_days": round(float(np.mean(durations)), 1) if len(durations) else 0.0,
+        }
+
    @staticmethod
    def _calc_stats(
        trades: list[TradeRecord],
@@ -1332,14 +1392,19 @@ class BacktestEngine:
        # 夏普 — 用交易收益标准差近似
        sharpe = float(np.mean(pnls) / np.std(pnls)) * np.sqrt(252) if np.std(pnls) > 0 else 0.0

+        # Sortino — 与 sharpe 同基准 (逐笔收益), 仅惩罚下行波动
+        sortino = BacktestEngine._sortino_ratio(pnls)
+
        # Calmar
        calmar = annual_return / abs(max_dd) if abs(max_dd) > 0.001 else 0.0

+        durations = np.array([t.duration for t in trades], dtype=float)
        return {
            "total_return": round(float(total_return), 4),
            "annual_return": round(float(annual_return), 4),
            "max_drawdown": round(float(max_dd), 4),
            "sharpe": round(float(sharpe), 2),
+            "sortino": round(float(sortino), 2) if sortino is not None else None,
            "calmar": round(float(calmar), 2),
            "win_rate": round(float(win_rate), 4),
            "profit_factor": round(float(profit_factor), 2) if np.isfinite(profit_factor) else None,
@@ -1347,6 +1412,8 @@ class BacktestEngine:
            "avg_pnl": round(float(np.mean(pnls)), 4),
            "avg_win": round(avg_win, 4),
            "avg_loss": round(avg_loss, 4),
+            **BacktestEngine._per_trade_block(pnls, durations),
+            **BacktestEngine._mc_drawdown_percentiles(pnls),
        }

    @staticmethod
@@ -1441,6 +1508,7 @@ class BacktestEngine:
        max_drawdown = float(drawdowns.min()) if len(drawdowns) else 0.0
        daily = np.array(daily_avg, dtype=float)
        sharpe = float(np.mean(daily) / np.std(daily) * np.sqrt(252)) if len(daily) > 1 and np.std(daily) > 0 else 0.0
+        sortino = BacktestEngine._sortino_ratio(daily)

        lo, hi, nbins = -0.20, 0.20, 20
        clipped = np.clip(pnls, lo, hi)
@@ -1471,8 +1539,10 @@ class BacktestEngine:
            "total_return": round(float(total_return), 4),
            "max_drawdown": round(float(max_drawdown), 4),
            "sharpe": round(float(sharpe), 2),
+            "sortino": round(float(sortino), 2) if sortino is not None else None,
            "return_distribution": dist,
            "execution": execution_stats,
+            **BacktestEngine._mc_drawdown_percentiles(pnls),
        }

        return SimResult(
@@ -1500,7 +1570,9 @@ class BacktestEngine:
        drawdowns = values / peaks - 1
        max_drawdown = float(drawdowns.min()) if len(drawdowns) else 0.0
        sharpe = float(np.mean(daily) / np.std(daily) * np.sqrt(252)) if len(daily) and np.std(daily) > 0 else 0.0
+        sortino = BacktestEngine._sortino_ratio(daily)
        pnls = np.array([t.pnl_pct for t in trades], dtype=float) if trades else np.array([])
+        durations = np.array([t.duration for t in trades], dtype=float) if trades else np.array([])
        exposures = np.array([float(r.get("exposure", 0.0)) for r in equity_curve], dtype=float)
        wins = pnls[pnls > 0]
        losses = pnls[pnls <= 0]
@@ -1511,6 +1583,7 @@ class BacktestEngine:
            "annual_return": round(float(annual_return), 4),
            "max_drawdown": round(float(max_drawdown), 4),
            "sharpe": round(float(sharpe), 2),
+            "sortino": round(float(sortino), 2) if sortino is not None else None,
            "calmar": round(float(annual_return / abs(max_drawdown)), 2) if abs(max_drawdown) > 0.001 else 0.0,
            "win_rate": round(float(len(wins) / len(pnls)), 4) if len(pnls) else 0.0,
            "profit_factor": round(float(avg_win / avg_loss), 2) if avg_loss > 0 else None,
@@ -1518,6 +1591,8 @@ class BacktestEngine:
            "avg_pnl": round(float(np.mean(pnls)), 4) if len(pnls) else 0.0,
            "avg_win": round(avg_win, 4),
            "avg_loss": round(avg_loss, 4),
+            **BacktestEngine._per_trade_block(pnls, durations),
+            **BacktestEngine._mc_drawdown_percentiles(pnls),
            "final_equity": round(final_equity, 2),
            "initial_capital": round(float(initial_capital), 2),
            "avg_exposure": round(float(np.mean(exposures)), 4) if len(exposures) else 0.0,
@@ -0,0 +1,164 @@
+"""稳健性指标测试 — Sortino + 蒙特卡罗回撤分位 + per-trade 明细。
+
+被测新增:
+- BacktestEngine._sortino_ratio(returns, periods_per_year): 下行波动调整收益比
+- BacktestEngine._mc_drawdown_percentiles(pnls, n_sims): 自助重抽样估计最大回撤分布
+- _calc_stats / _calc_portfolio_stats 输出新增 sortino / mc_maxdd_p50 / mc_maxdd_p95 /
+  median_pnl / best / worst / avg_holding_days 字段
+"""
+from __future__ import annotations
+
+from datetime import date
+
+import numpy as np
+
+from app.backtest.engine import BacktestEngine, TradeRecord
+
+# ---------------------------------------------------------------
+# Sortino
+# ---------------------------------------------------------------
+
+def test_sortino_all_losses_is_exact():
+    """全亏损序列: mean/downside_dev * sqrt(252) 可手算校验。"""
+    r = np.array([-0.1, -0.1])
+    # mean=-0.1; neg=[-0.1,-0.1]; downside_dev=sqrt(mean(0.01,0.01))=0.1
+    # sortino = -0.1/0.1 * sqrt(252) = -sqrt(252)
+    got = BacktestEngine._sortino_ratio(r)
+    assert abs(got - (-np.sqrt(252))) < 1e-6
+
+
+def test_sortino_no_downside_returns_none():
+    """无负收益 → 下行波动为 0, Sortino 未定义, 约定返回 None (不虚报 inf/0)。"""
+    r = np.array([0.05, 0.10, 0.02])
+    assert BacktestEngine._sortino_ratio(r) is None
+
+
+def test_sortino_exceeds_sharpe_when_downside_is_tamer():
+    """下行波动小于总波动时, Sortino 应高于 Sharpe (只惩罚下行的优势)。"""
+    # 大涨小跌: 上行贡献总波动但不进下行 → sortino > sharpe
+    r = np.array([0.20, -0.02, 0.20, -0.02])
+    mean = float(np.mean(r))
+    sharpe = mean / float(np.std(r)) * np.sqrt(252)
+    sortino = BacktestEngine._sortino_ratio(r)
+    assert sortino is not None
+    assert sortino > sharpe
+
+
+def test_sortino_too_few_points():
+    assert BacktestEngine._sortino_ratio(np.array([0.1])) == 0.0
+    assert BacktestEngine._sortino_ratio(np.array([])) == 0.0
+
+
+# ---------------------------------------------------------------
+# 蒙特卡罗最大回撤分位
+# ---------------------------------------------------------------
+
+# 固定种子 (42) + 固定输入下的快照值; 一旦有人改种子或算法, 立即红。
+_MC_INPUT = np.array([0.05, -0.03, 0.08, -0.06, 0.02, -0.04, 0.10, -0.05])
+_MC_P50 = -0.0976
+_MC_P95 = -0.2108
+
+
+def test_mc_drawdown_is_deterministic_snapshot():
+    """固定种子 → 结果既跨调用一致, 又等于钉死的快照值 (防有人把种子改成系统熵)。"""
+    a = BacktestEngine._mc_drawdown_percentiles(_MC_INPUT)
+    b = BacktestEngine._mc_drawdown_percentiles(_MC_INPUT)
+    assert a == b
+    assert a["mc_maxdd_p50"] == _MC_P50
+    assert a["mc_maxdd_p95"] == _MC_P95
+
+
+def test_mc_drawdown_p95_strictly_worse_and_negative():
+    """含亏损输入: 中位场景必有回撤 (p50<0), 且 P95 严格差于 P50 (非恒真的 <=)。"""
+    r = BacktestEngine._mc_drawdown_percentiles(_MC_INPUT)
+    assert r["mc_maxdd_p50"] < 0.0
+    assert r["mc_maxdd_p95"] < r["mc_maxdd_p50"]
+
+
+def test_mc_drawdown_ignores_non_finite():
+    """含 nan/inf 的收益应被剔除, 结果与纯净输入完全一致 (不污染分位/序列化)。"""
+    dirty = np.concatenate([_MC_INPUT, [np.nan, np.inf, -np.inf]])
+    assert BacktestEngine._mc_drawdown_percentiles(dirty) == BacktestEngine._mc_drawdown_percentiles(_MC_INPUT)
+
+
+def test_mc_drawdown_all_positive_has_zero_drawdown():
+    """全正收益: 任何重排都无回撤 → 分位均为 0。"""
+    pnls = np.array([0.01, 0.02, 0.03, 0.04, 0.05])
+    r = BacktestEngine._mc_drawdown_percentiles(pnls)
+    assert r["mc_maxdd_p50"] == 0.0
+    assert r["mc_maxdd_p95"] == 0.0
+
+
+def test_mc_drawdown_too_few_trades():
+    r = BacktestEngine._mc_drawdown_percentiles(np.array([0.1, -0.1]))
+    assert r["mc_maxdd_p50"] is None
+    assert r["mc_maxdd_p95"] is None
+
+
+# ---------------------------------------------------------------
+# 集成: stats 输出新字段
+# ---------------------------------------------------------------
+
+def _trades(pnls: list[float], durations: list[int]) -> list[TradeRecord]:
+    out = []
+    for p, d in zip(pnls, durations, strict=True):
+        out.append(TradeRecord(
+            symbol="A", entry_date=date(2024, 1, 1), exit_date=date(2024, 1, 1 + d),
+            entry_price=10.0, exit_price=10.0 * (1 + p), pnl_pct=p, duration=d,
+            exit_reason="signal",
+        ))
+    return out
+
+
+def test_calc_stats_emits_robustness_fields():
+    trades = _trades([0.10, -0.05, 0.08, -0.06], [3, 2, 5, 4])
+    stats = BacktestEngine._calc_stats(trades, 100_000, date(2024, 1, 1), date(2024, 6, 1))
+    for k in ("sortino", "mc_maxdd_p50", "mc_maxdd_p95", "median_pnl", "best", "worst", "avg_holding_days"):
+        assert k in stats, f"缺字段 {k}"
+    assert stats["best"] == round(0.10, 4)
+    assert stats["worst"] == round(-0.06, 4)
+    assert stats["median_pnl"] == round(float(np.median([0.10, -0.05, 0.08, -0.06])), 4)
+    assert stats["avg_holding_days"] == round(float(np.mean([3, 2, 5, 4])), 1)
+
+
+def test_calc_stats_empty_trades_safe():
+    """空交易不应因新字段计算崩溃。"""
+    stats = BacktestEngine._calc_stats([], 100_000, date(2024, 1, 1), date(2024, 6, 1))
+    assert stats["n_trades"] == 0
+
+
+def test_portfolio_stats_emits_robustness_fields():
+    """portfolio 分支同样输出 sortino / mc / per-trade 字段。"""
+    equity_curve = [
+        {"date": "2024-01-01", "value": 100_000.0, "exposure": 0.0},
+        {"date": "2024-01-02", "value": 103_000.0, "exposure": 0.5},
+        {"date": "2024-01-03", "value": 101_000.0, "exposure": 0.5},
+        {"date": "2024-01-04", "value": 105_000.0, "exposure": 0.5},
+    ]
+    trades = _trades([0.06, -0.02, 0.04], [2, 1, 3])
+    stats = BacktestEngine._calc_portfolio_stats(equity_curve, trades, 100_000)
+    for k in ("sortino", "mc_maxdd_p50", "mc_maxdd_p95", "median_pnl", "best", "worst", "avg_holding_days"):
+        assert k in stats, f"缺字段 {k}"
+
+
+def test_independent_candidate_stats_emits_sortino_and_mc():
+    """full 模式主路径 (_calc_independent_candidate_result) 必须输出 sortino / mc 字段。
+
+    这是前端 full 模式指标卡的真实数据来源, 若漏拼字典展开会导致 UI 显示空值。
+    """
+    # 构造足量交易 (>=3) 以触发 mc; 用引擎产出真实结果而非直接调私有函数
+    trades = _trades([0.10, -0.05, 0.08, -0.06, 0.03], [2, 1, 3, 2, 4])
+    result = BacktestEngine._calc_independent_candidate_result(
+        trades, n_candidates=5, execution_stats={},
+    )
+    for k in ("sortino", "mc_maxdd_p50", "mc_maxdd_p95"):
+        assert k in result.stats, f"independent 分支缺字段 {k}"
+    # mc 应为有效数值 (n=5>=3)
+    assert result.stats["mc_maxdd_p50"] is not None
+
+
+def test_calc_stats_all_wins_reports_sortino_none():
+    """全盈利交易在 stats 集成层: 无下行波动 → sortino 序列化为 None (非 0)。"""
+    trades = _trades([0.10, 0.05, 0.08], [3, 2, 4])
+    stats = BacktestEngine._calc_stats(trades, 100_000, date(2024, 1, 1), date(2024, 6, 1))
+    assert stats["sortino"] is None
@@ -1652,8 +1652,13 @@ export function StrategyBacktest() {
                <Stat label="超额收益" value={excessReturn != null ? fmtPct(excessReturn) : '—'}
                  color={statValueColor(excessReturn)} />
                <Stat label={<SharpeLabel />} value={pick('sharpe') != null ? Number(pick('sharpe')).toFixed(2) : '—'} />
+                <Stat label="索提诺" value={pick('sortino') != null ? Number(pick('sortino')).toFixed(2) : '—'} />
                <Stat label="最大回撤" value={pick('max_drawdown') != null ? fmtPct(pick('max_drawdown') as number) : '—'}
                  color="#34d399" />
+                <Stat label="蒙卡回撤(中位)" value={pick('mc_maxdd_p50') != null ? fmtPct(pick('mc_maxdd_p50') as number) : '—'}
+                  color="#34d399" />
+                <Stat label="蒙卡回撤(95%最坏)" value={pick('mc_maxdd_p95') != null ? fmtPct(pick('mc_maxdd_p95') as number) : '—'}
+                  color="#34d399" />
                <Stat label="胜率" value={pick('win_rate') != null ? fmtPct(pick('win_rate') as number) : '—'} />
                <Stat label="交易数" value={pick('n_trades') != null ? String(pick('n_trades')) : '—'} />
                {result.stats.full_kind === 'candidate_execution' ? (