Files
tick-stock-panel/backend/app/main.py
T
Jinfeng SunandClaude Opus 4.8 e5a94c42d5 feat: ETF 支持(选股 / 回测 / 监控) (#61)
* feat(screener): 选股引擎支持 ETF

- 12 个内置策略打 asset_types 白名单 + strategy_supports_asset;涨停类
  (连板/断板反包)仅股票,其余 10 个技术类对 ETF 开放
- ScreenerService(repo, asset_type) 分流取数,ETF 复用 kline_etf_enriched,
  跳过股票专用历史缓存与涨停信号;进程级 _history_cache key 含 asset_type
- API /run、/run_preset 透传 asset_type;/strategies 按资产过滤;
  股票专有策略在 ETF 下返回空
- 新增 enriched_dirname(asset_type) 共享 helper;get_enriched_latest_asset
  增 refresh 参数(供轮询线程避免冷缓存同步重算)
- 前端「策略」页加 股票/ETF 切换,ETF 走实时单跑(空日期→用 ETF 自身最新日);
  QK.screenerStrategies 按 asset_type keyed
- 测试:test_screener_etf.py

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(backtest): 回测支持 ETF(个股/因子/策略组合)

- 三条回测路径 + 共用 BacktestEngine 面板加载按 asset_type 路由到
  kline_etf_enriched(复用 enriched_dirname);PanelCache key 隔离资产;
  ETF 跳过股票专用 get_enriched_range 缓存
- 面板 compute_all/名称 JOIN 按 asset_type 取维表(get_instruments_asset),
  修复 ETF 策略回测用错股票维表致名称为空/涨停信号算错
- BacktestConfig/FactorConfig/StrategyBacktestConfig 增 asset_type
- 三个回测 API + SSE stream 透传 asset_type;_make_job_key 纳入 asset_type
  (修复 stream 与 cancel job_key 不对齐致取消失效的回归)
- 前端策略组合页/因子页加 股票/ETF 切换,标的搜索与策略列表跟随资产;
  assetType 持久化
- 测试:test_backtest_etf.py(含 job_key 一致性回归);既有回测测试替身同步

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(monitor): 监控规则支持 ETF

- engine.evaluate(df, asset_type) 按规则 asset_type 分轮评估;quote_service
  增开 ETF 评估轮(用 ETF enriched 快照),股票轮不受影响、不重置其策略结果
- ETF 评估轮独立 try(异常不丢弃已算出的股票告警)+ refresh=False(不在轮询
  线程触发 ETF 冷缓存同步重算)
- ETF 版历史加载器(main.py 注入)+ 按规则 asset_type 选加载器
- _strategy_pools 按 (sid, asset_type) 键,避免同策略股票/ETF 规则互相覆盖
- name_map 仅在有 ETF 规则时补 ETF 维表, setdefault 保股票名优先
- RuleModel/normalize 增 asset_type(默认 stock,持久化往返)
- 前端 RuleEditor 加 股票/ETF 选择,策略列表与标的搜索跟随资产
- 测试:test_monitor_etf.py

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(etf): 前端 API 绑定透传 asset_type + 文档

- api.ts: screener/backtest 绑定加 assetType 参数,MonitorRule 类型加 asset_type
- docs/features.md: 标注选股/回测/监控的 ETF 支持范围与前提

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(reliability): 管道并发/原子写/能力探测/监控告警多处加固

后端可靠性专项修复(均带回归测试, backend 全套 64 passed):

并发与数据完整性:
- 盘后管道单飞: JobStore.create() 去重纳入 pending∨running, 关闭"两次快速点击"
  并发双跑窗口; 新增 _heavy_run_lock 执行槽挡住 reap 后僵尸线程并发写 parquet
- adj_factor/minute 全部改走原子写(tmp+replace), 消除 kill/断电致 all.parquet 损坏
- 分块拉取失败聚合 WARNING 可见化(不再静默当成功); 复权失败标的会保持旧价已提示

能力探测:
- 周期重探(60min)热更新 app.state.capabilities, 付费 Key 过期/续费无需重启即可见
- 瞬时探测失败(超时/连接/5xx, 按 _is_transient 判定)不降级、保留旧付费档;
  真 401/无权限仍正常降级回落 free-api

监控告警:
- 评估仅在连续竞价(9:30-11:30/13:00-15:00)+ 快照当日新鲜度下进行, 避开集合竞价/
  收盘后陈旧价与节假日误告警
- scope=sector fail-closed(validate 拒绝新建 + _apply_scope 返回空), 修复板块规则
  对全市场刷屏
- 飞书 webhook 加退避重试并移到独立线程池 fire-and-forget, 不再阻塞行情轮询线程

单标的新鲜度: 新增 repo.symbols_lagging() 检测掉队标的并 WARNING + 计入 job 结果

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-08 12:12:29 +08:00

322 lines
13 KiB
Python

"""FastAPI 入口。"""
from __future__ import annotations
import logging
from contextlib import asynccontextmanager
from pathlib import Path
from fastapi import FastAPI, Request
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import FileResponse, JSONResponse
from fastapi.staticfiles import StaticFiles
from app import __version__
from app.api import analysis, auth as auth_api, backtest, data, ext_data, financials, indices, intraday, kline, market_recap, monitor_rules, alerts, overview, pipeline, rps, screener, settings as settings_api, signals, stock_analysis, strategy, watchlist
from app.api.routes import router as core_router
from app.config import settings
from app.jobs import daily_pipeline
from app.services.quote_service import QuoteService
from app.tickflow import client as tf_client
from app.tickflow.policy import detect_capabilities
from app.tickflow.repository import DataStore, KlineRepository
logging.basicConfig(
level=settings.log_level,
format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
)
logger = logging.getLogger(__name__)
@asynccontextmanager
async def lifespan(app: FastAPI):
logger.info(
"TickFlow Stock Panel v%s starting (mode=%s)",
__version__, tf_client.current_mode(),
)
# 首次启动: 若配置了 AUTH_PASSWORD 环境变量且未设过密码, 用它初始化。
# 公网部署免 SSH 端口转发; 已设过密码则不覆盖 (改密码走 UI)。
try:
from app.services import auth as auth_service
auth_service.bootstrap_from_env()
except Exception as e: # noqa: BLE001
logger.warning("auth bootstrap failed: %s", e)
# 数据层
store = DataStore()
repo = KlineRepository(store)
app.state.datastore = store
app.state.repo = repo
# 指标异步预热标志: enriched 缓存在后台线程构建, 完成后置 True
app.state.indicators_ready = False
repo._on_warmup_done = lambda: setattr(app.state, "indicators_ready", True) # noqa: SLF001
# Polars 缓存预热 — enriched 的重计算 (107万行 compute_indicators) 推后台,
# instruments/index/ETF 仍同步 (毫秒级)。应用立即 ready, 指标算完后自动替换。
repo.refresh_cache(background=True)
# 能力探测
capset = detect_capabilities()
app.state.capabilities = capset
logger.info("ready; %d capabilities active", len(capset.all()))
# 自定义数据源配置(可选): 失败只记录错误, 不影响 TickFlow 基准路径。
try:
from app.data_providers import custom as custom_sources
custom_sources.load_all()
logger.info("custom data sources loaded: %d", len(custom_sources.list_sources()))
except Exception as e: # noqa: BLE001
logger.warning("custom data sources init failed: %s", e)
# 全局行情服务
qs = QuoteService()
app.state.quote_service = qs
qs.set_repo(repo)
qs.boot_check()
# QuoteService 需要访问 strategy_monitor 等单例
# 先创建 strategy_monitor,再注入 app.state
from app.strategy.monitor import StrategyMonitorService
strategy_monitor = StrategyMonitorService()
app.state.strategy_monitor = strategy_monitor
qs.set_app_state(app.state)
# 五档盘口 sealed 服务(真假涨停/跌停, 独立旁路线)
from app.services.depth_service import DepthService
depth_service = DepthService()
depth_service.set_repo(repo)
depth_service.set_app_state(app.state)
app.state.depth_service = depth_service
# 启动调度器(若 enriched 数据为空,首次启动可手动 POST /api/pipeline/run)
try:
daily_pipeline.set_app_state(app.state) # 供 depth_finalize job 访问 depth_service
scheduler = daily_pipeline.start_scheduler(repo, capset)
app.state.scheduler = scheduler
except Exception as e: # noqa: BLE001
logger.warning("scheduler not started: %s", e)
app.state.scheduler = None
# depth sealed: 启动补跑(当天文件不存在) + 盘中轮询(有能力时)
try:
depth_service.boot_check()
depth_service.start_polling()
except Exception as e: # noqa: BLE001
logger.warning("depth_service init failed: %s", e)
# 扩展数据定时拉取
from app.services.ext_pull import pull_scheduler
pull_scheduler.start(store.data_dir)
pull_scheduler.refresh(store.data_dir)
app.state.pull_scheduler = pull_scheduler
# 内置扩展表 (概念/行业): 只创建 config (含拉取配置), 不自动拉数据
# 数据获取由用户在概念/行业页点「获取数据」手动触发 (POST /api/ext-data/presets/{id}/fetch)
try:
from app.services.ext_presets import ensure_builtin_presets
await ensure_builtin_presets(store.data_dir)
except Exception as e: # noqa: BLE001
logger.warning("内置扩展表初始化失败 (不影响启动): %s", e)
# 财务数据 (需 Expert 套餐): 仅初始化调度器供 /api/financials/sync/* 手动同步,
# 不启动自动调度——用户在「财务分析」页点「同步」手动拉取。
from app.services.financial_sync import financial_scheduler
financial_scheduler.start(store.data_dir, capset)
app.state.financial_scheduler = financial_scheduler
# 策略引擎
from app.strategy.engine import StrategyEngine
from app.strategy.monitor import StrategyMonitorService
from app.services.screener import ScreenerService
_screener_svc = ScreenerService(repo)
_etf_screener_svc = ScreenerService(repo, asset_type="etf")
strategy_dirs = [
Path(__file__).resolve().parent / "strategy" / "builtin",
store.data_dir / "strategies" / "custom",
store.data_dir / "strategies" / "ai",
]
strategy_engine = StrategyEngine(
enriched_loader=_screener_svc._load_enriched_for_date,
enriched_history_loader=_screener_svc._load_enriched_history,
strategy_dirs=strategy_dirs,
)
app.state.strategy_engine = strategy_engine
logger.info("strategy engine loaded: %d strategies", len(strategy_engine.list_strategies()))
# 通用监控规则引擎: 启动时 reload 规则到内存态 (修复重启后告警失效)
from app.strategy.monitor import MonitorRuleEngine
from app.strategy import monitor_rules as mr_store
from app.services import preferences
monitor_engine = MonitorRuleEngine()
monitor_engine.set_strategy_engine(strategy_engine)
monitor_engine.set_data_dir(store.data_dir)
# 复用 ScreenerService 的历史窗口加载器 (三级缓存, 启动预计算命中 ~0ms),
# 让声明 filter_history 的策略 (如反包) 也能在实时监控里跑选股 → 盘中触发通知。
monitor_engine.set_history_loader(_screener_svc._load_enriched_history)
# ETF 版历史加载器: asset_type=etf 的 strategy 型规则用 (读 kline_etf_enriched)。
monitor_engine.set_history_loader_etf(_etf_screener_svc._load_enriched_history)
# 自动迁移: 把旧 strategy_monitor_ids 同步为 type=strategy 规则 (统一到监控页)
try:
if preferences.get_strategy_monitor_enabled():
ids = preferences.get_strategy_monitor_ids()
if ids:
names = {s.id: s.name for s in strategy_engine.list_strategies()}
mr_store.migrate_strategy_monitors(store.data_dir, ids, names)
logger.info("strategy monitor migrated: %d strategies", len(ids))
except Exception as e: # noqa: BLE001
logger.warning("strategy monitor migration failed: %s", e)
try:
rules = mr_store.load_all(store.data_dir)
monitor_engine.set_rules(rules)
logger.info("monitor engine loaded: %d rules", monitor_engine.rule_count)
except Exception as e: # noqa: BLE001
logger.warning("monitor engine load failed: %s", e)
app.state.monitor_engine = monitor_engine
yield
if app.state.scheduler:
app.state.scheduler.shutdown(wait=False)
ps = getattr(app.state, "pull_scheduler", None)
if ps:
ps.stop()
fsc = getattr(app.state, "financial_scheduler", None)
if fsc:
fsc.stop()
qs = getattr(app.state, "quote_service", None)
if qs:
qs.stop()
dsvc = getattr(app.state, "depth_service", None)
if dsvc:
dsvc.stop_polling()
logger.info("shutdown")
app = FastAPI(
title="TickFlow Stock Panel",
version=__version__,
description="A 股选股 + 回测面板 — TickFlow 适配",
lifespan=lifespan,
)
# CORS: 允许局域网访问 (自托管场景, 放开所有来源)
# 注: allow_credentials=True 与 allow_origins=['*'] 不能共存 (浏览器规范),
# 本项目认证走 header (API Key), 不依赖 cookie, 故关闭 credentials 换取通配来源。
app.add_middleware(
CORSMiddleware,
allow_origins=["*"],
allow_credentials=False,
allow_methods=["*"],
allow_headers=["*"],
)
# ================================================================
# 访问认证中间件
# ================================================================
# 拦截所有 /api/ 请求, 三种状态:
# 1. 未设密码 + 本机/内网 → 放行(让本机用户访问面板 + 调 /api/auth/setup 设密码)
# 2. 未设密码 + 公网 → 拒绝(403, 防裸奔也防抢占; 引导本机设密码)
# 3. 已设密码 → 检查 session, 无效则 401(前端跳登录)
# 白名单: /api/auth/* (设密码/登录本身)、/health 等探活。
_AUTH_WHITELIST_PREFIX = ("/api/auth/",)
_AUTH_WHITELIST_EXACT = ("/health", "/api/health", "/openapi.json", "/docs", "/redoc")
@app.middleware("http")
async def auth_middleware(request: Request, call_next):
path = request.url.path
# 仅 /api/ 走认证; 静态资源(前端页面/assets)放行, 由前端处理跳转
if not path.startswith("/api/"):
return await call_next(request)
# 白名单放行(设密码/登录/探活本身不拦)
if path.startswith(_AUTH_WHITELIST_PREFIX) or path in _AUTH_WHITELIST_EXACT:
return await call_next(request)
from app.services import auth as auth_service
# 情况 1+2: 未设密码
if not auth_service.is_configured():
# 本机/内网 → 放行(服务器主人可访问, 并去 /login 设密码)
if auth_api._is_local_network(auth_api._client_ip(request)):
return await call_next(request)
# 公网 → 拒绝。不裸奔, 也不给公网设密码的机会(防抢占)
return JSONResponse(
status_code=403,
content={
"detail": "面板尚未初始化访问密码,请通过 SSH/本机浏览器访问以设置密码",
"code": "NOT_INITIALIZED",
},
)
# 情况 3: 已设密码, 检查会话
token = request.cookies.get(auth_api.COOKIE_NAME)
if token and auth_service.is_valid_session(token):
return await call_next(request)
# 未登录: 401(前端跳登录页)
return JSONResponse(status_code=401, content={"detail": "未登录或会话已过期"})
# 路由
app.include_router(core_router)
app.include_router(auth_api.router)
app.include_router(kline.router)
app.include_router(watchlist.router)
app.include_router(screener.router)
app.include_router(backtest.router)
app.include_router(intraday.router)
app.include_router(indices.router)
app.include_router(overview.router)
app.include_router(analysis.router)
app.include_router(pipeline.router)
app.include_router(data.router)
app.include_router(ext_data.router)
app.include_router(financials.router)
app.include_router(stock_analysis.router)
app.include_router(market_recap.router)
app.include_router(settings_api.router)
app.include_router(strategy.router)
app.include_router(signals.router)
app.include_router(monitor_rules.router)
app.include_router(alerts.router)
app.include_router(rps.router)
# 能力门控异常 → 403(而非默认 500)
# 业务代码用 capset.require(Cap.X) 断言能力,缺失时抛 CapabilityDenied;
# 若不注册 handler 会冒泡成 500 Internal Server Error,对前端不友好且语义错误。
from fastapi import Request
from fastapi.responses import JSONResponse
from app.tickflow.capabilities import CapabilityDenied
@app.exception_handler(CapabilityDenied)
async def capability_denied_handler(request: Request, exc: CapabilityDenied) -> JSONResponse:
return JSONResponse(
status_code=403,
content={"detail": str(exc), "suggestion": exc.suggestion},
)
# 生产期静态文件(前端 dist)
_static = Path(settings.static_dir)
if _static.exists():
if (_static / "assets").exists():
app.mount("/assets", StaticFiles(directory=_static / "assets"), name="assets")
@app.get("/{full_path:path}", include_in_schema=False)
def spa_fallback(full_path: str): # noqa: ARG001
"""所有未匹配路径回退到 index.html — React Router 接管。
index.html 禁止缓存 (Cache-Control: no-store), 确保浏览器每次拿到
最新版本引用的 JS/CSS 文件名 (assets 带 hash, 可长缓存)。
"""
index = _static / "index.html"
if index.exists():
return FileResponse(
index,
headers={"Cache-Control": "no-store, must-revalidate"},
)
return {"error": "frontend not built"}