aeroxw/tick-stock-panel
1
0
Fork 0
mirror of https://ghfast.top/https://github.com/aeroxw/tick-stock-panel.git synced 2026-09-12 17:54:15 +08:00
Code Issues Packages Projects Releases Wiki Activity

feat: ETF 支持(选股 / 回测 / 监控) (#61)

Browse Source 
* feat(screener): 选股引擎支持 ETF

- 12 个内置策略打 asset_types 白名单 + strategy_supports_asset;涨停类
  (连板/断板反包)仅股票,其余 10 个技术类对 ETF 开放
- ScreenerService(repo, asset_type) 分流取数,ETF 复用 kline_etf_enriched,
  跳过股票专用历史缓存与涨停信号;进程级 _history_cache key 含 asset_type
- API /run、/run_preset 透传 asset_type;/strategies 按资产过滤;
  股票专有策略在 ETF 下返回空
- 新增 enriched_dirname(asset_type) 共享 helper;get_enriched_latest_asset
  增 refresh 参数(供轮询线程避免冷缓存同步重算)
- 前端「策略」页加 股票/ETF 切换,ETF 走实时单跑(空日期→用 ETF 自身最新日);
  QK.screenerStrategies 按 asset_type keyed
- 测试:test_screener_etf.py

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(backtest): 回测支持 ETF(个股/因子/策略组合)

- 三条回测路径 + 共用 BacktestEngine 面板加载按 asset_type 路由到
  kline_etf_enriched(复用 enriched_dirname);PanelCache key 隔离资产;
  ETF 跳过股票专用 get_enriched_range 缓存
- 面板 compute_all/名称 JOIN 按 asset_type 取维表(get_instruments_asset),
  修复 ETF 策略回测用错股票维表致名称为空/涨停信号算错
- BacktestConfig/FactorConfig/StrategyBacktestConfig 增 asset_type
- 三个回测 API + SSE stream 透传 asset_type;_make_job_key 纳入 asset_type
  (修复 stream 与 cancel job_key 不对齐致取消失效的回归)
- 前端策略组合页/因子页加 股票/ETF 切换,标的搜索与策略列表跟随资产;
  assetType 持久化
- 测试:test_backtest_etf.py(含 job_key 一致性回归);既有回测测试替身同步

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(monitor): 监控规则支持 ETF

- engine.evaluate(df, asset_type) 按规则 asset_type 分轮评估;quote_service
  增开 ETF 评估轮(用 ETF enriched 快照),股票轮不受影响、不重置其策略结果
- ETF 评估轮独立 try(异常不丢弃已算出的股票告警)+ refresh=False(不在轮询
  线程触发 ETF 冷缓存同步重算)
- ETF 版历史加载器(main.py 注入)+ 按规则 asset_type 选加载器
- _strategy_pools 按 (sid, asset_type) 键,避免同策略股票/ETF 规则互相覆盖
- name_map 仅在有 ETF 规则时补 ETF 维表, setdefault 保股票名优先
- RuleModel/normalize 增 asset_type(默认 stock,持久化往返)
- 前端 RuleEditor 加 股票/ETF 选择,策略列表与标的搜索跟随资产
- 测试:test_monitor_etf.py

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(etf): 前端 API 绑定透传 asset_type + 文档

- api.ts: screener/backtest 绑定加 assetType 参数,MonitorRule 类型加 asset_type
- docs/features.md: 标注选股/回测/监控的 ETF 支持范围与前提

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(reliability): 管道并发/原子写/能力探测/监控告警多处加固

后端可靠性专项修复(均带回归测试, backend 全套 64 passed):

并发与数据完整性:
- 盘后管道单飞: JobStore.create() 去重纳入 pending∨running, 关闭"两次快速点击"
  并发双跑窗口; 新增 _heavy_run_lock 执行槽挡住 reap 后僵尸线程并发写 parquet
- adj_factor/minute 全部改走原子写(tmp+replace), 消除 kill/断电致 all.parquet 损坏
- 分块拉取失败聚合 WARNING 可见化(不再静默当成功); 复权失败标的会保持旧价已提示

能力探测:
- 周期重探(60min)热更新 app.state.capabilities, 付费 Key 过期/续费无需重启即可见
- 瞬时探测失败(超时/连接/5xx, 按 _is_transient 判定)不降级、保留旧付费档;
  真 401/无权限仍正常降级回落 free-api

监控告警:
- 评估仅在连续竞价(9:30-11:30/13:00-15:00)+ 快照当日新鲜度下进行, 避开集合竞价/
  收盘后陈旧价与节假日误告警
- scope=sector fail-closed(validate 拒绝新建 + _apply_scope 返回空), 修复板块规则
  对全市场刷屏
- 飞书 webhook 加退避重试并移到独立线程池 fire-and-forget, 不再阻塞行情轮询线程

单标的新鲜度: 新增 repo.symbols_lagging() 检测掉队标的并 WARNING + 计入 job 结果

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jinfeng Sun
2026-07-08 12:12:29 +08:00
committed by GitHub
co-authored by Claude Opus 4.8
parent 9c730e4b69
commit e5a94c42d5
35 changed files with 1135 additions and 213 deletions
Download Patch File Download Diff File Expand all files Collapse all files
backend/app/api/kline.py
+38 -21
View File
@@ -582,7 +582,7 @@ async def sync_minute(request: Request):
"""手动触发分钟 K 同步(全市场)。返回 pipeline job_id 可轮询进度。"""
import asyncio
from app.services.pipeline_jobs import job_store
from app.services.pipeline_jobs import job_store, release_run_slot, try_acquire_run_slot
from app.api.data import invalidate_storage_cache
from app.services.preferences import get_minute_sync_days
from app.tickflow.capabilities import Cap
@@ -594,19 +594,21 @@ async def sync_minute(request: Request):
if not _minute_allowed(capset):
raise HTTPException(status_code=403, detail="需要 Pro+ 权限")
job_id = job_store.create()
existing = job_store.get(job_id)
if existing and existing["status"] == "running":
job_id, is_new = job_store.create()
if not is_new:
return {"status": "reused", "job_id": job_id}
async def task() -> None:
job_store.start(job_id)
if not try_acquire_run_slot():
job_store.fail(job_id, "已有数据任务在运行(或上一次任务卡死未结束),请稍后再试")
return
loop = asyncio.get_event_loop()
def progress(stage: str, pct: int, msg: str) -> None:
job_store.progress(job_id, stage, pct, msg)
try:
job_store.start(job_id)
progress("sync_minute", 5, "解析标的池…")
universe = sorted(set(get_pool("watchlist")) | set(get_pool("CN_Equity_A")))
# 补充 instruments 全量标的，覆盖北交所、新股等
@@ -637,6 +639,8 @@ async def sync_minute(request: Request):
except Exception as e: # noqa: BLE001
job_store.fail(job_id, str(e))
invalidate_storage_cache()
finally:
release_run_slot()
asyncio.create_task(task())
return {"status": "started", "job_id": job_id}
@@ -668,16 +672,17 @@ async def extend_history(request: Request):
raise HTTPException(status_code=403, detail="需要 Pro+ 权限 (batch K-line)")
from app.services.extend_history import run_extend_history
from app.services.pipeline_jobs import job_store
from app.services.pipeline_jobs import job_store, release_run_slot, try_acquire_run_slot
from app.api.data import invalidate_storage_cache
job_id = job_store.create()
existing = job_store.get(job_id)
if existing and existing["status"] == "running":
job_id, is_new = job_store.create()
if not is_new:
return {"status": "reused", "job_id": job_id}
async def task() -> None:
job_store.start(job_id)
if not try_acquire_run_slot():
job_store.fail(job_id, "已有数据任务在运行(或上一次任务卡死未结束),请稍后再试")
return
loop = asyncio.get_event_loop()
def progress(stage: str, pct: int, msg: str,
@@ -686,6 +691,7 @@ async def extend_history(request: Request):
stage_pct=stage_pct, skip_log=skip_log)
try:
job_store.start(job_id)
result = await loop.run_in_executor(
_long_task_executor,
lambda: run_extend_history(repo, capset, value, unit, on_progress=progress),
@@ -699,6 +705,8 @@ async def extend_history(request: Request):
logger.exception("extend_history failed: job_id=%s", job_id)
job_store.fail(job_id, str(e))
invalidate_storage_cache()
finally:
release_run_slot()
asyncio.create_task(task())
return {"status": "started", "job_id": job_id}
@@ -719,16 +727,17 @@ async def rebuild_enriched(request: Request):
try:
repo = request.app.state.repo
from app.services.pipeline_jobs import job_store
from app.services.pipeline_jobs import job_store, release_run_slot, try_acquire_run_slot
from app.api.data import invalidate_storage_cache
job_id = job_store.create()
existing = job_store.get(job_id)
if existing and existing["status"] == "running":
job_id, is_new = job_store.create()
if not is_new:
return {"status": "reused", "job_id": job_id}
async def task() -> None:
job_store.start(job_id)
if not try_acquire_run_slot():
job_store.fail(job_id, "已有数据任务在运行(或上一次任务卡死未结束),请稍后再试")
return
loop = asyncio.get_event_loop()
def progress(stage: str, pct: int, msg: str,
@@ -737,6 +746,7 @@ async def rebuild_enriched(request: Request):
stage_pct=stage_pct, skip_log=skip_log)
try:
job_store.start(job_id)
progress("rebuild_enriched", 10, "全量计算 enriched…")
from app.indicators.pipeline import run_pipeline
@@ -777,6 +787,8 @@ async def rebuild_enriched(request: Request):
logger.exception("rebuild_enriched failed: job_id=%s", job_id)
job_store.fail(job_id, str(e))
invalidate_storage_cache()
finally:
release_run_slot()
asyncio.create_task(task())
return {"status": "started", "job_id": job_id}
@@ -838,16 +850,17 @@ async def extend_minute_history(request: Request):
if total_days <= 0:
raise HTTPException(status_code=400, detail="扩展范围无效")
from app.services.pipeline_jobs import job_store
from app.services.pipeline_jobs import job_store, release_run_slot, try_acquire_run_slot
from app.api.data import invalidate_storage_cache
job_id = job_store.create()
existing = job_store.get(job_id)
if existing and existing["status"] == "running":
job_id, is_new = job_store.create()
if not is_new:
return {"status": "reused", "job_id": job_id}
async def task() -> None:
job_store.start(job_id)
if not try_acquire_run_slot():
job_store.fail(job_id, "已有数据任务在运行(或上一次任务卡死未结束),请稍后再试")
return
loop = asyncio.get_event_loop()
def progress(stage: str, pct: int, msg: str,
@@ -856,6 +869,7 @@ async def extend_minute_history(request: Request):
stage_pct=stage_pct, skip_log=skip_log)
try:
job_store.start(job_id)
# 获取当前最早日期
earliest = repo.earliest_minute_date()
if not earliest:
@@ -925,7 +939,8 @@ async def extend_minute_history(request: Request):
else:
day_df = day_df.drop("_trade_date")
day_df = day_df.sort("symbol", "datetime")
day_df.write_parquet(out)
from app.services.kline_sync import _atomic_write_parquet
_atomic_write_parquet(day_df, out)
written += day_df.height
day_count += 1
@@ -955,6 +970,8 @@ async def extend_minute_history(request: Request):
logger.exception("extend_minute_history failed: job_id=%s", job_id)
job_store.fail(job_id, str(e))
invalidate_storage_cache()
finally:
release_run_slot()
asyncio.create_task(task())
return {"status": "started", "job_id": job_id}