diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index fecfc7e..88944c0 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -106,6 +106,7 @@ - 窗口、前 N 日和批次回算均按实际交易日，不得用自然日直接替代。 - A 股交易时段统一按北京时间处理；服务器时区不能成为业务逻辑的隐式输入。 +- 分钟 K 的 `datetime` 统一为北京时间墙钟（naive，如 `09:35:00`）；数据源入口（`kline_sync` 的 `_normalize_minute` / `_try_custom_minute`）强制归一，禁止 UTC 口径入库或下发。 - 日线、分钟线和实时快照必须明确交易日期归属，尤其注意午休、收盘后和跨日重启。 - 分钟 K 的股票、ETF、指数分开存储和路由，不得仅凭代码格式猜测资产类型。 @@ -129,6 +130,16 @@ - 插件加载失败、字段缺失和空数据必须有隔离测试，不能导致应用启动失败或其他数据源不可用。 - 新增数据源适配应同步更新 `docs/custom-data-source.md` 或 `docs/plugin-development.md` 中对应契约。 +### 能力路由矩阵契约 + +能力矩阵（`backend/app/data_providers/capabilities.py` 注册表 + `/api/settings/capability-matrix`）是能力路由的单一权威，遵循以下不变量： + +- 注册表集中声明每个能力的展示元数据、路由偏好字段与 TickFlow 档位要求；前端不硬编码能力清单。新增能力按既有模式扩展：注册表 + preferences getter + capability-matrix 注入 + 矩阵测试。 +- 各页面能力门控统一以矩阵的 `usable` 为准（生效源当前能否真正提供该能力），不是 TickFlow 套餐视角；缺能力提示统一引导到数据源配置。 +- 能力层中立：通用界面（侧栏徽章、能力路由卡、各页门控提示）不得出现 TickFlow 档位/订阅词汇；档位信息只在 TickFlow 专属详情卡展示。provider 名称作为路由事实可以出现。 +- 每个能力独立路由，禁止跟随/派生特殊值（`same_as_daily` 已下线）；存量非法偏好值由 preferences getter 回退默认自愈，不做迁移。 +- 边界注记：分时监控由分钟能力兜底（`intraday_monitor_support`），不单设分时能力；`depth5` 已进矩阵但插件数据集白名单暂未开放，当前仅 TickFlow 提供。 + ## 5. 领域专项要求 ### 5.1 策略 diff --git a/README.md b/README.md index 735a1ed..308ebdd 100644 --- a/README.md +++ b/README.md @@ -1,48 +1,38 @@
-# 📈 A股智能量化工作台 - -[![声明:个人开源](https://img.shields.io/badge/⚠️_声明-个人开源_非TickFlow官方项目-green?style=for-the-badge&labelColor=red)](https://github.com/shy3130/tick-stock-panel) - +# TSP--A股智能量化工作台 +[![声明: 本项目为个人开源, 非隶属任何官方项目](https://img.shields.io/badge/%E5%A3%B0%E6%98%8E-%E6%9C%AC%E9%A1%B9%E7%9B%AE%E4%B8%BA%E4%B8%AA%E4%BA%BA%E5%BC%80%E6%BA%90_%E9%9D%9E%E9%9A%B6%E5%B1%9E%E4%BB%BB%E4%BD%95%E5%AE%98%E6%96%B9%E9%A1%B9%E7%9B%AE-green?style=for-the-badge&labelColor=red)](https://github.com/shy3130/tick-stock-panel) **自托管、零运维的 A 股「选股 + 监控 + 回测」量化工作台** -**面向个人散户与量化爱好者而生** +**多数据源能力路由 · 分钟级策略执行 · 全时段异动监控 · AI 辅助研究** [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](./LICENSE) -[![Python](https://img.shields.io/badge/Python-≥3.11-blue.svg)](https://www.python.org/) -[![React](https://img.shields.io/badge/React-18-61dafb.svg)](https://react.dev/) -[![Data: TickFlow](https://img.shields.io/badge/Data-TickFlow-00b386.svg)](https://tickflow.org/auth/register?ref=V3KDKGXPEA) +[![Engine: Polars](https://img.shields.io/badge/Engine-Polars-6f42c1.svg)](https://pola.rs/) +[![Data: 多源插件化 · 能力路由](https://img.shields.io/badge/Data-%E5%A4%9A%E6%BA%90%E6%8F%92%E4%BB%B6%E5%8C%96%20%C2%B7%20%E8%83%BD%E5%8A%9B%E8%B7%AF%E7%94%B1-00b386.svg)](./docs/custom-data-source.md) [![Deploy: Docker](https://img.shields.io/badge/Deploy-Docker-2496ed.svg)](./Dockerfile) +[![Build: Docker CI](https://github.com/shy3130/tick-stock-panel/actions/workflows/docker.yml/badge.svg)](https://github.com/shy3130/tick-stock-panel/actions/workflows/docker.yml) [![GitHub stars](https://img.shields.io/github/stars/shy3130/tick-stock-panel?style=social)](https://github.com/shy3130/tick-stock-panel/stargazers)
- - -**[快速开始](#-快速开始)** · **[核心功能](#-核心功能)** · **[配置](#️-配置)** · **[完整文档](#-完整文档)** +**[快速开始](#-快速开始)** · **[核心功能](#-核心功能)** · **[技术架构](#️-技术架构)** · **[配置](#️-配置)** · **[完整文档](#-完整文档)**
- --- +**本项目以个人开源为主进行开发维护，数据源插件化，可任意接入第三方数据源。仅供学习研究使用，严禁商业用途。** - -**本项目个人开源，数据源插件化，可任意接入第三方数据源。仅供学习研究使用，严禁商业用途。** - - - - -> ⚠️ 小白请绕路，本开源项目谨作为本地量化提供解决思路Demo，不作为投资软件或者看盘软件。 +> ⚠️ 小白请绕路，本开源项目谨作为本地量化提供解决思路与方案，不作为投资软件或者看盘软件。 > > **明确不做**:不对标同花顺 / 通达信,不内置「AI 荐股 / 涨停预测」。 -有问题可以邮件415333856@qq.com。 +有任何项目问题或商务合作 / 广告投放等合作意向,可邮件联系 415333856@qq.com。 觉得有用可以点个 Star @@ -50,59 +40,60 @@ ## ✨ 核心功能 -| 模块 | 一句话 | 详见 | +| 模块         | 一句话 | 详见    | | :--------------- | :--------------------------------------------------------------------- | :-------------------------------- | -| 🔍 **选股引擎** | 18 个内置策略 + 自定义信号 + AI 生成 + 代码迁移,Polars 毫秒级扫全 A 股 | [strategy.md](./docs/strategy.md) | -| 📊 **指标流水线** | MA/EMA/MACD/RSI/KDJ/布林/量比等,一次扫表落盘 enriched Parquet | [features.md](./docs/features.md) | -| 🧪 **回测研究** | 因子/策略回测 + 财务快照因子(点时口径),T+1/费用/滑点约束,SSE 持久任务 | [features.md](./docs/features.md) | +| 🔀 **能力路由** | 多数据集(日K/除权/实时/分钟/盘口/财务,持续扩展)按源能力独立路由,任选组合 | [custom-data-source.md](./docs/custom-data-source.md) | +| 🔍 **选股引擎** | 18 个内置策略 + 分钟策略 + 自定义信号 + AI 生成,Polars 毫秒级扫全 A 股 | [strategy.md](./docs/strategy.md) | +| 📊 **指标流水线** | MA/EMA/MACD/RSI/KDJ/布林/量比等 68 列指标与信号,一次扫表落盘 enriched Parquet | [features.md](./docs/features.md) | +| 🧪 **回测研究** | 因子/策略/分钟回测 + 财务快照因子(点时口径),T+1/费用/滑点约束,结果可导出 | [features.md](./docs/features.md) | | ⛏️ **因子挖掘** | 嵌套样本外搜索多因子排名组合,与自有策略对照,候选库显式发布、永不自动上线 | [mining.md](./docs/mining.md) | | 🌡️ **市场环境** | 情绪周期 6 阶段(连板梯队驱动)+ 概念/行业主线排名,与 5 档环境分并存 | [market-phase.md](./docs/market-phase.md) | -| 🚨 **异动监控** | 交易所异动规则口径(3/10/30 日偏离值),盘中实时接近度,系统告警与推送接入 | — | +| 🚨 **异动监控** | 竞价/盘中/偏移三类异动一页覆盖:同花顺风向标 + 当日信号聚合 + 交易所偏离值口径 | — | | 📡 **监控中心** | 四类监控(策略/个股信号/价格/异动),多条件 AND/OR + 语音播报 + 飞书推送 | [features.md](./docs/features.md) | | 📈 **个股分析** | 9 类关键价位 + AI 四维分析(技术/基本面/财务/消息面) | [features.md](./docs/features.md) | -| 🏆 **连板梯队** | 连板层级统计 + 概念涨幅轮动 + 盘后 AI 复盘 + 炸板/翘板预警 | [features.md](./docs/features.md) | -| 🧰 **数据扩展** | 数据源插件化(stock-sdk 示例 + YAML 自定义源),扩展字段配成一级页面同台分析 | [custom-data-source.md](./docs/custom-data-source.md) | - - - - +| 🏆 **连板梯队** | 连板层级统计 + 概念涨幅轮动 + 盘后 AI 复盘(龙虎榜/盘前风向标注入) + 炸板/翘板预警 | [features.md](./docs/features.md) | +| 🧰 **数据扩展** | 数据源插件化(TickFlow/fuyao/stock-sdk + YAML 自定义源),扩展字段配成一级页面同台分析 | [custom-data-source.md](./docs/custom-data-source.md) |
📦 主要页面与功能 **📊 行情总览** -- **看板** Dashboard — 市场情绪评分 + 涨跌/成交额榜单 + 概念领涨领跌 + 大盘异动事件流,一日全貌 -- **自选** Watchlist — 自选股池,多分组管理(M:N),表格/卡片双视图,换手/量比/RSI 等实时指标 +- **看板** Dashboard — 市场情绪评分 + 涨跌/成交额榜单 + 概念/行业领涨领跌(点击板块直达成分股,领涨股带涨跌幅) + 大盘异动事件流,一日全貌 +- **自选** Watchlist — 自选股池,多分组管理(M:N),表格/卡片双视图,换手/量比/RSI 等实时指标,按档位分流实时刷新 - **指数** Indices — 沪深指数浏览与同步 **🔍 选股与回测** -- **策略** Screener — Polars 毫秒级扫描全 A 股,18 个内置策略卡片 + 自定义条件 -- **回测** Backtest — 两种模式: - - **因子回测** — IC/IR、分层收益、多空组合,先筛掉无效指标 - - **策略回测** — 净值曲线、回撤、夏普、胜率,支持 T+1/手续费/滑点/止损,SSE 流式进度 +- **策略** Screener — Polars 毫秒级扫描全 A 股,日线/分钟策略统一单池,按策略声明周期自动路由执行 +- **回测** Backtest — 四种研究视图: + - **因子回测** — IC/IR、分层收益、多空组合,62+ 因子目录先筛掉无效指标 + - **策略回测** — 净值曲线、回撤、夏普、胜率,T+1/手续费/滑点/止损,SSE 流式进度 + - **分钟策略回测** — 逐交易日回放信号、分钟收盘入场,分钟级成交明细 + - **验证** — 参数敏感性与滚动样本外 + - 研究闭环:结果导出 CSV(概要/净值/交易明细/分标的统计) → 保存候选 → **一键载入复测** - **挖掘** Mining — 嵌套样本外因子与策略挖掘:训练区间因子方向重估 + 相关性去重 + 多因子排名组合搜索,自有策略作对照轨;候选入库,显式确认后才发布,永不自动上线 **📈 个股与板块分析** - **个股分析** Stock Analysis (Beta) — 日K + 9 类关键价位 + AI 四维分析(技术/基本面/财务/消息面) -- **财务分析** Financials — 利润表/资负表/现金流/关键指标 + AI 解读 -- **概念分析** Concept Analysis — ths 概念涨幅轮动矩阵 + 领涨/领跌主线 + 个股穿透 -- **行业分析** Industry Analysis — 行业分层涨幅轮动 + 领涨/领跌主线 + 成分股 +- **财务分析** Financials — 利润表/资负表/现金流/关键指标(多源并集合并,fuyao 财务四表适配) + AI 解读 +- **概念分析 / 行业分析** — ths 维度涨幅轮动矩阵 + 领涨/领跌主线 + 个股穿透 - **市场环境** Regime — 情绪周期 6 阶段(冰点/启动/主升/高潮/退潮/修复,连板梯队驱动,EMA 平滑 + 2 日确认)+ 概念/行业主线排名,与 5 档环境分并存 - **连板梯队** Limit Up Ladder — 连板层级统计 + 概念/行业分布 + 封单监控(可切换连跌梯队) **🔔 监控与复盘** - **监控中心** Monitor — 策略/个股信号/价格/异动四类规则,支持自选分组作用域,盘中实时弹窗 + 语音播报(播报个股名称与信号) + 触发记录持久化 -- **异动监控** Abnormal Moves — 按交易所异动规则口径(3日 ±20%/±30%/±40% · 10日 +100% · 30日 +200%)实时计算个股偏离值接近度,盯住异动边缘名单;触发记录 + 站内通知/飞书·企微推送 -- **复盘** Review (Beta) — 盘后 AI 自动生成市场复盘,可定时执行、推送飞书、下载 Markdown +- **异动监控** Abnormal Moves — 按交易时间线三 tab: + - **竞价异动** — 同花顺盘前风向标(含当日/次日真实收益对照、追高风险标记)+ 全市场竞价扫描(待采集任务) + - **盘中异动** — 涨停/炸板/翘板/跌停/新高/新低/放量当日信号聚合,零新增采集 + - **偏移异动** — 交易所异动偏离值口径(主板 3 日 ±20%、创业板/科创板 ±30%、北交所 ±40%;10 日 +100%/−50%、30 日 +200%/−70%),实时接近度 +- **复盘** Review (Beta) — 盘后 AI 自动生成市场复盘,注入龙虎榜资金动向与盘前风向标对照;可定时执行、推送飞书、下载 Markdown **🗄️ 数据与扩展** - **数据** Data — 本地数据画像与同步状态(维表/日K/除权/Enriched/指数/ETF/分钟K/财务),盘后管道与历史扩展 - **扩展分析** (动态菜单) — 把任意第三方/扩展数据字段配成一级菜单,与内置数据同台分析 -- **设置** Settings — 数据源与能力检测、AI 接口、实时监控、扩展页面、信号库、菜单与系统设置 +- **设置** Settings — 数据源与能力检测(能力路由矩阵、档位徽章)、AI 接口、实时监控、扩展页面、信号库、菜单与系统设置
- --- ## 📸 界面预览 @@ -148,11 +139,116 @@ - --- +## 🏗️ 技术架构 +### 分层总览 +```mermaid +flowchart TB + subgraph DATA["数据源层 · 插件化"] + direction LR + D1["TickFlow SDK"] ~~~ D2["fuyao
同花顺 REST"] ~~~ D3["stock-sdk"] ~~~ D4["YAML 自定义源"] ~~~ D5["+ 更多插件…"] + end + + subgraph ROUTE["能力路由层"] + direction LR + R(["多数据集 · 按能力独立路由 · 档位探测"]) + end + + subgraph STORE["存储层"] + direction LR + ST1[("Parquet 分区表")] ~~~ ST2[("DuckDB")] ~~~ ST3[("JSON 按日缓存")] + end + + subgraph CALC["计算层 · Polars"] + direction LR + C1["指标流水线"] ~~~ C2["复权与信号"] + end + + subgraph RES["研究层"] + direction LR + G1["因子引擎"] ~~~ G2["回测引擎"] ~~~ G3["因子挖掘"] + end + + subgraph SVC["应用层 · FastAPI"] + direction LR + S1["REST · SSE"] ~~~ S2["监控引擎"] ~~~ S3["任务调度"] ~~~ S4["交易日探针"] + end + + subgraph FE["呈现层 · React 18"] + direction LR + F1["功能页面"] ~~~ F2["图表可视化"] ~~~ F3["实时推送"] + end + + subgraph EXT["二次开发与扩展 · 贯穿各层的插槽"] + direction LR + X1["自定义策略"] ~~~ X2["自定义信号"] ~~~ X3["扩展分析页面"] ~~~ X4["AI 接口"] + end + + DATA --- ROUTE + ROUTE --- STORE + STORE --- CALC + CALC --- RES + RES --- SVC + SVC --- FE + FE ~~~ EXT + + classDef fe fill:#eef2ff,stroke:#6366f1,color:#312e81 + classDef svc fill:#ecfeff,stroke:#06b6d4,color:#164e63 + classDef res fill:#fff7ed,stroke:#f97316,color:#7c2d12 + classDef calc fill:#f0f9ff,stroke:#0ea5e9,color:#0c4a6e + classDef store fill:#ecfdf5,stroke:#10b981,color:#064e3b + classDef data fill:#fdf2f8,stroke:#ec4899,color:#831843 + classDef pluginSlot fill:#fdf2f8,stroke:#ec4899,color:#831843,stroke-dasharray:5 4 + classDef route fill:#faf5ff,stroke:#8b5cf6,color:#6b21a8,stroke-width:2px + classDef ext fill:#f8fafc,stroke:#94a3b8,color:#334155,stroke-dasharray:5 4 + + class F1,F2,F3 fe + class S1,S2,S3,S4 svc + class G1,G2,G3 res + class C1,C2 calc + class ST1,ST2,ST3 store + class D1,D2,D3,D4 data + class D5 pluginSlot + class R route + class X1,X2,X3,X4 ext + + style FE fill:#f5f3ff,stroke:#c7d2fe,color:#3730a3 + style SVC fill:#ecfeff,stroke:#a5f3fc,color:#155e75 + style RES fill:#fff7ed,stroke:#fed7aa,color:#9a3412 + style CALC fill:#f0f9ff,stroke:#bae6fd,color:#075985 + style STORE fill:#ecfdf5,stroke:#a7f3d0,color:#065f46 + style ROUTE fill:#faf5ff,stroke:#ddd6fe,color:#6b21a8 + style DATA fill:#fdf2f8,stroke:#fbcfe8,color:#9d174d + style EXT fill:#f8fafc,stroke:#94a3b8,color:#334155,stroke-dasharray:7 5 +``` + +### 关键机制 + +| 机制 | 说明 | +| :--- | :--- | +| **能力路由矩阵** | 各数据集按源声明能力独立路由,注册表集中定义、可扩展:TICKFLOW 档位探测(None/Free/Starter/Pro/Expert)+ 插件源能力声明,fail-closed(声明 `pct_unit` 未声明即拒)。同一数据集可随时换源,指标与回测口径不变 | +| **交易日探针** | fuyao 交易日历(确定性,含调休)→ tickflow 全市场行情时间戳探针(OR 语义)→ 工作日兜底;节假日自动停掉实时轮询与分钟增量,零无效请求 | +| **财务多源合并** | 按 `(symbol, period_end)` 报告期累积,多源取并集、逐列按公告日取最新(PIT);公告前一律空值,绝不填 0 | +| **非路由数据集直连** | 龙虎榜/盘前风向标/交易日历等 fuyao 专有能力不进路由矩阵,由独立服务直连消费——按日 JSON 缓存(历史不可变)、交易日回退、四态降级 | +| **回测执行隔离** | 回测在 spawn worker 子进程运行,持久 run ID,刷新/切页重连不丢任务;子进程结果消息经锁保护回传 | +| **分层缓存** | enriched 读取时现算指标(存储仅 15 列基础数据,现算 68 列指标与信号)+ 进程内快照缓存;扩展字段按日分区快照,页面即配即用 | + +### 技术栈 + +| 层 | 选型 | +| :----------- | :------------------------------------------------------------------------------------------------ | +| **后端** | FastAPI · Pydantic v2 · APScheduler · sse-starlette | +| **数据** | Polars(计算)· DuckDB(查询)· Parquet(存储) | +| **回测** | 自研仓位模拟引擎(T+1/费用/滑点/分钟回放)· vectorbt(部分路径) | +| **数据源** | [TickFlow](https://tickflow.org/auth/register?ref=V3KDKGXPEA) 官方 SDK · fuyao(同花顺 REST) · 插件化扩展(stock-sdk 示例插件 · YAML 自定义源) | +| **AI**(可选) | OpenAI 兼容接口(DeepSeek / 通义 / Ollama 等) | +| **前端** | React 18 · Vite · TypeScript · Tailwind · Tanstack Query · [Lightweight Charts](https://www.tradingview.com/lightweight-charts/)(TradingView 开源) · ECharts · dnd-kit | +| **部署** | Docker 两阶段构建,前端 dist 拷进后端镜像,**单容器** | + +--- ## 🚀 快速开始 @@ -195,11 +291,11 @@ CODEX_CLI_VERSION=0.144.3 docker compose up --build ### 跑起来后的第一次使用 -1. **设置 → 凭据与能力** → 点 **重新检测**,确认档位标签 +1. **设置 → 凭据与能力** → 点 **重新检测**,确认档位标签与能力路由矩阵 2. **设置** → **立即跑盘后管道**:拉日 K + 计算 enriched 表(None / Free 走 free-api,当日数据盘后 1-2 小时可用) 3. **自选**页加标的 → **选股**页点策略卡片扫描 / 配自定义信号 -4. **回测**页选策略 + 区间 → 看净值 / 夏普 / 交易明细(SSE 实时进度) -5. **监控中心**配规则,盘中实时弹窗 + 持久化记录 +4. **回测**页选策略 + 区间 → 看净值 / 夏普 / 交易明细(SSE 实时进度),结果可导出 CSV、存候选一键复测 +5. **监控中心**配规则,盘中实时弹窗 + 持久化记录;**异动监控**覆盖竞价/盘中/偏移全时段 --- @@ -217,30 +313,17 @@ PORT=3018 # 服务端口 --- -## 🏗️ 技术栈 - -| 层 | 选型 | -| :----------- | :------------------------------------------------------------------------------------------------ | -| **后端** | FastAPI · Pydantic v2 · APScheduler · sse-starlette | -| **数据** | Polars(计算)· DuckDB(查询)· Parquet(存储) | -| **回测** | vectorbt(全项目唯一 pandas 边界) | -| **数据源** | [TickFlow](https://tickflow.org/auth/register?ref=V3KDKGXPEA) 官方 SDK · 插件化扩展(stock-sdk 示例插件 · YAML 自定义源) | -| **AI**(可选) | OpenAI 兼容接口(DeepSeek / 通义 / Ollama 等) | -| **前端** | React 18 · Vite · TypeScript · Tailwind · Tanstack Query · Lightweight Charts · ECharts · dnd-kit | -| **部署** | Docker 两阶段构建,前端 dist 拷进后端镜像,**单容器** | - ---- - ## 🗺️ 路线图 | Phase | 内容 | 状态 | | :----- | :----------------------------------------------------------------- | :--- | | 0-1 | 仓库骨架 · FastAPI 壳 · 能力探测 · K 线同步与分析页 | ✅ | -| 2-3 | Polars enriched 流水线 · Screener · vectorbt 回测(T+1/手续费/止损) | ✅ | +| 2-3 | Polars enriched 流水线 · Screener · 回测引擎(T+1/手续费/止损) | ✅ | | 4-5 | 监控引擎 · 四类监控规则 · 实时 SSE 推送 · 持久化记录 | ✅ | | 6 | 个股分析(专用日 K + 9 类关键价位 + AI 四维分析) | ✅ | | **v0.2** | 因子挖掘全链路 · 市场阶段与主线识别 · 异动监控 · 数据源插件化 | ✅ | -| **v2** | Webhook 推送· 板块异动 · 早晚报 · 更多扩展 | 🚧 | +| **v0.3** | 能力路由矩阵 · fuyao 数据源(财务/龙虎榜/风向标) · 分钟策略与回测 · 交易日探针 · 全时段异动中心 · 回测导出与候选复测 | ✅ | +| **v2** | Webhook 推送· 板块异动 · 早晚报 · 全市场竞价采集 · 更多扩展 | 🚧 | --- @@ -251,11 +334,11 @@ PORT=3018 # 服务端口 | [docs/deployment.md](./docs/deployment.md) | 部署方式(Dev / Docker / GH Actions)、老 CPU 兼容、更新代码、访问密码 | | [docs/configuration.md](./docs/configuration.md) | 所有 `.env` 配置项详解(数据源、AI、服务、密码、数据目录) | | [docs/features.md](./docs/features.md) | 各功能模块详细说明(选股/指标/回测/监控/个股分析/数据扩展) | -| [docs/custom-data-source.md](./docs/custom-data-source.md) | 自定义数据源接入、YAML 配置与 mock 联调示例 | +| [docs/custom-data-source.md](./docs/custom-data-source.md) | 自定义数据源接入、能力路由契约、YAML 配置与 mock 联调示例 | | [docs/strategy.md](./docs/strategy.md) | 策略体系(18 内置策略 + 三种扩展方式 + 文件结构) | | [docs/mining.md](./docs/mining.md) | 因子与策略挖掘口径、防泄漏、任务隔离和发布边界 | | [docs/market-phase.md](./docs/market-phase.md) | 市场情绪周期 6 阶段与概念/行业主线识别的口径与设计 | -| [docs/plugin-development.md](./docs/plugin-development.md) | 数据源插件开发规范(以 stock-sdk 为参考实现) | +| [docs/plugin-development.md](./docs/plugin-development.md) | 数据源插件开发规范(以 stock-sdk / fuyao 为参考实现) | | [docs/secondary-development.md](./docs/secondary-development.md) | 代码二次开发、前端插槽、后端策略接口与 AI 开发模板 | | [backend/app/strategy/prompts/strategy-guide.md](./backend/app/strategy/prompts/strategy-guide.md) | 策略开发完整规范(AI 生成与手写) | @@ -265,15 +348,42 @@ fork同时请点个star哦,欢迎 Issue 和 PR。 ## 💬 交流群 -欢迎加入交流群,讨论交流。 +欢迎加入交流群,一起讨论交流。作者个人维护的部分个性化接口,统一公布在群公告中,供大家免费使用。 交流群二维码 --- +## ❤️ 支持项目 + +
+ +如果这个项目对你有帮助,欢迎请作者喝杯咖啡 ☕ + + + + + + + + + + +
微信赞赏支付宝
微信赞赏码 · 感谢道友支持 愿一路长红支付宝收款码 · 打开支付宝扫一扫
+ +愿道友一路长红 📈 + +
+ +> 打赏完全自愿,金额不限;不用于购买任何功能、数据权限、投资建议 +> +> 作者精力有限,优先响应赞助回馈,希望理解 + +--- + ## ⚠️ 免责声明 -本项目仅供**学习与量化研究**,**不构成任何投资建议**。回测结果不代表未来收益。A 股有风险,入市需谨慎。数据准确性以数据源 TickFlow 官方为准。 +本项目仅供**学习与量化研究**,**不构成任何投资建议**。回测结果不代表未来收益。A 股有风险,入市需谨慎。数据准确性以数据源官方为准。 ## 📄 License @@ -281,6 +391,8 @@ fork同时请点个star哦,欢迎 Issue 和 PR。 本项目依赖 [TickFlow](https://tickflow.org/auth/register?ref=V3KDKGXPEA) 提供数据服务,使用前请遵守其服务条款 +内置数据源插件 [fuyao](https://fuyao.aicubes.cn/docs/api-reference/) 提供同花顺 REST 数据接口(行情 / 财务 / 龙虎榜 / 盘前风向标 / 交易日历等),需自备 API Key,使用前请遵守其服务条款 + 数据源插件 [stock-sdk](https://stock-sdk.linkdiary.cn) 遵循其各自的 ISC 协议。 ## 社区 diff --git a/VERSION b/VERSION index d663ead..f0cfd3b 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -v0.1.88 +v0.2.2 diff --git a/assets/support/alipay.jpg b/assets/support/alipay.jpg new file mode 100644 index 0000000..c376386 Binary files /dev/null and b/assets/support/alipay.jpg differ diff --git a/assets/support/wechat-appreciation.jpg b/assets/support/wechat-appreciation.jpg new file mode 100644 index 0000000..ff950eb Binary files /dev/null and b/assets/support/wechat-appreciation.jpg differ diff --git a/backend/app/__init__.py b/backend/app/__init__.py index d9f1257..b5cb0c0 100644 --- a/backend/app/__init__.py +++ b/backend/app/__init__.py @@ -2,7 +2,7 @@ import sys -__version__ = "0.2.1" +__version__ = "0.2.2" # Windows 默认 stdout/stderr 编码为 GBK(cp936),TickFlow SDK 内部输出含 emoji 的 # 指数/标的名称(如 \U0001f193)时会抛 UnicodeEncodeError,导致请求失败。 diff --git a/backend/app/api/abnormal.py b/backend/app/api/abnormal.py index e4f6027..c75ac85 100644 --- a/backend/app/api/abnormal.py +++ b/backend/app/api/abnormal.py @@ -1,13 +1,27 @@ -"""异动边缘监控 API — 按交易所异动规则口径统计接近触发的个股。""" +"""异动监控 API — 竞价/盘中/偏移三类异动。 + +- /intraday: 盘中量价信号聚合 (enriched 当日信号列, 零新增采集) +- /overview: 偏移异动边缘总览 (交易所异动规则口径的接近度) +""" from __future__ import annotations from fastapi import APIRouter, Query, Request -from app.services.abnormal_moves import build_overview +from app.services.abnormal_moves import build_intraday, build_overview router = APIRouter(prefix="/api/abnormal", tags=["abnormal"]) +@router.get("/intraday") +def abnormal_intraday( + request: Request, + limit: int = Query(500, ge=1, le=2000), +): + """盘中异动: 涨停/炸板/跌停翘板/跌停/新高/新低/放量 信号命中行。""" + repo = request.app.state.repo + return build_intraday(repo, limit=limit) + + @router.get("/overview") def abnormal_overview( request: Request, diff --git a/backend/app/api/backtest.py b/backend/app/api/backtest.py index 2605e73..5eccf99 100644 --- a/backend/app/api/backtest.py +++ b/backend/app/api/backtest.py @@ -346,6 +346,33 @@ class StrategyBacktestRequest(BaseModel): regime_filter: dict | None = None +def _guard_minute_strategy_backtest( + request: Request, strategy_id: str, start: date, asset_type: str, +) -> None: + """分钟策略回测入口守卫: 仅 A 股 + 本地分钟K覆盖检查 (fail-fast)。""" + engine = getattr(request.app.state, "strategy_engine", None) + if engine is None: + return + try: + s = engine.get(strategy_id) + except ValueError: + return + if s is None or s.execution_backend != "minute_filter": + return + if asset_type != "stock": + raise HTTPException(400, detail="分钟策略回测当前仅支持 A 股 (stock)") + earliest = request.app.state.repo.earliest_minute_date() + if earliest is None or start < earliest: + have = f"最早到 {earliest}, " if earliest else "" + raise HTTPException( + 400, + detail=( + f"本地分钟K{have}无法覆盖回测起始日 {start}。" + "请先用「扩展分钟K历史」拉取更多数据, 或缩小回测区间" + ), + ) + + @router.post("/strategy/run") def strategy_run(req: StrategyBacktestRequest, request: Request): """策略回测 — 复用 StrategyDef 体系做全周期回测。""" @@ -355,6 +382,7 @@ def strategy_run(req: StrategyBacktestRequest, request: Request): end = req.end or date.today() start = _resolve_start(req, end, FACTOR_DEFAULT_DAYS) _guard_server_backtest_range(start, end) + _guard_minute_strategy_backtest(request, req.strategy_id, start, req.asset_type) cfg = StrategyBacktestConfig( strategy_id=req.strategy_id, @@ -505,6 +533,7 @@ async def strategy_stream( # 空 start = 全部历史: 用本地最早日K日期, 查不到再回退到默认窗口 earliest = request.app.state.repo.earliest_daily_date() start_date = earliest or (end_date - timedelta(days=FACTOR_DEFAULT_DAYS)) + _guard_minute_strategy_backtest(request, strategy_id, start_date, asset_type) # 服务端范围保护 guard_violated = False diff --git a/backend/app/api/ext_data.py b/backend/app/api/ext_data.py index 6fc005c..67e81e9 100644 --- a/backend/app/api/ext_data.py +++ b/backend/app/api/ext_data.py @@ -7,6 +7,7 @@ import math import re import shutil import tempfile +import time from datetime import date, datetime from pathlib import Path from typing import Literal @@ -485,6 +486,198 @@ def dimension_members( } +# --------------------------------------------------------------------------- +# 板块分时 (dimension intraday) +# --------------------------------------------------------------------------- + +# 点击触发 + 60s 进程内缓存: 分钟分区是滚动底座 (minute_refresh / 盘后分钟同步), +# 不做后台预计算 — 板块基数大而单次聚合仅几十毫秒。 +_DIMENSION_INTRADAY_CACHE: dict[tuple[str, str, str, str | None], tuple[float, dict]] = {} +_DIMENSION_INTRADAY_CACHE_TTL_S = 60.0 +# 成分股网格化 ffill 上限: 超大板块退化为逐时间戳可得均值 (内存保护)。 +_DIMENSION_INTRADAY_FFILL_CAP = 2000 + + +def _bare_symbol_expr(col: str = "symbol") -> pl.Expr: + """'000001.SZ' → '000001'; 已是裸代码则原样。""" + return pl.col(col).cast(pl.String).str.strip_chars().str.split(".").list.first() + + +def _dimension_member_bares(matched: pl.DataFrame, config: ExtConfig) -> list[str]: + """成分股裸代码集合 (symbol 列优先级与 dimension-members 端点一致)。""" + if matched.is_empty(): + return [] + symbol_columns = ["symbol", "code", "股票代码", "代码"] + for mapping in (config.symbol_map, config.code_map): + if isinstance(mapping, dict) and mapping.get("type") == "mapped" and mapping.get("col"): + symbol_columns.append(str(mapping["col"])) + cols = [c for c in dict.fromkeys(symbol_columns) if c in matched.columns] + if not cols: + return [] + coalesced = pl.coalesce( + [pl.col(c).cast(pl.String).str.strip_chars().str.split(".").list.first() for c in cols] + ) + series = matched.select(coalesced.alias("_bare")).to_series() + return sorted({s for s in series.to_list() if s}) + + +def _prev_daily_close(data_dir: Path, target_date: str) -> pl.DataFrame | None: + """目标日前最近一个日K分区的收盘价 → (_bare, prev_close); 无则 None。""" + daily = data_dir / "kline_daily" + if not daily.exists(): + return None + dates = sorted( + d.name[5:] + for d in daily.iterdir() + if d.is_dir() and d.name.startswith("date=") and (d / "part.parquet").exists() + ) + prevs = [d for d in dates if d < target_date] + if not prevs: + return None + path = daily / f"date={prevs[-1]}" / "part.parquet" + if not path.exists(): + return None + df = pl.read_parquet(path, columns=["symbol", "close"]) + return ( + df.with_columns(_bare_symbol_expr().alias("_bare")) + .select([pl.col("_bare"), pl.col("close").cast(pl.Float64).alias("prev_close")]) + .unique(subset=["_bare"], keep="last") + ) + + +def _dimension_intraday_compute( + config: ExtConfig, + data_dir: Path, + field: str, + value: str, + snapshot_date: str | None, +) -> dict: + """板块等权分时: 成分股当日分钟K逐分钟平均涨跌幅 + 全市场对照线。 + + 口径: pct = 分钟close / ref − 1 (小数制, 与快照涨跌幅契约一致, 前端 ×100 显示), + ref 优先前一交易日日K收盘 (prev_close, 开盘跳空体现在曲线起点); + 日K缺失的标的退化为当日首根分钟close (混合基准)。 + 停牌/无成交分钟按成分股 forward-fill 后再平均, 全市场线取逐时间戳可得均值。 + """ + minute_dir = data_dir / "kline_minute" + partitions: list[str] = [] + if minute_dir.exists(): + partitions = sorted( + d.name[5:] + for d in minute_dir.iterdir() + if d.is_dir() and d.name.startswith("date=") and (d / "part.parquet").exists() + ) + if snapshot_date: + target = snapshot_date if snapshot_date in partitions else None + else: + target = partitions[-1] if partitions else None + if not target: + return {"status": "no_data", "reason": "minute_missing", "date": snapshot_date, "points": []} + + ext_df, _active = _read_ext_dataframe(config, data_dir) + if ext_df.is_empty() or field not in ext_df.columns: + return {"status": "empty", "reason": "no_members", "date": target, "points": []} + member_bares = _dimension_member_bares(_filter_dimension_member_rows(ext_df, field, value), config) + if not member_bares: + return {"status": "empty", "reason": "no_members", "date": target, "points": []} + + try: + bars = pl.read_parquet( + minute_dir / f"date={target}" / "part.parquet", + columns=["symbol", "datetime", "close"], + ) + except Exception as exc: # noqa: BLE001 + logger.warning("dimension-intraday read minute partition failed: %s", exc) + return {"status": "no_data", "reason": "minute_schema", "date": target, "points": []} + bars = bars.drop_nulls(subset=["datetime", "close"]) + if bars.is_empty(): + return {"status": "no_data", "reason": "minute_empty", "date": target, "points": []} + bars = bars.with_columns(_bare_symbol_expr().alias("_bare")) + + prev = _prev_daily_close(data_dir, target) + joined = bars.join(prev, on="_bare", how="left") if prev is not None else bars.with_columns( + pl.lit(None, dtype=pl.Float64).alias("prev_close") + ) + refs = joined.group_by("_bare").agg( + pl.col("prev_close").first().alias("_prev"), + pl.col("close").sort_by("datetime").first().alias("_first"), + ).with_columns(pl.coalesce(["_prev", "_first"]).alias("_ref")) + n_prev = refs["_prev"].is_not_null().sum() + basis = "prev_close" if n_prev == refs.height else ("first_close" if n_prev == 0 else "mixed") + joined = ( + joined.join(refs.select(["_bare", "_ref"]), on="_bare", how="left") + .with_columns((pl.col("close") / pl.col("_ref") - 1.0).alias("_pct")) + ) + + market = joined.group_by("datetime").agg(pl.col("_pct").mean().alias("_market")) + + member_bars = joined.filter(pl.col("_bare").is_in(member_bares)) + members_with_minute = member_bars["_bare"].n_unique() if not member_bars.is_empty() else 0 + if members_with_minute == 0: + return { + "status": "empty", "reason": "no_member_bars", "date": target, + "member_count": len(member_bares), "members_with_minute": 0, "points": [], + } + if members_with_minute <= _DIMENSION_INTRADAY_FFILL_CAP: + # 网格化 (成分股 × 全时间轴) + 逐股 ffill: 停牌分钟冻结在最后价而非退出均值 + grid = ( + member_bars.select(pl.col("_bare").unique()) + .join(joined.select(pl.col("datetime").unique()), how="cross") + ) + member_bars = ( + grid.join(member_bars.select(["_bare", "datetime", "_pct"]), on=["_bare", "datetime"], how="left") + .sort(["_bare", "datetime"]) + .with_columns(pl.col("_pct").forward_fill().over("_bare")) + ) + sector = member_bars.group_by("datetime").agg(pl.col("_pct").mean().alias("_sector")) + + combined = market.join(sector, on="datetime", how="left").sort("datetime") + + def _r4(v) -> float | None: + return round(float(v), 4) if v is not None and not (isinstance(v, float) and math.isnan(v)) else None + + points = [ + { + "time": row["datetime"].strftime("%H:%M"), + "sector": _r4(row["_sector"]), + "market": _r4(row["_market"]), + } + for row in combined.iter_rows(named=True) + ] + return { + "status": "ok", + "date": target, + "basis": basis, + "member_count": len(member_bares), + "members_with_minute": members_with_minute, + "points": points, + } + + +@router.get("/{config_id}/dimension-intraday") +def dimension_intraday( + request: Request, + config_id: str, + field: str = Query(..., min_length=1), + value: str = Query(..., min_length=1), + snapshot_date: str | None = Query(None, alias="date"), +): + """板块分时走势 (等权): 成分股 × 当日分钟K聚合; 60s 缓存, 点击触发不预计算。""" + config = _store(request).get(config_id) + if not config: + raise HTTPException(404, f"配置 '{config_id}' 不存在") + + cache_key = (config_id, field, value.strip(), snapshot_date) + now = time.monotonic() + hit = _DIMENSION_INTRADAY_CACHE.get(cache_key) + if hit is not None and now - hit[0] < _DIMENSION_INTRADAY_CACHE_TTL_S: + return hit[1] + + payload = _dimension_intraday_compute(config, _data_dir(request), field, value, snapshot_date) + _DIMENSION_INTRADAY_CACHE[cache_key] = (now, payload) + return payload + + # --------------------------------------------------------------------------- # 文件上传 # --------------------------------------------------------------------------- diff --git a/backend/app/api/kline.py b/backend/app/api/kline.py index 279fd0b..b3a61e6 100644 --- a/backend/app/api/kline.py +++ b/backend/app/api/kline.py @@ -10,7 +10,7 @@ from typing import Optional from fastapi import APIRouter, HTTPException, Query, Request from app.indicators.pipeline import compute_enriched, compute_enriched_single -from app.market_time import cn_now, cn_today +from app.market_time import cn_now, cn_today, in_continuous_session from app.price_limits import is_risk_warning_name, price_limit_pct from app.db_safe import is_valid_ext_ident from app.services import kline_sync @@ -780,11 +780,15 @@ def get_minute( request: Request, symbol: str = Query(..., description="标的代码"), trade_date: date | None = Query(None, alias="date", description="交易日期, 默认最新"), + live: bool = Query(False, description="当日盘中跳过本地优先, 直接实时拉取(个股详情分时轮询用)"), ): """读取某只股票某天的分钟 K 线。 - 本地有完整数据(240条) → 直接返回 - 本地无数据或不完整 → 从 TickFlow 实时拉取返回（不写入） + - live=true 且当日连续竞价时段 → 跳过本地优先直接实时拉取: + 盘中分钟增量落盘的本地分区按 ≥60s 轮次更新, 90% 完整度启发式会让 + 详情分时图停在上一增量轮, 与行情列表的节奏脱节 """ repo = request.app.state.repo asset_type = repo.resolve_asset_type(symbol) @@ -834,6 +838,19 @@ def get_minute( price_limit = _get_price_limit_info( repo, symbol, trade_date, asset_type, stock_name, ) + + if live and trade_date == cn_today() and in_continuous_session(): + # 详情分时轮询: 当日盘中实时拉取最新一根K, 不落盘; 拉空(源侧延迟/ + # 时段边界)则落回下方本地优先路径。 + live_df = kline_sync.fetch_minute_single(symbol, trade_date, asset_type=asset_type) + if not live_df.is_empty(): + return { + "symbol": symbol, "name": stock_name, "stock_info": stock_info, + "date": str(trade_date), "rows": live_df.to_dicts(), + "source": "live", "asset_type": asset_type, + "price_limit": price_limit, "prev_close": prev_close, + } + df = repo.get_minute(symbol, trade_date, asset_type=asset_type) # 完整交易日应有 240 条分钟K；如果是今天(盘中)，期望条数按已交易分钟估算 @@ -1078,9 +1095,11 @@ async def clear_minute(request: Request): removed = 0 if minute_dir.exists(): try: - result = repo.db.execute("SELECT COUNT(*) AS cnt FROM kline_minute").fetchone() + # execute_one (cursor+close): 直连 db.execute 的未消费结果集会在 Windows 上 + # 钉住分区句柄, 导致下方 rmtree 静默删不掉被钉文件 + result = repo.execute_one("SELECT COUNT(*) AS cnt FROM kline_minute") removed = result[0] if result else 0 - except Exception: # noqa: BLE001 + except Exception: pass # 仅删 kline_minute 目录, 绝不触碰其他目录 shutil.rmtree(minute_dir, ignore_errors=True) diff --git a/backend/app/api/market_recap.py b/backend/app/api/market_recap.py index 7215d23..9cc1c65 100644 --- a/backend/app/api/market_recap.py +++ b/backend/app/api/market_recap.py @@ -1,4 +1,4 @@ -"""AI 大盘复盘 API — 流式复盘 + 报告持久化。 +"""AI 大盘复盘 API — 流式复盘 + 报告持久化 + 龙虎榜 + 盘前风向标。 路由前缀: /api/market-recap @@ -7,16 +7,19 @@ GET /reports 历史复盘列表 POST /reports 保存一条复盘报告 DELETE /reports/{report_id} 删除一条复盘报告 + GET /dragon-tiger 龙虎榜三榜 (fuyao 专有, 历史按日缓存) + GET /auction-benchmark 盘前风向标 (fuyao 专有, 含当日/次日真实收益) """ from __future__ import annotations import logging +from datetime import date as date_cls -from fastapi import APIRouter, HTTPException, Request +from fastapi import APIRouter, HTTPException, Query, Request from fastapi.responses import StreamingResponse from pydantic import BaseModel -from app.services import market_recap_reports +from app.services import auction_benchmark, dragon_tiger, market_recap_reports from app.services.market_recap import recap_market_stream logger = logging.getLogger(__name__) @@ -30,6 +33,46 @@ class AnalyzeRequest(BaseModel): focus: str = "" # 可选:用户追加的复盘关注点 +@router.get("/dragon-tiger") +def get_dragon_tiger( + request: Request, + date: str | None = Query(default=None, description="复盘目标日 YYYY-MM-DD, 缺省取最近已发布交易日"), +): + """龙虎榜三榜 (全部/机构/游资)。fuyao 专有, 未配置时 state=source_unavailable。 + + 非交易日/当日未发布由服务层自动回退到上一交易日 (state=fallback_prev)。 + """ + target = None + if date: + try: + target = date_cls.fromisoformat(date) + except ValueError: + raise HTTPException(400, f"date 格式应为 YYYY-MM-DD, 收到: {date}") + return dragon_tiger.get_dragon_tiger( + request.app.state.repo.store.data_dir, target + ) + + +@router.get("/auction-benchmark") +def get_auction_benchmark( + request: Request, + date: str | None = Query(default=None, description="复盘目标日 YYYY-MM-DD, 缺省取最近交易日"), +): + """盘前风向标 (同花顺竞价筛选名单 + 当日/次日真实收益)。 + + fuyao 专有, 未配置时 state=source_unavailable; 非交易日由服务层自动回退。 + """ + target = None + if date: + try: + target = date_cls.fromisoformat(date) + except ValueError: + raise HTTPException(400, f"date 格式应为 YYYY-MM-DD, 收到: {date}") + return auction_benchmark.get_auction_benchmark( + request.app.state.repo.store.data_dir, target + ) + + @router.post("/analyze") async def analyze_market(request: Request, req: AnalyzeRequest): """AI 大盘复盘 — NDJSON 流式返回。 diff --git a/backend/app/api/monitor_rules.py b/backend/app/api/monitor_rules.py index 0d9286d..139296e 100644 --- a/backend/app/api/monitor_rules.py +++ b/backend/app/api/monitor_rules.py @@ -107,6 +107,11 @@ class RuleModel(BaseModel): # ladder 专属 (连板梯队封单监控) metric: str = "sealed_vol" # sealed_vol=封单量(手) | sealed_amount=封单额(元) threshold: float = 0 # 封单 <= 此值时报警 (原始单位: 量=手, 额=元) + # volume_delta 专属 (轮询放量监控): 相邻两次全市场快照的成交量增量 + threshold_volume: float = 9000 # 单轮增量 >= 此值(手)时报警 + threshold_amount: float = 1e6 # metric=amount 时: 单轮增量 >= 此值(元)时报警 + # 基础过滤 (与策略 basic_filter 语义对齐): 值为 null 表示不过滤 + basic_filter: dict = {} # ── 字段选项 ───────────────────────────────────────────── @@ -160,6 +165,7 @@ def get_options(request: Request): {"key": "strategy", "label": "策略监控"}, {"key": "abnormal", "label": "异动监控"}, {"key": "sector", "label": "板块监控"}, + {"key": "volume_delta", "label": "轮询放量"}, ], "scopes": [ {"key": "symbols", "label": "指定标的"}, diff --git a/backend/app/api/screener.py b/backend/app/api/screener.py index 784d105..fe6af72 100644 --- a/backend/app/api/screener.py +++ b/backend/app/api/screener.py @@ -317,7 +317,10 @@ def run_preset(req: PresetRequest, request: Request): raise HTTPException(status_code=status_code, detail=str(e)) from e safe_data = _safe(asdict(result)) - _update_cache_strategy(data_dir, str(as_of), req.strategy_id, safe_data) + # 分钟周期结果不写入盘后缓存 (strategy_cache 是日线语义, as_of/updated_at + # 混入分钟结果会污染页面秒加载路径)。 + if req.timeframe == "1d": + _update_cache_strategy(data_dir, str(as_of), req.strategy_id, safe_data) return _result_with_ext(safe_data, ext_values) @@ -583,8 +586,8 @@ def run_all(request: Request, body: Optional[dict] = None): elapsed = (time.perf_counter() - t_total) * 1000 logger.info("run_all: total took %.1fms (%d strategies)", elapsed, len(all_ids)) - # 写入策略缓存 (供页面秒加载) - if results: + # 写入策略缓存 (供页面秒加载); 分钟周期结果不落盘 (日线语义缓存) + if results and timeframe == "1d": try: strategy_cache.write_cache(data_dir, str(as_of), results) except Exception: # noqa: BLE001 diff --git a/backend/app/api/settings.py b/backend/app/api/settings.py index c10009e..8b305bb 100644 --- a/backend/app/api/settings.py +++ b/backend/app/api/settings.py @@ -386,6 +386,21 @@ def _realtime_allowed() -> bool: return QuoteService.is_realtime_allowed() +def _minute_history_days() -> int | None: + """当前分钟源的 1 分钟历史深度(交易日); None = 深历史(tickflow 基准)。 + + provider 可选类属性 minute_history_days 声明 (如 stock-sdk = 5, + 免费分时接口只保留最近 5 个交易日); 未声明或走 tickflow 时视为深历史。 + 前端分时档位/默认值据此收窄。 + """ + from app.services import kline_sync, preferences + provider_name = preferences.get_minute_data_provider() + provider, fallback, _err = kline_sync._resolve_minute_provider(provider_name) + if fallback or provider is None: + return None + return getattr(provider, "minute_history_days", None) + + class MinuteSyncPrefs(BaseModel): minute_sync_enabled: bool minute_sync_days: int = 5 @@ -397,6 +412,7 @@ class DataProvidersIn(BaseModel): daily_data_provider: str | None = None adj_factor_provider: str | None = None minute_data_provider: str | None = None + depth5_data_provider: str | None = None realtime_data_provider: str | None = None financial_data_provider: str | None = None @@ -478,14 +494,17 @@ def get_preferences() -> dict: "minute_sync_enabled": preferences.get_minute_sync_enabled(), "minute_sync_days": preferences.get_minute_sync_days(), "minute_sync_segment_days": preferences.get_minute_sync_segment_days(), + "minute_refresh_enabled": preferences.get_minute_refresh_enabled(), + "minute_refresh_interval": preferences.get_minute_refresh_interval(), "daily_data_provider": preferences.get_daily_data_provider(), "adj_factor_provider": preferences.get_adj_factor_provider(), "minute_data_provider": preferences.get_minute_data_provider(), + "minute_history_days": _minute_history_days(), + "depth5_data_provider": preferences.get_depth5_data_provider(), "realtime_data_provider": preferences.get_realtime_data_provider(), "financial_data_provider": preferences.get_financial_provider(), "data_source_job_timeout_s": preferences.get_data_source_job_timeout_s(), "data_source_long_job_timeout_s": preferences.get_data_source_long_job_timeout_s(), - "realtime_watchlist_symbols": preferences.get_realtime_watchlist_symbols(), **preferences.get_realtime_quote_scope(), "pipeline_pull_a_share": preferences.get_pipeline_pull_a_share(), "pipeline_pull_etf": preferences.get_pipeline_pull_etf(), @@ -541,6 +560,31 @@ def list_data_sources() -> dict: } +@router.get("/capability-matrix") +def get_capability_matrix() -> dict: + """能力 x 源路由矩阵: 能力注册表 + 各源能力声明 + 当前路由偏好, 设置页一次拉全。 + + 偏好值经 preferences getters 注入 (自带合法源校验, 非法值回退默认), + TickFlow 当前档位由 tickflow policy 注入 (候选按档位过滤), + 组装逻辑在 data_providers.capabilities, 本层保持薄。 + """ + from app.data_providers.capabilities import build_capability_matrix + from app.services import preferences + from app.tickflow import policy + + return build_capability_matrix( + { + "realtime_data_provider": preferences.get_realtime_data_provider(), + "daily_data_provider": preferences.get_daily_data_provider(), + "minute_data_provider": preferences.get_minute_data_provider(), + "depth5_data_provider": preferences.get_depth5_data_provider(), + "adj_factor_provider": preferences.get_adj_factor_provider(), + "financial_data_provider": preferences.get_financial_provider(), + }, + tickflow_tier=policy.base_tier_name(), + ) + + @router.post("/plugin-key") def save_plugin_key(req: PluginKeyIn) -> dict: """保存插件 API Key(先探后存, 对齐 /tickflow-key 语义)。 @@ -687,9 +731,8 @@ def delete_data_source(name: str, request: Request) -> dict: updates["realtime_data_provider"] = "tickflow" if preferences.get_financial_provider() == name: updates["financial_data_provider"] = "tickflow" - adj = preferences.get_adj_factor_provider() - if adj == name: - updates["adj_factor_provider"] = "same_as_daily" + if preferences.get_adj_factor_provider() == name: + updates["adj_factor_provider"] = "tickflow" if updates: preferences.save(updates) # 删除源可能触发偏好回退 tickflow, 同步刷新能力快照 @@ -735,6 +778,7 @@ def update_data_providers(req: DataProvidersIn, request: Request) -> dict: "daily_data_provider": preferences.get_daily_data_provider(), "adj_factor_provider": preferences.get_adj_factor_provider(), "minute_data_provider": preferences.get_minute_data_provider(), + "depth5_data_provider": preferences.get_depth5_data_provider(), "realtime_data_provider": preferences.get_realtime_data_provider(), "financial_data_provider": preferences.get_financial_provider(), } @@ -841,6 +885,15 @@ def update_minute_sync(req: MinuteSyncPrefs) -> dict: } +@router.get("/minute-refresh/status") +def minute_refresh_status(request: Request) -> dict: + """盘中分钟增量刷新服务状态 (开关/能力门控/最近一轮/下一轮)。""" + svc = getattr(request.app.state, "minute_refresh", None) + if svc is None: + return {"available": False} + return {"available": True, **svc.status()} + + class RealtimeQuotesPrefs(BaseModel): realtime_quotes_enabled: bool @@ -857,8 +910,8 @@ class RealtimeQuoteScopePrefs(BaseModel): def update_realtime_quotes(req: RealtimeQuotesPrefs, request: Request) -> dict: """保存全局实时行情开关。 - none 档无实时行情权限；free 档开启自选股实时；starter+ 开启全市场实时。 - 前端据此把开关置灰 / 回弹。 + 无实时能力的档位(TickFlow none/free)开关回弹强制关闭; + starter+ 或自定义实时源(如 fuyao)为全市场实时。前端据此把开关置灰 / 回弹。 """ from app.services import preferences qs = getattr(request.app.state, "quote_service", None) @@ -912,10 +965,6 @@ def update_realtime_quotes(req: RealtimeQuotesPrefs, request: Request) -> dict: + f"（任务 {job_id}）" ) raise HTTPException(status_code=409, detail=detail) - if req.realtime_quotes_enabled and qs and qs.realtime_mode() == "watchlist" and not preferences.get_realtime_watchlist_symbols(): - preferences.save({"realtime_quotes_enabled": False}) - _sync_depth_polling(False) - return {"realtime_quotes_enabled": False, "realtime_allowed": True, "mode": "watchlist", "error": "watchlist_empty"} preferences.save({"realtime_quotes_enabled": req.realtime_quotes_enabled}) if qs: @@ -936,18 +985,6 @@ def update_realtime_quote_scope(req: RealtimeQuoteScopePrefs) -> dict: return preferences.set_realtime_quote_scope(cfg) -class RealtimeWatchlistPrefs(BaseModel): - symbols: list[str] = [] - - -@router.put("/preferences/realtime-watchlist") -def update_realtime_watchlist(req: RealtimeWatchlistPrefs) -> dict: - """兼容旧入口；Free 实时标的由自选页前 5 个决定。""" - from app.services import preferences - symbols = preferences.set_realtime_watchlist_symbols(req.symbols) - return {"realtime_watchlist_symbols": symbols} - - class IndicesNavPinnedPrefs(BaseModel): indices_nav_pinned: bool @@ -981,6 +1018,9 @@ class RealtimeMonitorConfigIn(BaseModel): screener_auto_run: bool | None = None minute_intraday_refresh: bool | None = None minute_intraday_refresh_interval: int | None = None + # 盘中分钟增量落盘 (Expert 专有) — 交易时段常驻服务, 归实时监控配置 + minute_refresh_enabled: bool | None = None + minute_refresh_interval: int | None = None monitor_ext_fields: dict | None = None diff --git a/backend/app/backtest/engine.py b/backend/app/backtest/engine.py index b464462..6501fff 100644 --- a/backend/app/backtest/engine.py +++ b/backend/app/backtest/engine.py @@ -813,6 +813,14 @@ class BacktestEngine: matrix, raw_candidates, config, progress_cb, cancel_event, ) + @staticmethod + def _resolve_entry_prices(matrix: "MarketMatrix", config: "MatcherConfig") -> np.ndarray: + """入场价矩阵: 分钟策略的逐格覆盖有限值处优先, 否则按 open/close 惯例。""" + base = matrix.open if config.entry_fill == "open_t+1" else matrix.close + if matrix.entry_price is None: + return base + return np.where(np.isfinite(matrix.entry_price), matrix.entry_price, base) + def _simulate_independent_matrix( self, matrix: MarketMatrix, @@ -823,7 +831,7 @@ class BacktestEngine: options: SimulationOptions | None = None, ) -> SimResult: options = options or SimulationOptions() - entry_prices = matrix.open if config.entry_fill == "open_t+1" else matrix.close + entry_prices = self._resolve_entry_prices(matrix, config) exit_prices = matrix.open if config.exit_fill == "open_t+1" else matrix.close buy_cost_pct = config.buy_cost_pct() sell_cost_pct = config.sell_cost_pct() @@ -1722,7 +1730,7 @@ class BacktestEngine: ) -> SimResult: options = options or SimulationOptions() time_count, asset_count = matrix.shape - entry_prices = matrix.open if config.entry_fill == "open_t+1" else matrix.close + entry_prices = self._resolve_entry_prices(matrix, config) exit_prices = matrix.open if config.exit_fill == "open_t+1" else matrix.close buy_cost_pct = config.buy_cost_pct() sell_cost_pct = config.sell_cost_pct() diff --git a/backend/app/backtest/matrix.py b/backend/app/backtest/matrix.py index d9cfda8..a70c69f 100644 --- a/backend/app/backtest/matrix.py +++ b/backend/app/backtest/matrix.py @@ -551,6 +551,9 @@ class MarketMatrix: exit_signal_code: np.ndarray entry_signal_ids: tuple[str, ...] exit_signal_ids: tuple[str, ...] + # 逐格入场价覆盖 (time x asset, NaN=回退 open/close 惯例)。分钟策略回测用: + # 信号在盘中第 m 根触发, 入场价 = 触发分钟收盘价, 而非当日开盘/收盘。 + entry_price: np.ndarray | None = None @property def shape(self) -> tuple[int, int]: @@ -2302,11 +2305,14 @@ def build_market_matrix_from_signals( exit_delay_bars: int = 0, reference_price: np.ndarray | None = None, minute_exit_trigger: bool = False, + entry_price_override: np.ndarray | None = None, ) -> MarketMatrix: """Combine base data and strategy signals into the matcher input matrix.""" if entry_delay_bars not in (0, 1) or exit_delay_bars not in (0, 1): raise ValueError("phase-two MarketMatrix supports only zero or one bar delay") validate_signal_matrix(signals, market.shape) + if entry_price_override is not None and entry_price_override.shape != market.shape: + raise ValueError("entry_price_override shape does not match MarketDataMatrix") present = _present_matrix(market.open, market.high, market.low, market.close, market.volume) entry, entry_signal_time, entry_signal_code = _delay_signal_matrix( @@ -2379,6 +2385,11 @@ def build_market_matrix_from_signals( exit_signal_code=exit_signal_code, entry_signal_ids=signals.entry_signal_ids, exit_signal_ids=signals.exit_signal_ids, + entry_price=( + np.array(entry_price_override, dtype=np.float32, copy=True) + if entry_price_override is not None + else None + ), ) diff --git a/backend/app/backtest/minute_replay.py b/backend/app/backtest/minute_replay.py new file mode 100644 index 0000000..2e3dd89 --- /dev/null +++ b/backend/app/backtest/minute_replay.py @@ -0,0 +1,286 @@ +"""分钟策略回测回放器 — 逐交易日回放 filter_minute_history 产生入场信号。 + +与实盘选股 (ScreenerService 1m context) 走同一条 StrategyEngine.run 执行路径, +消除回测/实盘偏差。语义铁律: + +- 分钟侧: 传入当日全量分钟分区, 策略函数自身因果 (第 m 根只用 <=m 的K线); +- 日线侧: T 日的日线条件窗口只含 T-1 及更早的完成态日K — 与实盘盘中行为一致 + (当日成形K不进窗口), 杜绝未来函数; +- 按交易日精确对日: 缺分钟分区的日子显式跳过, 不做"回退最近分区" (那是实盘语义)。 +""" + +from __future__ import annotations + +import time +from collections.abc import Callable +from dataclasses import dataclass, field +from datetime import date, timedelta + +import polars as pl + +from app.price_limits import is_risk_warning_name, price_limit_pct +from app.strategy.engine import StrategyDataContext, StrategyDef, StrategyEngine + +# 日线面板列: 基础行情 + 涨停/炸板信号 (策略日线窗口契约) + 基础过滤/展示列。 +# raw_close 用于涨停价计算 (分钟价是未复权真实价, 涨停规则定义在原始价上)。 +MINUTE_DAILY_PANEL_COLUMNS = frozenset({ + "open", "high", "low", "close", "volume", "amount", + "raw_close", "raw_high", "raw_low", + "turnover_rate", + "signal_limit_up", "signal_limit_down", "signal_broken_limit_up", +}) +MINUTE_INSTRUMENT_COLUMNS = frozenset({"name", "total_shares", "float_shares"}) + + +def minute_replay_feature_plan(daily_bars: int): + """分钟回测的日线面板加载计划。 + + execution_backend 用 polars_expr 走"按需计算信号"路径: enriched 分区只落 + 基础列, 涨停/炸板信号由 load_panel_for_backtest 的 compute_limit_signals + 按 signal_columns 需求现算 (matrix_native 路径会跳过通用信号计算)。 + """ + # 函数级导入规避与 strategy.py 的循环依赖 (strategy 顶层导入本模块)。 + from app.backtest.strategy import ResolvedFeaturePlan + + return ResolvedFeaturePlan( + base_columns=MINUTE_DAILY_PANEL_COLUMNS, + intermediate_columns=frozenset(), + indicator_columns=frozenset(), + signal_columns=frozenset({ + "signal_limit_up", "signal_limit_down", "signal_broken_limit_up", + }), + matrix_columns=frozenset(), + instrument_columns=MINUTE_INSTRUMENT_COLUMNS, + warmup_bars=max(daily_bars, 1), + full_feature_fallback=False, + execution_backend="polars_expr", + ) + + +def minute_panel_start(start: date, daily_bars: int) -> date: + """日线面板加载起点: 覆盖首个回测日的 daily_bars 交易日窗口。 + + N 个交易日约需 N*2 自然日 (周末/节假日), 再留 warmup 余量。 + """ + calendar_days = max(daily_bars, 1) * 2 + 30 + return start - timedelta(days=calendar_days) + + +def _trigger_hhmm(value) -> str: + """从 last_datetime 提取北京时间 "HH:MM" 触发分钟。 + + 分区 datetime 为 UTC 存储 (tz-aware 或 naive-UTC), 统一折算到北京时区。 + """ + from app.market_time import CN_TZ + + if hasattr(value, "astimezone"): + if value.tzinfo is None: + from datetime import timezone + + value = value.replace(tzinfo=timezone.utc) + return value.astimezone(CN_TZ).strftime("%H:%M") + text = str(value or "") + if len(text) >= 16 and text[13] == ":": + return text[11:16] + return text[-5:] if text else "" + + +def _scalar_limit_up_price(prev_close: float, limit_pct: float) -> float: + """与 polars_limit_price 同口径的标量涨停价 (整数分半进位)。""" + cents = int(prev_close * 100 + 0.5) + numerator = round((1 + limit_pct) * 100) + return ((cents * numerator + 50) // 100) / 100 + + +@dataclass +class MinuteReplayHit: + """一个盘中入场信号: 触发分钟收盘买入。""" + + trade_date: date + symbol: str + # 已按当日 复权close/原始close 比例折算到复权价系的入场价, 与日线出场价同尺度。 + entry_price: float + trigger_time: str # "HH:MM" — 触发分钟K的时间戳 + score: float = 0.0 + + +@dataclass +class MinuteReplayResult: + hits: list[MinuteReplayHit] = field(default_factory=list) + skipped_days: list[date] = field(default_factory=list) + replayed_days: int = 0 + strategy_matches: int = 0 + buy_limit_up: int = 0 + elapsed_ms: float = 0.0 + + +class MinuteSignalReplayer: + """逐交易日回放分钟策略, 产出与实盘选股同源的入场命中。""" + + def __init__(self, engine, strategy_engine: StrategyEngine) -> None: + # engine: BacktestEngine — 只用其 repo (分钟分区读取)。 + self.engine = engine + self.strategy_engine = strategy_engine + + def replay( + self, + strategy: StrategyDef, + *, + panel: pl.DataFrame, + start: date, + end: date, + params: dict, + overrides: dict, + pool: list[str] | None = None, + symbols: list[str] | None = None, + progress_cb: Callable[[dict], None] | None = None, + cancel_event=None, + ) -> MinuteReplayResult: + t0 = time.perf_counter() + result = MinuteReplayResult() + repo = self.engine.repo + if panel.is_empty(): + return result + + universe = symbols if symbols else panel.get_column("symbol").unique().to_list() + daily_bars = int(strategy.minute_daily_bars or 0) + + # 面板交易日序列 (升序) — 日线窗口切片与缺分区日判定的基准。 + panel_dates = panel.get_column("date").unique().sort().to_list() + date_to_window: dict[date, tuple[date, date]] = {} + for i, day in enumerate(panel_dates): + window_start = panel_dates[max(0, i - daily_bars)] if daily_bars > 0 else day + date_to_window[day] = (window_start, day) + + # 逐分区日回放: 只回放 [start, end] 内有分钟分区的交易日。 + minute_days = repo.list_minute_dates(start, end, "stock") + minute_day_set = set(minute_days) + replay_days = [day for day in panel_dates if start <= day <= end] + result.skipped_days = [day for day in replay_days if day not in minute_day_set] + total = len(minute_days) + + # 逐标的的 T-1 原始收盘/复权收盘查表 (涨停价与复权折算用)。 + prev_raw_close: dict[str, float] = {} + prev_name: dict[str, str] = {} + adj_factor: dict[str, float] = {} + + for i, day in enumerate(minute_days): + if cancel_event is not None and cancel_event.is_set(): + break + if progress_cb is not None: + progress_cb({ + "day": i + 1, + "total": max(total, 1), + "date": str(day), + }) + + history = repo.get_minute_by_dates(universe, [day], "stock") + if history.is_empty(): + result.skipped_days.append(day) + continue + + # 日线窗口: 截至 T-1 的完成态日K (index of last panel date < day)。 + prior = [d for d in panel_dates if d < day] + if not prior: + # 面板起点之前的分区日 (窗口数据不足), 策略按数据不足自然不命中。 + daily_history = pl.DataFrame() + current = pl.DataFrame() + else: + last_prior = prior[-1] + window_start, _ = date_to_window[last_prior] + daily_history = panel.filter( + (pl.col("date") >= window_start) & (pl.col("date") <= last_prior) + ) if daily_bars > 0 else pl.DataFrame() + current = panel.filter(pl.col("date") == last_prior) + + # T-1 收盘/名称 + T 日复权因子查表。 + _refresh_day_lookups(prev_raw_close, prev_name, current, prior) + day_rows = panel.filter(pl.col("date") == day).select( + "symbol", "close", "raw_close", + ) + adj_factor.clear() + adj_factor.update(_adj_factors(day_rows)) + + context = StrategyDataContext( + asset_type="stock", + timeframe="1m", + as_of=day, + current=current if not current.is_empty() else None, + history=history, + daily_history=daily_history if not daily_history.is_empty() else None, + ) + try: + run_result = self.strategy_engine.run( + strategy.meta.get("id", ""), + context, + pool, + params, + overrides, + ) + except ValueError: + # 单日执行失败 (如窗口缺列) 记为跳过, 不中断整个回放。 + result.skipped_days.append(day) + continue + + result.replayed_days += 1 + result.strategy_matches += len(run_result.rows) + for row in run_result.rows: + symbol = row.get("symbol") + close = row.get("close") + if not symbol or close is None or float(close) <= 0: + continue + raw_close = float(close) + name = prev_name.get(str(symbol), "") + prev = prev_raw_close.get(str(symbol)) + # 涨停拒买: 触发分钟收盘已达当日涨停价 (按 T-1 原始收盘 + 板块规则)。 + if prev is not None and prev > 0: + limit_up = _scalar_limit_up_price( + prev, price_limit_pct(str(symbol), day, is_risk_warning=is_risk_warning_name(name)), + ) + if raw_close >= limit_up - 1e-9: + result.buy_limit_up += 1 + continue + trigger = row.get("last_datetime") + trigger_time = _trigger_hhmm(trigger) + result.hits.append(MinuteReplayHit( + trade_date=day, + symbol=str(symbol), + entry_price=raw_close * adj_factor.get(str(symbol), 1.0), + trigger_time=trigger_time, + score=float(run_result.scores.get(str(symbol), 0.0) or 0.0), + )) + + result.elapsed_ms = round((time.perf_counter() - t0) * 1000, 1) + return result + + +def _refresh_day_lookups( + prev_raw_close: dict[str, float], + prev_name: dict[str, str], + prior_snapshot: pl.DataFrame, + prior: list[date], +) -> None: + """从 T-1 快照刷新逐标的原始收盘与名称查表 (涨停价/ST 判定用)。""" + if prior_snapshot.is_empty(): + return + frame = prior_snapshot + if "raw_close" not in frame.columns: + frame = frame.with_columns(pl.col("close").alias("raw_close")) + if "name" not in frame.columns: + frame = frame.with_columns(pl.lit("").alias("name")) + prev_raw_close.clear() + prev_name.clear() + for symbol, raw_close, name in frame.select("symbol", "raw_close", "name").iter_rows(): + prev_raw_close[str(symbol)] = float(raw_close) if raw_close is not None else 0.0 + prev_name[str(symbol)] = str(name or "") + + +def _adj_factors(day_rows: pl.DataFrame) -> dict[str, float]: + """T 日 复权close/原始close 比例: 把分钟原始价折算到复权价系。""" + factors: dict[str, float] = {} + if day_rows.is_empty() or "raw_close" not in day_rows.columns: + return factors + for symbol, close, raw_close in day_rows.select("symbol", "close", "raw_close").iter_rows(): + if close and raw_close and float(raw_close) > 0: + factors[str(symbol)] = float(close) / float(raw_close) + return factors diff --git a/backend/app/backtest/strategy.py b/backend/app/backtest/strategy.py index 9ae6803..7a5d83d 100644 --- a/backend/app/backtest/strategy.py +++ b/backend/app/backtest/strategy.py @@ -28,13 +28,20 @@ from app.backtest.matrix import ( MatrixPipelineConfig, MatrixPrewarmCancelledError, MatrixStrategyPipeline, + SignalMatrix, apply_time_masks, + build_market_data_matrix, build_market_matrix, build_market_matrix_from_signals, rolling_mean, slice_market_data_matrix, slice_signal_matrix, ) +from app.backtest.minute_replay import ( + MinuteSignalReplayer, + minute_panel_start, + minute_replay_feature_plan, +) from app.backtest.minute_trigger import unsupported_minute_exit_signals from app.config import settings from app.indicators.pipeline import ( @@ -995,7 +1002,7 @@ class StrategyBacktestService: s, StrategyDataContext( asset_type=config.asset_type, - timeframe="1d", + timeframe="1m" if s.execution_backend == "minute_filter" else "1d", as_of=config.end, ), ) @@ -1044,6 +1051,26 @@ class StrategyBacktestService: overrides.get("score_max"), ) + if s.execution_backend == "minute_filter": + # 分钟策略回测: 逐交易日回放 filter_minute_history (与实盘选股同源), + # 信号分钟收盘价入场, 之后复用日K矩阵模拟的离场与组合管理。 + return self._run_minute_backtest( + config, s, params, overrides, + stop_loss=stop_loss, + take_profit=take_profit, + trailing_stop=trailing_stop, + trailing_take_profit_activate=trailing_take_profit_activate, + trailing_take_profit_drawdown=trailing_take_profit_drawdown, + max_hold_days=max_hold_days, + score_min=score_min, + score_max=score_max, + progress_cb=progress_cb, + cancel_event=cancel_event, + result_policy=result_policy, + run_id=run_id, + t0=t0, + ) + try: if s.execution_backend == "composite": # composite 回测: 子策略必须全为 matrix_native(否则 fail-closed), @@ -1629,6 +1656,300 @@ class StrategyBacktestService: elapsed_ms=round(elapsed, 1), ) + # ── 分钟策略回测: 逐日回放入场 + 日K矩阵离场 ── + + def _run_minute_backtest( + self, + config: StrategyBacktestConfig, + s: StrategyDef, + params: dict, + overrides: dict, + *, + stop_loss, + take_profit, + trailing_stop, + trailing_take_profit_activate, + trailing_take_profit_drawdown, + max_hold_days, + score_min, + score_max, + progress_cb, + cancel_event, + result_policy: BacktestResultPolicy, + run_id: str, + t0: float, + ) -> StrategyBacktestResult: + def _err(msg: str) -> StrategyBacktestResult: + return StrategyBacktestResult( + run_id=run_id, + config=self._config_to_dict(config), + error=msg, + elapsed_ms=(time.perf_counter() - t0) * 1000, + ) + + if config.asset_type != "stock": + return _err("分钟策略回测当前仅支持 A 股 (stock)") + if config.exit_fill == "signal_next_minute": + return _err("分钟策略回测暂不支持「信号触发卖出」离场口径") + + minute_days = self.engine.repo.list_minute_dates(config.start, config.end, "stock") + if not minute_days: + earliest = self.engine.repo.earliest_minute_date() + hint = f"本地分钟K最早到 {earliest}, " if earliest else "本地无分钟K数据, " + return _err( + f"回测区间内无分钟K数据: {hint}请先用「扩展分钟K历史」拉取, 或开启盘中分钟增量" + ) + + # 日线面板一次加载: 覆盖首个回测日的日线窗口 + 模拟区间 (含 full 模式尾部)。 + daily_bars = int(s.minute_daily_bars or 0) + feature_plan = minute_replay_feature_plan(daily_bars) + load_start = minute_panel_start(config.start, daily_bars) + full_horizon_days = int(max_hold_days or config.holding_days or 5) + load_end = config.end + if config.mode == "full": + load_end = config.end + timedelta(days=(full_horizon_days + 5) * 2) + sim_end = load_end if config.mode == "full" else config.end + + timing_ms: dict[str, float] = {} + t_load = time.perf_counter() + try: + panel = self.engine.load_panel_for_backtest( + config.symbols, + load_start, + load_end, + feature_plan, + asset_type="stock", + ) + except (ValueError, OSError, pl.exceptions.PolarsError) as e: + return _err(f"回测特征准备失败: {e}") + timing_ms["load_panel"] = round((time.perf_counter() - t_load) * 1000, 1) + if panel.is_empty(): + return _err("无日线数据, 请检查日期范围或先运行盘后管道") + + replayer = MinuteSignalReplayer(self.engine, self.strategy_engine) + replay = replayer.replay( + s, + panel=panel, + start=config.start, + end=config.end, + params=params, + overrides=overrides, + symbols=config.symbols, + progress_cb=progress_cb, + cancel_event=cancel_event, + ) + timing_ms["minute_replay"] = replay.elapsed_ms + if cancel_event is not None and cancel_event.is_set(): + return StrategyBacktestResult( + run_id=run_id, + config=self._config_to_dict(config), + error="cancelled", + elapsed_ms=round((time.perf_counter() - t0) * 1000, 1), + ) + if not replay.hits: + skipped_hint = ( + f" (区间内 {len(replay.skipped_days)} 个交易日缺分钟K分区被跳过)" + if replay.skipped_days else "" + ) + return _err("在指定区间内未产生买入信号" + skipped_hint) + + # 日频信号网格: 正式区间面板 → time x asset 矩阵, 命中格写入入场价覆盖。 + sim_panel = panel.filter( + (pl.col("date") >= config.start) & (pl.col("date") <= sim_end) + ) + if sim_panel.is_empty(): + return _err("正式回测区间内无数据") + axis_dates = sim_panel.get_column("date").unique().sort().to_list() + # 轴顺序必须与 build_market_data_matrix 的 _encode_axes 一致 (unique().sort()), + # 否则 (time, asset) 下标指向错误的标的。 + axis_symbols = sim_panel.get_column("symbol").cast(pl.Utf8).unique().sort().to_list() + time_index = {day: i for i, day in enumerate(axis_dates)} + asset_index = {sym: i for i, sym in enumerate(axis_symbols)} + shape = (len(axis_dates), len(axis_symbols)) + + entry = np.zeros(shape, dtype=np.uint8) + score = np.zeros(shape, dtype=np.float32) + entry_price_override = np.full(shape, np.nan, dtype=np.float32) + trigger_times: dict[tuple[str, date], str] = {} + dropped_axis_hits = 0 + for hit in replay.hits: + time_id = time_index.get(hit.trade_date) + asset_id = asset_index.get(hit.symbol) + if time_id is None or asset_id is None: + dropped_axis_hits += 1 + continue + entry[time_id, asset_id] = 1 + score[time_id, asset_id] = hit.score + entry_price_override[time_id, asset_id] = hit.entry_price + trigger_times[(hit.symbol, hit.trade_date)] = hit.trigger_time + raw_candidates = int(entry.sum()) + entry.setflags(write=False) + score.setflags(write=False) + entry_price_override.setflags(write=False) + exit_mask = np.zeros(shape, dtype=np.uint8) + exit_mask.setflags(write=False) + codes = np.zeros(shape, dtype=np.int16) + codes.setflags(write=False) + signals = SignalMatrix( + entry=entry, + exit=exit_mask, + score=score, + entry_signal_code=codes, + exit_signal_code=codes, + entry_signal_ids=(), + exit_signal_ids=(), + ) + + matcher_config = MatcherConfig( + matching=config.matching, + entry_fill="close_t", + exit_fill=config.exit_fill, + fees_pct=config.fees_pct, + commission_pct=config.commission_pct, + stamp_tax_pct=config.stamp_tax_pct, + slippage_bps=config.slippage_bps, + stop_loss_pct=stop_loss, + take_profit_pct=take_profit, + trailing_stop_pct=trailing_stop, + trailing_take_profit_activate_pct=trailing_take_profit_activate, + trailing_take_profit_drawdown_pct=trailing_take_profit_drawdown, + max_hold_days=max_hold_days, + max_positions=config.max_positions, + max_exposure_pct=config.max_exposure_pct, + score_min=score_min, + score_max=score_max, + initial_capital=config.initial_capital, + position_sizing=config.position_sizing, + # 分钟策略的成交价由 entry_price_override 提供 (触发分钟收盘), + # 不再叠加日线口径的分钟成交细化。 + minute_fill=False, + ) + + t_matrix = time.perf_counter() + market_data = build_market_data_matrix(sim_panel) + market_matrix = build_market_matrix_from_signals( + market_data, + signals, + # 入场即信号日盘中 (分钟价覆盖), 离场沿用日K口径。 + entry_delay_bars=0, + exit_delay_bars=1 if matcher_config.exit_fill == "open_t+1" else 0, + entry_price_override=entry_price_override, + ) + timing_ms["matrix_build"] = round((time.perf_counter() - t_matrix) * 1000, 1) + del sim_panel, market_data + + t_sim = time.perf_counter() + if config.mode == "full": + result = self.engine.simulate_independent_market_matrix( + market_matrix, + raw_candidates, + matcher_config, + progress_cb, + cancel_event, + result_policy.simulation_options(), + ) + else: + result = self.engine.simulate_market_matrix( + market_matrix, + matcher_config, + progress_cb, + cancel_event, + result_policy.simulation_options(), + ) + timing_ms["simulate"] = round((time.perf_counter() - t_sim) * 1000, 1) + timing_ms["statistics"] = float(result.stats.pop("statistics_ms", 0.0)) + + if cancel_event is not None and cancel_event.is_set(): + return StrategyBacktestResult( + run_id=run_id, + config=self._config_to_dict(config), + error="cancelled", + elapsed_ms=round((time.perf_counter() - t0) * 1000, 1), + ) + if result.stats.get("error"): + return _err(result.stats["error"]) + + execution = result.stats.get("execution") or {} + execution["buy_limit_up"] = int(execution.get("buy_limit_up", 0)) + replay.buy_limit_up + result.stats["execution"] = execution + timing_ms["total"] = round((time.perf_counter() - t0) * 1000, 1) + result.stats["timing_ms"] = timing_ms + result.stats["panel_rows"] = int(len(axis_dates) * len(axis_symbols)) + result.stats["panel_columns"] = 0 + result.stats["feature_columns"] = 0 + result.stats["execution_backend"] = s.execution_backend + result.stats["selection"] = { + "strategy_matches": replay.strategy_matches, + "entry_candidates": raw_candidates, + "entry_trigger_filtered": max(replay.strategy_matches - raw_candidates, 0), + "entry_trigger_enabled": False, + } + result.stats["minute_replay"] = { + "replayed_days": replay.replayed_days, + "skipped_days": [str(day) for day in replay.skipped_days[:50]], + "skipped_day_count": len(replay.skipped_days), + "dropped_axis_hits": dropped_axis_hits, + } + + benchmark_curve = ( + self._build_benchmark_curve(config.start, config.end) + if result_policy.include_benchmark + else [] + ) + strategy_info = { + "id": s.meta.get("id", config.strategy_id), + "name": s.meta.get("name", config.strategy_id), + "description": s.meta.get("description", ""), + "entry_signals": [], + "exit_signals": [], + "stop_loss": stop_loss, + "take_profit": take_profit, + "trailing_stop": trailing_stop, + "trailing_take_profit_activate": trailing_take_profit_activate, + "trailing_take_profit_drawdown": trailing_take_profit_drawdown, + "max_hold_days": max_hold_days, + "full_horizon_days": full_horizon_days, + "score_min": score_min, + "score_max": score_max, + "source": s.source, + "execution_backend": s.execution_backend, + } if result_policy.include_strategy_info else {} + + trades = ( + [self._trade_to_dict(t) for t in result.trades] + if result_policy.include_trades + else [] + ) + # 入场时间戳补分钟: 交易记录携带触发分钟 (HH:MM), 与日线回测的纯日期区分。 + for trade in trades: + entry_text = str(trade.get("entry_date") or "") + try: + key = (str(trade.get("symbol")), date.fromisoformat(entry_text[:10])) + except ValueError: + continue + trigger = trigger_times.get(key) + if trigger: + trade["entry_date"] = f"{entry_text[:10]} {trigger}" + + selected_stats = result_policy.select_stats(result.stats) + elapsed = (time.perf_counter() - t0) * 1000 + return StrategyBacktestResult( + run_id=run_id, + config=self._config_to_dict(config), + stats=selected_stats, + equity_curve=result.equity_curve if result_policy.include_curves else [], + drawdown_curve=result.drawdown_curve if result_policy.include_curves else [], + benchmark_curve=benchmark_curve, + trades=trades, + per_symbol_stats=( + result.per_symbol_stats + if result_policy.include_per_symbol_stats + else [] + ), + strategy_info=strategy_info, + elapsed_ms=round(elapsed, 1), + ) + # ── 全量模拟 (选股能力统计, 不建组合不算净值) ── def _run_full_simulation( diff --git a/backend/app/backtest/worker.py b/backend/app/backtest/worker.py index be1763f..6b4ca9d 100644 --- a/backend/app/backtest/worker.py +++ b/backend/app/backtest/worker.py @@ -257,9 +257,10 @@ def _worker_entry(task: dict[str, Any], event_queue, cancel_event) -> None: if store is not None: with suppress(Exception): store.db.close() - # 终态消息已入队: 显式冲刷队列后立即退出。大数据量任务跳过解释器 - # teardown (GC、DuckDB 线程 join、DLL 卸载), 否则收尾可达数十秒, - # 会撞上父进程 10s 退出预算。close+join_thread 保证消息完整落管。 + # 终态消息已入队: 显式冲刷队列后立即退出。put 只是入队, 实际写管道的 + # 是后台 feeder 线程, close+join_thread 保证消息完整落管 (否则父进程误判 + # "exited without result"); 大数据量任务再跳过解释器 teardown (GC、DuckDB + # 线程 join、DLL 卸载), 否则收尾可达数十秒, 撞上父进程 10s 退出预算。 with suppress(Exception): event_queue.close() event_queue.join_thread() @@ -320,6 +321,21 @@ def run_worker_task( elif message_type == "error": failure = message + # 子进程退出后, 队列读线程可能尚未把管道尾部的 result/error 搬进本地缓冲 + # (0.1s 轮询在系统高负载下会先看到 Empty+进程已死)。join 后做一次兜底排空, + # 只要消息完整刷入过管道就一定能取到。 + if result is None and failure is None: + for _ in range(2): + try: + message = events.get(timeout=1.0) + except queue.Empty: + break + message_type = message.get("type") + if message_type == "result": + result = message["payload"] + elif message_type == "error": + failure = message + process.join(timeout=10.0) worker_exit_forcibly = False if process.is_alive(): diff --git a/backend/app/data_providers/capabilities.py b/backend/app/data_providers/capabilities.py new file mode 100644 index 0000000..c6bb1d1 --- /dev/null +++ b/backend/app/data_providers/capabilities.py @@ -0,0 +1,201 @@ +"""能力注册表与能力路由矩阵 — 数据集维度的单一权威定义。 + +能力 (capability) = 一个标准化数据集 (CONTRIBUTING「数据源插件化要求」): +daily / adj_factor / realtime / minute / depth5 / financial (注册表顺序即设置页卡片顺序)。注册表集中声明每个 +能力的展示元数据、路由偏好字段与 TickFlow 档位要求, 前端设置页不再各自硬编码。 +depth5 目前仅 TickFlow 供 (插件数据集白名单未开放, 见 loader), 仍进矩阵是为了 +可用性门控诚实: 五档不可用时连板梯队封单/看板封单缺数据应有提示。 + +build_capability_matrix 把注册表、插件/自定义源的能力声明 (datasets) 和当前 +路由偏好合并为一个矩阵, 供设置页一次拉全。当前偏好由 API 层注入 +(preferences getters 自带合法源校验), 本模块不反向依赖 services 层。 + +候选契约: 每个能力的 candidates 只包含「当前确实可提供该能力」的源 — +TickFlow 按当前订阅档位过滤 (日K全档位, 其余按注册表 tf_tier 门槛), +未就绪的插件/自定义源 (依赖未装/Key 未配) 放入 pending 并携带原因, +供前端置灰提示。其他页面可以把 candidates 直接当作可用提供方名单。 + +usable 契约: 每个能力额外给出 usable = 生效源当前能否真正提供该能力 +(生效源在 candidates 中)。各页面的能力门控 (缺能力提示 → 数据源配置) +统一以 usable 为准, 而不是 TickFlow 套餐视角 — 路由到可用插件时同样可用, +路由到 TickFlow 但档位不足时同样不可用。 +""" + +from __future__ import annotations + +from app.data_providers import custom as custom_sources + +CAPABILITY_REGISTRY: list[dict] = [ + { + "id": "daily", + "label": "日K", + "desc": "历史K线与实时覆写", + "field": "daily_data_provider", + "default": "tickflow", + "tf_tier": "none", + }, + { + "id": "adj_factor", + "label": "除权因子", + "desc": "前复权计算基准", + "field": "adj_factor_provider", + "default": "tickflow", + "tf_tier": "starter", + # 独立路由 (曾经的「跟随日K」特殊值已下线: 每个能力单独配置, + # 复权口径一致性改由未来的一致性警示保障, 不做路由耦合) + }, + { + "id": "realtime", + "label": "实时行情", + "desc": "全市场实时快照", + "field": "realtime_data_provider", + "default": "tickflow", + "tf_tier": "starter", + }, + { + "id": "minute", + "label": "分钟K", + "desc": "分时图与分钟回测", + "field": "minute_data_provider", + "default": "tickflow", + "tf_tier": "pro", + }, + { + "id": "depth5", + "label": "五档盘口", + "desc": "连板梯队封单与盘口深度", + "field": "depth5_data_provider", + "default": "tickflow", + "tf_tier": "pro", + # 插件契约暂未开放 depth5 数据集 (loader 白名单), 当前仅 TickFlow 供 + }, + { + "id": "financial", + "label": "财务数据", + "desc": "财务指标与三大报表", + "field": "financial_data_provider", + "default": "tickflow", + "tf_tier": "expert", + }, + { + "id": "full_minute", + "label": "全量分钟", + "desc": "盘中全市场当日分钟落盘 (冷启动全天 + 标的池增量)", + "field": None, + "default": "tickflow", + "tf_tier": "expert", + # intraday.universe 能力 (TickFlow Expert 专有): 插件契约不开放此数据集, + # 生效源恒为 TickFlow — 不可路由, 无对应 provider 偏好字段 + }, +] + +_TICKFLOW_CANDIDATE = { + "name": "tickflow", + "display": "TickFlow", + "kind": "builtin", + "available": True, + "status": "ok", + "note": None, +} + +# 档位排序: none 最低 (无 Key/无效 Key, 仅免费通道历史日K), 未知档按 none 处理 (fail-closed) +_TIER_RANK = {"none": -1, "free": 0, "starter": 1, "pro": 2, "expert": 3} + + +def _tier_base(tier: str) -> str: + """归一化档位输入为基础名: "Pro +" -> "pro"; 空值归为 none。""" + text = str(tier or "").strip().lower() + if not text: + return "none" + return text.split()[0].split("+")[0] + + +def _declared_sources() -> list[dict]: + """插件 + 自定义源 → 统一能力声明视图。未注册 (hidden/加载失败) 的源不会出现。""" + rows: list[dict] = [] + for plugin in custom_sources.list_plugins(): + rows.append({ + "name": plugin["name"], + "display": plugin.get("display_name") or plugin["name"], + "datasets": set(plugin.get("datasets") or []), + "available": bool(plugin.get("available")), + "status": str(plugin.get("status") or ""), + "kind": "plugin", + }) + for source in custom_sources.list_sources(): + rows.append({ + "name": source["name"], + "display": source.get("display_name") or source["name"], + "datasets": set(source.get("datasets") or []), + # 自定义源注册即已通过加载校验, 视为可用 + "available": True, + "status": "ok", + "kind": "custom", + }) + return rows + + +def _display_of(sources: list[dict], name: str) -> str: + if name == "tickflow": + return "TickFlow" + for s in sources: + if s["name"] == name: + return s["display"] + return name + + +def build_capability_matrix(current: dict[str, str], tickflow_tier: str = "none") -> dict: + """注册表 + 源能力声明 + 当前偏好 → 能力路由矩阵。 + + current 为 {偏好字段: 当前值}, 由 API 层经 preferences getters 注入; + getters 已把非法值 (未注册源) 回退为默认, 这里直接信任。effective + 即当前值本身 (每个能力独立路由, 无跟随/派生特殊值)。 + + tickflow_tier 为 TickFlow 当前档位基础名 (none/free/starter/pro/expert), + 由 API 层从 tickflow policy 注入。当前档位不提供的能力里 TickFlow + 不进候选, 但偏好仍指向 tickflow 时以 tf_available=False 标记, + 供前端提示「档位不足」。未知档按 none 处理。 + """ + tier_base = _tier_base(tickflow_tier) + tier_rank = _TIER_RANK.get(tier_base, -1) + sources = _declared_sources() + + capabilities = [] + for cap in CAPABILITY_REGISTRY: + # field=None → 不可路由能力 (仅 TickFlow 提供, 无路由偏好), 生效源恒为默认 + effective = current.get(cap["field"], cap["default"]) if cap["field"] else cap["default"] + tf_available = tier_rank >= _TIER_RANK[cap["tf_tier"]] + candidates: list[dict] = [] + pending: list[dict] = [] + if tf_available: + candidates.append(dict(_TICKFLOW_CANDIDATE)) + for s in sources: + if cap["id"] not in s["datasets"]: + continue + entry = { + "name": s["name"], + "display": s["display"], + "kind": s["kind"], + "available": s["available"], + "status": s["status"], + "note": None if s["available"] else (s["status"] or "不可用"), + } + (candidates if s["available"] else pending).append(entry) + usable = any(c["name"] == effective for c in candidates) + capabilities.append({ + "id": cap["id"], + "label": cap["label"], + "desc": cap["desc"], + "field": cap["field"], + "default": cap["default"], + "tf_tier": cap["tf_tier"], + "tf_available": tf_available, + "usable": usable, + "current": effective, + "current_display": _display_of(sources, effective), + "effective": effective, + "effective_display": _display_of(sources, effective), + "candidates": candidates, + "pending": pending, + }) + return {"tickflow_tier": tier_base, "capabilities": capabilities} diff --git a/backend/app/data_providers/custom/config.py b/backend/app/data_providers/custom/config.py index cf4b732..e5004b3 100644 --- a/backend/app/data_providers/custom/config.py +++ b/backend/app/data_providers/custom/config.py @@ -37,6 +37,9 @@ class DatasetConfig: end_param: str = "end_time" asset_type_param: str | None = None freq_param: str | None = None + # realtime 比例字段(change_pct/amplitude/turnover_rate)的单位声明: + # "percent"(返回 3.66 表示 3.66%)或 "decimal"(返回 0.0366 表示 3.66%)。 + pct_unit: str | None = None @dataclass(frozen=True) @@ -75,6 +78,10 @@ def _dataset_from_dict(raw: dict[str, Any]) -> DatasetConfig: if not 0 < timeout <= MAX_TIMEOUT: raise ValueError(f"timeout must be between 0 and {MAX_TIMEOUT:g} seconds") + pct_unit = str(raw.get("pct_unit") or "").strip().lower() or None + if pct_unit not in (None, "percent", "decimal"): + raise ValueError(f"pct_unit must be 'percent' or 'decimal', got {pct_unit!r}") + return DatasetConfig( url=str(raw.get("url", "") or ""), method=str(raw.get("method", "GET") or "GET").upper(), @@ -91,6 +98,7 @@ def _dataset_from_dict(raw: dict[str, Any]) -> DatasetConfig: end_param=str(raw.get("end_param", "end_time") or "end_time").strip() or "end_time", asset_type_param=(str(raw.get("asset_type_param") or "").strip() or None), freq_param=(str(raw.get("freq_param") or "").strip() or None), + pct_unit=pct_unit, ) diff --git a/backend/app/data_providers/custom/loader.py b/backend/app/data_providers/custom/loader.py index 4f3aa1d..6f71d83 100644 --- a/backend/app/data_providers/custom/loader.py +++ b/backend/app/data_providers/custom/loader.py @@ -11,6 +11,7 @@ from pathlib import Path import yaml +from app import secrets_store from app.config import settings from app.data_providers.custom.config import ( DEFAULT_TIMEOUT, @@ -89,6 +90,19 @@ def list_plugins() -> list[dict]: return list(_PLUGIN_STATUS.values()) +def _plugin_key_masked(name: str, api_key_env: str) -> str: + """插件当前生效 Key 的脱敏串 (secrets.json 优先, .env 兜底), 未配置返回空。 + + 与 TickFlow Key 的展示契约一致 (settings API 的 tickflow_api_key_masked): + 完整 Key 永不出后端, 只出 mask() 结果, 供设置页常驻显示。 + """ + env = str(api_key_env or "").strip() + if not env: + return "" + key = secrets_store.get_env_backed_secret(f"{name.lower()}_api_key", env) + return secrets_store.mask(key) if key else "" + + def plugin_manifest(name: str) -> dict | None: """读取指定插件的 plugin.yaml 清单。""" plugin_dir = plugins_dir() / (name or "") @@ -353,6 +367,7 @@ def _config_to_dict(config: CustomSourceConfig) -> dict: } if ds_name != "realtime" else {}), **({"asset_type_param": ds.asset_type_param} if ds_name == "minute" and ds.asset_type_param else {}), **({"freq_param": ds.freq_param} if ds_name == "minute" and ds.freq_param else {}), + **({"pct_unit": ds.pct_unit} if ds_name == "realtime" and ds.pct_unit else {}), } return out @@ -476,6 +491,13 @@ def _sanitize_dataset(ds_name: str, ds_cfg: dict) -> dict: out["start_param"] = start_param if end_param: out["end_param"] = end_param + pct_unit = str(ds_cfg.get("pct_unit") or "").strip().lower() + if pct_unit: + if ds_name != "realtime": + raise ValueError(f"{ds_name}: pct_unit 仅用于 realtime 数据集") + if pct_unit not in ("percent", "decimal"): + raise ValueError(f"{ds_name}: pct_unit 必须是 percent 或 decimal") + out["pct_unit"] = pct_unit if ds_name == "minute": asset_type_param = str(ds_cfg.get("asset_type_param") or "").strip() freq_param = str(ds_cfg.get("freq_param") or "").strip() @@ -552,7 +574,9 @@ def _register_one_plugin(manifest: dict) -> None: "status": reason, "description": manifest.get("description", ""), "install_hint": manifest.get("install_hint", ""), + "homepage": manifest.get("homepage", ""), "api_key_env": manifest.get("api_key_env", ""), + "api_key_masked": _plugin_key_masked(name, manifest.get("api_key_env", "")), } if not available: return # 依赖没装: 不注册, 但状态已记录供 UI 显示 diff --git a/backend/app/data_providers/custom/provider.py b/backend/app/data_providers/custom/provider.py index 0d6f031..346e894 100644 --- a/backend/app/data_providers/custom/provider.py +++ b/backend/app/data_providers/custom/provider.py @@ -34,29 +34,56 @@ _REQUIRED = { "financial": {"symbol"}, } -# 小数制下 change_pct/amplitude/turnover_rate 的物理上限: A股最大涨跌停 30% (+容差)。 +# 小数制下 change_pct 的物理上限: A股最大涨跌停 30% (+容差)。 # 中位数口径下小数制批次不可能超过该值, 百分制批次(典型中位数 0.5~3)必然超过。 +# 仅对 change_pct 有效——amplitude/turnover_rate 的两种单位在数值区间上重叠 +# (百分制 0.05 = 0.05% 与小数制 0.05 = 5%), 无物理依据可判。 _PCT_FRACTION_MAX = 0.31 +_PCT_COLUMNS = ("change_pct", "amplitude", "turnover_rate") -def _normalize_pct_units(df: pl.DataFrame) -> pl.DataFrame: - """百分制源自适应归一为小数制 (契约: change_pct/amplitude/turnover_rate 为小数, - 0.0366 = 3.66%)。不少第三方接口(如 a-stock-data)直接返回 3.66 表示 3.66%, - 若不归一, 下游(行业/概念统计、前端 x100 展示)会整体放大 100 倍。 - 截面判定: 样本 >= 5 用 |值| 中位数(对个别无涨跌幅限制新股免疫), - 小样本退用最大值。整批同除 100, 避免逐值阈值在 0.3~1 区间的歧义。 +def _normalize_pct_units( + df: pl.DataFrame, + pct_unit: str | None = None, + transformed_cols: frozenset[str] = frozenset(), +) -> pl.DataFrame: + """比例字段单位归一为契约小数制 (change_pct/amplitude/turnover_rate, + 0.0366 = 3.66%, CONTRIBUTING §3.1)。单位只认显式声明, 不靠数值猜: + + - pct_unit="percent" → 三列无条件 /100 (声明即契约, 即使数值看着像小数制); + - pct_unit="decimal" → 原样透传 (即使数值看着像百分制也不动); + - 未声明 → change_pct 保留截面中位数判定(涨跌停 30% 上限使其物理可判: + 样本 >= 5 用 |值| 中位数, 小样本退用最大值, 整批同除 100); + amplitude/turnover_rate 置 None 交下游重算(enriched 管道按 + high/low/prev_close 与股本口径重算), 除非该列已被 transforms 显式 + 处理过(视为用户已接管单位, 原样透传)。 """ - for col in ("change_pct", "amplitude", "turnover_rate"): + dropped_undeclared = False + for col in _PCT_COLUMNS: if col not in df.columns: continue df = df.with_columns(pl.col(col).cast(pl.Float64, strict=False).alias(col)) - vals = df[col].drop_nulls().abs() - if vals.is_empty(): - continue - stat = vals.median() if vals.len() >= 5 else vals.max() - if stat > _PCT_FRACTION_MAX: + if pct_unit == "percent": df = df.with_columns((pl.col(col) / 100).alias(col)) + elif pct_unit == "decimal" or col in transformed_cols: + continue + elif col == "change_pct": + vals = df[col].drop_nulls().abs() + if vals.is_empty(): + continue + stat = vals.median() if vals.len() >= 5 else vals.max() + if stat > _PCT_FRACTION_MAX: + df = df.with_columns((pl.col(col) / 100).alias(col)) + else: + df = df.with_columns(pl.lit(None, dtype=pl.Float64).alias(col)) + dropped_undeclared = True + if dropped_undeclared: + logger.warning( + "自定义源 realtime 未声明 pct_unit: amplitude/turnover_rate 的单位" + "无法从数值判定, 已置 None 交由下游按股本/价格口径重算;" + "请在 realtime 数据集配置中显式声明 pct_unit: percent 或 decimal" + ) return df @@ -82,6 +109,11 @@ class GenericHTTPProvider: missing = sorted(required - mapped) if missing: errors.append(f"{dataset}: missing mapped fields: {', '.join(missing)}") + if cfg.pct_unit is not None: + if dataset != "realtime": + errors.append(f"{dataset}: pct_unit 仅用于 realtime 数据集") + elif cfg.pct_unit not in ("percent", "decimal"): + errors.append(f"{dataset}: pct_unit 必须是 percent 或 decimal") if dataset != "realtime": request_params = [cfg.symbols_param, cfg.start_param, cfg.end_param] if dataset == "minute": @@ -146,8 +178,13 @@ class GenericHTTPProvider: cfg = self._dataset("realtime") rows = self._request_rows(cfg) df = self._mapped_frame(cfg, rows) - # 百分制源(返回 3.66 表示 3.66%)截面归一为契约小数制 - df = _normalize_pct_units(df) + # 单位归一: 显式 pct_unit 声明优先; 未声明时 amplitude/turnover_rate + # fail-closed 置 None(交下游重算), change_pct 保留截面判定 + df = _normalize_pct_units( + df, + pct_unit=cfg.pct_unit, + transformed_cols=frozenset(cfg.transforms) & set(_PCT_COLUMNS), + ) if df.is_empty(): return [] return df.to_dicts() diff --git a/backend/app/jobs/daily_pipeline.py b/backend/app/jobs/daily_pipeline.py index 840f860..3dc47f5 100644 --- a/backend/app/jobs/daily_pipeline.py +++ b/backend/app/jobs/daily_pipeline.py @@ -303,8 +303,6 @@ def run_now( written_adj = 0 affected_symbols: list[str] = [] adj_provider = _prefs.get_adj_factor_provider() - if adj_provider == "same_as_daily": - adj_provider = _prefs.get_daily_data_provider() can_sync_adj = capset.has(Cap.ADJ_FACTOR) or adj_provider != "tickflow" if can_sync_adj: from datetime import datetime, timedelta diff --git a/backend/app/main.py b/backend/app/main.py index a65e130..801fa96 100644 --- a/backend/app/main.py +++ b/backend/app/main.py @@ -173,6 +173,16 @@ async def _application_lifespan(app: FastAPI): except Exception as e: # noqa: BLE001 logger.warning("depth_service init failed: %s", e) + # 盘中分钟增量刷新 (Expert 专有): 线程常驻, 开关/时段/能力门控在循环内每轮判断 + try: + from app.services.minute_refresh import MinuteRefreshService + minute_refresh = MinuteRefreshService(repo) + minute_refresh.set_app_state(app.state) + app.state.minute_refresh = minute_refresh + minute_refresh.start() + except Exception as e: + logger.warning("minute_refresh init failed: %s", e) + # 停机缺口自检: 延迟后台扫描, 发现最近交易日的盘中快照/缺口时自动创建 # 修复任务 (盘中停机→次日开实时场景, 不修则坏数据被"只刷今天"分支永久留存) try: @@ -356,6 +366,9 @@ async def _application_lifespan(app: FastAPI): wbot = getattr(app.state, "wecom_bot_service", None) if wbot: wbot.stop() + mrs = getattr(app.state, "minute_refresh", None) + if mrs: + mrs.stop() logger.info("shutdown") diff --git a/backend/app/market_time.py b/backend/app/market_time.py index f98835a..d25e212 100644 --- a/backend/app/market_time.py +++ b/backend/app/market_time.py @@ -28,6 +28,15 @@ def cn_today() -> date: return datetime.now(CN_TZ).date() +def in_continuous_session(now: datetime | None = None) -> bool: + """A股连续竞价时段 (北京时间): 9:30-11:30 / 13:00-15:00, 仅工作日。""" + now = now or cn_now() + return now.weekday() < 5 and ( + _MORNING_START <= now.time() <= _MORNING_END + or _AFTERNOON_START <= now.time() <= _AFTERNOON_END + ) + + def trading_minutes_elapsed_from_dt(dt: datetime) -> float: """根据北京时间 datetime 计算当日已交易分钟数。 diff --git a/backend/app/plugins/fuyao/client.py b/backend/app/plugins/fuyao/client.py index 821f97b..b090871 100644 --- a/backend/app/plugins/fuyao/client.py +++ b/backend/app/plugins/fuyao/client.py @@ -1,12 +1,17 @@ """扶摇(同花顺金融数据 API) HTTP 客户端。 -职责: 认证、统一信封解包、分页拉取快照。不知道 provider / services 层。 +职责: 认证、统一信封解包、快照分页、单标的日K、市场 dump 下载。不知道 provider / services 层。 文档: https://fuyao.aicubes.cn/docs — REST + X-api-key, 响应信封 {code, message, data}。 + +时间字段口径: 所有 *ms 字段(含 start/end 入参与 date_ms/ex_date_ms 出参)均为 +北京时间零点对应的 epoch ms(= UTC 前一日 16:00), 由 provider 层统一 +8h 换算。 """ + from __future__ import annotations import logging import time +from pathlib import Path import httpx @@ -14,8 +19,9 @@ logger = logging.getLogger(__name__) BASE_URL = "https://fuyao.aicubes.cn" -# A 股约 5400 只, 500/页约 11 页; 50 页上限防御 count 异常导致的死循环。 -_SNAPSHOT_PAGE_SIZE = 500 +# 单页 6000 覆盖全市场(实测 ~5600 含北交所, 2026-08 服务端不截断 limit=6000), +# 一次请求拉完; 分页循环兜底未来标的扩容或服务端改为截断的场景。 +_SNAPSHOT_PAGE_SIZE = 6000 _SNAPSHOT_MAX_PAGES = 50 _PAGE_INTERVAL_S = 0.15 # 页间隔, 降低触发限频 (code=4001) 的概率 @@ -59,7 +65,9 @@ class FuyaoClient: return payload.get("data") or {} # ---- 快照 ---- - def snapshot_page(self, limit: int = _SNAPSHOT_PAGE_SIZE, offset: int = 0) -> tuple[list[dict], int]: + def snapshot_page( + self, limit: int = _SNAPSHOT_PAGE_SIZE, offset: int = 0 + ) -> tuple[list[dict], int]: """拉取一页 A 股全市场快照。返回 (rows, total), total 为全市场总数。 实测响应(2026-08): data={timestamp, total, item}; 官方文档示例为 @@ -106,3 +114,167 @@ class FuyaoClient: if not out: raise FuyaoError("全市场快照为空") return out, server_ts + + # ---- 历史日K ---- + def historical_kline( + self, thscode: str, start_ms: int, end_ms: int, adjust: str = "none" + ) -> list[dict]: + """单标的日K(interval=1d 固定)。单次窗口 ≤10 年, 超出由调用方分片。 + + adjust 必须显式传 "none" 取原始价 — 服务端默认是 forward(前复权), + 官方前复权序列事件间存在逐日漂移, 项目内禁止使用。 + 返回 data.item 原始行: {date_ms, open_price, high_price, low_price, + close_price, volume(股), turnover(元)}。 + """ + data = self._get( + "/api/a-share/prices/historical", + { + "thscode": thscode, + "interval": "1d", + "adjust": adjust, + "start": int(start_ms), + "end": int(end_ms), + }, + ) + rows = data.get("item") + return rows if isinstance(rows, list) else [] + + # ---- 财务 ---- + # 端点均单标的(thscode 不接受逗号)。取数模式二选一: limit=最近N期 或 start/end 区间, + # 这里只用 limit。period=quarterly 覆盖每个季度末(含年报期), 与项目"各报告期累积"口径一致。 + _STATEMENT_ENDPOINTS = { + "income": "income-statements", + "balance_sheet": "balance-sheets", + "cash_flow": "cash-flow-statements", + } + + def financial_statements( + self, stmt: str, thscode: str, limit: int = 1 + ) -> list[dict]: + """单标的财务报表多期序列。stmt: income | balance_sheet | cash_flow。 + + 返回 data.item 原始行: 共有元数据(thscode/period/fiscal_year/fiscal_period/ + report_date_ms/period_end_ms/currency) + 各表字段。行内 null 表示该期未披露。 + """ + endpoint = self._STATEMENT_ENDPOINTS.get(stmt) + if endpoint is None: + raise FuyaoError(f"未知财务报表类型: {stmt}") + data = self._get( + f"/api/a-share/financials/{endpoint}", + {"thscode": thscode, "period": "quarterly", "limit": max(1, min(20, limit))}, + ) + rows = data.get("item") + return rows if isinstance(rows, list) else [] + + def financial_indicators(self, thscode: str, report: str) -> list[dict]: + """单标的单报告期财务指标(report 格式 yyyy-N, N=1..4 对应一季报..年报)。 + + 返回 data.abilities 原始列表 [{ability, indicators: [{index_id, value}]}]; + value 为保留原始精度的数值字符串(百分制指标即百分点数), 缺失为 null。 + 未披露报告期实测返回 code=5003(文档写 3002, 以实测为准) → 经 _get 抛 FuyaoError, + 由调用方按"该期无数据"处理。 + """ + data = self._get( + "/api/a-share/financials/indicators", + {"thscode": thscode, "report": report}, + ) + abilities = data.get("abilities") + return abilities if isinstance(abilities, list) else [] + + def valuations_snapshot(self, thscodes: list[str]) -> list[dict]: + """批量估值快照(pe_ttm/pe_mrq/pb_mrq/ps_ttm/pcf_ttm), 数值为最新口径。 + + 服务端单次上限 100 只(超出 code=1003), 分批由调用方负责。 + 返回 data.item 原始行。 + """ + data = self._get( + "/api/a-share/valuations/snapshot", + {"thscodes": ",".join(thscodes[:100])}, + ) + rows = data.get("item") + return rows if isinstance(rows, list) else [] + + def price_snapshot_batch(self, thscodes: list[str]) -> list[dict]: + """按 thscodes 批量行情快照(最新价等), 用于估值推导的分母。 + + 与全市场分页快照同一端点; thscodes 显式传入时不分页。 + 返回 data.item 原始行。 + """ + data = self._get( + "/api/a-share/prices/snapshot", + {"thscodes": ",".join(thscodes[:100])}, + ) + rows = data.get("item") + return rows if isinstance(rows, list) else [] + + def trading_days(self) -> list[dict]: + """近一年 A 股交易日序列 (固定窗口 [今日-1年, 今日], 无入参)。 + + 返回 data.item 原始行: {date_ms(上海零点), date(yyyyMMdd)}。 + 供交易日探针判定「今天在列表内 ⇔ 交易日」。 + """ + data = self._get("/api/a-share/calendar/trading-days", {}) + rows = data.get("item") + return rows if isinstance(rows, list) else [] + + def dragon_tiger_list(self, board_type: str = "all", date: str | None = None) -> dict: + """龙虎榜榜单 (特色数据)。board_type: all | org | hot_money。 + + 返回 data 原始容器: {trade_date, count, stock_count, stock_items[], + hot_money_items[]}。省略 date 时服务端自动取最近已发布交易日; + 显式传非交易日返回 code=1002 (由调用方做交易日回退)。 + 实测字段(2026-08): stock_items 12 个基础字段, org 榜额外带 4 个机构字段; + 文档中的 limit_reason / amount 实际不返回。 + """ + params: dict = {"board_type": board_type} + if date: + params["date"] = date + return self._get("/api/a-share/special-data/dragon-tiger-list", params) + + def short_term_benchmark(self, date: str | None = None) -> dict: + """短线风向标竞价基准 (同花顺竞价筛选, 每日约 5~6 只)。 + + 返回 data 原始容器: {date, date_ms, item[]}。item 行: + {thscode, ticker, name, auction_pct, tags[]}。支持一年内历史日期; + 显式传非交易日返回 code=1002 (由调用方做交易日回退)。 + """ + params: dict = {} + if date: + params["date"] = date + return self._get("/api/a-share/auction/short-term-benchmark", params) + + # ---- 市场 dump ---- + def dump_download_url(self, dump_kind: str) -> dict: + """获取 dump 预签名下载信息(约 300s 有效)。 + + dump_kind: adjustment-factors | daily-k-10d | daily-k。 + 返回 {presigned_url, presigned_url_expires_at, expires_in_seconds}; + release 版本号(如 20260828)嵌在 presigned_url 的 releases// 路径中, + 供调用方做缓存版本管理。 + """ + return self._get(f"/api/dump/market-dumps/{dump_kind}/download-url", {}) + + def download_dump(self, dump_kind: str, dest: Path) -> Path: + """下载 dump 到 dest(先写 .part 临时文件, 成功后原子改名)。失败抛 FuyaoError。 + + 预签名 URL 指向对象存储, 请求不得携带 X-api-key 头 → 用独立裸请求, + 不经过持有认证头的 self._http。 + """ + url = str(self.dump_download_url(dump_kind).get("presigned_url") or "") + if not url: + raise FuyaoError(f"dump {dump_kind} 未返回预签名 URL") + dest.parent.mkdir(parents=True, exist_ok=True) + tmp = dest.with_name(dest.name + ".part") + try: + with httpx.stream("GET", url, timeout=120.0, follow_redirects=True) as resp: + if resp.status_code != 200: + raise FuyaoError(f"dump {dump_kind} 下载失败 HTTP {resp.status_code}") + with open(tmp, "wb") as fh: + for chunk in resp.iter_bytes(1 << 20): + fh.write(chunk) + tmp.replace(dest) + except httpx.HTTPError as e: + raise FuyaoError(f"dump {dump_kind} 下载网络失败: {e}") from e + finally: + tmp.unlink(missing_ok=True) + return dest diff --git a/backend/app/plugins/fuyao/plugin.yaml b/backend/app/plugins/fuyao/plugin.yaml index aef54dd..cc063f7 100644 --- a/backend/app/plugins/fuyao/plugin.yaml +++ b/backend/app/plugins/fuyao/plugin.yaml @@ -7,8 +7,8 @@ display_name: "fuyao" runtime: none entry: app.plugins.fuyao.provider:FuyaoProvider check: app.plugins.fuyao.provider:availability -datasets: [realtime] +datasets: [realtime, daily, adj_factor, financial] api_key_env: FUYAO_API_KEY # 声明后设置页提供 Key 输入框(先探后存, secrets.json 优先) -hidden: true # 优化完成前不在数据源页展示; 删除此行即可恢复 -description: "同花顺官方 REST 数据 API。当前提供 A 股全市场实时快照(分页拉取);日K/分钟/财务未接入,自动回退 TickFlow。" +description: "同花顺官方 REST 数据 API。A 股实时快照、日K(原始价: 近端走 10d dump, 深窗口走 10 年全量 dump 一次下载秒级筛选, 兜底单标的接口)、除权因子(事件 dump + 本地日K dump 配价推导, 涨跌停自检, 秒级)、财务(利润表/资产负债表/现金流量表/指标, 字段映射至项目口径, bps 由估值反推; 股本无上游接口, 指标历史期建议切回 TickFlow 补齐);分钟未接入,自动回退 TickFlow。" install_hint: "点击卡片中的输入框配置 API Key(https://fuyao.aicubes.cn 申请),或在 .env 中配置 FUYAO_API_KEY" +homepage: "https://fuyao.aicubes.cn" diff --git a/backend/app/plugins/fuyao/provider.py b/backend/app/plugins/fuyao/provider.py index f0c7379..76ce1d9 100644 --- a/backend/app/plugins/fuyao/provider.py +++ b/backend/app/plugins/fuyao/provider.py @@ -3,35 +3,77 @@ 方法签名对齐 custom.GenericHTTPProvider(service 分流点按这套签名调用), 注入 custom loader 注册表后, 各 service 无需改动即可路由到本 provider。 -当前实现数据集: realtime (A 股全市场快照, 分页)。 -未声明 daily / minute / financial → provider_has_dataset 为 False, 自动回退 tickflow。 +实现数据集: + - realtime A 股全市场快照 (分页) + - daily A 股日K, 原始价; 近端窗口走 daily-k-10d 全市场 dump(1 次请求), + 深窗口走单标的 historical 接口(≤10 年/次自动分片) + - adj_factor A 股除权因子; adjustment-factors 事件 dump + 自家原始日K前收盘, + 按交易所公式推导单事件比值, 涨跌停自检 + - financial 财务五表(股本除外): 三表多期序列 + 指标单期, 字段映射为 TickFlow + canonical 列名, 扶摇独有字段原名透传为扩展列; bps 由估值 pb_mrq + 反推; shares 无上游接口恒空 +未声明 minute → provider_has_dataset 为 False, 自动回退 tickflow。 -单位口径 (CONTRIBUTING §3.1, 不可凭字段名推断): +单位与口径 (CONTRIBUTING §3.1, 不可凭字段名推断): - 扶摇 price_change_ratio_pct 为百分数数值 (1.74 = +1.74%), 本项目 realtime change_pct 契约为小数制 (0.0174 = 1.74%) → 此处显式 / 100。 - - volume 单位股、turnover 单位元, 与内部契约一致, 直接透传。 + - 扶摇 volume 单位为股, 本项目日K/实时契约均为手 → 统一 floor(股/100)。 + - turnover 单位元, 与内部一致, 直接透传。 + - 日K取数 adjust=none 锁定: 官方 forward 序列事件间有逐日漂移(2026-08 实测), + 项目内前复权一律由 indicators.pipeline 用本地因子计算。 + - ex_factor 为单事件比值(非累积), 累积链由 pipeline._apply_adj_factor 构建。 """ + from __future__ import annotations +import calendar import contextlib import logging +import math +import re import time +from collections.abc import Callable from dataclasses import dataclass, field +from datetime import UTC, date, datetime, timedelta +from pathlib import Path +import polars as pl + +from app.data_providers.normalizer import DAILY_COLS, normalize_daily +from app.indicators.pipeline import filter_halt_days from app.plugins.fuyao import client as fuyao_client from app.plugins.fuyao.client import FuyaoClient, FuyaoError logger = logging.getLogger(__name__) # 只声明真实提供的数据集; 其余数据集 provider_has_dataset 返回 False → 回退 tickflow -_DATASETS = ("realtime",) +_DATASETS = ("realtime", "daily", "adj_factor", "financial") API_KEY_ENV = "FUYAO_API_KEY" SECRETS_FIELD = "fuyao_api_key" # UI 配置的 Key 存 secrets.json, 优先级高于 .env +# 扶摇 *ms 时间字段为北京时间零点(= UTC 前一日 16:00), +8h 后按 UTC 解析即得交易日 +_SH_MS = 28_800_000 +_HIST_MAX_SPAN_MS = 3650 * 86_400_000 # historical 单次窗口上限 10 年, 超出由本层分片 +_HIST_INTERVAL_S = 0.12 # 单标的请求节流(实测 200+ 连发未触发 4001 限频) +_FINANCIAL_HISTORY_PERIODS = 8 # 财务首装全量历史: 最近 8 期季报(约 2 年) +_VALUATION_BATCH = 100 # 估值/价格快照端点单次上限 100 只 +# 项目财务表名 → 扶摇报表端点名 +_STATEMENT_ENDPOINTS = { + "income": "income-statements", + "balance_sheet": "balance-sheets", + "cash_flow": "cash-flow-statements", +} +_ADJ_DUMP_KIND = "adjustment-factors" +_DAILY10_DUMP_KIND = "daily-k-10d" +_DAILY_DUMP_KIND = "daily-k" # 10 年全量日K dump(约 172MB), 深窗口一次下载覆盖全市场 +_RECENT_DUMP_DAYS = 12 # 窗口跨度 ≤ 此天数时优先走 10d dump(覆盖 ≈10 个交易日) +_PREV_CLOSE_BACKDAYS = 30 # 推导因子时向前找"除权日前收盘"的回看天数(容忍长期停牌) + def get_api_key() -> str: from app import secrets_store + return secrets_store.get_env_backed_secret(SECRETS_FIELD, API_KEY_ENV) @@ -39,7 +81,8 @@ def availability() -> tuple[bool, str]: """loader 启动自检: API Key 已配置(secrets.json 或 .env)才注册为可切换数据源。不抛异常。""" if get_api_key(): return True, "ok" - return False, f"未配置 {API_KEY_ENV}(可在设置页数据源卡片中直接填写)" + # 状态行会拼在「未配置」标签之后, 文案不再重复"未配置"字样 + return False, f"缺少 API Key(可在下方输入框直接填写,或配置环境变量 {API_KEY_ENV})" def probe_api_key(api_key: str) -> tuple[bool, str]: @@ -85,6 +128,128 @@ def _first(row: dict, *names: str): return None +def _date_of_ms(value) -> date | None: + """扶摇 *ms(上海零点) → 交易日。None/非法值返回 None, 不伪造。""" + if value is None: + return None + try: + ms = int(value) + except (TypeError, ValueError): + return None + return datetime.fromtimestamp((ms + _SH_MS) // 1000, tz=UTC).date() + + +def _ms_of_date(d: date) -> int: + """交易日 → 扶摇 start/end 入参口径的 ms(该日上海零点的 epoch ms, 不依赖本机时区)。""" + return (calendar.timegm(d.timetuple()) - 28_800) * 1000 + + +def _iso_of_ms(value) -> str | None: + """扶摇 *ms → ISO 日期字符串(项目财务表 period_end/announce_date 的存储口径)。""" + d = _date_of_ms(value) + return d.isoformat() if d is not None else None + + +def _report_quarter(fiscal_period) -> int | None: + """fiscal_period(Q1..Q4/FY) → 指标接口 report 参数的季号 N(1..4)。""" + if not fiscal_period: + return None + text = str(fiscal_period).strip().upper() + if text == "FY": + return 4 + try: + return int(text.lstrip("Q")) + except ValueError: + return None + + +def _ref_price( + prev_close: float, dividend: float, bonus: float, allot: float, allot_price: float +) -> float | None: + """交易所除权参考价: (P - D + AR·AP) / (1 + S + AR), 四舍五入(half-up)保留 2 位。 + + half-up 是交易所口径; 银行家舍入会让约半数事件在第 2 位小数上偏离(对拍实证)。 + """ + denom = 1.0 + bonus + allot + if denom <= 0: + return None + x = (prev_close - dividend + allot * allot_price) / denom + return math.floor(x * 100 + 0.5) / 100 + + +def _price_limit(symbol: str) -> float: + """按代码前缀给涨跌停幅度(自检容差用): 创业板/科创板 20%, 北交所 30%, 主板 10%。""" + code = symbol.split(".")[0] + if code.startswith(("300", "301", "688", "689")): + return 0.20 + if code.startswith(("8", "4", "92")): + return 0.30 + return 0.10 + + +def _release_of(url: str) -> str: + """从预签名 URL 提取 release 版本号(releases// 路径), 提不到返回 unknown。""" + m = re.search(r"releases/(\d+)/", url or "") + return m.group(1) if m else "unknown" + + +def _cache_dir() -> Path: + from app.config import settings + + d = settings.data_dir / "cache" / "fuyao" + d.mkdir(parents=True, exist_ok=True) + return d + + +def _kline_rows(symbol: str, bars: list[dict]) -> list[dict]: + """historical/dump 原始行(价格元, volume 股) → 内部行(volume 手)。""" + out = [] + for b in bars: + v = _to_float(b.get("volume")) + out.append( + { + "symbol": symbol, + "date": _date_of_ms(b.get("date_ms")), + "open": _to_float(b.get("open_price")), + "high": _to_float(b.get("high_price")), + "low": _to_float(b.get("low_price")), + "close": _to_float(b.get("close_price")), + "volume": math.floor(v / 100.0) if v is not None else None, + "amount": _to_float(b.get("turnover")), + } + ) + return out + + +def _tail_ok(end_d: date, covered_max: date) -> bool: + """请求终点是否被覆盖到 covered_max: 周末/节假日的自然缺口(≤3 天)不算缺失。""" + if end_d <= covered_max: + return True + return (end_d - covered_max).days <= 3 and end_d.weekday() >= 5 + + +def _dump_covers(dump: pl.DataFrame, start_d: date, end_d: date) -> bool: + """dump 日期范围是否覆盖请求窗口: 起点必须落在 dump 内; 终点允许周末自然缺口。""" + if dump.is_empty() or "date_ms" not in dump.columns: + return False + dates = pl.from_epoch(dump["date_ms"].cast(pl.Int64) + _SH_MS, time_unit="ms").dt.date() + dmin, dmax = dates.min(), dates.max() + return start_d >= dmin and _tail_ok(end_d, dmax) + + +def _dump_date_range(path: Path) -> tuple[date | None, date | None]: + """lazy 读 parquet 的 date_ms 边界(走元数据/少量行组, 不整读大文件)。""" + row = ( + pl.scan_parquet(path) + .select( + pl.from_epoch(pl.col("date_ms").min() + _SH_MS, time_unit="ms").dt.date().alias("dmin"), + pl.from_epoch(pl.col("date_ms").max() + _SH_MS, time_unit="ms").dt.date().alias("dmax"), + ) + .collect() + ) + return row["dmin"][0], row["dmax"][0] + + def _map_snapshot_row(row: dict, fetched_ms: int) -> dict | None: """扶摇快照行 → 内部 realtime record。字段缺失时按依赖推导, 不伪造数据。 @@ -108,6 +273,8 @@ def _map_snapshot_row(row: dict, fetched_ms: int) -> dict | None: # 与 quote_service 的推导同口径: 小数制, 不乘 100 change_pct = change_amount / prev + volume = _to_float(row.get("volume")) + return { "symbol": symbol, "name": row.get("name"), # 快照无名称, 由下游维表关联 @@ -116,11 +283,11 @@ def _map_snapshot_row(row: dict, fetched_ms: int) -> dict | None: "open": _to_float(row.get("open_price")), "high": _to_float(_first(row, "high_price", "highest_price")), "low": _to_float(_first(row, "low_price", "lowest_price")), - "volume": _to_float(row.get("volume")), + "volume": math.floor(volume / 100.0) if volume is not None else None, # 股 → 手 "amount": _to_float(row.get("turnover")), "change_pct": change_pct, "change_amount": change_amount, - "amplitude": None, # 快照未提供, 不启发式计算 + "amplitude": None, # 快照未提供, 不启发式计算 "turnover_rate": None, # 需股本口径 (§3.4), 交给 enriched 管道用历史股本计算 "timestamp": fetched_ms, "session": None, @@ -136,18 +303,76 @@ class FuyaoProvider: def __init__(self) -> None: self.config = _FuyaoConfig() self._client: FuyaoClient | None = None + self._dump_memo: dict[str, pl.DataFrame] = {} + self._dump_path_memo: dict[str, Path] = {} def close(self) -> None: # loader.load_all 重建注册表时会对每个 provider 调 close if self._client is not None: with contextlib.suppress(Exception): self._client.close() self._client = None + self._dump_memo.clear() + self._dump_path_memo.clear() def _get_client(self) -> FuyaoClient: if self._client is None: self._client = fuyao_client.FuyaoClient(api_key=get_api_key()) return self._client + # ---- dump 缓存 ---- + def _ensure_dump_path(self, dump_kind: str, cache_prefix: str) -> Path: + """确保最新 release 的 dump 已落盘, 返回缓存路径(大文件不整读进内存)。 + + release 号取自预签名 URL 的 releases// 路径; 新 release 落盘后清理旧版缓存。 + """ + memo = self._dump_path_memo.get(dump_kind) + if memo is not None and memo.exists(): + return memo + client = self._get_client() + info = client.dump_download_url(dump_kind) + release = _release_of(str(info.get("presigned_url") or "")) + dest = _cache_dir() / f"{cache_prefix}__{release}.parquet" + if not dest.exists(): + client.download_dump(dump_kind, dest) + for old in dest.parent.glob(f"{cache_prefix}__*.parquet"): + if old.name != dest.name: + old.unlink(missing_ok=True) + logger.info("扶摇 dump %s(release %s)已下载: %s", dump_kind, release, dest.name) + self._dump_path_memo[dump_kind] = dest + return dest + + def _ensure_dump(self, dump_kind: str, cache_prefix: str) -> pl.DataFrame: + """小体量 dump(快照 10d / 因子)整读 + 进程内 memo, 避免重复打接口/读盘。""" + memo = self._dump_memo.get(dump_kind) + if memo is not None: + return memo + df = pl.read_parquet(self._ensure_dump_path(dump_kind, cache_prefix)) + self._dump_memo[dump_kind] = df + return df + + def _ensure_daily_big_dump(self, start_d: date) -> Path | None: + """10 年全量日K dump(约 172MB)。只要求覆盖窗口起点; 末端缺口由 10d dump 补。 + + 已有缓存覆盖起点就直接复用, 不追新 release(避免深窗口高频触发时日日重下 + 172MB — 旧 release 的中段历史不会变, 尾部新鲜度交给 1MB 的 10d dump)。 + """ + for f in sorted(_cache_dir().glob("daily_k__*.parquet"), reverse=True): + try: + dmin, _ = _dump_date_range(f) + except Exception: # 缓存损坏不致命, 换下一个/重拉 + continue + if dmin is not None and start_d >= dmin: + return f + try: + path = self._ensure_dump_path(_DAILY_DUMP_KIND, "daily_k") + except FuyaoError as e: + logger.warning("扶摇 10 年 dump 不可用, 回退单标的接口: %s", e) + return None + dmin, _ = _dump_date_range(path) + if dmin is None or start_d < dmin: + return None # 窗口比 10 年更早 → 单标的兜底 + return path + # ---- realtime ---- def get_realtime(self) -> list[dict]: """全市场实时快照 → 内部 realtime records。失败软返回空列表(不阻断轮询)。""" @@ -175,11 +400,661 @@ class FuyaoProvider: logger.info("扶摇实时行情拉取完成: %d 条(丢弃 %d 行)", len(records), dropped) return records + # ---- daily ---- + def get_daily( + self, + symbols: list[str], + start_time: datetime | None, + end_time: datetime | None, + asset_type: str = "stock", + on_chunk_done: Callable[[int, int], None] | None = None, + ) -> pl.DataFrame: + """A 股日K → 内部契约。原始价(adjust=none 锁定)、volume 股→手(floor /100)。 + + 取数三档(逐级降级, 全部失败才空手而归): + - 近端窗口(跨度 ≤12 天): daily-k-10d dump(约 1MB), 1 次请求覆盖全部标的; + - 深窗口: daily-k 10 年全量 dump(约 172MB, 缓存覆盖起点即复用, 不追新 release), + 末端缺口由 10d dump 补尾 — 全市场深回填从"逐票 34 分钟"降为"一次下载+秒级筛选"; + - 兜底: 单标的 historical 接口(窗口早于 dump 覆盖 / dump 不可用; 10 年自动分片, + 逐标的节流 + 进度回调)。 + """ + if not symbols or asset_type != "stock": + return pl.DataFrame() + end_dt = end_time or datetime.now() + start_dt = start_time or (end_dt - timedelta(days=365)) + start_d, end_d = start_dt.date(), end_dt.date() + + if (end_d - start_d).days <= _RECENT_DUMP_DAYS: + try: + dump = self._ensure_dump(_DAILY10_DUMP_KIND, "daily_k_10d") + if _dump_covers(dump, start_d, end_d): + df = self._daily_from_dump(dump, set(symbols), start_d, end_d) + if on_chunk_done: + on_chunk_done(1, 1) + logger.info("扶摇日K(10d dump)完成: %d 行 [%s ~ %s]", df.height, start_d, end_d) + return df + logger.info("扶摇 10d dump 未覆盖窗口 [%s ~ %s], 尝试 10 年 dump", start_d, end_d) + except FuyaoError as e: + logger.warning("扶摇日K 10d dump 不可用, 尝试 10 年 dump: %s", e) + + df = self._daily_from_big_dump(set(symbols), start_d, end_d) + if df is not None: + if on_chunk_done: + on_chunk_done(1, 1) + logger.info("扶摇日K(10 年 dump)完成: %d 行 [%s ~ %s]", df.height, start_d, end_d) + return df + df = self._daily_from_api(symbols, start_d, end_d, on_chunk_done) + logger.info("扶摇日K(单标的接口)完成: %d 行 [%s ~ %s]", df.height, start_d, end_d) + return df + + def _daily_from_dump( + self, dump: pl.DataFrame, symset: set[str], start_d: date, end_d: date + ) -> pl.DataFrame: + df = dump.with_columns( + pl.from_epoch(pl.col("date_ms") + _SH_MS, time_unit="ms").dt.date().alias("date") + ) + return self._map_daily_dump(df, symset, start_d, end_d) + + def _map_daily_dump( + self, df: pl.DataFrame, symset: set[str], start_d: date, end_d: date + ) -> pl.DataFrame: + """dump 原始行 → 内部日K契约(股→手、字段重命名、停牌过滤)。""" + if "adjusted" in df.columns: + df = df.filter(pl.col("adjusted") == "none") # 防御: dump 变为复权口径时拒绝落库 + out = ( + df.filter( + (pl.col("date") >= start_d) + & (pl.col("date") <= end_d) + & pl.col("thscode").is_in(sorted(symset)) + ) + .with_columns( + (pl.col("volume") / 100.0).floor().alias("volume"), # 股 → 手 + pl.col("thscode").alias("symbol"), + ) + .rename( + { + "open_price": "open", + "high_price": "high", + "low_price": "low", + "close_price": "close", + "turnover": "amount", + } + ) + ) + out = filter_halt_days(out) + cols = [c for c in DAILY_COLS if c in out.columns] + return out.select(cols).sort(["symbol", "date"]) if not out.is_empty() else out.select(cols) + + def _daily_from_big_dump( + self, symset: set[str], start_d: date, end_d: date + ) -> pl.DataFrame | None: + """深窗口主路径: 10 年全量 dump(lazy 按需筛) + 必要时 10d dump 补尾。 + + 覆盖不了(窗口早于 10 年 / dump 拉取失败)返回 None, 由调用方走单标的接口。 + """ + path = self._ensure_daily_big_dump(start_d) + if path is None: + return None + _, dmax = _dump_date_range(path) + big_hi = min(end_d, dmax) + # 窗口/标的过滤下推到 lazy 计划, 只物化需要的行(全量 10 年 ≈ 13.6M 行) + window = ( + pl.scan_parquet(path) + .with_columns( + pl.from_epoch(pl.col("date_ms") + _SH_MS, time_unit="ms").dt.date().alias("date") + ) + .filter( + (pl.col("date") >= start_d) + & (pl.col("date") <= big_hi) + & pl.col("thscode").is_in(sorted(symset)) + ) + .collect() + ) + parts = [self._map_daily_dump(window, symset, start_d, big_hi)] + if not _tail_ok(end_d, dmax): + # 末端缺口(如 10 年 dump 是旧 release, end 是最近交易日): 10d dump 补尾 + try: + ten = self._ensure_dump(_DAILY10_DUMP_KIND, "daily_k_10d") + ten_dates = pl.from_epoch( + ten["date_ms"].cast(pl.Int64) + _SH_MS, time_unit="ms" + ).dt.date() + ten_min, ten_max = ten_dates.min(), ten_dates.max() + if ( + ten_min is not None + and ten_min <= dmax + timedelta(days=1) + and _tail_ok(end_d, ten_max) + ): + tail_start = max(start_d, dmax + timedelta(days=1)) + parts.append(self._daily_from_dump(ten, symset, tail_start, end_d)) + else: + return None # 中段或尾部仍有缺口 → 单标的兜底, 不交缺口数据 + except FuyaoError as e: + logger.warning("扶摇 10d dump 补尾失败: %s", e) + return None + non_empty = [p for p in parts if not p.is_empty()] + if not non_empty: + return pl.DataFrame() + out = pl.concat(non_empty, how="vertical_relaxed") + return out.unique(subset=["symbol", "date"], keep="last").sort(["symbol", "date"]) + + def _daily_from_api( + self, + symbols: list[str], + start_d: date, + end_d: date, + on_chunk_done: Callable[[int, int], None] | None, + ) -> pl.DataFrame: + frames: list[pl.DataFrame] = [] + for i, sym in enumerate(symbols): + rows = self._historical_bars(sym, start_d, end_d) + time.sleep(_HIST_INTERVAL_S) + if rows: + df = normalize_daily(_kline_rows(sym, rows), default_symbol=sym, source=self.name) + if not df.is_empty(): + frames.append(df) + if on_chunk_done: + on_chunk_done(i + 1, len(symbols)) + return pl.concat(frames, how="diagonal_relaxed") if frames else pl.DataFrame() + + def _historical_bars(self, symbol: str, start_d: date, end_d: date) -> list[dict]: + """按 ≤10 年窗口分片拉取单标的原始日K。中途失败软返回已得行, 不抛出。""" + out: list[dict] = [] + s = _ms_of_date(start_d) + e = _ms_of_date(end_d) + while s <= e: + chunk_end = min(e, s + _HIST_MAX_SPAN_MS - 1) + try: + out.extend(self._get_client().historical_kline(symbol, s, chunk_end, adjust="none")) + except FuyaoError as err: + logger.warning("扶摇日K拉取失败 %s [%s ~ %s]: %s", symbol, start_d, end_d, err) + break + if s + _HIST_MAX_SPAN_MS <= e: + time.sleep(_HIST_INTERVAL_S) + s = chunk_end + 1 + return out + + # ---- adj_factor ---- + def get_adj_factors( + self, + symbols: list[str], + start_time: datetime | None, + end_time: datetime | None, + asset_type: str = "stock", + on_chunk_done: Callable[[int, int], None] | None = None, + ) -> pl.DataFrame: + """A 股除权因子 → 内部契约(symbol/trade_date/ex_factor, 单事件比值非累积)。 + + 数据链: adjustment-factors 全量事件 dump → 清洗(时区 +8h / 滤全零 / 同日成分合并) + → 前收盘价从本地日K dump 一次取齐(缺价标的回退单标的接口) → 交易所公式推导 + → 涨跌停自检剔除异常。 + 未来已公告事件无前收盘, 留给滚动增量窗口(15 天)自然补上。 + """ + schema = {"symbol": pl.String, "trade_date": pl.Date, "ex_factor": pl.Float64} + if not symbols or asset_type != "stock": + return pl.DataFrame(schema=schema) + try: + events = self._load_adj_events(set(symbols), start_time, end_time) + except FuyaoError as e: + logger.warning("扶摇除权因子 dump 加载失败: %s", e) + return pl.DataFrame(schema=schema) + if events.is_empty(): + return pl.DataFrame(schema=schema) + + out_rows: list[dict] = [] + syms = sorted(events["symbol"].unique().to_list()) + # 前收盘价优先从本地日K dump 一次取齐(全市场配价从逐标的 ~13 分钟降为秒级), + # 大 dump 不可用或个别标的缺价时回退单标的接口(节流保留)。 + bounds = events.group_by("symbol").agg( + pl.col("ex_date").min().alias("first_ex"), + pl.col("ex_date").max().alias("last_ex"), + ) + closes_by_sym = self._closes_from_dumps(bounds) + fallbacks = 0 + for i, sym in enumerate(syms): + evs = events.filter(pl.col("symbol") == sym).sort("ex_date") + first_ex: date = evs["ex_date"][0] + last_ex: date = evs["ex_date"][-1] + closes = (closes_by_sym or {}).get(sym) + if not closes or min(closes) > first_ex: + # dump 无该标的 / 覆盖不到首个事件前 → 单标的接口兜底 + closes = self._fetch_closes( + sym, first_ex - timedelta(days=_PREV_CLOSE_BACKDAYS), last_ex + ) + time.sleep(_HIST_INTERVAL_S) + fallbacks += 1 + if not closes: + logger.warning("扶摇除权因子: %s 原始日K为空, 跳过其 %d 个事件", sym, evs.height) + continue + days = sorted(closes) + for ev in evs.iter_rows(named=True): + exd: date = ev["ex_date"] + prev_days = [d for d in days if d < exd] + if not prev_days: + continue # 日K窗口未覆盖的远古事件(如 10 年分片边界之外) + p = closes[prev_days[-1]] + ref = _ref_price(p, ev["dividend"], ev["bonus"], ev["allot"], ev["allot_price"]) + if ref is None or ref <= 0: + logger.warning( + "扶摇除权因子: %s %s 参考价无法计算(P=%s D=%s S=%s AR=%s AP=%s), 跳过", + sym, + exd, + p, + ev["dividend"], + ev["bonus"], + ev["allot"], + ev["allot_price"], + ) + continue + factor = p / ref + ex_days = [d for d in days if d >= exd] + if ex_days: # 涨跌停自检: 错误因子会令复权后除权日涨跌幅超出板限 + ret = closes[ex_days[0]] / (p / factor) - 1.0 + if abs(ret) > _price_limit(sym) + 0.02: + logger.warning( + "扶摇除权因子自检剔除 %s %s: 复权后除权日涨跌幅 %.1f%% 超出涨跌停", + sym, + exd, + ret * 100, + ) + continue + out_rows.append({"symbol": sym, "trade_date": exd, "ex_factor": factor}) + if on_chunk_done: + on_chunk_done(i + 1, len(syms)) + if closes_by_sym is not None: + logger.info( + "扶摇除权因子: 本地 dump 配价 %d/%d 标的, 回退接口 %d 标的", + len(closes_by_sym), + len(syms), + fallbacks, + ) + if not out_rows: + return pl.DataFrame(schema=schema) + return ( + pl.DataFrame(out_rows, schema=schema) + .unique(subset=["symbol", "trade_date"], keep="last") + .sort(["symbol", "trade_date"]) + ) + + def _load_adj_events( + self, + symset: set[str], + start_time: datetime | None, + end_time: datetime | None, + ) -> pl.DataFrame: + """事件 dump → 清洗 → 窗口过滤。返回 symbol/ex_date/dividend/bonus/allot/allot_price。 + + 清洗规则(对拍实证, 见 2026-08 验证记录): + - ex_date_ms 为上海零点戳, +8h 转日期; + - 全零事件行(疑似特殊事件, dump 未给成分)过滤; + - 同日拆行(如分红/送转各一行)按成分合并后推导, 顺序不可反; + - 配股但配股价缺失 → 无法推导, 过滤。 + """ + today = datetime.now().date() + df = self._ensure_dump(_ADJ_DUMP_KIND, "adj_factors").rename({"thscode": "symbol"}) + df = ( + df.with_columns( + pl.from_epoch(pl.col("ex_date_ms") + _SH_MS, time_unit="ms") + .dt.date() + .alias("ex_date"), + pl.col("dividend_per_share").fill_null(0.0), + pl.col("per_share_bonus").fill_null(0.0), + pl.col("allotment_ratio").fill_null(0.0), + pl.col("allotment_price").fill_null(0.0), + ) + .filter( + (pl.col("ex_date") <= today) # 未来已公告事件无前收盘, 留给滚动增量 + & ( + (pl.col("dividend_per_share") != 0) + | (pl.col("per_share_bonus") != 0) + | (pl.col("allotment_ratio") != 0) + ) + ) + .group_by("symbol", "ex_date") + .agg( + pl.col("dividend_per_share").sum().alias("dividend"), + pl.col("per_share_bonus").sum().alias("bonus"), + pl.col("allotment_ratio").sum().alias("allot"), + pl.col("allotment_price").max().alias("allot_price"), # 同日拆行共享配股价 + ) + .filter(~((pl.col("allot") > 0) & (pl.col("allot_price") <= 0))) + .filter(pl.col("symbol").is_in(sorted(symset))) + ) + if start_time is not None: + df = df.filter(pl.col("ex_date") >= start_time.date()) + if end_time is not None: + df = df.filter(pl.col("ex_date") <= end_time.date()) + return df.select("symbol", "ex_date", "dividend", "bonus", "allot", "allot_price") + + # ---- financial ---- + # 字段映射: 扶摇原始字段 → 项目 canonical 列名(TickFlow 口径, 前端财务页与回测 + # FUNDAMENTAL_FACTORS 按此消费)。映射表之外的扶摇独有字段以原名透传为扩展列。 + _INCOME_FIELD_MAP = { + "operating_income": "revenue", + "operating_costs": "operating_cost", + "sales_fee": "selling_expense", + "manage_fee": "admin_expense", + "research_and_development_expenses": "rd_expense", + "operating_profit": "operating_profit", + "interest_expenses": "financial_expense", # 近似口径: 扶摇只给利息费用 + "profit_total": "total_profit", + "income_tax_expense": "income_tax", + "net_profit": "net_income", + "parent_holder_net_profit": "net_income_attributable", + "basic_eps": "basic_eps", + } + _BALANCE_FIELD_MAP = { + "assets_total": "total_assets", + "total_current_assets": "total_current_assets", + "non_current_nets_total": "total_non_current_assets", + "cash": "cash_and_equivalents", + "accounts_receivable": "accounts_receivable", + "total_debt": "total_liabilities", + "holder_equity_total": "total_equity", + } + _CASHFLOW_FIELD_MAP = { + "act_cash_flow_net": "net_operating_cash_flow", + "invest_cash_flow_net": "net_investing_cash_flow", + "financing_cash_flow_net": "net_financing_cash_flow", + "pay_fixed_assets_etc_cash": "capex", + "cash_equivalents_net_addition": "net_cash_change", + } + # 官方指标 index_id → canonical。归母净利同比近似 tickflow net_income_yoy; + # 实测 index_id 与文档有出入(calculate_ 前缀等), 以实测为准。 + _METRICS_FIELD_MAP = { + "index_weighted_avg_roe": "roe", + "total_assets_net_ratio": "roa", + "sale_gross_margin": "gross_margin", + "sale_net_interest_ratio": "net_margin", + "assets_debt_ratio": "debt_to_asset_ratio", + "calculate_operating_income_yoy_growth_ratio": "revenue_yoy", + "calculate_parent_holder_net_profit_yoy_growth_ratio": "net_income_yoy", + "operating_cash_flow_net_divide_income": "operating_cash_to_revenue", + "inventory_turnover_ratio": "inventory_turnover", + } + + def get_financials( + self, + table: str, + symbols: list[str], + latest_only: bool = True, + ) -> pl.DataFrame: + """拉取财务数据, 映射为 canonical 列(symbol/period_end/announce_date/指标)。 + + - 三大报表: 单股单请求, latest_only 决定最近 1 期还是 8 期季报; + - metrics: 指标接口为单股单期, 恒只拉最新一期(bps 由估值快照 pb_mrq 反推, + eps_basic 顺带取自利润表); 历史各期建议切回 TickFlow 同步补齐 — + 报告期合并写入会让两源数据共存, 互不覆盖; + - shares: 扶摇无股本接口, 恒返回空(已有存量靠合并写入保留)。 + """ + if table == "shares": + logger.info("扶摇无股本接口, shares 表跳过 (已有数据保留)") + return pl.DataFrame() + if table in _STATEMENT_ENDPOINTS: + field_map = { + "income": self._INCOME_FIELD_MAP, + "balance_sheet": self._BALANCE_FIELD_MAP, + "cash_flow": self._CASHFLOW_FIELD_MAP, + }[table] + return self._financial_statements(table, field_map, symbols, latest_only) + if table == "metrics": + return self._financial_metrics(symbols) + return pl.DataFrame() + + def _financial_statements( + self, + stmt: str, + field_map: dict[str, str], + symbols: list[str], + latest_only: bool, + ) -> pl.DataFrame: + client = self._get_client() + limit = 1 if latest_only else _FINANCIAL_HISTORY_PERIODS + rows_out: list[dict] = [] + for i, sym in enumerate(symbols): + if i: + time.sleep(_HIST_INTERVAL_S) + try: + rows = client.financial_statements(stmt, sym, limit=limit) + except FuyaoError as e: + logger.warning("扶摇财务 %s %s 失败: %s", stmt, sym, e) + continue + for r in rows: + row: dict = { + "symbol": sym, + "period_end": _iso_of_ms(r.get("period_end_ms")), + "announce_date": _iso_of_ms(r.get("report_date_ms")), + } + for src, dst in field_map.items(): + row[dst] = _to_float(r.get(src)) + # 扶摇独有字段以原名透传为扩展列 (canonical 之外的增量信息) + for src, value in r.items(): + if src not in field_map and src not in row and isinstance(value, (int, float)): + row[src] = value + rows_out.append(row) + return pl.DataFrame(rows_out) if rows_out else pl.DataFrame() + + def _financial_metrics(self, symbols: list[str]) -> pl.DataFrame: + client = self._get_client() + # 指标接口按 report(yyyy-N) 单期查询 → 先用利润表 limit=1 反查每股最新披露期 + latest: dict[str, dict] = {} + for i, sym in enumerate(symbols): + if i: + time.sleep(_HIST_INTERVAL_S) + try: + rows = client.financial_statements("income", sym, limit=1) + except FuyaoError as e: + logger.warning("扶摇财务 income %s 失败: %s", sym, e) + continue + if rows: + latest[sym] = rows[0] + if not latest: + return pl.DataFrame() + bps_by_sym = self._derive_bps(sorted(latest)) + rows_out: list[dict] = [] + for sym, r in latest.items(): + quarter = _report_quarter(r.get("fiscal_period")) + report = f"{r.get('fiscal_year')}-{quarter}" if quarter else None + row: dict = { + "symbol": sym, + "period_end": _iso_of_ms(r.get("period_end_ms")), + "announce_date": _iso_of_ms(r.get("report_date_ms")), + "eps_basic": _to_float(r.get("basic_eps")), + "bps": bps_by_sym.get(sym), + } + if report: + try: + abilities = client.financial_indicators(sym, report) + except FuyaoError as e: + logger.warning("扶摇指标 %s %s 失败: %s", sym, report, e) + abilities = [] + for ability in abilities: + for ind in ability.get("indicators") or []: + index_id = ind.get("index_id") + if not index_id: + continue + value = _to_float(ind.get("value")) + if value is not None: + row[self._METRICS_FIELD_MAP.get(index_id, index_id)] = value + rows_out.append(row) + return pl.DataFrame(rows_out) if rows_out else pl.DataFrame() + + def trading_days(self) -> set: + """近一年交易日集合 (供交易日探针)。失败抛 FuyaoError, 由探针兜为未知。""" + rows = self._get_client().trading_days() + return { + d + for d in (_date_of_ms(r.get("date_ms")) for r in rows) + if d is not None + } + + def dragon_tiger(self, board_type: str = "all", date: str | None = None) -> dict: + """龙虎榜单榜 (复盘页卡片 + AI 复盘上下文)。返回原始 data 容器。 + + 非路由数据集 (tickflow 无对应能力), 不进 plugin.yaml datasets, + 由 services.dragon_tiger 统一做三榜聚合/缓存/交易日回退。 + """ + return self._get_client().dragon_tiger_list(board_type, date) + + def short_term_benchmark(self, date: str | None = None) -> dict: + """短线风向标竞价基准 (复盘页卡片 + AI 复盘上下文)。返回原始 data 容器。 + + 非路由数据集 (tickflow 无对应能力), 不进 plugin.yaml datasets, + 由 services.auction_benchmark 统一做按日缓存/收益enrich/交易日回退。 + """ + return self._get_client().short_term_benchmark(date) + + def _derive_bps(self, symbols: list[str]) -> dict[str, float]: + """估值快照 pb_mrq 与行情快照最新价同源同刻 → bps = price / pb_mrq。 + + 与财报口径 bps 可能差几个百分点(上游权益基准不完全透明), 用于补齐 + metrics.bps 使回测 pb_latest 因子可用。接口失败只影响 bps 列, 不致命。 + """ + client = self._get_client() + pb: dict[str, float] = {} + price: dict[str, float] = {} + for i in range(0, len(symbols), _VALUATION_BATCH): + if i: + time.sleep(_HIST_INTERVAL_S) + batch = symbols[i : i + _VALUATION_BATCH] + for fetch, store in ( + (client.valuations_snapshot, pb), + (client.price_snapshot_batch, price), + ): + time.sleep(_HIST_INTERVAL_S) + try: + for r in fetch(batch): + value = _to_float(r.get("pb_mrq" if store is pb else "last_price")) + code = r.get("thscode") + if code and value is not None and value != 0: + store[code] = value + except FuyaoError as e: + logger.warning("扶摇 bps 推导快照失败(%d 只): %s", len(batch), e) + return { + sym: price[sym] / pb[sym] + for sym in symbols + if sym in pb and pb[sym] and sym in price + } + + def _fetch_closes(self, symbol: str, start_d: date, end_d: date) -> dict[date, float]: + rows = self._historical_bars(symbol, start_d, end_d) + out: dict[date, float] = {} + for r in rows: + d = _date_of_ms(r.get("date_ms")) + c = _to_float(r.get("close_price")) + if d is not None and c is not None: + out[d] = c + return out + + def _closes_from_dumps( + self, bounds: pl.DataFrame + ) -> dict[str, dict[date, float]] | None: + """从本地日K dump 一次取齐全部事件标的的收盘价。 + + 每标的开窗 [first_ex-30d, last_ex](与单标的接口同窗): 10 年大 dump 为 + 主体, 10d dump 叠加补末端新鲜度(除权日当天的自检需要 ex 日收盘)。 + 返回 symbol → {date: close}; 大 dump 不可用/窗口早于其覆盖/读盘失败 + → None, 由调用方整轮回退单标的接口。 + """ + try: + lo = bounds["first_ex"].min() - timedelta(days=_PREV_CLOSE_BACKDAYS) + path = self._ensure_daily_big_dump(lo) + if path is None: + return None + sources = [self._closes_scan(pl.scan_parquet(path))] + ten = self._dump_memo.get(_DAILY10_DUMP_KIND) + if ten is not None: + sources.append(self._closes_scan(ten.lazy())) + else: + try: + p10 = self._ensure_dump_path(_DAILY10_DUMP_KIND, "daily_k_10d") + except FuyaoError as e: + logger.info("扶摇 10d dump 不可用, 配价仅用 10 年 dump: %s", e) + else: + sources.append(self._closes_scan(pl.scan_parquet(p10))) + win = bounds.select( + "symbol", + (pl.col("first_ex") - pl.duration(days=_PREV_CLOSE_BACKDAYS)).alias("lo"), + "last_ex", + ) + # concat 顺序 = 叠加优先级: 同 (symbol, date) 时 10d dump(更新鲜)覆盖大 dump + df = ( + pl.concat(sources, how="vertical_relaxed") + .join(win.lazy(), on="symbol", how="inner") + .filter( + (pl.col("date") >= pl.col("lo")) & (pl.col("date") <= pl.col("last_ex")) + ) + .unique(subset=["symbol", "date"], keep="last", maintain_order=True) + .collect() + ) + return { + f["symbol"][0]: dict( + zip(f["date"].to_list(), f["close"].to_list(), strict=True) + ) + for f in df.partition_by("symbol") + } + except Exception as e: # 缓存损坏等不致命: 回退逐标的接口 + logger.warning("扶摇除权因子本地配价失败, 回退单标的接口: %s", e) + return None + + @staticmethod + def _closes_scan(lf: pl.LazyFrame) -> pl.LazyFrame: + """dump 行 → (symbol, date, close) lazy 投影; adjusted 列存在时锁 none。""" + if "adjusted" in lf.collect_schema().names(): + lf = lf.filter(pl.col("adjusted") == "none") + return lf.select( + pl.col("thscode").alias("symbol"), + pl.from_epoch(pl.col("date_ms").cast(pl.Int64) + _SH_MS, time_unit="ms") + .dt.date() + .alias("date"), + pl.col("close_price").cast(pl.Float64).alias("close"), + ).filter(pl.col("close").is_not_null()) + # ---- 测试(设置页试拉) ---- def test_dataset(self, dataset: str, symbols: list[str] | None = None) -> dict: + if dataset in ("daily", "adj_factor"): + syms = [s for s in (symbols or [])][:3] or ["000001.SZ"] + try: + if dataset == "daily": + df = self.get_daily(syms, datetime.now() - timedelta(days=30), datetime.now()) + else: + df = self.get_adj_factors( + syms, datetime.now() - timedelta(days=365), datetime.now() + ) + except FuyaoError as e: + return {"provider": self.name, "dataset": dataset, "rows": 0, "error": str(e)} + head = df.head(5).to_dicts() + for row in head: # date/datetime → ISO 字符串, 保证 JSON 可序列化 + for k, v in list(row.items()): + if isinstance(v, (date, datetime)): + row[k] = v.isoformat() + return { + "provider": self.name, + "dataset": dataset, + "rows": df.height, + "columns": df.columns, + "preview": head, + } + if dataset == "financial": + syms = [s for s in (symbols or [])][:1] or ["600519.SH"] + try: + df = self.get_financials("metrics", syms, latest_only=True) + except FuyaoError as e: + return {"provider": self.name, "dataset": dataset, "rows": 0, "error": str(e)} + head = df.head(5).to_dicts() + return { + "provider": self.name, + "dataset": dataset, + "rows": df.height, + "columns": df.columns, + "preview": head, + } if dataset != "realtime": - return {"provider": self.name, "dataset": dataset, "rows": 0, - "error": f"扶摇插件未接入 {dataset} 数据集(自动回退 TickFlow)"} + return { + "provider": self.name, + "dataset": dataset, + "rows": 0, + "error": f"扶摇插件未接入 {dataset} 数据集(自动回退 TickFlow)", + } try: rows, count = self._get_client().snapshot_page(limit=5) except FuyaoError as e: diff --git a/backend/app/plugins/stocksdk/provider.py b/backend/app/plugins/stocksdk/provider.py index c3481a4..c431fba 100644 --- a/backend/app/plugins/stocksdk/provider.py +++ b/backend/app/plugins/stocksdk/provider.py @@ -12,7 +12,8 @@ from __future__ import annotations import logging from collections.abc import Callable from dataclasses import dataclass, field -from datetime import datetime +from datetime import datetime, timedelta +from datetime import time as dtime import polars as pl @@ -51,6 +52,9 @@ class StockSDKProvider: name = "stocksdk" builtin = True + # 分钟历史深度能力(可选声明, 未声明视为深历史): stock-sdk 免费分时接口 + # 只保留最近 5 个交易日的 1 分钟数据, 分时档位/默认值据此收窄。 + minute_history_days = 5 def __init__(self) -> None: self.config = _StockSDKConfig() @@ -145,29 +149,96 @@ class StockSDKProvider: return pl.DataFrame() period = "".join(ch for ch in str(freq) if ch.isdigit()) or "1" logger.info("stock-sdk minute 拉取开始(%d symbols, period=%s)", len(symbols), period) + + # 上游区间查询的分钟 open 为日级常量(伪值), 单日查询才给最新交易日真实 + # 分钟 open → 末尾 3 个自然日逐日单拉(跳过周末, 覆盖周五收盘后场景), + # 其余历史段仍走单个区间任务控制桥接成本。伪 open 由 _null_degenerate_opens + # 在 _minute_df 内置 null。 + windows: list[tuple[datetime | None, datetime | None]] = [] + if ( + start_time is not None + and end_time is not None + and end_time.date() > start_time.date() + ): + tail_start = end_time.date() - timedelta(days=3) + if tail_start > start_time.date(): + head_end = datetime.combine(tail_start - timedelta(days=1), dtime.max) + windows.append((start_time, head_end)) + else: + tail_start = start_time.date() + day = tail_start + while day <= end_time.date(): + if day.weekday() < 5: + windows.append(( + datetime.combine(day, dtime.min), + datetime.combine(day, dtime.max), + )) + day += timedelta(days=1) + else: + windows.append((start_time, end_time)) + frames: list[pl.DataFrame] = [] chunks = chunked(symbols, _BATCH) - for i, chunk in enumerate(chunks): - job = { - "op": "minute", - "symbols": chunk, - "period": period, - "start": _yyyymmdd(start_time), - "end": _yyyymmdd(end_time), - } - try: - result = bridge.run_job(job, timeout=180) - except bridge.StockSDKBridgeError as e: - logger.warning("stock-sdk minute 拉取失败(%d symbols): %s", len(chunk), e) - result = {"rows": {}} - for sym, rows in (result.get("rows") or {}).items(): - df = self._minute_df(rows, sym) - if not df.is_empty(): - frames.append(df) - if on_chunk_done: - on_chunk_done(i + 1, len(chunks)) + total = len(chunks) * len(windows) + step = 0 + for win_start, win_end in windows: + for chunk in chunks: + step += 1 + job = { + "op": "minute", + "symbols": chunk, + "period": period, + "start": _yyyymmdd(win_start), + "end": _yyyymmdd(win_end), + } + try: + result = bridge.run_job(job, timeout=180) + except bridge.StockSDKBridgeError as e: + logger.warning("stock-sdk minute 拉取失败(%d symbols): %s", len(chunk), e) + result = {"rows": {}} + for sym, rows in (result.get("rows") or {}).items(): + df = self._minute_df(rows, sym) + if not df.is_empty(): + frames.append(df) + if on_chunk_done: + on_chunk_done(step, total) + # 末窗口(最新一日)的真实 open 与首窗口可能重叠同日(时区/边界), keep="last" + # 由上层 _write_minute_partition 的 unique 处理; 这里仅拼接。 return pl.concat(frames, how="diagonal_relaxed") if frames else pl.DataFrame() + @staticmethod + def _null_degenerate_opens(df: pl.DataFrame) -> pl.DataFrame: + """把"日级常量"的伪分钟 open 置 null。 + + stock-sdk 上游对历史日的分钟 open 只给全天常量(如涨跌停价/日开), + 并非真实分钟开盘价; 只有最新交易日在单日查询下给真实值。伪 open 入库 + 会让分钟K的 close-vs-open 口径全偏(如分时量恒红), 故按日检测: + 同日 rows>10 且 open 唯一值<=3 而 close 唯一值>10 → open 判定非分钟级, + 置 null(fail-closed, 不伪造 prev_close 替代)。 + """ + if df.is_empty() or "open" not in df.columns or "datetime" not in df.columns: + return df + stats = df.group_by(pl.col("datetime").dt.date()).agg( + pl.len().alias("n"), + pl.col("open").n_unique().alias("uo"), + pl.col("close").n_unique().alias("uc"), + ) + fake_dates = stats.filter( + (pl.col("n") > 10) & (pl.col("uo") <= 3) & (pl.col("uc") > 10) + )["datetime"] + if fake_dates.is_empty(): + return df + logger.warning( + "stock-sdk minute open 为日级常量, 置 null: %s %s", + df["symbol"][0] if "symbol" in df.columns else "?", fake_dates.to_list(), + ) + return df.with_columns( + pl.when(pl.col("datetime").dt.date().is_in(fake_dates)) + .then(None) + .otherwise(pl.col("open")) + .alias("open") + ) + @staticmethod def _minute_df(rows: list[dict], symbol: str) -> pl.DataFrame: if not rows: @@ -192,6 +263,7 @@ class StockSDKProvider: for col in ("open", "high", "low", "close", "volume", "amount"): if col in df.columns: df = df.with_columns(pl.col(col).cast(pl.Float64, strict=False)) + df = StockSDKProvider._null_degenerate_opens(df) keep = [c for c in _MINUTE_CANONICAL if c in df.columns] return df.select(keep) if "datetime" in keep else pl.DataFrame() diff --git a/backend/app/services/abnormal_moves.py b/backend/app/services/abnormal_moves.py index c324570..7e583b1 100644 --- a/backend/app/services/abnormal_moves.py +++ b/backend/app/services/abnormal_moves.py @@ -223,3 +223,60 @@ def build_overview( "counts": counts, "rows": out_rows[:limit], } + + +# ================================================================ +# 盘中异动 (量价信号聚合, 异动监控「盘中」tab) +# +# 数据源: enriched 最新快照的当日消息号列 (零新增采集): +# 涨停/跌停/跌停翘板/炸板/放量(量比≥2)/创60日新高/新低。 +# 行序 = 信号优先级 (涨停 > 炸板 > 翘板 > 跌停 > 新高 > 新低 > 放量), +# 同级按 |今日涨跌| 降序; counts 供前端筛选 chips 展示各类型数量。 +# ================================================================ + +_INTRADAY_SIGNALS: tuple[tuple[str, str], ...] = ( + ("signal_limit_up", "limit_up"), + ("signal_broken_limit_up", "broken"), + ("signal_limit_down_recovery", "recovery"), + ("signal_limit_down", "limit_down"), + ("signal_n_day_high", "new_high"), + ("signal_n_day_low", "new_low"), + ("signal_volume_surge", "volume_surge"), +) +_INTRADAY_PRIORITY = {key: i for i, (_, key) in enumerate(_INTRADAY_SIGNALS)} +_INTRADAY_COLS = ("symbol", "name", "close", "change_pct", "amplitude", + "vol_ratio_5d", "turnover_rate", "consecutive_limit_ups") + + +def build_intraday(repo: Any, limit: int = 500) -> dict[str, Any]: + """enriched 最新快照 → 当日异动信号命中行 (含各类型计数)。""" + df, cache_date = repo.get_enriched_latest() + empty = {"cache_date": cache_date.isoformat() if cache_date else None, + "counts": {}, "rows": []} + if df.is_empty() or "symbol" not in df.columns: + return empty + present = [(c, k) for c, k in _INTRADAY_SIGNALS if c in df.columns] + if not present: + return empty + + hits = df.filter(pl.any_horizontal([pl.col(c).fill_null(False) for c, _ in present])) + if hits.is_empty(): + return empty + counts = {k: int(hits[c].fill_null(False).sum()) for c, k in present} + + sig_cols = {k: hits[c].fill_null(False).to_list() for c, k in present} + base_cols = [c for c in _INTRADAY_COLS if c in hits.columns] + base = hits.select(base_cols).to_dicts() + rows: list[dict[str, Any]] = [] + for i, r in enumerate(base): + signals = [k for k, flags in sig_cols.items() if flags[i]] + rows.append({ + **{c: r.get(c) for c in base_cols}, + "signals": signals, + "_prio": min((_INTRADAY_PRIORITY[s] for s in signals), default=99), + }) + rows.sort(key=lambda r: (r["_prio"], -abs(r.get("change_pct") or 0.0))) + for r in rows: + r.pop("_prio", None) + return {"cache_date": cache_date.isoformat() if cache_date else None, + "counts": counts, "rows": rows[:limit]} diff --git a/backend/app/services/auction_benchmark.py b/backend/app/services/auction_benchmark.py new file mode 100644 index 0000000..334b5c8 --- /dev/null +++ b/backend/app/services/auction_benchmark.py @@ -0,0 +1,288 @@ +"""短线风向标服务 (fuyao 专有) — 复盘页卡片 + AI 复盘上下文。 + +非路由数据集: tickflow 无对应能力, 直接经 custom_sources 调 fuyao provider; +fuyao 未配置时返回 source_unavailable 状态, 前端降级提示。 + +数据契约 (实测 2026-08-30, 60 交易日回测 353 样本): +- 每日 5~6 只, 服务端筛选的竞价异动股, 附概念标签 +- 名单当日 (开盘买→收盘卖) 均值 +0.54% vs 全市场 +0.10%, 有真实当日选股能力; + 但高开≥5% 子集当日 -1.97% (追高陷阱) → 前端对高开子集标「追高风险」 +- 次日无显著优势 (+0.08%), 定位为「当日观察名单」而非隔夜轮动信号 + +缓存策略 (与 dragon_tiger 同模式): +- 历史名单不可变 → 按日落 JSON 缓存 (data/auction_benchmark/date=YYYY-MM-DD.json), + 缓存命中不触发插件注册表加载 +- 收益 enrich (当日oc/全天/次日) 不落缓存 — 次日数据晚到, 读取时现算 +- 当日名单不缓存 (竞价阶段名单可能变动, 以现拉为准) +- 显式日期失败 → 回退上一交易日一次 (state=fallback_prev) + +日期解析: 接口对显式非交易日报 code=1002, 本层用本地 kline_daily 分区日期 +把目标日回退到「≤目标日的最近交易日」, 规避报错。 +""" + +from __future__ import annotations + +import contextlib +import json +import logging +import re +from datetime import date as date_cls +from pathlib import Path + +import polars as pl + +from app.market_time import cn_today + +logger = logging.getLogger(__name__) + +_DATE_DIR_RE = re.compile(r"^date=(\d{4}-\d{2}-\d{2})$") + + +def _local_trading_days(data_dir: Path) -> list[date_cls]: + """本地日K分区日期 = 已知交易日集合 (升序)。扫描失败返回空。""" + root = data_dir / "kline_daily" + out: list[date_cls] = [] + try: + for d in root.iterdir(): + m = _DATE_DIR_RE.match(d.name) + if d.is_dir() and m: + try: + out.append(date_cls.fromisoformat(m.group(1))) + except ValueError: + continue + except OSError: + return [] + return sorted(out) + + +def resolve_trade_date(data_dir: Path, target: date_cls | None) -> date_cls | None: + """目标日 → ≤目标日的最近本地交易日。None → None (由 fuyao 取当日)。""" + if target is None: + return None + days = _local_trading_days(data_dir) + if not days: + return target + candidates = [d for d in days if d <= target] + return max(candidates) if candidates else target + + +def _prev_trading_day(data_dir: Path, d: date_cls) -> date_cls | None: + days = _local_trading_days(data_dir) + earlier = [x for x in days if x < d] + return max(earlier) if earlier else None + + +def _next_trading_day(data_dir: Path, d: date_cls) -> date_cls | None: + days = _local_trading_days(data_dir) + later = [x for x in days if x > d] + return min(later) if later else None + + +def _provider(): + from app.data_providers import custom as custom_sources + + if not custom_sources.is_custom_provider("fuyao"): + return None + return custom_sources.get_provider("fuyao") + + +def _cache_path(data_dir: Path, d: date_cls) -> Path: + return data_dir / "auction_benchmark" / f"date={d.isoformat()}.json" + + +def _load_cache(path: Path) -> dict | None: + try: + return json.loads(path.read_text(encoding="utf-8")) + except (OSError, ValueError): + return None + + +def _store_cache(path: Path, payload: dict) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + tmp = path.with_name(path.name + ".part") + tmp.write_text(json.dumps(payload, ensure_ascii=False), encoding="utf-8") + tmp.replace(path) + + +def _raw_items(data: dict) -> list[dict]: + items = data.get("item") + return [r for r in items if isinstance(r, dict)] if isinstance(items, list) else [] + + +def _read_kline_closes(data_dir: Path, d: date_cls) -> dict[str, float]: + """某交易日全市场 {symbol: close}; 无分区/读失败返回空。""" + root = data_dir / "kline_daily" / f"date={d.isoformat()}" + try: + files = sorted(root.glob("*.parquet")) + if not files: + return {} + df = pl.concat([pl.read_parquet(f, columns=["symbol", "close"]) for f in files]) + return dict(zip(df["symbol"].to_list(), df["close"].to_list())) + except (OSError, pl.exceptions.PolarsError): + return {} + + +def _enrich(data_dir: Path, trade_date: date_cls, items: list[dict]) -> list[dict]: + """名单叠加真实收益: day0_oc (开盘买→收盘卖) / day0_pct (全天) / d1_pct (次日)。 + + 用相邻 kline_daily 分区现算, 不落缓存 — 次日分区晚到时先给 None, 到了自然补上。 + """ + root = data_dir / "kline_daily" / f"date={trade_date.isoformat()}" + day0: dict[str, tuple[float, float]] = {} # symbol -> (open, close) + try: + files = sorted(root.glob("*.parquet")) + if files: + df = pl.concat([pl.read_parquet(f, columns=["symbol", "open", "close"]) for f in files]) + day0 = dict(zip(df["symbol"].to_list(), zip(df["open"].to_list(), df["close"].to_list()))) + except (OSError, pl.exceptions.PolarsError): + day0 = {} + + prev = _prev_trading_day(data_dir, trade_date) + prev_close = _read_kline_closes(data_dir, prev) if prev else {} + nxt = _next_trading_day(data_dir, trade_date) + next_close = _read_kline_closes(data_dir, nxt) if nxt else {} + + out: list[dict] = [] + for r in items: + sym = str(r.get("thscode") or "") + oc = pct = d1 = None + if sym in day0: + o, c = day0[sym] + if o and o > 0 and c is not None: + oc = c / o - 1.0 + pc = prev_close.get(sym) + if pc and pc > 0 and c is not None: + pct = c / pc - 1.0 + nc = next_close.get(sym) + if c and c > 0 and nc is not None: + d1 = nc / c - 1.0 + out.append({ + "thscode": sym, + "ticker": r.get("ticker"), + "name": r.get("name"), + "auction_pct": r.get("auction_pct"), + "tags": [str(t) for t in (r.get("tags") or [])], + "day0_oc": oc, + "day0_pct": pct, + "d1_pct": d1, + }) + return out + + +def _base_payload(data_dir: Path, trade_date: date_cls, data: dict) -> dict: + return { + "state": "ok", + "requested_date": trade_date.isoformat(), + "trade_date": str(data.get("date") or trade_date.isoformat()), + "count": len(_raw_items(data)), + "raw_items": _raw_items(data), + } + + +def get_auction_benchmark(data_dir: Path, target: date_cls | None = None) -> dict: + """取短线风向标名单 (含当日/次日真实收益)。返回给前端的统一容器。 + + state: ok (正常) | fallback_prev (目标日拉取失败, 已回退上一期) + | source_unavailable (未配置 fuyao) | no_data (拉取失败) + """ + days = _local_trading_days(data_dir) + if target is None: + target = max(days) if days else None + trade_date = resolve_trade_date(data_dir, target) + today = cn_today() + + # 历史日缓存优先 (纯本地, 不触发插件注册表加载) + if trade_date is not None and trade_date < today: + cached = _load_cache(_cache_path(data_dir, trade_date)) + if cached is not None: + return _respond(data_dir, trade_date, cached, cached.get("state") or "ok") + + provider = _provider() + if provider is None: + return {"state": "source_unavailable"} + + from app.plugins.fuyao.client import FuyaoError + + explicit = trade_date.isoformat() if trade_date is not None else None + try: + data = provider.short_term_benchmark(explicit) + try: + actual = date_cls.fromisoformat(str(data.get("date"))) + except ValueError: + actual = trade_date + base = _base_payload(data_dir, actual, data) + # 历史日不可变 → 落缓存; 当日不缓存 (竞价阶段名单可能变动) + if actual is not None and actual < today: + with contextlib.suppress(OSError): + _store_cache(_cache_path(data_dir, actual), base) + return _respond(data_dir, actual, base, "ok") + except FuyaoError as e: + # 显式日期失败 (非交易日/边界日) → 回退上一交易日一次 + if trade_date is not None: + prev = _prev_trading_day(data_dir, trade_date) + if prev is not None: + try: + cached_prev = _load_cache(_cache_path(data_dir, prev)) + if cached_prev is not None: + return _respond(data_dir, prev, cached_prev, "fallback_prev", + requested=explicit) + data_prev = provider.short_term_benchmark(prev.isoformat()) + base = _base_payload(data_dir, prev, data_prev) + with contextlib.suppress(OSError): + _store_cache(_cache_path(data_dir, prev), base) + return _respond(data_dir, prev, base, "fallback_prev", + requested=explicit) + except FuyaoError: + pass + logger.warning("短线风向标拉取失败: %s", e) + return {"state": "no_data", "message": str(e)} + + +def _respond( + data_dir: Path, + trade_date: date_cls | None, + base: dict, + state: str, + requested: str | None = None, +) -> dict: + """缓存/现拉的原始容器 → 叠加收益 enrich 后的前端容器。""" + if trade_date is None: + return {**base, "state": state} + payload = { + "state": state, + "requested_date": requested if requested is not None else base.get("requested_date"), + "trade_date": base.get("trade_date") or trade_date.isoformat(), + "count": base.get("count") or 0, + "items": _enrich(data_dir, trade_date, base.get("raw_items") or []), + } + return payload + + +def build_recap_context(data_dir: Path) -> str: + """AI 复盘的盘前风向标摘要段 (纯文本, 失败返回空串不影响复盘)。""" + try: + payload = get_auction_benchmark(data_dir, None) + if payload.get("state") not in ("ok", "fallback_prev"): + return "" + items = payload.get("items") or [] + if not items: + return "" + trade_date = payload.get("trade_date") or "" + lines = [f"(数据日期: {trade_date})"] + segs = [] + for i in items: + seg = (f"{i.get('name')}({i.get('thscode')}) 竞价{(i.get('auction_pct') or 0):+.2f}%" + f"[{'·'.join(i.get('tags') or [])}]") + if i.get("day0_oc") is not None: + seg += f" → 当日开盘买{i['day0_oc']*100:+.2f}%" + if i.get("d1_pct") is not None: + seg += f", 次日{i['d1_pct']*100:+.2f}%" + segs.append(seg) + lines.append("盘前风向标名单: " + "; ".join(segs)) + ocs = [i["day0_oc"] for i in items if i.get("day0_oc") is not None] + if ocs: + lines.append(f"名单当日(开盘买→收盘卖)均值 {sum(ocs)/len(ocs)*100:+.2f}%") + return "\n".join(lines) + except Exception as e: # noqa: BLE001 — 摘要失败不影响复盘主流程 + logger.debug("盘前风向标复盘摘要构建失败: %s", e) + return "" diff --git a/backend/app/services/dragon_tiger.py b/backend/app/services/dragon_tiger.py new file mode 100644 index 0000000..1d919f8 --- /dev/null +++ b/backend/app/services/dragon_tiger.py @@ -0,0 +1,227 @@ +"""龙虎榜数据服务 (fuyao 专有) — 复盘页卡片 + AI 复盘上下文。 + +非路由数据集: tickflow 无对应能力, 直接经 custom_sources 调 fuyao provider; +fuyao 未配置时返回 source_unavailable 状态, 前端降级提示。 + +数据契约 (实测 2026-08, 文档的 limit_reason/amount 实际不返回): +- 三榜一次取齐: all/org 为股票表 (org 额外带机构 4 字段), hot_money 为席位表 +- 历史榜单不可变 → 按日落 JSON 缓存 (data/dragon_tiger/date=YYYY-MM-DD.json) +- 当日榜单盘中未发布 → 自动回退上一交易日并标记 state=fallback_prev + +日期解析: 接口对显式非交易日报 code=1002, 本层用本地 kline_daily 分区日期 +把目标日回退到「≤目标日的最近交易日」, 规避报错。 +""" + +from __future__ import annotations + +import contextlib +import json +import logging +import re +from datetime import date as date_cls +from pathlib import Path + +from app.market_time import cn_today + +logger = logging.getLogger(__name__) + +_BOARDS = ("all", "org", "hot_money") +_DATE_DIR_RE = re.compile(r"^date=(\d{4}-\d{2}-\d{2})$") + + +def _local_trading_days(data_dir: Path) -> list[date_cls]: + """本地日K分区日期 = 已知交易日集合 (升序)。扫描失败返回空。""" + root = data_dir / "kline_daily" + out: list[date_cls] = [] + try: + for d in root.iterdir(): + m = _DATE_DIR_RE.match(d.name) + if d.is_dir() and m: + try: + out.append(date_cls.fromisoformat(m.group(1))) + except ValueError: + continue + except OSError: + return [] + return sorted(out) + + +def resolve_trade_date(data_dir: Path, target: date_cls | None) -> date_cls | None: + """目标日 → ≤目标日的最近本地交易日。None 或本地无更早分区 → None (由 fuyao 默认)。 + + 目标日早于全部本地分区 (极老的历史复盘) → 原样返回, 由调用方试显式日期, + 失败如实报 no_data, 不静默给错日期的数据。 + """ + if target is None: + return None + days = _local_trading_days(data_dir) + if not days: + return target + candidates = [d for d in days if d <= target] + return max(candidates) if candidates else target + + +def _prev_trading_day(data_dir: Path, d: date_cls) -> date_cls | None: + days = _local_trading_days(data_dir) + earlier = [x for x in days if x < d] + return max(earlier) if earlier else None + + +def _provider(): + from app.data_providers import custom as custom_sources + + if not custom_sources.is_custom_provider("fuyao"): + return None + return custom_sources.get_provider("fuyao") + + +def _fetch_boards(provider, d: date_cls) -> dict: + """三榜取齐。任一榜失败抛 FuyaoError (整日失败, 不留半套缓存)。""" + return { + bt: _boards_of(provider, bt, d.isoformat()) for bt in _BOARDS + } + + +def _boards_of(provider, board_type: str, iso: str | None) -> dict: + data = provider.dragon_tiger(board_type, iso) + return { + "trade_date": data.get("trade_date"), + "stock_count": data.get("stock_count"), + "count": data.get("count"), + "stock_items": data.get("stock_items") or [], + "hot_money_items": data.get("hot_money_items") or [], + } + + +def _cache_path(data_dir: Path, d: date_cls) -> Path: + return data_dir / "dragon_tiger" / f"date={d.isoformat()}.json" + + +def _load_cache(path: Path) -> dict | None: + try: + return json.loads(path.read_text(encoding="utf-8")) + except (OSError, ValueError): + return None + + +def _store_cache(path: Path, payload: dict) -> None: + path.parent.mkdir(parents=True, exist_ok=True) + tmp = path.with_name(path.name + ".part") + tmp.write_text(json.dumps(payload, ensure_ascii=False), encoding="utf-8") + tmp.replace(path) + + +def get_dragon_tiger(data_dir: Path, target: date_cls | None = None) -> dict: + """取龙虎榜 (三榜)。返回给前端的统一容器。 + + state: ok (正常) | fallback_prev (当日未发布, 已回退上一期并带上一期数据) + | source_unavailable (未配置 fuyao) | no_data (拉取失败) + """ + # 无目标日 → 最近本地交易日 (走缓存判定; 本地无任何分区才让 fuyao 自取默认) + if target is None: + days = _local_trading_days(data_dir) + target = max(days) if days else None + trade_date = resolve_trade_date(data_dir, target) + today = cn_today() + + # 历史日缓存优先 (纯本地, 不触发插件注册表加载) + if trade_date is not None and trade_date < today: + cached = _load_cache(_cache_path(data_dir, trade_date)) + if cached is not None: + return cached + + provider = _provider() + if provider is None: + return {"state": "source_unavailable"} + + from app.plugins.fuyao.client import FuyaoError + + # 日期未知 (target=None / 本地无分区) → 省略 date 让 fuyao 取最近已发布交易日 + explicit = trade_date.isoformat() if trade_date is not None else None + try: + raw = {bt: _boards_of(provider, bt, explicit) for bt in _BOARDS} + try: + actual = date_cls.fromisoformat(str(raw["all"].get("trade_date"))) + except ValueError: + actual = None + payload = { + "state": "ok", + "requested_date": explicit, + "trade_date": raw["all"].get("trade_date"), + **raw, + } + # 历史日不可变 → 落缓存; 当日不缓存 (盘中 fallback / 盘后补充都以现拉为准) + if actual is not None and actual < today: + with contextlib.suppress(OSError): + _store_cache(_cache_path(data_dir, actual), payload) + return payload + except FuyaoError as e: + # 显式日期失败 (当日未发布 / 边界日) → 回退上一交易日一次 + if trade_date is not None: + prev = _prev_trading_day(data_dir, trade_date) + if prev is not None: + try: + cached_prev = _load_cache(_cache_path(data_dir, prev)) + if cached_prev is not None: + base = dict(cached_prev) + base.pop("state", None) + return {**base, "state": "fallback_prev", + "requested_date": explicit} + raw_prev = _fetch_boards(provider, prev) + base = { + "requested_date": prev.isoformat(), + "trade_date": prev.isoformat(), + **raw_prev, + } + with contextlib.suppress(OSError): + _store_cache(_cache_path(data_dir, prev), {**base, "state": "ok"}) + return {**base, "state": "fallback_prev", + "requested_date": explicit} + except FuyaoError: + pass + logger.warning("龙虎榜拉取失败: %s", e) + return {"state": "no_data", "message": str(e)} + + +def build_recap_context(data_dir: Path) -> str: + """AI 复盘的龙虎榜摘要段 (纯文本, 失败返回空串不影响复盘)。""" + try: + payload = get_dragon_tiger(data_dir, None) + if payload.get("state") not in ("ok", "fallback_prev"): + return "" + items = payload.get("all", {}).get("stock_items") or [] + org_items = payload.get("org", {}).get("stock_items") or [] + hm_items = payload.get("hot_money", {}).get("hot_money_items") or [] + trade_date = payload.get("trade_date") or "" + lines = [f"(数据日期: {trade_date})"] + + top_buy = sorted( + [i for i in items if (i.get("net_value") or 0) > 0], + key=lambda x: x.get("net_value") or 0, reverse=True, + )[:5] + if top_buy: + lines.append("净买入居前: " + "; ".join( + f"{i.get('name')}({i.get('thscode')}) 净买{float(i.get('net_value') or 0)/1e8:.2f}亿" + f" 涨跌{(i.get('change') or 0)*100:.1f}%" + + (" [3日榜]" if i.get("range_days") == 3 else "") + for i in top_buy)) + top_sell = sorted( + [i for i in items if (i.get("net_value") or 0) < 0], + key=lambda x: x.get("net_value") or 0, + )[:5] + if top_sell: + lines.append("净卖出居前: " + "; ".join( + f"{i.get('name')} 净卖{abs(float(i.get('net_value') or 0))/1e8:.2f}亿" + for i in top_sell)) + if org_items: + lines.append("机构净买居前: " + "; ".join( + f"{i.get('name')} 机构净买{float(i.get('org_net_value') or 0)/1e8:.2f}亿" + for i in sorted(org_items, key=lambda x: x.get("org_net_value") or 0, reverse=True)[:5])) + if hm_items: + lines.append("活跃游资: " + "; ".join( + f"{h.get('name')} 净买{float(h.get('buying') or 0)/1e8:.2f}亿" + for h in sorted(hm_items, key=lambda x: x.get('buying') or 0, reverse=True)[:5])) + return "\n".join(lines) + except Exception as e: # noqa: BLE001 — 摘要失败不影响复盘主流程 + logger.debug("龙虎榜复盘摘要构建失败: %s", e) + return "" diff --git a/backend/app/services/extend_history.py b/backend/app/services/extend_history.py index dc0037a..0249ad4 100644 --- a/backend/app/services/extend_history.py +++ b/backend/app/services/extend_history.py @@ -167,8 +167,6 @@ def run_extend_history( from app.services import preferences as _prefs adj_provider = _prefs.get_adj_factor_provider() - if adj_provider == "same_as_daily": - adj_provider = _prefs.get_daily_data_provider() can_sync_adj = capset.has(Cap.ADJ_FACTOR) or adj_provider != "tickflow" if can_sync_adj: emit("extend_history", 48, f"获取除权因子 [{adj_start_str} ~ {adj_end_str}]…") diff --git a/backend/app/services/financial_sync.py b/backend/app/services/financial_sync.py index 7b857eb..6655496 100644 --- a/backend/app/services/financial_sync.py +++ b/backend/app/services/financial_sync.py @@ -155,6 +155,13 @@ def _sync_table( def _merge_report_history(*frames: pl.DataFrame) -> pl.DataFrame: + """按 (symbol, period_end) 合并各报告期, 同期多行逐列取最新非空值。 + + 语义(区分"覆盖"与"填空"): 每列独立取 announce_date 最新的非空值 — + 新同步行有值则覆盖旧值, 新行缺的列(如 fuyao 不提供的字段)由旧行补齐, + 实现多数据源并集共存。历史报告期不可变, 合并不会引入过期数据。 + 无 announce_date 的帧按输入顺序, 后写优先(与旧行为 keep="last" 一致)。 + """ valid = [ frame for frame in frames @@ -166,11 +173,15 @@ def _merge_report_history(*frames: pl.DataFrame) -> pl.DataFrame: pl.concat(valid, how="diagonal_relaxed") .filter(pl.col("symbol").is_not_null() & pl.col("period_end").is_not_null()) ) - # 同一 (symbol, period_end) 多条时保留 announce_date 最新一条 (业绩修正以最新公告为准)。 - if "announce_date" in merged.columns: - merged = merged.sort(["symbol", "period_end", "announce_date"], nulls_last=True) - return merged.unique(subset=["symbol", "period_end"], keep="last").sort( - ["symbol", "period_end"] + sort_keys = ["symbol", "period_end"] + ( + ["announce_date"] if "announce_date" in merged.columns else [] + ) + merged = merged.sort(sort_keys, nulls_last=True) + value_cols = [c for c in merged.columns if c not in ("symbol", "period_end")] + return ( + merged.group_by("symbol", "period_end") + .agg([pl.col(c).drop_nulls().last() for c in value_cols]) + .sort(["symbol", "period_end"]) ) diff --git a/backend/app/services/kline_sync.py b/backend/app/services/kline_sync.py index 46789c1..a0ec361 100644 --- a/backend/app/services/kline_sync.py +++ b/backend/app/services/kline_sync.py @@ -20,7 +20,7 @@ from app.services import preferences from app.tickflow.capabilities import Cap, CapabilitySet from app.tickflow.client import get_client from app.tickflow.rate_limits import chunked, resolve_limit, sleep_between_batches -from app.tickflow.repository import KlineRepository +from app.tickflow.repository import KlineRepository, replace_with_retry logger = logging.getLogger(__name__) @@ -32,11 +32,12 @@ def _atomic_write_parquet(df: pl.DataFrame, out) -> None: 单文件、每次「读→concat→原地写」, 直接 write_parquet(out) 在进程被 kill (dev.sh 清端口用 kill -9)、reap 超时或断电时会留下半截文件, 之后复权视图 scan_parquet 整条链路报错、enriched 全市场重算不出。临时文件后缀 .tmp 不匹配 - *.parquet glob, 不会被扫描误读。 + *.parquet glob, 不会被扫描误读。Windows 下目标正被并发读取时由 + replace_with_retry 短退避穿过。 """ tmp = out.with_name(out.name + ".tmp") df.write_parquet(tmp) - tmp.replace(out) # 同目录 rename, POSIX/NTFS 均为原子操作 + replace_with_retry(tmp, out) # 标准列(无论 SDK 返回什么形状,我们把它规范成这套) @@ -335,8 +336,6 @@ def sync_adj_factor(symbols: list[str], repo: KlineRepository, return 0, [] provider_name = preferences.get_adj_factor_provider() - if provider_name == "same_as_daily": - provider_name = preferences.get_daily_data_provider() if provider_name != "tickflow": from app.data_providers import custom as custom_sources if custom_sources.provider_has_dataset(provider_name, "adj_factor"): @@ -446,8 +445,80 @@ CANONICAL_MINUTE_COLS = [ ] +# 北京墙钟特征时段(含集合竞价 09:15 与收盘 15:00): 上午 09-11, 下午 13-15 +_BJ_HOURS = [9, 10, 11, 13, 14, 15] +# 上述时段 -8h 的 UTC 墙钟特征: 上午 01-03, 下午 05-07 +_UTC_SHIFTED_HOURS = [1, 2, 3, 5, 6, 7] + + +def _enforce_minute_beijing_wallclock(df: pl.DataFrame, *, source: str) -> pl.DataFrame: + """分钟 K datetime 时区契约守卫: 统一为北京墙钟 (naive)。 + + 契约 (CONTRIBUTING §3.3): kline_minute.datetime 必须是北京时间墙钟, 如 09:35:00。 + 在两个源头入口强制 —— _normalize_minute (TickFlow 帧) 与 _try_custom_minute + (插件/自定义源帧); 落盘 (_write_minute_partition) 与内存消费 (监控/补拉/脉冲) + 均在其下游, 这里收口即全覆盖: + - tz-aware → 转 Asia/Shanghai 后去时区; + - naive 且时刻落在 A 股交易时段 → 直通 (已是北京墙钟); + - naive 且整体呈"交易时段 -8h"的 UTC 特征 → 自动 +8 纠偏并记日志; + - 无法识别的口径 → fail-closed 抛 ValueError, 不让脏时间入库或下发。 + 幂等: 纠偏后的帧再过守卫直通, 不会二次改写。 + """ + if df.is_empty() or "datetime" not in df.columns: + return df + dtype = df.schema["datetime"] + if not isinstance(dtype, pl.Datetime): + # trade_time 等字符串路径: 先解析成 Datetime (失败置 null), 再做时段分类 + if dtype == pl.Utf8: + df = df.with_columns(pl.col("datetime").str.to_datetime(strict=False)) + else: + df = df.with_columns(pl.col("datetime").cast(pl.Datetime("us"), strict=False)) + dtype = df.schema["datetime"] + if not isinstance(dtype, pl.Datetime): + return df # 仍非 Datetime: 维持原行为交由下游处理 + if isinstance(dtype, pl.Datetime) and dtype.time_zone is not None: + df = df.with_columns( + pl.col("datetime") + .dt.convert_time_zone("Asia/Shanghai") + .dt.replace_time_zone(None) + .cast(pl.Datetime("us")) + ) + logger.info("minute datetime tz-aware input converted to Beijing wallclock (source=%s)", source) + return df + + hour = pl.col("datetime").dt.hour() + beijing = int(df.select(hour.is_in(_BJ_HOURS).sum()).item() or 0) + utc_shifted = int(df.select(hour.is_in(_UTC_SHIFTED_HOURS).sum()).item() or 0) + if beijing == 0 and utc_shifted == 0: + if df["datetime"].null_count() == df.height: + return df # 全 null: 维持原行为, 由下游落盘过滤 + raise ValueError( + f"minute datetime 口径无法识别 (source={source}, rows={df.height}, " + f"sample={df['datetime'].drop_nulls().head(2).to_list()}): " + "契约要求北京墙钟 (09:30-15:00), 既非交易时段也非 UTC 平移特征" + ) + if utc_shifted > beijing: + if beijing: + logger.warning( + "minute datetime mixed convention, shifting all by +8h per UTC majority " + "(source=%s, utc=%d, beijing=%d)", source, utc_shifted, beijing, + ) + else: + logger.info( + "minute datetime UTC wallclock detected, shifted +8h to Beijing " + "(source=%s, rows=%d)", source, utc_shifted, + ) + return df.with_columns(pl.col("datetime") + pl.duration(hours=8)) + if utc_shifted: + logger.warning( + "minute datetime has %d UTC-like rows among %d Beijing rows, left as-is " + "(source=%s)", utc_shifted, beijing, source, + ) + return df + + def _normalize_minute(df_in, default_symbol: str | None = None) -> pl.DataFrame: - """把 SDK 返回的分钟 K 数据规范成 canonical 列。""" + """把 SDK 返回的分钟 K 数据规范成 canonical 列 (datetime 收口为北京墙钟)。""" if df_in is None or len(df_in) == 0: return pl.DataFrame() @@ -465,8 +536,14 @@ def _normalize_minute(df_in, default_symbol: str | None = None) -> pl.DataFrame: # datetime 列:优先用 timestamp(毫秒精度),其次 trade_time if "timestamp" in df.columns: + # TickFlow 毫秒时间戳为 UTC 基准; 契约要求北京墙钟 naive + # (与 stock-sdk provider 归一口径一致, 见 CONTRIBUTING §3.3) df = df.with_columns( - pl.from_epoch("timestamp", time_unit="ms").alias("datetime"), + pl.from_epoch(pl.col("timestamp").cast(pl.Int64), time_unit="ms") + .dt.replace_time_zone("UTC") + .dt.convert_time_zone("Asia/Shanghai") + .dt.replace_time_zone(None) + .alias("datetime") ).drop("timestamp") for drop_col in ("trade_time", "trade_date"): if drop_col in df.columns: @@ -478,6 +555,10 @@ def _normalize_minute(df_in, default_symbol: str | None = None) -> pl.DataFrame: elif "trade_date" in df.columns: df = df.rename({"trade_date": "datetime"}) + if "datetime" in df.columns: + # 时区契约守卫: 须在下方通用 us-cast 之前, 避免带时区列被静默剥成 UTC-naive + df = _enforce_minute_beijing_wallclock(df, source="tickflow") + if "symbol" not in df.columns and default_symbol is not None: df = df.with_columns(pl.lit(default_symbol).alias("symbol")) @@ -606,11 +687,18 @@ def _try_custom_minute( symbols, start_time=start_time, end_time=end_time, asset_type=asset_type, freq=freq, on_chunk_done=wrapped_cb, ) - return (df, False) - except Exception as e: # noqa: BLE001 + except Exception as e: logger.warning("custom minute provider %s call failed, falling back to TickFlow: %s", provider_name, e) return (None, True) + try: + # 时区契约守卫: 插件/自定义源帧同样收口为北京墙钟 (CONTRIBUTING §3.3) + df = _enforce_minute_beijing_wallclock(df, source=provider_name) + except Exception as e: + logger.warning("custom minute provider %s datetime 契约校验失败, falling back to TickFlow: %s", + provider_name, e) + return (None, True) + return (df, False) def sync_minute_batch( @@ -828,6 +916,110 @@ def fetch_intraday_monitor_batch( return pl.concat(frames, how="diagonal_relaxed") if frames else pl.DataFrame() +def fetch_intraday_full_market_burst( + symbols: list[str], + capset: CapabilitySet | None, + *, + count: int = 300, +) -> tuple[pl.DataFrame, int]: + """全市场当日分钟K并发脉冲拉取 (盘中增量刷新专用, 不落盘)。 + + 与 fetch_intraday_monitor_batch 的区别: + - 监控路径每轮只拉少量标的 (≤ batch 上限, 单请求); + 本函数按 batch_size 把全市场切块后用线程池一次全部打出 + (5546/200 = 28 并发), 配合 >=60s 的固定轮节奏, 任何 60s + 滑动窗口至多一个脉冲 (28 < 48 安全 rpm)。 + - 单块失败不拖垮整轮: 成功块照常返回落盘; 失败块立即单独重试一次, + 仍失败则跳过该块 (本函数每轮拉全天, 下一轮天然自愈)。失败块过多 + (>4, 系统性故障/限流风暴) 时跳过重试, 避免向已过载的服务端加压。 + + 限流口径: 只用 intraday.batch 独立池 (Cap.INTRADAY_BATCH, Expert 专有), + 不与 kline.minute.batch (盘后分钟同步) 共享配额。 + 返回 (当日全市场分钟K, 请求数)。 + """ + if not symbols: + return (pl.DataFrame(), 0) + limits = capset.limits(Cap.INTRADAY_BATCH) if capset and capset.has(Cap.INTRADAY_BATCH) else None + batch_size = max(1, int(limits.batch) if limits and limits.batch else 200) + chunks = list(chunked(symbols, batch_size)) + if not chunks: + return (pl.DataFrame(), 0) + + from concurrent.futures import ThreadPoolExecutor + + tf = get_client() + + def _fetch(chunk: list[str]) -> tuple[list[pl.DataFrame], Exception | None]: + # 单块独立容错: 异常作为返回值上交而不是抛出, 避免一个块把整轮 + # 已成功的数据一起拖垮 (pool.map 迭代中抛异常会废弃全部已收 frames) + try: + raw = tf.klines.intraday_batch( + chunk, count=count, as_dataframe=True, show_progress=False, + batch_size=len(chunk), + ) + return (_normalize_intraday_raw(raw), None) + except Exception as e: + return ([], e) + + frames: list[pl.DataFrame] = [] + failed: list[list[str]] = [] + with ThreadPoolExecutor(max_workers=min(len(chunks), 32)) as pool: + for chunk, (sub, err) in zip(chunks, pool.map(_fetch, chunks), strict=True): + if err is not None: + failed.append(chunk) + else: + frames.extend(sub) + + requests = len(chunks) + if failed: + if len(failed) > 4: + missed = sum(len(chunk) for chunk in failed) + logger.warning( + "intraday burst: %d/%d chunks failed (%d symbols), systemic — skip retry, next round re-pulls full day", + len(failed), len(chunks), missed, + ) + else: + logger.warning("intraday burst: %d/%d chunks failed, retrying once", len(failed), len(chunks)) + for chunk in failed: + sub, err = _fetch(chunk) + requests += 1 + if err is None: + frames.extend(sub) + else: + logger.warning( + "intraday burst: chunk retry still failed, skip %d symbols this round: %s", + len(chunk), err, + ) + if not frames: + return (pl.DataFrame(), requests) + return (pl.concat(frames, how="diagonal_relaxed"), requests) + + +def fetch_intraday_universe_increment( + universe: str = "CN_Equity_A", + *, + count: int = 3, +) -> tuple[pl.DataFrame, int]: + """全市场当日分钟K增量拉取 (盘中稳态轮专用, 不落盘)。 + + /v1/klines/intraday/universe: 传 universe ID 一次请求返回全市场每只标的 + 最新 count 根分钟K (服务端实测上限 3 根/标的), 替代稳态场景下 28 块并发 + 的 intraday.batch 脉冲 (请求量 28→1, 传输量 ~40 倍降)。缺口回补 + (冷启动/长时间断档/全天修复) 仍走 fetch_intraday_full_market_burst。 + 返回 (增量分钟K, 请求数); 拉取失败返回空 df 由调用方按失败轮处理。 + """ + tf = get_client() + try: + raw = tf.klines.intraday_universe(universe, count=count, as_dataframe=True) + except Exception as e: + logger.warning("intraday universe fetch failed (%s): %s", universe, e) + return (pl.DataFrame(), 0) + frames = _normalize_intraday_raw(raw) + if not frames: + return (pl.DataFrame(), 0) + return (pl.concat(frames, how="diagonal_relaxed"), 1) + + def fetch_minute_single( symbol: str, trade_date: date, diff --git a/backend/app/services/market_overview_builder.py b/backend/app/services/market_overview_builder.py index 5d7f622..a6c549b 100644 --- a/backend/app/services/market_overview_builder.py +++ b/backend/app/services/market_overview_builder.py @@ -258,10 +258,12 @@ def _dimension_rank(rows: list[dict], repo, kind: str, limit: int = 5, level: in store = ExtConfigStore(repo.store.data_dir) groups: dict[str, dict[str, dict]] = {} + group_source: dict[str, str] = {} # 组名 → 首个命中的扩展字段 "configId.field" (看板成分股弹窗用) for config in store.load_all(): field = _dimension_field(config, kind) if not field: continue + source_field = f"{config.id}.{field}" for ext_row in _read_ext_rows(repo.store.data_dir, config, field): quote = None for key in _symbol_keys(ext_row, config): @@ -277,6 +279,7 @@ def _dimension_rank(rows: list[dict], repo, kind: str, limit: int = 5, level: in parts = value.split("-") value = parts[level - 1] if level <= len(parts) else parts[-1] groups.setdefault(value, {})[symbol] = quote + group_source.setdefault(value, source_field) items = [] for name, by_symbol in groups.items(): @@ -293,6 +296,7 @@ def _dimension_rank(rows: list[dict], repo, kind: str, limit: int = 5, level: in "up_count": sum(1 for v in changes if v > 0), "down_count": sum(1 for v in changes if v < 0), "amount": sum(_finite(s.get("amount")) or 0 for s in stocks), + "source_field": group_source.get(name), "leader": { "symbol": leader.get("symbol"), "name": leader.get("name"), diff --git a/backend/app/services/market_recap.py b/backend/app/services/market_recap.py index 5be1ff6..9be3bb7 100644 --- a/backend/app/services/market_recap.py +++ b/backend/app/services/market_recap.py @@ -179,8 +179,9 @@ def _build_emotion_block(overview: dict) -> str: return "\n".join(lines) -def _build_user_prompt(overview: dict, news: list[dict], focus: str) -> str: - """构建用户消息:复盘日期 + 市场数据精简切片 + 新闻 + 关注点。""" +def _build_user_prompt(overview: dict, news: list[dict], focus: str, lhb_context: str = "", + bench_context: str = "") -> str: + """构建用户消息:复盘日期 + 市场数据精简切片 + 龙虎榜(可选) + 盘前风向标(可选) + 新闻 + 关注点。""" as_of = overview.get("as_of") or "今日" parts: list[str] = [ @@ -202,6 +203,22 @@ def _build_user_prompt(overview: dict, news: list[dict], focus: str) -> str: _build_sector_block(overview.get("industry_rank"), "行业"), ] + # 龙虎榜资金动向 (fuyao 数据源, 摘要自带数据日期; 无数据源/失败时为空不占段) + if lhb_context: + parts.extend([ + "", + "## 龙虎榜资金动向", + lhb_context, + ]) + + # 盘前风向标 (fuyao 竞价筛选名单 + 当日实际表现对照; 失败为空不占段) + if bench_context: + parts.extend([ + "", + "## 盘前风向标(竞价)", + bench_context, + ]) + if news: news_lines = [] for i, n in enumerate(news[:8], 1): @@ -300,7 +317,16 @@ async def recap_market_stream( try: from app.services.ai_provider import stream_ai_text - user_prompt = _build_user_prompt(overview, news or [], focus) + # 龙虎榜摘要 (fuyao 专有): 拉取失败/未配置 → 空串, 复盘主流程不受影响 + from app.services import dragon_tiger as dragon_tiger_svc + + lhb_ctx = dragon_tiger_svc.build_recap_context(repo.store.data_dir) + + # 盘前风向标摘要 (fuyao 专有): 失败/未配置 → 空串 + from app.services import auction_benchmark as auction_benchmark_svc + + bench_ctx = auction_benchmark_svc.build_recap_context(repo.store.data_dir) + user_prompt = _build_user_prompt(overview, news or [], focus, lhb_ctx, bench_ctx) got_content = False async for delta in stream_ai_text( [ diff --git a/backend/app/services/minute_refresh.py b/backend/app/services/minute_refresh.py new file mode 100644 index 0000000..3b21cae --- /dev/null +++ b/backend/app/services/minute_refresh.py @@ -0,0 +1,305 @@ +"""盘中分钟K增量落盘服务 (Expert 专有)。 + +两段式拉取 (见 feat/minute-strategy 方案): +- 全天修复轮: intraday.batch (日内分时批量) 并发脉冲一次拉全市场当日全部 + 分钟K — 冷启动 (如 10 点才开服务, 补 9:30 起缺口) / 覆盖滞后超阈值 / + 连续空轮自愈时触发。 +- 稳态增量轮: intraday.universe 传 CN_Equity_A 标的池, 单请求返回全市场 + 每只最新 3 根 (服务端上限), 靠 _write_minute_partition 的 + unique(symbol,datetime) 幂等合并滚出全天。 + +单轮合并写入当日 kline_minute 分区, 供分钟策略 (minute_filter) 读到新鲜数据。 + +设计约束: +- Expert 专有: 能力门控 Cap.INTRADAY_UNIVERSE (全量分钟) — 仅 TickFlow + Expert 档具备, 天然排他 (自定义分钟源无此能力, 且服务本就让位插件)。 +- 修复轮并发脉冲: 全市场按 batch_size 分块 (5546/200 = 28 块) 一次打出。 + 任何 60s 滑动窗口至多一个脉冲 (28 < 48 安全 rpm); 单块失败不拖垮整轮, + 失败块单独重试一次 (见 fetch_intraday_full_market_burst)。 +- 稳态轮单请求: 无脉冲并发, 间隔可低至 3s; 实际节奏 = max(间隔, 单轮完成), + 服务端响应 ~5s 时自动退化为响应节奏, 不会重叠请求。 +- 固定节奏: 默认 6s 一轮 (clamp [3, 300]), 不补跑 (missed 轮次直接跳过)。 +- 仅连续竞价时段运行 (9:30-11:30 / 13:00-15:00), 午休/收盘自动暂停与恢复; + 午休后恢复因覆盖滞后会多跑一次修复轮, 幂等无害。 +- 不与其他分钟能力冲突: 与 盘后分钟同步 (kline.minute.batch) / 分时监控路径 + 分属不同限流池; 落盘走 _write_minute_partition 的 unique(symbol,datetime) + 合并, 与盘后同步写同一分区安全幂等。 +- 数据源插件化让位: 配置了自定义分钟源 (minute_data_provider != tickflow) 时 + 服务不启动 — 盘中增量交由插件自管, 本服务不抢占。 + +分层: 本模块只做调度/落盘/状态; TickFlow SDK 调用全部在 kline_sync 边界层 +(fetch_intraday_full_market_burst / fetch_intraday_universe_increment), +保持插件化边界不泄漏。 +""" +from __future__ import annotations + +import contextlib +import threading +import time +from dataclasses import dataclass, field +from typing import Any + +import polars as pl + +from app.market_time import cn_now, cn_today, in_continuous_session +from app.services import preferences + +# 轮询间隔允许范围 (秒): 稳态轮单请求无并发脉冲, 下限 3s; 上限防误配。 +REFRESH_INTERVAL_MIN = 3 +REFRESH_INTERVAL_MAX = 300 +# 等待步长 (秒): 循环小步睡眠, 便于快速停止与偏好热生效。 +_LOOP_STEP_S = 2.0 +# 当日覆盖滞后超过该分钟数 (≈ universe 单请求 3 根余量) → 触发全天修复轮。 +_REPAIR_LAG_MINUTES = 3.0 +# 连续空轮达到该次数 → 强制全天修复轮 (自愈 universe 端点持续异常)。 +_EMPTY_ROUNDS_TO_REPAIR = 2 + + +def _in_continuous_session(now=None) -> bool: + """A股连续竞价时段 (北京时间): 9:30-11:30 / 13:00-15:00, 仅工作日。""" + return in_continuous_session(now) + + +@dataclass +class _RefreshState: + """服务运行状态 (status() 的内存镜像, 循环线程内更新)。""" + + rounds: int = 0 + last_round_at: float | None = None # epoch 秒 + last_round_ms: float | None = None # 单轮耗时 + last_rows: int = 0 # 上轮写入行数 (合并后) + last_symbols: int = 0 # 上轮覆盖标的数 + last_requests: int = 0 # 上轮请求数 (增量恒 1, 修复=分块数+重试) + last_mode: str | None = None # 上轮模式: "increment" / "full" + last_error: str | None = None + next_round_at: float | None = None # epoch 秒 + extra: dict[str, Any] = field(default_factory=dict) + + +class MinuteRefreshService: + """盘中分钟增量刷新: 单实例挂 app.state.minute_refresh, 后台守护线程。""" + + def __init__(self, repo) -> None: + self._repo = repo + self._app_state: Any | None = None + self._thread: threading.Thread | None = None + self._stop = threading.Event() + self._state = _RefreshState() + self._round_lock = threading.Lock() # 同时只允许一轮 (手动触发与定时轮互斥) + self._empty_rounds = 0 # 连续空轮计数 (escalate 到全天修复) + + # ------------------------------------------------------------------ + # 生命周期 + # ------------------------------------------------------------------ + + def set_repo(self, repo) -> None: + self._repo = repo + + def set_app_state(self, app_state: Any) -> None: + self._app_state = app_state + + def start(self) -> bool: + """启动后台线程 (幂等)。开关/时段/能力判断都在循环内每轮做, 热生效。""" + if self._thread is not None and self._thread.is_alive(): + return True + self._stop.clear() + self._thread = threading.Thread( + target=self._loop, name="minute-refresh", daemon=True, + ) + self._thread.start() + return True + + def stop(self) -> None: + self._stop.set() + + # ------------------------------------------------------------------ + # 门控 + # ------------------------------------------------------------------ + + def capability_ok(self) -> bool: + """Cap.INTRADAY_UNIVERSE (全量分钟) 存在。能力探测结果缓存在 app.state。 + + 门控挂在稳态增量的主能力上; 全天修复轮用的 intraday.batch 与其 + 同属 Expert 档 (tiers.yaml), 目前两者必然同时持有。 + """ + capset = getattr(self._app_state, "capabilities", None) if self._app_state else None + if capset is None: + return False + try: + from app.tickflow.capabilities import Cap + + return capset.has(Cap.INTRADAY_UNIVERSE) + except Exception: + return False + + def custom_provider_active(self) -> bool: + """配置了自定义分钟源 → 让位插件, 本服务不启动。""" + try: + return preferences.get_minute_data_provider() != "tickflow" + except Exception: + return False + + def _gate_reason(self) -> str | None: + """返回本轮不执行的原因 (None = 放行)。""" + if not preferences.get_minute_refresh_enabled(): + return "disabled" + if self.custom_provider_active(): + return "custom_minute_provider" + if not self.capability_ok(): + return "capability" + if not _in_continuous_session(): + return "outside_trading_hours" + # 节假日 (工作日但休市): 周几门控覆盖不到, 由交易日探针剔除。 + # 未知 (None) 维持现状 — 空轮升级机制兜底 (探针失灵时的第二道防线)。 + from app.services import trading_day + + if trading_day.is_trading_day() is False: + return "holiday" + return None + + # ------------------------------------------------------------------ + # 主循环 + # ------------------------------------------------------------------ + + def _loop(self) -> None: + while not self._stop.is_set(): + try: + reason = self._gate_reason() + if reason is None: + interval = preferences.get_minute_refresh_interval() + started = time.time() + self._run_round() + # 固定节奏: 下一轮 = max(本轮起点+间隔, 本轮完成), 不补跑 + finish = time.time() + self._state.next_round_at = max(started + interval, finish) + # 等到下一轮 (小步睡眠保持可停/偏好热切换) + while not self._stop.is_set(): + now = time.time() + gate = self._gate_reason() + if gate is not None: + self._state.next_round_at = None + break # 门控关闭 → 回外层等待重评估 + if now >= self._state.next_round_at: + break + self._stop.wait(min(_LOOP_STEP_S, max(0.0, self._state.next_round_at - now))) + continue + except Exception as e: + self._state.last_error = f"round failed: {e}" + self._stop.wait(_LOOP_STEP_S) + + # ------------------------------------------------------------------ + # 单轮 + # ------------------------------------------------------------------ + + def _today_coverage_lag_minutes(self) -> float | None: + """当日分区最新K距现在的分钟数; None = 当日无数据。 + + 覆盖度探测只读当日分区文件 (毫秒级); 任何异常按无数据处理 → + 本轮走全天修复, 不会因探测失败而丢增量。 + """ + with contextlib.suppress(Exception): + part = ( + self._repo.store.data_dir / "kline_minute" + / f"date={cn_today().isoformat()}" / "part.parquet" + ) + if not part.exists(): + return None + mx = pl.read_parquet(part, columns=["datetime"])["datetime"].max() + if mx is None: + return None + # 分区 datetime 为北京墙钟 naive, cn_now 带时区 → 剥齐再比 + return (cn_now().replace(tzinfo=None) - mx).total_seconds() / 60 + return None + + def _select_mode(self) -> str: + """选轮次模式: 稳态增量 (universe 单请求) vs 全天修复 (burst 脉冲)。 + + 当日已有数据且覆盖滞后 ≤ _REPAIR_LAG_MINUTES (≈ 3 根余量) → 增量; + 冷启动 / 断档超阈值 / 连续空轮 → 全天修复。 + """ + if self._empty_rounds >= _EMPTY_ROUNDS_TO_REPAIR: + return "full" + lag = self._today_coverage_lag_minutes() + if lag is None or lag > _REPAIR_LAG_MINUTES: + return "full" + return "increment" + + def _run_round(self) -> None: + from app.services import kline_sync + + t0 = time.perf_counter() + mode = self._select_mode() + with self._round_lock: + if mode == "increment": + df, requests = kline_sync.fetch_intraday_universe_increment() + self._state.last_symbols = ( + df["symbol"].n_unique() if not df.is_empty() else 0 + ) + else: + symbols = self._universe() + self._state.last_symbols = len(symbols) + if not symbols: + self._state.last_error = "empty universe (instruments 未加载)" + return + capset = getattr(self._app_state, "capabilities", None) if self._app_state else None + df, requests = kline_sync.fetch_intraday_full_market_burst(symbols, capset) + self._state.last_requests = requests + if df.is_empty(): + self._empty_rounds += 1 + self._state.last_error = f"intraday {mode} returned no data" + return + self._empty_rounds = 0 + written = kline_sync._write_minute_partition( + df, self._repo.store.data_dir / "kline_minute", + ) + + self._state.rounds += 1 + self._state.last_round_at = time.time() + self._state.last_round_ms = (time.perf_counter() - t0) * 1000 + self._state.last_rows = written + self._state.last_mode = mode + self._state.last_error = None + + def _universe(self) -> list[str]: + """全市场 A 股标的 (instruments 维表, 与盘后分钟同步同一来源)。""" + inst = self._repo.get_instruments() + if inst.is_empty() or "symbol" not in inst.columns: + return [] + return inst["symbol"].cast(pl.Utf8).unique().sort().to_list() + + # ------------------------------------------------------------------ + # 状态 + # ------------------------------------------------------------------ + + def status(self) -> dict[str, Any]: + import contextlib + + with contextlib.suppress(Exception): + enabled = preferences.get_minute_refresh_enabled() + running = self._thread is not None and self._thread.is_alive() + gate = self._gate_reason() + return { + "enabled": enabled, + "running": running, + "interval_seconds": preferences.get_minute_refresh_interval(), + "capability_ok": self.capability_ok(), + "custom_provider_active": self.custom_provider_active(), + "in_trading_hours": _in_continuous_session(), + "gate_reason": gate if (enabled and running) else (gate or "disabled"), + "rounds": self._state.rounds, + "last_round_at": self._state.last_round_at, + "last_round_ms": self._state.last_round_ms, + "last_rows": self._state.last_rows, + "last_symbols": self._state.last_symbols, + "last_requests": self._state.last_requests, + "last_mode": self._state.last_mode, + "next_round_at": self._state.next_round_at, + "last_error": self._state.last_error, + } + + def trigger_manual_round(self) -> dict[str, Any]: + """手动触发一轮 (无视时段门控, 但仍受能力/插件门控); 供状态页「立即刷新」。""" + if self.custom_provider_active() or not self.capability_ok(): + return {"ok": False, "reason": self._gate_reason() or "capability"} + threading.Thread(target=self._run_round, daemon=True, name="minute-refresh-manual").start() + return {"ok": True} diff --git a/backend/app/services/pipeline_jobs.py b/backend/app/services/pipeline_jobs.py index 365a216..08160e5 100644 --- a/backend/app/services/pipeline_jobs.py +++ b/backend/app/services/pipeline_jobs.py @@ -2,8 +2,12 @@ 设计: - job_store/ 文件夹,每个 job 一个 {id}.json,最多保留 max_jobs 个文件 - - running/pending 状态的 job 仅存内存(高频读写) - - succeeded/failed 后写入独立文件并从内存释放 + - create()/start() 即落盘 pending/running 快照(进度只更新内存) —— + 进程意外死亡(uvicorn --reload 热重载 / 被 kill)时记录不蒸发 + - succeeded/failed 后写入终态并从内存释放 + - 实例化时扫描磁盘,把上个进程遗留的 pending/running 孤儿记录补标为 + failed(中断);finished_at 取文件 mtime(最后已知存活时刻),不用下次 + 开机时间虚增时长 - 列表查询 = 内存中的活跃 job + 磁盘文件扫描,按时间排序 - 单个查询 = 内存优先,没有则读磁盘 - 创建新 job 前检查文件数量,>= max_jobs 时删除最老的文件 @@ -15,7 +19,7 @@ import logging import os import threading import uuid -from datetime import datetime +from datetime import UTC, datetime from pathlib import Path from typing import Any, Literal @@ -105,11 +109,12 @@ class JobStore: self._active_id: str | None = None self._lock = threading.Lock() self._store_dir.mkdir(parents=True, exist_ok=True) + self._reap_orphans() # ===== persistence ===== def _write_file(self, job: dict[str, Any]) -> None: - """将终态 job 写入独立 JSON 文件。""" + """将 job 快照写入独立 JSON 文件(create/start/终态均落盘)。""" path = self._store_dir / f"{job['id']}.json" try: path.write_text( @@ -154,6 +159,42 @@ class JobStore: jobs.sort(key=lambda j: j.get("started_at") or "", reverse=True) return jobs + def _reap_orphans(self) -> None: + """启动补录: 把上个进程遗留的 pending/running 记录标为中断。 + + 单例在进程启动时实例化,此时磁盘上的 pending/running 必然来自已死 + 亡的进程(uvicorn --reload 热重载 / 被 kill)—— 不补录则这些记录 + 永远停留在「运行中」, 同步历史里既看不到结果也看不到失败, 即 + 「数据在、记录丢」。finished_at 取文件 mtime(最后一次落盘 = 最后 + 已知存活时刻), 避免拿下次开机时间虚增 duration。 + """ + for f in self._store_dir.glob("*.json"): + try: + j = json.loads(f.read_text("utf-8")) + except Exception: + continue + orig_status = j.get("status") + if not j.get("id") or orig_status not in ("pending", "running"): + continue + j["status"] = "failed" + j["error"] = "后端重启,任务中断(启动时补录)" + try: + mtime = datetime.fromtimestamp(f.stat().st_mtime, tz=UTC) + end = mtime.strftime("%Y-%m-%dT%H:%M:%SZ") + except Exception: + end = datetime.utcnow().isoformat(timespec="seconds") + "Z" + # mtime 早于 started_at(时钟回拨等)时夹住, 避免 duration 为负 + if j.get("started_at") and end < j["started_at"]: + end = j["started_at"] + j["finished_at"] = end + j["duration_s"] = _duration_s(j) + logger.warning( + "job_store: 补录中断任务 %s (上个进程遗留 %s 记录)", + j["id"], + orig_status, + ) + self._write_file(j) + # ===== lifecycle ===== def create( @@ -189,7 +230,7 @@ class JobStore: return self._active_id, False job_id = uuid.uuid4().hex[:10] - self._active_jobs[job_id] = { + job = { "id": job_id, "status": "pending", "stage": "init", @@ -204,7 +245,11 @@ class JobStore: "error": None, "timeout_s": timeout_s, } + self._active_jobs[job_id] = job self._active_id = job_id + # pending 即落盘: 进程在 start() 前死亡时记录也不丢 + self._delete_oldest() + self._write_file(job) _register_cancel_flag(job_id) return job_id, True @@ -218,6 +263,8 @@ class JobStore: # 心跳基准初始化为启动时刻: start() 到首次 progress() 之间的 # 初始化阶段(解析标的池等)同样计入停滞计时。 j["last_progress_at"] = j["started_at"] + # running 快照落盘: 进程死亡后由下次启动的 _reap_orphans 补录 + self._write_file(j) def succeed(self, job_id: str, result: Any) -> None: with self._lock: diff --git a/backend/app/services/preferences.py b/backend/app/services/preferences.py index f5470ff..13e06ab 100644 --- a/backend/app/services/preferences.py +++ b/backend/app/services/preferences.py @@ -84,29 +84,6 @@ def get_realtime_quote_interval() -> float: return load().get("realtime_quote_interval", 6.0) -def get_realtime_watchlist_symbols() -> list[str]: - """Free 档自选实时监控标的:直接取自选页前 5 个。""" - try: - from app.services import watchlist - rows = watchlist.list_symbols() - except Exception as e: # noqa: BLE001 - logger.warning("load watchlist for realtime failed: %s", e) - return [] - out: list[str] = [] - for row in rows: - symbol = str((row or {}).get("symbol") or "").strip().upper() - if symbol and symbol not in out: - out.append(symbol) - if len(out) >= 5: - break - return out - - -def set_realtime_watchlist_symbols(symbols: list[str]) -> list[str]: # noqa: ARG001 - """兼容旧接口: Free 实时标的现在由自选页前 5 个决定。""" - return get_realtime_watchlist_symbols() - - def set_realtime_quote_interval(interval: float) -> float: """保存行情轮询间隔（不在此做 min/max 校验，由调用方按档位限制）。""" current = load() @@ -214,6 +191,27 @@ def get_minute_sync_segment_days() -> int: """ return max(5, min(30, load().get("minute_sync_segment_days", 20))) +# ===== 盘中分钟增量刷新 (Expert 专有) ===== + +# 稳态轮为 intraday.universe 单请求增量, 无脉冲并发, 间隔可低至 3s; +# 全天修复轮 (intraday.batch 28 块爆发) 的 rpm 安全与间隔无关, 由轮次 +# 调度 max(间隔, 单轮完成) 天然防重叠。 +_MINUTE_REFRESH_INTERVAL_MIN = 3 +_MINUTE_REFRESH_INTERVAL_MAX = 300 + + +def get_minute_refresh_enabled() -> bool: + """盘中分钟K增量落盘开关。默认关闭; 能力门控 (Expert) 在服务层判断。""" + return bool(load().get("minute_refresh_enabled", False)) + + +def get_minute_refresh_interval() -> int: + """盘中分钟增量刷新间隔(秒)。默认 6,范围 [3, 300]。""" + return max( + _MINUTE_REFRESH_INTERVAL_MIN, + min(_MINUTE_REFRESH_INTERVAL_MAX, int(load().get("minute_refresh_interval", 6))), + ) + # ===== 数据源选择 (默认 TickFlow；第一阶段仅日K切换入口) ===== @@ -260,10 +258,9 @@ def get_daily_data_provider() -> str: def get_adj_factor_provider() -> str: - provider = str(load().get("adj_factor_provider", "same_as_daily") or "same_as_daily").lower() - if provider == "same_as_daily": - return provider - return provider if provider in _allowed_data_providers() else "same_as_daily" + # 「跟随日K」(same_as_daily) 特殊值已下线: 存量配置里的旧值按非法值回退 tickflow + provider = str(load().get("adj_factor_provider", "tickflow") or "tickflow").lower() + return provider if provider in _allowed_data_providers() else "tickflow" def get_minute_data_provider() -> str: @@ -271,6 +268,11 @@ def get_minute_data_provider() -> str: return provider if provider in _allowed_data_providers() else "tickflow" +def get_depth5_data_provider() -> str: + provider = str(load().get("depth5_data_provider", "tickflow") or "tickflow").lower() + return provider if provider in _allowed_data_providers() else "tickflow" + + def get_realtime_data_provider() -> str: provider = str(load().get("realtime_data_provider", "tickflow") or "tickflow").lower() return provider if provider in _allowed_data_providers() else "tickflow" @@ -921,6 +923,13 @@ def set_realtime_monitor_config(cfg: dict) -> dict: updates["minute_intraday_refresh_interval"] = max( _INTRADAY_REFRESH_INTERVAL_MIN, min(_INTRADAY_REFRESH_INTERVAL_MAX, int(cfg["minute_intraday_refresh_interval"]))) + if "minute_refresh_enabled" in cfg: + updates["minute_refresh_enabled"] = bool(cfg["minute_refresh_enabled"]) + if "minute_refresh_interval" in cfg: + # clamp 到 [3, 300], 与 getter 一致, 防前端传越界值 + updates["minute_refresh_interval"] = max( + _MINUTE_REFRESH_INTERVAL_MIN, + min(_MINUTE_REFRESH_INTERVAL_MAX, int(cfg["minute_refresh_interval"]))) if "monitor_ext_fields" in cfg: raw = cfg["monitor_ext_fields"] or {} updates["monitor_ext_fields"] = { @@ -942,6 +951,8 @@ def get_realtime_monitor_config() -> dict: "screener_auto_run": get_screener_auto_run(), "minute_intraday_refresh": get_minute_intraday_refresh(), "minute_intraday_refresh_interval": get_minute_intraday_refresh_interval(), + "minute_refresh_enabled": get_minute_refresh_enabled(), + "minute_refresh_interval": get_minute_refresh_interval(), "monitor_ext_fields": get_monitor_ext_fields(), } diff --git a/backend/app/services/quote_service.py b/backend/app/services/quote_service.py index 9844a5c..fade3c4 100644 --- a/backend/app/services/quote_service.py +++ b/backend/app/services/quote_service.py @@ -28,7 +28,7 @@ import threading import time from concurrent.futures import ThreadPoolExecutor from contextlib import contextmanager -from datetime import date, time as dt_time +from datetime import date, datetime, time as dt_time import polars as pl @@ -167,13 +167,15 @@ class QuoteService: CORE_INDEX_SYMBOLS = ("000001.SH", "399001.SZ", "399006.SZ", "000680.SH") - # 档位 → 最小轮询间隔 (秒) + # 档位 → 最小轮询间隔 (秒) — TickFlow 档位限速保护, 仅实时源为 tickflow 时适用 TIER_MIN_INTERVAL = { "expert": 1.0, "pro": 3.0, "starter": 6.0, "free": 6.0, } + # 插件/自定义源: 不受 TickFlow 档位保护约束, 通用下限 1s (默认间隔仍为 DEFAULT_INTERVAL) + CUSTOM_PROVIDER_MIN_INTERVAL = 1.0 DEFAULT_INTERVAL = 6.0 MAX_INTERVAL = 60.0 @@ -218,6 +220,17 @@ class QuoteService: # 午休/收盘最终同步状态: 到边界后必须成功拉取一版行情, 再进入休盘态。 self._final_sync_done: set[tuple[date, str]] = set() self._final_sync_failed: dict[tuple[date, str], str] = {} + self._holiday_active = False # 交易日探针当前是否判休市 (日志去重) + # 轮询放量 (volume_delta 规则): 上一轮全市场股票快照的 (累计成交量[手], 累计成交额[元])。 + # 每轮全量快照后更新 (含非连续竞价时段, 保证 13:00 恢复时 prev 是 12:59 + # 而非 11:30); 跨交易日清空; cur < prev (数据源重置) 时丢弃该轮差值。 + self._prev_stock_volume: dict[str, tuple[float, float]] | None = None + self._prev_volume_fetched_at: float | None = None # epoch 毫秒 + self._prev_volume_date: date | None = None + # 最近一轮的有效差值 (vol_delta[手], amt_delta[元]) - 仅连续竞价时段内、 + # prev 不早于本时段开盘时计算 + self._volume_delta: dict[str, tuple[float, float]] = {} + self._volume_delta_span_s: float = 0.0 # ================================================================ # 生命周期 @@ -423,15 +436,18 @@ class QuoteService: @classmethod def realtime_mode(cls) -> str: - """当前实时行情模式: none / watchlist / full_market。""" + """当前实时行情模式: none / full_market。 + + TickFlow 免费档不再提供"自选前 5 只"降级实时(自定义源 fuyao 的全市场 + 快照已全面覆盖且免费); TickFlow 免费档 = 无实时, 接入自定义实时源 + (如 fuyao)或升级 TickFlow 后恢复全市场模式。 + """ from app.services import preferences if preferences.get_realtime_data_provider() != "tickflow": return "full_market" tier = cls._current_tier() - if tier == "none": + if tier in ("none", "free"): return "none" - if tier == "free": - return "watchlist" return "full_market" @classmethod @@ -441,6 +457,11 @@ class QuoteService: @classmethod def _tier_min_interval(cls) -> float: + # 实时源路由到插件/自定义源时, TickFlow 档位限速不适用 (中立能力原则): + # 下限放宽到通用 1s, 默认/已保存间隔不变 + from app.services import preferences + if preferences.get_realtime_data_provider() != "tickflow": + return cls.CUSTOM_PROVIDER_MIN_INTERVAL tier = cls._current_tier() return cls.TIER_MIN_INTERVAL.get(tier, cls.DEFAULT_INTERVAL) @@ -494,7 +515,6 @@ class QuoteService: def status(self) -> dict: """返回行情服务状态。""" - from app.services import preferences age = (time.perf_counter() - self._fetch_time) * 1000 if self._fetch_time else -1 mode = self.realtime_mode() phase = self._market_phase() @@ -507,7 +527,6 @@ class QuoteService: "paused": self._paused, "mode": mode, "realtime_allowed": mode != "none", - "watchlist_symbol_count": len(preferences.get_realtime_watchlist_symbols()), "interval_s": self._interval, "symbol_count": self._symbol_count, "index_symbol_count": self._index_symbol_count, @@ -561,15 +580,12 @@ class QuoteService: waited += 0.5 def _fetch_quotes(self, *, final: bool = False) -> bool: - """按当前档位拉取行情。加锁串行化 (后台轮询 vs 手动 refresh)。返回本轮是否成功更新。""" + """拉取行情。加锁串行化 (后台轮询 vs 手动 refresh)。返回本轮是否成功更新。""" with self._fetch_lock: before = self._fetched_at if final: logger.info("最终行情同步开始") - if self.realtime_mode() == "watchlist": - self._fetch_watchlist_quotes() - else: - self._fetch_full_market_quotes() + self._fetch_full_market_quotes() return self._fetched_at > before def _fetch_full_market_quotes(self) -> None: @@ -719,6 +735,9 @@ class QuoteService: _persist_last_fetch(fetched_at) logger.info("行情刷新: %d 只股票, %d 只ETF, %d 只指数, 耗时 %.0fms", len(stock_records), len(etf_records), len(index_records), fetch_ms) + # 轮询放量状态更新 (volume_delta 规则的差值来源) + self._update_volume_delta(stock_records, fetched_at) + # ---- 写 kline_daily (不复权原始价格, 只有 OHLCV) ---- daily_df = self._build_daily(stock_records) if not daily_df.is_empty() and self._repo: @@ -765,150 +784,11 @@ class QuoteService: # ---- 策略监控 + 告警评估 ---- self._evaluate_monitors(daily_df, quote_extra) - def _fetch_watchlist_quotes(self) -> None: - """Free 档自选股实时: 按 capability batch 上限分批拉取。""" - from app.services import preferences - from app.tickflow.client import get_paid_realtime_client - from app.tickflow.capabilities import Cap - from app.tickflow.policy import detect_capabilities - from app.tickflow.rate_limits import chunked, resolve_limit, sleep_between_batches - - symbols = preferences.get_realtime_watchlist_symbols() - # 指数监控规则标的并入轮询 (与股票共享 batch 额度) - engine = getattr(self._app_state, "monitor_engine", None) if self._app_state else None - if engine: - for _r in list(engine.rules.values()): - if _r.get("enabled", True) and _r.get("asset_type") == "index" and _r.get("scope") == "symbols": - for _s in _r.get("symbols", []): - if _s and _s not in symbols: - symbols.append(_s) - if not symbols: - logger.info("自选实时未配置标的, 跳过行情拉取") - return - - tf = get_paid_realtime_client() - if tf is None: - logger.warning("自选实时拉取失败:未配置付费服务器 API Key") - return - - # 按 capability batch 上限分批: 股票+指数共享额度, 超过上限会导致整轮失败 - capset = detect_capabilities() - lim = resolve_limit(capset, Cap.QUOTE_BY_SYMBOL, default_batch=5) - batches = chunked(symbols, lim.batch) - - t0 = time.perf_counter() - now_ts = time.perf_counter() - resp = [] - for i, batch in enumerate(batches): - sleep_between_batches(i, lim.rpm) - try: - resp.extend(tf.quotes.get(symbols=batch) or []) - except Exception as e: # noqa: BLE001 - logger.warning("自选实时批次 %d/%d 拉取失败: %s", i + 1, len(batches), e) - - if not resp: - logger.warning("自选实时行情数据为空") - return - - records = [] - for q in resp: - ext = q.get("ext") or {} - last_price = q.get("last_price") - prev_close = q.get("prev_close") - change_amount = ext.get("change_amount") - change_pct = ext.get("change_pct") - if change_amount is None and last_price is not None and prev_close is not None: - change_amount = float(last_price) - float(prev_close) - if change_pct is None and change_amount is not None and prev_close not in (None, 0): - # 小数制, 与 ext.change_pct / enriched 口径一致 (不乘 100) - change_pct = float(change_amount) / float(prev_close) - records.append({ - "symbol": q.get("symbol"), - "name": q.get("name") or ext.get("name"), - "last_price": last_price, - "prev_close": prev_close, - "open": q.get("open"), - "high": q.get("high"), - "low": q.get("low"), - "volume": q.get("volume"), - "amount": q.get("amount"), - "change_pct": change_pct, - "change_amount": change_amount, - "amplitude": ext.get("amplitude"), - "turnover_rate": ext.get("turnover_rate"), - "timestamp": q.get("timestamp"), - "session": q.get("session"), - }) - - index_set = self._repo.get_index_symbol_set() if self._repo else set() - etf_set = self._repo.get_etf_symbol_set() if self._repo else set() - index_records, etf_records, stock_records = self._split_records_by_asset(records, index_set, etf_set) - - fetch_ms = (time.perf_counter() - t0) * 1000 - fetched_at = time.time() * 1000 - with self._lock: - self._fetch_time = now_ts - self._fetch_ms = fetch_ms - self._fetched_at = fetched_at - self._symbol_count = len(stock_records) - self._index_symbol_count = len(index_records) - self._etf_symbol_count = len(etf_records) - self._index_quotes_cache = self._build_index_quotes(index_records) if index_records else None - - _persist_last_fetch(fetched_at) - logger.info("自选实时刷新: %d 只股票, %d 只ETF, %d 只指数, 耗时 %.0fms", - len(stock_records), len(etf_records), len(index_records), fetch_ms) - - daily_df = self._build_daily(stock_records) - quote_extra = self._build_quote_extra(stock_records) - if not daily_df.is_empty() and self._repo: - try: - self._repo.merge_live_daily_asset("stock", daily_df) - except Exception as e: # noqa: BLE001 - logger.warning("自选实时日K写盘失败: %s", e) - self._flush_live_enriched(daily_df, quote_extra, asset_type="stock", merge=True) - - # ETF/指数进自选前5时按各自资产落盘, 不污染股票表 - etf_daily_df = self._build_daily(etf_records) - if not etf_daily_df.is_empty() and self._repo: - try: - self._repo.merge_live_daily_asset("etf", etf_daily_df) - except Exception as e: # noqa: BLE001 - logger.warning("自选实时 ETF 日K写盘失败: %s", e) - self._flush_live_enriched(etf_daily_df, self._build_quote_extra(etf_records), asset_type="etf", merge=True) - index_daily_df = self._build_daily(index_records) - if not index_daily_df.is_empty() and self._repo: - try: - self._repo.merge_live_daily_asset("index", index_daily_df) - except Exception as e: # noqa: BLE001 - logger.warning("自选实时指数日K写盘失败: %s", e) - self._flush_live_enriched(index_daily_df, self._build_quote_extra(index_records), asset_type="index", merge=True) - - self._broadcast_quote_updated() - self._evaluate_monitors(daily_df, quote_extra) - # ================================================================ # 工具 # ================================================================ @staticmethod - def _split_records_by_asset( - records: list[dict], index_set: set[str], etf_set: set[str], - ) -> tuple[list[dict], list[dict], list[dict]]: - """把行情 records 按资产拆成 (index, etf, stock)。判定顺序与 resolve_asset_type 一致: 先 ETF 后指数。""" - index_records: list[dict] = [] - etf_records: list[dict] = [] - stock_records: list[dict] = [] - for r in records: - sym = r.get("symbol") - if sym in etf_set: - etf_records.append(r) - elif sym in index_set: - index_records.append(r) - else: - stock_records.append(r) - return index_records, etf_records, stock_records - @staticmethod def _build_daily(records: list[dict]) -> pl.DataFrame: """将 API records 转为日K格式 DataFrame (OHLCV + quote_ts, 写 kline_daily 用)。""" @@ -1041,8 +921,29 @@ class QuoteService: return (cn_today(), "close") return None + def _holiday_gate(self) -> bool: + """交易日探针门控: 确定休市 → False (停止轮询, 含 final 定版)。 + + 探针未知 (None, 未配置 fuyao 且 tickflow 不可用/开盘缓冲窗内) → True, + 维持周几近似现状行为。探针是纯读, 不落盘; 休市结论带 TTL 定期复探, + 误判自愈。首次判定变化打一条日志, 避免每拍刷屏。 + """ + from app.services import trading_day + + holiday = trading_day.is_trading_day() is False + if holiday != self._holiday_active: + self._holiday_active = holiday + if holiday: + logger.info("交易日探针判定休市, 行情轮询暂停 (30 分钟复探)") + return not holiday + def _should_poll_for_phase(self, phase: str) -> bool: - """是否处于会主动拉行情的阶段。final 阶段成功后即停止。""" + """是否处于会主动拉行情的阶段。final 阶段成功后即停止。 + + 节假日 (工作日但休市) 由交易日探针剔除 — 周几门控覆盖不到的部分。 + """ + if not self._holiday_gate(): + return False if phase in {"preopen", "morning", "pre_afternoon", "afternoon"}: return True key = self._final_sync_key(phase) @@ -1117,6 +1018,8 @@ class QuoteService: eval_df = enriched_today if engine.has_rule_type("ladder"): eval_df = self._inject_sealed_vol(enriched_today, enriched_date) + if engine.has_rule_type("volume_delta"): + eval_df = self._inject_volume_delta(eval_df) eval_df = self._inject_intraday_signals(eval_df, engine, "stock") rule_events = engine.evaluate(eval_df, asset_type="stock") if engine.consume_strategy_result_updates(): @@ -1203,7 +1106,8 @@ class QuoteService: "window_change_pct", "coverage_ratio", "valid_count", "total_count", "up_count", "down_count", "leader", "abnormal_window", "abnormal_value", "abnormal_threshold", - "abnormal_closeness", + "abnormal_closeness", "volume_delta", "volume_delta_span", + "volume_delta_amount", ): if key in ev: alert[key] = ev[key] @@ -1352,6 +1256,88 @@ class QuoteService: ) return self._intraday_signal_evaluator.inject(enriched, signals) + @staticmethod + def _continuous_session_start_ms() -> float: + """当前连续竞价时段的起点 (北京时间 9:30 或 13:00) 的 epoch 毫秒。""" + now = cn_now() + start_time = dt_time(13, 0) if now.time() >= dt_time(13, 0) else dt_time(9, 30) + return datetime.combine(now.date(), start_time, tzinfo=now.tzinfo).timestamp() * 1000.0 + + def _update_volume_delta(self, stock_records: list[dict], fetched_at_ms: float) -> None: + """全市场相邻两次快照的股票累计成交量差值 (手), 供 volume_delta 规则。 + + - prev 每轮都更新 (含非连续竞价时段); 差值只在连续竞价时段内计算 + - 开盘保护: prev 早于本时段起点 (9:30/13:00) 时本轮差值无效 -- 避免 + 9:25 集合竞价撮合量 / 午休缺口被当成"突然放量" + - cur < prev (数据源重置/口径跳变) 的个股丢弃差值; 跨交易日清空 + """ + today = cn_today() + if self._prev_volume_date != today: + self._prev_stock_volume = None + self._prev_volume_fetched_at = None + self._prev_volume_date = today + self._volume_delta = {} + + cur: dict[str, tuple[float, float]] = {} + for r in stock_records: + sym = r.get("symbol") + vol = r.get("volume") + amt = r.get("amount") + if not sym or not isinstance(vol, (int, float)): + continue + cur[str(sym)] = ( + float(vol), + float(amt) if isinstance(amt, (int, float)) else 0.0, + ) + + prev = self._prev_stock_volume + prev_ts = self._prev_volume_fetched_at + if ( + prev is not None + and prev_ts is not None + and self._is_continuous_trading() + and prev_ts >= self._continuous_session_start_ms() + ): + delta = { + sym: (v - prev[sym][0], a - prev[sym][1]) + for sym, (v, a) in cur.items() + if sym in prev and v >= prev[sym][0] and a >= prev[sym][1] and v - prev[sym][0] > 0 + } + self._volume_delta = delta + self._volume_delta_span_s = max((fetched_at_ms - prev_ts) / 1000.0, 0.001) + else: + self._volume_delta = {} + + self._prev_stock_volume = cur + self._prev_volume_fetched_at = fetched_at_ms + + def _inject_volume_delta(self, enriched_today: pl.DataFrame) -> pl.DataFrame: + """把最近一轮快照差值作为临时列注入 enriched 副本。 + + _volume_delta (手) / _volume_delta_amount (元) / _volume_delta_span (秒, 快照间隔)。 + 无有效差值 (首轮/开盘保护/暂停后恢复) 时返回原 df, 规则安全降级不触发。 + """ + try: + delta = self._volume_delta + if not delta: + return enriched_today + span = self._volume_delta_span_s + delta_df = pl.DataFrame({ + "symbol": list(delta.keys()), + "_volume_delta": [v for v, _ in delta.values()], + "_volume_delta_amount": [a for _, a in delta.values()], + "_volume_delta_span": [span] * len(delta), + }) + drop_cols = [ + c for c in ("_volume_delta", "_volume_delta_amount", "_volume_delta_span") + if c in enriched_today.columns + ] + df = enriched_today.drop(drop_cols) if drop_cols else enriched_today + return df.join(delta_df, on="symbol", how="left") + except Exception as e: # noqa: BLE001 + logger.debug("快照差值注入失败 (volume_delta 规则将不触发): %s", e) + return enriched_today + def _inject_sealed_vol(self, enriched_today: pl.DataFrame, enriched_date) -> pl.DataFrame: """从 depth_service 取封单量, 作为临时列 _sealed_vol 注入 enriched 副本。 @@ -1414,7 +1400,7 @@ class QuoteService: source_labels = { "strategy": "策略", "signal": "信号", "price": "价格", "market": "异动", "ladder": "连板梯队", - "sector": "板块", + "sector": "板块", "volume_delta": "放量", } rules = engine.rules if engine is not None else {} enqueued = 0 diff --git a/backend/app/services/screener.py b/backend/app/services/screener.py index 1e62e29..c11cf64 100644 --- a/backend/app/services/screener.py +++ b/backend/app/services/screener.py @@ -386,6 +386,27 @@ class ScreenerService: if current is None: current = self._load_enriched_for_date(as_of) + if timeframe == "1m": + # 分钟策略数据源是本地当日分钟K分区 (单分区文件直读), 与日线 + # enriched 历史窗口无关, 不走 required_history_bars 日线路径。 + history = self._load_minute_history(as_of, current) + # 策略声明 META["daily_history_bars"] 时额外装配日线 enriched 窗口, + # 供分钟策略叠加日线维度条件 (如 N 日内涨停过)。 + daily_history = None + if engine is not None: + daily_bars = engine.minute_daily_history_bars(strategy_ids) + if daily_bars > 0: + daily_history = self._load_enriched_history(as_of, daily_bars) + return StrategyDataContext( + asset_type=self.asset_type, + timeframe=timeframe, + as_of=as_of, + current=current, + history=history, + daily_history=daily_history, + market=None, + cache_key=cache_key, + ) history_bars = engine.required_history_bars( strategy_ids, params_map=params_map, @@ -404,6 +425,31 @@ class ScreenerService: cache_key=cache_key, ) + def _load_minute_history(self, as_of: date, current: pl.DataFrame | None) -> pl.DataFrame: + """分钟策略数据源: 优先 as_of 当日分钟分区, 缺失时回退全市场最近分区。 + + 只按日期直读单个分区文件 (get_minute_by_dates), 与全量 glob 扫描解耦, + 内存只随当日分区大小 (~67万行) 走。标的池限定为 enriched 快照 universe; + 分区与快照的日期差是允许的 (分钟分区可能比 enriched 更新, 行自带时间戳)。 + """ + if self.asset_type != "stock": + raise ValueError("分钟策略当前仅支持 A 股") + symbols: list[str] = [] + if current is not None and not current.is_empty(): + symbols = current["symbol"].cast(pl.Utf8).unique().to_list() + if not symbols: + return pl.DataFrame() + df = self.repo.get_minute_by_dates(symbols, [as_of]) + if df.is_empty(): + fallback = self.repo.latest_minute_date_global() + if fallback is None: + raise ValueError( + "无分钟K数据 — 请先在 数据→分钟K 完成同步, 或开启盘中增量刷新" + ) + if fallback != as_of: + df = self.repo.get_minute_by_dates(symbols, [fallback]) + return df + def latest_date(self) -> date | None: if self.asset_type != "stock": _, d = self.repo.get_enriched_latest_asset(self.asset_type) diff --git a/backend/app/services/trading_day.py b/backend/app/services/trading_day.py new file mode 100644 index 0000000..bfca3f9 --- /dev/null +++ b/backend/app/services/trading_day.py @@ -0,0 +1,139 @@ +"""交易日探针 (oracle) — 回答「今天是否 A 股交易日」。 + +消费方 (实时行情轮询 / 盘中分钟增量) 在周几+时段门控之后调用, 用于把 +「工作日但休市」的节假日从轮询窗口里剔除; 返回 None (未知) 时调用方 +维持现状行为 (周几近似 + 快照新鲜度判据兜底), 不引入新依赖。 + +探测链 (按确定性排序, 先到先得): + 1. fuyao 交易日历 (已配置 fuyao 时): GET /api/a-share/calendar/trading-days, + 今天在近一年交易日列表内 ⇔ 交易日。权威日历, 无时段依赖, 无开盘缓冲问题。 + 2. tickflow 实时行情时间戳: 拉一篮流动性票快照 (单请求), max(timestamp) + 日期 == 今天 ⇔ 交易日。非交易日全市场戳停在上一交易日 (2026-08-29 周六 + 实测 5551/5551, 含停牌股 — 戳是快照定版时刻, 非最后成交时刻); + 交易日集合竞价阶段 (9:15-9:30) 戳是否已翻新未实测 → 开盘缓冲窗内 + 戳过期不作数, 保守视为未知。 + 3. 均不可用 → None: 调用方按周几近似继续。 + +安全约束: + - 周末直接返回 False (周几判断零成本, 不打任何请求)。 + - 探针是纯读: 只产出一个布尔判定, 不落盘、不进行情管道、不碰归属链路。 + - 只用于「降档」(休市不轮询); 休市结论 TTL 较短 (30 分钟) 定期复探, + 探针误判最坏损失一段快照且可自愈; 未知结论短 TTL (5 分钟) 防止 + 轮询循环每拍重打失败的探测。 +""" + +from __future__ import annotations + +import threading +import time +from dataclasses import dataclass +from datetime import datetime, time as dt_time + +from app.market_time import CN_TZ, cn_now + +# tickflow 戳探针的开盘缓冲窗: 此时刻之前戳仍是上一交易日属正常 (集合竞价), +# 不据此判休市。周一实测竞价戳翻新时机后可收紧。仅上午首个窗口需要。 +_STALE_BUFFER_UNTIL = dt_time(9, 40) + +# 一篮流动性票: 探 max(timestamp), 任一戳为今日即交易日 (OR 语义)。 +# 大盘蓝筹同日全部停牌 = 市场性事件, 与休市同处理无碍。 +_BASKET = ("000001.SZ", "600519.SH", "600036.SH", "601318.SH", "000651.SZ") + +_TTL_TRADING_S = 3600.0 # 交易日结论每小时复探 (跨日天然失效) +_TTL_HOLIDAY_S = 1800.0 # 休市结论 30 分钟复探, 误判自愈上限 +_TTL_UNKNOWN_S = 300.0 # 未知结论 5 分钟后重试探测 + +_CACHE_LOCK = threading.Lock() + + +@dataclass +class _Cache: + day: object | None = None + verdict: bool | None = None + probed_at: float = 0.0 + + +_CACHE = _Cache() + + +def reset_cache() -> None: + """清空探针缓存 (测试用)。""" + with _CACHE_LOCK: + _CACHE.day = None + _CACHE.verdict = None + _CACHE.probed_at = 0.0 + + +def _probe_fuyao(now: datetime) -> bool | None: + """fuyao 交易日历: 今天在列表内 ⇔ 交易日。未配置 fuyao / 失败 → None。""" + try: + from app.data_providers import custom as custom_sources + + if not custom_sources.is_custom_provider("fuyao"): + return None + provider = custom_sources.get_provider("fuyao") + days = provider.trading_days() + return now.date() in days if days else None + except Exception: # noqa: BLE001 — 探针失败按未知处理, 不上抛 + return None + + +def _probe_tickflow(now: datetime) -> bool | None: + """tickflow 行情时间戳: max(timestamp) 日期 == 今天 ⇔ 交易日。 + + 戳停在上一交易日: 开盘缓冲窗内 → None (可能是竞价未翻新), 窗后 → False。 + 无实时权限 / 网络失败 / 无有效戳 → None。 + """ + try: + from app.tickflow.client import get_client + + rows = get_client().quotes.get(symbols=list(_BASKET)) or [] + stamps = [r.get("timestamp") for r in rows if isinstance(r, dict)] + valid = [int(t) for t in stamps if isinstance(t, (int, float)) and t] + if not valid: + return None + latest_day = datetime.fromtimestamp(max(valid) / 1000, tz=CN_TZ).date() + if latest_day == now.date(): + return True + if now.time() < _STALE_BUFFER_UNTIL: + return None + return False + except Exception: # noqa: BLE001 — 无权限/网络失败按未知处理 + return None + + +def is_trading_day(now: datetime | None = None) -> bool | None: + """今天是否 A 股交易日。True=交易日, False=确定休市, None=未知 (维持周几近似)。 + + 周末零成本直判; 工作日走探测链 (fuyao 日历 → tickflow 时间戳), + 结论按 TTL 缓存。线程安全: 实时行情与分钟增量两个线程共用。 + """ + now = now or cn_now() + if now.weekday() >= 5: + return False + + with _CACHE_LOCK: + if ( + _CACHE.day == now.date() + and _CACHE.verdict is not None + and (time.monotonic() - _CACHE.probed_at) < _ttl_of(_CACHE.verdict) + ): + return _CACHE.verdict + + verdict = _probe_fuyao(now) + if verdict is None: + verdict = _probe_tickflow(now) + + with _CACHE_LOCK: + _CACHE.day = now.date() + _CACHE.verdict = verdict + _CACHE.probed_at = time.monotonic() + return verdict + + +def _ttl_of(verdict: bool | None) -> float: + if verdict is True: + return _TTL_TRADING_S + if verdict is False: + return _TTL_HOLIDAY_S + return _TTL_UNKNOWN_S diff --git a/backend/app/strategy/engine.py b/backend/app/strategy/engine.py index aa7e581..517e897 100644 --- a/backend/app/strategy/engine.py +++ b/backend/app/strategy/engine.py @@ -155,6 +155,9 @@ class StrategyDataContext: as_of: date current: pl.DataFrame | None = None history: pl.DataFrame | None = None + # 仅 1m 分支: 策略声明 META["daily_history_bars"] 时注入的日线 enriched 窗口, + # 供分钟策略叠加日线维度条件 (如 N 日内涨停过); 未声明时为 None。 + daily_history: pl.DataFrame | None = None market: Any | None = None cache_key: str | None = None @@ -199,6 +202,11 @@ class StrategyDef: execution_backend: str = "polars_expr" matrix_strategy: Any | None = None composite: CompositeSpec | None = None # 仅 backend=="composite" 时非空 + # 仅 backend=="minute_filter" 时非空: 输入为当日分钟K窗口, 输出为命中标的行 + filter_minute_history_fn: Callable[[pl.DataFrame, dict], pl.DataFrame] | None = None + # 仅 minute_filter: META["daily_history_bars"] 声明需要的日线历史窗口 (0=不需要; + # >0 时 filter_minute_history 必须接受 daily 关键字, 引擎注入 context.daily_history) + minute_daily_bars: int = 0 @dataclass @@ -471,6 +479,7 @@ class StrategyEngine: filter_fn = getattr(mod, "filter", None) filter_history_fn = getattr(mod, "filter_history", None) + filter_minute_history_fn = getattr(mod, "filter_minute_history", None) execution_backend = str( getattr( mod, @@ -481,7 +490,7 @@ class StrategyEngine: ), ) ) - valid_backends = {"polars_expr", "matrix_native", "python_history_legacy", "composite"} + valid_backends = {"polars_expr", "matrix_native", "python_history_legacy", "composite", "minute_filter"} if execution_backend not in valid_backends: raise ValueError( f"unsupported execution backend {execution_backend!r}; " @@ -490,6 +499,7 @@ class StrategyEngine: matrix_strategy = getattr(mod, "MATRIX_STRATEGY", None) composite_spec: CompositeSpec | None = None + minute_daily_bars = 0 if execution_backend == "matrix_native": from app.backtest.matrix import MatrixStrategy @@ -515,6 +525,39 @@ class StrategyEngine: "composite strategy must not declare filter, filter_history or MATRIX_STRATEGY" ) composite_spec = _parse_composite_children(meta.get("children")) + elif execution_backend == "minute_filter": + # 分钟形态策略: 只声明 filter_minute_history; 数据源是本地当日分钟K分区 + # (由 ScreenerService.build_strategy_context 的 1m 分支注入), 因此 timeframes + # 必须且只能是 ["1m"] — 混入 1d 会让日线 context 走错数据路径。 + if ( + filter_minute_history_fn is None + or filter_fn is not None + or filter_history_fn is not None + or matrix_strategy is not None + ): + raise ValueError( + "minute_filter strategy must declare only filter_minute_history" + ) + if meta.get("timeframes") != ["1m"]: + raise ValueError( + "minute_filter strategy must declare timeframes == ['1m']" + ) + # 可选日线历史窗口: 声明 daily_history_bars 时 fn 必须接受 daily 关键字, + # 引擎会把 context.daily_history (enriched 日线窗口) 注入进来。 + minute_daily_bars = int(meta.get("daily_history_bars") or 0) + if minute_daily_bars < 0 or minute_daily_bars > 250: + raise ValueError( + "minute_filter daily_history_bars must be within [0, 250]" + ) + if minute_daily_bars > 0: + import inspect + + sig = inspect.signature(filter_minute_history_fn) + if "daily" not in sig.parameters: + raise ValueError( + "minute_filter daily_history_bars requires " + "filter_minute_history to accept a 'daily' keyword" + ) elif filter_history_fn is None or filter_fn is not None: raise ValueError("python_history_legacy strategy must declare only filter_history") @@ -538,6 +581,8 @@ class StrategyEngine: execution_backend=execution_backend, matrix_strategy=matrix_strategy, composite=composite_spec, + filter_minute_history_fn=filter_minute_history_fn, + minute_daily_bars=minute_daily_bars, ) def reload(self) -> None: @@ -652,6 +697,16 @@ class StrategyEngine: return None return max(0, int(value)) + def minute_daily_history_bars(self, strategy_ids: list[str]) -> int: + """1m 分支需要的日线 enriched 窗口大小: 各 minute_filter 策略声明的 + META["daily_history_bars"] 取 max, 未声明 (纯分钟策略) 为 0。""" + required = 0 + for strategy_id in strategy_ids: + strategy = self.get(strategy_id) + if strategy.execution_backend == "minute_filter": + required = max(required, strategy.minute_daily_bars) + return required + def required_history_bars( self, strategy_ids: list[str], @@ -885,7 +940,26 @@ class StrategyEngine: exit_signal_hits = self._collect_signal_hits(signal_df, exit_signals) # 普通策略只读目标日期；历史策略读取调用方注入的历史窗口。 - if s.filter_history_fn: + if s.execution_backend == "minute_filter": + # 分钟策略: 读取调用方注入的当日分钟K窗口。无 date 列, 不按 as_of 过滤, + # 每个命中行自带最后K线时间戳 (last_datetime)。 + if history is None: + raise ValueError(f"strategy {strategy_id} requires minute history data") + if history.is_empty(): + return StrategyResult( + as_of=as_of, + strategy_id=strategy_id, + exit_signal_hits=exit_signal_hits, + ) + if s.minute_daily_bars > 0: + df = s.filter_minute_history_fn(history, params, daily=context.daily_history) + else: + df = s.filter_minute_history_fn(history, params) + # 基础过滤/展示列 (name/total_shares/change_pct 等) 来自 enriched 快照, + # 在命中结果上事后联表, 避免把 enriched 列铺到全市场分钟行上。 + if current is not None and not current.is_empty(): + df = self._join_basic_columns(df, current) + elif s.filter_history_fn: if history is None: raise ValueError(f"strategy {strategy_id} requires history data") df = history @@ -945,7 +1019,9 @@ class StrategyEngine: # Stage 3: 评分 df = self._apply_scoring(df, scoring, scoring_directions) entry_signal_hits = self._collect_signal_hits(df, entry_signals) - if not entry_signals and (s.filter_history_fn or s.filter_fn): + if not entry_signals and ( + s.filter_history_fn or s.filter_fn or s.execution_backend == "minute_filter" + ): entry_signal_hits = [ {"symbol": str(symbol), "signals": []} for symbol in df["symbol"].cast(pl.Utf8).unique().to_list() @@ -1036,7 +1112,8 @@ class StrategyEngine: history_strats = [ (sid, strategy) for sid, strategy in selected - if strategy.filter_history_fn or strategy.execution_backend == "matrix_native" + if strategy.filter_history_fn + or strategy.execution_backend in ("matrix_native", "minute_filter") ] shared_history = context.history if history_strats and shared_history is None: @@ -1466,6 +1543,25 @@ class StrategyEngine: return df.filter(expr) return df + # 分钟策略命中行需要从事后联表补齐的 enriched 列: 基础过滤引用 + 前端展示。 + # close 不在列 — 分钟策略输出的 close 是最后一根分钟K收盘价, 优先于日线快照。 + MINUTE_JOIN_COLUMNS: tuple[str, ...] = ( + "name", "total_shares", "float_shares", "amount", + "turnover_rate", "change_pct", "pre_close", + ) + + @staticmethod + def _join_basic_columns(df: pl.DataFrame, current: pl.DataFrame) -> pl.DataFrame: + """把 enriched 快照列按 symbol 联到分钟策略输出上, 只补 df 缺失的列。""" + cols = [ + c for c in StrategyEngine.MINUTE_JOIN_COLUMNS + if c in current.columns and c not in df.columns + ] + if not cols: + return df + extra = current.select(["symbol", *cols]).unique(subset=["symbol"], keep="last") + return df.join(extra, on="symbol", how="left") + # ================================================================ # 内部: 评分 # ================================================================ diff --git a/backend/app/strategy/monitor.py b/backend/app/strategy/monitor.py index 162b324..a6afdfd 100644 --- a/backend/app/strategy/monitor.py +++ b/backend/app/strategy/monitor.py @@ -46,7 +46,8 @@ _SIGNAL_CN: dict[str, str] = { # 行情字段 "close": "收盘价", "open": "开盘价", "high": "最高价", "low": "最低价", "change_pct": "涨跌幅", "change_amount": "涨跌额", "amplitude": "振幅", - "turnover_rate": "换手率", "volume": "成交量", "amount": "成交额", + "turnover_rate": "换手率", "volume": "成交量", "amount": "成交额", + "_volume_delta": "轮询成交量差值(手)", "_sealed_vol": "封单量(手)", # 均线 "ma5": "MA5", "ma10": "MA10", "ma20": "MA20", "ma30": "MA30", "ma60": "MA60", "ema5": "EMA5", "ema10": "EMA10", "ema20": "EMA20", @@ -983,6 +984,9 @@ class MonitorRuleEngine: elif rtype == "ladder": # 连板梯队封单监控: 独立处理 (需带预警封单值, 走专属 message) return self._evaluate_ladder(scoped, rule, now) + elif rtype == "volume_delta": + # 轮询放量监控: 相邻两次全市场快照的成交量差值, 独立处理走专属 message + return self._evaluate_volume_delta(scoped, rule, now) else: # signal / price / market: 通用条件匹配 for sym, name, price, pct, hit_sigs in self._match_conditions(scoped, rule): @@ -1364,6 +1368,140 @@ class MonitorRuleEngine: results.append((sym, name, price, pct, hit_sigs)) return results + @staticmethod + def _volume_delta_basic_mask(df: pl.DataFrame, bf: dict, name_map: dict[str, str]) -> pl.Expr | None: + """轮询放量基础过滤掩码 (与策略 basic_filter 语义对齐, 字段缺失时该项跳过)。 + + 支持: price_min/max (收盘价), market_cap_min (总市值=close x total_shares), + float_cap_min/max (流通市值), amount_min (当日累计成交额), exclude_st (名称含 ST)。 + """ + masks: list[pl.Expr] = [] + if bf.get("price_min") is not None: + masks.append(pl.col("close") >= float(bf["price_min"])) + if bf.get("price_max") is not None: + masks.append(pl.col("close") <= float(bf["price_max"])) + if bf.get("amount_min") is not None and "amount" in df.columns: + masks.append(pl.col("amount") >= float(bf["amount_min"])) + if bf.get("market_cap_min") is not None and "total_shares" in df.columns: + masks.append((pl.col("close") * pl.col("total_shares")) >= float(bf["market_cap_min"])) + if bf.get("float_cap_min") is not None and "float_shares" in df.columns: + masks.append((pl.col("close") * pl.col("float_shares")) >= float(bf["float_cap_min"])) + if bf.get("float_cap_max") is not None and "float_shares" in df.columns: + masks.append((pl.col("close") * pl.col("float_shares")) <= float(bf["float_cap_max"])) + if bf.get("exclude_st") and name_map: + st_symbols = [ + sym for sym, name in name_map.items() + if name and "ST" in str(name).upper() + ] + if st_symbols: + masks.append(~pl.col("symbol").is_in(st_symbols)) + if not masks: + return None + return pl.all_horizontal(masks) + + def _evaluate_volume_delta(self, scoped: pl.DataFrame, rule: dict, now: float) -> list[dict]: + """评估轮询放量监控: 相邻两次全市场快照的成交量/成交额差值。 + + 差值列 _volume_delta(手)/_volume_delta_amount(元)/间隔列 _volume_delta_span + 由 quote_service 评估前注入。metric=volume 按手数、amount 按金额比较阈值; + basic_filter 先行过滤 (股价/市值/成交额/ST, 与策略 basic_filter 语义对齐)。 + 命中 >5 只时合并为一条批量事件防刷屏。 + """ + if "_volume_delta" not in scoped.columns: + return [] # 无差值数据 (首轮/开盘保护/非全市场轮询), 安全降级 + + metric = rule.get("metric", "volume") + if metric == "amount" and "_volume_delta_amount" in scoped.columns: + cmp_col, threshold = "_volume_delta_amount", rule.get("threshold_amount", 1e6) + th_text = f"{threshold / 1e4:,.0f} 万元" + else: + cmp_col, threshold = "_volume_delta", rule.get("threshold_volume", 9000) + th_text = f"{threshold:,.0f} 手" + + cooldown = rule.get("cooldown_seconds", 300) + severity = rule.get("severity", "warn") + span_s = 0.0 + if "_volume_delta_span" in scoped.columns and scoped.height > 0: + v = scoped["_volume_delta_span"][0] + span_s = float(v) if v is not None else 0.0 + span_text = f" (间隔 {span_s:.0f}s)" if span_s > 0 else "" + + candidate = scoped + bf = rule.get("basic_filter") or {} + if bf: + mask = self._volume_delta_basic_mask(candidate, bf, self._name_map) + if mask is not None: + candidate = candidate.filter(mask) + + hit = candidate.filter( + pl.col(cmp_col).is_not_null() & (pl.col(cmp_col) >= threshold) + ).sort(cmp_col, descending=True) + if hit.is_empty(): + return [] + hit_rows = list(hit.iter_rows(named=True)) + + def _name_of(row: dict) -> str: + sym = row.get("symbol", "") + return row.get("name") or self._name_map.get(sym) or sym + + def _fmt(v) -> str: + if metric == "amount": + return f"{v / 1e4:,.0f} 万元" + return f"{v:,.0f} 手" + + def _event(symbol: str, name: str, message: str, *, delta=None, price=None, pct=None) -> dict: + ev = { + "ts": int(now * 1000), + "rule_id": rule["id"], + "rule_name": rule.get("name", ""), + "source": "volume_delta", + "type": "轮询放量", + "symbol": symbol, + "name": name, + "message": message, + "price": price, + "change_pct": pct, + "signals": [], + "severity": severity, + "conditions": [], + "logic": "and", + "volume_delta": delta, + "volume_delta_span": round(span_s, 1), + } + if metric == "amount": + ev["volume_delta_amount"] = delta + return ev + + if len(hit_rows) > 5: + top = "、".join(_name_of(r) for r in hit_rows[:8]) + suffix = "等" if len(hit_rows) > 8 else "" + message = ( + f"放量 · 单轮增量 >= {th_text}{span_text} · " + f"共 {len(hit_rows)} 只: {top}{suffix}" + ) + key = (rule["id"], "_volume_delta_batch", "volume_delta") + last = self._last_fire.get(key) + if last is not None and (now - last) < cooldown: + return [] + self._last_fire[key] = now + return [_event("", "", message)] + + events: list[dict] = [] + for row in hit_rows: + sym = row.get("symbol", "") + key = (rule["id"], sym, "volume_delta") + last = self._last_fire.get(key) + if last is not None and (now - last) < cooldown: + continue + self._last_fire[key] = now + delta = row.get(cmp_col) + message = f"放量 · 单轮增量 {_fmt(delta)} >= {th_text}{span_text}" + events.append(_event( + sym, _name_of(row), message, + delta=delta, price=row.get("close"), pct=row.get("change_pct"), + )) + return events + def _evaluate_ladder(self, scoped: pl.DataFrame, rule: dict, now: float) -> list[dict]: """评估连板梯队封单监控规则。 diff --git a/backend/app/strategy/monitor_rules.py b/backend/app/strategy/monitor_rules.py index 35cd0bc..2d6571b 100644 --- a/backend/app/strategy/monitor_rules.py +++ b/backend/app/strategy/monitor_rules.py @@ -28,7 +28,7 @@ logger = logging.getLogger(__name__) # ── 常量 ──────────────────────────────────────────────── ID_RE = re.compile(r"^[a-z0-9_]{1,40}$") -RULE_TYPES = {"strategy", "signal", "price", "market", "ladder", "sector", "abnormal"} +RULE_TYPES = {"strategy", "signal", "price", "market", "ladder", "sector", "abnormal", "volume_delta"} SCOPES = {"symbols", "all", "sector", "watchlist_group"} LOGICS = {"and", "or"} DIRECTIONS = {"entry", "exit", "both"} @@ -45,6 +45,19 @@ SECTOR_WINDOWS = {1, 3, 5, 10, 15} # abnormal 规则 (异动边缘): 接近度方向 / 关注窗口 ABNORMAL_DIRECTIONS = {"up", "down", "both"} ABNORMAL_WINDOWS = {"any", "3d", "10d", "30d"} +# volume_delta 规则 (轮询放量): 阈值口径 (手数 / 成交额) +VD_METRICS = {"volume", "amount"} +# volume_delta 基础过滤默认值 (与策略 DEFAULT_BASIC_FILTER 核心子集对齐: +# 价格 3-300 元, 总市值 >=10 亿, 当日成交额 >=2000 万, 剔除 ST) +VD_BASIC_FILTER_DEFAULTS: dict = { + "price_min": 3, + "price_max": 300, + "market_cap_min": 10e8, + "float_cap_min": None, + "float_cap_max": None, + "amount_min": 0.2e8, + "exclude_st": True, +} # 布尔信号列前缀 (op=truth 时 field 取这些) _SIGNAL_PREFIXES = ("signal_", "csg_") @@ -190,6 +203,39 @@ def validate(rule: dict) -> None: threshold_pct = rule.get("threshold_pct") if not isinstance(threshold_pct, (int, float)) or not 1 <= threshold_pct <= 150: raise ValueError("异动接近度阈值必须是 1 到 150 之间的百分比数字") + elif rule.get("type") == "volume_delta": + # 轮询放量监控: 相邻两次全市场快照的成交量/成交额差值, 不用 conditions + if rule.get("asset_type", "stock") != "stock": + raise ValueError("轮询放量监控仅支持个股 (依赖全市场股票快照)") + if rule.get("scope", "all") == "sector": + raise ValueError("轮询放量监控不支持板块作用域") + if rule.get("metric", "volume") not in VD_METRICS: + raise ValueError(f"metric 必须是 {VD_METRICS} 之一 (volume=手数, amount=金额)") + if rule.get("metric", "volume") == "amount": + thr = rule.get("threshold_amount") + if isinstance(thr, bool) or not isinstance(thr, (int, float)) or not math.isfinite(thr) or thr < 1: + raise ValueError("threshold_amount 必须是 >=1 的数字 (单轮成交额增量, 单位元)") + else: + thr = rule.get("threshold_volume") + if isinstance(thr, bool) or not isinstance(thr, (int, float)) or not math.isfinite(thr) or thr < 1: + raise ValueError("threshold_volume 必须是 >=1 的数字 (单轮成交量增量, 单位手)") + bf = rule.get("basic_filter") + if bf is not None: + if not isinstance(bf, dict): + raise ValueError("basic_filter 必须是对象") + for key, value in bf.items(): + if key == "exclude_st": + if not isinstance(value, bool): + raise ValueError("basic_filter.exclude_st 必须是布尔值") + elif key in ("price_min", "price_max", "market_cap_min", "float_cap_min", + "float_cap_max", "amount_min"): + if value is not None and ( + isinstance(value, bool) or not isinstance(value, (int, float)) + or not math.isfinite(value) or value <= 0 + ): + raise ValueError(f"basic_filter.{key} 必须是正数字或 null") + else: + raise ValueError(f"basic_filter 不支持字段: {key}") else: # 信号/价格/市场类型: 需要 conditions conds = rule.get("conditions") @@ -254,7 +300,7 @@ def normalize(rule: dict) -> dict: r.setdefault("enabled", True) r.setdefault("asset_type", "stock") # sector/abnormal 默认全市场 (sector 随后强制 all; abnormal 支持指定标的) - r.setdefault("scope", "all" if r.get("type") in {"sector", "abnormal"} else "symbols") + r.setdefault("scope", "all" if r.get("type") in {"sector", "abnormal", "volume_delta"} else "symbols") r.setdefault("symbols", []) r.setdefault("group_id", None) # watchlist_group 作用域: 成员动态来自分组, symbols 不参与; 其他作用域清掉残留 group_id @@ -290,6 +336,15 @@ def normalize(rule: dict) -> dict: # ladder 专属默认字段 r.setdefault("metric", "sealed_vol") r.setdefault("threshold", 0) + # volume_delta 专属默认字段 (轮询放量): 冷却期默认 300s 而非 3600s -- + # 持续放量会连续多轮达标, 1 小时只提醒一次太迟钝。 + if r.get("type") == "volume_delta": + if r.get("cooldown_seconds") is None: + r["cooldown_seconds"] = 300 + r["metric"] = r["metric"] if r.get("metric") in VD_METRICS else "volume" + r.setdefault("threshold_volume", 9000) + r.setdefault("threshold_amount", 1e6) + r["basic_filter"] = {**VD_BASIC_FILTER_DEFAULTS, **(r.get("basic_filter") or {})} if r.get("type") == "sector": r["scope"] = "all" r["symbols"] = [] diff --git a/backend/app/tickflow/capabilities.py b/backend/app/tickflow/capabilities.py index 60bae72..cbce736 100644 --- a/backend/app/tickflow/capabilities.py +++ b/backend/app/tickflow/capabilities.py @@ -20,6 +20,7 @@ class Cap(StrEnum): KLINE_MINUTE_BATCH = "kline.minute.batch" INTRADAY = "intraday" INTRADAY_BATCH = "intraday.batch" + INTRADAY_UNIVERSE = "intraday.universe" DEPTH5 = "depth5" DEPTH5_BATCH = "depth5.batch" WEBSOCKET = "websocket" diff --git a/backend/app/tickflow/policy.py b/backend/app/tickflow/policy.py index 54b475a..a54802f 100644 --- a/backend/app/tickflow/policy.py +++ b/backend/app/tickflow/policy.py @@ -32,7 +32,8 @@ _CAPSET_CACHE_FILE = "capabilities.json" # v2: 拆分 depth5 → depth5(单只) + depth5.batch(批量) # v3: 探测补全 quote.batch(此前 tiers.yaml 声明了但 _probe_real 漏探测) # v5: Free 档补充付费服务器 quote.by_symbol(10rpm/5标的),用于自选股实时监控。 -_CACHE_SCHEMA_VERSION = 5 +# v6: 新增 intraday.universe(全量分钟) 探测。 +_CACHE_SCHEMA_VERSION = 6 # 探测用最小代价请求:挑流通性最好的 1 只标的试 _PROBE_SYMBOL = "600000.SH" # 浦发银行,长期不会退市 @@ -234,6 +235,11 @@ def _probe_real(tiers: dict) -> tuple[CapabilitySet, list[str], set[Cap]]: lambda: tf.klines.intraday_batch([_PROBE_SYMBOL], count=1, as_dataframe=False), defaults(Cap.INTRADAY_BATCH)) + # intraday.universe — 全量分钟: 标的池单请求拉全市场最新 N 根 (Expert) + try_call(Cap.INTRADAY_UNIVERSE, + lambda: tf.klines.intraday_universe("CN_Equity_A", count=1, as_dataframe=False), + defaults(Cap.INTRADAY_UNIVERSE)) + # depth5 — 按标的查(单只) try_call(Cap.DEPTH5, lambda: tf.depth.get(_PROBE_SYMBOL), @@ -316,8 +322,6 @@ def _augment_custom_sources(capset: CapabilitySet) -> None: daily_provider = preferences.get_daily_data_provider() adj_provider = preferences.get_adj_factor_provider() - if adj_provider == "same_as_daily": - adj_provider = daily_provider active_providers = { "daily": daily_provider, "adj_factor": adj_provider, @@ -466,6 +470,7 @@ _CAP_ALIASES: dict[Cap, str] = { Cap.KLINE_MINUTE_BY_SYMBOL: "分钟K", Cap.INTRADAY: "分时", Cap.INTRADAY_BATCH: "批量分时", + Cap.INTRADAY_UNIVERSE: "全量分钟", Cap.DEPTH5: "五档", Cap.DEPTH5_BATCH: "批量五档", Cap.WEBSOCKET: "WS", diff --git a/backend/app/tickflow/repository.py b/backend/app/tickflow/repository.py index 64f89b7..f84cb55 100644 --- a/backend/app/tickflow/repository.py +++ b/backend/app/tickflow/repository.py @@ -38,6 +38,33 @@ from app.parquet import scan_enriched_parquet logger = logging.getLogger(__name__) +def replace_with_retry(src: Path, dst: Path, *, attempts: int = 10, delay_s: float = 0.5) -> None: + """os.replace 的 Windows 读锁重试版。 + + 分区 parquet 的读端 (polars scan_parquet / DuckDB read_parquet 视图) 在扫描进行 + 期间持有句柄; Windows 不允许替换"仍被读端打开"的目标文件 (PermissionError, + WinError 5), Linux 的 inode 交换语义则无此限制。读端扫描通常亚秒级完成, + 短退避重试即可穿过并发读窗口; attempts 次仍被占用则原样抛出, 由上层记录失败。 + """ + last: PermissionError | None = None + for i in range(attempts): + try: + src.replace(dst) + if i: + logger.info("parquet replace succeeded after %d blocked attempt(s): %s", i, dst) + return + except PermissionError as e: + last = e + if i == 0: + logger.warning( + "parquet replace blocked by concurrent reader, retrying (total <= %.1fs): %s", + attempts * delay_s, dst, + ) + if i < attempts - 1: + time.sleep(delay_s) + raise last # type: ignore[misc] # attempts >= 1 时 last 必已赋值 + + def enriched_dirname(asset_type: str) -> str: """asset_type → enriched parquet 目录名。ETF 走独立目录, 其余(stock)用日K enriched。""" return "kline_etf_enriched" if asset_type == "etf" else "kline_daily_enriched" @@ -1813,13 +1840,15 @@ class KlineRepository: # ================================================================ def latest_minute_date(self, symbol: str, asset_type: str = "stock") -> date | None: + # 注意: 必须走 execute_one (cursor+close)。直连 self.db.execute(...).fetchone() + # 的未消费结果集会把首个分区 parquet 的句柄钉在共享连接上, Windows 下阻塞 + # 同步写入的 os.replace → 个股分时"补齐数据"500。 table = "kline_etf_minute" if asset_type == "etf" else "kline_minute" try: - with self._lock: - row = self.db.execute( - f"SELECT max(CAST(datetime AS DATE)) FROM {table} WHERE symbol = ?", - [symbol], - ).fetchone() + row = self.execute_one( + f"SELECT max(CAST(datetime AS DATE)) FROM {table} WHERE symbol = ?", + [symbol], + ) if row and row[0]: return row[0] if isinstance(row[0], date) else date.fromisoformat(str(row[0])) except duckdb.CatalogException: @@ -1829,10 +1858,9 @@ class KlineRepository: def latest_minute_date_global(self) -> date | None: """全市场最近分钟K日期 (不分 symbol)。用于非交易日回退到上一交易日。""" try: - with self._lock: - row = self.db.execute( - "SELECT max(CAST(datetime AS DATE)) FROM kline_minute", - ).fetchone() + row = self.execute_one( + "SELECT max(CAST(datetime AS DATE)) FROM kline_minute", + ) if row and row[0]: return row[0] if isinstance(row[0], date) else date.fromisoformat(str(row[0])) except Exception: # noqa: BLE001 @@ -1841,10 +1869,9 @@ class KlineRepository: def earliest_daily_date(self) -> date | None: """本地日K数据的最早日期。""" try: - with self._lock: - res = self.db.execute( - "SELECT min(date) FROM kline_daily", - ).fetchone() + res = self.execute_one( + "SELECT min(date) FROM kline_daily", + ) if res and res[0]: d = res[0] return d if isinstance(d, date) else date.fromisoformat(str(d)) @@ -1855,10 +1882,9 @@ class KlineRepository: def earliest_minute_date(self) -> date | None: """本地分钟K数据的最早日期。""" try: - with self._lock: - res = self.db.execute( - "SELECT min(CAST(datetime AS DATE)) FROM kline_minute", - ).fetchone() + res = self.execute_one( + "SELECT min(CAST(datetime AS DATE)) FROM kline_minute", + ) if res and res[0]: d = res[0] return d if isinstance(d, date) else date.fromisoformat(str(d)) @@ -1866,13 +1892,35 @@ class KlineRepository: return None return None + def list_minute_dates(self, start: date, end: date, asset_type: str = "stock") -> list[date]: + """枚举 [start, end] 内存在的分钟K分区日 (目录名直读, 零 parquet 扫描)。 + + 分钟回测按交易日精确对日: 缺分区的日子由调用方显式跳过, + 不做"回退最近分区" (那是实盘选股的语义, 回放会串日)。 + """ + dirname = "kline_minute" if asset_type == "stock" else f"kline_{asset_type}_minute" + minute_dir = self.store.data_dir / dirname + if not minute_dir.exists(): + return [] + out: list[date] = [] + for entry in minute_dir.iterdir(): + if not (entry.is_dir() and entry.name.startswith("date=")): + continue + try: + day = date.fromisoformat(entry.name[5:]) + except ValueError: + continue + if start <= day <= end: + out.append(day) + out.sort() + return out + def latest_daily_date(self) -> date | None: """本地日K数据的最新日期。""" try: - with self._lock: - res = self.db.execute( - "SELECT max(date) FROM kline_daily", - ).fetchone() + res = self.execute_one( + "SELECT max(date) FROM kline_daily", + ) if res and res[0]: d = res[0] return d if isinstance(d, date) else date.fromisoformat(str(d)) @@ -1924,10 +1972,9 @@ class KlineRepository: def _latest_enriched_date_duckdb(self) -> date | None: try: - with self._lock: - res = self.db.execute( - "SELECT max(date) FROM kline_enriched", - ).fetchone() + res = self.execute_one( + "SELECT max(date) FROM kline_enriched", + ) if res and res[0]: d = res[0] return d if isinstance(d, date) else date.fromisoformat(str(d)) @@ -2095,10 +2142,11 @@ class KlineRepository: 直接 write_parquet(out) 在进程被 kill (dev.sh 清端口用 kill -9) 或断电时会留下半截文件, 之后 scan_parquet glob 整条链路报错。 临时文件后缀 .tmp 不匹配 *.parquet glob, 不会被扫描误读。 + Windows 下目标正被并发读取时由 replace_with_retry 短退避穿过。 """ tmp = out.with_name(out.name + ".tmp") df.write_parquet(tmp) - tmp.replace(out) # 同目录 rename, POSIX/NTFS 均为原子操作 + replace_with_retry(tmp, out) def _write_daily_partition(self, df: pl.DataFrame, table: str) -> None: """按 date 分区写入 parquet，每个日期一个文件，支持 merge-upsert。""" diff --git a/backend/pyproject.toml b/backend/pyproject.toml index 6042c2d..a3737bc 100644 --- a/backend/pyproject.toml +++ b/backend/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "tickflow-stock-panel-backend" -version = "0.2.1" +version = "0.2.2" description = "A 股选股 + 监控 + 回测面板 — TickFlow 适配" requires-python = ">=3.11" license = { text = "MIT" } diff --git a/backend/tests/backtest/test_matrix_strategy.py b/backend/tests/backtest/test_matrix_strategy.py index c1db050..f0cab1e 100644 --- a/backend/tests/backtest/test_matrix_strategy.py +++ b/backend/tests/backtest/test_matrix_strategy.py @@ -319,6 +319,7 @@ def test_builtin_matrix_strategies_use_their_declared_formula_modules(): path for path in strategy_dir.glob("*.py") if path.name != "__init__.py" ) + # 分钟形态策略 (minute_red_streak) 已迁至自定义策略目录, 内置策略全部 matrix 后端 assert len(strategy_files) == 19 for strategy_path in strategy_files: strategy = StrategyEngine._load_file(strategy_path) @@ -784,7 +785,10 @@ def test_registered_builtin_matrix_strategies_share_one_cache_profile(): strategy_dirs=[REPO_ROOT / "backend" / "app" / "strategy" / "builtin"] ) profile = build_matrix_cache_profile(engine, "stock") - strategies = engine.strategy_definitions() + strategies = tuple( + s for s in engine.strategy_definitions() + if s.execution_backend != "minute_filter" + ) assert len(strategies) == 19 assert all(strategy.execution_backend == "matrix_native" for strategy in strategies) diff --git a/backend/tests/backtest/test_minute_backtest.py b/backend/tests/backtest/test_minute_backtest.py new file mode 100644 index 0000000..7ea69f1 --- /dev/null +++ b/backend/tests/backtest/test_minute_backtest.py @@ -0,0 +1,330 @@ +"""分钟策略回测端到端集成测试 (三期 v1)。 + +用合成的分钟K分区 + 合成日线面板 + 专用测试策略, 完整跑通 +StrategyBacktestService.run() 的 minute_filter 分支: +逐日回放 (与实盘选股同一条 StrategyEngine.run 路径) → 信号分钟收盘入场 +→ 涨停拒买 → 日K矩阵离场 → 交易记录携带分钟时间戳。 + +核心断言: +- 日线窗口因果性: T 日的日线条件窗口只含 T-1 及更早 (测试策略内置守卫, + 窗口含 T 则拒绝命中 — 若回放器传错窗口, 全部用例的信号归零); +- 入场价 = 触发分钟收盘价 (entry_price_override 机制); +- 涨停拒买: 触发分钟收盘 >= 当日涨停价 (T-1 收盘 + 板块规则) 不成交; +- 缺分钟分区的交易日显式跳过 (不回退最近分区); +- 离场复用日K口径 (max_hold → 次日开盘)。 +""" +from __future__ import annotations + +from datetime import date, datetime, timedelta +from pathlib import Path + +import polars as pl +import pytest + +from app.backtest.engine import BacktestEngine +from app.backtest.strategy import StrategyBacktestConfig, StrategyBacktestService +from app.strategy.engine import StrategyEngine + +# ── 测试策略: 内置因果性守卫 ────────────────────────────────────── +# 命中条件: 当日某分钟 close > T-1 close * 1.05, 触发分钟 = 首根满足条件的K。 +# daily 窗口的最后一个日期必须 < 触发日, 否则返回空 (回放器传错窗口时信号归零)。 +TEST_STRATEGY_SOURCE = ''' +import polars as pl + +META = { + "id": "test_minute_ping", + "name": "test_minute_ping", + "asset_types": ["stock"], + "timeframes": ["1m"], + "daily_history_bars": 5, + "order_by": "close", + "descending": True, + "limit": 100, +} +EXECUTION_BACKEND = "minute_filter" + + +def filter_minute_history(df, params, *, daily=None): + if daily is None or daily.is_empty(): + return pl.DataFrame() + trigger_day = df.select(pl.col("datetime").max()).item().date() + # 因果性守卫: 日线窗口不得包含触发日。 + if daily.get_column("date").max() >= trigger_day: + return pl.DataFrame() + prev = ( + daily.sort("date").group_by("symbol").last() + .select(pl.col("symbol"), pl.col("close").alias("prev_close")) + ) + joined = df.join(prev, on="symbol", how="inner") + hits = joined.filter(pl.col("close") > pl.col("prev_close") * 1.05) + if hits.is_empty(): + return pl.DataFrame() + return ( + hits.sort("datetime").group_by("symbol").first() + .select( + pl.col("symbol"), + pl.col("datetime").alias("last_datetime"), + pl.col("close"), + ) + ) +''' + + +# ── 合成数据 ───────────────────────────────────────────────────── +def _trading_days(n: int, start: date = date(2026, 7, 1)) -> list[date]: + days: list[date] = [] + cur = start + while len(days) < n: + if cur.weekday() < 5: + days.append(cur) + cur += timedelta(days=1) + return days + + +def _daily_panel(days: list[date], symbols: list[str]) -> pl.DataFrame: + """合成日线面板: 三个符号的慢涨走势, raw_close == close (复权因子 1)。""" + rows = [] + for sym_idx, sym in enumerate(symbols): + base = 10.0 + sym_idx * 4.0 + for t, day in enumerate(days): + close = round(base * (1 + t * 0.002), 3) + open_p = round(close - 0.05, 3) + rows.append({ + "symbol": sym, + "date": day, + "open": open_p, + "high": round(close + 0.08, 3), + "low": round(open_p - 0.06, 3), + "close": close, + "raw_close": close, + # 成交额需过 DEFAULT_BASIC_FILTER.amount_min (2e8) — 命中行的 + # amount 由 T-1 enriched 快照联表注入 (与实盘同路径)。 + "volume": 2e7, + "amount": round(close * 2e7, 3), + "name": f"股票{sym_idx}", + "total_shares": 5e8, + "float_shares": 4e8, + "signal_limit_up": False, + "signal_limit_down": False, + }) + return pl.DataFrame(rows).sort(["symbol", "date"]).with_columns( + pl.col("date").cast(pl.Date), + ) + + +def _minute_frame(day: date, bars: list[tuple[str, str, float]]) -> pl.DataFrame: + """bars: (symbol, "HH:MM"(北京), close)。分区 datetime 为 naive-UTC 存储 (北京 - 8h)。""" + rows = [] + for sym, hm, close in bars: + local = datetime(day.year, day.month, day.day, int(hm[:2]), int(hm[3:])) + rows.append({ + "symbol": sym, + "datetime": local - timedelta(hours=8), + "open": close - 0.01, + "high": close + 0.01, + "low": close - 0.02, + "close": close, + "volume": 1000.0, + "amount": close * 1000.0, + }) + return pl.DataFrame(rows).sort(["symbol", "datetime"]).with_columns( + pl.col("datetime").cast(pl.Datetime("us")), + ) + + +class _FakeMinuteRepo: + """仅实现分钟回测所需的最小 repo 接口。""" + + def __init__(self, minute_frames: dict[date, pl.DataFrame]) -> None: + self.minute_frames = minute_frames + self.store = None + + def list_minute_dates(self, start, end, asset_type="stock"): + return sorted(d for d in self.minute_frames if start <= d <= end) + + def get_minute_by_dates(self, symbols, dates, asset_type="stock"): + frames = [self.minute_frames[d] for d in dates if d in self.minute_frames] + if not frames: + return pl.DataFrame( + schema={"symbol": pl.Utf8, "datetime": pl.Datetime("us"), + "open": pl.Float64, "high": pl.Float64, "low": pl.Float64, + "close": pl.Float64, "volume": pl.Float64, "amount": pl.Float64}, + ) + df = pl.concat(frames) + if symbols: + df = df.filter(pl.col("symbol").is_in(list(symbols))) + return df.sort(["symbol", "datetime"]) + + def earliest_minute_date(self): + return min(self.minute_frames) if self.minute_frames else None + + def get_index_daily(self, *args, **kwargs) -> pl.DataFrame: + return pl.DataFrame() + + +def _make_service( + tmp_path: Path, panel: pl.DataFrame, minute_frames: dict[date, pl.DataFrame], +) -> StrategyBacktestService: + strat_dir = tmp_path / "strategies" + strat_dir.mkdir(exist_ok=True) + (strat_dir / "test_minute_ping.py").write_text(TEST_STRATEGY_SOURCE, encoding="utf-8") + strategy_engine = StrategyEngine(strategy_dirs=[strat_dir]) + + repo = _FakeMinuteRepo(minute_frames) + bt_engine = BacktestEngine(repo) + + def _load_panel(self, symbols, start, end, feature_plan, asset_type="stock", **kw): + df = panel.filter((pl.col("date") >= start) & (pl.col("date") <= end)) + if symbols: + df = df.filter(pl.col("symbol").is_in(list(symbols))) + keep = set(feature_plan.base_columns) | set(feature_plan.instrument_columns) | {"symbol", "date"} + return df.select(sorted(c for c in df.columns if c in keep)) + + bt_engine.load_panel_for_backtest = _load_panel.__get__(bt_engine) + return StrategyBacktestService(bt_engine, strategy_engine) + + +def _config(start: date, end: date, **kw) -> StrategyBacktestConfig: + defaults = dict( + strategy_id="test_minute_ping", + symbols=None, + start=start, + end=end, + exit_fill="open_t+1", + max_positions=10, + mode="position", + holding_days=1, + overrides={"max_hold_days": 1}, + ) + defaults.update(kw) + return StrategyBacktestConfig(**defaults) + + +@pytest.fixture() +def scenario(tmp_path: Path): + """三个符号 x 三个回测日。面板共 30 个交易日 (指数慢涨, 涨停价按 T-1 收盘 +10%)。 + + - 000001.SZ: T1 触发 (close 10.72 > prev 10.19*1.05), T2/T3 不再触发; + - 000002.SZ: 三天都不触发 (涨幅不足 5%); + - 600000.SH: T2 触发但触发分钟收盘已达涨停价 → 拒买; T3 正常触发。 + """ + days = _trading_days(30) + t1, t2, t3 = days[-4], days[-3], days[-2] # 留一天做 T+1 离场 + symbols = ["000001.SZ", "000002.SZ", "600000.SH"] + panel = _daily_panel(days, symbols) + + def _prev_close(sym: str, before: date) -> float: + return panel.filter( + (pl.col("symbol") == sym) & (pl.col("date") < before) + ).sort("date").get_column("close")[-1] + + minute_frames = { + t1: _minute_frame(t1, [ + ("000001.SZ", "09:31", round(_prev_close("000001.SZ", t1) * 1.005, 3)), + ("000001.SZ", "09:35", round(_prev_close("000001.SZ", t1) * 1.07, 3)), # 触发 + ("000002.SZ", "09:31", round(_prev_close("000002.SZ", t1) * 1.01, 3)), + ("600000.SH", "09:31", round(_prev_close("600000.SH", t1) * 1.01, 3)), + ]), + t2: _minute_frame(t2, [ + ("000001.SZ", "09:31", round(_prev_close("000001.SZ", t2) * 1.004, 3)), + ("000002.SZ", "09:31", round(_prev_close("000002.SZ", t2) * 1.01, 3)), + # 涨停拒买: 触发分钟收盘 = T-1收盘 * 1.10 (主板涨停价, 半进位后相等) + ("600000.SH", "09:40", round(_prev_close("600000.SH", t2) * 1.10, 3)), + ]), + t3: _minute_frame(t3, [ + ("000001.SZ", "09:31", round(_prev_close("000001.SZ", t3) * 1.004, 3)), + ("000002.SZ", "09:31", round(_prev_close("000002.SZ", t3) * 1.01, 3)), + ("600000.SH", "09:50", round(_prev_close("600000.SH", t3) * 1.06, 3)), # 触发 + ]), + } + service = _make_service(tmp_path, panel, minute_frames) + return service, panel, {"t1": t1, "t2": t2, "t3": t3, "t4": days[-1]}, minute_frames + + +def test_entry_at_trigger_minute_price(scenario): + service, panel, days, _ = scenario + result = service.run(_config(days["t1"], days["t3"])) + assert not result.error, result.error + entries = [t for t in result.trades if t["symbol"] == "000001.SZ"] + assert len(entries) == 1 + trade = entries[0] + # 入场价 = 触发分钟 (09:35) 收盘价, 入场时间戳精确到分钟 + prev_close = panel.filter( + (pl.col("symbol") == "000001.SZ") & (pl.col("date") < days["t1"]) + ).sort("date").get_column("close")[-1] + expected_price = round(prev_close * 1.07, 3) + assert trade["entry_price"] == pytest.approx(expected_price, abs=1e-6) + assert trade["entry_date"].startswith(f"{days['t1']} 09:35") + + +def test_daily_window_strictly_before_trigger_day(scenario): + """因果性: 测试策略拒绝含触发日的日线窗口 — 有信号即证明窗口止于 T-1。""" + service, _, days, _ = scenario + result = service.run(_config(days["t1"], days["t3"])) + assert not result.error, result.error + assert result.trades, "日线窗口若含触发日, 测试策略会拒绝命中 — 信号归零" + + +def test_limit_up_entry_rejected(scenario): + service, panel, days, _ = scenario + result = service.run(_config(days["t1"], days["t3"])) + assert not result.error, result.error + # T2 的 600000.SH 触发分钟收盘 = 涨停价 → 拒买; T3 才有它的成交 + entries_600000 = [t for t in result.trades if t["symbol"] == "600000.SH"] + assert all(t["entry_date"][:10] == str(days["t3"]) for t in entries_600000) + execution = result.stats.get("execution", {}) + assert execution.get("buy_limit_up", 0) >= 1 + replay_stats = result.stats.get("minute_replay", {}) + assert replay_stats.get("replayed_days") == 3 + + +def test_missing_partition_day_skipped(tmp_path): + days = _trading_days(30) + t1, t2, t3 = days[-4], days[-3], days[-2] + symbols = ["000001.SZ"] + panel = _daily_panel(days, symbols) + + def _prev(before: date) -> float: + return panel.filter( + (pl.col("symbol") == "000001.SZ") & (pl.col("date") < before) + ).sort("date").get_column("close")[-1] + + frames = { + t1: _minute_frame(t1, [("000001.SZ", "09:35", round(_prev(t1) * 1.07, 3))]), + t3: _minute_frame(t3, [("000001.SZ", "09:35", round(_prev(t3) * 1.06, 3))]), + # t2 无分区 → 应被跳过, 而不是回退到 t1/t3 的数据 + } + service = _make_service(tmp_path, panel, frames) + result = service.run(_config(t1, t3)) + assert not result.error, result.error + replay_stats = result.stats.get("minute_replay", {}) + assert replay_stats.get("replayed_days") == 2 + assert str(t2) in replay_stats.get("skipped_days", []) + entry_days = {t["entry_date"][:10] for t in result.trades} + assert str(t2) not in entry_days + + +def test_exit_reuses_daily_next_open(scenario): + """离场复用日K口径: max_hold=1 → 次日开盘卖出。""" + service, panel, days, _ = scenario + result = service.run(_config(days["t1"], days["t3"])) + assert not result.error, result.error + trade = next(t for t in result.trades if t["symbol"] == "000001.SZ") + entry_day = date.fromisoformat(trade["entry_date"][:10]) + exit_day = date.fromisoformat(str(trade["exit_date"])[:10]) + assert exit_day > entry_day + next_open = panel.filter( + (pl.col("symbol") == "000001.SZ") & (pl.col("date") == exit_day) + ).get_column("open")[0] + assert trade["exit_price"] == pytest.approx(next_open, abs=1e-6) + + +def test_guards(scenario, tmp_path): + service, panel, days, _ = scenario + # 信号触发卖出离场口径不支持 (通用校验或分钟分支守卫, 任一拒绝即可) + result = service.run(_config(days["t1"], days["t3"], exit_fill="signal_next_minute")) + assert result.error and "分钟" in result.error + # 无分钟分区 → 明确报错 + empty_service = _make_service(tmp_path, panel, {}) + result = empty_service.run(_config(days["t1"], days["t3"])) + assert "分钟K" in (result.error or "") diff --git a/backend/tests/fixtures/strategies/minute_red_streak.py b/backend/tests/fixtures/strategies/minute_red_streak.py new file mode 100644 index 0000000..9522e54 --- /dev/null +++ b/backend/tests/fixtures/strategies/minute_red_streak.py @@ -0,0 +1,169 @@ +"""分钟红7 — 开盘 N 根 (当日最早) 分钟K多数收红, 且最高的 top_red 根全红。 + +数据契约: filter_minute_history 接收当日全市场分钟K窗口 +(symbol, datetime, open, high, low, close, volume, amount), +由 ScreenerService.build_strategy_context 的 1m 分支从本地 kline_minute +分区注入; 策略本身不感知数据来源 (本地同步 / 盘中增量刷新对它透明)。 + +META["daily_history_bars"] 声明叠加日线维度的条件 (N 日内涨停过): +引擎会以 daily= 关键字注入日线 enriched 窗口, 涨停判定直接复用 +enriched 预计算信号 — signal_limit_up (收盘封板) 或 signal_broken_limit_up +(炸板: 盘中触及涨停未封住), 任一命中即算"盘中涨停过"。 + +本文件是分钟红7的参考实现 (测试夹具): 该策略按用户要求以自定义策略形态 +交付, 正式位置为运行时 data/strategies/custom/minute_red_streak.py +(gitignore, 用户可自行修改); 引擎按 id 加载, 参数覆盖不受位置影响。 +""" + +import polars as pl + +META = { + "id": "minute_red_streak", + "name": "分钟红7", + "description": "开盘前7根1分钟K至少5根收红, 最高的2根(按最高价)都是红K, 且近20日盘中触及过涨停", + "tags": ["分钟", "形态", "短线"], + "asset_types": ["stock"], + "timeframes": ["1m"], + # 日线 enriched 窗口 (交易日语义, 含 as_of): 覆盖 limit_up_days 参数上限 + "daily_history_bars": 60, + "params": [ + { + "id": "bars", + "label": "开盘K线数", + "type": "int", + "default": 7, + "min": 5, + "max": 15, + "step": 1, + }, + { + "id": "min_red", + "label": "最少红K数", + "type": "int", + "default": 5, + "min": 1, + "max": 15, + "step": 1, + }, + { + "id": "top_red", + "label": "最高K需红数", + "type": "int", + "default": 2, + "min": 1, + "max": 3, + "step": 1, + }, + { + "id": "rank_by_close", + "label": "最高K按收盘价排序", + "type": "bool", + "default": False, + }, + { + "id": "require_limit_up", + "label": "要求N日内涨停过", + "type": "bool", + "default": True, + }, + { + "id": "limit_up_days", + "label": "涨停回看天数", + "type": "int", + "default": 20, + "min": 5, + "max": 60, + "step": 1, + }, + ], + "order_by": "red_count", + "descending": True, + "limit": 100, +} + +EXECUTION_BACKEND = "minute_filter" +ENTRY_SIGNALS: list[str] = [] +EXIT_SIGNALS: list[str] = [] + + +def _recent_limit_ups(daily: pl.DataFrame | None, lookback: int) -> pl.DataFrame: + """日线窗口 → (symbol, recent_limit_ups) 近 lookback 个交易日的涨停次数。 + + 涨停过 = signal_limit_up (收盘封板) 或 signal_broken_limit_up (炸板触及)。 + 日线窗口缺失 / 无涨停信号列 → 返回空表 (调用方 inner join 即失败闭合, + 宁可漏过不可错报)。 + """ + empty = pl.DataFrame(schema={"symbol": pl.Utf8, "recent_limit_ups": pl.UInt32}) + if daily is None or daily.is_empty(): + return empty + if not {"signal_limit_up", "signal_broken_limit_up"}.issubset(daily.columns): + return empty + return ( + daily.select("symbol", "date", "signal_limit_up", "signal_broken_limit_up") + .sort(["symbol", "date"]) + .filter(pl.int_range(pl.len()).over("symbol") >= pl.len().over("symbol") - lookback) + .group_by("symbol") + .agg( + recent_limit_ups=( + pl.col("signal_limit_up").fill_null(False) + | pl.col("signal_broken_limit_up").fill_null(False) + ).sum() + ) + .filter(pl.col("recent_limit_ups") > 0) + ) + + +def filter_minute_history(df: pl.DataFrame, params: dict, *, daily: pl.DataFrame | None = None) -> pl.DataFrame: + """红K形态过滤: 全向量化, 无逐行 Python 循环。 + + - 每标的按时间取当日最早 bars 根 (开盘窗口); 不足 bars 根不触发 + - 红 = close > open; 窗口内红K数 >= min_red + - 按 rank_by (high / close) 降序取前 top_red 根, 同值取时间更晚者, 需全红 + - require_limit_up: 近 limit_up_days 个交易日盘中触及过涨停 (日线维度, + 由 daily 窗口的预计算涨停信号判定; 窗口缺失时失败闭合不触发) + """ + bars = int(params.get("bars") or 7) + min_red = min(int(params.get("min_red") or 5), bars) + top_red = min(int(params.get("top_red") or 2), bars) + rank_by = "close" if params.get("rank_by_close") else "high" + if rank_by not in df.columns: + rank_by = "high" + + windowed = ( + df.sort(["symbol", "datetime"]) + .filter(pl.int_range(pl.len()).over("symbol") < bars) + .with_columns(_red=(pl.col("close") > pl.col("open")).cast(pl.Int32)) + ) + + window = windowed.group_by("symbol").agg( + bars_checked=pl.len(), + red_count=pl.col("_red").sum(), + last_datetime=pl.col("datetime").max(), + # 输出列名用 close: 基础过滤的股价区间作用于开盘窗口末根收盘价 + close=pl.col("close").sort_by("datetime").last(), + window_high=pl.col("high").max(), + window_low=pl.col("low").min(), + window_volume=pl.col("volume").sum(), + window_amount=pl.col("amount").sum(), + ) + + top = ( + windowed.sort([rank_by, "datetime"], descending=[True, True]) + .filter(pl.int_range(pl.len()).over("symbol") < top_red) + .group_by("symbol") + .agg(top_red_count=pl.col("_red").sum()) + ) + + result = ( + window.join(top, on="symbol", how="inner") + .filter( + (pl.col("bars_checked") >= bars) + & (pl.col("red_count") >= min_red) + & (pl.col("top_red_count") >= top_red) + ) + .drop("bars_checked") + ) + if params.get("require_limit_up", True): + lookback = max(5, min(int(params.get("limit_up_days") or 20), 60)) + result = result.join(_recent_limit_ups(daily, lookback), on="symbol", how="inner") + return result diff --git a/backend/tests/test_abnormal_intraday.py b/backend/tests/test_abnormal_intraday.py new file mode 100644 index 0000000..64d0258 --- /dev/null +++ b/backend/tests/test_abnormal_intraday.py @@ -0,0 +1,75 @@ +"""盘中异动聚合测试 (build_intraday, 不依赖真实网络/enriched)。 + +覆盖: 信号命中过滤、counts 计数、优先级排序 (涨停 > 炸板 > …)、 +多信号行、limit 截断、空快照与缺信号列的降级。 +""" + +from __future__ import annotations + +from datetime import date + +import polars as pl + +from app.services.abnormal_moves import build_intraday + + +class _FakeRepo: + def __init__(self, df: pl.DataFrame): + self._df = df + + def get_enriched_latest(self): + return self._df, date(2026, 8, 28) + + +def _df(rows: list[dict]) -> pl.DataFrame: + cols = ["symbol", "name", "close", "change_pct", "amplitude", "vol_ratio_5d", + "turnover_rate", "consecutive_limit_ups", + "signal_limit_up", "signal_broken_limit_up", "signal_limit_down_recovery", + "signal_limit_down", "signal_n_day_high", "signal_n_day_low", + "signal_volume_surge"] + base = {c: [] for c in cols} + for r in rows: + for c in cols: + base[c].append(r.get(c)) + return pl.DataFrame(base) + + +def test_counts_filter_and_priority(): + repo = _FakeRepo(_df([ + {"symbol": "A1", "name": "甲", "close": 10.0, "change_pct": 0.1, + "signal_limit_up": True, "signal_n_day_high": True}, + {"symbol": "B1", "name": "乙", "close": 5.0, "change_pct": -0.05, + "signal_limit_down": True}, + {"symbol": "C1", "name": "丙", "close": 8.0, "change_pct": 0.02, + "signal_volume_surge": True}, + {"symbol": "D1", "name": "丁", "close": 7.0, "change_pct": None}, # 无信号 → 不出现 + ])) + out = build_intraday(repo) + assert out["cache_date"] == "2026-08-28" + assert out["counts"] == {"limit_up": 1, "broken": 0, "recovery": 0, + "limit_down": 1, "new_high": 1, "new_low": 0, + "volume_surge": 1} + syms = [r["symbol"] for r in out["rows"]] + assert syms == ["A1", "B1", "C1"] # 优先级: 涨停 > 跌停 > 放量; 无信号被过滤 + assert out["rows"][0]["signals"] == ["limit_up", "new_high"] # 多信号按优先级序 + + +def test_limit_truncates(): + repo = _FakeRepo(_df([ + {"symbol": f"S{i}", "signal_volume_surge": True, "change_pct": 0.01} for i in range(10) + ])) + out = build_intraday(repo, limit=3) + assert len(out["rows"]) == 3 + assert out["counts"]["volume_surge"] == 10 # counts 不受 limit 影响 + + +def test_empty_snapshot(): + repo = _FakeRepo(pl.DataFrame({"symbol": [], "name": []})) + out = build_intraday(repo) + assert out["rows"] == [] and out["counts"] == {} + + +def test_missing_signal_columns_degrades(): + repo = _FakeRepo(pl.DataFrame({"symbol": ["A1"], "name": ["甲"]})) + out = build_intraday(repo) + assert out["rows"] == [] and out["counts"] == {} diff --git a/backend/tests/test_atomic_write_retry.py b/backend/tests/test_atomic_write_retry.py new file mode 100644 index 0000000..94f22a7 --- /dev/null +++ b/backend/tests/test_atomic_write_retry.py @@ -0,0 +1,161 @@ +"""Windows 读锁竞态下 parquet 原子替换的重试测试。 + +根因: polars scan_parquet / DuckDB read_parquet 扫描进行中持有分区句柄, +Windows os.replace 替换"仍被打开"的目标文件抛 PermissionError (WinError 5); +Linux 的 inode 交换语义无此限制。表现为个股分时"补齐数据"500。 + +修复: replace_with_retry 短退避重试穿过读窗口; 永久占用则原样抛出。 +两处 _atomic_write_parquet (repository / kline_sync) 均接入。 + +另含 DuckDB 句柄泄漏回归: latest_minute_date 等曾用 self.db.execute(...) +.fetchone() 直连共享连接, 未消费结果集把首个分区句柄钉死在连接上, +导致同步 os.replace 永久被拒 (修为 execute_one cursor+close)。 +""" +from __future__ import annotations + +import os +import sys +from datetime import date, datetime + +import polars as pl +import pytest + +from app.services import kline_sync +from app.tickflow import repository + +try: + import psutil + + _PSUTIL = True +except ImportError: # pragma: no cover + _PSUTIL = False + + +def _minute_frame() -> pl.DataFrame: + return pl.DataFrame({ + "symbol": ["600519.SH"], + "datetime": [datetime(2026, 1, 15, 9, 30)], + "open": [10.0], "high": [10.5], "low": [9.5], "close": [10.2], + "volume": [100.0], "amount": [1020.0], + }) + + +def _flaky_replace(monkeypatch, fail_times: int) -> dict: + """os.replace 前 fail_times 次 raise PermissionError, 之后正常执行。""" + real_replace = os.replace + state = {"calls": 0} + + def _flaky(src, dst): + state["calls"] += 1 + if state["calls"] <= fail_times: + raise PermissionError(5, "拒绝访问。") + return real_replace(src, dst) + + monkeypatch.setattr(os, "replace", _flaky) + return state + + +# ---------- replace_with_retry 本体 ---------- + +def test_retry_succeeds_after_transient_blocks(tmp_path, monkeypatch): + out = tmp_path / "part.parquet" + out.write_bytes(b"old") + src = tmp_path / "part.parquet.tmp" + src.write_bytes(b"new") + state = _flaky_replace(monkeypatch, fail_times=2) + + repository.replace_with_retry(src, out, attempts=5, delay_s=0) + + assert out.read_bytes() == b"new" + assert state["calls"] == 3 + assert not src.exists() + + +def test_retry_exhausted_raises_last_error(tmp_path, monkeypatch): + monkeypatch.setattr(os, "replace", lambda s, d: (_ for _ in ()).throw(PermissionError(5, "拒绝访问。"))) + src = tmp_path / "a.tmp" + src.write_bytes(b"x") + + with pytest.raises(PermissionError, match="拒绝访问"): + repository.replace_with_retry(src, tmp_path / "a.parquet", attempts=3, delay_s=0) + assert src.exists() # 未被消费, 目标未生成 + + +def test_retry_no_block_single_attempt(tmp_path, monkeypatch): + out = tmp_path / "part.parquet" + out.write_bytes(b"old") + src = tmp_path / "part.parquet.tmp" + src.write_bytes(b"new") + state = _flaky_replace(monkeypatch, fail_times=0) + + repository.replace_with_retry(src, out, attempts=5, delay_s=0) + + assert state["calls"] == 1 + assert out.read_bytes() == b"new" + + +# ---------- 两处 _atomic_write_parquet 接入 ---------- + +def test_kline_sync_atomic_write_survives_transient_lock(tmp_path, monkeypatch): + state = _flaky_replace(monkeypatch, fail_times=1) + out = tmp_path / "date=2026-01-15" / "part.parquet" + out.parent.mkdir(parents=True) + + kline_sync._atomic_write_parquet(_minute_frame(), out) + + assert out.exists() + assert state["calls"] == 2 + assert pl.read_parquet(out).height == 1 + + +def test_repository_atomic_write_survives_transient_lock(tmp_path, monkeypatch): + state = _flaky_replace(monkeypatch, fail_times=1) + out = tmp_path / "kline_minute" / "date=2026-01-15" / "part.parquet" + out.parent.mkdir(parents=True) + + repository.KlineRepository._atomic_write_parquet(_minute_frame(), out) + + assert out.exists() + assert state["calls"] == 2 + + +def test_write_minute_partition_survives_reader_race(tmp_path, monkeypatch): + """集成: _write_minute_partition 读旧→concat→写新全程有读锁竞态仍完成。""" + state = _flaky_replace(monkeypatch, fail_times=2) + # 预置旧分区 (读改写路径) + old_dir = tmp_path / "date=2026-01-15" + old_dir.mkdir(parents=True) + _minute_frame().write_parquet(old_dir / "part.parquet") + + written = kline_sync._write_minute_partition(_minute_frame(), tmp_path) + + assert written == 1 + assert state["calls"] >= 3 # 至少经历了重试 + + +# ---------- DuckDB 句柄泄漏回归 (Windows 实测语义) ---------- + +@pytest.mark.skipif(sys.platform != "win32" or not _PSUTIL, reason="Windows 句柄语义 + psutil") +def test_minute_date_queries_do_not_pin_partition_handles(tmp_path): + """latest_minute_date 等查询后不得残留分区句柄。 + + 旧实现 self.db.execute(...).fetchone() 的未消费结果集经 DuckDB buffer + manager 钉住首个分区句柄, 后续同步 os.replace 永久 PermissionError。 + """ + from app.tickflow.repository import DataStore, KlineRepository + + minute_dir = tmp_path / "kline_minute" + kline_sync._write_minute_partition( + _minute_frame(), minute_dir) # date=2026-01-15 + repo = KlineRepository(DataStore(data_dir=tmp_path)) + + assert repo.latest_minute_date("600519.SH") == date(2026, 1, 15) + assert repo.latest_minute_date_global() == date(2026, 1, 15) + assert repo.earliest_minute_date() == date(2026, 1, 15) + + me = psutil.Process() + held = [f.path for f in me.open_files() if "kline_minute" in f.path] + assert held == [] + + # 钉住场景的端到端后果: 查询后重写同一分区必须成功 (旧实现在此 PermissionError) + assert kline_sync._write_minute_partition(_minute_frame(), minute_dir) == 1 diff --git a/backend/tests/test_auction_benchmark.py b/backend/tests/test_auction_benchmark.py new file mode 100644 index 0000000..3c24263 --- /dev/null +++ b/backend/tests/test_auction_benchmark.py @@ -0,0 +1,178 @@ +"""盘前风向标服务测试 (不依赖真实网络)。 + +覆盖: 交易日回退、历史日 JSON 缓存命中与落盘、收益 enrich 数学 (当日oc/全天/次日)、 +fuyao 未配置降级、目标日失败 fallback_prev、彻底失败 no_data、AI 复盘摘要段。 +日期用 2026-08-26/27/28 (写作时为过去交易日), 与仓库既有绝对日期测试风格一致。 +""" + +from __future__ import annotations + +import json +from datetime import date +from pathlib import Path + +import polars as pl +import pytest + +from app.plugins.fuyao.client import FuyaoError +from app.services import auction_benchmark as ab + + +def _write_kline(data_dir: Path, day: str, rows: list[tuple[str, float, float]]) -> None: + part = data_dir / "kline_daily" / f"date={day}" + part.mkdir(parents=True, exist_ok=True) + df = pl.DataFrame( + { + "symbol": [r[0] for r in rows], + "open": [r[1] for r in rows], + "close": [r[2] for r in rows], + } + ) + df.write_parquet(part / "part-0.parquet") + + +@pytest.fixture() +def data_dir(tmp_path: Path) -> Path: + for d in ("2026-08-26", "2026-08-27", "2026-08-28"): + (tmp_path / "kline_daily" / f"date={d}").mkdir(parents=True, exist_ok=True) + _write_kline(tmp_path, "2026-08-26", [("600519.SH", 1690.0, 1700.0), ("000858.SZ", 130.0, 131.0)]) + _write_kline(tmp_path, "2026-08-27", [("600519.SH", 1717.0, 1734.0), ("000858.SZ", 132.0, 130.0)]) + _write_kline(tmp_path, "2026-08-28", [("600519.SH", 1734.0, 1768.68), ("000858.SZ", 129.0, 133.0)]) + return tmp_path + + +class _FakeProvider: + """记录调用; fail_dates 中的日期抛 FuyaoError。""" + + def __init__(self, fail_dates: set[str] | None = None): + self.calls: list[str | None] = [] + self.fail_dates = fail_dates or set() + + def short_term_benchmark(self, date_iso: str | None) -> dict: + self.calls.append(date_iso) + if date_iso in self.fail_dates: + raise FuyaoError(f"code=3002: {date_iso} 未就绪") + return { + "date": date_iso or "2026-08-28", + "date_ms": 0, + "item": [ + {"thscode": "600519.SH", "ticker": "600519", "name": "贵州茅台", + "auction_pct": 1.0, "tags": ["白酒", "超级品牌"]}, + {"thscode": "000858.SZ", "ticker": "000858", "name": "五粮液", + "auction_pct": -2.5, "tags": ["白酒"]}, + ], + } + + +def _use_provider(monkeypatch, provider) -> _FakeProvider: + monkeypatch.setattr(ab, "_provider", lambda: provider) + return provider + + +# ---- 交易日解析 ---- + +def test_resolve_rolls_back_non_trading_day(data_dir): + assert ab.resolve_trade_date(data_dir, date(2026, 8, 30)) == date(2026, 8, 28) + assert ab.resolve_trade_date(data_dir, date(2026, 8, 27)) == date(2026, 8, 27) + + +# ---- 状态与缓存 ---- + +def test_source_unavailable_without_fuyao(data_dir, monkeypatch): + monkeypatch.setattr(ab, "_provider", lambda: None) + out = ab.get_auction_benchmark(data_dir, None) + assert out["state"] == "source_unavailable" + + +def test_fetch_stores_cache_then_hits_cache(data_dir, monkeypatch): + provider = _use_provider(monkeypatch, _FakeProvider()) + out = ab.get_auction_benchmark(data_dir, None) # 默认 → 最近分区 08-28 + assert out["state"] == "ok" and out["trade_date"] == "2026-08-28" + assert out["count"] == 2 and len(out["items"]) == 2 + assert provider.calls == ["2026-08-28"] + assert (data_dir / "auction_benchmark" / "date=2026-08-28.json").exists() + + provider.calls.clear() + out2 = ab.get_auction_benchmark(data_dir, date(2026, 8, 30)) # 周日 → 08-28 → 命中缓存 + assert out2["state"] == "ok" and out2["trade_date"] == "2026-08-28" + assert provider.calls == [] + + +def test_explicit_history_date_uses_cache(data_dir, monkeypatch): + provider = _use_provider(monkeypatch, _FakeProvider()) + ab.get_auction_benchmark(data_dir, date(2026, 8, 27)) + assert provider.calls == ["2026-08-27"] + provider.calls.clear() + ab.get_auction_benchmark(data_dir, date(2026, 8, 27)) + assert provider.calls == [] + + +def test_failure_falls_back_to_prev(data_dir, monkeypatch): + provider = _use_provider(monkeypatch, _FakeProvider(fail_dates={"2026-08-28"})) + out = ab.get_auction_benchmark(data_dir, date(2026, 8, 28)) + assert out["state"] == "fallback_prev" + assert out["trade_date"] == "2026-08-27" + assert out["requested_date"] == "2026-08-28" + # 回退日缓存以 ok 落盘 (不污染直查) + cached = json.loads((data_dir / "auction_benchmark" / "date=2026-08-27.json").read_text(encoding="utf-8")) + assert cached["state"] == "ok" + + +def test_total_failure_returns_no_data(data_dir, monkeypatch): + _use_provider(monkeypatch, _FakeProvider(fail_dates={"2026-08-28", "2026-08-27"})) + out = ab.get_auction_benchmark(data_dir, date(2026, 8, 28)) + assert out["state"] == "no_data" + assert "2026-08-28" in out.get("message", "") + + +def test_corrupt_cache_refetches(data_dir, monkeypatch): + provider = _use_provider(monkeypatch, _FakeProvider()) + cache = data_dir / "auction_benchmark" / "date=2026-08-28.json" + cache.parent.mkdir(parents=True, exist_ok=True) + cache.write_text("{broken json", encoding="utf-8") + out = ab.get_auction_benchmark(data_dir, date(2026, 8, 28)) + assert out["state"] == "ok" + assert provider.calls # 缓存损坏 → 重新拉取 + + +# ---- 收益 enrich ---- + +def test_enrich_math_with_local_kline(data_dir, monkeypatch): + # 显式查 08-27: prev=08-26, next=08-28 + _use_provider(monkeypatch, _FakeProvider()) + out = ab.get_auction_benchmark(data_dir, date(2026, 8, 27)) + by = {i["thscode"]: i for i in out["items"]} + mt = by["600519.SH"] + # day0_oc = 1734/1717-1; day0_pct = 1734/1700-1; d1 = 1768.68/1734-1 + assert mt["day0_oc"] == pytest.approx(1734.0 / 1717.0 - 1) + assert mt["day0_pct"] == pytest.approx(1734.0 / 1700.0 - 1) + assert mt["d1_pct"] == pytest.approx(1768.68 / 1734.0 - 1) + wly = by["000858.SZ"] + assert wly["day0_oc"] == pytest.approx(130.0 / 132.0 - 1) + assert wly["d1_pct"] == pytest.approx(133.0 / 130.0 - 1) + # 原始字段透传 + assert mt["auction_pct"] == 1.0 and mt["tags"] == ["白酒", "超级品牌"] + + +def test_enrich_missing_kline_gives_none(data_dir, monkeypatch): + # 最新分区 08-28 无次日 → d1_pct=None; kline 行存在则 oc/pct 正常 + _use_provider(monkeypatch, _FakeProvider()) + out = ab.get_auction_benchmark(data_dir, None) + for i in out["items"]: + assert i["d1_pct"] is None + assert i["day0_oc"] is not None + + +# ---- AI 复盘摘要 ---- + +def test_build_recap_context_contains_summary(data_dir, monkeypatch): + _use_provider(monkeypatch, _FakeProvider()) + ctx = ab.build_recap_context(data_dir) + assert "盘前风向标名单" in ctx and "贵州茅台" in ctx + assert "白酒" in ctx # 概念标签 + assert "当日" in ctx # 收益对照 + + +def test_build_recap_context_empty_without_source(data_dir, monkeypatch): + monkeypatch.setattr(ab, "_provider", lambda: None) + assert ab.build_recap_context(data_dir) == "" diff --git a/backend/tests/test_capability_augment.py b/backend/tests/test_capability_augment.py index 5543be6..0044649 100644 --- a/backend/tests/test_capability_augment.py +++ b/backend/tests/test_capability_augment.py @@ -12,7 +12,7 @@ from app.tickflow.capabilities import Cap, CapabilityLimits, CapabilitySet from app.tickflow.policy import _augment_custom_sources -def _set_providers(monkeypatch, *, daily="tickflow", adj="same_as_daily", +def _set_providers(monkeypatch, *, daily="tickflow", adj="tickflow", minute="tickflow", financial="tickflow") -> None: """mock preferences 各数据集 provider getter。""" from app.services import preferences @@ -42,9 +42,9 @@ def test_daily_custom_source_grants_daily_batch(monkeypatch): assert not capset.has(Cap.FINANCIAL) -def test_adj_same_as_daily_resolves_to_daily_provider(monkeypatch): - """adj_factor_provider=same_as_daily → 跟随 daily provider 判定。""" - _set_providers(monkeypatch, daily="mock_src", adj="same_as_daily") +def test_adj_custom_source_grants_adj_factor(monkeypatch): + """adj 显式路由到声明除权的自定义源 → 补授能力 (跟随日K已下线, 独立判定)。""" + _set_providers(monkeypatch, adj="mock_src") _set_datasets(monkeypatch, {"adj_factor"}) capset = CapabilitySet() _augment_custom_sources(capset) diff --git a/backend/tests/test_capability_matrix.py b/backend/tests/test_capability_matrix.py new file mode 100644 index 0000000..351de72 --- /dev/null +++ b/backend/tests/test_capability_matrix.py @@ -0,0 +1,256 @@ +"""能力路由矩阵契约测试。 + +覆盖: 注册表与路由偏好字段一一对应、候选按各源 datasets 声明过滤、 +候选只含当前可用源 (未就绪插件进 pending 并携带原因)、TickFlow 候选 +按当前订阅档位过滤、偏好指向 tickflow 但档位不足时的 tf_available +标记、usable 跟随生效源 (各页能力门控的统一判定)、每能力独立路由 +(「跟随日K」特殊值已下线, 存量旧值由 preferences getter 自愈回退)、 +偏好指向未知源时的回退形态。全部用假插件/自定义源, +不依赖真实网络与本地 data/ 目录。 +""" +from __future__ import annotations + +from app.data_providers import custom as custom_sources +from app.data_providers.capabilities import CAPABILITY_REGISTRY, build_capability_matrix + +DEFAULT_CURRENT = { + "daily_data_provider": "tickflow", + "adj_factor_provider": "tickflow", + "minute_data_provider": "tickflow", + "depth5_data_provider": "tickflow", + "realtime_data_provider": "tickflow", + "financial_data_provider": "tickflow", +} + + +def _fake_sources(monkeypatch, plugins: list[dict], customs: list[dict] | None = None) -> None: + monkeypatch.setattr(custom_sources, "list_plugins", lambda: plugins) + monkeypatch.setattr(custom_sources, "list_sources", lambda: customs or []) + + +def _by_id(matrix: dict) -> dict[str, dict]: + return {c["id"]: c for c in matrix["capabilities"]} + + +def test_registry_covers_all_routing_fields(): + """注册表是能力的单一权威: 可路由能力与偏好键一一对应、无重复; + full_minute 为不可路由能力 (field=None, 仅 TickFlow Expert 提供)。""" + routable = [c["field"] for c in CAPABILITY_REGISTRY if c["field"] is not None] + assert sorted(routable) == sorted(DEFAULT_CURRENT) + assert len(set(routable)) == len(routable) + assert {c["id"] for c in CAPABILITY_REGISTRY} == { + "realtime", "daily", "minute", "full_minute", "depth5", "adj_factor", "financial", + } + full_minute = next(c for c in CAPABILITY_REGISTRY if c["id"] == "full_minute") + assert full_minute["field"] is None + assert full_minute["tf_tier"] == "expert" + for cap in CAPABILITY_REGISTRY: + assert cap["default"] == "tickflow" + assert cap["tf_tier"] in ("none", "starter", "pro", "expert") + assert "follow" not in cap + + +def test_matrix_without_third_party_sources(monkeypatch): + """无插件无自定义源: 每个能力只剩 TickFlow 候选, 默认路由全部生效。""" + _fake_sources(monkeypatch, []) + matrix = build_capability_matrix(dict(DEFAULT_CURRENT), tickflow_tier="expert") + assert matrix["tickflow_tier"] == "expert" + assert len(matrix["capabilities"]) == 7 + for cap in matrix["capabilities"]: + names = [c["name"] for c in cap["candidates"]] + assert names == ["tickflow"] + assert cap["candidates"][0]["kind"] == "builtin" + assert cap["tf_available"] is True + assert cap["usable"] is True + assert cap["pending"] == [] + assert cap["current"] == cap["effective"] == "tickflow" + + +def test_candidates_only_available_unready_goes_pending(monkeypatch): + """候选只包含声明了该能力且可用的源; 未就绪插件进 pending 并携带原因。""" + _fake_sources( + monkeypatch, + [ + {"name": "fuyao", "display_name": "fuyao", "datasets": ["realtime"], + "available": True, "status": "ok"}, + {"name": "sdk", "display_name": "SDK", "datasets": ["daily", "minute"], + "available": False, "status": "依赖未安装"}, + ], + [{"name": "myhttp", "display_name": "MyHTTP", "datasets": ["financial", "realtime"]}], + ) + caps = _by_id(build_capability_matrix(dict(DEFAULT_CURRENT), tickflow_tier="expert")) + assert [c["name"] for c in caps["realtime"]["candidates"]] == ["tickflow", "fuyao", "myhttp"] + assert [c["name"] for c in caps["daily"]["candidates"]] == ["tickflow"] + assert [c["name"] for c in caps["daily"]["pending"]] == ["sdk"] + assert caps["daily"]["pending"][0]["available"] is False + assert caps["daily"]["pending"][0]["note"] == "依赖未安装" + assert [c["name"] for c in caps["financial"]["candidates"]] == ["tickflow", "myhttp"] + assert caps["financial"]["candidates"][1]["kind"] == "custom" + assert [c["name"] for c in caps["adj_factor"]["candidates"]] == ["tickflow"] + + +def test_tickflow_candidates_filtered_by_tier(monkeypatch): + """TickFlow 只出现在当前档位确实提供的能力候选里 (free: 仅日K)。""" + _fake_sources(monkeypatch, []) + caps = _by_id(build_capability_matrix(dict(DEFAULT_CURRENT), tickflow_tier="free")) + assert caps["daily"]["tf_available"] is True + assert [c["name"] for c in caps["daily"]["candidates"]] == ["tickflow"] + for cap_id in ("realtime", "minute", "depth5", "adj_factor", "financial"): + assert caps[cap_id]["tf_available"] is False + assert [c["name"] for c in caps[cap_id]["candidates"]] == [] + # starter 解锁实时与除权, 分钟/五档/财务仍锁 + caps = _by_id(build_capability_matrix(dict(DEFAULT_CURRENT), tickflow_tier="starter")) + assert caps["realtime"]["tf_available"] is True + assert caps["adj_factor"]["tf_available"] is True + assert caps["minute"]["tf_available"] is False + assert caps["depth5"]["tf_available"] is False + assert caps["financial"]["tf_available"] is False + + +def test_current_tickflow_unmet_tier_flagged(monkeypatch): + """偏好仍指向 tickflow 但档位不足: current 不动, tf_available=False 供前端警示。""" + _fake_sources(monkeypatch, []) + cap = _by_id( + build_capability_matrix(dict(DEFAULT_CURRENT), tickflow_tier="free"), + )["realtime"] + assert cap["current"] == cap["effective"] == "tickflow" + assert cap["tf_available"] is False + assert cap["usable"] is False + assert [c["name"] for c in cap["candidates"]] == [] + + +def test_usable_follows_effective_provider(monkeypatch): + """usable 跟随生效源而非 TickFlow 套餐: 各页能力门控的统一判定。 + + - 生效源是可用插件 → usable (即使 TickFlow 档位不足); + - 生效源是未就绪插件 → 不可用 (即使有其他可用候选); + - 除权独立路由, 档位门槛/插件可用性同样生效。 + """ + _fake_sources( + monkeypatch, + [ + {"name": "fuyao", "display_name": "fuyao", "datasets": ["realtime", "minute"], + "available": True, "status": "ok"}, + {"name": "sdk", "display_name": "SDK", "datasets": ["daily", "adj_factor", "minute"], + "available": False, "status": "依赖未安装"}, + ], + ) + caps = _by_id( + build_capability_matrix( + dict(DEFAULT_CURRENT, realtime_data_provider="fuyao", minute_data_provider="sdk"), + tickflow_tier="free", + ), + ) + # 路由到可用插件: TickFlow 档位不足不影响 usable + assert caps["realtime"]["tf_available"] is False + assert caps["realtime"]["usable"] is True + # 路由到未就绪插件: 有可用候选 (fuyao) 也不算 usable + minute = caps["minute"] + assert [c["name"] for c in minute["candidates"]] == ["fuyao"] + assert minute["usable"] is False + # 除权默认路由 tickflow: 自身档位门槛 (starter+) 生效, free 档不可用 + # (独立路由, 不再随日K联动) + assert caps["adj_factor"]["effective"] == "tickflow" + assert caps["adj_factor"]["tf_available"] is False + assert caps["adj_factor"]["usable"] is False + # 除权显式路由到未就绪 sdk → 不可用 (有 tickflow 候选也不算, 日K不受影响) + caps = _by_id( + build_capability_matrix( + dict(DEFAULT_CURRENT, adj_factor_provider="sdk"), tickflow_tier="expert", + ), + ) + assert [c["name"] for c in caps["adj_factor"]["candidates"]] == ["tickflow"] + assert caps["adj_factor"]["effective"] == "sdk" + assert caps["adj_factor"]["usable"] is False + assert caps["daily"]["usable"] is True + + +def test_unknown_or_empty_tier_fails_closed(monkeypatch): + """未知档 (探测缺失) 与空档按 none 处理: 仅全档位能力 (日K) 保留 TickFlow。""" + _fake_sources(monkeypatch, []) + for tier in ("", "unknown", None): + matrix = build_capability_matrix(dict(DEFAULT_CURRENT), tickflow_tier=tier) + caps = _by_id(matrix) + assert [c["name"] for c in caps["daily"]["candidates"]] == ["tickflow"] + assert caps["realtime"]["tf_available"] is False + assert caps["realtime"]["candidates"] == [] + assert caps["minute"]["tf_available"] is False + + +def test_adj_factor_routes_independently(monkeypatch): + """除权独立路由 (跟随日K已下线): 显式切到声明除权的插件即生效, 与日K当前源无关。 + + 历史遗留值 same_as_daily 不再被矩阵特判 — 存量配置经 preferences getter + 按非法源回退 tickflow (getter 层自愈), 矩阵只信任注入值。 + """ + _fake_sources( + monkeypatch, + [{"name": "sdk", "display_name": "SDK", "datasets": ["adj_factor"], + "available": True, "status": "ok"}], + ) + # 日K走 tickflow, 除权显式走 sdk → 互不影响 (TickFlow none 档下插件照常可用) + caps = _by_id( + build_capability_matrix( + dict(DEFAULT_CURRENT, adj_factor_provider="sdk"), tickflow_tier="none", + ), + ) + adj = caps["adj_factor"] + assert adj["current"] == adj["effective"] == "sdk" + assert adj["usable"] is True + assert caps["daily"]["effective"] == "tickflow" + assert caps["daily"]["usable"] is True + + +def test_depth5_capability_semantics(monkeypatch): + """五档: pro+ 档 TickFlow 可供 (usable); 档位不足时不可用且无候选。 + + 插件数据集白名单未开放 depth5, 假插件即使声明其他数据集也不进五档候选; + 未来契约开放后声明 depth5 的源会自然成为候选 (candidates 按 datasets 过滤)。 + """ + _fake_sources( + monkeypatch, + [{"name": "fuyao", "display_name": "fuyao", "datasets": ["realtime"], + "available": True, "status": "ok"}], + ) + # pro 档: TickFlow 进候选, 默认路由 tickflow → usable + cap = _by_id(build_capability_matrix(dict(DEFAULT_CURRENT), tickflow_tier="pro"))["depth5"] + assert cap["tf_available"] is True + assert [c["name"] for c in cap["candidates"]] == ["tickflow"] + assert cap["usable"] is True + # starter 档: 档位不足 → 无候选, usable False (连板梯队封单缺数据) + cap = _by_id(build_capability_matrix(dict(DEFAULT_CURRENT), tickflow_tier="starter"))["depth5"] + assert cap["tf_available"] is False + assert cap["candidates"] == [] + assert cap["usable"] is False + + +def test_unknown_current_display_falls_back_to_name(monkeypatch): + """偏好指向未注册源 (正常经 getters 校验不会发生): 展示回退为原始名, 不抛异常。""" + _fake_sources(monkeypatch, []) + caps = _by_id( + build_capability_matrix( + dict(DEFAULT_CURRENT, realtime_data_provider="ghost"), tickflow_tier="expert", + ), + ) + assert caps["realtime"]["current"] == "ghost" + assert caps["realtime"]["current_display"] == "ghost" + assert caps["realtime"]["effective_display"] == "ghost" + + +def test_full_minute_row_is_non_routable_expert_only(monkeypatch): + """全量分钟行: field=None 不可路由, 生效源恒为 TickFlow, 按 expert 档判定可用。""" + _fake_sources(monkeypatch, []) + # 即使有插件声明别的数据集也不会成为全量分钟候选 (契约不开放该数据集) + caps = _by_id(build_capability_matrix(dict(DEFAULT_CURRENT), tickflow_tier="expert")) + fm = caps["full_minute"] + assert fm["field"] is None + assert [c["name"] for c in fm["candidates"]] == ["tickflow"] + assert fm["usable"] is True + assert fm["tf_available"] is True + assert fm["effective"] == "tickflow" + + caps_pro = _by_id(build_capability_matrix(dict(DEFAULT_CURRENT), tickflow_tier="pro")) + fm_pro = caps_pro["full_minute"] + assert fm_pro["candidates"] == [] + assert fm_pro["usable"] is False + assert fm_pro["tf_available"] is False diff --git a/backend/tests/test_custom_pct_units.py b/backend/tests/test_custom_pct_units.py index 50aeab6..616351d 100644 --- a/backend/tests/test_custom_pct_units.py +++ b/backend/tests/test_custom_pct_units.py @@ -1,15 +1,19 @@ -"""自定义源实时行情涨跌幅单位自适应归一测试。 +"""自定义源实时行情比例字段单位归一测试 (CONTRIBUTING §3.1)。 -契约要求 change_pct/amplitude/turnover_rate 用小数制 (0.0366 = 3.66%), -但不少第三方接口(如 a-stock-data)直接返回 3.66 表示 3.66%。未归一会把 -行业/概念统计与前端 x100 展示整体放大 100 倍(用户反馈)。 +契约: change_pct/amplitude/turnover_rate 为小数制 (0.0366 = 3.66%)。 +单位只认显式声明 pct_unit: percent|decimal, 不靠数值猜: + - 声明 percent → 无条件 /100; 声明 decimal → 无条件透传; + - 未声明 → change_pct 保留截面中位数判定(涨跌停 30% 上限物理可判), + amplitude/turnover_rate 置 None 交下游重算(fail-closed), + 已被 transforms 显式处理过的列视为用户接管单位, 透传。 """ + from __future__ import annotations import polars as pl import pytest -from app.data_providers.custom.config import CustomSourceConfig, DatasetConfig +from app.data_providers.custom.config import CustomSourceConfig, DatasetConfig, config_from_dict from app.data_providers.custom.provider import GenericHTTPProvider, _normalize_pct_units @@ -22,24 +26,64 @@ def _df(pcts, amps=None, turnovers=None): return pl.DataFrame(data) -def test_percent_unit_batch_is_divided_by_100(): - out = _normalize_pct_units(_df( - [1.5, -2.2, 0.9, 2.8, -1.1, 0.6, 3.3, -0.8], - amps=[2.0, 3.5, 1.8, 4.0, 2.5, 1.2, 5.0, 1.6], - turnovers=[0.5, 1.2, 0.8, 2.0, 0.9, 0.4, 1.5, 0.7], - )) +# ---- 显式声明: percent ---- + + +def test_declared_percent_divides_all_columns(): + out = _normalize_pct_units( + _df( + [1.5, -2.2, 0.9, 2.8, -1.1, 0.6, 3.3, -0.8], + amps=[2.0, 3.5, 1.8, 4.0, 2.5, 1.2, 5.0, 1.6], + turnovers=[0.5, 1.2, 0.8, 2.0, 0.9, 0.4, 1.5, 0.7], + ), + pct_unit="percent", + ) assert out["change_pct"][0] == pytest.approx(0.015) assert out["amplitude"][0] == pytest.approx(0.02) assert out["turnover_rate"][0] == pytest.approx(0.005) -def test_fraction_unit_batch_untouched(): +def test_declared_percent_wins_even_when_values_look_decimal(): + # 百分制低波动日: 0.25 表示 0.25%, 数值落在小数制区间内——声明优先, 不靠猜 + out = _normalize_pct_units( + _df( + [0.25, 0.30, 0.28, 0.27, 0.26, 0.22], + amps=[0.4, 0.5, 0.45, 0.6, 0.5, 0.4], + turnovers=[0.05, 0.08, 0.06, 0.1, 0.07, 0.05], + ), + pct_unit="percent", + ) + assert out["change_pct"][0] == pytest.approx(0.0025) + assert out["amplitude"][0] == pytest.approx(0.004) + assert out["turnover_rate"][0] == pytest.approx(0.0005) + + +# ---- 显式声明: decimal ---- + + +def test_declared_decimal_passes_through(): pcts = [0.015, -0.022, 0.009, 0.028, -0.011, 0.006, 0.033, -0.008] - out = _normalize_pct_units(_df(pcts, amps=[0.02, 0.035, 0.018, 0.04, 0.025, 0.012, 0.05, 0.016])) + out = _normalize_pct_units( + _df(pcts, amps=[0.02, 0.035, 0.018, 0.04, 0.025, 0.012, 0.05, 0.016]), pct_unit="decimal" + ) assert out["change_pct"].to_list() == pcts assert out["amplitude"][0] == pytest.approx(0.02) +def test_declared_decimal_wins_even_when_values_look_percent(): + # 用户声明了小数制就按小数制契约透传, 不替用户"修正"数据 + out = _normalize_pct_units(_df([3.66, -2.15, 0.9, 2.8, 1.1]), pct_unit="decimal") + assert out["change_pct"][0] == pytest.approx(3.66) + + +# ---- 未声明: change_pct 保留截面判定(物理可判) ---- + + +def test_undeclared_change_pct_percent_batch_normalized(): + out = _normalize_pct_units(_df([1.5, -2.2, 0.9, 2.8, 3.3, 0.6])) + assert out["change_pct"][0] == pytest.approx(0.015) + + def test_limit_up_fraction_30cm_not_divided(): # 北交所 30% 涨跌停的小数制极值不应被误判为百分制 out = _normalize_pct_units(_df([0.30, 0.29, 0.28, 0.27, 0.26])) @@ -60,38 +104,85 @@ def test_string_values_are_cast(): assert out["change_pct"][0] == pytest.approx(0.015) +# ---- 未声明: amplitude/turnover_rate fail-closed (核心修复) ---- + + +def test_undeclared_amplitude_and_turnover_are_nulled(): + # 百分制 0.05 = 0.05% 与小数制 0.05 = 5% 数值相同, 不可判定 → 置 None + out = _normalize_pct_units( + _df( + [1.5, -2.2, 0.9, 2.8, 3.3, 0.6], + amps=[2.0, 3.5, 1.8, 4.0, 5.0, 1.6], + turnovers=[0.05, 1.2, 0.8, 2.0, 1.5, 0.7], + ) + ) + assert out["amplitude"].null_count() == 6 + assert out["turnover_rate"].null_count() == 6 + # change_pct 仍正常归一 + assert out["change_pct"][0] == pytest.approx(0.015) + + +def test_undeclared_transformed_column_passes_through(): + # 用户已用 transforms 显式处理过单位(如 value / 100)的列: 视为接管, 不置 None + out = _normalize_pct_units( + _df([1.5, -2.2, 0.9, 2.8, 3.3, 0.6], turnovers=[0.005, 0.012, 0.008, 0.02, 0.015, 0.007]), + transformed_cols=frozenset({"turnover_rate"}), + ) + assert out["turnover_rate"][0] == pytest.approx(0.005) + # 未 transform 的 amplitude 仍 fail-closed + assert "amplitude" not in out.columns + + def test_missing_or_null_columns_noop(): out = _normalize_pct_units(pl.DataFrame({"close": [1.0, 2.0]})) assert out.columns == ["close"] out2 = _normalize_pct_units(_df([None, None, None, None, None, None])) assert out2["change_pct"].null_count() == 6 + # 全 null 的不可判定列保持 null + out3 = _normalize_pct_units(_df([1.5, -2.2, 0.9, 2.8, 3.3, 0.6], turnovers=[None] * 6)) + assert out3["turnover_rate"].null_count() == 6 -def _realtime_provider(rows): - provider = GenericHTTPProvider(CustomSourceConfig( - name="pct_source", - display_name="Pct Source", - datasets={"realtime": DatasetConfig( - url="https://example.test/realtime", - field_map={ - "code": "symbol", "price": "last_price", "pre_close": "prev_close", - "pct": "change_pct", "amp": "amplitude", "turnover": "turnover_rate", +# ---- provider 集成 ---- + + +def _realtime_provider(rows, **ds_kwargs): + provider = GenericHTTPProvider( + CustomSourceConfig( + name="pct_source", + display_name="Pct Source", + datasets={ + "realtime": DatasetConfig( + url="https://example.test/realtime", + field_map={ + "code": "symbol", + "price": "last_price", + "pre_close": "prev_close", + "pct": "change_pct", + "amp": "amplitude", + "turnover": "turnover_rate", + }, + **ds_kwargs, + ) }, - )}, - )) + ) + ) provider._request_rows = lambda cfg, **kwargs: rows return provider -def test_get_realtime_normalizes_percent_source(): - provider = _realtime_provider([ - {"code": "S1", "price": 10.0, "pre_close": 9.85, "pct": 1.52, "amp": 2.4, "turnover": 1.1}, - {"code": "S2", "price": 20.0, "pre_close": 20.44, "pct": -2.15, "amp": 3.1, "turnover": 0.8}, - {"code": "S3", "price": 30.0, "pre_close": 29.8, "pct": 0.67, "amp": 1.9, "turnover": 0.5}, - {"code": "S4", "price": 40.0, "pre_close": 38.9, "pct": 2.83, "amp": 4.2, "turnover": 2.0}, - {"code": "S5", "price": 50.0, "pre_close": 50.55, "pct": -1.09, "amp": 2.0, "turnover": 0.9}, - {"code": "S6", "price": 60.0, "pre_close": 59.64, "pct": 0.60, "amp": 1.6, "turnover": 0.7}, - ]) +_ROWS = [ + {"code": "S1", "price": 10.0, "pre_close": 9.85, "pct": 1.52, "amp": 2.4, "turnover": 1.1}, + {"code": "S2", "price": 20.0, "pre_close": 20.44, "pct": -2.15, "amp": 3.1, "turnover": 0.8}, + {"code": "S3", "price": 30.0, "pre_close": 29.8, "pct": 0.67, "amp": 1.9, "turnover": 0.5}, + {"code": "S4", "price": 40.0, "pre_close": 38.9, "pct": 2.83, "amp": 4.2, "turnover": 2.0}, + {"code": "S5", "price": 50.0, "pre_close": 50.55, "pct": -1.09, "amp": 2.0, "turnover": 0.9}, + {"code": "S6", "price": 60.0, "pre_close": 59.64, "pct": 0.60, "amp": 1.6, "turnover": 0.7}, +] + + +def test_get_realtime_declared_percent_source(): + provider = _realtime_provider(_ROWS, pct_unit="percent") try: rows = provider.get_realtime() finally: @@ -101,3 +192,120 @@ def test_get_realtime_normalizes_percent_source(): assert by_sym["S1"]["amplitude"] == pytest.approx(0.024) assert by_sym["S1"]["turnover_rate"] == pytest.approx(0.011) assert by_sym["S2"]["change_pct"] == pytest.approx(-0.0215) + + +def test_get_realtime_undeclared_nulls_ambiguous_columns(): + provider = _realtime_provider(_ROWS) + try: + rows = provider.get_realtime() + finally: + provider.close() + by_sym = {r["symbol"]: r for r in rows} + # change_pct 截面判定仍归一 + assert by_sym["S1"]["change_pct"] == pytest.approx(0.0152) + # 不可判定列 fail-closed + assert by_sym["S1"]["amplitude"] is None + assert by_sym["S1"]["turnover_rate"] is None + + +def test_get_realtime_transformed_turnover_kept(): + provider = _realtime_provider(_ROWS, transforms={"turnover_rate": "value / 100"}) + try: + rows = provider.get_realtime() + finally: + provider.close() + by_sym = {r["symbol"]: r for r in rows} + assert by_sym["S1"]["turnover_rate"] == pytest.approx(0.011) + assert by_sym["S1"]["amplitude"] is None + + +# ---- 配置解析与校验 ---- + + +def test_config_parses_pct_unit(): + cfg = config_from_dict( + { + "name": "s", + "datasets": { + "realtime": { + "url": "https://example.test", + "pct_unit": "Percent", + } + }, + } + ) + assert cfg.datasets["realtime"].pct_unit == "percent" + + +def test_config_rejects_invalid_pct_unit(): + with pytest.raises(ValueError, match="pct_unit"): + config_from_dict( + { + "name": "s", + "datasets": { + "realtime": { + "url": "https://example.test", + "pct_unit": "basis_point", + } + }, + } + ) + + +def test_validate_flags_pct_unit_on_non_realtime(): + provider = GenericHTTPProvider( + CustomSourceConfig( + name="s", + display_name="S", + datasets={ + "daily": DatasetConfig( + url="https://example.test", + field_map={ + "c": "symbol", + "d": "date", + "o": "open", + "h": "high", + "l": "low", + "cl": "close", + "v": "volume", + "a": "amount", + }, + pct_unit="percent", + ) + }, + ) + ) + try: + errors = provider.validate() + finally: + provider.close() + assert any("pct_unit" in e and "realtime" in e for e in errors) + + +def test_validate_flags_invalid_pct_unit_value(): + provider = GenericHTTPProvider( + CustomSourceConfig( + name="s", + display_name="S", + datasets={ + "realtime": DatasetConfig( + url="https://example.test", + field_map={ + "c": "symbol", + "p": "last_price", + "pc": "prev_close", + "o": "open", + "h": "high", + "l": "low", + "v": "volume", + }, + pct_unit="bp", + ) + }, + ) + ) + try: + errors = provider.validate() + finally: + provider.close() + assert any("pct_unit" in e for e in errors) diff --git a/backend/tests/test_dimension_intraday.py b/backend/tests/test_dimension_intraday.py new file mode 100644 index 0000000..b98d9ba --- /dev/null +++ b/backend/tests/test_dimension_intraday.py @@ -0,0 +1,158 @@ +"""板块分时 (dimension-intraday) 纯函数测试。 + +夹具: snapshot 扩展配置 (所属概念) + kline_minute/kline_daily 分区, +验证等权口径、停牌 ffill、prev_close/首根基准与各降级状态。 +""" +from __future__ import annotations + +from datetime import datetime +from pathlib import Path + +import polars as pl + +from app.api.ext_data import _dimension_intraday_compute +from app.services.ext_data import ExtConfig + + +def _mk_config() -> ExtConfig: + return ExtConfig(id="ext_gn", label="测试概念", mode="snapshot", fields=[]) + + +def _write_ext(data_dir: Path, values: dict[str, str]) -> None: + """snapshot 扩展数据: symbol → 所属概念 标签串。""" + cfg_dir = data_dir / "ext_data" / "ext_gn" + cfg_dir.mkdir(parents=True, exist_ok=True) + df = pl.DataFrame({ + "symbol": list(values.keys()), + "所属概念": list(values.values()), + }) + df.write_parquet(cfg_dir / "part.parquet") + + +def _write_minute(data_dir: Path, day: str, rows: list[tuple[str, str, float]]) -> None: + part = data_dir / "kline_minute" / f"date={day}" / "part.parquet" + part.parent.mkdir(parents=True, exist_ok=True) + df = pl.DataFrame( + { + "symbol": [r[0] for r in rows], + "datetime": [datetime.fromisoformat(r[1]) for r in rows], + "close": [r[2] for r in rows], + }, + schema_overrides={"datetime": pl.Datetime("us")}, + ) + df.write_parquet(part) + + +def _write_daily(data_dir: Path, day: str, closes: dict[str, float]) -> None: + part = data_dir / "kline_daily" / f"date={day}" / "part.parquet" + part.parent.mkdir(parents=True, exist_ok=True) + pl.DataFrame({ + "symbol": list(closes.keys()), + "close": list(closes.values()), + }).write_parquet(part) + + +def test_dimension_intraday_equal_weight_and_ffill(tmp_path: Path) -> None: + data_dir = tmp_path / "data" + _write_ext(data_dir, {"000001.SZ": "人工智能、芯片", "000002.SZ": "人工智能"}) + # 前收: 000001=10.0 (+5%/+6%/+4%), 000002=20.0, 600000 非成分股 + _write_daily(data_dir, "2026-08-27", {"000001.SZ": 10.0, "000002.SZ": 20.0, "600000.SH": 5.0}) + # 000002 在 09:32 无成交 (停牌分钟) → ffill 沿用 20.4 + _write_minute(data_dir, "2026-08-28", [ + ("000001.SZ", "2026-08-28T09:31:00", 10.5), + ("000002.SZ", "2026-08-28T09:31:00", 20.4), + ("600000.SH", "2026-08-28T09:31:00", 5.05), + ("000001.SZ", "2026-08-28T09:32:00", 10.6), + ("600000.SH", "2026-08-28T09:32:00", 5.10), + ("000001.SZ", "2026-08-28T09:33:00", 10.4), + ("000002.SZ", "2026-08-28T09:33:00", 20.8), + ("600000.SH", "2026-08-28T09:33:00", 5.20), + ]) + + payload = _dimension_intraday_compute(_mk_config(), data_dir, "所属概念", "人工智能", None) + + assert payload["status"] == "ok" + assert payload["date"] == "2026-08-28" + assert payload["basis"] == "prev_close" + assert payload["member_count"] == 2 + assert payload["members_with_minute"] == 2 + points = payload["points"] + assert [p["time"] for p in points] == ["09:31", "09:32", "09:33"] + # 09:31: 成分等权 (5% + 2%)/2 = 3.5%; 全市场 (5+2+1)/3 ≈ 2.667% (小数制) + assert points[0]["sector"] == 0.035 + assert points[0]["market"] == 0.0267 + # 09:32: 000002 ffill 20.4 → (6% + 2%)/2 = 4.0% (无 ffill 会是 6.0%) + assert points[1]["sector"] == 0.04 + assert points[1]["market"] == 0.04 + # 09:33: (4% + 4%)/2 = 4.0% + assert points[2]["sector"] == 0.04 + + +def test_dimension_intraday_tag_no_partial_match(tmp_path: Path) -> None: + data_dir = tmp_path / "data" + _write_ext(data_dir, {"000001.SZ": "人工智能体"}) + _write_minute(data_dir, "2026-08-28", [("000001.SZ", "2026-08-28T09:31:00", 10.5)]) + payload = _dimension_intraday_compute(_mk_config(), data_dir, "所属概念", "人工智能", None) + assert payload["status"] == "empty" + assert payload["reason"] == "no_members" + + +def test_dimension_intraday_no_minute_store(tmp_path: Path) -> None: + data_dir = tmp_path / "data" + _write_ext(data_dir, {"000001.SZ": "人工智能"}) + payload = _dimension_intraday_compute(_mk_config(), data_dir, "所属概念", "人工智能", None) + assert payload["status"] == "no_data" + assert payload["reason"] == "minute_missing" + + +def test_dimension_intraday_requested_date_absent(tmp_path: Path) -> None: + data_dir = tmp_path / "data" + _write_ext(data_dir, {"000001.SZ": "人工智能"}) + _write_minute(data_dir, "2026-08-28", [("000001.SZ", "2026-08-28T09:31:00", 10.5)]) + payload = _dimension_intraday_compute(_mk_config(), data_dir, "所属概念", "人工智能", "2026-08-27") + assert payload["status"] == "no_data" + assert payload["reason"] == "minute_missing" + + +def test_dimension_intraday_members_without_bars(tmp_path: Path) -> None: + """成分股全是 ETF 等无分钟数据的标的 → empty/no_member_bars。""" + data_dir = tmp_path / "data" + _write_ext(data_dir, {"510050.SH": "人工智能"}) + _write_daily(data_dir, "2026-08-27", {"510050.SH": 3.0, "000001.SZ": 10.0}) + _write_minute(data_dir, "2026-08-28", [("000001.SZ", "2026-08-28T09:31:00", 10.5)]) + payload = _dimension_intraday_compute(_mk_config(), data_dir, "所属概念", "人工智能", None) + assert payload["status"] == "empty" + assert payload["reason"] == "no_member_bars" + assert payload["member_count"] == 1 + + +def test_dimension_intraday_first_close_basis(tmp_path: Path) -> None: + """无前一交易日日K → 基准退化为当日首根 close, 曲线起点 ≈ 0。""" + data_dir = tmp_path / "data" + _write_ext(data_dir, {"000001.SZ": "人工智能"}) + _write_minute(data_dir, "2026-08-28", [ + ("000001.SZ", "2026-08-28T09:31:00", 10.0), + ("000001.SZ", "2026-08-28T09:32:00", 10.3), + ]) + payload = _dimension_intraday_compute(_mk_config(), data_dir, "所属概念", "人工智能", None) + assert payload["status"] == "ok" + assert payload["basis"] == "first_close" + assert payload["points"][0]["sector"] == 0.0 + assert payload["points"][1]["sector"] == 0.03 + + +def test_dimension_intraday_explicit_date_uses_that_partition(tmp_path: Path) -> None: + data_dir = tmp_path / "data" + _write_ext(data_dir, {"000001.SZ": "人工智能"}) + _write_daily(data_dir, "2026-08-26", {"000001.SZ": 10.0}) + _write_daily(data_dir, "2026-08-27", {"000001.SZ": 11.0}) + _write_minute(data_dir, "2026-08-27", [("000001.SZ", "2026-08-27T09:31:00", 10.5)]) + _write_minute(data_dir, "2026-08-28", [("000001.SZ", "2026-08-28T09:31:00", 12.1)]) + # 默认取最新分区 2026-08-28 → prev 为 08-27 的 11.0 → +10% + latest = _dimension_intraday_compute(_mk_config(), data_dir, "所属概念", "人工智能", None) + assert latest["date"] == "2026-08-28" + assert latest["points"][0]["sector"] == 0.1 + # 显式指定 08-27 → prev 为 08-26 的 10.0 → +5% + explicit = _dimension_intraday_compute(_mk_config(), data_dir, "所属概念", "人工智能", "2026-08-27") + assert explicit["date"] == "2026-08-27" + assert explicit["points"][0]["sector"] == 0.05 diff --git a/backend/tests/test_dragon_tiger.py b/backend/tests/test_dragon_tiger.py new file mode 100644 index 0000000..bca90c6 --- /dev/null +++ b/backend/tests/test_dragon_tiger.py @@ -0,0 +1,157 @@ +"""龙虎榜服务测试 (不依赖真实网络)。 + +覆盖: 交易日回退 (非交易日/目标日解析)、历史日 JSON 缓存命中与落盘、 +fuyao 未配置降级、当日未发布 fallback_prev、彻底失败 no_data、 +AI 复盘摘要段构建。 +日期用 2026-08-27/28 (写作时为过去交易日), 与仓库既有绝对日期测试风格一致。 +""" + +from __future__ import annotations + +import json +from datetime import date +from pathlib import Path + +import pytest + +from app.plugins.fuyao.client import FuyaoError +from app.services import dragon_tiger as dt + + +def _mk_days(data_dir: Path, *days: str) -> None: + for d in days: + (data_dir / "kline_daily" / f"date={d}").mkdir(parents=True, exist_ok=True) + + +@pytest.fixture() +def data_dir(tmp_path: Path) -> Path: + _mk_days(tmp_path, "2026-08-26", "2026-08-27", "2026-08-28") + return tmp_path + + +class _FakeProvider: + """按 (board, date) 记录调用; fail_dates 中的日期抛 FuyaoError。""" + + def __init__(self, fail_dates: set[str] | None = None): + self.calls: list[tuple] = [] + self.fail_dates = fail_dates or set() + + def dragon_tiger(self, board_type: str, date: str | None) -> dict: + self.calls.append((board_type, date)) + if date in self.fail_dates: + raise FuyaoError(f"code=3002: {date} 未就绪") + iso = date or "2026-08-28" + return { + "trade_date": iso, + "count": 2, + "stock_count": 2, + "stock_items": [ + {"thscode": "600519.SH", "ticker": "600519", "name": "贵州茅台", + "change": 0.05, "net_value": 1.5e8, "net_rate": 0.01, + "buy_value": 2e8, "sell_value": 0.5e8, "hot_rank": 3, "range_days": 1, + "org_net_value": 0.8e8}, + {"thscode": "000858.SZ", "ticker": "000858", "name": "五粮液", + "change": -0.03, "net_value": -0.6e8, "net_rate": -0.004, + "buy_value": 0.4e8, "sell_value": 1e8, "range_days": 3}, + ], + "hot_money_items": [ + {"name": "宁波桑田路", "buying": 1.9e8, + "rows": [{"thscode": "600519.SH", "name": "贵州茅台", + "hot_money_item_net_value": 1.2e8}]}, + ], + } + + +def _use_provider(monkeypatch, provider) -> _FakeProvider: + monkeypatch.setattr(dt, "_provider", lambda: provider) + return provider + + +# ---- 交易日解析 ---- + +def test_resolve_rolls_back_non_trading_day(data_dir): + # 周日 08-30 → 最近分区 08-28 + assert dt.resolve_trade_date(data_dir, date(2026, 8, 30)) == date(2026, 8, 28) + # 08-27 是交易日 → 原样 + assert dt.resolve_trade_date(data_dir, date(2026, 8, 27)) == date(2026, 8, 27) + + +def test_resolve_older_than_all_partitions_returns_target(data_dir): + # 早于全部本地分区 → 原样返回, 由调用方显式尝试 (失败如实 no_data) + assert dt.resolve_trade_date(data_dir, date(2020, 1, 1)) == date(2020, 1, 1) + + +# ---- 状态与缓存 ---- + +def test_source_unavailable_without_fuyao(data_dir, monkeypatch): + monkeypatch.setattr(dt, "_provider", lambda: None) + out = dt.get_dragon_tiger(data_dir, None) + assert out["state"] == "source_unavailable" + + +def test_fetch_stores_cache_then_hits_cache(data_dir, monkeypatch): + provider = _use_provider(monkeypatch, _FakeProvider()) + out = dt.get_dragon_tiger(data_dir, None) # 默认 → 最近分区 08-28 + assert out["state"] == "ok" + assert out["trade_date"] == "2026-08-28" + assert len(provider.calls) == 3 # 三榜一次取齐 + cache = data_dir / "dragon_tiger" / "date=2026-08-28.json" + assert cache.exists() + + provider.calls.clear() + out2 = dt.get_dragon_tiger(data_dir, date(2026, 8, 30)) # 周日 → 08-28 → 命中缓存 + assert out2["state"] == "ok" and out2["trade_date"] == "2026-08-28" + assert provider.calls == [] # 纯本地, 不打网络 + + +def test_explicit_history_date_uses_cache(data_dir, monkeypatch): + provider = _use_provider(monkeypatch, _FakeProvider()) + dt.get_dragon_tiger(data_dir, date(2026, 8, 27)) + assert provider.calls[0] == ("all", "2026-08-27") + provider.calls.clear() + dt.get_dragon_tiger(data_dir, date(2026, 8, 27)) + assert provider.calls == [] + + +def test_unpublished_today_falls_back_to_prev(data_dir, monkeypatch): + # 08-28 拉取失败 (未发布/未就绪) → 回退 08-27, state=fallback_prev + provider = _use_provider(monkeypatch, _FakeProvider(fail_dates={"2026-08-28"})) + out = dt.get_dragon_tiger(data_dir, date(2026, 8, 28)) + assert out["state"] == "fallback_prev" + assert out["trade_date"] == "2026-08-27" + assert out["requested_date"] == "2026-08-28" + # 回退日缓存以 ok 状态落盘 (不污染直查) + cached = json.loads((data_dir / "dragon_tiger" / "date=2026-08-27.json").read_text(encoding="utf-8")) + assert cached["state"] == "ok" + + +def test_total_failure_returns_no_data(data_dir, monkeypatch): + provider = _use_provider(monkeypatch, _FakeProvider(fail_dates={"2026-08-28", "2026-08-27"})) + out = dt.get_dragon_tiger(data_dir, date(2026, 8, 28)) + assert out["state"] == "no_data" + assert "2026-08-28" in out.get("message", "") + + +def test_corrupt_cache_refetches(data_dir, monkeypatch): + provider = _use_provider(monkeypatch, _FakeProvider()) + cache = data_dir / "dragon_tiger" / "date=2026-08-28.json" + cache.parent.mkdir(parents=True, exist_ok=True) + cache.write_text("{broken json", encoding="utf-8") + out = dt.get_dragon_tiger(data_dir, date(2026, 8, 28)) + assert out["state"] == "ok" + assert provider.calls # 缓存损坏 → 重新拉取 + + +# ---- AI 复盘摘要 ---- + +def test_build_recap_context_contains_summary(data_dir, monkeypatch): + _use_provider(monkeypatch, _FakeProvider()) + ctx = dt.build_recap_context(data_dir) + assert "净买入居前" in ctx and "贵州茅台" in ctx + assert "机构净买居前" in ctx + assert "宁波桑田路" in ctx + + +def test_build_recap_context_empty_without_source(data_dir, monkeypatch): + monkeypatch.setattr(dt, "_provider", lambda: None) + assert dt.build_recap_context(data_dir) == "" diff --git a/backend/tests/test_full_minute_capability.py b/backend/tests/test_full_minute_capability.py new file mode 100644 index 0000000..dc9713e --- /dev/null +++ b/backend/tests/test_full_minute_capability.py @@ -0,0 +1,52 @@ +"""全量分钟能力 (Cap.INTRADAY_UNIVERSE) 契约。 + +- 能力位存在且值为 "intraday.universe" +- tiers.yaml 仅 expert 档声明该能力 (Pro/自定义源天然没有) +- 探测层注册了该能力的探测调用, 显示标签为「全量分钟」 +- 缓存 schema 已 bump (旧 capabilities.json 触发重探测) +- 盘中分钟服务门控挂在该能力位上 +""" +from pathlib import Path +from types import SimpleNamespace + +import polars as pl + +from app.services.minute_refresh import MinuteRefreshService +from app.tickflow.capabilities import Cap, CapabilityLimits, CapabilitySet +from app.tickflow.policy import _CACHE_SCHEMA_VERSION, _CAP_ALIASES, _load_tiers_yaml + + +def test_capability_enum_value(): + assert Cap("intraday.universe") is Cap.INTRADAY_UNIVERSE + + +def test_tiers_yaml_grants_universe_to_expert_only(): + tiers = _load_tiers_yaml() + assert "intraday.universe" in tiers["expert"] + for tier in ("free", "starter", "pro"): + assert "intraday.universe" not in tiers[tier] + + +def test_policy_labels_and_cache_schema(): + assert _CAP_ALIASES[Cap.INTRADAY_UNIVERSE] == "全量分钟" + assert _CACHE_SCHEMA_VERSION >= 6 + + +def test_service_gate_requires_universe_not_batch_alone(): + class _Repo: + store = SimpleNamespace(data_dir=Path(".")) + + def get_instruments(self) -> pl.DataFrame: + return pl.DataFrame({"symbol": []}) + + def _svc_with(caps: dict) -> MinuteRefreshService: + svc = MinuteRefreshService(_Repo()) + svc.set_app_state(SimpleNamespace(capabilities=CapabilitySet(caps))) + return svc + + universe = {Cap.INTRADAY_UNIVERSE: CapabilityLimits(rpm=20)} + batch_only = {Cap.INTRADAY_BATCH: CapabilityLimits(rpm=60, batch=200)} + + assert _svc_with(universe).capability_ok() is True + assert _svc_with(batch_only).capability_ok() is False # 仅有 intraday.batch 不放行 + assert _svc_with({}).capability_ok() is False diff --git a/backend/tests/test_fuyao_financial.py b/backend/tests/test_fuyao_financial.py new file mode 100644 index 0000000..e594a9c --- /dev/null +++ b/backend/tests/test_fuyao_financial.py @@ -0,0 +1,242 @@ +"""fuyao 财务适配测试 (不依赖真实网络)。 + +覆盖: 三大报表字段映射 (canonical 列名 + 扩展列透传 + ISO 日期口径)、 +latest_only 分档 (limit 1 vs 8)、metrics 组装 (eps_basic 顺带 / bps 估值反推 / +指标 index_id 映射与未知 id 透传 / 单股指标失败不弃行)、shares 恒空、 +报告期合并写入的逐列填空语义 (并集共存, 新行缺列不覆盖旧值)。 +""" + +from __future__ import annotations + +import polars as pl +import pytest + +from app.plugins.fuyao import client as fc +from app.plugins.fuyao import provider as fp +from app.plugins.fuyao.provider import FuyaoProvider +from app.services.financial_sync import _merge_report_history + + +class _FakeFinClient: + """财务端点假客户端: 记录调用入参, 按表返回预置行。""" + + def __init__( + self, + statements: dict[str, list[dict]] | None = None, + indicators: dict[str, list[dict]] | None = None, + indicator_error: Exception | None = None, + valuations: list[dict] | None = None, + prices: list[dict] | None = None, + ): + self.statements = statements or {} + self.indicators = indicators or {} + self.indicator_error = indicator_error + self.valuations = valuations or [] + self.prices = prices or [] + self.stmt_calls: list[tuple] = [] + self.ind_calls: list[str] = [] + + def financial_statements(self, stmt, thscode, limit=1): + self.stmt_calls.append((stmt, thscode, limit)) + return [dict(r, thscode=thscode) for r in self.statements.get(stmt, [])] + + def financial_indicators(self, thscode, report): + self.ind_calls.append(f"{thscode}@{report}") + if self.indicator_error: + raise self.indicator_error + return self.indicators.get(report, []) + + def valuations_snapshot(self, thscodes): + return [r for r in self.valuations if r.get("thscode") in thscodes] + + def price_snapshot_batch(self, thscodes): + return [r for r in self.prices if r.get("thscode") in thscodes] + + +def _provider_with(monkeypatch, fake): + monkeypatch.setattr( + fp, "fuyao_client", type("M", (), {"FuyaoClient": lambda **kw: fake}) + ) + monkeypatch.setattr(fp, "get_api_key", lambda: "test-key") + monkeypatch.setattr(fp, "_HIST_INTERVAL_S", 0) + return FuyaoProvider() + + +# period_end_ms: 2026-06-30 上海零点; report_date_ms: 2026-08-15 上海零点 +_INCOME_ROW = { + "period": "quarterly", + "fiscal_year": 2026, + "fiscal_period": "Q2", + "report_date_ms": 1786723200000, + "period_end_ms": 1782748800000, + "currency": "CNY", + "operating_income": 90703260964.48, + "net_profit": 46033330566.78, + "parent_holder_net_profit": 44516880421.86, + "basic_eps": 35.57, + "operating_expenses": 50000000000.0, # 扶摇独有 → 扩展列 +} + + +def test_income_mapping_canonical_columns(monkeypatch): + fake = _FakeFinClient(statements={"income": [_INCOME_ROW]}) + provider = _provider_with(monkeypatch, fake) + df = provider.get_financials("income", ["600519.SH"], latest_only=True) + row = df.to_dicts()[0] + assert row["symbol"] == "600519.SH" + assert row["period_end"] == "2026-06-30" + assert row["announce_date"] == "2026-08-15" + assert row["revenue"] == pytest.approx(_INCOME_ROW["operating_income"]) + assert row["net_income"] == pytest.approx(_INCOME_ROW["net_profit"]) + assert row["net_income_attributable"] == pytest.approx( + _INCOME_ROW["parent_holder_net_profit"] + ) + assert row["basic_eps"] == 35.57 + # 扩展列: 扶摇独有数字字段原名透传; 字符串元数据不透传 + assert row["operating_expenses"] == 50000000000.0 + assert "thscode" not in df.columns and "ticker" not in df.columns + # 原始名不残留 (已映射字段) + assert "operating_income" not in df.columns and "net_profit" not in df.columns + + +def test_statements_limit_latest_vs_history(monkeypatch): + fake = _FakeFinClient(statements={"income": [_INCOME_ROW]}) + provider = _provider_with(monkeypatch, fake) + provider.get_financials("income", ["600519.SH"], latest_only=True) + assert fake.stmt_calls == [("income", "600519.SH", 1)] + provider.get_financials("income", ["600519.SH"], latest_only=False) + assert fake.stmt_calls[-1] == ("income", "600519.SH", fp._FINANCIAL_HISTORY_PERIODS) + + +def test_balance_and_cashflow_mapping(monkeypatch): + balance = { + "period_end_ms": 1782748800000, + "report_date_ms": 1786723200000, + "assets_total": 309050784569.31, + "total_debt": 46954432394.95, + "holder_equity_total": 262096352174.36, + } + cashflow = { + "period_end_ms": 1782748800000, + "report_date_ms": 1786723200000, + "act_cash_flow_net": 92000000000.0, + "invest_cash_flow_net": -3000000000.0, + "pay_dividends_profits_interest_cash": 64000000000.0, # 扶摇独有 → 扩展列 + } + fake = _FakeFinClient( + statements={"balance_sheet": [balance], "cash_flow": [cashflow]} + ) + provider = _provider_with(monkeypatch, fake) + bal = provider.get_financials("balance_sheet", ["600519.SH"]).to_dicts()[0] + assert bal["total_assets"] == pytest.approx(balance["assets_total"]) + assert bal["total_liabilities"] == pytest.approx(balance["total_debt"]) + assert bal["total_equity"] == pytest.approx(balance["holder_equity_total"]) + cf = provider.get_financials("cash_flow", ["600519.SH"]).to_dicts()[0] + assert cf["net_operating_cash_flow"] == pytest.approx(92000000000.0) + assert cf["net_investing_cash_flow"] == pytest.approx(-3000000000.0) + assert cf["pay_dividends_profits_interest_cash"] == pytest.approx(64000000000.0) + + +_METRICS_ABILITIES = [ + { + "ability": "profitability", + "indicators": [ + {"index_id": "index_weighted_avg_roe", "value": "16.7500"}, + {"index_id": "sale_gross_margin", "value": "89.5552"}, + ], + }, + { + "ability": "growth", + # 实测 index_id 与文档有出入; 未映射 id 原名透传 + "indicators": [{"index_id": "fixed_asset_invest_expansion_ratio", "value": "2.12587300"}], + }, + { + "ability": "solvency", + "indicators": [{"index_id": "earned_interest_multiple", "value": None}], # null → 不写列 + }, +] + + +def test_metrics_assembly(monkeypatch): + fake = _FakeFinClient( + statements={"income": [_INCOME_ROW]}, + indicators={"2026-2": _METRICS_ABILITIES}, + valuations=[{"thscode": "600519.SH", "pb_mrq": 6.455055}], + prices=[{"thscode": "600519.SH", "last_price": 1297.4}], + ) + provider = _provider_with(monkeypatch, fake) + df = provider.get_financials("metrics", ["600519.SH"], latest_only=True) + row = df.to_dicts()[0] + assert row["period_end"] == "2026-06-30" + assert row["announce_date"] == "2026-08-15" + assert row["eps_basic"] == 35.57 # 顺带取自利润表 + assert row["bps"] == pytest.approx(1297.4 / 6.455055) # 估值反推 + assert row["roe"] == pytest.approx(16.75) # 字符串 → float + assert row["gross_margin"] == pytest.approx(89.5552) + assert row["fixed_asset_invest_expansion_ratio"] == pytest.approx(2.125873) + assert "earned_interest_multiple" not in df.columns # 全空指标不成列 + assert fake.ind_calls == ["600519.SH@2026-2"] # report 由利润表最新期反推 + + +def test_metrics_indicator_failure_keeps_row(monkeypatch): + """指标端点单股失败 (如未披露期 code=5003) → 行仍写入 (eps/bps 保留)。""" + fake = _FakeFinClient( + statements={"income": [_INCOME_ROW]}, + indicator_error=fc.FuyaoError("code=5003"), + valuations=[{"thscode": "600519.SH", "pb_mrq": 6.455055}], + prices=[{"thscode": "600519.SH", "last_price": 1297.4}], + ) + provider = _provider_with(monkeypatch, fake) + df = provider.get_financials("metrics", ["600519.SH"], latest_only=True) + row = df.to_dicts()[0] + assert row["symbol"] == "600519.SH" + assert row["eps_basic"] == 35.57 + assert "roe" not in df.columns + + +def test_metrics_skips_symbol_without_income(monkeypatch): + fake = _FakeFinClient(statements={"income": []}) + provider = _provider_with(monkeypatch, fake) + assert provider.get_financials("metrics", ["600519.SH"]).is_empty() + + +def test_shares_returns_empty(monkeypatch): + provider = _provider_with(monkeypatch, _FakeFinClient()) + assert provider.get_financials("shares", ["600519.SH"]).is_empty() + + +def test_merge_fills_missing_cells_from_old_rows(): + """逐列填空: 同报告期新行缺的列由旧行补齐, 有值则覆盖 (并集共存语义)。""" + old = pl.DataFrame({ + "symbol": ["600519.SH", "600519.SH"], + "period_end": ["2026-03-31", "2026-06-30"], + "announce_date": ["2026-04-20", "2026-08-10"], + "diluted_eps": [68.1, 70.2], # tickflow 提供, fuyao 没有 + "net_income": [280.0, 460.0], + }) + new = pl.DataFrame({ + "symbol": ["600519.SH"], + "period_end": ["2026-06-30"], + "announce_date": ["2026-08-15"], # 更晚公告 → 该期以新行为基准 + "net_income": [461.5], # 修正值覆盖 + # diluted_eps 缺失 → 由旧行 70.2 补齐 + "bps": [200.99], # fuyao 扩展列, 旧行没有 + }) + merged = _merge_report_history(old, new).to_dicts() + assert len(merged) == 2 + q2 = next(r for r in merged if r["period_end"] == "2026-06-30") + assert q2["net_income"] == pytest.approx(461.5) # 新值覆盖 + assert q2["diluted_eps"] == pytest.approx(70.2) # 旧行补齐 + assert q2["bps"] == pytest.approx(200.99) # 扩展列并入 + q1 = next(r for r in merged if r["period_end"] == "2026-03-31") + assert q1["diluted_eps"] == pytest.approx(68.1) # 未触碰期原样保留 + # 旧公告覆盖新公告的倒序场景: announce 早的行不覆盖晚的 + reversed_new = pl.DataFrame({ + "symbol": ["600519.SH"], + "period_end": ["2026-06-30"], + "announce_date": ["2026-08-01"], + "net_income": [999.0], + }) + q2b = _merge_report_history(old, reversed_new).to_dicts()[1] + # 公告更晚的 old 行 (08-10) 胜出, 早公告的新行不覆盖 → 业绩修正以最新公告为准 + assert q2b["net_income"] == pytest.approx(460.0) diff --git a/backend/tests/test_fuyao_provider.py b/backend/tests/test_fuyao_provider.py index 6bfdfed..c5d3693 100644 --- a/backend/tests/test_fuyao_provider.py +++ b/backend/tests/test_fuyao_provider.py @@ -1,11 +1,18 @@ """FuyaoProvider 契约与单位标准化测试。 -不依赖真实网络: 用假 FuyaoClient 返回样例快照页, 验证字段映射、 -百分数→小数制转换 (CONTRIBUTING §3.1)、分页合并、软失败、 -能力声明 (未声明数据集回退 tickflow) 与设置页试拉。 +不依赖真实网络: 用假 FuyaoClient 返回样例快照页/历史K线/dump, 验证字段映射、 +单位口径 (CONTRIBUTING §3.1: 百分数→小数、volume 股→手)、日K取数分档 +(10d dump / 单标的接口)、除权因子推导 (交易所公式 + half-up 舍入 + +同日合并 + 涨跌停自检)、能力声明 (未声明数据集回退 tickflow) 与设置页试拉。 """ + from __future__ import annotations +import calendar +import itertools +from datetime import date, datetime, timedelta + +import polars as pl import pytest from app.plugins.fuyao import client as fc @@ -16,7 +23,13 @@ from app.plugins.fuyao.provider import FuyaoProvider class _FakeClient: """按调用次数返回预置页, 记录调用供分页断言。snapshot_all 同真实客户端语义。""" - def __init__(self, pages: list[list[dict]], count: int, error: Exception | None = None, server_ts: int = 0): + def __init__( + self, + pages: list[list[dict]], + count: int, + error: Exception | None = None, + server_ts: int = 0, + ): self.pages = pages self.count = count self.error = error @@ -73,7 +86,9 @@ def _row(thscode: str = "600519.SH", **over): def _provider_with(monkeypatch, pages, count=None, error=None, **fake_kwargs): - fake = _FakeClient(pages, count if count is not None else sum(len(p) for p in pages), error, **fake_kwargs) + fake = _FakeClient( + pages, count if count is not None else sum(len(p) for p in pages), error, **fake_kwargs + ) monkeypatch.setattr(fp, "fuyao_client", type("M", (), {"FuyaoClient": lambda **kw: fake})) monkeypatch.setattr(fp, "get_api_key", lambda: "test-key") return FuyaoProvider(), fake @@ -81,6 +96,7 @@ def _provider_with(monkeypatch, pages, count=None, error=None, **fake_kwargs): # ---- 单位与字段映射 ---- + def test_snapshot_units_and_field_mapping(monkeypatch): """核心口径: price_change_ratio_pct 百分数 → change_pct 小数制 (1.72 → 0.0172)。""" provider, _ = _provider_with(monkeypatch, [[_row()]]) @@ -94,7 +110,8 @@ def test_snapshot_units_and_field_mapping(monkeypatch): assert r["open"] == 1460.0 assert r["high"] == 1490.5 assert r["low"] == 1455.0 - assert r["volume"] == 1234500 + # volume 单位股(1234500) → 内部契约手: floor(1234500/100) = 12345 + assert r["volume"] == 12345 assert r["amount"] == 1.83e9 assert r["timestamp"] > 0 # 快照不提供的字段必须为 None, 不启发式伪造 @@ -130,10 +147,12 @@ def test_all_rows_unrecognized_returns_empty_with_no_fake_data(monkeypatch): # ---- 客户端信封解析 (实测结构 vs 文档示例) ---- + def _patch_http(monkeypatch, payload, status_code=200): class _Resp: def json(self): return payload + _Resp.status_code = status_code class _Http: @@ -148,11 +167,14 @@ def _patch_http(monkeypatch, payload, status_code=200): def test_client_parses_real_world_envelope(monkeypatch): """实测信封(2026-08): data={timestamp, total, item}。""" - _patch_http(monkeypatch, { - "code": 0, "message": "success", - "data": {"timestamp": 1787542612000, "total": 2, - "item": [_row(), _row("000001.SZ")]}, - }) + _patch_http( + monkeypatch, + { + "code": 0, + "message": "success", + "data": {"timestamp": 1787542612000, "total": 2, "item": [_row(), _row("000001.SZ")]}, + }, + ) c = fc.FuyaoClient(api_key="k") rows, total = c.snapshot_page() assert total == 2 and len(rows) == 2 @@ -161,10 +183,14 @@ def test_client_parses_real_world_envelope(monkeypatch): def test_client_parses_documented_envelope(monkeypatch): """官方文档示例信封: data={count, data}。""" - _patch_http(monkeypatch, { - "code": 0, "message": "OK", - "data": {"count": 3, "data": [_row()]}, - }) + _patch_http( + monkeypatch, + { + "code": 0, + "message": "OK", + "data": {"count": 3, "data": [_row()]}, + }, + ) c = fc.FuyaoClient(api_key="k") rows, total = c.snapshot_page() assert total == 3 and len(rows) == 1 @@ -179,13 +205,20 @@ def test_client_raises_on_error_code(monkeypatch): # ---- 字段名兼容与服务端时间戳 ---- + def test_doc_style_field_names_fallback(monkeypatch): """文档示例字段名 (highest_price/lowest_price/prev_close_price) 也能映射。""" row = { - "thscode": "600519.SH", "last_price": 1480.0, "price_change": 25.0, - "price_change_ratio_pct": 1.72, "open_price": 1460.0, - "highest_price": 1490.5, "lowest_price": 1455.0, "prev_close_price": 1455.0, - "volume": 1234500, "turnover": 1.83e9, + "thscode": "600519.SH", + "last_price": 1480.0, + "price_change": 25.0, + "price_change_ratio_pct": 1.72, + "open_price": 1460.0, + "highest_price": 1490.5, + "lowest_price": 1455.0, + "prev_close_price": 1455.0, + "volume": 1234500, + "turnover": 1.83e9, } provider, _ = _provider_with(monkeypatch, [[row]]) r = provider.get_realtime()[0] @@ -206,6 +239,7 @@ def test_realtime_falls_back_to_local_time_without_server_ts(monkeypatch): # ---- 分页 ---- + def test_snapshot_pagination_merges_pages(monkeypatch): page1 = [_row(f"{600000 + i}.SH") for i in range(2)] page2 = [_row(f"{688000 + i}.SH") for i in range(1)] @@ -224,8 +258,11 @@ def test_snapshot_stops_when_page_empty(monkeypatch): # ---- 软失败 ---- + def test_realtime_error_returns_empty_list(monkeypatch): - provider, _ = _provider_with(monkeypatch, [[]], error=fc.FuyaoError("扶摇接口错误 code=4001: 频率超限")) + provider, _ = _provider_with( + monkeypatch, [[]], error=fc.FuyaoError("扶摇接口错误 code=4001: 频率超限") + ) assert provider.get_realtime() == [] @@ -236,19 +273,23 @@ def test_client_requires_api_key(): # ---- 能力声明与注册 ---- -def test_datasets_declaration_realtime_only(): - """只声明 realtime; 其他数据集 provider_has_dataset 必须为 False (回退 tickflow)。""" + +def test_datasets_declaration(): + """声明 realtime/daily/adj_factor/financial; minute 未声明 (回退 tickflow)。""" config = FuyaoProvider().config assert "realtime" in config.datasets - assert "daily" not in config.datasets + assert "daily" in config.datasets + assert "adj_factor" in config.datasets + assert "financial" in config.datasets assert "minute" not in config.datasets - assert "financial" not in config.datasets # ---- API Key 解析 (secrets.json > .env, 对齐 tickflow 语义) ---- + def test_get_api_key_secrets_store_takes_priority(monkeypatch): from app import secrets_store + monkeypatch.delenv(fp.API_KEY_ENV, raising=False) monkeypatch.setattr(secrets_store, "load", lambda: {fp.SECRETS_FIELD: "sk-from-ui"}) assert fp.get_api_key() == "sk-from-ui" @@ -256,6 +297,7 @@ def test_get_api_key_secrets_store_takes_priority(monkeypatch): def test_get_api_key_falls_back_to_env(monkeypatch): from app import secrets_store + monkeypatch.setenv(fp.API_KEY_ENV, "sk-from-env") monkeypatch.setattr(secrets_store, "load", lambda: {}) assert fp.get_api_key() == "sk-from-env" @@ -263,6 +305,7 @@ def test_get_api_key_falls_back_to_env(monkeypatch): def test_availability_accepts_secrets_store_key(monkeypatch): from app import secrets_store + monkeypatch.delenv(fp.API_KEY_ENV, raising=False) monkeypatch.setattr(secrets_store, "load", lambda: {fp.SECRETS_FIELD: "sk-from-ui"}) assert fp.availability() == (True, "ok") @@ -270,6 +313,7 @@ def test_availability_accepts_secrets_store_key(monkeypatch): def test_availability_requires_env_key(monkeypatch): from app import secrets_store + monkeypatch.delenv(fp.API_KEY_ENV, raising=False) monkeypatch.setattr(secrets_store, "load", lambda: {}) ok, reason = fp.availability() @@ -278,6 +322,7 @@ def test_availability_requires_env_key(monkeypatch): # ---- 先探后存 (probe_api_key) ---- + def _patch_client_cls(monkeypatch, fake): monkeypatch.setattr(fp, "fuyao_client", type("M", (), {"FuyaoClient": lambda **kw: fake})) @@ -289,7 +334,10 @@ def test_probe_api_key_ok(monkeypatch): def test_probe_api_key_invalid_key(monkeypatch): - _patch_client_cls(monkeypatch, _FakeClient([[]], 0, error=fc.FuyaoError("扶摇接口错误 code=1001: 无效 api key"))) + _patch_client_cls( + monkeypatch, + _FakeClient([[]], 0, error=fc.FuyaoError("扶摇接口错误 code=1001: 无效 api key")), + ) ok, reason = fp.probe_api_key("sk-bad") assert ok is False and "无效" in reason @@ -298,13 +346,16 @@ def test_loader_probe_plugin_key_dispatch(monkeypatch): import app.plugins.fuyao.provider as provider_mod from app.data_providers.custom import loader - monkeypatch.setattr(provider_mod, "probe_api_key", lambda key: (True, "ok") if key == "good" else (False, "bad")) + monkeypatch.setattr( + provider_mod, "probe_api_key", lambda key: (True, "ok") if key == "good" else (False, "bad") + ) assert loader.probe_plugin_key("fuyao", "good") == (True, "ok") assert loader.probe_plugin_key("fuyao", "bad") == (False, "bad") def test_loader_probe_plugin_key_unsupported_plugin(): from app.data_providers.custom import loader + # stock-sdk 未声明 api_key_env → 不支持界面配 Key ok, reason = loader.probe_plugin_key("stocksdk", "x") assert ok is False and "不支持" in reason @@ -314,13 +365,16 @@ def test_loader_probe_plugin_key_unsupported_plugin(): # ---- 保存/清除端点 (直接调用 handler, 先探后存语义) ---- + def test_save_plugin_key_invalid_key_not_persisted(monkeypatch): from app.api import settings as settings_api from app.data_providers import custom as custom_sources saved: dict = {} monkeypatch.setattr(custom_sources, "probe_plugin_key", lambda n, k: (False, "Key 无效")) - monkeypatch.setattr(settings_api.secrets_store, "save", lambda updates: saved.update(updates) or updates) + monkeypatch.setattr( + settings_api.secrets_store, "save", lambda updates: saved.update(updates) or updates + ) out = settings_api.save_plugin_key(settings_api.PluginKeyIn(plugin="fuyao", api_key="bad")) assert out["ok"] is False and out["reason"] == "invalid" assert saved == {} # 无效 Key 不落盘 @@ -333,10 +387,16 @@ def test_save_plugin_key_valid_persists_and_rescans(monkeypatch): saved: dict = {} reloaded = [] monkeypatch.setattr(custom_sources, "probe_plugin_key", lambda n, k: (True, "ok")) - monkeypatch.setattr(settings_api.secrets_store, "save", lambda updates: saved.update(updates) or updates) - monkeypatch.setattr(settings_api.secrets_store, "mask", lambda key, prefix=4, suffix=4: "abcd••••wxyz") + monkeypatch.setattr( + settings_api.secrets_store, "save", lambda updates: saved.update(updates) or updates + ) + monkeypatch.setattr( + settings_api.secrets_store, "mask", lambda key, prefix=4, suffix=4: "abcd••••wxyz" + ) monkeypatch.setattr(custom_sources, "load_all", lambda: reloaded.append(1)) - monkeypatch.setattr(custom_sources, "list_plugins", lambda: [{"name": "fuyao", "available": True}]) + monkeypatch.setattr( + custom_sources, "list_plugins", lambda: [{"name": "fuyao", "available": True}] + ) out = settings_api.save_plugin_key(settings_api.PluginKeyIn(plugin="fuyao", api_key="good-key")) assert out["ok"] is True assert saved == {"fuyao_api_key": "good-key"} # 字段名与 provider.SECRETS_FIELD 一致 @@ -352,34 +412,79 @@ def test_clear_plugin_key(monkeypatch): monkeypatch.setattr(custom_sources, "is_builtin", lambda n: n == "fuyao") monkeypatch.setattr(settings_api.secrets_store, "clear", lambda *keys: cleared.extend(keys)) monkeypatch.setattr(custom_sources, "load_all", lambda: None) - monkeypatch.setattr(custom_sources, "list_plugins", lambda: [{"name": "fuyao", "available": False}]) + monkeypatch.setattr( + custom_sources, "list_plugins", lambda: [{"name": "fuyao", "available": False}] + ) out = settings_api.clear_plugin_key("fuyao") assert out["ok"] is True and out["plugin_available"] is False assert cleared == ["fuyao_api_key"] -def test_manifest_declares_realtime_dataset(): +def test_manifest_declares_datasets(): from app.data_providers.custom import loader + manifest = loader.plugin_manifest("fuyao") assert manifest is not None assert manifest["entry"] == "app.plugins.fuyao.provider:FuyaoProvider" - assert "realtime" in (manifest.get("datasets") or []) + assert {"realtime", "daily", "adj_factor"} <= set(manifest.get("datasets") or []) assert manifest.get("runtime") == "none" assert manifest.get("api_key_env") == fp.API_KEY_ENV def test_hidden_plugin_not_registered(): - """hidden: true 的插件不注册、不在数据源页展示 (优化完成前隐藏 fuyao)。""" + """fuyao 已取消隐藏 (plugin.yaml 不再声明 hidden); hidden 机制本身仍生效。 + + 用合成清单验证: hidden: true 的插件不注册、不在数据源页展示。 + """ from app.data_providers.custom import loader + manifest = loader.plugin_manifest("fuyao") - assert manifest.get("hidden") is True + assert manifest is not None and not manifest.get("hidden"), ( + "fuyao 应保持可见; 如需重新隐藏请在 plugin.yaml 声明 hidden 并更新本测试" + ) loader._register_one_plugin(manifest) - assert "fuyao" not in loader._PLUGIN_STATUS - assert "fuyao" not in loader._PROVIDERS + assert "fuyao" in loader._PLUGIN_STATUS + + hidden_manifest = dict(manifest, name="hidden_probe", hidden=True) + loader._register_one_plugin(hidden_manifest) + assert "hidden_probe" not in loader._PLUGIN_STATUS + assert "hidden_probe" not in loader._PROVIDERS + + +# ---- 插件 Key 脱敏展示 (与 TickFlow Key 契约一致) ---- + + +def test_plugin_key_masked_from_secrets_then_env(monkeypatch): + """api_key_masked 随插件状态返回: secrets.json 优先, .env 兜底, 未配置为空。 + + 完整 Key 不出后端, 只出 mask() 结果 — 与 settings API 的 + tickflow_api_key_masked 同一展示契约。 + """ + from app import secrets_store + from app.data_providers.custom import loader + + # 未声明 api_key_env / 未配置 Key → 空 + assert loader._plugin_key_masked("x", "") == "" + monkeypatch.setattr(secrets_store, "load", lambda: {}) + monkeypatch.delenv(fp.API_KEY_ENV, raising=False) + assert loader._plugin_key_masked("fuyao", fp.API_KEY_ENV) == "" + + # .env 兜底 + monkeypatch.setenv(fp.API_KEY_ENV, "env-secret-key-123456") + assert loader._plugin_key_masked("fuyao", fp.API_KEY_ENV) == "env-••••••3456" + + # secrets.json 优先于 .env + monkeypatch.setattr(secrets_store, "load", lambda: {"fuyao_api_key": "stored-secret-key-999"}) + assert loader._plugin_key_masked("fuyao", fp.API_KEY_ENV) == "stor••••••-999" + + # 注册进插件状态: 数据源列表接口据此常驻展示 (而非仅保存后瞬时显示) + loader._register_one_plugin(loader.plugin_manifest("fuyao")) + assert loader._PLUGIN_STATUS["fuyao"]["api_key_masked"] == "stor••••••-999" # ---- 设置页试拉 ---- + def test_test_dataset_realtime_preview(monkeypatch): provider, _ = _provider_with(monkeypatch, [[_row(), _row("000001.SZ")]], count=5400) out = provider.test_dataset("realtime") @@ -399,3 +504,851 @@ def test_close_is_idempotent(monkeypatch): provider, _ = _provider_with(monkeypatch, [[_row()]]) provider.close() provider.close() + + +# ===================================================================== +# daily / adj_factor (2026-08 接入) +# ===================================================================== + + +def _sh_ms(d: date) -> int: + """交易日 → 扶摇口径 ms(该日上海零点), 与 provider._ms_of_date 同式。""" + return (calendar.timegm(d.timetuple()) - 28_800) * 1000 + + +def _bar(d: date, close: float, volume: float = 1_612_611, open_: float | None = None): + """historical/dump 原始行(价格元, volume 股)。""" + return { + "date_ms": _sh_ms(d), + "open_price": open_ if open_ is not None else round(close * 0.99, 2), + "high_price": round(close * 1.01, 2), + "low_price": round(close * 0.98, 2), + "close_price": close, + "volume": volume, + "turnover": close * volume, + } + + +def _dump_bar(sym: str, d: date, close: float) -> dict: + """daily dump 行(11 列形状, 含 thscode/adjusted)。""" + return { + "thscode": sym, + "currency": "CNY", + "interval": "1d", + "adjusted": "none", + **_bar(d, close), + } + + +class _FakeHistClient: + """单标的历史K线假客户端: 按窗口过滤预置 bars, 记录调用参数。""" + + def __init__(self, bars_by_symbol: dict | None = None, error_syms: tuple = ()): + self.bars = bars_by_symbol or {} + self.errors = set(error_syms) + self.calls: list[dict] = [] + + def historical_kline(self, thscode, start_ms, end_ms, adjust="none"): + self.calls.append({"thscode": thscode, "start": start_ms, "end": end_ms, "adjust": adjust}) + if thscode in self.errors: + raise fc.FuyaoError("扶摇接口错误 code=4001: 频率超限") + return [b for b in self.bars.get(thscode, []) if start_ms <= b["date_ms"] <= end_ms] + + def dump_download_url(self, dump_kind): + raise fc.FuyaoError("测试环境无 dump") + + def close(self): + pass + + +def _hist_provider(monkeypatch, fake: _FakeHistClient, allow_dumps: bool = False) -> FuyaoProvider: + p = FuyaoProvider() + monkeypatch.setattr(fp, "fuyao_client", type("M", (), {"FuyaoClient": lambda **kw: fake})) + monkeypatch.setattr(fp, "get_api_key", lambda: "test-key") + monkeypatch.setattr(fp, "_HIST_INTERVAL_S", 0.0) + if not allow_dumps: + # 默认禁用 dump 档(单标的接口路径测试用); 大 dump 测试传 allow_dumps=True + p._ensure_daily_big_dump = lambda start_d: None # type: ignore[assignment] + return p + + +def _daily10_dump(rows: list[dict]) -> pl.DataFrame: + """11 列 daily-k-10d dump 形状。""" + return pl.DataFrame( + rows, + schema={ + "thscode": pl.String, + "currency": pl.String, + "interval": pl.String, + "adjusted": pl.String, + "date_ms": pl.Int64, + "open_price": pl.Float64, + "high_price": pl.Float64, + "low_price": pl.Float64, + "close_price": pl.Float64, + "volume": pl.Float64, + "turnover": pl.Float64, + }, + ) + + +def _adj_dump(events: list[tuple]) -> pl.DataFrame: + """8 列 adjustment-factors dump 形状。events: (thscode, ex_date, D, S, AR, AP)。""" + rows = [ + { + "thscode": s, + "ticker": s.split(".")[0], + "ex_date_ms": _sh_ms(d), + "dividend_per_share": dv, + "per_share_bonus": bn, + "allotment_ratio": ar, + "allotment_price": ap, + "currency": "CNY", + } + for s, d, dv, bn, ar, ap in events + ] + return pl.DataFrame( + rows, + schema={ + "thscode": pl.String, + "ticker": pl.String, + "ex_date_ms": pl.Int64, + "dividend_per_share": pl.Float64, + "per_share_bonus": pl.Float64, + "allotment_ratio": pl.Float64, + "allotment_price": pl.Float64, + "currency": pl.String, + }, + ) + + +def _adj_provider(monkeypatch, events: list[tuple], bars_by_symbol: dict) -> FuyaoProvider: + p = _hist_provider(monkeypatch, _FakeHistClient(bars_by_symbol)) + p._dump_memo[fp._ADJ_DUMP_KIND] = _adj_dump(events) + return p + + +# ---- daily: 单标的接口路径 ---- + + +def test_daily_api_units_volume_shares_to_lots(monkeypatch): + """核心口径: 原始价 + volume 股→手 floor(/100) + 上海零点时区。""" + bars = [ + _bar(date(2026, 8, 26), 11.10), + _bar(date(2026, 8, 27), 11.05), + _bar(date(2026, 8, 28), 11.65), + ] + provider = _hist_provider(monkeypatch, _FakeHistClient({"000001.SZ": bars})) + # 窗口跨度 28 天 > _RECENT_DUMP_DAYS → 走单标的接口 + df = provider.get_daily(["000001.SZ"], datetime(2026, 8, 1), datetime(2026, 8, 28)) + assert df.height == 3 + assert df.columns == ["symbol", "date", "open", "high", "low", "close", "volume", "amount"] + assert df.schema["date"] == pl.Date + assert df["date"].to_list() == [date(2026, 8, 26), date(2026, 8, 27), date(2026, 8, 28)] + assert df["volume"].to_list() == [16126.0] * 3 # 1,612,611 股 → 16126 手 + assert df["close"].to_list() == [11.10, 11.05, 11.65] + assert df["amount"].to_list() == [b["turnover"] for b in bars] + + +def test_daily_api_adjust_locked_to_none(monkeypatch): + """adjust=none 锁定: 服务端默认 forward, 官方前复权序列不可用(事件间逐日漂移)。""" + provider = _hist_provider( + monkeypatch, _FakeHistClient({"000001.SZ": [_bar(date(2026, 8, 27), 11.05)]}) + ) + provider.get_daily(["000001.SZ"], datetime(2026, 8, 1), datetime(2026, 8, 28)) + fake = provider._get_client() + assert fake.calls and all(c["adjust"] == "none" for c in fake.calls) + + +def test_daily_api_deep_window_splits_into_10y_chunks(monkeypatch): + """超 10 年窗口自动分片(25.6 年 → 3 片), 分片连续不重叠且各 ≤10 年。""" + provider = _hist_provider(monkeypatch, _FakeHistClient({"000001.SZ": []})) + provider.get_daily(["000001.SZ"], datetime(2001, 1, 1), datetime(2026, 8, 28)) + calls = provider._get_client().calls + assert len(calls) == 3 + for c in calls: + assert c["end"] - c["start"] < fp._HIST_MAX_SPAN_MS + for a, b in itertools.pairwise(calls): + assert b["start"] == a["end"] + 1 # 连续且不重叠 + + +def test_daily_api_soft_fail_per_symbol(monkeypatch): + """单标的失败软跳过, 不阻断其余标的。""" + bars = {"000001.SZ": [_bar(date(2026, 8, 27), 11.05)]} + provider = _hist_provider(monkeypatch, _FakeHistClient(bars, error_syms=("600519.SH",))) + df = provider.get_daily(["600519.SH", "000001.SZ"], datetime(2026, 8, 1), datetime(2026, 8, 28)) + assert df["symbol"].unique().to_list() == ["000001.SZ"] + + +def test_daily_empty_symbols_or_non_stock_returns_empty(monkeypatch): + provider = _hist_provider(monkeypatch, _FakeHistClient({})) + assert provider.get_daily([], None, None).is_empty() + assert provider.get_daily( + ["510300.SH"], datetime(2026, 8, 1), datetime(2026, 8, 28), asset_type="etf" + ).is_empty() + + +# ---- daily: 10d dump 路径 ---- + + +def _recent_daily_dump() -> pl.DataFrame: + rows = [] + for d in [date(2026, 8, 25), date(2026, 8, 26), date(2026, 8, 27), date(2026, 8, 28)]: + b = _bar(d, 11.0 + hash(d) % 3, volume=97_570_170) + rows.append( + {"thscode": "000001.SZ", "currency": "CNY", "interval": "1d", "adjusted": "none", **b} + ) + rows.append( + { + "thscode": "600519.SH", + "currency": "CNY", + "interval": "1d", + "adjusted": "none", + **_bar(d, 1480.0, volume=1_234_500), + } + ) + # 停牌行(open=high=0)应被过滤 + rows.append( + { + "thscode": "000001.SZ", + "currency": "CNY", + "interval": "1d", + "adjusted": "none", + **_bar(date(2026, 8, 27), 11.0, open_=0.0), + } + ) + rows[-1]["high_price"] = 0.0 + return _daily10_dump(rows) + + +def test_daily_recent_window_uses_dump_without_api_calls(monkeypatch): + """近端窗口: 一次 dump 覆盖全部标的, 不打单标的接口; volume 股→手; 停牌行过滤。""" + provider = _hist_provider(monkeypatch, _FakeHistClient({})) + provider._dump_memo[fp._DAILY10_DUMP_KIND] = _recent_daily_dump() + progress = [] + df = provider.get_daily( + ["000001.SZ"], + datetime(2026, 8, 26), + datetime(2026, 8, 28), + on_chunk_done=lambda c, t: progress.append((c, t)), + ) + assert provider._get_client().calls == [] # 未走单标的接口 + assert progress == [(1, 1)] + assert set(df["symbol"].to_list()) == {"000001.SZ"} # 未请求的标的不混入 + assert df["volume"].max() == 975_701.0 # 97,570,170 股 → 975,701 手 + assert date(2026, 8, 26) in df["date"].to_list() + # 停牌行(open=high=0)被 filter_halt_days 剔除, 08-27 仅保留正常行 + assert df.filter(pl.col("date") == date(2026, 8, 27)).height == 1 + + +def test_daily_dump_stale_falls_back_to_api(monkeypatch): + """dump 末端落后且 end 是工作日 → 视为有缺口, 回退单标的接口。""" + bars = {"000001.SZ": [_bar(date(2026, 8, 27), 11.05)]} + provider = _hist_provider(monkeypatch, _FakeHistClient(bars)) + provider._dump_memo[fp._DAILY10_DUMP_KIND] = _recent_daily_dump() # max=08-28 + provider.get_daily(["000001.SZ"], datetime(2026, 8, 24), datetime(2026, 9, 4)) # 09-04 周五 + assert provider._get_client().calls != [] + + +def test_daily_dump_weekend_end_covered(monkeypatch): + """end 为周末且紧随 dump 末端 → 自然缺口, 仍走 dump。""" + provider = _hist_provider(monkeypatch, _FakeHistClient({})) + provider._dump_memo[fp._DAILY10_DUMP_KIND] = _recent_daily_dump() # max=周五 08-28 + df = provider.get_daily(["000001.SZ"], datetime(2026, 8, 26), datetime(2026, 8, 30)) # 周日 + assert provider._get_client().calls == [] + assert not df.is_empty() + + +def test_daily_dump_rejects_non_raw_adjustment(monkeypatch): + """防御: dump 变为复权口径(adjusted != none)时拒绝输出, 不污染原始K线库。""" + dump = _recent_daily_dump().with_columns(pl.lit("forward").alias("adjusted")) + provider = _hist_provider(monkeypatch, _FakeHistClient({})) + provider._dump_memo[fp._DAILY10_DUMP_KIND] = dump + assert provider.get_daily( + ["000001.SZ"], datetime(2026, 8, 26), datetime(2026, 8, 28) + ).is_empty() + + +def test_daily_old_window_bypasses_dump(monkeypatch): + """深窗口且 dump 不可用 → 单标的接口兜底。""" + provider = _hist_provider(monkeypatch, _FakeHistClient({"000001.SZ": []})) + df = provider.get_daily(["000001.SZ"], datetime(2020, 1, 1), datetime(2026, 8, 28)) + assert df.is_empty() + assert provider._get_client().calls # 走了单标的接口 + + +# ---- daily: 10 年全量 dump 档 ---- + + +def _bigdump_provider( + monkeypatch, tmp_path, big_rows: list[dict], ten_memo: pl.DataFrame | None = None +): + """缓存目录重定向到 tmp_path 并放置 daily_k__.parquet; 单标的接口若被调用会被断言暴露。""" + provider = _hist_provider(monkeypatch, _FakeHistClient({}), allow_dumps=True) + monkeypatch.setattr(fp, "_cache_dir", lambda: tmp_path) + _daily10_dump(big_rows).write_parquet(tmp_path / "daily_k__20260101.parquet") + if ten_memo is not None: + provider._dump_memo[fp._DAILY10_DUMP_KIND] = ten_memo + return provider + + +def test_daily_deep_window_uses_big_dump(monkeypatch, tmp_path): + """深窗口走 10 年 dump: 不打单标的接口, volume 股→手, 未请求标的不混入。""" + rows = [ + { + "thscode": "000001.SZ", + "currency": "CNY", + "interval": "1d", + "adjusted": "none", + **_bar(date(2026, 1, 5), 11.0, volume=97_570_170), + }, + { + "thscode": "000001.SZ", + "currency": "CNY", + "interval": "1d", + "adjusted": "none", + **_bar(date(2026, 6, 12), 11.7, volume=1_234_500), + }, + { + "thscode": "000001.SZ", + "currency": "CNY", + "interval": "1d", + "adjusted": "none", + **_bar(date(2026, 8, 14), 11.4, volume=1_234_500), + }, + { + "thscode": "600519.SH", + "currency": "CNY", + "interval": "1d", + "adjusted": "none", + **_bar(date(2026, 8, 14), 1480.0, volume=1_234_500), + }, + ] + provider = _bigdump_provider(monkeypatch, tmp_path, rows) + progress = [] + df = provider.get_daily( + ["000001.SZ"], + datetime(2026, 1, 5), + datetime(2026, 8, 14), + on_chunk_done=lambda c, t: progress.append((c, t)), + ) + assert provider._get_client().calls == [] + assert progress == [(1, 1)] + assert set(df["symbol"].to_list()) == {"000001.SZ"} + assert df["date"].to_list() == [date(2026, 1, 5), date(2026, 6, 12), date(2026, 8, 14)] + assert df["volume"].to_list() == [975_701.0, 12_345.0, 12_345.0] + + +def test_daily_big_dump_tail_filled_by_10d(monkeypatch, tmp_path): + """大 dump 末端缺口(dmax 旧)由 10d dump 补尾, 两段拼接无缝。""" + big_rows = [ + { + "thscode": "000001.SZ", + "currency": "CNY", + "interval": "1d", + "adjusted": "none", + **_bar(date(2026, 8, 10), 10.9), + }, + { + "thscode": "000001.SZ", + "currency": "CNY", + "interval": "1d", + "adjusted": "none", + **_bar(date(2026, 8, 14), 11.0), + }, + ] + ten_rows = [ + { + "thscode": "000001.SZ", + "currency": "CNY", + "interval": "1d", + "adjusted": "none", + **_bar(date(2026, 8, 15), 11.2), + }, + { + "thscode": "000001.SZ", + "currency": "CNY", + "interval": "1d", + "adjusted": "none", + **_bar(date(2026, 8, 28), 11.65), + }, + ] + provider = _bigdump_provider(monkeypatch, tmp_path, big_rows, ten_memo=_daily10_dump(ten_rows)) + df = provider.get_daily(["000001.SZ"], datetime(2026, 8, 10), datetime(2026, 8, 28)) + assert df["date"].to_list() == [ + date(2026, 8, 10), + date(2026, 8, 14), + date(2026, 8, 15), + date(2026, 8, 28), + ] + assert df["close"].to_list() == [10.9, 11.0, 11.2, 11.65] + + +def test_daily_big_dump_midgap_falls_back(monkeypatch, tmp_path): + """大 dump 与 10d dump 之间有中段缺口 → 整体回退, 不拼缺口数据。""" + big_rows = [ + { + "thscode": "000001.SZ", + "currency": "CNY", + "interval": "1d", + "adjusted": "none", + **_bar(date(2026, 8, 1), 10.9), + }, + { + "thscode": "000001.SZ", + "currency": "CNY", + "interval": "1d", + "adjusted": "none", + **_bar(date(2026, 8, 5), 11.0), + }, + ] + # 10d 从 08-27 起, 与大 dump 末端 08-05 之间有中段缺口 + ten_rows = [ + { + "thscode": "000001.SZ", + "currency": "CNY", + "interval": "1d", + "adjusted": "none", + **_bar(date(2026, 8, 27), 11.5), + }, + ] + provider = _bigdump_provider(monkeypatch, tmp_path, big_rows, ten_memo=_daily10_dump(ten_rows)) + df = provider.get_daily(["000001.SZ"], datetime(2026, 8, 1), datetime(2026, 8, 28)) + assert df.is_empty() # 兜底走单标的接口(fake 为空) → 不返回缺口拼接数据 + + +def test_daily_big_dump_start_before_dmin_falls_back(monkeypatch, tmp_path): + """窗口早于大 dump 起点(>10 年) → 单标的接口兜底。""" + rows = [ + { + "thscode": "000001.SZ", + "currency": "CNY", + "interval": "1d", + "adjusted": "none", + **_bar(date(2020, 1, 2), 10.0), + }, + ] + provider = _bigdump_provider(monkeypatch, tmp_path, rows) + # 缓存不覆盖起点 → 会尝试拉最新 release; 模拟"最新 release 就是这份缓存文件" + provider._ensure_dump_path = lambda kind, prefix: tmp_path / "daily_k__20260101.parquet" # type: ignore[assignment] + bars = {"000001.SZ": [_bar(date(2015, 6, 1), 9.0)]} + provider._dump_memo.clear() + monkeypatch.setattr( + fp, "fuyao_client", type("M", (), {"FuyaoClient": lambda **kw: _FakeHistClient(bars)}) + ) + df = provider.get_daily(["000001.SZ"], datetime(2015, 1, 1), datetime(2015, 12, 31)) + assert df.height == 1 and df["close"][0] == 9.0 # 单标的兜底成功 + + +def test_tail_ok_weekend_tolerance(): + """末端容忍: 周末/节假日的自然缺口(≤3 天且 end 是周末)不算缺失。""" + assert fp._tail_ok(date(2026, 8, 28), date(2026, 8, 28)) is True + assert fp._tail_ok(date(2026, 8, 29), date(2026, 8, 28)) is True # 周六 + assert fp._tail_ok(date(2026, 8, 30), date(2026, 8, 28)) is True # 周日 + assert fp._tail_ok(date(2026, 8, 31), date(2026, 8, 28)) is False # 周一(可能有行情) + assert fp._tail_ok(date(2026, 9, 5), date(2026, 8, 28)) is False # 隔了一周 + + +# ---- adj_factor: 推导 ---- + + +def test_adj_dividend_factor_derivation(monkeypatch): + """纯分红: P=32.8, D=0.68 → ref=32.12 → factor=32.8/32.12 (与 tickflow 对拍值一致)。""" + events = [("600519.SH", date(2026, 6, 12), 0.68, 0.0, 0.0, 0.0)] + bars = {"600519.SH": [_bar(date(2026, 6, 11), 32.8), _bar(date(2026, 6, 12), 32.0)]} + provider = _adj_provider(monkeypatch, events, bars) + df = provider.get_adj_factors(["600519.SH"], datetime(2026, 1, 1), datetime(2026, 12, 31)) + assert df.height == 1 + row = df.row(0, named=True) + assert row["trade_date"] == date(2026, 6, 12) + assert row["ex_factor"] == pytest.approx(32.8 / 32.12) + + +def test_adj_bonus_and_allotment_formula(monkeypatch): + """送转+配股混合: P=5.23, AR=0.4, AP=3.36 → ref=(5.23+1.344)/1.4=4.6957→4.70(half-up)。""" + events = [("300176.SZ", date(2026, 8, 21), 0.0, 0.0, 0.4, 3.36)] + bars = {"300176.SZ": [_bar(date(2026, 8, 20), 5.23), _bar(date(2026, 8, 21), 4.68)]} + provider = _adj_provider(monkeypatch, events, bars) + df = provider.get_adj_factors(["300176.SZ"], datetime(2026, 1, 1), datetime(2026, 12, 31)) + assert df.height == 1 + assert df["ex_factor"][0] == pytest.approx(5.23 / 4.70) + + +def test_adj_half_up_rounding_not_bankers(monkeypatch): + """舍入口径: x=2.625 → half-up 2.63 (银行家舍入会给 2.62, 对拍实证偏离)。""" + events = [("600000.SH", date(2026, 6, 12), 8.0, 0.0, 0.0, 0.0)] + bars = {"600000.SH": [_bar(date(2026, 6, 11), 10.625), _bar(date(2026, 6, 12), 2.70)]} + provider = _adj_provider(monkeypatch, events, bars) + df = provider.get_adj_factors(["600000.SH"], datetime(2026, 1, 1), datetime(2026, 12, 31)) + assert df["ex_factor"][0] == pytest.approx(10.625 / 2.63) + + +def test_adj_price_limit_selfcheck_drops_impossible_factor(monkeypatch): + """涨跌停自检: 复权后除权日涨幅超板限的事件剔除, 不让坏因子落库。""" + events = [("600000.SH", date(2026, 6, 12), 8.0, 0.0, 0.0, 0.0)] + bars = {"600000.SH": [_bar(date(2026, 6, 11), 10.625), _bar(date(2026, 6, 12), 4.05)]} # +54% + provider = _adj_provider(monkeypatch, events, bars) + df = provider.get_adj_factors(["600000.SH"], datetime(2026, 1, 1), datetime(2026, 12, 31)) + assert df.is_empty() + + +def test_adj_merges_same_day_components_before_derivation(monkeypatch): + """同日拆行(分红行+送股行)必须先合并再推导。""" + events = [ + ("000812.SZ", date(2026, 6, 12), 0.3, 0.0, 0.0, 0.0), + ("000812.SZ", date(2026, 6, 12), 0.0, 0.1, 0.0, 0.0), + ] + bars = {"000812.SZ": [_bar(date(2026, 6, 11), 10.0), _bar(date(2026, 6, 12), 9.0)]} + provider = _adj_provider(monkeypatch, events, bars) + df = provider.get_adj_factors(["000812.SZ"], datetime(2026, 1, 1), datetime(2026, 12, 31)) + assert df.height == 1 # 合并成一个事件 + assert df["ex_factor"][0] == pytest.approx(10.0 / 8.82) # ref=(10-0.3)/1.1=8.8181→8.82 + + +def test_adj_filters_zero_rows_and_missing_allot_price(monkeypatch): + """全零行与配股价缺失行过滤, 不参与推导。""" + events = [ + ("000001.SZ", date(2026, 6, 12), 0.0, 0.0, 0.0, 0.0), # 全零 + ("000002.SZ", date(2026, 6, 12), 0.0, 0.0, 0.3, 0.0), # 配股无价 + ("600519.SH", date(2026, 6, 12), 0.68, 0.0, 0.0, 0.0), # 正常 + ] + bars = { + "600519.SH": [_bar(date(2026, 6, 11), 32.8), _bar(date(2026, 6, 12), 32.0)], + "000001.SZ": [_bar(date(2026, 6, 11), 10.0), _bar(date(2026, 6, 12), 10.0)], + "000002.SZ": [_bar(date(2026, 6, 11), 8.0), _bar(date(2026, 6, 12), 8.0)], + } + provider = _adj_provider(monkeypatch, events, bars) + df = provider.get_adj_factors(["000001.SZ", "000002.SZ", "600519.SH"], None, None) + assert df["symbol"].unique().to_list() == ["600519.SH"] + + +def test_adj_skips_future_announced_events(monkeypatch): + """未来已公告事件无前收盘, 跳过留给滚动增量窗口。""" + future = date.today() + timedelta(days=5) + events = [("600519.SH", future, 0.68, 0.0, 0.0, 0.0)] + provider = _adj_provider(monkeypatch, events, {"600519.SH": []}) + df = provider.get_adj_factors(["600519.SH"], datetime(2026, 1, 1), None) + assert df.is_empty() + + +def test_adj_window_filter(monkeypatch): + events = [ + ("600519.SH", date(2026, 1, 10), 0.5, 0.0, 0.0, 0.0), + ("600519.SH", date(2026, 6, 12), 0.68, 0.0, 0.0, 0.0), + ] + bars = { + "600519.SH": [ + _bar(date(2026, 1, 9), 30.0), + _bar(date(2026, 1, 10), 30.0), + _bar(date(2026, 6, 11), 32.8), + _bar(date(2026, 6, 12), 32.0), + ] + } + provider = _adj_provider(monkeypatch, events, bars) + df = provider.get_adj_factors(["600519.SH"], datetime(2026, 3, 1), datetime(2026, 12, 31)) + assert df["trade_date"].to_list() == [date(2026, 6, 12)] + + +def test_adj_output_schema_sorted_and_deduped(monkeypatch): + events = [ + ("000001.SZ", date(2026, 6, 12), 0.36, 0.0, 0.0, 0.0), + ("600519.SH", date(2026, 6, 12), 0.68, 0.0, 0.0, 0.0), + ("600519.SH", date(2025, 6, 20), 0.5, 0.0, 0.0, 0.0), + ] + bars = { + "600519.SH": [ + _bar(date(2025, 6, 19), 31.0), + _bar(date(2025, 6, 20), 31.0), + _bar(date(2026, 6, 11), 32.8), + _bar(date(2026, 6, 12), 32.0), + ], + "000001.SZ": [_bar(date(2026, 6, 11), 11.7), _bar(date(2026, 6, 12), 11.5)], + } + provider = _adj_provider(monkeypatch, events, bars) + df = provider.get_adj_factors(["000001.SZ", "600519.SH"], None, None) + assert df.columns == ["symbol", "trade_date", "ex_factor"] + assert df.schema["trade_date"] == pl.Date and df.schema["ex_factor"] == pl.Float64 + assert df.height == 3 + assert df.sort(["symbol", "trade_date"]).equals(df) # 已排序 + assert df["ex_factor"].min() > 1.0 # 分红因子必 >1 + + +def test_adj_etf_and_empty_symbols_return_empty(monkeypatch): + provider = _adj_provider(monkeypatch, [], {}) + empty = provider.get_adj_factors([], None, None) + assert empty.is_empty() and empty.columns == ["symbol", "trade_date", "ex_factor"] + etf = provider.get_adj_factors(["510300.SH"], None, None, asset_type="etf") + assert etf.is_empty() + + +def test_adj_dump_unavailable_returns_empty(monkeypatch): + """dump 加载失败软返回空(不阻断管道), 不抛异常。""" + provider = _hist_provider(monkeypatch, _FakeHistClient({})) + + def _boom(dump_kind, cache_prefix): + raise fc.FuyaoError("dump 下载网络失败") + + provider._ensure_dump = _boom # type: ignore[assignment] + df = provider.get_adj_factors(["600519.SH"], None, None) + assert df.is_empty() and df.columns == ["symbol", "trade_date", "ex_factor"] + + +def test_adj_progress_callback_per_symbol(monkeypatch): + events = [ + ("000001.SZ", date(2026, 6, 12), 0.36, 0.0, 0.0, 0.0), + ("600519.SH", date(2026, 6, 12), 0.68, 0.0, 0.0, 0.0), + ] + bars = { + "600519.SH": [_bar(date(2026, 6, 11), 32.8), _bar(date(2026, 6, 12), 32.0)], + "000001.SZ": [_bar(date(2026, 6, 11), 11.7), _bar(date(2026, 6, 12), 11.5)], + } + provider = _adj_provider(monkeypatch, events, bars) + progress = [] + provider.get_adj_factors( + ["600519.SH", "000001.SZ"], None, None, on_chunk_done=lambda c, t: progress.append((c, t)) + ) + assert progress == [(1, 2), (2, 2)] + + +# ---- adj_factor: 本地 dump 配价(2026-08 优化) ---- + + +def test_adj_closes_from_local_dump_no_http(monkeypatch, tmp_path): + """前收盘从本地日K dump 一次取齐: 零单标的请求, 因子值与接口路径同公式。""" + events = [("000001.SZ", date(2026, 6, 12), 0.3, 0.0, 0.0, 0.0)] + big_rows = [ + _dump_bar("000001.SZ", date(2026, 5, 10), 10.2), # 覆盖窗口起点(first_ex-30d) + _dump_bar("000001.SZ", date(2026, 6, 11), 10.625), + _dump_bar("000001.SZ", date(2026, 6, 12), 9.6), + ] + provider = _bigdump_provider(monkeypatch, tmp_path, big_rows) + provider._dump_memo[fp._ADJ_DUMP_KIND] = _adj_dump(events) + progress = [] + df = provider.get_adj_factors( + ["000001.SZ"], + datetime(2026, 1, 1), + datetime(2026, 12, 31), + on_chunk_done=lambda c, t: progress.append((c, t)), + ) + assert provider._get_client().calls == [] # 零 HTTP + assert progress == [(1, 1)] + assert df.height == 1 + assert df["trade_date"][0] == date(2026, 6, 12) + assert df["ex_factor"][0] == pytest.approx(10.625 / fp._ref_price(10.625, 0.3, 0.0, 0.0, 0.0)) + + +def test_adj_missing_symbol_in_dump_falls_back_to_http(monkeypatch, tmp_path): + """dump 缺价的标的回退单标的接口; dump 内标的仍零请求。""" + events = [ + ("000001.SZ", date(2026, 6, 12), 0.3, 0.0, 0.0, 0.0), + ("600519.SH", date(2026, 6, 12), 1.0, 0.0, 0.0, 0.0), + ] + big_rows = [ + _dump_bar("000001.SZ", date(2026, 5, 10), 10.2), + _dump_bar("000001.SZ", date(2026, 6, 11), 10.625), + _dump_bar("000001.SZ", date(2026, 6, 12), 9.6), + ] + provider = _bigdump_provider(monkeypatch, tmp_path, big_rows) + provider._dump_memo[fp._ADJ_DUMP_KIND] = _adj_dump(events) + provider._get_client().bars["600519.SH"] = [ + _bar(date(2026, 6, 11), 1500.0), + _bar(date(2026, 6, 12), 1400.0), + ] + df = provider.get_adj_factors( + ["000001.SZ", "600519.SH"], datetime(2026, 1, 1), datetime(2026, 12, 31) + ) + assert [c["thscode"] for c in provider._get_client().calls] == ["600519.SH"] + assert set(df["symbol"].to_list()) == {"000001.SZ", "600519.SH"} + + +def test_adj_ten_day_dump_overlays_fresh_ex_close(monkeypatch, tmp_path): + """大 dump 缺除权日收盘(发布滞后)时由 10d dump 叠加补, 涨跌停自检仍生效。""" + events = [("000001.SZ", date(2026, 8, 28), 8.0, 0.0, 0.0, 0.0)] # 异常大额分红 + big_rows = [ + _dump_bar("000001.SZ", date(2026, 7, 25), 10.0), # 覆盖窗口起点 + _dump_bar("000001.SZ", date(2026, 8, 27), 10.625), # 大 dump 只到除权前一日 + ] + ten_rows = [_dump_bar("000001.SZ", date(2026, 8, 28), 4.05)] # +54% 超涨跌停 + provider = _bigdump_provider(monkeypatch, tmp_path, big_rows, ten_memo=_daily10_dump(ten_rows)) + provider._dump_memo[fp._ADJ_DUMP_KIND] = _adj_dump(events) + df = provider.get_adj_factors(["000001.SZ"], datetime(2026, 1, 1), datetime(2026, 12, 31)) + assert df.is_empty() # 自检剔除(无叠加时 ex 日收盘缺失, 该行会被保留) + + +def test_adj_local_pricing_corrupt_dump_falls_back(monkeypatch, tmp_path): + """本地配价读盘异常不致命: 整轮回退单标的接口。""" + events = [("000001.SZ", date(2026, 6, 12), 0.3, 0.0, 0.0, 0.0)] + big_rows = [ + _dump_bar("000001.SZ", date(2026, 5, 10), 10.2), + _dump_bar("000001.SZ", date(2026, 6, 11), 10.625), + _dump_bar("000001.SZ", date(2026, 6, 12), 9.6), + ] + provider = _bigdump_provider(monkeypatch, tmp_path, big_rows) + provider._dump_memo[fp._ADJ_DUMP_KIND] = _adj_dump(events) + + def _boom(_lf): # 签名对齐被替换的 _closes_scan + raise OSError("disk error") + + provider._closes_scan = _boom # type: ignore[assignment] + provider._get_client().bars["000001.SZ"] = [ + _bar(date(2026, 6, 11), 10.625), + _bar(date(2026, 6, 12), 9.6), + ] + df = provider.get_adj_factors(["000001.SZ"], datetime(2026, 1, 1), datetime(2026, 12, 31)) + assert df.height == 1 + assert provider._get_client().calls # 回退了接口 + + +# ---- 客户端: 历史日K / dump ---- + + +class _RecordingHttp: + def __init__(self, payload): + self.payload = payload + self.calls: list[tuple] = [] + + def get(self, path, params=None): + self.calls.append((path, params)) + resp = type("R", (), {"json": lambda self_: self.payload, "status_code": 200})() + return resp + + def close(self): + pass + + +def test_client_historical_kline_sends_params(monkeypatch): + http = _RecordingHttp({"code": 0, "data": {"item": [_bar(date(2026, 8, 28), 11.65)]}}) + monkeypatch.setattr(fc.httpx, "Client", lambda **kw: http) + c = fc.FuyaoClient(api_key="k") + rows = c.historical_kline("000001.SZ", 1, 2, adjust="none") + assert len(rows) == 1 + path, params = http.calls[0] + assert path == "/api/a-share/prices/historical" + assert params == { + "thscode": "000001.SZ", + "interval": "1d", + "adjust": "none", + "start": 1, + "end": 2, + } + + +def test_client_historical_kline_missing_item(monkeypatch): + http = _RecordingHttp({"code": 0, "data": {}}) + monkeypatch.setattr(fc.httpx, "Client", lambda **kw: http) + c = fc.FuyaoClient(api_key="k") + assert c.historical_kline("000001.SZ", 1, 2) == [] + + +def test_client_dump_download_url(monkeypatch): + info = { + "presigned_url": "https://o.thsi.cn/.../releases/20260828/x.parquet?sig=1", + "presigned_url_expires_at": "2026-08-29T12:00:00+08:00", + "expires_in_seconds": 300, + } + http = _RecordingHttp({"code": 0, "data": info}) + monkeypatch.setattr(fc.httpx, "Client", lambda **kw: http) + c = fc.FuyaoClient(api_key="k") + assert c.dump_download_url("adjustment-factors") == info + assert http.calls[0][0] == "/api/dump/market-dumps/adjustment-factors/download-url" + + +def test_client_download_dump_atomic_and_key_never_leaked(monkeypatch, tmp_path): + """/S3 预签名下载不带 X-api-key; 成功后原子改名, .part 清理。""" + http = _RecordingHttp( + { + "code": 0, + "data": { + "presigned_url": "https://o.thsi.cn/fuyao/market-dump/adj/releases/20260828/a.parquet?sig=1" + }, + } + ) + monkeypatch.setattr(fc.httpx, "Client", lambda **kw: http) + + captured: dict = {} + + class _Resp: + status_code = 200 + + def iter_bytes(self, n): + yield b"parquet-bytes-" + + class _Ctx: + def __enter__(self): + return _Resp() + + def __exit__(self, *a): + return False + + def _stream(method, url, **kw): + captured["method"], captured["url"], captured["kw"] = method, url, kw + return _Ctx() + + monkeypatch.setattr(fc.httpx, "stream", _stream) + dest = tmp_path / "adj__20260828.parquet" + fc.FuyaoClient(api_key="k").download_dump("adjustment-factors", dest) + assert captured["method"] == "GET" and "o.thsi.cn" in captured["url"] + assert "headers" not in captured["kw"] or "X-api-key" not in ( + captured["kw"].get("headers") or {} + ) + assert dest.read_bytes() == b"parquet-bytes-" + assert not list(tmp_path.glob("*.part")) + + +def test_client_download_dump_http_error(monkeypatch, tmp_path): + http = _RecordingHttp({"code": 0, "data": {"presigned_url": "https://o.thsi.cn/x.parquet"}}) + monkeypatch.setattr(fc.httpx, "Client", lambda **kw: http) + + class _Resp: + status_code = 403 + + class _Ctx: + def __enter__(self): + return _Resp() + + def __exit__(self, *a): + return False + + monkeypatch.setattr(fc.httpx, "stream", lambda m, u, **kw: _Ctx()) + dest = tmp_path / "x.parquet" + with pytest.raises(fc.FuyaoError, match="403"): + fc.FuyaoClient(api_key="k").download_dump("adjustment-factors", dest) + assert not dest.exists() and not list(tmp_path.glob("*.part")) + + +# ---- 时区与 release 解析 ---- + + +def test_shanghai_midnight_ms_roundtrip(): + """上海零点戳 ↔ 日期: +8h 换算不得偏移一天。""" + d = date(2026, 6, 12) + assert fp._date_of_ms(_sh_ms(d)) == d + assert fp._ms_of_date(d) == _sh_ms(d) + + +def test_release_of_extracts_from_presigned_url(): + url = "https://o.thsi.cn/x/market-dump/adj/releases/20260828/a.parquet?sig=1" + assert fp._release_of(url) == "20260828" + assert fp._release_of("https://no-release-here/x") == "unknown" + + +# ---- 设置页试拉: daily / adj_factor ---- + + +def test_test_dataset_daily_preview(monkeypatch): + bars = {"000001.SZ": [_bar(date(2026, 8, 27), 11.05), _bar(date(2026, 8, 28), 11.65)]} + provider = _hist_provider(monkeypatch, _FakeHistClient(bars)) + out = provider.test_dataset("daily", ["000001.SZ"]) + assert out["provider"] == "fuyao" and out["dataset"] == "daily" + assert out["rows"] == 2 + assert out["preview"][0]["date"] == "2026-08-27" # date → ISO 字符串 + + +def test_test_dataset_adj_factor_preview(monkeypatch): + events = [("600519.SH", date(2026, 6, 12), 0.68, 0.0, 0.0, 0.0)] + bars = {"600519.SH": [_bar(date(2026, 6, 11), 32.8), _bar(date(2026, 6, 12), 32.0)]} + provider = _adj_provider(monkeypatch, events, bars) + out = provider.test_dataset("adj_factor", ["600519.SH"]) + assert out["rows"] == 1 + assert out["preview"][0]["trade_date"] == "2026-06-12" + assert out["preview"][0]["ex_factor"] == pytest.approx(32.8 / 32.12) diff --git a/backend/tests/test_intraday_burst_fault_isolation.py b/backend/tests/test_intraday_burst_fault_isolation.py new file mode 100644 index 0000000..a4e9aac --- /dev/null +++ b/backend/tests/test_intraday_burst_fault_isolation.py @@ -0,0 +1,90 @@ +"""fetch_intraday_full_market_burst 单块容错契约。 + +一个块失败不得拖垮整轮: 成功块必须照常返回供落盘; 失败块单独重试一次; +失败块过多 (系统性故障) 时跳过重试。全部用假 client, 不发真实网络请求。 +""" +from types import SimpleNamespace +from unittest.mock import patch + +import polars as pl + +from app.services import kline_sync +from app.tickflow.capabilities import Cap, CapabilityLimits, CapabilitySet + + +def _capset(batch: int = 2) -> CapabilitySet: + return CapabilitySet({Cap.INTRADAY_BATCH: CapabilityLimits(rpm=60, batch=batch)}) + + +def _frame() -> pl.DataFrame: + # _normalize_minute 的最小输入: 毫秒 timestamp → 北京墙钟 datetime。 + # 时间必须落在交易时段 (时区契约守卫会拒绝非交易小时的脏数据) + from datetime import datetime + from zoneinfo import ZoneInfo + ts = int(datetime(2026, 8, 28, 9, 31, tzinfo=ZoneInfo("Asia/Shanghai")).timestamp() * 1000) + return pl.DataFrame({ + "timestamp": [ts], + "open": [1.0], "high": [1.0], "low": [1.0], "close": [1.0], + "volume": [100.0], "amount": [100.0], + }) + + +class _FakeKlines: + """intraday_batch 假实现: fail_once 首次失败重试成功, fail_always 恒失败。""" + + def __init__(self, fail_once: set[str], fail_always: set[str]) -> None: + self.fail_once = set(fail_once) + self.fail_always = set(fail_always) + self.calls: list[list[str]] = [] + + def intraday_batch(self, chunk, **kwargs): + self.calls.append(list(chunk)) + syms = set(chunk) + if syms & self.fail_always: + raise RuntimeError("permanent failure") + if syms & self.fail_once: + self.fail_once -= syms + raise RuntimeError("transient failure") + return {s: _frame() for s in chunk} + + +def _run(symbols: list[str], fake: _FakeKlines, batch: int = 2): + client = SimpleNamespace(klines=fake) + with patch.object(kline_sync, "get_client", return_value=client): + return kline_sync.fetch_intraday_full_market_burst(symbols, _capset(batch)) + + +def test_transient_chunk_failure_retried_and_all_symbols_returned(): + symbols = [f"S{i}" for i in range(6)] # 3 chunks (batch=2) + fake = _FakeKlines(fail_once={"S0", "S1"}, fail_always=set()) + df, requests = _run(symbols, fake) + # 失败块 (S0,S1) 重试后成功 → 6 只全在, 请求数 = 3 块 + 1 次重试 + assert set(df["symbol"].to_list()) == set(symbols) + assert requests == 4 + + +def test_permanent_chunk_failure_skipped_without_losing_other_chunks(): + symbols = [f"S{i}" for i in range(6)] + fake = _FakeKlines(fail_once=set(), fail_always={"S4", "S5"}) + df, requests = _run(symbols, fake) + # 失败块重试仍失败 → 只跳过该块, 其余 4 只必须返回 (旧实现会整轮丢弃) + assert set(df["symbol"].to_list()) == {"S0", "S1", "S2", "S3"} + assert requests == 4 + + +def test_all_chunks_succeed_requests_equals_chunk_count(): + symbols = [f"S{i}" for i in range(6)] + fake = _FakeKlines(fail_once=set(), fail_always=set()) + df, requests = _run(symbols, fake) + assert set(df["symbol"].to_list()) == set(symbols) + assert requests == 3 + + +def test_systemic_failure_skips_retry_to_avoid_pressuring_overloaded_server(): + symbols = [f"S{i}" for i in range(12)] # 6 chunks (batch=2) + # 5 个块恒失败 (>4) → 系统性故障, 不再重试 + fake = _FakeKlines(fail_once=set(), fail_always={s for s in symbols if s not in ("S0", "S1")}) + df, requests = _run(symbols, fake) + assert set(df["symbol"].to_list()) == {"S0", "S1"} + assert requests == 6 # 无重试 + assert len(fake.calls) == 6 diff --git a/backend/tests/test_job_record_persistence.py b/backend/tests/test_job_record_persistence.py new file mode 100644 index 0000000..9adfdd7 --- /dev/null +++ b/backend/tests/test_job_record_persistence.py @@ -0,0 +1,125 @@ +"""回归测试: job 记录跨进程死亡持久化(「数据在、记录丢」补丁)。 + +背景(用户反馈): 全市场同步 12:11~12:42 成功结束后 0.7s, uvicorn --reload +检测到代码变更杀死 worker, 恰好落在管道完成与 job_store.succeed() 落盘之间 +—— 数据已写盘但同步历史无任何记录。旧实现 pending/running 仅存内存、终态才 +落盘, 存在整段丢失窗口。 + +修复后契约: + - create()/start() 即落盘 pending/running 快照; + - 下次进程启动(= 新 JobStore 实例, 同目录)把遗留的 pending/running + 孤儿记录补标为 failed(中断), finished_at 取文件 mtime; + - 终态记录不受补录影响; 终态写入覆盖 running 快照(同一文件)。 +均为纯逻辑, 不触网。 +""" +from __future__ import annotations + +import json + +from app.services.pipeline_jobs import JobStore + + +def _read_disk(d, jid: str) -> dict: + return json.loads((d / f"{jid}.json").read_text("utf-8")) + + +# ── 创建/启动即落盘 ────────────────────────────────────────────────────── + +def test_create_writes_pending_snapshot_to_disk(tmp_path): + d = tmp_path / "jobs" + store = JobStore(store_dir=d) + jid, _ = store.create(timeout_s=60) + + disk = _read_disk(d, jid) + assert disk["status"] == "pending" + assert disk["stage"] == "init" + + +def test_start_updates_disk_snapshot_to_running(tmp_path): + d = tmp_path / "jobs" + store = JobStore(store_dir=d) + jid, _ = store.create(timeout_s=60) + store.start(jid) + + disk = _read_disk(d, jid) + assert disk["status"] == "running" + assert disk["started_at"] is not None + + +# ── 进程死亡 → 下次启动补录 ────────────────────────────────────────────── + +def test_orphan_running_record_is_reaped_on_next_boot(tmp_path): + """核心场景: 进程死在 running(甚至工作已做完但未终态), 记录必须可见。""" + d = tmp_path / "jobs" + dead = JobStore(store_dir=d) + jid, _ = dead.create(timeout_s=60) + dead.start(jid) + dead.progress(jid, "sync", 50, "halfway") # 进度只更新内存 + + # 新进程 = 同目录新实例(内存为空, 只有磁盘) + revived = JobStore(store_dir=d) + j = revived.get(jid) + assert j is not None + assert j["status"] == "failed" + assert "中断" in j["error"] + assert j["finished_at"] is not None + # finished_at 基于文件 mtime(≈ start 时刻), 时长不得虚增为负或巨大 + assert j["duration_s"] is not None + assert 0 <= j["duration_s"] <= 60 + # 同步历史列表可见 + assert any(x["id"] == jid for x in revived.list_recent()) + + +def test_orphan_pending_record_is_reaped(tmp_path): + """进程死在 create() 与 start() 之间: 记录同样可见, 时长为 None。""" + d = tmp_path / "jobs" + dead = JobStore(store_dir=d) + jid, _ = dead.create(timeout_s=60) + # 未 start 即死亡 + + revived = JobStore(store_dir=d) + j = revived.get(jid) + assert j["status"] == "failed" + assert j["duration_s"] is None + + +def test_reap_does_not_touch_terminal_records(tmp_path): + d = tmp_path / "jobs" + store = JobStore(store_dir=d) + jid, _ = store.create(timeout_s=60) + store.start(jid) + store.succeed(jid, {"daily_rows": 100}) + + revived = JobStore(store_dir=d) + j = revived.get(jid) + assert j["status"] == "succeeded" + assert j["result"] == {"daily_rows": 100} + + +def test_reap_allows_new_job_after_dead_orphan(tmp_path): + """补录后旧 job 已 failed: 新进程 create() 不被死孤儿阻塞(单飞只看内存)。""" + d = tmp_path / "jobs" + dead = JobStore(store_dir=d) + old_jid, _ = dead.create(timeout_s=60) + dead.start(old_jid) + + revived = JobStore(store_dir=d) + new_jid, is_new = revived.create(timeout_s=60) + assert is_new is True + assert new_jid != old_jid + + +# ── 终态覆盖快照 ───────────────────────────────────────────────────────── + +def test_terminal_write_replaces_running_snapshot(tmp_path): + d = tmp_path / "jobs" + store = JobStore(store_dir=d) + jid, _ = store.create(timeout_s=60) + store.start(jid) + store.fail(jid, "boom") + + files = list(d.glob("*.json")) + assert len(files) == 1 + disk = _read_disk(d, jid) + assert disk["status"] == "failed" + assert disk["error"] == "boom" diff --git a/backend/tests/test_kline_minute_live.py b/backend/tests/test_kline_minute_live.py new file mode 100644 index 0000000..7764024 --- /dev/null +++ b/backend/tests/test_kline_minute_live.py @@ -0,0 +1,137 @@ +"""个股详情分时轮询的 live 直拉路径测试。 + +背景: 盘中分钟增量落盘后, 当日本地分区很快达到 90% 完整度, +/api/kline/minute 的本地优先启发式会拦截实时补拉, 详情分时图停在 +上一增量轮 (≥60s 滞后)。live=1 让详情轮询在连续竞价时段绕过本地优先。 +""" +from __future__ import annotations + +from datetime import date, datetime + +import polars as pl +import pytest +from fastapi import FastAPI +from fastapi.testclient import TestClient + +from app.api.kline import router +from app.market_time import CN_TZ, in_continuous_session + +# 2026-08-26 是周三; 10:00 处于上午连续竞价, expected(已交易分钟) = 30 +_NOW = datetime(2026, 8, 26, 10, 0, tzinfo=CN_TZ) +_TODAY = date(2026, 8, 26) +_LOCAL_ROWS = 30 + + +class _FakeRepo: + def resolve_asset_type(self, symbol: str) -> str: + return "stock" + + def get_instruments(self) -> pl.DataFrame: + return pl.DataFrame( + {"symbol": [], "name": [], "total_shares": [], "float_shares": []} + ) + + def get_daily_asset(self, asset_type, symbol, start, end, columns=None): + return pl.DataFrame({"date": [], "close": []}) + + def get_minute(self, symbol, trade_date, asset_type="stock") -> pl.DataFrame: + return pl.DataFrame({ + "datetime": [ + datetime(2026, 8, 26, 9, 30 + offset // 60, offset % 60) + for offset in range(_LOCAL_ROWS) + ], + "close": [10.0] * _LOCAL_ROWS, + }) + + +def _client() -> TestClient: + app = FastAPI() + app.include_router(router) + app.state.repo = _FakeRepo() + return TestClient(app) + + +def _patch_market(monkeypatch, *, in_session: bool) -> None: + import app.api.kline as kline_api + + monkeypatch.setattr(kline_api, "cn_now", lambda: _NOW) + monkeypatch.setattr(kline_api, "cn_today", lambda: _TODAY) + monkeypatch.setattr(kline_api, "in_continuous_session", lambda: in_session) + + +def _patch_live_fetch(monkeypatch) -> None: + import app.api.kline as kline_api + + def _fake_fetch(symbol, trade_date, asset_type="stock"): + return pl.DataFrame({ + "datetime": [datetime(2026, 8, 26, 9, 59)], + "close": [11.11], + }) + + monkeypatch.setattr( + kline_api.kline_sync, "fetch_minute_single", _fake_fetch + ) + + +def test_minute_live_param_bypasses_local_first_during_session(monkeypatch): + _patch_market(monkeypatch, in_session=True) + _patch_live_fetch(monkeypatch) + + resp = _client().get( + "/api/kline/minute", params={"symbol": "600000.SH", "live": 1} + ) + + assert resp.status_code == 200 + body = resp.json() + assert body["source"] == "live" + assert body["rows"][0]["close"] == 11.11 + + +def test_minute_without_live_keeps_local_first(monkeypatch): + _patch_market(monkeypatch, in_session=True) + _patch_live_fetch(monkeypatch) + + resp = _client().get( + "/api/kline/minute", params={"symbol": "600000.SH"} + ) + + assert resp.status_code == 200 + body = resp.json() + # 本地 30 根 >= expected(30)*0.9 → 完整, 走本地 + assert body["source"] == "local" + assert body["rows"][0]["close"] == 10.0 + + +def test_minute_live_param_falls_back_to_local_off_session(monkeypatch): + _patch_market(monkeypatch, in_session=False) + _patch_live_fetch(monkeypatch) + + resp = _client().get( + "/api/kline/minute", params={"symbol": "600000.SH", "live": 1} + ) + + assert resp.status_code == 200 + body = resp.json() + assert body["source"] == "local" + + +@pytest.mark.parametrize( + ("hour", "minute", "expected"), + [ + (9, 29, False), + (9, 30, True), + (11, 30, True), + (11, 31, False), + (12, 30, False), + (13, 0, True), + (15, 0, True), + (15, 1, False), + ], +) +def test_in_continuous_session_boundaries(hour: int, minute: int, expected: bool): + now = datetime(2026, 8, 26, hour, minute, tzinfo=CN_TZ) # 周三 + assert in_continuous_session(now) is expected + + +def test_in_continuous_session_rejects_weekend(): + assert in_continuous_session(datetime(2026, 8, 29, 10, 0, tzinfo=CN_TZ)) is False diff --git a/backend/tests/test_minute_history_days.py b/backend/tests/test_minute_history_days.py new file mode 100644 index 0000000..c114365 --- /dev/null +++ b/backend/tests/test_minute_history_days.py @@ -0,0 +1,62 @@ +"""分钟源历史深度能力 (minute_history_days) 契约测试。 + +provider 可选类属性 minute_history_days 声明 1 分钟历史深度(交易日): +- stock-sdk = 5 (免费分时接口仅保留最近 5 个交易日) +- 未声明 / 走 tickflow → None (深历史) +preferences GET 带出该字段, 前端分时档位据此收窄 (浅源默认 5日, 深源默认 20日)。 +""" +from __future__ import annotations + +from types import SimpleNamespace + +from app.api import settings +from app.services import preferences + + +def _mock_resolver(monkeypatch, provider, fallback, err=None): + monkeypatch.setattr( + "app.services.kline_sync._resolve_minute_provider", + lambda name: (provider, fallback, err), + ) + + +def test_stocksdk_declares_five_day_history(): + from app.plugins.stocksdk.provider import StockSDKProvider + + assert StockSDKProvider.minute_history_days == 5 + + +def test_history_days_from_custom_provider(monkeypatch): + """自定义浅源 → 声明值; 前端据此只显示 1/5 日档。""" + _mock_resolver(monkeypatch, SimpleNamespace(minute_history_days=5), False) + monkeypatch.setattr(preferences, "get_minute_data_provider", lambda: "stocksdk") + assert settings._minute_history_days() == 5 + + +def test_history_days_none_for_undeclared_provider(monkeypatch): + """未声明的自定义源 → None (深历史基准)。""" + _mock_resolver(monkeypatch, SimpleNamespace(), False) + monkeypatch.setattr(preferences, "get_minute_data_provider", lambda: "my_source") + assert settings._minute_history_days() is None + + +def test_history_days_none_for_tickflow(monkeypatch): + """tickflow (回退路径) → None (深历史)。""" + _mock_resolver(monkeypatch, None, True) + monkeypatch.setattr(preferences, "get_minute_data_provider", lambda: "tickflow") + assert settings._minute_history_days() is None + + +def test_history_days_none_when_resolver_fails(monkeypatch): + """resolver 异常 (registry 损坏) → 降级 None, 不抛 500。""" + _mock_resolver(monkeypatch, None, True, err="registry broken") + monkeypatch.setattr(preferences, "get_minute_data_provider", lambda: "stocksdk") + assert settings._minute_history_days() is None + + +def test_preferences_get_includes_history_days(monkeypatch): + """GET /preferences 响应包含 minute_history_days 字段。""" + _mock_resolver(monkeypatch, SimpleNamespace(minute_history_days=5), False) + payload = settings.get_preferences() + assert payload["minute_history_days"] == 5 + assert "minute_data_provider" in payload diff --git a/backend/tests/test_minute_refresh.py b/backend/tests/test_minute_refresh.py new file mode 100644 index 0000000..bbe38da --- /dev/null +++ b/backend/tests/test_minute_refresh.py @@ -0,0 +1,305 @@ +"""盘中分钟增量刷新服务 (minute_refresh) 测试。 + +覆盖: +- 连续竞价时段判定 (含边界) +- 门控链: 开关关闭 / 自定义分钟源让位 / 能力缺失 / 时段外 / 放行 +- 单轮: mock 边界层脉冲 + 落盘, 校验状态字段与 universe 来源 +- 偏好读写: 默认关闭、间隔 clamp [60, 300] +- API: /minute-refresh/status 无服务时 available=false + +不发起真实网络请求: fetch_intraday_full_market_burst 与 _write_minute_partition +均 monkeypatch 替换。 +""" +from __future__ import annotations + +from datetime import datetime + +import polars as pl + +from app.services import minute_refresh, preferences +from app.services.minute_refresh import MinuteRefreshService, _in_continuous_session + + +def _isolated_prefs(tmp_path, monkeypatch): + path = tmp_path / "preferences.json" + monkeypatch.setattr(preferences, "_path", lambda: path) + preferences._invalidate_cache() + return path + + +class _FakeCapSet: + def __init__(self, has_intraday_batch: bool): + self._has = has_intraday_batch + + def has(self, cap) -> bool: + from app.tickflow.capabilities import Cap + + # 服务门控挂 INTRADAY_UNIVERSE, 修复轮用 INTRADAY_BATCH — 两者同档, 一起授/不授 + return self._has and cap in (Cap.INTRADAY_BATCH, Cap.INTRADAY_UNIVERSE) + + +class _FakeAppState: + def __init__(self, has_intraday_batch: bool): + self.capabilities = _FakeCapSet(has_intraday_batch) + + +class _FakeRepo: + def __init__(self, symbols: list[str]): + from pathlib import Path + self._inst = pl.DataFrame({"symbol": symbols}) + self.store = type("S", (), {"data_dir": Path(".")})() + + def get_instruments(self) -> pl.DataFrame: + return self._inst + + +# ── 时段判定 ──────────────────────────────────────────────────────── + + +def test_continuous_session_boundaries(): + wk = datetime(2026, 8, 25, 10, 0) # 周二 + assert _in_continuous_session(wk) + assert not _in_continuous_session(datetime(2026, 8, 25, 9, 29)) + assert not _in_continuous_session(datetime(2026, 8, 25, 11, 31)) # 午休 + assert _in_continuous_session(datetime(2026, 8, 25, 13, 0)) # 午后恢复 + assert _in_continuous_session(datetime(2026, 8, 25, 15, 0)) # 收盘瞬时 + assert not _in_continuous_session(datetime(2026, 8, 25, 15, 1)) + assert not _in_continuous_session(datetime(2026, 8, 22, 10, 0)) # 周六 + + +# ── 门控链 ────────────────────────────────────────────────────────── + + +def _svc(tmp_path, monkeypatch, *, enabled=True, custom_provider=False, capability=True, in_hours=True): + _isolated_prefs(tmp_path, monkeypatch) + preferences.save({"minute_refresh_enabled": enabled}) + if custom_provider: + # 模拟已注册的自定义分钟源 (真实注册表在测试环境未加载) + monkeypatch.setattr(preferences, "get_minute_data_provider", lambda: "a-stock-data") + svc = MinuteRefreshService(_FakeRepo(["600000.SH"])) + svc.set_app_state(_FakeAppState(capability)) + monkeypatch.setattr(minute_refresh, "_in_continuous_session", lambda now=None: in_hours) + # 交易日探针默认未知 (None → 放行): 隔离真实网络探测, holiday 分支在 + # test_trading_day.py 单独覆盖 + from app.services import trading_day + monkeypatch.setattr(trading_day, "is_trading_day", lambda now=None: None) + return svc + + +def test_gate_disabled(tmp_path, monkeypatch): + assert _svc(tmp_path, monkeypatch, enabled=False)._gate_reason() == "disabled" + + +def test_gate_custom_provider_yields(tmp_path, monkeypatch): + svc = _svc(tmp_path, monkeypatch, custom_provider=True) + assert svc._gate_reason() == "custom_minute_provider" + + +def test_gate_capability_missing(tmp_path, monkeypatch): + svc = _svc(tmp_path, monkeypatch, capability=False) + assert svc._gate_reason() == "capability" + + +def test_gate_outside_trading_hours(tmp_path, monkeypatch): + svc = _svc(tmp_path, monkeypatch, in_hours=False) + assert svc._gate_reason() == "outside_trading_hours" + + +def test_gate_pass(tmp_path, monkeypatch): + svc = _svc(tmp_path, monkeypatch) + assert svc._gate_reason() is None + assert svc.capability_ok() and not svc.custom_provider_active() + + +# ── 单轮 ──────────────────────────────────────────────────────────── + + +def test_run_round_writes_partition_and_updates_status(tmp_path, monkeypatch): + svc = _svc(tmp_path, monkeypatch) + minute_df = pl.DataFrame({ + "symbol": ["600000.SH"], + "datetime": [datetime(2026, 8, 25, 1, 30)], + "open": [10.0], "high": [10.5], "low": [9.9], "close": [10.2], + "volume": [1000.0], "amount": [10200.0], + }) + calls: dict = {} + + def fake_burst(symbols, capset, *, count=300): + calls["symbols"] = list(symbols) + return (minute_df, 1) + + def fake_write(df, minute_dir): + calls["dir"] = minute_dir + calls["rows"] = df.height + return df.height + + monkeypatch.setattr( + "app.services.kline_sync.fetch_intraday_full_market_burst", fake_burst + ) + monkeypatch.setattr("app.services.kline_sync._write_minute_partition", fake_write) + + svc._run_round() + + assert calls["symbols"] == ["600000.SH"] + assert calls["rows"] == 1 + st = svc.status() + assert st["rounds"] == 1 + assert st["last_rows"] == 1 + assert st["last_symbols"] == 1 + assert st["last_requests"] == 1 + assert st["last_round_at"] is not None + assert st["last_error"] is None + assert st["capability_ok"] is True + + +def test_run_round_records_error_when_burst_empty(tmp_path, monkeypatch): + svc = _svc(tmp_path, monkeypatch) + monkeypatch.setattr( + "app.services.kline_sync.fetch_intraday_full_market_burst", + lambda symbols, capset, *, count=300: (pl.DataFrame(), 3), + ) + svc._run_round() + st = svc.status() + assert st["rounds"] == 0 + assert "no data" in st["last_error"] + assert st["last_requests"] == 3 + + +# ── 两段式模式选择: 冷启动全天 → 稳态增量 ─────────────────────────── + + +def _patch_round(monkeypatch, *, lag, inc_df, burst_df): + calls: dict = {"modes": []} + + monkeypatch.setattr( + minute_refresh.MinuteRefreshService, "_today_coverage_lag_minutes", + lambda self: lag, raising=True, + ) + monkeypatch.setattr( + "app.services.kline_sync.fetch_intraday_universe_increment", + lambda *a, **k: (calls["modes"].append("increment"), (inc_df, 1))[1], + ) + monkeypatch.setattr( + "app.services.kline_sync.fetch_intraday_full_market_burst", + lambda symbols, capset, *, count=300: (calls["modes"].append("full"), (burst_df, 28))[1], + ) + monkeypatch.setattr( + "app.services.kline_sync._write_minute_partition", + lambda df, minute_dir: df.height, + ) + return calls + + +def _inc_df(): + return pl.DataFrame({ + "symbol": ["600000.SH", "000001.SZ"], + "datetime": [datetime(2026, 8, 25, 10, 0)] * 2, + "open": [10.0] * 2, "high": [10.5] * 2, "low": [9.9] * 2, "close": [10.2] * 2, + "volume": [1000.0] * 2, "amount": [10200.0] * 2, + }) + + +def _full_df(): + return _inc_df() + + +def test_cold_start_no_local_data_uses_full_mode(tmp_path, monkeypatch): + """当日无数据 (lag=None, 如 10 点冷启动) → 全天修复轮。""" + svc = _svc(tmp_path, monkeypatch) + calls = _patch_round(monkeypatch, lag=None, inc_df=_inc_df(), burst_df=_full_df()) + svc._run_round() + assert calls["modes"] == ["full"] + st = svc.status() + assert st["last_mode"] == "full" + assert st["last_rows"] == 2 + + +def test_healthy_coverage_uses_increment_mode(tmp_path, monkeypatch): + """当日覆盖新鲜 (lag ≤ 3 分钟) → universe 单请求增量, 不打 burst。""" + svc = _svc(tmp_path, monkeypatch) + calls = _patch_round(monkeypatch, lag=0.2, inc_df=_inc_df(), burst_df=_full_df()) + svc._run_round() + assert calls["modes"] == ["increment"] + st = svc.status() + assert st["last_mode"] == "increment" + assert st["last_requests"] == 1 + assert st["last_symbols"] == 2 + assert st["last_rows"] == 2 + + +def test_stale_coverage_beyond_bar_headroom_falls_back_to_full(tmp_path, monkeypatch): + """覆盖滞后超过 3 分钟 (超过 universe 3 根余量) → 全天修复轮。""" + svc = _svc(tmp_path, monkeypatch) + calls = _patch_round(monkeypatch, lag=5.0, inc_df=_inc_df(), burst_df=_full_df()) + svc._run_round() + assert calls["modes"] == ["full"] + + +def test_consecutive_empty_rounds_escalate_to_full(tmp_path, monkeypatch): + """universe 连续 2 轮空返回 → 第 3 轮自动升级全天修复 (自愈)。""" + svc = _svc(tmp_path, monkeypatch) + calls = _patch_round( + monkeypatch, + lag=0.2, + inc_df=pl.DataFrame(), # 增量恒空 (模拟 universe 端点持续异常) + burst_df=_full_df(), + ) + svc._run_round() + svc._run_round() + assert calls["modes"] == ["increment", "increment"] + assert svc.status()["rounds"] == 0 + svc._run_round() + assert calls["modes"] == ["increment", "increment", "full"] + assert svc.status()["last_mode"] == "full" + + +def test_status_reports_gate_reason_when_stopped(tmp_path, monkeypatch): + svc = _svc(tmp_path, monkeypatch, enabled=False) + st = svc.status() + assert st["enabled"] is False + assert st["running"] is False + assert st["gate_reason"] == "disabled" + assert st["interval_seconds"] == 6 + + +# ── 偏好 ──────────────────────────────────────────────────────────── + + +def test_refresh_preferences_defaults_and_clamp(tmp_path, monkeypatch): + _isolated_prefs(tmp_path, monkeypatch) + assert preferences.get_minute_refresh_enabled() is False + assert preferences.get_minute_refresh_interval() == 6 + preferences.save({"minute_refresh_interval": 1}) + assert preferences.get_minute_refresh_interval() == 3 # 下限 + preferences.save({"minute_refresh_interval": 999}) + assert preferences.get_minute_refresh_interval() == 300 # 上限 + preferences.save({"minute_refresh_interval": 15}) + assert preferences.get_minute_refresh_interval() == 15 + +def test_realtime_monitor_config_owns_refresh_keys(tmp_path, monkeypatch): + """盘中增量配置归属实时监控端点 (set_realtime_monitor_config), 并 clamp 到 [3,300]。""" + _isolated_prefs(tmp_path, monkeypatch) + saved = preferences.set_realtime_monitor_config({ + "minute_refresh_enabled": True, + "minute_refresh_interval": 1, # 越界 → clamp 到下限 + }) + assert saved["minute_refresh_enabled"] is True + assert saved["minute_refresh_interval"] == 3 + saved = preferences.set_realtime_monitor_config({"minute_refresh_interval": 400}) + assert saved["minute_refresh_interval"] == 300 + saved = preferences.set_realtime_monitor_config({"minute_refresh_interval": 6}) + assert saved["minute_refresh_interval"] == 6 + + +def test_status_endpoint_without_service(): + from fastapi import FastAPI + from fastapi.testclient import TestClient + + from app.api.settings import router + + app = FastAPI() + app.include_router(router) + client = TestClient(app) + resp = client.get("/api/settings/minute-refresh/status") + assert resp.status_code == 200 + assert resp.json() == {"available": False} diff --git a/backend/tests/test_minute_strategy.py b/backend/tests/test_minute_strategy.py new file mode 100644 index 0000000..44a5198 --- /dev/null +++ b/backend/tests/test_minute_strategy.py @@ -0,0 +1,526 @@ +"""分钟策略 (minute_filter 后端) 测试。 + +覆盖: +- minute_red_streak 形态: 命中 / 不足根数不触发 / 最高K不红 / rank_by 两口径 / + 乱序输入 / 最高价并列取更晚K线 / 开盘窗口(当日最早N根, 与最近N根区分) +- 引擎加载校验: 只能声明 filter_minute_history、timeframes 必须且只能是 ["1m"] +- 引擎 1m 运行: enriched 联表基础过滤 (剔除ST / 股价区间)、entry hits、 + 日线 context 拒绝 +- ScreenerService 1m context: 当日分区优先、缺失回退最近分区、空库报错、 + 非股票资产拒绝 +""" +from __future__ import annotations + +import datetime as _dt +import importlib.util +from datetime import date, datetime +from pathlib import Path + +import polars as pl + +from app.services.screener import ScreenerService +from app.strategy.engine import StrategyDataContext, StrategyEngine + +# 分钟红7 已从内置策略改为自定义策略 (运行时 data/strategies/custom/, 不入库); +# 测试通过仓库内的参考实现夹具加载, 覆盖同一份策略逻辑。 +STRATEGY_FIXTURE_DIR = Path(__file__).resolve().parent / "fixtures" / "strategies" +_spec = importlib.util.spec_from_file_location( + "minute_red_streak_fixture", STRATEGY_FIXTURE_DIR / "minute_red_streak.py" +) +minute_red_streak = importlib.util.module_from_spec(_spec) +_spec.loader.exec_module(minute_red_streak) + + +def _bars(symbol: str, candles: list[tuple[float, float, float]], start_hour: int = 9) -> pl.DataFrame: + """candles: (open, close, high) 序列, 时间从 start_hour:30 起每分钟一根。""" + n = len(candles) + base = datetime(2026, 8, 25, start_hour, 30) + return pl.DataFrame({ + "symbol": [symbol] * n, + "datetime": [base + _dt.timedelta(minutes=i) for i in range(n)], + "open": [float(c[0]) for c in candles], + "high": [float(c[2]) for c in candles], + "low": [float(min(c[0], c[1])) for c in candles], + "close": [float(c[1]) for c in candles], + "volume": [100.0] * n, + "amount": [10000.0] * n, + }) + + +# ── 形态 ──────────────────────────────────────────────────────────── + + +def test_pattern_hits_five_red_of_seven_with_red_top_two(): + # 7根: 5红2绿, 绿K的最高价都压得比红K低 → 最高的两根(10.9/10.7)都是红 + candles = [ + (10.0, 10.2, 10.30), # 红 + (10.2, 10.1, 10.25), # 绿 (低高点) + (10.1, 10.4, 10.50), # 红 + (10.4, 10.6, 10.70), # 红 (次高) + (10.6, 10.5, 10.65), # 绿 (低高点) + (10.5, 10.7, 10.80), # 红 + (10.7, 10.8, 10.90), # 红 (最高) + ] + out = minute_red_streak.filter_minute_history(_bars("600000.SH", candles), {"require_limit_up": False}) + assert out["symbol"].to_list() == ["600000.SH"] + row = out.row(0, named=True) + assert row["red_count"] == 5 + assert row["top_red_count"] == 2 + assert row["close"] == 10.8 + + +def test_pattern_insufficient_bars_never_triggers(): + out = minute_red_streak.filter_minute_history(_bars("600000.SH", [(10.0, 10.2, 10.3)] * 6), {"require_limit_up": False}) + assert out.is_empty() + + +def test_pattern_green_at_top_blocks_hit(): + # 5红, 但最高的一根是绿 (高开回落) → 最高两根不全红, 不触发 + candles = [ + (10.0, 10.2, 10.30), # 红 + (10.1, 10.4, 10.50), # 红 + (10.3, 10.6, 10.70), # 红 + (10.6, 10.5, 10.65), # 绿 (低高点) + (10.4, 10.5, 10.55), # 红 (低高点) + (11.5, 11.0, 12.00), # 绿 (最高) + (11.0, 11.4, 11.90), # 红 (次高) + ] + out = minute_red_streak.filter_minute_history(_bars("600000.SH", candles), {"require_limit_up": False}) + assert out.is_empty() + + +def test_pattern_rank_by_close_uses_close_not_high(): + # high 口径最高两根是绿K冲高; close 口径最高两根是红K → 仅 close 口径命中 + candles = [ + (10.0, 10.5, 10.60), # 红 + (10.5, 10.9, 11.50), # 绿 (high 最高, 并列) + (10.9, 11.2, 11.40), # 红 + (11.2, 11.3, 11.35), # 红 + (11.3, 11.4, 11.45), # 红 (close 次高) + (11.4, 11.1, 11.50), # 绿 (high 最高, 并列) + (11.1, 11.5, 11.55), # 红 (close 最高) + ] + by_high = minute_red_streak.filter_minute_history(_bars("600000.SH", candles), {"require_limit_up": False}) + by_close = minute_red_streak.filter_minute_history( + _bars("600000.SH", candles), {"rank_by_close": True, "require_limit_up": False} + ) + assert by_high.is_empty() + assert by_close["symbol"].to_list() == ["600000.SH"] + + +def test_pattern_sorts_unordered_input_by_datetime(): + bars = pl.concat([ + _bars("600000.SH", [(10.0, 10.2, 10.30)]), + _bars("600000.SH", [ + (10.2, 10.1, 10.25), (10.1, 10.4, 10.50), (10.4, 10.6, 10.70), + (10.6, 10.5, 10.65), (10.5, 10.7, 10.80), (10.7, 10.8, 10.90), + ]), + ]).sample(fraction=1.0, shuffle=True, seed=7) + out = minute_red_streak.filter_minute_history(bars, {"require_limit_up": False}) + assert out["symbol"].to_list() == ["600000.SH"] + assert out.row(0, named=True)["close"] == 10.8 # 最后一根(时间最大)的收盘 + + +def test_pattern_three_way_high_tie_prefers_later_bars(): + # 三根 high 并列最高: 更早的绿K应被更晚的两根红K挤出 top2 → 命中 + # (若并列取更早, top2 = {红, 绿} → 不命中; 该测试固定 "同值取更晚" 契约) + candles = [ + (10.0, 10.2, 10.30), # 红 + (10.1, 10.4, 10.50), # 红 + (10.2, 10.1, 10.25), # 绿 (低高点) + (10.3, 10.6, 10.70), # 红 + (10.8, 10.5, 10.90), # 绿 (并列最高, 最早 → 被 top2 排除) + (10.5, 10.6, 10.90), # 红 (并列最高, 中间) + (10.6, 10.8, 10.90), # 红 (并列最高, 最晚) + ] + out = minute_red_streak.filter_minute_history(_bars("600000.SH", candles), {"require_limit_up": False}) + assert out["symbol"].to_list() == ["600000.SH"] + assert out.row(0, named=True)["top_red_count"] == 2 + + +def test_pattern_min_red_threshold_respected(): + # 4红3绿, 最高的两根红 → min_red=5 不命中, min_red=4 命中 + candles = [ + (10.0, 10.2, 10.30), # 红 + (10.2, 10.1, 10.25), # 绿 + (10.1, 10.4, 10.50), # 红 + (10.4, 10.3, 10.45), # 绿 + (10.3, 10.6, 10.70), # 红 + (10.6, 10.5, 10.65), # 绿 + (10.5, 10.8, 10.90), # 红 + ] + bars = _bars("600000.SH", candles) + assert minute_red_streak.filter_minute_history(bars, {"min_red": 5, "require_limit_up": False}).is_empty() + assert not minute_red_streak.filter_minute_history(bars, {"min_red": 4, "require_limit_up": False}).is_empty() + + +def test_pattern_uses_opening_bars_even_if_day_turns_green(): + # 开盘7根 = 5红2绿命中; 第8/9根大绿回落 → 开盘窗口语义下仍命中, + # 且 close 取窗口末根 (10.8) 而非全天最新价 + candles = [ + (10.0, 10.2, 10.30), # 红 + (10.2, 10.1, 10.25), # 绿 (低高点) + (10.1, 10.4, 10.50), # 红 + (10.4, 10.6, 10.70), # 红 (次高) + (10.6, 10.5, 10.65), # 绿 (低高点) + (10.5, 10.7, 10.80), # 红 + (10.7, 10.8, 10.90), # 红 (最高) ← 窗口末根 + (10.8, 10.0, 10.85), # 开盘窗口外的绿 + (10.0, 9.5, 10.05), # 开盘窗口外的绿 + ] + out = minute_red_streak.filter_minute_history(_bars("600000.SH", candles), {"require_limit_up": False}) + assert out["symbol"].to_list() == ["600000.SH"] + row = out.row(0, named=True) + assert row["red_count"] == 5 + assert row["close"] == 10.8 # 窗口末根收盘, 不是第9根的 9.5 + assert row["last_datetime"] == datetime(2026, 8, 25, 9, 36) + + +def test_pattern_opening_window_miss_not_rescued_by_late_reds(): + # 开盘7根仅4红不命中; 第8/9根转红 (最近7根口径会命中) → 开盘窗口仍不触发 + candles = [ + (10.0, 10.2, 10.30), # 红 + (10.2, 10.1, 10.25), # 绿 + (10.1, 10.4, 10.50), # 红 + (10.4, 10.3, 10.45), # 绿 + (10.3, 10.6, 10.70), # 红 + (10.6, 10.5, 10.65), # 绿 + (10.5, 10.8, 10.90), # 红 + (10.8, 10.9, 11.00), # 红 (窗口外) + (10.9, 11.0, 11.10), # 红 (窗口外) + ] + out = minute_red_streak.filter_minute_history(_bars("600000.SH", candles), {"require_limit_up": False}) + assert out.is_empty() + + +# ── 涨停条件 (日线维度) ───────────────────────────────────────────── + + +_HIT_CANDLES = [ + (10.0, 10.2, 10.30), # 红 + (10.2, 10.1, 10.25), # 绿 (低高点) + (10.1, 10.4, 10.50), # 红 + (10.4, 10.6, 10.70), # 红 (次高) + (10.6, 10.5, 10.65), # 绿 (低高点) + (10.5, 10.7, 10.80), # 红 + (10.7, 10.8, 10.90), # 红 (最高) +] + + +def _daily( + symbol: str, + days: int, + flag_on: set[int] | None = None, + *, + broken: bool = False, +) -> pl.DataFrame: + """days 个交易日的日线帧; flag_on 指定第几天 (0=最早) 触发涨停信号。""" + flag_on = flag_on or set() + base = date(2026, 8, 25) + return pl.DataFrame({ + "symbol": [symbol] * days, + "date": [base - _dt.timedelta(days=days - i) for i in range(days)], + "signal_limit_up": [i in flag_on and not broken for i in range(days)], + "signal_broken_limit_up": [i in flag_on and broken for i in range(days)], + }) + + +def test_pattern_limit_up_condition_filters_by_daily_signals(): + bars = pl.concat([ + _bars("600001.SH", _HIT_CANDLES), + _bars("600002.SH", _HIT_CANDLES), + _bars("600003.SH", _HIT_CANDLES), + ]) + daily = pl.concat([ + _daily("600001.SH", 20, {3}), # 收盘涨停 → 过 + _daily("600002.SH", 20, {15}, broken=True), # 炸板触及 → 过 + _daily("600003.SH", 20), # 无涨停 → 剔除 + ]) + out = minute_red_streak.filter_minute_history(bars, {}, daily=daily) + assert sorted(out["symbol"].to_list()) == ["600001.SH", "600002.SH"] + assert sorted(out["recent_limit_ups"].to_list()) == [1, 1] + + +def test_pattern_limit_up_lookback_window_boundary(): + # 25 个交易日, 涨停仅发生在第 5 天 (0=最早): 回看 20 日窗口 = 最后 20 根 + # (索引 5..24), 第 5 天在窗外 → 不命中; 回看放宽到 25 → 命中 + bars = _bars("600000.SH", _HIT_CANDLES) + daily = _daily("600000.SH", 25, {4}) + assert minute_red_streak.filter_minute_history(bars, {}, daily=daily).is_empty() + out = minute_red_streak.filter_minute_history( + bars, {"limit_up_days": 25}, daily=daily + ) + assert out["symbol"].to_list() == ["600000.SH"] + + +def test_pattern_limit_up_fails_closed_without_daily(): + # 日线窗口缺失时失败闭合 (宁可漏过不可错报) + out = minute_red_streak.filter_minute_history(_bars("600000.SH", _HIT_CANDLES), {}) + assert out.is_empty() + + +def test_pattern_limit_up_disabled_ignores_daily(): + out = minute_red_streak.filter_minute_history( + _bars("600000.SH", _HIT_CANDLES), {"require_limit_up": False} + ) + assert out["symbol"].to_list() == ["600000.SH"] + assert "recent_limit_ups" not in out.columns + + +# ── 引擎加载与运行 ────────────────────────────────────────────────── + + +def test_custom_minute_strategy_loads_with_minute_filter_backend(): + # 自定义策略与内置策略共用同一加载器: 夹具目录即一个 custom 目录 + engine = StrategyEngine(strategy_dirs=[STRATEGY_FIXTURE_DIR]) + assert not [e for e in engine.load_errors() if "minute" in e["file"]] + s = engine.get("minute_red_streak") + assert s.execution_backend == "minute_filter" + assert s.filter_minute_history_fn is not None + assert s.meta["timeframes"] == ["1m"] + assert s.source == "custom" + + +def _minute_code(sid: str, timeframes: str = '["1m"]', extra: str = "") -> str: + return f'''import polars as pl +META = {{"id": "{sid}", "name": "{sid}", "asset_types": ["stock"], "timeframes": {timeframes}}} +EXECUTION_BACKEND = "minute_filter" +{extra} +def filter_minute_history(df, params): + return df.group_by("symbol").agg( + close=pl.col("close").max(), last_datetime=pl.col("datetime").max() + ) +''' + + +def test_minute_filter_backend_validation(tmp_path): + (tmp_path / "ok.py").write_text(_minute_code("m_ok")) + (tmp_path / "bad_filter.py").write_text( + _minute_code("m_bad1", extra="def filter(df, params):\n return pl.lit(True)") + ) + (tmp_path / "bad_tf.py").write_text(_minute_code("m_bad2", timeframes='["1d", "1m"]')) + engine = StrategyEngine(strategy_dirs=[tmp_path]) + ids = {m["id"] for m in engine.list_strategies(include_research=True)} + assert "m_ok" in ids + assert "m_bad1" not in ids + assert "m_bad2" not in ids + assert any("only filter_minute_history" in e["error"] for e in engine.load_errors()) + assert any("timeframes" in e["error"] for e in engine.load_errors()) + + +def test_minute_filter_daily_history_validation(tmp_path): + # 声明 daily_history_bars: fn 必须接受 daily 关键字, 且范围 [0, 250] + (tmp_path / "m_daily_ok.py").write_text( + 'import polars as pl\n' + 'META = {"id": "m_daily_ok", "name": "x", "asset_types": ["stock"], ' + '"timeframes": ["1m"], "daily_history_bars": 20}\n' + 'EXECUTION_BACKEND = "minute_filter"\n' + 'def filter_minute_history(df, params, *, daily=None):\n' + ' return df.group_by("symbol").agg(close=pl.col("close").max())\n' + ) + (tmp_path / "m_daily_kw.py").write_text( + 'import polars as pl\n' + 'META = {"id": "m_daily_kw", "name": "x", "asset_types": ["stock"], ' + '"timeframes": ["1m"], "daily_history_bars": 20}\n' + 'EXECUTION_BACKEND = "minute_filter"\n' + 'def filter_minute_history(df, params):\n' + ' return df.group_by("symbol").agg(close=pl.col("close").max())\n' + ) + (tmp_path / "m_daily_range.py").write_text( + 'import polars as pl\n' + 'META = {"id": "m_daily_range", "name": "x", "asset_types": ["stock"], ' + '"timeframes": ["1m"], "daily_history_bars": 300}\n' + 'EXECUTION_BACKEND = "minute_filter"\n' + 'def filter_minute_history(df, params, *, daily=None):\n' + ' return df.group_by("symbol").agg(close=pl.col("close").max())\n' + ) + engine = StrategyEngine(strategy_dirs=[tmp_path]) + assert engine.has("m_daily_ok") + assert engine.get("m_daily_ok").minute_daily_bars == 20 + assert not engine.has("m_daily_kw") + assert not engine.has("m_daily_range") + assert any("'daily' keyword" in e["error"] for e in engine.load_errors()) + assert any("[0, 250]" in e["error"] for e in engine.load_errors()) + + +def test_minute_run_injects_daily_history(tmp_path): + # fn 直接消费 daily (对涨停信号求和), 验证引擎把 context.daily_history 注入 + (tmp_path / "m_use_daily.py").write_text( + 'import polars as pl\n' + 'META = {"id": "m_use_daily", "name": "x", "asset_types": ["stock"], ' + '"timeframes": ["1m"], "daily_history_bars": 10}\n' + 'EXECUTION_BACKEND = "minute_filter"\n' + 'def filter_minute_history(df, params, *, daily=None):\n' + ' if daily is None:\n' + ' return pl.DataFrame(schema={"symbol": pl.Utf8})\n' + ' return daily.group_by("symbol").agg(\n' + ' close=pl.col("signal_limit_up").sum() + 10.0)\n' + ) + engine = StrategyEngine(strategy_dirs=[tmp_path]) + context = StrategyDataContext( + asset_type="stock", + timeframe="1m", + as_of=date(2026, 8, 25), + current=pl.DataFrame({ + "symbol": ["600001.SH"], + "name": ["正常股"], + "total_shares": [1e8], + "float_shares": [5e7], + "amount": [3e8], + "change_pct": [0.01], + }), + history=_bars("600001.SH", [(10.0, 10.2, 10.3)] * 7), + daily_history=_daily("600001.SH", 10, {2}), + ) + result = engine.run("m_use_daily", context) + assert result.total == 1 + assert result.rows[0]["close"] == 11 # 10 + 窗口内 1 次收盘涨停 + + +def test_minute_context_run_applies_enriched_basic_filter(tmp_path): + (tmp_path / "m_basic.py").write_text(_minute_code("m_basic")) + engine = StrategyEngine(strategy_dirs=[tmp_path]) + + hist = pl.concat([ + _bars("600001.SH", [(10.0, 20.0, 25.0)] * 7), # 命中, 收盘 20 + _bars("600002.SH", [(10.0, 20.0, 25.0)] * 7), # 命中但 ST → 剔除 + _bars("600003.SH", [(10.0, 20.0, 25.0)] * 7), # 命中 + _bars("600004.SH", [(100.0, 200.0, 250.0)] * 7), # 命中但收盘 200 → 超上限剔除 + ]) + current = pl.DataFrame({ + "symbol": ["600001.SH", "600002.SH", "600003.SH", "600004.SH"], + "name": ["正常股", "ST垃圾", "正常股2", "高价股"], + "total_shares": [1e8, 1e8, 1e8, 1e8], + "float_shares": [5e7, 5e7, 5e7, 5e7], + "amount": [3e8, 3e8, 3e8, 3e8], + "change_pct": [0.01, 0.01, 0.01, 0.01], + }) + context = StrategyDataContext( + asset_type="stock", + timeframe="1m", + as_of=date(2026, 8, 25), + current=current, + history=hist, + ) + result = engine.run( + "m_basic", context, overrides={"basic_filter": {"price_max": 150.0}} + ) + symbols = {r["symbol"] for r in result.rows} + assert symbols == {"600001.SH", "600003.SH"} + assert all("name" in r for r in result.rows) # enriched 列已联表 + assert {h["symbol"] for h in result.entry_signal_hits} == symbols + + +def test_minute_strategy_rejects_daily_context(tmp_path): + (tmp_path / "m_daily.py").write_text(_minute_code("m_daily")) + engine = StrategyEngine(strategy_dirs=[tmp_path]) + context = StrategyDataContext( + asset_type="stock", + timeframe="1d", + as_of=date(2026, 8, 25), + current=pl.DataFrame({"symbol": ["600001.SH"]}), + ) + try: + engine.run("m_daily", context) + raise AssertionError("expected ValueError") + except ValueError as e: + assert "timeframe" in str(e) + + +# ── ScreenerService 1m context ────────────────────────────────────── + + +class _FakeMinuteRepo: + def __init__(self, partitions: dict[date, pl.DataFrame]): + self.partitions = partitions + + def get_minute_by_dates(self, symbols, dates, asset_type="stock"): + frames = [self.partitions[d] for d in dates if d in self.partitions] + if not frames: + return pl.DataFrame() + return pl.concat(frames).filter(pl.col("symbol").is_in(symbols)) + + def latest_minute_date_global(self): + return max(self.partitions) if self.partitions else None + + +def _svc(partitions: dict[date, pl.DataFrame], asset_type: str = "stock") -> ScreenerService: + return ScreenerService(_FakeMinuteRepo(partitions), asset_type=asset_type) # type: ignore[arg-type] + + +def test_minute_context_prefers_as_of_partition(): + d1, d2 = date(2026, 8, 24), date(2026, 8, 25) + svc = _svc({ + d1: _bars("600001.SH", [(10.0, 10.2, 10.3)] * 3), + d2: _bars("600001.SH", [(10.0, 10.2, 10.3)] * 4), + }) + ctx = svc.build_strategy_context( + None, d1, [], timeframe="1m", + current=pl.DataFrame({"symbol": ["600001.SH"], "name": ["x"]}), + ) + assert ctx.history.height == 3 # as_of 当日分区, 不取更新的 d2 + assert ctx.timeframe == "1m" + + +def test_minute_context_falls_back_to_latest_partition(): + d1, d2 = date(2026, 8, 24), date(2026, 8, 25) + svc = _svc({ + d1: _bars("600001.SH", [(10.0, 10.2, 10.3)] * 3), + d2: _bars("600001.SH", [(10.0, 10.2, 10.3)] * 4), + }) + ctx = svc.build_strategy_context( + None, date(2026, 8, 20), [], timeframe="1m", + current=pl.DataFrame({"symbol": ["600001.SH"]}), + ) + assert ctx.history.height == 4 # 回退到最近分区 d2 + + +def test_minute_context_empty_store_raises_with_guidance(): + svc = _svc({}) + try: + svc.build_strategy_context( + None, date(2026, 8, 25), [], timeframe="1m", + current=pl.DataFrame({"symbol": ["600001.SH"]}), + ) + raise AssertionError("expected ValueError") + except ValueError as e: + assert "分钟K" in str(e) + + +def test_minute_context_rejects_non_stock_asset(): + svc = _svc({date(2026, 8, 25): _bars("510300.SH", [(10.0, 10.2, 10.3)] * 3)}, asset_type="etf") + try: + svc.build_strategy_context( + None, date(2026, 8, 25), [], timeframe="1m", + current=pl.DataFrame({"symbol": ["510300.SH"]}), + ) + raise AssertionError("expected ValueError") + except ValueError as e: + assert "A 股" in str(e) + + +def test_minute_context_loads_daily_history_for_declared_strategies(): + class _FakeEngine: + def minute_daily_history_bars(self, strategy_ids): + return 5 + + daily = _daily("600001.SH", 6, {1}) + repo = _FakeMinuteRepo({date(2026, 8, 25): _bars("600001.SH", [(10.0, 10.2, 10.3)] * 3)}) + repo.get_enriched_history = lambda target_date, lookback_days: daily # type: ignore[method-assign] + repo.get_instruments_asset = lambda asset_type: None # type: ignore[method-assign] + svc = ScreenerService(repo, asset_type="stock") # type: ignore[arg-type] + ctx = svc.build_strategy_context( + _FakeEngine(), date(2026, 8, 25), ["m_x"], timeframe="1m", + current=pl.DataFrame({"symbol": ["600001.SH"], "name": ["x"]}), + ) + assert ctx.daily_history is not None + assert ctx.daily_history.height == 6 # 引擎声明 5 → 装配日线窗口 + + +def test_minute_context_without_engine_skips_daily_history(): + svc = _svc({date(2026, 8, 25): _bars("600001.SH", [(10.0, 10.2, 10.3)] * 3)}) + ctx = svc.build_strategy_context( + None, date(2026, 8, 25), [], timeframe="1m", + current=pl.DataFrame({"symbol": ["600001.SH"]}), + ) + assert ctx.daily_history is None # 无引擎声明 → 不装配日线 diff --git a/backend/tests/test_minute_timezone_contract.py b/backend/tests/test_minute_timezone_contract.py new file mode 100644 index 0000000..e3912a4 --- /dev/null +++ b/backend/tests/test_minute_timezone_contract.py @@ -0,0 +1,185 @@ +"""分钟 K datetime 北京墙钟契约测试。 + +契约 (CONTRIBUTING §3.3): kline_minute.datetime 必须是北京墙钟 naive。 +守卫 _enforce_minute_beijing_wallclock 在两个源头入口强制: +- _normalize_minute (TickFlow 帧, timestamp 毫秒为 UTC 基准) +- _try_custom_minute (插件/自定义源帧) + +覆盖: 显式转换 / 北京墙钟直通 / UTC 特征自愈 +8 / tz-aware 换算 / +fail-closed 拒收 / 路由级契约违规回退 TickFlow。 +""" +from __future__ import annotations + +from datetime import UTC, datetime +from unittest.mock import MagicMock + +import polars as pl +import pytest + +from app.services import kline_sync + + +def _minute_frame(datetimes: list, symbol: str = "600519.SH") -> pl.DataFrame: + n = len(datetimes) + return pl.DataFrame({ + "symbol": [symbol] * n, + "datetime": datetimes, + "open": [10.0] * n, + "high": [10.5] * n, + "low": [9.5] * n, + "close": [10.2] * n, + "volume": [100.0] * n, + "amount": [1020.0] * n, + }) + + +def _beijing_day() -> list[datetime]: + """一个正常交易日墙钟样本: 开盘/午盘首/收盘。""" + return [ + datetime(2026, 1, 15, 9, 30), + datetime(2026, 1, 15, 13, 0), + datetime(2026, 1, 15, 15, 0), + ] + + +# ---------- TickFlow 路径: timestamp 毫秒 (UTC 基准) → 北京墙钟 ---------- + +def test_tickflow_timestamp_normalizes_to_beijing_wallclock(): + """09:30 北京 = 01:30 UTC; SDK 帧 timestamp 毫秒归一后必须回到 09:30。""" + ts_ms = [ + int(datetime(2026, 1, 15, 1, 30, tzinfo=UTC).timestamp() * 1000), # 09:30 北京 + int(datetime(2026, 1, 15, 5, 0, tzinfo=UTC).timestamp() * 1000), # 13:00 北京 + int(datetime(2026, 1, 15, 7, 0, tzinfo=UTC).timestamp() * 1000), # 15:00 北京 + ] + df = pl.DataFrame({ + "symbol": ["600519.SH"] * 3, + "timestamp": ts_ms, + "open": [10.0] * 3, "high": [10.5] * 3, + "low": [9.5] * 3, "close": [10.2] * 3, + "volume": [100.0] * 3, "amount": [1020.0] * 3, + }) + out = kline_sync._normalize_minute(df) + assert out["datetime"].to_list() == _beijing_day() + + +def test_tickflow_timestamp_partial_day_lunch_unaffected(): + """午间 11:30 (03:30 UTC) 同样正确归一, 不被误判为越界。""" + df = pl.DataFrame({ + "symbol": ["000001.SZ"], + "timestamp": [int(datetime(2026, 1, 15, 3, 30, tzinfo=UTC).timestamp() * 1000)], + "open": [1.0], "high": [1.0], "low": [1.0], "close": [1.0], + "volume": [1.0], "amount": [1.0], + }) + out = kline_sync._normalize_minute(df) + assert out["datetime"].to_list() == [datetime(2026, 1, 15, 11, 30)] + + +# ---------- 守卫: 各口径分类 ---------- + +def test_guard_beijing_naive_passthrough(): + df = _minute_frame(_beijing_day()) + out = kline_sync._enforce_minute_beijing_wallclock(df, source="t") + assert out["datetime"].to_list() == _beijing_day() + + +def test_guard_utc_naive_selfhealed_plus8(): + """01:30/05:00/07:00 (UTC 墙钟特征) → 自动 +8 → 09:30/13:00/15:00。""" + df = _minute_frame([ + datetime(2026, 1, 15, 1, 30), + datetime(2026, 1, 15, 5, 0), + datetime(2026, 1, 15, 7, 0), + ]) + out = kline_sync._enforce_minute_beijing_wallclock(df, source="t") + assert out["datetime"].to_list() == _beijing_day() + + +def test_guard_selfheal_is_idempotent(): + df = _minute_frame([datetime(2026, 1, 15, 1, 30), datetime(2026, 1, 15, 3, 0)]) + once = kline_sync._enforce_minute_beijing_wallclock(df, source="t") + twice = kline_sync._enforce_minute_beijing_wallclock(once, source="t") + assert once["datetime"].to_list() == twice["datetime"].to_list() + + +def test_guard_tzaware_utc_converted(): + """tz-aware UTC 01:30 → 北京墙钟 09:30 (naive)。""" + df = _minute_frame([datetime(2026, 1, 15, 1, 30, tzinfo=UTC)]) + out = kline_sync._enforce_minute_beijing_wallclock(df, source="t") + assert out["datetime"].dtype == pl.Datetime("us") + assert out["datetime"].to_list() == [datetime(2026, 1, 15, 9, 30)] + + +def test_guard_tzaware_shanghai_converted(): + """tz-aware +08:00 09:30 → 北京墙钟 09:30 (naive), 数值不变。""" + df = _minute_frame([datetime(2026, 1, 15, 9, 30, tzinfo=_shanghai_tz())]) + out = kline_sync._enforce_minute_beijing_wallclock(df, source="t") + assert out["datetime"].to_list() == [datetime(2026, 1, 15, 9, 30)] + + +def _shanghai_tz(): + from zoneinfo import ZoneInfo + return ZoneInfo("Asia/Shanghai") + + +def test_guard_unrecognized_convention_fails_closed(): + """21:30/22:15 (境外墙钟特征) 既非北京时段也非 UTC 平移 → 拒收。""" + df = _minute_frame([datetime(2026, 1, 15, 21, 30), datetime(2026, 1, 15, 22, 15)]) + with pytest.raises(ValueError, match="口径无法识别"): + kline_sync._enforce_minute_beijing_wallclock(df, source="t") + + +def test_guard_all_null_datetimes_passthrough(): + """全 null datetime 维持原行为 (下游落盘过滤), 不误伤。""" + df = _minute_frame([None, None]) + out = kline_sync._enforce_minute_beijing_wallclock(df, source="t") + assert out.height == 2 + assert out["datetime"].null_count() == 2 + + +def test_guard_string_datetimes_classified_after_parse(): + """trade_time 字符串路径: 先解析再分类 (UTC 特征串同样自愈)。""" + df = pl.DataFrame({ + "symbol": ["600519.SH"], + "trade_time": ["2026-01-15 01:30:00"], + "open": [1.0], "high": [1.0], "low": [1.0], "close": [1.0], + "volume": [1.0], "amount": [1.0], + }).rename({"trade_time": "datetime"}) + out = kline_sync._enforce_minute_beijing_wallclock(df, source="t") + assert out["datetime"].to_list() == [datetime(2026, 1, 15, 9, 30)] + + +# ---------- 路由级: 自定义源契约违规 → 回退 TickFlow ---------- + +def _setup_custom_provider(monkeypatch, provider: object) -> None: + monkeypatch.setattr(kline_sync.preferences, "get_minute_data_provider", lambda: "mock_src") + monkeypatch.setattr("app.data_providers.custom.provider_has_dataset", lambda name, ds: True) + monkeypatch.setattr("app.data_providers.custom.get_provider", lambda name: provider) + + +def test_custom_provider_utc_frame_selfhealed(monkeypatch): + """插件返回 UTC 墙钟帧 → 路由层守卫 +8 后下发, 不回退。""" + mock_provider = MagicMock() + mock_provider.get_minute = MagicMock(return_value=_minute_frame( + [datetime(2026, 1, 15, 1, 30), datetime(2026, 1, 15, 5, 0)])) + _setup_custom_provider(monkeypatch, mock_provider) + + df, fallback = kline_sync._try_custom_minute( + ["600519.SH"], datetime(2026, 1, 15, 9, 25), datetime(2026, 1, 15, 15, 5), + asset_type="stock", + ) + assert fallback is False + assert df["datetime"].to_list() == [datetime(2026, 1, 15, 9, 30), datetime(2026, 1, 15, 13, 0)] + + +def test_custom_provider_garbage_datetime_falls_back(monkeypatch): + """插件返回无法识别口径 → fail-closed 回退 TickFlow。""" + mock_provider = MagicMock() + mock_provider.get_minute = MagicMock(return_value=_minute_frame( + [datetime(2026, 1, 15, 21, 30)])) + _setup_custom_provider(monkeypatch, mock_provider) + + df, fallback = kline_sync._try_custom_minute( + ["600519.SH"], datetime(2026, 1, 15, 9, 25), datetime(2026, 1, 15, 15, 5), + asset_type="stock", + ) + assert fallback is True + assert df is None diff --git a/backend/tests/test_quote_interval_min.py b/backend/tests/test_quote_interval_min.py new file mode 100644 index 0000000..99efac3 --- /dev/null +++ b/backend/tests/test_quote_interval_min.py @@ -0,0 +1,48 @@ +"""实时行情轮询间隔下限契约测试。 + +下限语义 (中立能力原则): +- 实时源路由到插件/自定义源时, TickFlow 档位限速保护不适用 → 通用下限 1s; +- 实时源为 tickflow 时, 仍按当前订阅档位查表 (none/free=6s, starter=6s, pro=3s, expert=1s); +- 默认间隔 6s 不因路由变化而改变 (只放宽下限, 不动存量偏好)。 +""" +from __future__ import annotations + +from app.services.quote_service import QuoteService + + +def _route(monkeypatch, provider: str, tier: str) -> None: + from app.services import preferences + monkeypatch.setattr(preferences, "get_realtime_data_provider", lambda: provider) + monkeypatch.setattr(QuoteService, "_current_tier", classmethod(lambda cls: tier)) + + +def _bare() -> QuoteService: + """绕过 __init__ (单例/线程副作用), 只用无状态方法。""" + return QuoteService.__new__(QuoteService) + + +def test_custom_provider_min_interval_1s(monkeypatch): + """插件/自定义源: 不受 TickFlow 档位保护, 下限放宽到 1s。""" + _route(monkeypatch, "fuyao", "none") + assert _bare().get_min_interval() == 1.0 + + +def test_custom_provider_clamp_allows_1s(monkeypatch): + """fuyao 路由下设置 1s 不再被抬到 6s; 超过上限仍被压回。""" + _route(monkeypatch, "fuyao", "none") + qs = _bare() + assert qs._clamp_interval(1.0) == 1.0 + assert qs._clamp_interval(0.5) == 1.0 + assert qs._clamp_interval(120.0) == QuoteService.MAX_INTERVAL + + +def test_tickflow_tier_floor_unchanged(monkeypatch): + """TickFlow 路由: 档位查表行为不变 (none/free/starter=6s, pro=3s, expert=1s)。""" + for tier, expect in (("none", 6.0), ("free", 6.0), ("starter", 6.0), ("pro", 3.0), ("expert", 1.0)): + _route(monkeypatch, "tickflow", tier) + assert _bare().get_min_interval() == expect, tier + + +def test_default_interval_unchanged(): + """默认间隔仍是 6s — 放宽的只是下限, 不是默认值。""" + assert QuoteService.DEFAULT_INTERVAL == 6.0 diff --git a/backend/tests/test_realtime_mode.py b/backend/tests/test_realtime_mode.py new file mode 100644 index 0000000..12f90f7 --- /dev/null +++ b/backend/tests/test_realtime_mode.py @@ -0,0 +1,31 @@ +"""回归测试: 实时行情模式判定 — 免费档不再提供自选实时降级。 + +watchlist(自选前 5 只)模式已于 2026-08 移除: 自定义实时源(如 fuyao)的 +全市场快照免费且更优, TickFlow 免费档不再保留降级通路。锁定判定结果, +防止该通路被无意恢复。 +""" +from app.services.quote_service import QuoteService + + +def test_custom_realtime_source_is_full_market(monkeypatch): + """自定义实时源(如 fuyao)无视 TickFlow 档位, 恒为全市场。""" + from app.services import preferences + monkeypatch.setattr(preferences, "get_realtime_data_provider", lambda: "fuyao") + monkeypatch.setattr(QuoteService, "_current_tier", lambda: "free") + assert QuoteService.realtime_mode() == "full_market" + + +def test_tickflow_free_has_no_realtime(monkeypatch): + """TickFlow 免费档 = 无实时(不再降级为自选模式)。""" + from app.services import preferences + monkeypatch.setattr(preferences, "get_realtime_data_provider", lambda: "tickflow") + monkeypatch.setattr(QuoteService, "_current_tier", lambda: "free") + assert QuoteService.realtime_mode() == "none" + assert QuoteService.is_realtime_allowed() is False + + +def test_tickflow_paid_is_full_market(monkeypatch): + from app.services import preferences + monkeypatch.setattr(preferences, "get_realtime_data_provider", lambda: "tickflow") + monkeypatch.setattr(QuoteService, "_current_tier", lambda: "pro") + assert QuoteService.realtime_mode() == "full_market" diff --git a/backend/tests/test_screener_etf.py b/backend/tests/test_screener_etf.py index c201f40..7d37ead 100644 --- a/backend/tests/test_screener_etf.py +++ b/backend/tests/test_screener_etf.py @@ -41,6 +41,7 @@ def test_all_builtin_strategies_declare_asset_types_and_timeframes(): assert engine.load_errors() == [] for meta in engine.list_strategies(): assert meta["asset_types"] + # 分钟红7已迁至自定义策略目录, 内置策略均为日线 assert meta["timeframes"] == ["1d"] @@ -48,11 +49,13 @@ def test_all_builtin_strategies_use_matrix_backend_only(): engine = _engine() assert engine.load_errors() == [] strategies = [engine.get(meta["id"]) for meta in engine.list_strategies()] - assert len(strategies) == 18 - assert all(strategy.execution_backend == "matrix_native" for strategy in strategies) - assert all(strategy.matrix_strategy is not None for strategy in strategies) - assert all(strategy.filter_fn is None for strategy in strategies) - assert all(strategy.filter_history_fn is None for strategy in strategies) + matrix_strategies = [s for s in strategies if s.execution_backend == "matrix_native"] + assert len(matrix_strategies) == 18 + assert all(s.matrix_strategy is not None for s in matrix_strategies) + assert all(s.filter_fn is None for s in matrix_strategies) + assert all(s.filter_history_fn is None for s in matrix_strategies) + # 分钟形态策略 (minute_filter) 已迁至自定义策略目录, 不在 builtin 加载范围 + assert [s.meta["id"] for s in strategies if s.execution_backend == "minute_filter"] == [] def test_all_builtin_matrix_formulas_accept_base_market_matrix(): @@ -79,10 +82,14 @@ def test_all_builtin_matrix_formulas_accept_base_market_matrix(): from app.backtest.matrix import build_market_data_matrix fields = set() - for strategy in (engine.get(meta["id"]) for meta in engine.list_strategies()): + matrix_metas = [ + m for m in engine.list_strategies() + if engine.get(m["id"]).execution_backend == "matrix_native" + ] + for strategy in (engine.get(meta["id"]) for meta in matrix_metas): fields.update(engine._matrix_field_columns(strategy)) market = build_market_data_matrix(panel, field_columns=fields) - for meta in engine.list_strategies(): + for meta in matrix_metas: strategy = engine.get(meta["id"]) signals = strategy.matrix_strategy.compute_signals(market, {}) assert signals.shape == market.shape, meta["id"] diff --git a/backend/tests/test_stocksdk_provider.py b/backend/tests/test_stocksdk_provider.py index d6ddc57..de2453f 100644 --- a/backend/tests/test_stocksdk_provider.py +++ b/backend/tests/test_stocksdk_provider.py @@ -226,3 +226,82 @@ def test_builtin_not_editable(): raise AssertionError("expected ValueError for builtin") except ValueError: pass + + +# ---------- 分钟 open 数据卫生 (stock-sdk 上游区间模式给日级常量伪 open) ---------- + +def _sdk_rows(day: str, opens, closes): + """构造 bridge 返回形状的分钟行 (timestamp 为北京墙钟对应 UTC 毫秒)。""" + from datetime import UTC, datetime, timedelta + rows = [] + for i, (o, c) in enumerate(zip(opens, closes, strict=True)): + dt = datetime.fromisoformat(f"{day} 09:30:00") + timedelta(minutes=i) + ts = int(dt.replace(tzinfo=UTC).timestamp() * 1000) - 8 * 3600_000 + rows.append({"timestamp": ts, "open": o, "high": max(o, c), "low": min(o, c), + "close": c, "volume": 100 + i, "amount": 1000.0 + i}) + return rows + + +def test_minute_degenerate_open_nulled(): + """历史日 open 为全天常量(uniq=1)而 close 多值 → open 置 null。""" + n = 30 + rows = _sdk_rows("2026-08-27", [8.0] * n, [10 + i * 0.01 for i in range(n)]) + df = StockSDKProvider._minute_df(rows, "600664.SH") + assert df.height == n + assert df["open"].null_count() == n # 伪 open 全部置 null + assert df["close"].null_count() == 0 # close/high/low 保留 + + +def test_minute_real_open_kept(): + """真实分钟 open (多唯一值) 原样保留。""" + n = 30 + rows = _sdk_rows("2026-08-28", [10 + i * 0.01 for i in range(n)], [10.05 + i * 0.01 for i in range(n)]) + df = StockSDKProvider._minute_df(rows, "600664.SH") + assert df["open"].null_count() == 0 + assert df["open"].n_unique() == n + + +def test_minute_short_day_open_kept(): + """短交易日 (rows<=10, 如半日/首日少量bar) 不误杀。""" + rows = _sdk_rows("2026-08-26", [8.0] * 6, [8.0 + i * 0.1 for i in range(6)]) + df = StockSDKProvider._minute_df(rows, "600664.SH") + assert df["open"].null_count() == 0 + + +def test_get_minute_splits_tail_into_single_day_jobs(monkeypatch): + """多日区间 → 末尾 3 自然日(跳过周末)逐日单拉 (保住最新交易日真实 open)。""" + from datetime import datetime + + jobs = [] + + def fake_run_job(job, timeout=None): + jobs.append(job) + return {"ok": True, "op": "minute", "rows": {}} + + monkeypatch.setattr(sp.bridge, "run_job", fake_run_job) + p = StockSDKProvider() + p.get_minute(["600519.SH"], datetime(2026, 8, 20), datetime(2026, 8, 29, 23, 0)) + + # 末尾 4 个自然日 (08-26..08-29) 逐日单拉, 周六 08-29 跳过; + # 前段 = [08-20, 08-25] 一个区间任务 + spans = [(j["start"], j["end"]) for j in jobs] + for day in ("20260826", "20260827", "20260828"): + assert (day, day) in spans + assert ("20260829", "20260829") not in spans # 周六不单拉 + assert ("20260820", "20260825") in spans + + +def test_get_minute_single_day_no_split(monkeypatch): + """单日区间不拆分, 保持一个任务。""" + from datetime import datetime + + jobs = [] + + def fake_run_job(job, timeout=None): + jobs.append(job) + return {"ok": True, "op": "minute", "rows": {}} + + monkeypatch.setattr(sp.bridge, "run_job", fake_run_job) + StockSDKProvider().get_minute(["600519.SH"], datetime(2026, 8, 28), datetime(2026, 8, 28, 15, 0)) + assert len(jobs) == 1 + assert jobs[0]["start"] == "20260828" diff --git a/backend/tests/test_trading_day.py b/backend/tests/test_trading_day.py new file mode 100644 index 0000000..baa6322 --- /dev/null +++ b/backend/tests/test_trading_day.py @@ -0,0 +1,257 @@ +"""交易日探针 (trading_day oracle) 与两个消费方接入的测试。 + +不依赖真实网络: 探测函数 (_probe_fuyao / _probe_tickflow) 全部 monkeypatch。 +覆盖: 周末零成本直判、探测链优先级 (fuyao 日历权威, 无开盘缓冲问题)、 +tickflow 戳的 OR 语义与开盘缓冲窗、失败/无权限 → None、TTL 缓存、 +实时行情门控与分钟增量 gate_reason 的 holiday 分支。 +""" + +from __future__ import annotations + +from datetime import date, datetime, time as dt_time, timezone, timedelta + +import pytest + +from app.services import trading_day +from app.services.trading_day import is_trading_day, reset_cache + +CN = timezone(timedelta(hours=8)) + + +@pytest.fixture(autouse=True) +def _clean_cache(): + reset_cache() + yield + reset_cache() + + +def _no_probes(monkeypatch): + """探测函数替换为爆炸 — 用于验证未被打到。""" + monkeypatch.setattr(trading_day, "_probe_fuyao", lambda now: (_ for _ in ()).throw(AssertionError("不应探测"))) + monkeypatch.setattr(trading_day, "_probe_tickflow", lambda now: (_ for _ in ()).throw(AssertionError("不应探测"))) + + +# ---- 周末零成本直判 ---- + +def test_weekend_returns_false_without_probing(monkeypatch): + sat = datetime(2026, 8, 29, 10, 0, tzinfo=CN) # 周六 + sun = datetime(2026, 8, 30, 10, 0, tzinfo=CN) # 周日 + _no_probes(monkeypatch) + assert is_trading_day(sat) is False + assert is_trading_day(sun) is False + + +# ---- 探测链优先级 ---- + +def test_fuyao_calendar_is_authoritative_even_before_open(monkeypatch): + """fuyao 日历结论无时段依赖: 开盘缓冲窗内也直接生效。""" + holiday_mon = datetime(2026, 9, 7, 9, 31, tzinfo=CN) # 周一 (缓冲窗内) + monkeypatch.setattr(trading_day, "_probe_fuyao", lambda now: False) + monkeypatch.setattr( + trading_day, "_probe_tickflow", + lambda now: (_ for _ in ()).throw(AssertionError("fuyao 已有结论不应继续探测")), + ) + assert is_trading_day(holiday_mon) is False + + +def test_chain_falls_through_to_tickflow_when_fuyao_unknown(monkeypatch): + monday = datetime(2026, 9, 7, 10, 0, tzinfo=CN) + monkeypatch.setattr(trading_day, "_probe_fuyao", lambda now: None) + monkeypatch.setattr(trading_day, "_probe_tickflow", lambda now: True) + assert is_trading_day(monday) is True + + +def test_all_probes_unknown_returns_none(monkeypatch): + monday = datetime(2026, 9, 7, 10, 0, tzinfo=CN) + monkeypatch.setattr(trading_day, "_probe_fuyao", lambda now: None) + monkeypatch.setattr(trading_day, "_probe_tickflow", lambda now: None) + assert is_trading_day(monday) is None + + +# ---- tickflow 戳语义 ---- + +def test_tickflow_stale_stamp_before_buffer_is_unknown(monkeypatch): + """开盘缓冲窗内戳停在昨日: 可能是集合竞价未翻新, 保守判未知。""" + monday_935 = datetime(2026, 9, 7, 9, 35, tzinfo=CN) + import app.tickflow.client as tf_client_mod + + class _FakeQuotes: + def get(self, symbols): + # 周一 9:35 拉到上周五 15:30 的戳 + return [{"symbol": s, "timestamp": 1787902251001} for s in symbols] + + class _FakeClient: + quotes = _FakeQuotes() + + monkeypatch.setattr(tf_client_mod, "get_client", lambda: _FakeClient()) + monkeypatch.setattr(trading_day, "_probe_fuyao", lambda now: None) + assert trading_day._probe_tickflow(monday_935) is None + + +def test_tickflow_stale_stamp_after_buffer_is_holiday(monkeypatch): + monday_1041 = datetime(2026, 9, 7, 10, 41, tzinfo=CN) + import app.tickflow.client as tf_client_mod + + class _FakeQuotes: + def get(self, symbols): + return [{"symbol": s, "timestamp": 1787902251001} for s in symbols] # 上周五 + + class _FakeClient: + quotes = _FakeQuotes() + + monkeypatch.setattr(tf_client_mod, "get_client", lambda: _FakeClient()) + assert trading_day._probe_tickflow(monday_1041) is False + + +def test_tickflow_fresh_stamp_is_trading_anytime(monkeypatch): + """缓冲窗内只要戳是今天就判交易日 (OR 语义, 任一翻新即可)。""" + monday_920 = datetime(2026, 9, 7, 9, 20, tzinfo=CN) + import app.tickflow.client as tf_client_mod + + class _FakeQuotes: + def get(self, symbols): + # 一只翻新 + 其余停在周五 → max 为今日 + fresh_ms = int(monday_920.timestamp() * 1000) + return [ + {"symbol": "000001.SZ", "timestamp": fresh_ms}, + {"symbol": "600519.SH", "timestamp": 1787902251001}, + ] + + class _FakeClient: + quotes = _FakeQuotes() + + monkeypatch.setattr(tf_client_mod, "get_client", lambda: _FakeClient()) + assert trading_day._probe_tickflow(monday_920) is True + + +def test_tickflow_error_returns_none(monkeypatch): + monday = datetime(2026, 9, 7, 10, 41, tzinfo=CN) + import app.tickflow.client as tf_client_mod + + def _boom(): + raise RuntimeError("无实时权限 (free 档)") + + monkeypatch.setattr(tf_client_mod, "get_client", _boom) + assert trading_day._probe_tickflow(monday) is None + + +# ---- TTL 缓存 ---- + +def test_verdict_cached_within_ttl(monkeypatch): + monday = datetime(2026, 9, 7, 10, 0, tzinfo=CN) + calls = {"n": 0} + + def _counting_probe(now): + calls["n"] += 1 + return True + + monkeypatch.setattr(trading_day, "_probe_fuyao", _counting_probe) + assert is_trading_day(monday) is True + assert is_trading_day(monday) is True + assert is_trading_day(monday) is True + assert calls["n"] == 1 # 命中缓存, 只探一次 + + +def test_unknown_verdict_retries_after_short_ttl(monkeypatch): + monday = datetime(2026, 9, 7, 10, 0, tzinfo=CN) + calls = {"n": 0} + + def _counting_probe(now): + calls["n"] += 1 + return None + + monkeypatch.setattr(trading_day, "_probe_fuyao", _counting_probe) + monkeypatch.setattr(trading_day, "_probe_tickflow", lambda now: None) # 隔离真实网络 + assert is_trading_day(monday) is None + # 手动把缓存时间拨回 10 分钟前 (超过 unknown TTL 300s) → 重探 + with trading_day._CACHE_LOCK: + trading_day._CACHE.probed_at -= 600 + assert is_trading_day(monday) is None + assert calls["n"] == 2 + + +# ---- 消费方 1: 实时行情门控 ---- + +def test_quote_service_holiday_gate_blocks_polling(monkeypatch): + from app.services.quote_service import QuoteService + + qs = QuoteService() + monkeypatch.setattr(trading_day, "is_trading_day", lambda now=None: False) + assert qs._should_poll_for_phase("morning") is False + assert qs._should_poll_for_phase("preopen") is False + # final 定版同样被剔除: 休市日没有需要定版的当日行情 + assert qs._should_poll_for_phase("close_final") is False + + +def test_quote_service_polls_when_trading_or_unknown(monkeypatch): + from app.services.quote_service import QuoteService + + qs = QuoteService() + monkeypatch.setattr(trading_day, "is_trading_day", lambda now=None: True) + assert qs._should_poll_for_phase("morning") is True + # 未知 → 维持现状 (周几近似) + monkeypatch.setattr(trading_day, "is_trading_day", lambda now=None: None) + assert qs._should_poll_for_phase("morning") is True + + +# ---- 消费方 2: 分钟增量 gate_reason ---- + +def _minute_service(monkeypatch): + from app.services.minute_refresh import MinuteRefreshService + + svc = MinuteRefreshService.__new__(MinuteRefreshService) + svc._app_state = None + monkeypatch.setattr( + "app.services.minute_refresh.preferences.get_minute_refresh_enabled", + lambda: True, + ) + monkeypatch.setattr(svc, "custom_provider_active", lambda: False) + monkeypatch.setattr(svc, "capability_ok", lambda: True) + return svc + + +def test_minute_refresh_gate_returns_holiday(monkeypatch): + """周几+时段门控放行 (周一盘中) 但探针判休市 → holiday。""" + svc = _minute_service(monkeypatch) + monday_1030 = datetime(2026, 9, 7, 10, 30, tzinfo=CN) + monkeypatch.setattr( + "app.services.minute_refresh._in_continuous_session", lambda now=None: True + ) + monkeypatch.setattr(trading_day, "is_trading_day", lambda now=None: False) + assert svc._gate_reason() == "holiday" + + +def test_minute_refresh_gate_passes_when_trading(monkeypatch): + svc = _minute_service(monkeypatch) + monkeypatch.setattr( + "app.services.minute_refresh._in_continuous_session", lambda now=None: True + ) + monkeypatch.setattr(trading_day, "is_trading_day", lambda now=None: True) + assert svc._gate_reason() is None + + +# ---- fuyao 日历解析 ---- + +def test_fuyao_provider_trading_days_conversion(monkeypatch): + from app.plugins.fuyao.provider import FuyaoProvider + from app.plugins.fuyao import provider as fp + + class _CalClient: + def trading_days(self): + # 上海零点戳按 provider 的 _ms_of_date 口径构造 + def _midnight_ms(d): + import calendar + return (calendar.timegm(d.timetuple()) - 28800) * 1000 + + d4, d7 = date(2026, 9, 4), date(2026, 9, 7) + return [ + {"date": "20260904", "date_ms": _midnight_ms(d4)}, + {"date": "20260907", "date_ms": _midnight_ms(d7)}, + ] + + monkeypatch.setattr( + fp, "fuyao_client", type("M", (), {"FuyaoClient": lambda **kw: _CalClient()}) + ) + monkeypatch.setattr(fp, "get_api_key", lambda: "test-key") + days = FuyaoProvider().trading_days() + assert days == {date(2026, 9, 4), date(2026, 9, 7)} diff --git a/backend/tests/test_volume_delta_monitor.py b/backend/tests/test_volume_delta_monitor.py new file mode 100644 index 0000000..df1a36d --- /dev/null +++ b/backend/tests/test_volume_delta_monitor.py @@ -0,0 +1,274 @@ +"""轮询放量监控 (volume_delta) 测试: 引擎命中/冷却/批量合并 + 基础过滤 + 快照差值边界。""" +from __future__ import annotations + +from datetime import date + +import polars as pl +import pytest + +from app.strategy import monitor_rules +from app.strategy.monitor import MonitorRuleEngine + + +def _df(rows: list[dict]) -> pl.DataFrame: + """rows 每项: symbol/_volume_delta 必填, 其余可选 (close/amount/total_shares/float_shares)。""" + base = { + "symbol": [], "close": [], "change_pct": [], + "_volume_delta": [], "_volume_delta_amount": [], "_volume_delta_span": [], + } + optional = ["amount", "total_shares", "float_shares"] + for r in rows: + base["symbol"].append(r["symbol"]) + base["close"].append(r.get("close", 10.0)) + base["change_pct"].append(0.01) + base["_volume_delta"].append(r["_volume_delta"]) + base["_volume_delta_amount"].append(r.get("_volume_delta_amount", r["_volume_delta"] * 1000.0)) + base["_volume_delta_span"].append(6.0) + data = {k: v for k, v in base.items()} + for col in optional: + vals = [r.get(col) for r in rows] + if any(v is not None for v in vals): + data[col] = [v if v is not None else 0.0 for v in vals] + return pl.DataFrame(data) + + +def _rule(**kw): + r = { + "id": "vd1", "name": "轮询放量", "type": "volume_delta", + "asset_type": "stock", "scope": "all", "enabled": True, + "threshold_volume": 9000, "cooldown_seconds": 300, + "severity": "warn", + } + r.update(kw) + return r + + +def test_volume_delta_hits_above_threshold(): + eng = MonitorRuleEngine() + eng.set_rules([_rule()]) + events = eng.evaluate(_df([ + {"symbol": "S1.SH", "_volume_delta": 9500.0, "close": 10.0}, + {"symbol": "S2.SH", "_volume_delta": 8999.0, "close": 20.0}, + ])) + assert [e["symbol"] for e in events] == ["S1.SH"] + ev = events[0] + assert ev["source"] == "volume_delta" + assert "9,500" in ev["message"] and "9,000" in ev["message"] and "间隔 6s" in ev["message"] + assert ev["volume_delta"] == 9500.0 + + +def test_volume_delta_no_column_degrades_silently(): + eng = MonitorRuleEngine() + eng.set_rules([_rule()]) + plain = pl.DataFrame({"symbol": ["S1.SH"], "close": [10.0]}) + assert eng.evaluate(plain) == [] + + +def test_volume_delta_cooldown_suppresses_repeat(): + eng = MonitorRuleEngine() + eng.set_rules([_rule(cooldown=300)]) + df = _df([{"symbol": "S1.SH", "_volume_delta": 12000.0}]) + assert len(eng.evaluate(df)) == 1 + assert eng.evaluate(df) == [] + + +def test_volume_delta_batch_merge_over_five(): + eng = MonitorRuleEngine() + eng.set_rules([_rule()]) + rows = [{"symbol": f"S{i}.SH", "_volume_delta": 20000.0 + i} for i in range(8)] + events = eng.evaluate(_df(rows)) + assert len(events) == 1 + assert events[0]["symbol"] == "" + assert "共 8 只" in events[0]["message"] + + +def test_volume_delta_scope_filters(): + eng = MonitorRuleEngine() + eng.set_rules([_rule(scope="symbols", symbols=["S2.SH"])]) + events = eng.evaluate(_df([ + {"symbol": "S1.SH", "_volume_delta": 9500.0}, + {"symbol": "S2.SH", "_volume_delta": 9500.0}, + ])) + assert [e["symbol"] for e in events] == ["S2.SH"] + + +def test_volume_delta_metric_amount(): + eng = MonitorRuleEngine() + eng.set_rules([_rule(metric="amount", threshold_amount=5e6)]) + events = eng.evaluate(_df([ + {"symbol": "S1.SH", "_volume_delta": 100.0, "_volume_delta_amount": 6e6}, + {"symbol": "S2.SH", "_volume_delta": 20000.0, "_volume_delta_amount": 4.9e6}, + ])) + assert [e["symbol"] for e in events] == ["S1.SH"] + assert "万元" in events[0]["message"] + + +def test_volume_delta_basic_filter_price_and_amount(): + eng = MonitorRuleEngine() + eng.set_rules([_rule(basic_filter={ + "price_min": 5, "price_max": 100, "amount_min": 1e8, "exclude_st": False, + })]) + events = eng.evaluate(_df([ + # 价低被滤 + {"symbol": "LOW.SH", "_volume_delta": 20000.0, "close": 3.0, "amount": 5e8}, + # 价过高被滤 + {"symbol": "HIGH.SH", "_volume_delta": 20000.0, "close": 200.0, "amount": 5e8}, + # 成交额不足被滤 + {"symbol": "THIN.SH", "_volume_delta": 20000.0, "close": 10.0, "amount": 5e7}, + # 通过 + {"symbol": "OK.SH", "_volume_delta": 20000.0, "close": 10.0, "amount": 5e8}, + ])) + assert [e["symbol"] for e in events] == ["OK.SH"] + + +def test_volume_delta_basic_filter_market_cap(): + eng = MonitorRuleEngine() + eng.set_rules([_rule(basic_filter={ + "market_cap_min": 20e8, "price_min": None, "price_max": None, + "amount_min": None, "exclude_st": False, + })]) + # close × total_shares: BIG 10×3e8=30亿 通过; SMALL 10×1e8=10亿 被滤 + events = eng.evaluate(_df([ + {"symbol": "BIG.SH", "_volume_delta": 20000.0, "total_shares": 3e8}, + {"symbol": "SMALL.SH", "_volume_delta": 20000.0, "total_shares": 1e8}, + ])) + assert [e["symbol"] for e in events] == ["BIG.SH"] + + +def test_volume_delta_basic_filter_exclude_st(): + eng = MonitorRuleEngine() + eng.set_name_map({"STOCK.SH": "平安银行", "STK.SH": "ST 某某"}) + eng.set_rules([_rule(basic_filter={ + "price_min": None, "price_max": None, "amount_min": None, "exclude_st": True, + })]) + events = eng.evaluate(_df([ + {"symbol": "STOCK.SH", "_volume_delta": 20000.0}, + {"symbol": "STK.SH", "_volume_delta": 20000.0}, + ])) + assert [e["symbol"] for e in events] == ["STOCK.SH"] + + +def test_validate_and_normalize_defaults(): + r = monitor_rules.normalize({"id": "vd2", "type": "volume_delta"}) + assert r["threshold_volume"] == 9000 + assert r["scope"] == "all" + assert r["cooldown_seconds"] == 300 + assert r["metric"] == "volume" + assert r["basic_filter"]["price_min"] == 3 + assert r["basic_filter"]["exclude_st"] is True + # 用户字段覆盖默认 + r2 = monitor_rules.normalize({"id": "vd5", "type": "volume_delta", "basic_filter": {"price_min": 1, "exclude_st": False}}) + assert r2["basic_filter"]["price_min"] == 1 + assert r2["basic_filter"]["exclude_st"] is False + assert r2["basic_filter"]["price_max"] == 300 # 未覆盖项保留默认 + monitor_rules.validate({"id": "vd2", "name": "n", "type": "volume_delta", "threshold_volume": 1}) + with pytest.raises(ValueError): + monitor_rules.validate({"id": "vd3", "name": "n", "type": "volume_delta", "threshold_volume": 0}) + with pytest.raises(ValueError): + monitor_rules.validate({"id": "vd4", "name": "n", "type": "volume_delta", "asset_type": "etf"}) + with pytest.raises(ValueError): + monitor_rules.validate({"id": "vd6", "name": "n", "type": "volume_delta", + "metric": "amount", "threshold_amount": 0}) + with pytest.raises(ValueError): + monitor_rules.validate({"id": "vd7", "name": "n", "type": "volume_delta", + "basic_filter": {"price_min": -1}}) + with pytest.raises(ValueError): + monitor_rules.validate({"id": "vd8", "name": "n", "type": "volume_delta", + "basic_filter": {"unknown_field": 1}}) + + +# ── 快照差值状态 (QuoteService) ────────────────────────── + +def _qs(monkeypatch, *, continuous=True): + from app.services.quote_service import QuoteService + qs = QuoteService.__new__(QuoteService) + qs._prev_stock_volume = None + qs._prev_volume_fetched_at = None + qs._prev_volume_date = None + qs._volume_delta = {} + qs._volume_delta_span_s = 0.0 + monkeypatch.setattr(QuoteService, "_is_continuous_trading", lambda self: continuous) + monkeypatch.setattr( + QuoteService, "_continuous_session_start_ms", + staticmethod(lambda: 0.0), + ) + monkeypatch.setattr("app.services.quote_service.cn_today", lambda: date(2026, 8, 25)) + return qs + + +def test_delta_computed_and_prev_updated(monkeypatch): + qs = _qs(monkeypatch) + t0 = 1_000_000.0 + qs._update_volume_delta( + [{"symbol": "S1.SH", "volume": 10000, "amount": 5e6}, + {"symbol": "S2.SH", "volume": 500, "amount": 1e6}], t0, + ) + assert qs._volume_delta == {} # 首轮无 prev + qs._update_volume_delta( + [{"symbol": "S1.SH", "volume": 19500, "amount": 9.5e6}, + {"symbol": "S2.SH", "volume": 400, "amount": 2e6}], t0 + 6000, + ) + # S2 volume cur < prev (重置) → 丢弃; S1 差值 (9500 手, 450 万元) + assert qs._volume_delta == {"S1.SH": (9500.0, 4.5e6)} + assert qs._volume_delta_span_s == 6.0 + + +def test_delta_cross_day_reset(monkeypatch): + import app.services.quote_service as qsm + qs = _qs(monkeypatch) + qs._update_volume_delta([{"symbol": "S1.SH", "volume": 10000}], 1000.0) + assert qs._prev_volume_date == date(2026, 8, 25) + monkeypatch.setattr(qsm, "cn_today", lambda: date(2026, 8, 26)) + qs._update_volume_delta([{"symbol": "S1.SH", "volume": 20000}], 2000.0) + assert qs._volume_delta == {} + assert qs._prev_volume_date == date(2026, 8, 26) + + +def test_delta_open_protection(monkeypatch): + qs = _qs(monkeypatch) + # 9:29 的 prev (早于 9:30 时段起点) → 9:31 本轮不触发 + session_start = 1_000_000.0 + monkeypatch.setattr( + type(qs), "_continuous_session_start_ms", + staticmethod(lambda: session_start), + ) + qs._update_volume_delta([{"symbol": "S1.SH", "volume": 10000}], session_start - 60_000) + qs._update_volume_delta([{"symbol": "S1.SH", "volume": 99999}], session_start + 60_000) + assert qs._volume_delta == {} + # 之后一轮 prev 已在时段内 → 恢复计算 + qs._update_volume_delta([{"symbol": "S1.SH", "volume": 109999}], session_start + 66_000) + assert qs._volume_delta == {"S1.SH": (10000.0, 0.0)} + + +def test_delta_not_continuous_trading(monkeypatch): + qs = _qs(monkeypatch, continuous=False) + qs._update_volume_delta([{"symbol": "S1.SH", "volume": 10000}], 1000.0) + qs._update_volume_delta([{"symbol": "S1.SH", "volume": 99999}], 7000.0) + # 非连续竞价 (如午休) 不产差值, 但 prev 持续更新 + assert qs._volume_delta == {} + assert qs._prev_stock_volume == {"S1.SH": (99999.0, 0.0)} + + +def test_inject_volume_delta_join(): + from app.services.quote_service import QuoteService + qs = QuoteService.__new__(QuoteService) + qs._volume_delta = {"S1.SH": (900.0, 9e5), "S9.SH": (500.0, 5e5)} + qs._volume_delta_span_s = 6.0 + base = pl.DataFrame({"symbol": ["S1.SH", "S2.SH"], "close": [10.0, 20.0]}) + out = qs._inject_volume_delta(base) + assert out.filter(pl.col("symbol") == "S1.SH")["_volume_delta"][0] == 900.0 + assert out.filter(pl.col("symbol") == "S1.SH")["_volume_delta_amount"][0] == 9e5 + # 未命中股票为 null (不触发) + assert out.filter(pl.col("symbol") == "S2.SH")["_volume_delta"][0] is None + # 空差值原样返回 + qs._volume_delta = {} + assert qs._inject_volume_delta(base).columns == ["symbol", "close"] + + +def test_session_start_ms_matches_clock(): + from app.services.quote_service import QuoteService + from datetime import datetime, time as dt_time, timedelta, timezone + + now = QuoteService._continuous_session_start_ms() / 1000.0 + start_dt = datetime.fromtimestamp(now, tz=timezone(timedelta(hours=8))) + assert start_dt.time() in (dt_time(9, 30), dt_time(13, 0)) diff --git a/backend/tests/test_watchlist_batch.py b/backend/tests/test_watchlist_batch.py deleted file mode 100644 index 251d240..0000000 --- a/backend/tests/test_watchlist_batch.py +++ /dev/null @@ -1,123 +0,0 @@ -"""回归测试: Free 档自选实时 symbols 超过 capability batch 上限时分批请求 (PR #46 问题 4)。""" -from contextlib import ExitStack -from unittest.mock import MagicMock, patch - -import polars as pl - -from app.services.quote_service import QuoteService -from app.tickflow.capabilities import Cap, CapabilityLimits, CapabilitySet - - -def _make_svc(engine_rules: dict) -> QuoteService: - """创建最小可用的 QuoteService 实例 (跳过 __init__)。""" - svc = QuoteService.__new__(QuoteService) - svc._app_state = MagicMock() - svc._repo = MagicMock() - svc._lock = MagicMock() - - engine = MagicMock() - engine.rules = engine_rules - svc._app_state.monitor_engine = engine - svc._app_state.repo = svc._repo - - svc._repo.get_index_symbol_set.return_value = {"000001.SH"} - svc._repo.get_etf_symbol_set.return_value = set() - return svc - - -def _run_fetch(svc, tf, watchlist: list[str], capset: CapabilitySet): - """在完整 patch 环境下执行 _fetch_watchlist_quotes。""" - with ExitStack() as stack: - stack.enter_context(patch( - "app.services.preferences.get_realtime_watchlist_symbols", - return_value=watchlist, - )) - stack.enter_context(patch( - "app.tickflow.client.get_paid_realtime_client", return_value=tf, - )) - stack.enter_context(patch( - "app.tickflow.policy.detect_capabilities", return_value=capset, - )) - stack.enter_context(patch("app.tickflow.rate_limits.sleep_between_batches")) - # patch 分批之后的下游处理 - stack.enter_context(patch.object( - QuoteService, "_build_daily", return_value=pl.DataFrame(), - )) - stack.enter_context(patch.object( - QuoteService, "_build_quote_extra", return_value=pl.DataFrame(), - )) - stack.enter_context(patch.object( - QuoteService, "_build_index_quotes", return_value=pl.DataFrame(), - )) - stack.enter_context(patch.object(QuoteService, "_broadcast_quote_updated")) - stack.enter_context(patch.object(QuoteService, "_evaluate_monitors")) - stack.enter_context(patch("app.services.quote_service._persist_last_fetch")) - svc._fetch_watchlist_quotes() - - -def test_watchlist_batch_respects_capability_limit(): - """6 symbols / batch 5 → 分 2 批请求, 不整轮失败。""" - engine_rules = { - "r_idx": {"enabled": True, "asset_type": "index", "scope": "symbols", - "symbols": ["000001.SH"]}, - } - svc = _make_svc(engine_rules) - - tf = MagicMock() - tf.quotes.get.return_value = [ - {"symbol": "600000.SH", "last_price": 10.0, "prev_close": 9.9, "ext": {}}, - ] - capset = CapabilitySet({Cap.QUOTE_BY_SYMBOL: CapabilityLimits(batch=5, rpm=60)}) - - _run_fetch(svc, tf, - ["600000.SH", "600001.SH", "600002.SH", "600003.SH", "600004.SH"], - capset) - - # 5 股票 + 1 指数 = 6 symbols, batch 5 → 2 批 - assert tf.quotes.get.call_count == 2 - first_batch = tf.quotes.get.call_args_list[0][1]["symbols"] - second_batch = tf.quotes.get.call_args_list[1][1]["symbols"] - assert len(first_batch) == 5 - assert len(second_batch) == 1 - assert "000001.SH" in second_batch - - -def test_watchlist_batch_partial_failure_keeps_other_batches(): - """某一批拉取失败不影响其他批次 (已有股票实时刷新不丢失)。""" - engine_rules = { - "r_idx": {"enabled": True, "asset_type": "index", "scope": "symbols", - "symbols": ["000001.SH"]}, - } - svc = _make_svc(engine_rules) - - tf = MagicMock() - # 第一批 (股票) 成功, 第二批 (指数) 失败 - tf.quotes.get.side_effect = [ - [{"symbol": "600000.SH", "last_price": 10.0, "prev_close": 9.9, "ext": {}}], - ConnectionError("timeout"), - ] - capset = CapabilitySet({Cap.QUOTE_BY_SYMBOL: CapabilityLimits(batch=5, rpm=60)}) - - _run_fetch(svc, tf, - ["600000.SH", "600001.SH", "600002.SH", "600003.SH", "600004.SH"], - capset) - - # 两批都被尝试 (第二批失败不阻断) - assert tf.quotes.get.call_count == 2 - - -def test_watchlist_no_index_rules_no_extra_symbols(): - """无指数监控规则时, symbols 不追加指数标的。""" - svc = _make_svc({}) # 无规则 - - tf = MagicMock() - tf.quotes.get.return_value = [ - {"symbol": "600000.SH", "last_price": 10.0, "prev_close": 9.9, "ext": {}}, - ] - capset = CapabilitySet({Cap.QUOTE_BY_SYMBOL: CapabilityLimits(batch=5, rpm=60)}) - - _run_fetch(svc, tf, ["600000.SH", "600001.SH"], capset) - - # 2 symbols / batch 5 → 1 批 - assert tf.quotes.get.call_count == 1 - assert tf.quotes.get.call_args_list[0][1]["symbols"] == ["600000.SH", "600001.SH"] diff --git a/backend/tests/test_watchlist_realtime_split.py b/backend/tests/test_watchlist_realtime_split.py deleted file mode 100644 index 7529a96..0000000 --- a/backend/tests/test_watchlist_realtime_split.py +++ /dev/null @@ -1,19 +0,0 @@ -"""Free 档自选实时资产分流测试。""" -from app.services.quote_service import QuoteService - - -def test_split_records_by_asset(): - records = [ - {"symbol": "600000.SH"}, {"symbol": "510300.SH"}, {"symbol": "000001.SH"}, - ] - index, etf, stock = QuoteService._split_records_by_asset( - records, {"000001.SH"}, {"510300.SH"}, - ) - assert [r["symbol"] for r in index] == ["000001.SH"] - assert [r["symbol"] for r in etf] == ["510300.SH"] - assert [r["symbol"] for r in stock] == ["600000.SH"] - # etf 优先于 index (与 resolve_asset_type 判定顺序一致) - index2, etf2, stock2 = QuoteService._split_records_by_asset( - [{"symbol": "X"}], {"X"}, {"X"}, - ) - assert etf2 and not index2 and not stock2 diff --git a/backend/uv.lock b/backend/uv.lock index e90c8ec..941ec69 100644 --- a/backend/uv.lock +++ b/backend/uv.lock @@ -2493,15 +2493,15 @@ wheels = [ [[package]] name = "tickflow" -version = "0.1.24" +version = "0.1.25" source = { registry = "https://pypi.org/simple" } dependencies = [ { name = "httpx" }, { name = "typing-extensions" }, ] -sdist = { url = "https://files.pythonhosted.org/packages/aa/56/d911f7d03363a06f69838878df4f92dd01235c899431197f94fb4c0e36ad/tickflow-0.1.24.tar.gz", hash = "sha256:13f6464a9dd1bdf98a312bc8c313fcc24ad0ad83c16d4f83b76bee83df6e11df", size = 36867, upload-time = "2026-06-20T02:39:35.891Z" } +sdist = { url = "https://files.pythonhosted.org/packages/d1/c9/facd0cd7568ea3c7dcfb0d52ded7d0b52ad871ca45fa97d7df0925b26091/tickflow-0.1.25.tar.gz", hash = "sha256:a86929bc99167014567c3d8b99af1c2f498922ad476bb4507edb57164c576d04", size = 37357, upload-time = "2026-08-29T04:43:27.255Z" } wheels = [ - { url = "https://files.pythonhosted.org/packages/4c/9d/6c03706054f3bcca8a7113a60258e1a52a762077a555d52a5a84e4f6895f/tickflow-0.1.24-py3-none-any.whl", hash = "sha256:e898867b0e3e668618135c78e3a367542f81b7a289567335d298c707452e5f42", size = 43031, upload-time = "2026-06-20T02:39:34.307Z" }, + { url = "https://files.pythonhosted.org/packages/ea/37/f80b8f6e435f1825ea8384b1605a01c323998f00b1c112c59e72a9d7dd9a/tickflow-0.1.25-py3-none-any.whl", hash = "sha256:69687235e85b44eae077262325ddef15e82d5adafda075945612f8cf0815a44c", size = 43510, upload-time = "2026-08-29T04:43:25.785Z" }, ] [package.optional-dependencies] @@ -2513,7 +2513,7 @@ all = [ [[package]] name = "tickflow-stock-panel-backend" -version = "0.2.1" +version = "0.2.2" source = { editable = "." } dependencies = [ { name = "apscheduler" }, diff --git a/docs/custom-data-source.md b/docs/custom-data-source.md index e4cc73d..104d701 100644 --- a/docs/custom-data-source.md +++ b/docs/custom-data-source.md @@ -125,7 +125,23 @@ datasets: 建议实时接口额外提供 `amount`、`change_pct`、`change_amount`、`amplitude`、`turnover_rate`、`name`。缺失时部分字段会由 pipeline 回算,但精度取决于可用输入。 -`change_pct` 和 `amplitude` 使用小数制,例如 `0.0366` 表示 `3.66%`(`turnover_rate` 同)。若接口直接返回百分数值 `3.66`,实时行情会按截面中位数自动归一为小数制,但仍建议接口直接提供小数制以避免小样本歧义。 +`change_pct`、`amplitude`、`turnover_rate` 统一使用小数制,例如 `0.0366` 表示 `3.66%`。百分制单位必须在 realtime 数据集上**显式声明**,不做数值猜测(数值无法区分两种单位:`0.05` 既可能是 0.05% 也可能是 5%): + +```yaml +datasets: + realtime: + url: https://api.example.com/snapshot + pct_unit: percent # 接口返回 3.66 表示 3.66%;小数制源声明 decimal 或省略 +``` + +处理规则: + +| 声明 | 行为 | +| --- | --- | +| `pct_unit: percent` | `change_pct` / `amplitude` / `turnover_rate` 无条件 `/100` | +| `pct_unit: decimal` | 三列原样透传 | +| 未声明 | `change_pct` 按截面中位数归一(A 股涨跌停 30% 上限使两种单位物理可分);`amplitude` / `turnover_rate` **置 `None`** 交由 pipeline 按价格与股本口径重算,并记录 WARNING | +| 列已配置 `transforms` | 视为用户已接管该列单位,原样透传 | ## 请求约定 @@ -210,7 +226,7 @@ cp docs/examples/custom-data-source/mock_source.yaml data/data_sources/mock_sour 5. 保存数据源选择: - 日K: `mock_source` -- 除权因子: `same_as_daily` 或 `mock_source` +- 除权因子: `mock_source` (或保持默认 `tickflow`) - 实时行情: `mock_source` 6. 触发同步或开启实时行情。 @@ -273,12 +289,16 @@ cp docs/examples/custom-data-source/mock_source.yaml data/data_sources/mock_sour amount = 成交额 change_pct = 涨跌幅 (小数, 0.0366 = 3.66%) change_amount = 涨跌额 - amplitude = 振幅 - turnover_rate = 换手率 (小数, 0.05 = 5%; 若上游返回 5 表示 5%, 配置 transforms: turnover_rate: "value / 100") + amplitude = 振幅 (小数, 0.024 = 2.4%) + turnover_rate = 换手率 (小数, 0.05 = 5%) + # 上游若返回百分数值 (3.66 表示 3.66%), 在 realtime 数据集声明 pct_unit: percent, + # 不要依赖数值自动识别; 逐列转换也可用 transforms: turnover_rate: "value / 100" 分钟K (minute): symbol = 股票代码 - datetime = 时间戳 (YYYY-MM-DD HH:MM:SS) + # datetime 必须是北京时间墙钟 (如 2026-08-28 09:35:00), 不要返回 UTC; + # 入口守卫会自动纠偏 UTC 特征帧, 但契约仍要求源头写对 + datetime = 北京时间墙钟 (YYYY-MM-DD HH:MM:SS) open / high / low / close = OHLC volume = 成交量 amount = 成交额 diff --git a/docs/features.md b/docs/features.md index 1afbfbe..19d545f 100644 --- a/docs/features.md +++ b/docs/features.md @@ -16,7 +16,7 @@ | 量价 / 涨停 | 量价齐升 · 高换手强势 · 连板股 · 断板反包 · 涨停动量 | | 反转 / 波动 | 超跌反弹 · 超卖反转 · 新低反转 · 低波动龙头 · 回踩 MA20 | -全 A 股一次扫表,Polars 毫秒级返回。选股页点策略卡片即可扫描,结果支持导出。 +全 A 股一次扫表,Polars 毫秒级返回。选股页点策略卡片即可扫描,结果支持导出。**分钟策略**与日线策略共用同一个标的池,按策略声明的周期自动路由执行(分钟策略依赖数据页开启分钟数据落盘)。 **ETF 支持**:选股页顶部可切换 `股票 / ETF`。ETF 复用已算好的 `kline_etf_enriched` 技术指标,仅开放**技术类内置策略**(趋势/量价/反转/波动);依赖涨停信号的策略(连板股、断板反包)为股票专有,ETF 模式下不显示。需先在数据页开启 ETF 拉取(`pipeline_pull_etf`)并跑一次盘后管道。 @@ -26,7 +26,7 @@ ## 📊 指标流水线(Indicators) -原生 Polars 向量化,全 A 股一次扫表落盘 enriched Parquet: +原生 Polars 向量化,全 A 股一次扫表落盘 enriched Parquet(基础行情列),指标在读取时现算并带进程内快照缓存: - **均线 / 趋势**:MA(5-60) · EMA · MACD · 动量 · 布林带 - **震荡 / 波动**:RSI · KDJ · ATR · 年化波动率 · 振幅 @@ -54,6 +54,10 @@ 输出净值曲线 · 夏普 · 最大回撤 · 胜率 · 交易明细。SSE 流式进度支持切页重连,不会丢失回测任务。 +**分钟策略回测**:分钟级策略逐交易日回放信号、以分钟收盘价入场,输出分钟级成交明细,与日线回测同一套 T+1 / 费用约束。 + +**结果沉淀**:回测结果可**导出 CSV**(概要 / 净值曲线 / 交易明细 / 分标的统计四段),并可**保存候选**(完整参数与指标快照);候选在回测页「候选方案」中查看,支持**一键载入复测**,回填全部参数继续研究。 + **因子与策略挖掘**:复用已有日频因子和 matrix-native 策略，通过 T-1 市场环境、相关去重和嵌套样本外验证生成研究候选。任务在 spawn worker 中运行，支持持久 run ID、取消、刷新重连和显式发布；自动周度任务默认关闭且永不自动发布。完整口径见 [因子与策略挖掘](./mining.md)。 **市场阶段与主线**:市场环境页在原 5 档状态之外新增情绪周期阶段(冰点/启动/主升/高潮/退潮/修复,由连板梯队的高度、宽度、晋级率、梯队完整度判定,平均段长约 10 天)与主线识别(概念/行业维度的涨停梯队聚合排名,可配置宽基标签过滤)。完整口径见 [市场阶段与主线识别](./market-phase.md)。 @@ -88,6 +92,20 @@ --- +## 🚨 异动监控(Abnormal Moves) + +独立的异动页,按交易时间线组织为三个 tab,覆盖盘前到多日: + +| Tab | 口径 | +| :--- | :--- | +| 竞价异动(盘前) | fuyao 同花顺盘前风向标:当日竞价强势榜(5-6 只/日,含竞价涨幅与概念标签),附带**当日 / 次日真实收益对照**与**追高风险标记**(高开 ≥5% 追高当日均值显著为负,来自 60 日回测) | +| 盘中异动 | 涨停 / 炸板 / 翘板 / 跌停 / 新高 / 新低 / 放量等当日信号聚合,按信号优先级 + 涨跌幅排序,复用 enriched 信号列,零新增采集 | +| 偏移异动(多日累计) | 交易所异动偏离值口径(按板块分档:主板 3 日 ±20% · 创业板/科创板 ±30% · 北交所 ±40%;10 日 +100%/−50%,30 日 +200%/−70%),实时计算接近度(≥100% 已触发 · ≥70% 边缘 · ≥50% 观察) | + +竞价数据来自 fuyao 数据源(历史按日缓存,当日不缓存);未配置 fuyao key 时竞价 tab 显示引导提示。 + +--- + ## 📈 个股分析(Beta) 以「行情 + 关键价位」为主体的单标的决策页: @@ -102,7 +120,7 @@ - **连板梯队**:实时统计各连板层级(首板 / 2 连板 / 3 连板...)的标的与封单,捕捉市场情绪与题材热度 - **概念涨幅轮动**:基于 ths 概念 / 行业,统计概念板块涨幅与 RPS 轮动,AI 分析资金主线 -- **盘后 AI 复盘**:盘后自动生成市场复盘,可推送至飞书群 +- **盘后 AI 复盘**:盘后自动生成市场复盘,注入**龙虎榜资金动向**(席位净买/机构与游资 Top5)与**盘前风向标**对照,可定时执行、推送飞书群、下载 Markdown;复盘页内亦有龙虎榜卡片(可排序席位明细) --- @@ -110,7 +128,7 @@ ### 数据源插件化 -内置 [TickFlow](https://tickflow.org) 官方 SDK(日 K / 分钟 K / 指数 / 财务 / 实时行情),同时支持接入第三方数据源:YAML 声明自有 HTTP 接口,或开发插件源(参考实现 stock-sdk,Docker 镜像已内置)。五个数据集(daily / adj_factor / realtime / minute / financial)按源声明自由组合,在「设置 → 数据源」切换。 +内置 [TickFlow](https://tickflow.org) 官方 SDK(日 K / 分钟 K / 指数 / 财务 / 实时行情),同时支持接入第三方数据源:YAML 声明自有 HTTP 接口,或开发插件源(参考实现 stock-sdk,Docker 镜像已内置)。多个数据集(日K / 除权因子 / 实时行情 / 分钟K / 五档盘口 / 财务,注册表持续扩展)按源声明自由组合,在「设置 → 数据源」切换。 详见 [custom-data-source.md](./custom-data-source.md) 与 [plugin-development.md](./plugin-development.md)。 diff --git a/docs/plugin-development.md b/docs/plugin-development.md index 9d04a4f..9e1ccc9 100644 --- a/docs/plugin-development.md +++ b/docs/plugin-development.md @@ -1,9 +1,13 @@ # 数据源插件开发指南 -数据源插件是可选的行情数据来源(stock-sdk、akshare 等),作为独立模块放在 -`backend/app/plugins/` 下。用户**手动安装依赖**后才可用(开发模式);不安装完全不影响主功能。 +数据源插件是可选的行情数据来源(fuyao、stock-sdk、akshare 等),作为独立模块放在 +`backend/app/plugins/` 下。services 层(kline_sync / quote_service / financial_sync) +全部通过统一路由点分流:插件声明了某数据集就走插件,未声明自动回退 TickFlow。 +因此**一个合格的插件只需要正确实现契约,不需要改动任何 service / API 代码**; +反过来,插件也必须遵守内部数据契约(单位、代码格式、复权口径),框架不会替你转换。 -> ⚠️ **Docker 默认不打包 stock-sdk**(合规考虑:它抓取第三方财经网站接口,存在版权与反爬风险)。如需在 Docker 中启用,构建时传 `--build-arg INCLUDE_STOCKSDK=1`,使用风险自负。下方"手动安装依赖"适用于开发模式及自定义 Docker 构建。 +> 无代码接入(纯 HTTP YAML 配置)请看 [custom-data-source.md](./custom-data-source.md), +> 两种方式遵循同一套内部数据契约。 ## 快速上手 @@ -13,7 +17,7 @@ backend/app/plugins// ├── plugin.yaml # 清单(必需) ├── provider.py # Provider 实现(必需) -├── ... # 桥接/依赖文件(按需) +├── ... # client/桥接/依赖文件(按需) ``` ### plugin.yaml 字段 @@ -21,16 +25,20 @@ backend/app/plugins// ```yaml name: my_source # 唯一标识, 只允许 [a-z0-9_], 也是 provider name display_name: "我的数据源" # 设置页显示名 -runtime: python # 运行时类型: node | python | none +runtime: none # 运行时类型: node | python | none entry: app.plugins.my_source.provider:MyProvider # provider 类的导入路径 check: app.plugins.my_source.bridge:availability # 可用性检测函数(可选) -datasets: [daily, adj_factor, minute, realtime] # 支持的数据集 +datasets: [realtime] # 支持的数据集: daily/adj_factor/minute/realtime/financial api_key_env: MY_SOURCE_API_KEY # (可选)声明后设置页提供 Key 输入框 hidden: false # (可选)true = 已加载但对设置页隐藏,不注册不展示 description: "数据源描述" install_hint: "pip install xxx" # 未装依赖时显示的安装提示 +homepage: "https://example.com" # (可选)官网/申请地址, 显示在设置页 Key 配置说明中 ``` +只声明真实提供的数据集;未声明的数据集 `provider_has_dataset` 返回 False,自动回退 +TickFlow。不要声明做不了的数据集(粒度含义见下文"能力声明的粒度")。 + #### api_key_env(界面配置 API Key) 声明 `api_key_env` 的插件可以在设置页的数据源卡片中直接填写 Key, 对齐 @@ -49,36 +57,74 @@ TickFlow 的「先探后存」语义: | runtime | 含义 | 典型场景 | |---|---|---| | `python` | 纯 Python 依赖, `pip install` | akshare、tushare | -| `node` | 需要 Node.js 运行时, `npm install` | stock-sdk(Docker 默认不打包,见 [deployment.md](./deployment.md)) | - -> stock-sdk 在 Docker 中默认不打包(合规考虑);如需启用,构建时传 `--build-arg INCLUDE_STOCKSDK=1`,开发模式下需手动 `npm install`。 +| `node` | 需要 Node.js 运行时, `npm install` | stock-sdk | | `none` | 无额外依赖 | 纯 HTTP API 源 | +> ⚠️ stock-sdk 在 Docker 中默认不打包(合规考虑:它抓取第三方财经网站接口,存在版权与 +> 反爬风险)。如需启用,构建时传 `--build-arg INCLUDE_STOCKSDK=1`,使用风险自负。 +> 详见 [deployment.md](./deployment.md)。 + `runtime` 字段当前仅用于 UI 展示, 实际依赖检测由 `check` 函数负责。 ### check 函数 -插件自己负责检测依赖是否已安装。后端启动时会调用此函数: +插件自己负责检测依赖/Key 是否就绪。后端启动时会调用此函数: ```python -# app/plugins/my_source/bridge.py +# app/plugins/my_source/provider.py (或 bridge.py) def availability() -> tuple[bool, str]: """返回 (是否可用, 原因)。不抛异常。""" - try: - import akshare # noqa: F401 - return True, "ok" - except ImportError: - return False, "未安装 akshare, 运行: pip install akshare" + if not get_api_key(): + return False, "未配置 MY_SOURCE_API_KEY(可在设置页数据源卡片中直接填写)" + return True, "ok" ``` - **可用** → 插件注册进路由表, 设置页可切换 -- **不可用** → 设置页显示插件卡片但灰显, 展示 `install_hint` +- **不可用** → 设置页显示插件卡片但灰显, 展示原因/`install_hint` + +## 内部数据契约(所有数据集必须遵守) + +以下口径是全项目红线(详见 CONTRIBUTING §3)。金融数据错误往往不抛异常,而是生成 +**看似合理的错误结果**——单位、代码格式、复权口径错了,页面照样能渲染,只是数字全错。 +插件必须在 provider 内完成适配。 + +### 代码格式 + +- symbol 统一带交易所后缀: `600519.SH` / `000001.SZ` / `300750.SZ`; ETF、指数同格式。 +- 接口返回裸代码(如 `600519`)或异构格式时,在 client 层实测一页并归一,不要直接透传。 + +### 单位制 + +| 字段 | 契约 | 说明 | +| --- | --- | --- | +| `change_pct` | **小数制**, `0.0366` = 3.66% | 接口给百分数(3.66)时必须在 provider 内显式 /100 | +| `turnover_rate`(realtime 入口) | **小数制**, `0.05` = 5% | 下游 enriched 管道统一转百分数值存储 | +| `volume` | 股 | | +| `amount` / `turnover` | 元 | | +| 日K OHLC | **不复权原始价** | 复权由 adj_factor + enriched 管道处理, provider 不得自行复权 | + +### 缺字段与空数据 + +- 接口不提供的字段返回 `None`,禁止"数值小于 1 就乘 100"之类启发式补全——那会掩盖 + 真实的数据错误。 +- 可推导字段按固定口径推导: `change_pct = change_amount / prev_close`(小数制,不乘 100)。 +- 接口结构整体变化(如所有行都识别不出 symbol)要打明确告警日志,不要静默返回空数据。 + +## 能力声明的粒度(重要) + +`datasets` 声明是**数据集级**的,不是资产类型级的:声明了 `realtime`,整个全市场实时 +轮询周期(含指数与 ETF 部分)就全部路由给插件。若你的快照只覆盖 A 股股票: + +- 指数行情自动降级为日线推导值(非实时),不报错; +- ETF 实时计数为 0。 + +这是当前框架的设计行为。要么在数据里尽量覆盖指数/ETF,要么接受降级并在 +`description` 里向用户说明覆盖范围。 ## Provider 接口契约 -Provider 是一个普通 Python 类(无需继承基类), 实现以下方法签名。方法签名对齐 -`GenericHTTPProvider`, 这样 services 层(kline_sync / quote_service 等)的路由逻辑 -零改动即可路由到插件。 +Provider 是普通 Python 类(无需继承基类),方法签名对齐 `GenericHTTPProvider`, +services 层零改动即可路由。只实现已声明数据集对应的方法,其余可缺省。 ```python class MyProvider: @@ -91,22 +137,72 @@ class MyProvider: def close(self) -> None: """清理资源(load_all 重建注册表时会调)。""" - def get_daily(self, symbols, start_time, end_time, asset_type="stock", on_chunk_done=None) -> pl.DataFrame: - """日K: 返回 schema [symbol, date, open, high, low, close, volume, amount]""" + def get_daily(self, symbols, start_time, end_time, asset_type="stock", + on_chunk_done=None) -> pl.DataFrame: + """日K: [symbol, date, open, high, low, close, volume, amount]; 不复权""" - def get_adj_factors(self, symbols, start_time, end_time, asset_type="stock", on_chunk_done=None) -> pl.DataFrame: - """除权因子: 返回 schema [symbol, trade_date, ex_factor]""" + def get_adj_factors(self, symbols, start_time, end_time, asset_type="stock", + on_chunk_done=None) -> pl.DataFrame: + """除权因子: [symbol, trade_date, ex_factor]""" - def get_minute(self, symbols, start_time, end_time, asset_type="stock", on_chunk_done=None, freq="1m") -> pl.DataFrame: - """分钟K: 返回 schema [symbol, datetime, open, high, low, close, volume, amount]""" + def get_minute(self, symbols, start_time, end_time, asset_type="stock", + on_chunk_done=None, freq="1m") -> pl.DataFrame: + """分钟K: [symbol, datetime(北京墙钟), open, high, low, close, volume, amount]""" def get_realtime(self) -> list[dict]: - """全市场实时快照: 返回 list[dict], 每行含 symbol/last_price/prev_close/open/high/low/volume""" + """全市场实时快照 → list[dict]。失败软返回 [], 不抛异常(不阻断轮询线程)。""" + + def get_financials(self, table, symbols, latest_only=False) -> pl.DataFrame: + """财务数据(声明 financial 数据集时实现, table 见 financial_sync 调用)。""" def get_instruments(self, asset_type="stock") -> list[dict]: - """标的维表(可选): 返回 tickflow Instrument 形状的行, 供 instrument_sync 复用 flatten""" + """(可选)标的维表: 返回 tickflow Instrument 形状的行, 供 instrument_sync 复用 flatten""" + + def test_dataset(self, dataset: str, symbols=None) -> dict: + """(强烈建议)设置页"试拉"按钮。 + 返回 {provider, dataset, rows, columns, preview, error?}; 未支持的数据集 + 返回 error 字段说明会回退 TickFlow。""" ``` +### get_minute 的 datetime 时区契约 + +`datetime` 必须是**北京时间墙钟**（naive，如 `2026-08-28 09:35:00`），与日K的 +`date` 语义对齐；不要返回 UTC 或带时区的时间。前端分时图按交易时段时轴 +（09:30–11:30 / 13:00–15:00）映射每根K线，UTC 口径的帧会导致全部点位落在时轴外、 +分时图空白。 + +入口守卫（`kline_sync._enforce_minute_beijing_wallclock`）对所有分钟源强制归一： +带时区 → 自动换算成北京墙钟；naive 但整体呈 UTC 特征（如 01:30）→ 自动 +8 纠偏并 +记日志；完全无法识别的口径 → 拒收并回退 TickFlow。契约仍要求源头写对，守卫只是兜底。 + +可选类属性 `minute_history_days = 5` 声明 1 分钟历史深度（交易日）；未声明视为 +深历史（TickFlow 基准）。浅源（如 stock-sdk 免费分时仅保留最近 5 个交易日）声明后， +个股分时档位自动收窄为可行选项并默认 5 日，深源默认 20 日。 + +### 异常语义 + +| 方法 | 失败行为 | +| --- | --- | +| `get_realtime` | **软失败**: 返回 `[]` + warning 日志, 保证轮询线程不中断 | +| `get_minute` | 抛异常时调用方自动回退 TickFlow 重试 | +| `get_daily` / `get_adj_factors` / `get_financials` | 异常由上层同步流程捕获记录; 无数据返回空 DataFrame | + +### get_realtime 行字段 + +| 字段 | 必需 | 契约 | +| --- | --- | --- | +| `symbol` | ✅ | 标准代码带后缀 | +| `last_price` | ✅ | 最新价 | +| `prev_close` | ✅ | 昨收, 涨跌幅推导基准 | +| `open` / `high` / `low` | ✅ | 当日 OHLC | +| `volume` | ✅ | 股 | +| `amount` | 建议 | 成交额(元) | +| `change_pct` | 建议 | **小数制**; 缺失时下游按 change_amount/prev_close 推导 | +| `change_amount` | 建议 | 涨跌额(元) | +| `timestamp` | 建议 | 毫秒; 优先用服务端时间(行情归属), 缺失退本地时间 | +| `name` | 可选 | 快照无名称时置 None, 下游用标的维表关联 | +| `amplitude` / `turnover_rate` / `session` | 可选 | 缺失置 None, 不启发式伪造; turnover_rate 入口为小数制 | + ### config.datasets 的作用 `provider_has_dataset(name, dataset)` 通过 `dataset in provider.config.datasets` 判断。 @@ -118,14 +214,40 @@ class MyConfig: datasets = {"daily": ..., "realtime": ...} # key 是数据集名, value 任意 ``` +## 限频与性能 + +- realtime 默认 6s 轮询一轮。优先确认服务端单次 limit 上限: fuyao 实测单页 + limit=6000 可一次拉完全市场(~5600 只), 1 请求/轮; 若服务端强制小页, 必须做 + 页间隔/自限速(参考 fuyao 的 0.15s 页间隔兜底), 并建议用户把轮询间隔调大(15-30s)。 +- 分页必须有页数上限(防 count 异常导致死循环)和空页终止条件。 +- 拉取由 fetch 锁串行化, 慢不会并发重叠; 实际刷新周期 = 轮询间隔 + 拉取耗时, + 串行分页的全量快照本身就需要数秒, 不要按"6s 内必须完成"设计。 + +## 测试要求 + +插件 PR 必须带契约测试(CONTRIBUTING §9), **不依赖真实网络与 API Key**——用假 +Client/桥接注入。以 `backend/tests/test_fuyao_provider.py` 为范本, 至少覆盖: + +1. 字段映射与单位转换: 百分数→小数制、volume 股→手、*ms 零点戳时区换算、缺失字段按口径推导、缺失字段置 None 不伪造 +2. 接口响应结构变体: 实测结构 vs 官方文档示例双兼容(供应商文档与实际不一致是常态) +3. 分页: 多页合并、空页终止、页数上限 +4. 软失败: 接口报错返回 []; 整页 schema 变化有告警而非静默空数据 +5. 能力声明: 未声明数据集 `provider_has_dataset` 为 False +6. Key 语义: 先探后存(无效不落盘)、secrets.json > .env 优先级、availability 两态 +7. loader 集成: 清单解析后正确注册(或 hidden 时正确跳过) + +```bash +cd backend && uv run --extra dev python -m pytest tests/test__provider.py -q +uv run --extra dev python -m ruff check app/plugins// tests/test__provider.py +``` + ## 现有插件参考 - **`backend/app/plugins/fuyao/`** — 同花顺官方 REST 数据源(runtime: none, 纯 HTTP 零依赖) - - 当前提供 `realtime`(A 股全市场快照, 分页拉取); Key 在设置页卡片直接配置(先探后存), 或 `.env` 配 `FUYAO_API_KEY` - - `client.py` — httpx 客户端(X-api-key 认证 + 统一信封解包 + 分页) - - `provider.py` — Provider 实现(字段映射、百分数→小数制单位转换、软失败、Key 探测) - - 单位口径注意: 扶摇 `price_change_ratio_pct` 为百分数数值(1.74 = +1.74%), - 内部 `change_pct` 契约为小数制, provider 内显式 / 100(见 CONTRIBUTING §3.1) + - 提供 `realtime`(A 股全市场快照, 分页拉取)、`daily`(原始价日K三档: 近端窗口走 daily-k-10d dump, 深窗口走 daily-k 10 年全量 dump(172MB 一次下载、缓存复用、10d 补尾), 兜底单标的接口按 10 年自动分片)、`adj_factor`(事件 dump + 前收盘价从本地日K dump 一次取齐、缺价标的回退单标的接口, 按交易所公式推导单事件比值, 涨跌停自检; 全市场配价从逐标的 ~13 分钟降为秒级); Key 在设置页卡片直接配置(先探后存), 或 `.env` 配 `FUYAO_API_KEY` + - `client.py` — httpx 客户端(X-api-key 认证 + 统一信封解包 + 分页 + 页间隔限频 + 单标的日K + dump 预签名下载, S3 下载不带 Key 头) + - `provider.py` — Provider 实现(实测/文档双字段名映射、百分数→小数制、volume 股→手、上海零点戳 +8h 时区、dump 按 release 版本缓存、软失败、Key 探测) + - `tests/test_fuyao_provider.py` — 73 个契约测试, 是新插件的测试范本 - **`backend/app/plugins/stocksdk/`** — Node 型插件, 通过 subprocess 桥接调用 stock-sdk - `bridge.py` — Python↔Node 桥接 + availability 检测 - `bridge.mjs` — Node 端(并发池、重试、SDK 解析) @@ -133,9 +255,9 @@ class MyConfig: ## 路由机制(无需关心, 仅参考) -后端启动时, `loader.py` 的 `_load_builtin_plugins()` 扫描 `plugins/` 目录: +后端启动时, `loader.py` 扫描 `plugins/` 目录: 1. 读每个子目录的 `plugin.yaml` -2. 调 `check` 函数检测可用性 +2. `hidden: true` → 跳过(不注册不展示); 否则调 `check` 函数检测可用性 3. 可用 → 动态 import `entry` 指向的 Provider 类 → 注册进 `_PROVIDERS` 4. 不可用 → 记录状态, 设置页显示但不可切换 diff --git a/frontend/package.json b/frontend/package.json index c031875..ad460d1 100644 --- a/frontend/package.json +++ b/frontend/package.json @@ -1,7 +1,7 @@ { "name": "tickflow-stock-panel-frontend", "private": true, - "version": "0.2.1", + "version": "0.2.2", "type": "module", "scripts": { "dev": "vite", diff --git a/frontend/src/components/CandlestickChart.tsx b/frontend/src/components/CandlestickChart.tsx index 9b766fb..6c26dd7 100644 --- a/frontend/src/components/CandlestickChart.tsx +++ b/frontend/src/components/CandlestickChart.tsx @@ -66,6 +66,8 @@ export function CandlestickChart({ data, height = 480 }: Props) { width: el.clientWidth, height, layout: { + // 关闭 TV 角标 (licence 归属改由 README 技术栈外链承担) + attributionLogo: false, background: { color: THEME.background }, textColor: ctRef.current.text, fontFamily: 'JetBrains Mono, monospace', diff --git a/frontend/src/components/DimensionMembersDialog.tsx b/frontend/src/components/DimensionMembersDialog.tsx index 8f72e58..d67ef6f 100644 --- a/frontend/src/components/DimensionMembersDialog.tsx +++ b/frontend/src/components/DimensionMembersDialog.tsx @@ -1,12 +1,22 @@ import { useEffect, useMemo, useRef, useState } from 'react' import { useQuery } from '@tanstack/react-query' import { useVirtualizer } from '@tanstack/react-virtual' -import { Building2, ChevronRight, RefreshCw, Search, Tags, Users, X } from 'lucide-react' +import { Link } from 'react-router-dom' +import { + createChart, + LineStyle, + type IChartApi, + type ISeriesApi, + type LineData, + type Time, +} from 'lightweight-charts' +import { Activity, Building2, ChevronRight, Database, RefreshCw, Search, Tags, Users, X } from 'lucide-react' import { Modal } from '@/components/Modal' import { boardTag } from '@/components/stock-table/primitives' -import { api, type MarketSnapshotRow } from '@/lib/api' +import { api, type DimensionIntradayPoint, type MarketSnapshotRow } from '@/lib/api' import { QK } from '@/lib/queryKeys' import { fmtBigNum, fmtPct, fmtPrice, priceColorClass } from '@/lib/format' +import { useChartTheme } from '@/lib/theme' export type DimensionKind = 'concept' | 'industry' @@ -218,6 +228,16 @@ function DimensionMembersDialogContent({ target, onClose, onStockClick }: Omit

+ {source && ( + + )} +

@@ -300,3 +320,219 @@ function Summary({ label, value, className }: { label: string; value: string | n
) } + +// --------------------------------------------------------------------------- +// 板块分时 (等权): 点击触发 + 60s 轮询续期, 不预计算 +// --------------------------------------------------------------------------- + +const INTRADAY_SECTOR_COLOR: Record = { + concept: '#F97316', + industry: '#0EA5E9', +} +const INTRADAY_MARKET_COLOR = '#94A3B8' +// 横轴伪时间戳基点 (2020-01-01 UTC), 每点 +60s 保持均匀间距 +const INTRADAY_BASE_TS = 1577836800 + +function lastNonNull(points: DimensionIntradayPoint[], key: 'sector' | 'market'): number | null { + for (let i = points.length - 1; i >= 0; i--) { + const value = points[i]?.[key] + if (value != null) return value + } + return null +} + +function DimensionIntradaySection({ configId, field, value, date, kind }: { + configId: string + field: string + value: string + date?: string + kind: DimensionKind +}) { + const query = useQuery({ + queryKey: QK.dimensionIntraday(configId, field, value, date), + queryFn: () => api.dimensionIntraday(configId, { field, value, date }), + staleTime: 15_000, + refetchInterval: 60_000, + }) + const data = query.data + + return ( +
+
+ + 分时走势 · 等权 + {data?.member_count != null && data.members_with_minute != null && ( + + {data.members_with_minute}/{data.member_count}只 + + )} + {data?.basis && data.basis !== 'prev_close' && ( + + 基准:当日首价 + + )} + {data?.status === 'ok' && ( +
+ + + 板块 + + {fmtPct(lastNonNull(data.points, 'sector'))} + + + + + 全市场 + + {fmtPct(lastNonNull(data.points, 'market'))} + + + {data.date && {data.date}} +
+ )} +
+ + {query.isLoading ? ( +
+ ) : query.isError ? ( +
+ 分时加载失败:{String((query.error as Error).message)} +
+ ) : data?.status === 'no_data' ? ( +
+ +

分钟数据未落盘, 暂无分时走势

+

需 TickFlow Pro+ 盘后分钟同步 / Expert 盘中增量, 或自定义分钟源

+ 前往数据页 → +
+ ) : !data || data.status === 'empty' || data.points.length < 2 ? ( +
+ {data?.reason === 'no_member_bars' ? '成分股当日无分钟数据 (ETF 等标的无分钟落盘)' : '暂无成分股分时数据'} +
+ ) : ( + + )} +
+ ) +} + +function IntradayChart({ points, kind }: { points: DimensionIntradayPoint[]; kind: DimensionKind }) { + const containerRef = useRef(null) + const chartRef = useRef(null) + const sectorRef = useRef | null>(null) + const marketRef = useRef | null>(null) + const ct = useChartTheme() + const ctRef = useRef(ct) + ctRef.current = ct + // v4 不支持字符串时间: 用均匀伪时间戳作横轴, 标签经 formatter 映射回 HH:MM + const labelsRef = useRef([]) + const labelAt = (time: number) => labelsRef.current[time - INTRADAY_BASE_TS] ?? '' + + useEffect(() => { + const el = containerRef.current + if (!el) return + + const chart = createChart(el, { + width: el.clientWidth, + height: 132, + layout: { + // 关闭 TV 角标 (licence 归属改由 README 技术栈外链承担) + attributionLogo: false, + background: { color: 'transparent' }, + textColor: ctRef.current.text, + fontFamily: 'JetBrains Mono, monospace', + fontSize: 10, + }, + grid: { + vertLines: { color: ctRef.current.grid }, + horzLines: { color: ctRef.current.grid }, + }, + rightPriceScale: { borderColor: ctRef.current.border, scaleMargins: { top: 0.12, bottom: 0.04 } }, + timeScale: { + borderColor: ctRef.current.border, + rightOffset: 2, + barSpacing: 4, + tickMarkFormatter: (time: number) => labelAt(time), + }, + localization: { + timeFormatter: (time: number) => labelAt(time), + }, + crosshair: { + vertLine: { labelVisible: false }, + horzLine: { labelVisible: true }, + }, + handleScroll: false, + handleScale: false, + }) + const sector = chart.addLineSeries({ + color: INTRADAY_SECTOR_COLOR[kind], + lineWidth: 2, + priceLineVisible: false, + lastValueVisible: true, + priceFormat: { type: 'custom', formatter: (v: number) => `${(v * 100).toFixed(2)}%`, minMove: 0.0001 }, + crosshairMarkerRadius: 3, + }) + const market = chart.addLineSeries({ + color: INTRADAY_MARKET_COLOR, + lineWidth: 1, + lineStyle: LineStyle.Dashed, + priceLineVisible: false, + lastValueVisible: false, + crosshairMarkerRadius: 2, + }) + sector.createPriceLine({ + price: 0, + color: ctRef.current.border, + lineWidth: 1, + lineStyle: LineStyle.Dashed, + axisLabelVisible: false, + }) + chartRef.current = chart + sectorRef.current = sector + marketRef.current = market + + const observer = new ResizeObserver(() => { + chart.applyOptions({ width: el.clientWidth }) + }) + observer.observe(el) + return () => { + observer.disconnect() + chart.remove() + chartRef.current = null + sectorRef.current = null + marketRef.current = null + } + }, [kind]) + + useEffect(() => { + chartRef.current?.applyOptions({ + layout: { textColor: ct.text }, + grid: { vertLines: { color: ct.grid }, horzLines: { color: ct.grid } }, + rightPriceScale: { borderColor: ct.border }, + timeScale: { borderColor: ct.border }, + }) + }, [ct]) + + useEffect(() => { + const sectorSeries = sectorRef.current + const marketSeries = marketRef.current + if (!sectorSeries || !marketSeries) return + labelsRef.current = points.map(p => p.time) + const toData = (key: 'sector' | 'market'): LineData[] => + points + .map((p, i) => ({ time: (INTRADAY_BASE_TS + i * 60) as Time, value: p[key] })) + .filter((d): d is LineData => d.value != null) + sectorSeries.setData(toData('sector')) + marketSeries.setData(toData('market')) + chartRef.current?.timeScale().fitContent() + }, [points]) + + return ( +
+
+
+ ) +} diff --git a/frontend/src/components/EChartsIntraday.tsx b/frontend/src/components/EChartsIntraday.tsx index ad86406..dae6b00 100644 --- a/frontend/src/components/EChartsIntraday.tsx +++ b/frontend/src/components/EChartsIntraday.tsx @@ -72,6 +72,9 @@ function buildOption(data: MinuteKlineRow[], prevClose: number | undefined, avgP const volumes = new Array(FULL_DAY_TIMES.length).fill(null) as (any | null)[] const volNeutral = 'rgba(161,161,170,0.5)' + // 量柱着色基准: 前一分钟 close; 第一根用昨收。 + // 不用 row.open — stock-sdk 历史日无真实分钟 open(为 null), close-vs-open 会全偏。 + let prevRef: number | null = prevClose ?? null for (let i = 0; i < data.length; i++) { const timeKey = formatMinuteTime(data[i].datetime) const idx = timeIndexMap.get(timeKey) @@ -83,9 +86,16 @@ function buildOption(data: MinuteKlineRow[], prevClose: number | undefined, avgP volumes[idx] = { value: data[i].volume, itemStyle: { - color: data[i].close > data[i].open ? THEME.volUp : data[i].close < data[i].open ? THEME.volDown : volNeutral, + color: prevRef == null + ? volNeutral + : data[i].close > prevRef + ? THEME.volUp + : data[i].close < prevRef + ? THEME.volDown + : volNeutral, }, } + prevRef = data[i].close } } @@ -570,7 +580,7 @@ export function EChartsIntraday({ <> {date && {date}} 开 - {d.open.toFixed(2)} + {d.open != null ? d.open.toFixed(2) : '—'} 高 {d.high.toFixed(2)} 低 diff --git a/frontend/src/components/EChartsMultiDayIntraday.tsx b/frontend/src/components/EChartsMultiDayIntraday.tsx index 63e0cf5..1db4dc3 100644 --- a/frontend/src/components/EChartsMultiDayIntraday.tsx +++ b/frontend/src/components/EChartsMultiDayIntraday.tsx @@ -74,6 +74,9 @@ function buildModel(sessions: MinuteKlineSession[]) { const dayValues: (number | null)[] = [] const dayAverages: (number | null)[] = [] + // 量柱着色基准: 前一分钟 close; 当日第一根用 session 昨收。 + // 不用 row.open — stock-sdk 历史日无真实分钟 open(为 null), close-vs-open 会全偏。 + let prevRef: number | null = session.prev_close for (const time of FULL_DAY_TIMES) { const point = rowsByTime.get(time) const index = categories.length @@ -91,13 +94,16 @@ function buildModel(sessions: MinuteKlineSession[]) { volumeData.push({ value: row.volume, itemStyle: { - color: row.close > row.open - ? COLORS.volumeUp - : row.close < row.open - ? COLORS.volumeDown - : COLORS.volumeFlat, + color: prevRef == null + ? COLORS.volumeFlat + : row.close > prevRef + ? COLORS.volumeUp + : row.close < prevRef + ? COLORS.volumeDown + : COLORS.volumeFlat, }, }) + prevRef = row.close priceValues.push(row.low, row.high, average) pointByIndex.set(index, { date: session.date, @@ -408,7 +414,7 @@ export function EChartsMultiDayIntraday({ {info ? ( <> {info.date} {formatMinuteTime(info.row.datetime)} - 开{info.row.open.toFixed(2)} + 开{info.row.open != null ? info.row.open.toFixed(2) : '—'} 高{info.row.high.toFixed(2)} 低{info.row.low.toFixed(2)} 收{info.row.close.toFixed(2)} diff --git a/frontend/src/components/Layout.tsx b/frontend/src/components/Layout.tsx index 2d68a6d..bc34732 100644 --- a/frontend/src/components/Layout.tsx +++ b/frontend/src/components/Layout.tsx @@ -1,4 +1,4 @@ -import { useEffect, useMemo, useRef, useState, Suspense } from 'react' +import { useEffect, useLayoutEffect, useMemo, useRef, useState, Suspense } from 'react' import { NavLink, Outlet, useNavigate, useLocation } from 'react-router-dom' import { useQuery, useQueryClient } from '@tanstack/react-query' import { motion } from 'framer-motion' @@ -10,7 +10,7 @@ import { AiReportBubble } from '@/components/financials/AiReportBubble' import { StockAnalysisHost } from '@/components/stock-analysis/StockAnalysisHost' import { StockAnalysisBubble } from '@/components/stock-analysis/StockAnalysisBubble' import { - useCapabilities, + useCapabilityMatrix, useSettings, usePreferences, useQuoteStatus, @@ -53,7 +53,7 @@ import { PanelLeftOpen, } from 'lucide-react' import { Logo } from './Logo' -import { api, type IndexQuote } from '@/lib/api' +import { api, type CapabilityMatrix, type IndexQuote } from '@/lib/api' import { cn } from '@/lib/cn' import { resolveWatchlistGroupColor } from '@/lib/watchlist-group-colors' import { computeGroupPcts, groupPctColor, groupPctTitle } from '@/lib/watchlistGroupStats' @@ -171,80 +171,140 @@ function SidebarIndexQuotes({ rows, items }: { rows: IndexQuote[] | undefined; i ) } -// ===== 档位卡片 ===== -function TierBadge({ label, hasKey, providerName, isTickflow }: { label: string; hasKey?: boolean; providerName: string; isTickflow: boolean }) { - const base = label.split(' ')[0].split('+')[0].toLowerCase() - const isNone = base === 'none' +// ===== 数据源能力健康卡 ===== +// 能力路由架构下的侧栏状态: 不再展示「主数据源 + TickFlow 档位」(单源时代遗留 — +// 五个能力各自路由, 拿日K的源代表全局是随意的), 改为回答「各能力当前是否都有源在供」。 +// 档位/订阅信息归设置页 TickFlow 介绍卡 (档位词仅出现在 TickFlow 专属界面的设计规则)。 +// 单能力方格: 可用=绿 / 日K缺失=红 / 其他缺失=琥珀 (与悬浮卡中同色, 一眼对应) +function capSquareCls(c: { id: string; usable: boolean }) { + return c.usable ? 'bg-accent' : c.id === 'daily' ? 'bg-danger' : 'bg-warning/80' +} - const tierConfig: Record = { - none: { - desc: '未配置 Key · 仅历史日K', - dotStyle: { background: '#52525b' }, - tagBg: { background: 'rgba(113,113,122,0.15)' }, - labelTextStyle: { color: '#71717a' }, - }, - free: { - desc: '基础日K · 自选实时', - dotStyle: { background: '#71717a' }, - tagBg: { background: 'rgba(113,113,122,0.3)' }, - labelTextStyle: { color: '#a1a1aa' }, - }, - starter: { - desc: '批量同步 · 行情池', - dotStyle: { background: '#3b82f6' }, - tagBg: { background: 'rgba(59,130,246,0.2)' }, - labelTextStyle: { color: '#60a5fa' }, - }, - pro: { - desc: '分钟K · 实时行情 · 盘口', - dotStyle: { background: 'linear-gradient(135deg, #a855f7, #7c3aed)' }, - tagBg: { background: 'linear-gradient(135deg, rgba(168,85,247,0.2), rgba(124,58,237,0.15))' }, - labelTextStyle: { background: 'linear-gradient(135deg, #c084fc, #a855f7)', WebkitBackgroundClip: 'text', backgroundClip: 'text', color: 'transparent' }, - }, - expert: { - desc: 'WebSocket · 财务数据', - dotStyle: { background: 'linear-gradient(135deg, #3b82f6, #a855f7, #f59e0b)' }, - tagBg: { background: 'linear-gradient(135deg, rgba(59,130,246,0.2), rgba(168,85,247,0.2), rgba(245,158,11,0.2))' }, - labelTextStyle: { background: 'linear-gradient(135deg, #60a5fa, #c084fc, #fbbf24)', WebkitBackgroundClip: 'text', backgroundClip: 'text', color: 'transparent' }, - }, +function DataSourceHealthBadge({ matrix }: { matrix: CapabilityMatrix | undefined }) { + const caps = matrix?.capabilities ?? [] + const loading = caps.length === 0 + const usableCount = caps.filter(c => c.usable).length + const down = caps.filter(c => !c.usable) + // 日K是核心能力 (其他一切派生于它): 挂了用危险色; 一般缺项琥珀; 全可用绿 + const level = loading + ? 'loading' + : down.length === 0 ? 'ok' : down.some(c => c.id === 'daily') ? 'danger' : 'warn' + const countCls = level === 'ok' ? 'text-accent/80' + : level === 'danger' ? 'text-danger' + : level === 'warn' ? 'text-warning' + : 'text-muted' + + // 悬浮卡: 侧栏 aside 是 overflow-hidden, 用 fixed 定位逃逸裁剪 (坐标取自徽标实时位置)。 + // 徽标靠近屏幕顶部时居中定位会把卡片上半截推出视口 → 渲染后按实际高度钳制进视口。 + const linkRef = useRef(null) + const popRef = useRef(null) + const closeTimer = useRef(undefined) + const [popPos, setPopPos] = useState<{ left: number; top: number } | null>(null) + const openPop = () => { + window.clearTimeout(closeTimer.current) + const rect = linkRef.current?.getBoundingClientRect() + if (rect) setPopPos({ left: rect.right, top: rect.top + rect.height / 2 }) } - - const t = tierConfig[base] || tierConfig.none - const displayLabel = isNone ? 'None' : (label || 'None') - const descText = isNone && !hasKey ? '配置 Key 解锁更多能力' : t.desc + const closePop = () => { + closeTimer.current = window.setTimeout(() => setPopPos(null), 80) + } + useEffect(() => () => window.clearTimeout(closeTimer.current), []) + useLayoutEffect(() => { + if (!popPos || !popRef.current) return + const h = popRef.current.offsetHeight + const margin = 8 + const minCenter = margin + h / 2 + const maxCenter = window.innerHeight - margin - h / 2 + const clamped = Math.min(maxCenter, Math.max(minCenter, popPos.top)) + if (clamped !== popPos.top) setPopPos({ ...popPos, top: clamped }) + }, [popPos]) return ( - - - - - {providerName || '数据源'} - - - {isTickflow && ( - - {displayLabel} + <> + { if (e.key === 'Escape') setPopPos(null) }} + className="group relative flex items-center gap-2 overflow-hidden rounded-md py-1.5 pl-2.5 pr-2 transition-colors duration-150 hover:bg-elevated/70" + > + + + {/* 能力方格 (按注册顺序: 实时/日K/分钟/除权/财务), 与悬浮卡逐格同色对应 */} + + {loading + ? Array.from({ length: 5 }, (_, i) => ( + + )) + : caps.map(c => ( + + ))} + {!loading && ( + + {usableCount}/{caps.length} + + )} + + {popPos && ( +
window.clearTimeout(closeTimer.current)} + onMouseLeave={closePop} + > + +
+ + + 数据源能力 + + + {loading ? '获取中…' : `${usableCount}/${caps.length} 可用`} + +
+
+ {loading ? ( +
正在获取能力路由状态…
+ ) : caps.map(c => ( +
+ + {c.label} + + {c.usable ? ( + <> + {c.effective_display} + + + ) : ( + 未接入 + )} + +
+ ))} +
+ {/* 分时有分钟K功能替身 (intraday_monitor_support 三路可达), 不单独占能力格, 在此备注 */} +
+ 分时信号监控可由分钟 K 数据驱动，不单独设能力格 +
+
+ 点击前往数据源配置 + +
+ +
)} - + ) } @@ -275,8 +335,8 @@ function AIConfigBadge({ configured, model }: { configured?: boolean; model?: st export function Layout() { // ===== 共享 hooks (替代内联 useQuery) ===== - const { data: caps } = useCapabilities() const { data: settingsState } = useSettings() + const { data: matrix } = useCapabilityMatrix() const { data: versionData } = useVersion() const { data: prefs } = usePreferences() // 数据源列表 (用于实时行情状态显示当前数据源名称) @@ -446,13 +506,6 @@ export function Layout() { ? '关闭实时行情' : '开启实时行情' - // 当前主数据源 (用于侧边栏数据源状态卡) - const activeProvider = prefs?.daily_data_provider || 'tickflow' - const activeProviderName = activeProvider === 'tickflow' - ? 'TickFlow' - : (dataSources?.custom?.find(s => s.name === activeProvider)?.display_name || activeProvider) - const isCustomActive = activeProvider !== 'tickflow' - // 轮询触发记录总数 → 更新监控中心徽标 (每 15 秒; 后台标签页由 SSE 事件驱动, 不轮询) const alertsTotalQuery = useQuery({ queryKey: ['alerts-total'], @@ -522,10 +575,6 @@ export function Layout() { toast('当前数据源无实时行情能力, 请先配置数据源', 'error') return } - if (fresh.mode === 'watchlist' && (prefs?.realtime_watchlist_symbols?.length ?? 0) === 0) { - navigate('/watchlist') - return - } } await toggleQuote.mutateAsync(enabled) // 仅在交易时段立即获取一次行情 @@ -572,15 +621,10 @@ export function Layout() {
- {/* 状态卡 — 收起时隐藏 */} - {!navCollapsed && ( -
- + {/* 状态卡 — 收起时隐藏 */} + {!navCollapsed && ( +
+
)} + {draft.type === 'volume_delta' && ( +
+
+
+ 阈值口径 +
+ {([['volume', '按手数'], ['amount', '按金额']] as const).map(([key, label]) => ( + + ))} +
+
+ +
+ +
+ 基础过滤 (与策略选股口径对齐, 留空不过滤) +
+ + + + + + +
+
+ +
+ 捕捉单次轮询间隔内的突发放量 (大单连续扫货)。开盘首轮与暂停恢复后的第一轮不触发, + 防止集合竞价撮合量误报; 冷却期内同一标的不重复提醒, 命中超过 5 只时合并为一条批量通知。 +
+
+ )} + {/* 作用范围 */} {draft.type !== 'sector' &&
作用范围 @@ -1106,7 +1247,7 @@ export function RuleEditor({ rule, preset, simple, onClose, onSaved }: Props) {
} {/* 触发条件 (非 strategy) */} - {draft.type !== 'strategy' && draft.type !== 'sector' && draft.type !== 'abnormal' && ( + {draft.type !== 'strategy' && draft.type !== 'sector' && draft.type !== 'abnormal' && draft.type !== 'volume_delta' && (
触发条件 @@ -1402,7 +1543,7 @@ export function RuleEditor({ rule, preset, simple, onClose, onSaved }: Props) { return (

{unconfigured.join('、')}尚未配置, - 前往设置页配置 → + 前往设置页配置 →

) })()} diff --git a/frontend/src/components/screener/StrategyCard.tsx b/frontend/src/components/screener/StrategyCard.tsx index f2efbff..7a9a580 100644 --- a/frontend/src/components/screener/StrategyCard.tsx +++ b/frontend/src/components/screener/StrategyCard.tsx @@ -92,12 +92,14 @@ interface StrategyCardProps { monitored?: boolean /** 切换策略监控 (点击 RadioTower 图标) */ onToggleMonitor?: () => void + /** 周期徽章 (如 '分钟'); 日线策略不传 */ + timeframeBadge?: string } export function StrategyCard({ name, description, source, active, count, expiredCount, loading, cardSize, - onRun, disabled, onSettings, monitored, onToggleMonitor, + onRun, disabled, onSettings, monitored, onToggleMonitor, timeframeBadge, }: StrategyCardProps) { const cs = CARD_STYLES[cardSize] const activeCls = active @@ -125,6 +127,9 @@ export function StrategyCard({ className="flex flex-col items-start cursor-pointer disabled:opacity-50 disabled:cursor-wait w-full">
{srcLabel} + {timeframeBadge && ( + {timeframeBadge} + )} {name}
{description && ( @@ -164,6 +169,9 @@ export function StrategyCard({ className="flex flex-col items-start cursor-pointer disabled:opacity-50 disabled:cursor-wait min-w-0">
{srcLabel} + {timeframeBadge && ( + {timeframeBadge} + )} {name} {count != null && !loading && ( {count} diff --git a/frontend/src/components/screener/StrategyPoolDialog.tsx b/frontend/src/components/screener/StrategyPoolDialog.tsx index d6dcc21..31f6b47 100644 --- a/frontend/src/components/screener/StrategyPoolDialog.tsx +++ b/frontend/src/components/screener/StrategyPoolDialog.tsx @@ -24,6 +24,8 @@ const SOURCE_LABEL: Record = { invalid: '失效', } +const TF_BADGE_CLS = 'text-[8px] px-1 py-px rounded border leading-tight shrink-0 border-sky-500/30 bg-sky-500/10 text-sky-400' + type SourceTab = 'all' | 'builtin' | 'custom' | 'ai' const TABS: { id: SourceTab; label: string }[] = [ @@ -57,7 +59,8 @@ export function StrategyPoolDialog({ pool, onConfirm, onClose }: Props) { const loadStrategies = useCallback(async () => { setLoading(true) try { - const d = await api.strategyList() + // 不按周期过滤: 日线+分钟策略合并展示, 分钟策略以徽章区分 + const d = await api.strategyList(undefined, 'all') setAllStrategies(d.strategies) } catch { setAllStrategies([]) @@ -236,6 +239,9 @@ export function StrategyPoolDialog({ pool, onConfirm, onClose }: Props) { {SOURCE_LABEL[s.source] ?? '内置'} + {s.timeframes?.includes('1m') && ( + 分钟 + )} ))} @@ -280,6 +286,9 @@ export function StrategyPoolDialog({ pool, onConfirm, onClose }: Props) { {SOURCE_LABEL[src] ?? '内置'} + {s?.timeframes?.includes('1m') && ( + 分钟 + )} + ) + })} +
+ {TAB_META.find(t => t.key === tab)?.desc} +
+ +
+ {tab === 'auction' && ( + setPreview({ symbol: s, name: n ?? s })} /> + )} + {tab === 'intraday' && ( + setPreview({ symbol: r.symbol, name: r.name ?? r.symbol })} /> + )} + {tab === 'deviation' && ( + setPreview({ symbol: r.symbol, name: r.name ?? r.symbol })} /> + )} +
+ + {preview && ( + setPreview(null)} + /> + )} +
+ ) +} + +// ================================================================ +// 竞价异动 tab +// ================================================================ + +/** 追高风险阈值: 60日回测高开≥5%子集当日开盘买 -1.97% (温和高开才是名单 alpha 来源) */ +const _BENCH_CHASE_RISK_PCT = 5 + +function AuctionView({ onOpenStock }: { + onOpenStock: (symbol: string, name?: string | null) => void +}) { + const q = useQuery({ + queryKey: ['auction-benchmark', 'latest'], + queryFn: () => api.auctionBenchmark(), + staleTime: 5 * 60_000, + retry: 1, + }) + + // fuyao 未配置: 整个 tab 的统一引导态 (风向标与全市场扫描都依赖 fuyao), + // 不再展示零散的降级卡/占位卡 — 与偏移 tab「监控未开启」空态同款式 + if (q.data?.state === 'source_unavailable') { + return ( +
+
+ + + +
竞价数据源未配置
+

+ 竞价异动 (同花顺盘前风向标与全市场竞价扫描) 依赖 fuyao 数据源, + 复盘页的龙虎榜同样来自该数据源。在「设置 → 数据源」配置 fuyao API Key 后即可使用。 +

+ + 前往配置数据源 + + +
+
+ ) + } + + return ( +
+ + + {/* 全市场竞价扫描: 采集任务启用后填充 (接口与批量能力已验证) */} +
+
+ + + +
+
+ 全市场竞价扫描 + + 待采集任务启用 + +
+

+ 9:25 竞价终态后扫描全市场 (实测 5547 只约 2 秒), 自动筛出高开 ≥5% 且竞价量比 ≥10 + 的标的并按日落盘积累历史。竞价明细无历史接口, 数据从采集启用之日起积累。 +

+
+
+
+ +

+ 风向标为同花顺盘前竞价筛选名单 (每日约 5~6 只)。60 日回测: 名单当日开盘买入均值 +0.54% + (超额 +0.44%), 但高开 ≥5% 子集当日 -1.97% — 追高是陷阱, 次日无显著优势, 仅作当日观察。 +

+
+ ) +} + +function BenchmarkCard({ q, onOpenStock }: { + q: UseQueryResult + onOpenStock: (symbol: string, name?: string | null) => void +}) { + const d = q.data + + if (q.isLoading) { + return ( +
+ + + +
+ + + +
+
+ ) + } + + // source_unavailable (fuyao 未配置) 由 AuctionView 统一引导态处理, 此处不再分支 + + if (!d || d.state === 'no_data') { + return ( +
+ + + + 盘前风向标暂不可用{d?.message ? ` (${d.message.slice(0, 40)})` : ''} + +
+ ) + } + + const items = d.items ?? [] + const isFallback = d.state === 'fallback_prev' + const ocs = items.map(i => i.day0_oc).filter((v): v is number => v != null) + const avgOc = ocs.length ? ocs.reduce((a, b) => a + b, 0) / ocs.length : null + + return ( +
+ {/* 头部 */} +
+ + + + + + 盘前风向标 + {isFallback && ( + + 显示上一期 + + )} + + + {d.trade_date} · {items.length} 只 · 同花顺竞价筛选 + {avgOc != null && ( + <> · 当日开盘买均值 {fmtPct(avgOc)} + )} + + + + 次日无优势
仅当日观察 + +
+ + {/* 名单 */} + {items.length === 0 ? ( +

本期无名单数据

+ ) : ( +
+
+ 竞价 + 股票 + 当日 + 次日 +
+ {items.map((i: AuctionBenchmarkItem) => { + const gap = i.auction_pct ?? null + const chase = (gap ?? 0) >= _BENCH_CHASE_RISK_PCT + return ( + + ) + })} +
+ )} +
+ ) +} + +// ================================================================ +// 盘中异动 tab +// ================================================================ + +function IntradayView({ onPreview }: { + onPreview: (r: AbnormalIntradayRow) => void +}) { + const [sigFilter, setSigFilter] = useState<'all' | IntradaySignalKey>('all') + const [boardFilter, setBoardFilter] = useState<'all' | (typeof BOARDS)[number]>('all') + const [query, setQuery] = useState('') + const [excludeSt, setExcludeSt] = useState(true) + + const q = useQuery({ + queryKey: QK.abnormalIntraday(500), + queryFn: () => api.abnormalIntraday(500), + refetchInterval: REFRESH_MS, + }) + const data = q.data + const counts = data?.counts ?? {} + + const rows = useMemo(() => { + let list = data?.rows ?? [] + if (sigFilter !== 'all') list = list.filter(r => r.signals.includes(sigFilter)) + if (boardFilter !== 'all') { + // boardTag: 创/科/北有徽章, 主板返回 null + const want = boardFilter === '主板' ? null + : boardFilter === '创业板' ? '创' + : boardFilter === '科创板' ? '科' : '北' + list = list.filter(r => (boardTag(r.symbol)?.label ?? null) === want) + } + if (excludeSt) list = list.filter(r => !(r.name ?? '').toUpperCase().includes('ST')) + const s = query.trim().toLowerCase() + if (s) list = list.filter(r => `${r.symbol} ${r.name ?? ''}`.toLowerCase().includes(s)) + return list + }, [data, sigFilter, boardFilter, excludeSt, query]) + + const total = (data?.rows ?? []).length + + return ( +
+ {/* 信号筛选 chips (带各类型计数) + 工具行 */} +
+ setSigFilter('all')} label="全部" count={total} /> + {SIGNAL_KEYS.map(k => ( + setSigFilter(k)} + label={SIGNAL_META[k].label} + count={counts[k] ?? 0} + cls={SIGNAL_META[k].cls} + /> + ))} + + 数据截至 {data?.cache_date ?? '—'} + {q.isFetching && ' · 更新中…'} + +
+ setBoardFilter(v)} + options={[ + { value: 'all' as const, label: '全板块' }, + ...BOARDS.map(b => ({ value: b, label: b })), + ]} + /> + + +
+ + setQuery(e.target.value)} + placeholder="搜索代码/名称" + className="h-7 w-40 rounded border border-border bg-base pl-7 pr-2 text-[11px] text-foreground" + /> +
+
+
+ + {/* 主表 */} +
+ + + + + + + + + + + + + + + {q.isLoading ? ( + + ) : rows.length === 0 ? ( + + ) : ( + rows.map((r, i) => ( + onPreview(r)} /> + )) + )} + +
#代码 / 名称现价今日信号量比振幅换手
正在加载盘中信号…
{data ? '当前筛选下没有命中标的' : '暂无数据'}
+
+
+ ) +} + +function SigChip({ active, onClick, label, count, cls }: { + active: boolean + onClick: () => void + label: string + count: number + cls?: string +}) { + return ( + + ) +} + +function IntradayRowView({ row, rank, onPreview }: { + row: AbnormalIntradayRow + rank: number + onPreview: () => void +}) { + const board = boardTag(row.symbol) + const clu = row.consecutive_limit_ups ?? 0 + return ( + + {rank} + + + + {fmtPrice(row.close)} + + {fmtPct(row.change_pct)} + + +
+ {row.signals.map(s => ( + + {SIGNAL_META[s].label} + + ))} +
+ + + {row.vol_ratio_5d != null ? row.vol_ratio_5d.toFixed(2) : '—'} + + + {row.amplitude != null ? fmtPct(row.amplitude, 2) : '—'} + + + {row.turnover_rate != null ? `${Number(row.turnover_rate).toFixed(2)}%` : '—'} + + + ) +} + +// ================================================================ +// 偏移异动 tab (原有异动边缘监控, 逻辑保持不变) +// ================================================================ + +function DeviationView({ onPreview }: { + onPreview: (r: AbnormalRow) => void +}) { // 主开关: 默认关闭, 开启后才轮询计算 (仅控制本页计算, 后台告警由监控规则驱动) const [enabled, setEnabled] = useState(() => storage.abnormalEnabled.get(false)) - // 规则口径面板 (标题栏「?」) + // 规则口径面板 (工具栏「?」) const [rulesOpen, setRulesOpen] = useState(false) // 上次计算结果: 开启时每次成功计算都落本地, 关闭后仍展示 const [lastResult, setLastResult] = useState( @@ -56,7 +584,6 @@ export function AbnormalMoves() { const [watchlistOnly, setWatchlistOnly] = useState(false) // 默认过滤 ST/*ST 风险警示股票 (口径与后端 is_st_name 一致: 名称含 ST) const [excludeSt, setExcludeSt] = useState(true) - const [preview, setPreview] = useState<{ symbol: string; name: string } | null>(null) const overview = useQuery({ queryKey: QK.abnormalOverview(minCloseness, 300), @@ -124,70 +651,8 @@ export function AbnormalMoves() { const updating = overview.isFetching return ( - // 整页占满视口: 头部/筛选固定, 只有表格列表区滚动 -
-
- - - {enabled && ( - - )} - - - 告警规则 - - {/* 主开关: 开启后才开始轮询计算 */} - -
- } - /> -
-
- - {/* 规则口径面板 (标题栏「?」展开) */} +
+ {/* 规则口径面板 (工具栏「?」展开) */} {rulesOpen && (
@@ -252,7 +717,7 @@ export function AbnormalMoves() {
)} - {/* 统计 + 筛选 */} + {/* 统计 + 控制 */}
@@ -264,6 +729,49 @@ export function AbnormalMoves() { {data ? ` · 基准指数今日 ${(data.bench_rt_pct * 100).toFixed(2)}%` : ''} )} +
+ + {enabled && ( + + )} + {/* 主开关: 开启后才开始轮询计算 */} + +
@@ -377,7 +885,7 @@ export function AbnormalMoves() { key={r.symbol} row={r} rank={i + 1} - onPreview={() => setPreview({ symbol: r.symbol, name: r.name ?? r.symbol })} + onPreview={() => onPreview(r)} /> )) )} @@ -386,15 +894,6 @@ export function AbnormalMoves() {
)} - -
- {preview && ( - setPreview(null)} - /> - )}
) @@ -508,7 +1007,7 @@ function AbnormalRowView({ row, rank, onPreview }: {
diff --git a/frontend/src/pages/Backtest.tsx b/frontend/src/pages/Backtest.tsx index 3b405fd..4994750 100644 --- a/frontend/src/pages/Backtest.tsx +++ b/frontend/src/pages/Backtest.tsx @@ -6,6 +6,7 @@ import { FactorDiscovery } from './backtest/FactorDiscovery' import { ResearchCandidatesDialog } from './backtest/ResearchCandidatesDialog' import { RobustnessValidation } from './backtest/RobustnessValidation' import { StrategyBacktest } from './backtest/StrategyBacktest' +import { type ResearchCandidate } from '@/lib/api' type Tab = 'factor' | 'strategy' | 'robustness' @@ -31,6 +32,8 @@ export function Backtest() { const [searchParams, setSearchParams] = useSearchParams() const requestedTab = searchParams.get('tab') const [candidatesOpen, setCandidatesOpen] = useState(false) + // 候选「载入复测」: 弹窗选定 → 关闭弹窗切到策略页 → StrategyBacktest 消费后清空 + const [pendingLoad, setPendingLoad] = useState(null) // 旧链接兼容: 挖掘已升级为一级路由 /mining, 保留 run/candidate 参数重定向 if (requestedTab === 'mining') { @@ -99,11 +102,25 @@ export function Backtest() {
{activeTab === 'factor' && } - {activeTab === 'strategy' && } + {activeTab === 'strategy' && ( + setPendingLoad(null)} + /> + )} {activeTab === 'robustness' && }
- {candidatesOpen && setCandidatesOpen(false)} />} + {candidatesOpen && ( + setCandidatesOpen(false)} + onLoadStrategy={candidate => { + setPendingLoad(candidate) + setCandidatesOpen(false) + if (activeTab !== 'strategy') changeTab('strategy') + }} + /> + )}
) } diff --git a/frontend/src/pages/Dashboard.tsx b/frontend/src/pages/Dashboard.tsx index 8cbd3ad..98405b0 100644 --- a/frontend/src/pages/Dashboard.tsx +++ b/frontend/src/pages/Dashboard.tsx @@ -7,6 +7,7 @@ import { DatePicker } from '@/components/DatePicker' import { api, type MarketSnapshotRow, type OverviewDimensionRankItem, type OverviewMarket, type AlertEvent } from '@/lib/api' import { QK } from '@/lib/queryKeys' import { fmtBigNum, fmtPct } from '@/lib/format' +import { DimensionMembersDialog, dimensionKindForSourceField, type DimensionMembersTarget } from '@/components/DimensionMembersDialog' import { useDataStatus, useCapabilities, useSettings, usePreferences } from '@/lib/useSharedQueries' import { SealedBadge } from '@/components/SealedBadge' import { StockPreviewDialog } from '@/components/StockPreviewDialog' @@ -465,18 +466,36 @@ function StockList({ title, rows, mode, onStockClick }: { ) } -function RankColumn({ title, rows, tone, onStockClick }: { +function RankColumn({ title, rows, tone, onStockClick, onDimensionClick }: { title: string; rows: OverviewDimensionRankItem[]; tone: 'bull' | 'bear'; - onStockClick?: (symbol: string, name?: string) => void; + onStockClick?: (symbol: string, name?: string) => void + onDimensionClick?: (target: DimensionMembersTarget) => void }) { return (
{title}
- {rows.slice(0, 5).map((r, idx) => ( -
+ {rows.slice(0, 5).map((r, idx) => { + const kind = r.source_field ? dimensionKindForSourceField(r.source_field) : null + const clickable = !!(r.source_field && kind && onDimensionClick) + return ( +
clickable && onDimensionClick!({ + kind: kind!, + value: r.name, + sourceField: r.source_field!, + })} + title={clickable ? `查看「${r.name}」成分股` : undefined} + className={`grid grid-cols-[14px_1fr_auto] items-center gap-1 rounded-md bg-elevated/40 px-1.5 py-1 border border-transparent transition-colors ${ + clickable ? 'cursor-pointer hover:border-accent/40 hover:bg-elevated/70' : 'hover:border-border/60' + }`} + > {idx + 1}
-
{r.name}
+
+ {r.name} + {clickable && ↗} +
{r.count}只 · @@ -489,6 +508,11 @@ function RankColumn({ title, rows, tone, onStockClick }: { ) : ( {r.leader?.name ?? '—'} )} + {r.leader?.change_pct != null && ( + + {fmtStockPct(r.leader.change_pct)} + + )} {r.leader?.symbol && (() => { const board = boardTag(r.leader!.symbol!) return board ? ( @@ -501,24 +525,26 @@ function RankColumn({ title, rows, tone, onStockClick }: {
{fmtStockPct(r.avg_pct)}
- ))} + ) + })} {rows.length === 0 &&
暂无数据
}
) } -function HotRankCard({ title, rank, configUrl, onStockClick }: { +function HotRankCard({ title, rank, configUrl, onStockClick, onDimensionClick }: { title: string; rank?: OverviewMarket['concept_rank']; configUrl: string; - onStockClick?: (symbol: string, name?: string) => void; + onStockClick?: (symbol: string, name?: string) => void + onDimensionClick?: (target: DimensionMembersTarget) => void }) { const hasData = (rank?.leading?.length ?? 0) > 0 || (rank?.lagging?.length ?? 0) > 0 return (
- + {hasData ? (
- - + +
) : (
@@ -540,6 +566,8 @@ export function Dashboard() { const [selectedDate, setSelectedDate] = useState() const [manualFetching, setManualFetching] = useState(false) const [previewStock, setPreviewStock] = useState<{symbol: string; name?: string; alert?: AlertEvent} | null>(null) + // 板块成分股弹窗 (概念/行业热度卡片行点击) + const [dimensionTarget, setDimensionTarget] = useState(null) // 首次使用(无数据 + 未完成引导)自动弹窗: 同一会话只弹一次 const [showWelcomeModal, setShowWelcomeModal] = useState(false) const dataStatus = useDataStatus({ staleTime: 60_000 }) @@ -820,8 +848,12 @@ export function Dashboard() {
- setPreviewStock({symbol, name})} /> - setPreviewStock({symbol, name})} /> + setPreviewStock({symbol, name})} + onDimensionClick={setDimensionTarget} /> + setPreviewStock({symbol, name})} + onDimensionClick={setDimensionTarget} />
@@ -867,6 +899,14 @@ export function Dashboard() { } : null} onClose={() => setPreviewStock(null)} /> + setDimensionTarget(null)} + onStockClick={(symbol, name) => { + setDimensionTarget(null) + setPreviewStock({ symbol, name }) + }} + />
) } @@ -899,11 +939,13 @@ function FetchDataCard({ ? '可通过 TickFlow 免费服务器拉取近 1 年全 A 股日K' : `将从当前数据源「${providerLabel}」拉取近 1 年全 A 股日K`}(约 5500 只),预计 1-3 分钟,期间可继续浏览其他页面。

- {isTickflowProvider && ( -

- ⓘ 获取数据后即可进行策略定制、回测验证、选股扫描等本地分析功能。 -

- )} +

+ ⓘ 获取数据后即可进行策略定制、回测验证、选股扫描等本地分析功能。 +

+

+ 💡 配置 fuyao(同花顺 REST) Key 可解锁财务四表 / 龙虎榜 / 盘前风向标 / 竞价异动: + 前往设置 → +

{isFetching ? (
@@ -985,11 +1027,22 @@ function WelcomeFetchModal({ : `将从当前数据源「${providerLabel}」拉取近 1 年全 A 股日K`}(约 5500 只),预计 1-3 分钟。 同步期间可继续浏览其他页面,完成后看板自动刷新。

- {isTickflowProvider && ( -
- ⓘ 获取数据后即可进行策略定制、回测验证等本地分析功能。 -
- )} +
+
获取完成后的推荐步骤
+
    +
  1. 1. 配置 fuyao(同花顺 REST) Key — 解锁财务四表 / 龙虎榜 / 盘前风向标 / 竞价异动
    2. +
  2. 2. 分钟数据落盘(可选) — 分钟策略回测与板块分时走势需要
    3. +
  3. 3. 开始研究 — 自选加标的 → 策略扫描 → 回测验证
    4. +
+ + 前往设置 → 数据源 + + +
@@ -661,7 +661,7 @@ export function Data() { 当前无需 API Key,历史日K将使用免费通道获取。 实时行情、分钟K等能力取决于所选数据源,可在 - + 数据源设置 中配置。 @@ -973,7 +973,7 @@ export function Data() { {openSettings === 'daily' && ( setOpenSettings(null)}> setOpenSettings(null)} @@ -986,7 +986,7 @@ export function Data() { {showRepair && ( setShowRepair(false)}> setShowRepair(false)} @@ -1038,7 +1038,7 @@ export function Data() { {openSettings === 'page-settings' && ( setOpenSettings(null)}> - + )} @@ -1147,7 +1147,7 @@ export function Data() { {openSettings === 'minute' && ( setOpenSettings(null)}> - { setActiveJobId(jobId); setOpenSettings(null) }} /> + { setActiveJobId(jobId); setOpenSettings(null) }} /> )} diff --git a/frontend/src/pages/Financials.tsx b/frontend/src/pages/Financials.tsx index 1c46508..b083c70 100644 --- a/frontend/src/pages/Financials.tsx +++ b/frontend/src/pages/Financials.tsx @@ -1,8 +1,10 @@ import { useState, useEffect } from 'react' +import { Link } from 'react-router-dom' import { RefreshCw, Download, Lock, Loader2, X, Search, FileText, Database, Clock, CheckCircle2, Hourglass, Lightbulb, ExternalLink, ChartPie } from 'lucide-react' import { PageHeader } from '@/components/PageHeader' import { EmptyState } from '@/components/EmptyState' -import { useCapabilities } from '@/lib/useSharedQueries' +import { useCapabilities, useCapabilityMatrix } from '@/lib/useSharedQueries' +import { routeCapUsable } from '@/lib/capability-labels' import { useFinancialStatus, useFinancialSync } from '@/lib/useFinancials' import { StockFinancialSearch } from '@/components/financials/StockFinancialSearch' import { StockFinancialDetail } from '@/components/financials/StockFinancialDetail' @@ -30,8 +32,12 @@ const TABLE_ICON: Record = { export function Financials() { const { data: caps } = useCapabilities() + const { data: matrix } = useCapabilityMatrix() const { data: status, isLoading } = useFinancialStatus() - const hasFinancial = caps?.capabilities?.['financial'] != null || status?.available === true + // 路由感知门控: 生效源当前能否提供财务数据 (含插件/自定义源); + // 矩阵未加载时回退 TickFlow 套餐视角 + 后端可用状态 + const hasFinancial = routeCapUsable(matrix, 'financial') + ?? (caps?.capabilities?.['financial'] != null || status?.available === true) const syncMut = useFinancialSync() // 同步进行中 = 服务端真值(status.syncing)或本地乐观态(请求已发出待确认)。 // 乐观窗口:点击后到 invalidate 触发的 refetch 返回之间,status.syncing 暂为 false, @@ -72,6 +78,12 @@ export function Financials() {

当前数据源未提供财务数据。配置提供财务数据的数据源后,此页自动显示财务数据面板。

+ + 前往数据源配置 + {/* 当前财务数据源(TickFlow)需付费,后续将接入免费数据源;期间欢迎在 issues 推荐免费源 */}
@@ -274,7 +286,7 @@ export function Financials() {
暂无财务数据
-
点击右上角"全部同步"从 TickFlow 拉取
+
点击右上角"全部同步"从当前数据源拉取
) : ( <> diff --git a/frontend/src/pages/LimitUpLadder.tsx b/frontend/src/pages/LimitUpLadder.tsx index d267741..41de236 100644 --- a/frontend/src/pages/LimitUpLadder.tsx +++ b/frontend/src/pages/LimitUpLadder.tsx @@ -421,7 +421,9 @@ function MonitorMenu({ stock, direction, sealMode, monitorRule, anchorRect, hasD { key: '100000000', label: '亿元', mult: 100000000 }, ] - const [metric, setMetric] = useState<'sealed_vol' | 'sealed_amount'>(existing?.metric ?? (sealMode === 'amount' ? 'sealed_amount' : 'sealed_vol')) + const [metric, setMetric] = useState<'sealed_vol' | 'sealed_amount'>( + existing?.metric === 'sealed_amount' || (!existing && sealMode === 'amount') ? 'sealed_amount' : 'sealed_vol' + ) const units = metric === 'sealed_amount' ? AMT_UNITS : VOL_UNITS // 已有规则: 反算到最大便捷单位 (选能整除的最大倍率); 新建: 额默认亿元, 量默认万手 const initUnit = (() => { diff --git a/frontend/src/pages/Monitor.tsx b/frontend/src/pages/Monitor.tsx index 8a4d838..a7e1a19 100644 --- a/frontend/src/pages/Monitor.tsx +++ b/frontend/src/pages/Monitor.tsx @@ -19,11 +19,11 @@ import { markSeen, resetBadge, leaveMonitorPage } from '@/lib/monitorBadge' import { RuleEditor } from '@/components/monitor/RuleEditor' import { StockPreviewDialog } from '@/components/StockPreviewDialog' import { DimensionMembersDialog, type DimensionKind, type DimensionMembersTarget } from '@/components/DimensionMembersDialog' -import { usePreferences } from '@/lib/useSharedQueries' +import { usePreferences, useQuoteStatus } from '@/lib/useSharedQueries' const TYPE_LABEL: Record = { signal: '信号', price: '价格/涨跌', market: '市场异动', strategy: '策略监控', sector: '板块监控', - abnormal: '异动监控', + abnormal: '异动监控', volume_delta: '轮询放量', } /** 严重级别 → 左侧色条 + 图标 */ @@ -39,6 +39,7 @@ const SOURCE_BADGE_STYLE: Record = { market: 'bg-purple-500/10 text-purple-400 border-purple-500/20', sector: 'bg-cyan-500/10 text-cyan-700 border-cyan-500/20 dark:text-cyan-300', abnormal: 'bg-orange-500/10 text-orange-500 border-orange-500/20 dark:text-orange-400', + volume_delta: 'bg-rose-500/10 text-rose-400 border-rose-500/20 dark:text-rose-300', } /** @@ -132,12 +133,16 @@ export function Monitor() { }, [searchParams, setSearchParams]) // 触发记录: 过滤 + 统计 (提升到主组件, 供 header 行使用) - const [filter, setFilter] = useState<'all' | 'strategy' | 'signal' | 'price' | 'market' | 'sector' | 'abnormal'>('all') + const [filter, setFilter] = useState<'all' | 'strategy' | 'signal' | 'price' | 'market' | 'sector' | 'abnormal' | 'volume_delta'>('all') const [confirmClear, setConfirmClear] = useState(false) const [confirmClearRules, setConfirmClearRules] = useState(false) // 全局 ext 字段配置 (监控中心个股通知带行业/概念标签) const { data: prefs } = usePreferences() + // 实时行情可用性: mode=none 表示当前生效数据源完全无法提供实时行情 + // (TickFlow 无有效 Key, 或路由源未就绪) — 监控/预警收不到最新价, 顶部提示去数据源配置。 + const { data: quoteStatus } = useQuoteStatus() + const realtimeUnavailable = quoteStatus?.mode === 'none' const monitorExtFields = prefs?.monitor_ext_fields ?? { concept: { field: 'ext_gn_ths.所属概念' }, industry: { field: 'ext_hy_ths.所属同花顺行业' }, @@ -185,6 +190,22 @@ export function Monitor() { return (
+ {realtimeUnavailable && ( +
+
+ + + 实时行情当前不可用 — 监控与预警收不到最新价。可接入提供实时行情的数据源。 + + + 前往数据源配置 + +
+
+ )}
{/* 左栏: 触发记录 */} @@ -193,7 +214,7 @@ export function Monitor() { {/* 过滤标签 */}
- {(['all', 'strategy', 'signal', 'price', 'market', 'sector', 'abnormal'] as const).map(f => ( + {(['all', 'strategy', 'signal', 'price', 'market', 'sector', 'abnormal', 'volume_delta'] as const).map(f => (
+ ) : r.type === 'volume_delta' ? ( +
+ + {r.metric === 'amount' + ? `单轮增量 ≥ ${Math.round((r.threshold_amount ?? 1e6) / 1e4).toLocaleString()} 万元` + : `单轮增量 ≥ ${(r.threshold_volume ?? 9000).toLocaleString()} 手`} + + + 冷却 {Math.round((r.cooldown_seconds ?? 300) / 60)} 分钟 + + {r.basic_filter && Object.values(r.basic_filter).some(v => v !== null && v !== false) && ( + + 基础过滤{r.basic_filter.exclude_st ? ' · 剔除ST' : ''} + + )} +
) : r.type === 'strategy' && r.strategy_id ? (
{(r.score_min != null || r.score_max != null) && ( diff --git a/frontend/src/pages/Onboarding.tsx b/frontend/src/pages/Onboarding.tsx index dbd5f27..292c908 100644 --- a/frontend/src/pages/Onboarding.tsx +++ b/frontend/src/pages/Onboarding.tsx @@ -1,4 +1,4 @@ -import { useState } from 'react' +import { useState, useEffect, useRef } from 'react' import { useNavigate } from 'react-router-dom' import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query' import { motion, AnimatePresence } from 'framer-motion' @@ -12,7 +12,6 @@ import { LineChart, ScanSearch, Flame, - Zap, Radar, ShieldCheck, BellRing, @@ -20,20 +19,19 @@ import { FileText, Landmark, Database, - Plus, Puzzle, + KeyRound, + Route, } from 'lucide-react' -import { api } from '@/lib/api' -import { useCapabilities, usePreferences, useSettings } from '@/lib/useSharedQueries' +import { api, type ProviderField } from '@/lib/api' +import { usePreferences, useSettings } from '@/lib/useSharedQueries' import { QK } from '@/lib/queryKeys' -import { CAP_LABELS } from '@/lib/capability-labels' import { Logo } from '@/components/Logo' -import { DataSourceEditor } from '@/pages/settings/DataSourceEditor' // ===== 引导页:5 步向导 ===== -// 0. 声明 1. 欢迎 2. 输入 Key(可跳过) 3. 能力探测结果 4. 完成 → 写标记 → 进面板 +// 0. 声明 1. 欢迎 2. 数据源与 Key 3. 能力路由检测 4. 完成 → 写标记 → 进面板 -const STEPS = ['声明', '欢迎', '数据源', '能力探测', '完成'] as const +const STEPS = ['声明', '欢迎', '数据源', '能力路由', '完成'] as const const BRAND = '#8B5CF6' @@ -287,63 +285,86 @@ const DATASET_LABELS: Record = { function DataSourceStep({ onNext, onBack }: { onNext: () => void; onBack: () => void }) { const qc = useQueryClient() - const prefs = usePreferences() + const settings = useSettings() const sources = useQuery({ queryKey: QK.dataSources, queryFn: api.dataSources, staleTime: 60_000, }) - // null = 跟随后端当前激活源 (首次使用即默认内置源) - const [picked, setPicked] = useState(null) - // 是否展开「添加自有数据源」编辑器 - const [adding, setAdding] = useState(false) + // 两个常驻 Key 表单 (不收起): 输入/错误/已保存提示按数据源名分槽 + const [inputs, setInputs] = useState>({}) + const [errors, setErrors] = useState>({}) + const [savedMsg, setSavedMsg] = useState>({}) - const builtin = sources.data?.builtin ?? [] const plugins = sources.data?.plugins ?? [] + const builtin = sources.data?.builtin ?? [] const custom = sources.data?.custom ?? [] const items = [ ...builtin.map(s => ({ ...s, kind: 'builtin' as const })), ...plugins.map(s => ({ ...s, kind: 'plugin' as const })), ...custom.map(s => ({ ...s, kind: 'custom' as const })), ] - const byName = new Map(items.map(s => [s.name, s])) - const activeName = prefs.data?.daily_data_provider || 'tickflow' - const selected = picked ?? activeName + // 已配置态徽标: tickflow 看 settings.mode (free/api_key 均为已配 Key), fuyao 看插件 api_key_masked + const tfConfigured = settings.data?.mode === 'free' || settings.data?.mode === 'api_key' + const fuyaoConfigured = !!plugins.find(p => p.name === 'fuyao')?.api_key_masked - // 切换数据源 —— 与设置页同一套偏好接口: - // 内置源 5 个数据集全量切换; 其他源仅切换其声明支持的数据集, 其余回落内置源 - const switchProvider = useMutation({ - mutationFn: (name: string) => { - if (name === 'tickflow') { - return api.updateDataProviders({ - daily_data_provider: 'tickflow', - adj_factor_provider: 'same_as_daily', - realtime_data_provider: 'tickflow', - minute_data_provider: 'tickflow', - financial_data_provider: 'tickflow', - }) + // 内联 Key 配置 (先探后存): 验证通过才落盘 + const saveKey = useMutation({ + mutationFn: ({ name, key }) => (name === 'tickflow' + ? api.saveTickflowKey(key) + : api.savePluginKey(name, key)), + onSuccess: (data: any, vars) => { + qc.invalidateQueries({ queryKey: QK.dataSources }) + qc.invalidateQueries({ queryKey: QK.capabilities }) + qc.invalidateQueries({ queryKey: QK.capabilityMatrix }) + const setError = (msg: string | null) => setErrors(s => ({ ...s, [vars.name]: msg })) + const setSaved = (msg: string | null) => { + setSavedMsg(s => ({ ...s, [vars.name]: msg })) + if (msg) setTimeout(() => setSavedMsg(s => ({ ...s, [vars.name]: null })), 6000) + } + if (data.ok) { + setInputs(s => ({ ...s, [vars.name]: '' })) + setError(null) + if (vars.name === 'tickflow') { + setSaved(`TickFlow Key 已保存${data.tier_label ? `,当前档位:${data.tier_label}` : ''}`) + } else if (vars.name === 'fuyao' && data.plugin_available) { + // fuyao 定位为增强源: 仅实时行情 + 除权因子路由过去, 其余数据集保持 TickFlow + routeFuyaoEnhanced.mutate() + setSaved('fuyao Key 已保存:实时行情与除权因子已切换到 fuyao,其余数据集保持 TickFlow') + } else if (data.plugin_available) { + setSaved('Key 已保存,该数据源已就绪') + } + } else { + setError(data.error || (data.reason === 'invalid' ? 'Key 验证失败,请检查后重试' : '保存失败,请重试')) } - const supported = new Set(byName.get(name)?.datasets ?? []) - const pick = (dataset: string) => (supported.has(dataset) ? name : 'tickflow') - return api.updateDataProviders({ - daily_data_provider: pick('daily'), - adj_factor_provider: 'same_as_daily', - realtime_data_provider: pick('realtime'), - minute_data_provider: pick('minute'), - financial_data_provider: pick('financial'), - }) }, + onError: (e: Error, vars) => setErrors(s => ({ ...s, [vars.name]: `保存失败: ${e.message}` })), + }) + + // fuyao 增强路由: 只切实时行情 + 除权因子两个字段 (updateDataProviders 部分更新, 其余不动) + const routeFuyaoEnhanced = useMutation({ + mutationFn: () => api.updateDataProviders({ + realtime_data_provider: 'fuyao', + adj_factor_provider: 'fuyao', + }), onSuccess: () => qc.invalidateQueries({ queryKey: QK.preferences }), }) - const choose = (name: string, available: boolean) => { - if (!available || name === selected || switchProvider.isPending) return - setAdding(false) - setPicked(name) - switchProvider.mutate(name) - } + // 常驻展开的两个 Key 表单元数据 (卡片只读, Key 是向导里唯一可操作项) + const keyForms = [ + { + name: 'tickflow', display: 'TickFlow', env: 'TICKFLOW_API_KEY', configured: tfConfigured, autoFocus: true, + copy: '留空即免费 None 模式,可直接使用;填写 Key 后按订阅档位解锁实时 / 分钟 / 盘口 / 财务等更多数据集。仅存本地 (secrets.json),先验证后保存。', + register: { label: '前往 TickFlow 注册获取 Key', url: 'https://tickflow.org/auth/register?ref=V3KDKGXPEA' }, + }, + { + name: 'fuyao', display: 'fuyao', env: 'FUYAO_API_KEY', configured: fuyaoConfigured, autoFocus: false, + copy: '仅存本地 (secrets.json),先验证后保存。保存成功后仅「实时行情」与「除权因子」切换到 fuyao,其余数据集保持 TickFlow。', + register: { label: '前往 fuyao 官网申请 Key', url: 'https://fuyao.aicubes.cn' }, + }, + ] return (
@@ -354,12 +375,12 @@ function DataSourceStep({ onNext, onBack }: { onNext: () => void; onBack: () =>

配置数据源

- 所有数据源均为第三方服务,按需选择或添加自有接口;随时可在 - 设置 → 数据源 - 中调整。 + 默认使用内置 TickFlow 数据源(无需 Key 即可同步历史日K)。 + 可按下方的说明填写 API Key 增强数据能力;数据源的切换与增删随时在 + 设置 → 数据源 中进行。

- {/* 数据源卡片选择器 */} + {/* 数据源卡片 (只读展示): 不在向导中切换, 保留设置页完整能力 */} {sources.isLoading ? (
@@ -368,47 +389,39 @@ function DataSourceStep({ onNext, onBack }: { onNext: () => void; onBack: () => ) : (
{items.map(item => { - const isSelected = selected === item.name - const unavailable = item.kind === 'plugin' && !item.available const plugin = item.kind === 'plugin' ? plugins.find(p => p.name === item.name) : undefined - // 切换中的目标卡片: 圆点位置显示转圈, 其余卡片压暗 - const switchingToThis = switchProvider.isPending && switchProvider.variables === item.name + // 缺 Key 型插件 (如 fuyao): 表单常驻下方, 卡片仅提示, 不可交互 + const needsKey = item.kind === 'plugin' && !item.available && !!plugin?.api_key_env + const unavailable = item.kind === 'plugin' && !item.available && !needsKey return ( - +
) })} - - {/* 添加自有数据源卡片 */} -
)} - {/* 插件化提示: 标识数据源体系已插件化, 并给出两条接入路径与文档指引 */} + {/* API Key 配置 (常驻展开, 不收起): TickFlow 解锁档位 + fuyao 实时/除权增强路由 */} + {keyForms.map(f => { + const val = inputs[f.name] ?? '' + const err = errors[f.name] ?? null + const msg = savedMsg[f.name] ?? null + const pending = saveKey.isPending && saveKey.variables?.name === f.name + return ( +
+
+ + 配置 {f.display} API Key + {f.env} + {f.configured ? ( + + + 已配置 + + ) : f.name === 'tickflow' && ( + 可选 + )} +
+

{f.copy}

+ + {f.register.label} ↗ + +
{ + e.preventDefault() + if (val.trim() && !saveKey.isPending) saveKey.mutate({ name: f.name, key: val.trim() }) + }} + > + { + setInputs(s => ({ ...s, [f.name]: e.target.value })) + setErrors(s => ({ ...s, [f.name]: null })) + }} + placeholder={`粘贴 ${f.env}`} + autoFocus={f.autoFocus} + className="h-8 flex-1 rounded-input border border-border bg-surface px-3 font-mono text-xs text-foreground placeholder:text-muted/60 focus:border-accent/60 focus:outline-none" + /> + +
+ {err && ( +

+ + {err} +

+ )} + {msg && ( +

+ + {msg} +

+ )} +
+ ) + })} + + {/* 插件化提示: 自有数据源接入与切换在设置页, 向导保持极简 */}
数据源已插件化 - ,接入自有行情有两条路径:用 YAML 描述自有 HTTP 接口,放入 - data/data_sources/*.yaml - (也可用上方表单配置);或开发插件源,放入 - backend/app/plugins/ - 目录。接入方法与字段映射详见 + ,接入自有行情或切换数据源请前往 + 设置 → 数据源 + ,方法详见 docs/custom-data-source.md 与 docs/plugin-development.md @@ -456,38 +526,6 @@ function DataSourceStep({ onNext, onBack }: { onNext: () => void; onBack: () =>
- {switchProvider.isError && ( -
- - 数据源切换失败:{String((switchProvider.error as any)?.message ?? '')} -
- )} - - {/* 添加自有数据源: 复用设置页编辑器 (命名/鉴权/数据集字段映射/测试/启用) */} - - {adding && ( - - setAdding(false)} - onSaved={() => { - qc.invalidateQueries({ queryKey: QK.dataSources }) - setAdding(false) - }} - activeName={activeName} - onActivate={name => switchProvider.mutate(name)} - /> - - )} - - {/* 底部操作 */}
+ ) +} + +function _DtSummaryRow({ label, items, pick, onOpenStock }: { + label?: string + items: DragonTigerStockItem[] + pick: (i: DragonTigerStockItem) => number | null | undefined + onOpenStock: (s: string) => void +}) { + if (!items.length) return null + return ( +
+ {/* 固定宽度标签槽: 无标签行(净卖)也占位, 保证四行药丸左缘对齐 */} + + {label && ( + + {label} + + )} + + {items.map((i, idx) => ( + <_DtPill key={`${i.thscode}-${idx}`} item={i} idx={idx} value={pick(i)} onOpenStock={onOpenStock} /> + ))} +
+ ) +} + +/** 可排序列 key — 与 DragonTigerStockItem 数值字段对应 */ +type _DtSortKey = 'change' | 'net_value' | 'net_rate' | 'org_net_value' | 'buy_value' | 'sell_value' + +/** 排序表头: 点击切换列/方向, 箭头指示当前排序 (hover 才显隐未激活箭头) */ +function _DtTh({ label, sortKey, sort, onSort, className }: { + label: string + sortKey: _DtSortKey + sort: { key: _DtSortKey; desc: boolean } + onSort: (k: _DtSortKey) => void + className?: string +}) { + const active = sort.key === sortKey + return ( + + ) +} + +function _DtStockTable({ items, tab, onOpenStock }: { + items: DragonTigerStockItem[] + tab: _DtTabKey + onOpenStock: (s: string) => void +}) { + const isOrg = tab === 'org' + const [sort, setSort] = useState<{ key: _DtSortKey; desc: boolean }>({ key: isOrg ? 'org_net_value' : 'net_value', desc: true }) + const onSort = (k: _DtSortKey) => + setSort(s => (s.key === k ? { key: k, desc: !s.desc } : { key: k, desc: true })) + const sorted = [...items].sort((a, b) => { + const av = a[sort.key] ?? null + const bv = b[sort.key] ?? null + if (av == null && bv == null) return 0 + if (av == null) return 1 // 空值恒垫底, 不随方向翻转 + if (bv == null) return -1 + return (av - bv) * (sort.desc ? -1 : 1) + }) + const maxAbs = Math.max(1e-12, ...items.map(i => Math.abs(i.net_value ?? 0))) + return ( +
+ {/* 表头 */} +
+ # + <_DtTh label="涨跌幅" sortKey="change" sort={sort} onSort={onSort} /> + 股票 + <_DtTh label="净买额" sortKey="net_value" sort={sort} onSort={onSort} className="w-full justify-end" /> + <_DtTh label="占比" sortKey="net_rate" sort={sort} onSort={onSort} className="w-full justify-end" /> + {isOrg && <_DtTh label="机构净买" sortKey="org_net_value" sort={sort} onSort={onSort} className="w-full justify-end" />} + <_DtTh label="买入额" sortKey="buy_value" sort={sort} onSort={onSort} className="w-full justify-end" /> + <_DtTh label="卖出额" sortKey="sell_value" sort={sort} onSort={onSort} className="w-full justify-end" /> + 榜期 +
+ {sorted.map((i, idx) => { + const barPct = Math.max(2, Math.min(100, (Math.abs(i.net_value ?? 0) / maxAbs) * 100)) + const positive = (i.net_value ?? 0) >= 0 + return ( + + ) + })} +
+ ) +} + +function _DtSeatList({ seats, onOpenStock }: { + seats: { name?: string | null; buying?: number | null; rows?: DragonTigerStockItem[] | null }[] + onOpenStock: (s: string) => void +}) { + if (!seats.length) { + return

本期无游资上榜数据

+ } + return ( +
+ {seats.map((s, idx) => ( +
+ + {idx + 1} + +
+
+ {s.name ?? '—'} + + {fmtVolume(s.buying ?? null)} + +
+
+ {(s.rows ?? []).map(r => ( + + ))} +
+
+
+ ))} +
+ ) +} + +/** 追高风险阈值: 60日回测高开≥5%子集当日开盘买 -1.97% (温和高开才是名单 alpha 来源) */ + +function DragonTigerCard({ date, onOpenStock }: { + date?: string + onOpenStock: (symbol: string) => void +}) { + const [expanded, setExpanded] = useState(false) + const [tab, setTab] = useState<_DtTabKey>('all') + const q = useQuery({ + queryKey: ['dragon-tiger', date ?? 'latest'], + queryFn: () => api.dragonTiger(date), + staleTime: 5 * 60_000, + retry: 1, + }) + const d = q.data + + if (q.isLoading) { + return ( +
+ + + +
+ + + +
+
+ ) + } + + // fuyao 未配置: 显式降级提示 (不用死区块, 给配置入口) + if (d?.state === 'source_unavailable') { + return ( +
+ + + + 龙虎榜需要 fuyao 数据源 (同花顺特色数据) + + 前往配置 + +
+ ) + } + + // 拉取失败: 轻提示 + 重试 (不占复盘页视觉重量) + if (!d || d.state === 'no_data') { + return ( +
+ + + + 龙虎榜暂不可用{d?.message ? ` (${d.message.slice(0, 40)})` : ''} + +
+ ) + } + + const allItems = d.all?.stock_items ?? [] + const orgItems = d.org?.stock_items ?? [] + const seats = d.hot_money?.hot_money_items ?? [] + const topBuy = _dtSorted(allItems.filter(i => (i.net_value ?? 0) > 0), 'net_value').slice(0, 5) + const topOrg = _dtSorted(orgItems, 'org_net_value').slice(0, 5) + const topSell = _dtSorted(allItems.filter(i => (i.net_value ?? 0) < 0), 'net_value', true).slice(0, 5) + const botOrg = _dtSorted(orgItems.filter(i => (i.org_net_value ?? 0) < 0), 'org_net_value', true).slice(0, 5) + const isFallback = d.state === 'fallback_prev' + + return ( +
+ {/* 头部: 奖杯徽章 + 概要, 常驻可点击展开 */} + + + {/* 收起态: 排名药丸摘要 */} + {!expanded && ( +
+ <_DtSummaryRow label="净买 Top5" items={topBuy} pick={i => i.net_value} onOpenStock={onOpenStock} /> + <_DtSummaryRow items={topSell} pick={i => i.net_value} onOpenStock={onOpenStock} /> + <_DtSummaryRow label="机构 Top5" items={topOrg} pick={i => i.org_net_value} onOpenStock={onOpenStock} /> + <_DtSummaryRow items={botOrg} pick={i => i.org_net_value} onOpenStock={onOpenStock} /> +
+ )} + + {/* 展开态: 分段式 tab + 三榜内容 */} + + {expanded && ( + +
+ {/* 分段式 tab */} +
+ {_DT_TABS.map(t => ( + + ))} +
+ + {tab === 'hot_money' ? ( + <_DtSeatList seats={seats} onOpenStock={onOpenStock} /> + ) : ( + <_DtStockTable + key={tab} + items={_dtSorted(tab === 'org' ? orgItems : allItems, tab === 'org' ? 'org_net_value' : 'net_value')} + tab={tab} + onOpenStock={onOpenStock} + /> + )} +
+ + )} + +
+ ) +} diff --git a/frontend/src/pages/Screener.tsx b/frontend/src/pages/Screener.tsx index bc2910e..5f4d388 100644 --- a/frontend/src/pages/Screener.tsx +++ b/frontend/src/pages/Screener.tsx @@ -40,6 +40,9 @@ const SHOW_STRATEGY_STORE = false export function Screener() { const [assetType, setAssetType] = useState<'stock' | 'etf'>('stock') + // 周期显示筛选: 全部 / 日线 / 分钟 — 只过滤卡片显示, 不影响池和执行; + // 执行按每个策略自己声明的 timeframes 路由 (日线走盘后缓存, 分钟走本地分钟K分区) + const [tfFilter, setTfFilter] = useState<'all' | '1d' | '1m'>('all') const [activeStrategy, setActiveStrategy] = useState(null) const [result, setResult] = useState(null) const [asOf, setAsOf] = useState('') @@ -127,12 +130,22 @@ export function Screener() { const { data: prefs } = usePreferences() const screenerAutoRun = prefs?.screener_auto_run ?? true + // 统一列表: 不按周期过滤, 日线+分钟策略合并返回, 分钟策略带 timeframes 标识 const strategies = useQuery({ - queryKey: QK.screenerStrategies('all'), - queryFn: () => api.screenerStrategies(), + queryKey: [...QK.screenerStrategies('all'), 'all'], + queryFn: () => api.screenerStrategies(undefined, 'all'), }) + // 激活策略自身的执行周期 (决定走缓存还是分钟实时跑)。 + // 在 queries 之前独立计算, 避免依赖下方 strategyMap 的定义顺序。 + const activeStrategyTimeframe = useMemo(() => { + if (!activeStrategy) return '1d' as const + const meta = (strategies.data?.presets ?? []).find(s => s.id === activeStrategy) + return meta?.timeframes?.includes('1m') ? ('1m' as const) : ('1d' as const) + }, [strategies.data, activeStrategy]) + // 卡片首屏只读取轻量摘要；明细在点击策略或“全部”时按需加载。 + // 摘要只覆盖日线缓存; 分钟策略命中数来自手动单跑。 const summaryQuery = useQuery({ queryKey: QK.screenerCachedSummary, queryFn: api.screenerCachedSummary, @@ -142,13 +155,14 @@ export function Screener() { const fullCachedQuery = useQuery({ queryKey: QK.screenerCached(asOf, extColumnsParam), queryFn: () => api.screenerCached(extColumnsParam || undefined), - enabled: assetType === 'stock' && showAll, + enabled: assetType === 'stock' && tfFilter !== '1m' && showAll, }) const singleCachedQuery = useQuery({ queryKey: QK.screenerCachedResult(activeStrategy ?? '', asOf, extColumnsParam), queryFn: () => api.screenerCachedResult(activeStrategy!, extColumnsParam || undefined), enabled: assetType === 'stock' + && activeStrategyTimeframe === '1d' && !showAll && !!activeStrategy && summaryQuery.data?.results[activeStrategy]?.as_of === asOf, @@ -195,11 +209,25 @@ export function Screener() { ) const visiblePool = useMemo(() => pool.filter(id => availableStrategyIds.has(id)), [pool, availableStrategyIds]) + // 卡片显示: 按周期筛选 (all=全部, 1d=仅日线, 1m=仅分钟); 未声明 timeframes 视为日线 + const displayPool = useMemo(() => visiblePool.filter(id => { + if (tfFilter === 'all') return true + const isMinute = strategyMap.get(id)?.timeframes?.includes('1m') ?? false + return tfFilter === '1m' ? isMinute : !isMinute + }), [visiblePool, strategyMap, tfFilter]) + + // runAll/盘后缓存只覆盖日线策略; 池中分钟策略由手动单跑实时计算 + const dailyPoolIds = useMemo( + () => visiblePool.filter(id => !(strategyMap.get(id)?.timeframes?.includes('1m') ?? false)), + [visiblePool, strategyMap], + ) + // 策略列表加载后,自动清除池中失效的自定义策略(如本地开发残留的、 // 当前后端已不存在的策略 ID),避免"策略池"对话框持续显示失效项。 // 关键: 仅当本次拉取成功且返回非空列表时才 prune。 // 拉取中/失败/返回空(如引擎 reload 瞬时把某策略跳过)时一律不碰池, // 否则会把用户池里仍有效的 ID 永久清空并写入 localStorage,导致卡片全没。 + // 日线/分钟池按周期隔离, 各自用自身周期的列表清理, 互不影响。 useEffect(() => { if (strategies.isError) return // 拉取失败: 不 prune if (!strategies.isSuccess) return // 加载中: 不 prune @@ -215,12 +243,12 @@ export function Screener() { } }, [loadErrors]) - // 进入页面自动跑策略池中的策略，获取命中数 + // 进入页面自动跑策略池中的策略，获取命中数 (仅日线策略; 分钟策略手动单跑) const runAll = useMutation({ mutationFn: ({ date, strategyIds }: { date?: string; strategyIds?: string[] } = {}) => api.screenerRunAll( date, - strategyIds ?? visiblePool, + strategyIds ?? dailyPoolIds, assetType, ), onSuccess: (data) => { @@ -235,10 +263,10 @@ export function Screener() { }) const missingStrategyIds = useMemo( - () => visiblePool.filter(id => summaryQuery.data?.results[id]?.as_of !== asOf), - [visiblePool, summaryQuery.data, asOf], + () => dailyPoolIds.filter(id => summaryQuery.data?.results[id]?.as_of !== asOf), + [dailyPoolIds, summaryQuery.data, asOf], ) - const cacheCoversPool = visiblePool.length > 0 && missingStrategyIds.length === 0 + const cacheCoversPool = dailyPoolIds.length > 0 && missingStrategyIds.length === 0 // 防止 reload / auto-run / StrictMode 叠出并发 run_all（后端 Numba 会崩溃） // 用 ref 同步门闩，避免同一渲染周期内 isPending 尚未更新导致重复触发 @@ -445,9 +473,10 @@ export function Screener() { // 缓存命中时秒加载; 未命中时, 仅当 screener_auto_run 开启才自动触发 runAll useEffect(() => { // ETF 模式无股票盘后缓存/ runAll, 单策略走实时单跑, 不触发 runAll - if (assetType !== 'stock') return - if (!asOf || strategyPresets.length === 0 || !summaryQuery.isSuccess || runAll.isPending || visiblePool.length === 0) return - const runKey = `${asOf}|${visiblePool.join(',')}` + // 分钟筛选视图下不跑日线缓存 (切回 全部/日线 视图时本 effect 会重新评估) + if (assetType !== 'stock' || tfFilter === '1m') return + if (!asOf || strategyPresets.length === 0 || !summaryQuery.isSuccess || runAll.isPending || dailyPoolIds.length === 0) return + const runKey = `${asOf}|${dailyPoolIds.join(',')}` if (runAllDateRef.current === runKey) return // 缓存已覆盖当前策略池 → 秒加载, 不触发 runAll if (cacheCoversPool) { @@ -458,11 +487,12 @@ export function Screener() { if (!screenerAutoRun) return runAllDateRef.current = runKey requestRunAll({ date: asOf, strategyIds: missingStrategyIds }) - }, [asOf, strategyPresets.length, summaryQuery.isSuccess, visiblePool, cacheCoversPool, missingStrategyIds, screenerAutoRun, assetType, runAll.isPending, requestRunAll]) + }, [asOf, strategyPresets.length, summaryQuery.isSuccess, dailyPoolIds, cacheCoversPool, missingStrategyIds, screenerAutoRun, assetType, tfFilter, runAll.isPending, requestRunAll]) + // 执行周期由策略自身声明决定: 日线走盘后缓存/单跑, 分钟走本地分钟K分区实时跑 const run = useMutation({ - mutationFn: ({ id, date }: { id: string; date: string }) => - api.screenerRunPreset(id, undefined, date || undefined, extColumnsParam || undefined, assetType), + mutationFn: ({ id, date, timeframe: tf }: { id: string; date: string; timeframe: '1d' | '1m' }) => + api.screenerRunPreset(id, undefined, date || undefined, extColumnsParam || undefined, assetType, tf), onSuccess: (data, vars) => { setResult(data) // 同步更新卡片上的命中数 @@ -477,15 +507,16 @@ export function Screener() { setActiveStrategy(s.id) setShowAll(false) if (result?.strategy !== s.id || result.as_of !== asOf) setResult(null) - // ETF 模式: 无股票盘后缓存, 始终实时单跑。 - // 传空日期让后端用 ETF 自己的最新交易日 (asOf 跟随的是股票 enriched, 两者可能不同日)。 - if (assetType !== 'stock') { - run.mutate({ id: s.id, date: '' }) + const tf = s.timeframes?.includes('1m') ? '1m' as const : '1d' as const + // ETF 模式无股票盘后缓存、分钟策略走本地分钟分区 → 始终实时单跑。 + // 传空日期让后端用自身的最新交易日 (ETF 与分钟分区跟股票 enriched 可能不同日)。 + if (assetType !== 'stock' || tf === '1m') { + run.mutate({ id: s.id, date: '', timeframe: tf }) return } // 摘要命中时由 singleCachedQuery 按需加载明细；缺失时才单独计算。 if (summaryQuery.data?.results[s.id]?.as_of === asOf || runAll.isPending) return - run.mutate({ id: s.id, date: asOf }) + run.mutate({ id: s.id, date: asOf, timeframe: tf }) } // 日期变化交给统一 effect 计算一次，避免这里与 effect 重复请求。 @@ -602,14 +633,14 @@ export function Screener() { subtitle="基于本地 enriched 表 · 毫秒级 SQL" right={
- {/* 资产类型切换: 股票 / ETF */} + {/* 资产类型切换: 股票 / ETF (分钟策略 asset_types 仅股票, ETF 列表自然不含) */}
{(['stock', 'etf'] as const).map(t => ( ))}
+ {/* 周期筛选: 全部 / 日线 / 分钟 — 只过滤卡片显示, 不影响池与执行路由 */} +
+ {(['all', '1d', '1m'] as const).map(tf => ( + + ))} +
{/* 重新运行策略：重载策略文件并重跑全部策略，更新命中个股 */}
diff --git a/frontend/src/pages/backtest/ResearchCandidatesDialog.tsx b/frontend/src/pages/backtest/ResearchCandidatesDialog.tsx index 2aa7b74..7c8c711 100644 --- a/frontend/src/pages/backtest/ResearchCandidatesDialog.tsx +++ b/frontend/src/pages/backtest/ResearchCandidatesDialog.tsx @@ -1,6 +1,6 @@ import { useMemo, useState } from 'react' import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query' -import { ArrowDown, ArrowUp, BookmarkCheck, CheckCircle2, Clock3, Link2, Loader2, Trash2, X, XCircle } from 'lucide-react' +import { ArrowDown, ArrowUp, BookmarkCheck, CheckCircle2, Clock3, Link2, Loader2, RotateCcw, Trash2, X, XCircle } from 'lucide-react' import { Modal } from '@/components/Modal' import { toast } from '@/components/Toast' import { api, type ResearchCandidate, type ResearchCandidateStatus, type ScoringDirection } from '@/lib/api' @@ -41,7 +41,11 @@ function metricSummary(item: ResearchCandidate) { ].filter(Boolean).join(' · ') || '暂无指标摘要' } -export function ResearchCandidatesDialog({ onClose }: { onClose: () => void }) { +export function ResearchCandidatesDialog({ onClose, onLoadStrategy }: { + onClose: () => void + /** 策略候选「载入复测」: 把保存的 config 回填到回测表单 (由回测页接线) */ + onLoadStrategy?: (candidate: ResearchCandidate) => void +}) { const queryClient = useQueryClient() const [kind, setKind] = useState<'all' | 'factor' | 'strategy'>('all') const [linkDraft, setLinkDraft] = useState(null) @@ -219,6 +223,17 @@ export function ResearchCandidatesDialog({ onClose }: { onClose: () => void }) { ))}
+ {item.kind === 'strategy' && onLoadStrategy && ( + + )} {item.kind === 'factor' && (
+ )}
+ {/* 分钟策略提示条: 数据窗口 + 成交语义 */} + {isMinuteStrategy && ( +
+ +
+ 分钟策略回测 + ：逐日回放分钟K，信号分钟收盘价买入；日线条件按 T-1 完成态评估。 + {minuteDataStatus?.minute?.earliest_date + ? ` 本地分钟K ${minuteDataStatus.minute.earliest_date} ~ ${minuteDataStatus.minute.latest_date}（${minuteDataStatus.minute.trading_days} 个交易日），缺分区的日子自动跳过。` + : ' 本地暂无分钟K数据，请先在数据页拉取。'} + {minuteStartMismatch && ( + + 当前开始日期 {start} 早于分钟数据起点 {minuteEarliest}，运行会被拒绝 — 请把开始日期调整到 {minuteEarliest} 之后，或先用「扩展分钟K历史」拉取。 + + )} +
+
+ )} {/* 分钟K开启时的提示条 */} - {highGranularity && hasMinuteBatch && ( + {highGranularity && hasMinuteBatch && !isMinuteStrategy && (
@@ -1465,6 +1618,9 @@ export function StrategyBacktest() { }`} > {st.name} + {st.timeframes?.includes('1m') && ( + 分钟 + )} {st.source && st.source !== 'builtin' && ( {SRC_MAP[st.source] ?? ''} @@ -1619,10 +1775,18 @@ export function StrategyBacktest() {
- + {isMinuteStrategy ? ( +
+ + 信号分钟收盘 + 分钟 +
+ ) : ( + + )}
@@ -1633,12 +1797,12 @@ export function StrategyBacktest() { > - {highGranularity && minuteExitTriggerSupported && ( + {highGranularity && minuteExitTriggerSupported && !isMinuteStrategy && ( )}
- {(entryFill === 'close_t' || exitFill === 'close_t') && ( + {!isMinuteStrategy && (entryFill === 'close_t' || exitFill === 'close_t') && (
信号日收盘仅适合收盘前已确认的信号 @@ -1931,6 +2095,15 @@ export function StrategyBacktest() { )} 持有 {result.config?.holding_days ?? 5} 天 + {String(result.config?.start).slice(0,10)} ~ {String(result.config?.end).slice(0,10)} @@ -2047,6 +2220,15 @@ export function StrategyBacktest() { {fmtDuration(result.elapsed_ms)} )} +
)} diff --git a/frontend/src/pages/settings/AI.tsx b/frontend/src/pages/settings/AI.tsx index 0e7a248..727cc19 100644 --- a/frontend/src/pages/settings/AI.tsx +++ b/frontend/src/pages/settings/AI.tsx @@ -1,4 +1,4 @@ -import { useState, useEffect, useRef } from 'react' +import { useState, useEffect, useRef, createContext, useContext } from 'react' import { useMutation, useQueryClient } from '@tanstack/react-query' import { Save, Loader2, Check, Wifi, WifiOff, Eye, EyeOff, Shield, @@ -8,6 +8,7 @@ import { import { useSettings } from '@/lib/useSharedQueries' import { api, type SettingsState } from '@/lib/api' import { QK } from '@/lib/queryKeys' +import { useCardFlash, cardFlashCls } from '@/lib/useCardFlash' // 统一的输入框样式(与项目其他设置页一致) const INPUT_CLS = @@ -70,7 +71,7 @@ const findPreset = (provider: string, baseUrl: string, codexCommand: string) => return provider === CODEX_PROVIDER ? p.codexCommand === codexCommand : p.url === baseUrl }) ?? PRESETS[0] -export function SettingsAIPanel() { +export function SettingsAIPanel({ highlight }: { highlight?: string } = {}) { const qc = useQueryClient() const settings = useSettings() const s = settings.data @@ -306,8 +307,9 @@ export function SettingsAIPanel() { } return ( +
- @@ -519,20 +521,27 @@ export function SettingsAIPanel() {
)}
+ ) } // ===== 通用卡片(与 Keys 页风格统一) ===== +// 卡片定位锚点: highlight= 时滚动到视口中央并闪烁 (见 useCardFlash) +const HighlightContext = createContext('') + interface CardProps { icon: React.ComponentType<{ className?: string }> title: string right?: React.ReactNode children: React.ReactNode + anchor?: string } -function Card({ icon: Icon, title, right, children }: CardProps) { - return ( +function Card({ icon: Icon, title, right, children, anchor }: CardProps) { + const highlight = useContext(HighlightContext) + const { ref, flash } = useCardFlash(anchor ? highlight : undefined, anchor ?? '') + const inner = (
@@ -544,6 +553,12 @@ function Card({ icon: Icon, title, right, children }: CardProps) { {children}
) + if (!anchor) return inner + return ( +
+ {inner} +
+ ) } // ===== 表单字段(统一 label + 输入框样式) ===== diff --git a/frontend/src/pages/settings/CustomSignals.tsx b/frontend/src/pages/settings/CustomSignals.tsx index d445b22..c19b12d 100644 --- a/frontend/src/pages/settings/CustomSignals.tsx +++ b/frontend/src/pages/settings/CustomSignals.tsx @@ -6,6 +6,7 @@ import { QK } from '@/lib/queryKeys' import { BUILTIN_SIGNAL_DEFINITIONS, type SignalKind } from '@/lib/signals' import { CustomSignalDialog } from '@/components/signals/CustomSignalDialog' import { Skeleton } from '@/components/data/Skeleton' +import { AnchorWrap } from '@/lib/useCardFlash' type SignalSection = 'builtin' | 'custom' @@ -16,7 +17,7 @@ const KIND_CLASS: Record = { both: 'bg-muted/10 text-muted', } -export function SettingsCustomSignalsPanel() { +export function SettingsCustomSignalsPanel({ highlight }: { highlight?: string } = {}) { const qc = useQueryClient() const list = useQuery({ queryKey: QK.customSignals, queryFn: api.customSignalsList }) const options = useQuery({ queryKey: QK.customSignalsOptions, queryFn: api.customSignalsOptions }) @@ -87,6 +88,7 @@ export function SettingsCustomSignalsPanel() { return (
+
@@ -133,6 +135,7 @@ export function SettingsCustomSignalsPanel() {
+ {activeSection === 'builtin' && (
diff --git a/frontend/src/pages/settings/DataSources.tsx b/frontend/src/pages/settings/DataSources.tsx index 06f90f3..6eb5e48 100644 --- a/frontend/src/pages/settings/DataSources.tsx +++ b/frontend/src/pages/settings/DataSources.tsx @@ -1,56 +1,79 @@ import { useState } from 'react' import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query' import { motion, AnimatePresence } from 'framer-motion' -import { Check, Database, Eye, EyeOff, KeyRound, Plus, RefreshCw, Zap, FileWarning, Puzzle, AlertCircle, CheckCircle2, Loader2, Save, Trash2 } from 'lucide-react' -import { api, type DataSourceItem, type PluginDataSourceItem } from '@/lib/api' +import { + AlertCircle, + AlertTriangle, + CandlestickChart as CandlestickIcon, + Check, + CheckCircle2, + Database, + ExternalLink, + Eye, + EyeOff, + FileWarning, + KeyRound, + Landmark, + ListChecks, + Loader2, + Lock, + Plus, + Puzzle, + Radio, + RefreshCw, + Route, + Save, + Scale, + Timer, + Trash2, + Zap, +} from 'lucide-react' +import { + api, + type CapabilityMatrix, + type CapabilityRoute, + type DataSourceItem, + type PluginDataSourceItem, + type Preferences, + type ProviderField, +} from '@/lib/api' import { QK } from '@/lib/queryKeys' import { useCapabilities, usePreferences } from '@/lib/useSharedQueries' -import { TIER_RANK, tierRank, tierStyle } from '@/lib/capability-labels' +import { AnchorWrap } from '@/lib/useCardFlash' +import { CAP_LABELS, TIER_RANK, tierRank, tierStyle, TierTag } from '@/lib/capability-labels' import { toast } from '@/components/Toast' import { DataSourceEditor } from './DataSourceEditor' -import { TickFlowKeyConfig } from './Keys' +import { TickFlowKeySection, TierHelpPopover, useInvalidateTierRelated } from './Keys' const DATASET_LABEL: Record = { - daily: '日K', - adj_factor: '除权', realtime: '实时', + daily: '日K', minute: '分钟', + adj_factor: '除权', + depth5: '五档', financial: '财务', + full_minute: '全量分钟', } -/** 数据集 → 路由偏好字段 + 默认值 + 展示标签 (financial 无后端路由消费方, 仅展示不参与切换) */ -/** 能力卡片定义: 数据集 + 说明 (路由选择嵌入每张卡片) */ -const CAPABILITY_CARDS = [ - { dataset: 'daily', label: '日K', desc: '历史 + 实时覆写' }, - { dataset: 'adj_factor', label: '除权因子', desc: '复权计算' }, - { dataset: 'realtime', label: '实时行情', desc: '全市场快照' }, - { dataset: 'minute', label: '分钟K', desc: '分时图 · 回测' }, -] as const - -/** 数据集 → 路由偏好字段 + 默认值 (financial 无后端路由消费方, 仅展示) */ -const DATASET_ROUTE: Record = { - daily: { field: 'daily_data_provider', def: 'tickflow' }, - adj_factor: { field: 'adj_factor_provider', def: 'same_as_daily' }, - minute: { field: 'minute_data_provider', def: 'tickflow' }, - realtime: { field: 'realtime_data_provider', def: 'tickflow' }, +/** 能力图标 (纯展示; 能力清单本身由后端注册表驱动) */ +const CAP_ICON: Record> = { + realtime: Radio, + daily: CandlestickIcon, + minute: Timer, + full_minute: Zap, + adj_factor: Scale, + financial: Landmark, } -/** 各能力在 TickFlow 需要的最低订阅档位 (对照 tiers.yaml: 日K 全档位可用, - * 全市场实时/除权因子需 Starter+, 分钟K需 Pro+, 财务需 Expert+) */ -const TICKFLOW_TIER_REQ: Record = { - daily: 'none', - adj_factor: 'starter', - realtime: 'starter', - minute: 'pro', - financial: 'expert', +/** TickFlow 档位要求文本: none → 全档位, 其余 → starter+ 形式 */ +function tierReqText(tier: string): string { + return tier === 'none' ? '全档位' : `${tier}+` } -/** TickFlow 档位要求徽标: 按所需档位配色(与左侧菜单/Key 页一致), 当前档位不足时琥珀描边提示 */ +/** TickFlow 档位要求徽标: 按所需档位配色, 当前档位不足时琥珀描边提示。 + * 仅用于 TickFlow 详情的档位介绍表 (能力卡上不展示档位信息)。 */ function TierReqChip({ tier, currentLabel }: { tier: string; currentLabel?: string }) { - const text = tier === 'none' ? '全档位' : `${tier}+` + const text = tierReqText(tier) const req = TIER_RANK[tier] ?? -1 const unmet = currentLabel != null && tierRank(currentLabel) < req const t = tierStyle(tier) @@ -82,153 +105,280 @@ function AllTiersBadge({ size = 'text-[10px]' }: { size?: string }) { ) } -/** 卡片内静态数据集标签: 只展示该源适配了哪些数据集 (路由选择在下方能力卡片) */ -function DatasetChipRow({ datasets }: { datasets: string[] }) { - if (datasets.length === 0) return null - return ( -
- 已适配 - {datasets.map(ds => ( - - {DATASET_LABEL[ds] || ds} - - ))} -
- ) -} - -/** 按能力路由网格: 每个数据源详情下展示一张能力卡,卡片上以可点标签列出 - * 所有具备该能力的数据源 — 点谁,该数据集就立刻由谁提供,可跨源自由组合。 - * TickFlow 详情含全部能力; 其他源只列自己参与的能力。 */ -function SourceCapabilityGrid({ sourceName, sourceDisplay, datasets, candidatesOf, providerOf, pending, onSelect, anyCustom, onReset }: { - sourceName: string - sourceDisplay: string - datasets: string[] - /** 该数据集的所有候选提供方 (含 TickFlow), 按推荐顺序 */ - candidatesOf: (dataset: string) => { name: string; display: string }[] - /** 该数据集当前的原始路由偏好值 (adj_factor 可能是 same_as_daily) */ - providerOf: (dataset: string) => string - pending?: boolean - onSelect: (dataset: string, provider: string) => void - anyCustom?: boolean - onReset?: () => void -}) { - const isDefault = sourceName === 'tickflow' - const dsList = isDefault ? [...datasets, 'financial'] : datasets - const caps = useCapabilities() - if (dsList.length === 0) return null - - return ( -
-
- - {isDefault - ? '每个能力可单独选择提供方 — 点标签即刻切换,未单独设置的由 TickFlow 提供' - : `${sourceDisplay} 参与的能力 — 每个能力都可单独选择由哪个数据源提供`} - - {isDefault && anyCustom && ( - - )} -
- -
- {dsList.map(ds => { - const route = DATASET_ROUTE[ds] - const meta = CAPABILITY_CARDS.find(c => c.dataset === ds) - const label = meta?.label || DATASET_LABEL[ds] || ds - const desc = meta?.desc || '' - if (!route) { - return ( -
-
-
-
{label}
- {desc &&
{desc}
} -
-
- {isDefault && } - 固定 -
-
-
- ) - } - const raw = providerOf(ds) - const candidates = candidatesOf(ds) - return ( -
-
-
-
{label}
- {desc &&
{desc}
} -
- {/* 右上角: TickFlow 所需档位 (当前提供方由下方高亮标签指示) */} - {isDefault && } -
- {/* 提供方标签: 点谁该数据集就由谁提供,当前项高亮 */} -
- {ds === 'adj_factor' && ( - - )} - {candidates.map(c => ( - - ))} -
-
- ) - })} -
-
- ) -} - -/** 提供方标签样式: 当前项高亮(accent), 其余弱化可点 */ -function tagCls(active: boolean) { - return `px-1.5 py-0.5 rounded text-[10px] transition-colors select-none disabled:opacity-50 cursor-pointer ${ +/** 提供方标签样式: 当前项高亮(accent), 其余弱化可点; 未就绪源禁用置灰 */ +function tagCls(active: boolean, disabled = false, interactive = true) { + const base = 'inline-flex items-center gap-1 px-1.5 py-0.5 rounded text-[10px] transition-colors select-none' + if (disabled) return `${base} bg-elevated/40 text-muted/50 cursor-not-allowed` + if (!interactive) return `${base} cursor-default ${active ? 'bg-accent/15 text-accent font-medium' : 'bg-elevated/60 text-muted/70'}` + return `${base} cursor-pointer disabled:opacity-50 ${ active ? 'bg-accent/15 text-accent font-medium' : 'bg-elevated/60 text-muted/70 hover:bg-accent/15 hover:text-accent' }` } - -/** 详情组件所需的路由上下文(由面板构造) */ -interface RouteCtx { - /** 数据集 → 候选提供方列表 (含 TickFlow) */ - candidatesOf: (dataset: string) => { name: string; display: string }[] - /** 数据集 → 当前原始路由偏好值 (adj_factor 可能是 same_as_daily) */ - providerOf: (dataset: string) => string - displayOf: (name?: string) => string - pending: boolean - onSelect: (dataset: string, provider: string) => void - anyCustom: boolean - onReset: () => void +/** 源名 → 展示名 (特殊值 + 候选/待就绪表查找, 找不到回退原始名) */ +function displayOfName(matrix: CapabilityMatrix | undefined, name: string): string { + if (name === 'tickflow') return 'TickFlow' + if (name === 'same_as_daily') return '跟随日K' + for (const cap of matrix?.capabilities ?? []) { + const hit = cap.candidates.find(c => c.name === name) ?? cap.pending.find(c => c.name === name) + if (hit) return hit.display + } + return name } -/** 插件 API Key 配置区 (布局对齐 TickFlowKeyConfig: 状态 + 输入 + 保存并检测)。 +/** 乐观更新: 把一组偏好字段变更应用到矩阵缓存 (每能力独立路由, current 即生效) */ +function patchMatrix( + matrix: CapabilityMatrix, + changes: Partial>, +): CapabilityMatrix { + const caps = matrix.capabilities.map(c => ({ ...c })) + for (const cap of caps) { + const value = cap.field != null ? changes[cap.field] : undefined + if (value !== undefined) { + cap.current = value + cap.current_display = displayOfName({ capabilities: caps }, value) + cap.effective = value + cap.effective_display = cap.current_display + } + } + return { ...matrix, capabilities: caps } +} + +const DEFAULT_ROUTING: Record = { + daily_data_provider: 'tickflow', + adj_factor_provider: 'tickflow', + minute_data_provider: 'tickflow', + depth5_data_provider: 'tickflow', + realtime_data_provider: 'tickflow', + financial_data_provider: 'tickflow', +} + +/** 单个能力卡: 当前生效提供方 + 候选切换标签。 + * candidates 只含当前可提供该能力的源; 未就绪源 (pending) 置灰提示; + * 生效源无法提供该能力 (usable=False, 无论档位不足还是源未就绪) 时显示琥珀警示。 */ +function CapabilityCard({ cap, pendingKey, onSelect }: { + cap: CapabilityRoute + pendingKey: string | null + onSelect: (field: ProviderField, provider: string) => void +}) { + const Icon = CAP_ICON[cap.id] ?? Database + const busy = (provider: string) => pendingKey === `${cap.field}:${provider}` + // 能力中立判定: usable=False 即当前路由的源供不了 (TickFlow 档位不足或插件未就绪同待遇) + const unmet = !cap.usable + const chipsEmpty = cap.candidates.length === 0 && cap.pending.length === 0 + return ( +
+
+ + + +
{cap.label}
+
+ {cap.desc &&
{cap.desc}
} + + {/* 当前生效提供方 */} +
+ 当前 + {unmet ? ( + + + 能力不可用 + + ) : ( + <> + + {cap.effective_display} + + )} +
+ + {/* 候选标签: 点谁该能力就由谁提供 */} +
+ {cap.candidates.map(c => { + const active = cap.current === c.name + // field=null → 不可路由能力 (仅 TickFlow 提供): 渲染为非交互标签, + // 保持激活高亮但不可点击 (用 button+disabled 会被 opacity-50 冲淡成灰色) + const routable = cap.field != null + if (!routable) { + return ( + + {c.display} + + ) + } + return ( + + ) + })} + {/* 未就绪源: 声明了该能力但依赖/Key 未配好, 置灰并说明原因 */} + {cap.pending.map(c => ( + + + {c.display} + + ))} + {chipsEmpty && ( + + 暂无可用提供方 + + )} +
+
+ ) +} + +/** 能力路由区 (页面主视图): 每个能力一张卡, 点候选标签即刻切换 (乐观更新) */ +function CapabilityRoutingSection() { + const qc = useQueryClient() + const matrix = useQuery({ queryKey: QK.capabilityMatrix, queryFn: api.capabilityMatrix }) + const [pendingKey, setPendingKey] = useState(null) + + const invalidateRouting = () => { + qc.invalidateQueries({ queryKey: QK.capabilityMatrix }) + qc.invalidateQueries({ queryKey: QK.preferences }) + qc.invalidateQueries({ queryKey: QK.capabilities }) + qc.invalidateQueries({ queryKey: QK.quoteStatus }) + } + + /** 切换前先把变更写进矩阵/偏好缓存, 界面零延迟响应; 失败回滚 */ + const applyOptimistic = (changes: Partial>) => { + const prevMatrix = qc.getQueryData(QK.capabilityMatrix) + const prevPrefs = qc.getQueryData(QK.preferences) + if (prevMatrix) qc.setQueryData(QK.capabilityMatrix, patchMatrix(prevMatrix, changes)) + if (prevPrefs) qc.setQueryData(QK.preferences, { ...prevPrefs, ...changes }) + return { prevMatrix, prevPrefs } + } + + const rollback = (ctx: { prevMatrix?: CapabilityMatrix; prevPrefs?: Preferences } | undefined) => { + if (ctx?.prevMatrix) qc.setQueryData(QK.capabilityMatrix, ctx.prevMatrix) + if (ctx?.prevPrefs) qc.setQueryData(QK.preferences, ctx.prevPrefs) + } + + const routeMut = useMutation({ + mutationFn: ({ field, provider }: { field: ProviderField; provider: string }) => + // 动态键经运行时字段名收敛为合法偏好键 (字段名来自后端注册表) + api.updateDataProviders({ [field]: provider } as Partial>), + onMutate: async (v) => { + setPendingKey(`${v.field}:${v.provider}`) + await qc.cancelQueries({ queryKey: QK.capabilityMatrix }) + await qc.cancelQueries({ queryKey: QK.preferences }) + return applyOptimistic({ [v.field]: v.provider }) + }, + onSuccess: (_d, v) => { + const cap = matrix.data?.capabilities.find(c => c.field === v.field) + toast(`「${cap?.label || v.field}」已切换为 ${displayOfName(matrix.data, v.provider)}`, 'success') + }, + onError: (e: Error, _v, ctx) => { + rollback(ctx) + toast(`路由切换失败: ${e.message}`, 'error') + }, + onSettled: () => { + setPendingKey(null) + invalidateRouting() + }, + }) + + const resetMut = useMutation({ + mutationFn: () => api.updateDataProviders(DEFAULT_ROUTING), + onMutate: async () => { + await qc.cancelQueries({ queryKey: QK.capabilityMatrix }) + await qc.cancelQueries({ queryKey: QK.preferences }) + return applyOptimistic(DEFAULT_ROUTING) + }, + onSuccess: () => toast('能力路由已恢复默认', 'success'), + onError: (e: Error, _v, ctx) => { + rollback(ctx) + toast(`恢复失败: ${e.message}`, 'error') + }, + onSettled: invalidateRouting, + }) + + const list = matrix.data?.capabilities ?? [] + const anyCustom = list.some(c => c.current !== c.default) + + return ( +
+
+
+ +

能力路由

+ {list.length} 个能力 +
+ {anyCustom && ( + + )} +
+

+ 每个能力独立选择提供方 — 点标签即刻生效。选项只列出当前可提供该能力的源 + (各源按自身可用性过滤, 详见下方数据源介绍); 未就绪的源置灰提示。 +

+ + {matrix.isError ? ( +
+
+ + 能力矩阵加载失败: {(matrix.error as Error)?.message || '未知错误'} +
+ +
+ ) : matrix.isLoading || list.length === 0 ? ( +
+ {Array.from({ length: 5 }).map((_, i) => ( +
+ ))} +
+ ) : ( +
+ {list.map(cap => ( + routeMut.mutate({ field, provider })} + /> + ))} +
+ )} +
+ ) +} + +/** 插件 API Key 配置区块 (嵌入插件详情卡, 不再独立成卡)。 * 先探后存: 后端用候选 Key 实探一次, 无效不落盘; secrets.json 优先于 .env。 */ function PluginKeyConfig({ plugin }: { plugin: PluginDataSourceItem }) { const qc = useQueryClient() @@ -238,6 +388,7 @@ function PluginKeyConfig({ plugin }: { plugin: PluginDataSourceItem }) { const invalidate = () => { qc.invalidateQueries({ queryKey: QK.dataSources }) + qc.invalidateQueries({ queryKey: QK.capabilityMatrix }) qc.invalidateQueries({ queryKey: QK.capabilities }) qc.invalidateQueries({ queryKey: QK.quoteStatus }) } @@ -272,15 +423,33 @@ function PluginKeyConfig({ plugin }: { plugin: PluginDataSourceItem }) { }) return ( -
-
- -

API Key

+
+
+ +

API Key

{plugin.api_key_env}
+ + {/* 申请说明 + 官网链接 (对齐 TickFlow Key 区话术) */}

- Key 保存为本地文件(secrets.json, 优先级高于 .env),不会上传任何第三方。保存前会先用该 Key - 实探一次数据接口,无效则不落盘。 + {plugin.homepage ? ( + <> + 在{' '} + + {plugin.display_name} 官网 + + + {' '}申请获取。 + + ) : ( + <>向该数据源官方申请 API Key。 + )} + Key 仅存本地 (secrets.json 优先, 环境变量 {plugin.api_key_env} 兜底),不会上传任何第三方,请妥善保管。

{/* 当前状态 */} @@ -292,8 +461,11 @@ function PluginKeyConfig({ plugin }: { plugin: PluginDataSourceItem }) { <> 已配置 - {save.data?.ok && save.data.api_key_masked && ( - {save.data.api_key_masked} + {/* 生效 Key 脱敏串 (secrets.json 优先, .env 兜底) — 常驻显示, 与 TickFlow Key 一致 */} + {plugin.api_key_masked && ( + + {plugin.api_key_masked} + )} ) : ( @@ -344,7 +516,7 @@ function PluginKeyConfig({ plugin }: { plugin: PluginDataSourceItem }) {
)} -
+
) } -export function SettingsDataSourcesPanel() { +/** 能力芯片: 三态 — 服务中(高亮+勾) / 已适配(灰) / 档位锁定(锁, 仅 TickFlow) */ +function CapabilityChips({ caps, servingSet, isTickFlow }: { + caps: CapabilityRoute[] + servingSet: Set + isTickFlow: boolean +}) { + if (caps.length === 0) return 未声明能力 + return ( +
+ {caps.map(cap => { + const servingNow = servingSet.has(cap.id) + const locked = isTickFlow && !cap.tf_available + const cls = servingNow + ? 'bg-accent/15 text-accent' + : locked + ? 'bg-warning/8 text-warning/70' + : 'bg-elevated/60 text-muted/70' + const title = servingNow + ? `正在提供「${cap.label}」` + : locked + ? `TickFlow 需 ${tierReqText(cap.tf_tier)} · 当前档位未解锁` + : `已适配「${cap.label}」` + return ( + + {servingNow + ? + : locked ? : null} + {DATASET_LABEL[cap.id] || cap.id} + + ) + })} +
+ ) +} + +export function SettingsDataSourcesPanel({ highlight }: { highlight?: string } = {}) { const qc = useQueryClient() const prefs = usePreferences() const sources = useQuery({ queryKey: QK.dataSources, queryFn: api.dataSources }) - const [selected, setSelected] = useState('tickflow') // 当前在右侧编辑的源 name + const matrix = useQuery({ queryKey: QK.capabilityMatrix, queryFn: api.capabilityMatrix }) + const [selected, setSelected] = useState('tickflow') // 当前在下方配置的源 name const [confirmDelete, setConfirmDelete] = useState(null) + const builtin: DataSourceItem[] = sources.data?.builtin ?? [] + const pluginList: PluginDataSourceItem[] = sources.data?.plugins ?? [] + const customList: DataSourceItem[] = sources.data?.custom ?? [] + const errors = sources.data?.errors ?? [] + const activeName = prefs.data?.daily_data_provider || 'tickflow' + + const pluginItems: DataSourceItem[] = pluginList.map(p => ({ + name: p.name, display_name: p.display_name, datasets: p.datasets, + })) + const allItems = [ + ...builtin, + ...pluginItems, + ...customList, + ] + const customNames = new Set(customList.map(c => c.name)) + + const selectedCustom = customList.find(s => s.name === selected) + + // ===== 各能力当前的有效提供方 (除权 same_as_daily = 跟随日K) ===== + // 用于能力芯片"服务中"态与详情卡标识: 路由切换在上方能力路由区, 这里只读展示 + const dailyPref = prefs.data?.daily_data_provider || 'tickflow' + const adjPref = prefs.data?.adj_factor_provider || 'same_as_daily' + const effProvider: Record = { + daily: dailyPref, + adj_factor: adjPref === 'same_as_daily' ? dailyPref : adjPref, + minute: prefs.data?.minute_data_provider || 'tickflow', + realtime: prefs.data?.realtime_data_provider || 'tickflow', + depth5: prefs.data?.depth5_data_provider || 'tickflow', + financial: prefs.data?.financial_data_provider || 'tickflow', + } + const servingDatasets = (name: string) => { + const ids = Object.entries(effProvider).filter(([, v]) => v === name).map(([k]) => k) + if (name === 'tickflow') { + // 不可路由能力 (field=null, 如全量分钟): 仅 TickFlow 提供, usable 即服务中 + ids.push(...(matrix.data?.capabilities ?? []) + .filter(c => c.field == null && c.usable).map(c => c.id)) + } + return ids + } + const servingSetOf = (name: string) => new Set(servingDatasets(name)) + + const matrixCaps = matrix.data?.capabilities ?? [] + + // 数据源集合/插件可用性/档位变化都会改变能力候选集 → 统一连带失效 + const invalidateSources = () => { + qc.invalidateQueries({ queryKey: QK.dataSources }) + qc.invalidateQueries({ queryKey: QK.capabilityMatrix }) + qc.invalidateQueries({ queryKey: QK.preferences }) + qc.invalidateQueries({ queryKey: QK.capabilities }) + qc.invalidateQueries({ queryKey: QK.quoteStatus }) + } + const reload = useMutation({ mutationFn: api.reloadDataSources, onSuccess: () => { - qc.invalidateQueries({ queryKey: QK.dataSources }) - // 重载可能改变数据集声明 → 能力与实时模式随之变化 - qc.invalidateQueries({ queryKey: QK.capabilities }) - qc.invalidateQueries({ queryKey: QK.quoteStatus }) + invalidateSources() toast('配置已重新加载', 'success') }, }) @@ -388,10 +649,7 @@ export function SettingsDataSourcesPanel() { const remove = useMutation({ mutationFn: (name: string) => api.deleteDataSource(name), onSuccess: () => { - qc.invalidateQueries({ queryKey: QK.dataSources }) - qc.invalidateQueries({ queryKey: QK.preferences }) - qc.invalidateQueries({ queryKey: QK.capabilities }) - qc.invalidateQueries({ queryKey: QK.quoteStatus }) + invalidateSources() setSelected('tickflow') setConfirmDelete(null) toast('数据源已删除', 'success') @@ -400,37 +658,29 @@ export function SettingsDataSourcesPanel() { const switchProvider = useMutation({ mutationFn: async (name: string) => { - // tickflow: 全量重置为默认路由 + // 一键套用: 该源适配了哪些数据集就接管哪些, 其余回默认 if (name === 'tickflow') { - return api.updateDataProviders({ - daily_data_provider: 'tickflow', - adj_factor_provider: 'same_as_daily', - realtime_data_provider: 'tickflow', - minute_data_provider: 'tickflow', - financial_data_provider: 'tickflow', - }) + return api.updateDataProviders(DEFAULT_ROUTING) } - // 非 tickflow: 该源适配了哪些数据集就接管哪些, 其余回退默认 const supported = new Set( allItems.find(s => s.name === name)?.datasets ?? [] ) const pick = (dataset: string) => - supported.has(dataset) ? name : (dataset === 'adj_factor' ? 'same_as_daily' : 'tickflow') + supported.has(dataset) ? name : DEFAULT_ROUTING[`${dataset}_data_provider` as ProviderField] ?? 'tickflow' return api.updateDataProviders({ daily_data_provider: pick('daily'), adj_factor_provider: pick('adj_factor'), realtime_data_provider: pick('realtime'), minute_data_provider: pick('minute'), - financial_data_provider: 'tickflow', + financial_data_provider: pick('financial'), }) }, - onSuccess: () => { - qc.invalidateQueries({ queryKey: QK.preferences }) - qc.invalidateQueries({ queryKey: QK.capabilities }) - // 切换会改变实时行情 provider → 模式(none/watchlist/full_market)立即刷新 - qc.invalidateQueries({ queryKey: QK.quoteStatus }) - toast('数据源已切换', 'success') + onSuccess: (_d, name) => { + invalidateSources() + const display = allItems.find(s => s.name === name)?.display_name || name + toast(`已让「${display}」接管其适配的能力`, 'success') }, + onError: (e: Error) => toast(`切换失败: ${e.message}`, 'error'), }) const editExisting = useMutation({ @@ -441,9 +691,7 @@ export function SettingsDataSourcesPanel() { const installMut = useMutation({ mutationFn: (name: string) => api.installPlugin(name), onSuccess: (data) => { - qc.invalidateQueries({ queryKey: QK.dataSources }) - qc.invalidateQueries({ queryKey: QK.capabilities }) - qc.invalidateQueries({ queryKey: QK.quoteStatus }) + invalidateSources() if (data.install_ok) { toast('插件依赖安装成功', 'success') } else { @@ -456,10 +704,7 @@ export function SettingsDataSourcesPanel() { const uninstallMut = useMutation({ mutationFn: (name: string) => api.uninstallPlugin(name), onSuccess: (data) => { - qc.invalidateQueries({ queryKey: QK.dataSources }) - qc.invalidateQueries({ queryKey: QK.preferences }) - qc.invalidateQueries({ queryKey: QK.capabilities }) - qc.invalidateQueries({ queryKey: QK.quoteStatus }) + invalidateSources() if (data.uninstall_ok) { toast(data.uninstall_message || '已卸载', 'success') } else { @@ -469,126 +714,20 @@ export function SettingsDataSourcesPanel() { onError: (e: Error) => toast(`卸载失败: ${e.message}`, 'error'), }) - const builtin: DataSourceItem[] = sources.data?.builtin ?? [] - const pluginList: PluginDataSourceItem[] = sources.data?.plugins ?? [] - const customList: DataSourceItem[] = sources.data?.custom ?? [] - const errors = sources.data?.errors ?? [] - const activeName = prefs.data?.daily_data_provider || 'tickflow' - // 插件 name → 状态 (供卡片渲染时判断 available/installing 等) const pluginMap = new Map(pluginList.map(p => [p.name, p])) - const pluginNames = new Set(pluginList.map(p => p.name)) - - // 顶部数据源选择列表 (内置 + 所有插件 + 自定义 + 新增) - const pluginItems: DataSourceItem[] = pluginList.map(p => ({ - name: p.name, display_name: p.display_name, datasets: p.datasets, - })) - const allItems = [ - ...builtin, - ...pluginItems, - ...customList, - ] - - const selectedCustom = customList.find(s => s.name === selected) - - // ===== 各数据集当前的有效提供方 (除权 same_as_daily = 跟随日K) ===== - // 用于"服务中"徽标: 改单个能力路由只影响对应数据集, 不再产生"当前数据源被切换"的表现 - const dailyPref = prefs.data?.daily_data_provider || 'tickflow' - const adjPref = prefs.data?.adj_factor_provider || 'same_as_daily' - const effProvider: Record = { - daily: dailyPref, - adj_factor: adjPref === 'same_as_daily' ? dailyPref : adjPref, - minute: prefs.data?.minute_data_provider || 'tickflow', - realtime: prefs.data?.realtime_data_provider || 'tickflow', - } - const servingDatasets = (name: string) => - Object.entries(effProvider).filter(([, v]) => v === name).map(([k]) => k) - const servingLabels = (name: string) => - servingDatasets(name).map(k => DATASET_LABEL[k] || k) - - const displayOf = (name?: string) => - name === 'tickflow' ? 'TickFlow' - : name === 'same_as_daily' ? '跟随日K' - : allItems.find(s => s.name === name)?.display_name || name || '' - - const invalidateRouting = () => { - qc.invalidateQueries({ queryKey: QK.preferences }) - qc.invalidateQueries({ queryKey: QK.capabilities }) - qc.invalidateQueries({ queryKey: QK.quoteStatus }) - } - - // 单个能力的提供方切换: 点标签即刻生效 (same_as_daily 仅除权有效 = 跟随日K) - const routeMut = useMutation({ - mutationFn: ({ dataset, provider }: { dataset: string; provider: string }) => { - const route = DATASET_ROUTE[dataset] - if (!route) throw new Error(`数据集 ${dataset} 不支持切换数据源`) - return api.updateDataProviders({ [route.field]: provider }) - }, - onSuccess: (_d, v) => { - invalidateRouting() - const label = DATASET_LABEL[v.dataset] || v.dataset - toast(`「${label}」已切换为 ${displayOf(v.provider)}`, 'success') - }, - onError: (e: Error) => toast(`路由切换失败: ${e.message}`, 'error'), - }) - - // 恢复默认: 路由全部回 TickFlow - const resetRouteMut = useMutation({ - mutationFn: () => api.updateDataProviders({ - daily_data_provider: 'tickflow', - adj_factor_provider: 'same_as_daily', - minute_data_provider: 'tickflow', - realtime_data_provider: 'tickflow', - }), - onSuccess: () => { - invalidateRouting() - toast('数据集路由已恢复默认(TickFlow)', 'success') - }, - onError: (e: Error) => toast(`恢复失败: ${e.message}`, 'error'), - }) - - const anyCustomRouting = Object.values(effProvider).some(v => v !== 'tickflow') || adjPref !== 'same_as_daily' - - // 某数据集的全部候选提供方 (TickFlow 恒在首位, 其余按声明该数据集的数据源列出) - const candidatesOf = (dataset: string) => { - const list = [{ name: 'tickflow', display: 'TickFlow' }] - for (const item of allItems) { - if (item.name !== 'tickflow' && item.datasets.includes(dataset)) { - list.push({ name: item.name, display: item.display_name || item.name }) - } - } - return list - } - - // 数据集 → 当前原始路由偏好值 (adj_factor 保留 same_as_daily 以驱动"跟随日K"标签态) - const providerOf = (dataset: string) => { - switch (dataset) { - case 'daily': return dailyPref - case 'adj_factor': return adjPref - case 'minute': return prefs.data?.minute_data_provider || 'tickflow' - case 'realtime': return prefs.data?.realtime_data_provider || 'tickflow' - default: return 'tickflow' - } - } - - // 传给各数据源详情的路由上下文(能力卡标签选择 + 当前提供方展示) - const routeCtx: RouteCtx = { - candidatesOf, - providerOf, - displayOf, - pending: routeMut.isPending, - onSelect: (dataset, provider) => routeMut.mutate({ dataset, provider }), - anyCustom: anyCustomRouting, - onReset: () => resetRouteMut.mutate(), - } return (
- {/* ===== 顶部: 当前数据源 + 数据源选择 (一个大卡片) ===== */} + {/* ===== 上区: 能力路由 (能力为主视图, 点标签切换提供方) ===== */} + + + {/* ===== 下区: 数据源 (源为主视图, 配置/接入) ===== */} +
-
- +
+

数据源

reload.mutate()} disabled={reload.isPending} - className="inline-flex items-center gap-1.5 px-2.5 py-1 rounded-btn text-xs text-muted hover:text-foreground hover:bg-elevated transition-colors disabled:opacity-50" + className="inline-flex items-center gap-1.5 px-2.5 py-1 rounded-btn text-xs text-muted hover:text-foreground hover:bg-elevated transition-colors disabled:opacity-50 shrink-0" > 重新加载 @@ -620,15 +759,28 @@ export function SettingsDataSourcesPanel() {
- {/* 数据源选择 - 横向卡片列表 */} + {/* 数据源卡片 - 横向网格 (配置入口) */}
{allItems.map(item => { - const serving = servingLabels(item.name) + const servingSet = servingSetOf(item.name) + const isTf = item.name === 'tickflow' + if (isTf) { + // 名义路由到 tickflow 但当前档位提供不了的能力 → 按锁定态展示, 不算服务中 + for (const c of matrixCaps) { + if (!c.tf_available) servingSet.delete(c.id) + } + } + const servingCount = servingSet.size const isSelected = selected === item.name const plugin = pluginMap.get(item.name) const pluginUnavailable = plugin && !plugin.available const installing = installMut.isPending && installMut.variables === item.name const uninstalling = uninstallMut.isPending && uninstallMut.variables === item.name + const declared = new Set(item.datasets) + // TickFlow 展示注册表全量能力 (含档位锁定态); 其余源按声明过滤 + const chipCaps = isTf + ? matrixCaps + : matrixCaps.filter(c => declared.has(c.id)) return (
c.name === item.name)) { + if (customNames.has(item.name)) { editExisting.mutate(item.name) } }} @@ -649,93 +801,82 @@ export function SettingsDataSourcesPanel() { : 'border-border/60 bg-elevated/20 hover:bg-elevated/40 cursor-pointer' }`} > -
+ {/* 名称行: 服务点 + 名称 + 档位/第三方标识 */} +
0 ? 'bg-accent' : 'bg-transparent border border-muted/40' + pluginUnavailable ? 'bg-muted/30' : servingCount > 0 ? 'bg-accent' : 'bg-transparent border border-muted/40' }`} /> - 0 ? 'font-medium text-foreground' : 'text-secondary'}`}> + 0 ? 'font-medium text-foreground' : 'text-secondary'}`}> {item.display_name} - {serving.length > 0 && ( - - 服务中 - + {isTf && matrix.data?.tickflow_tier && ( + )} - {item.name === 'tickflow' && ( - <> - 第三方 - - - )} - {pluginNames.has(item.name) && ( + {!customNames.has(item.name) && ( 第三方 )} - {/* 右侧操作区: 插件未安装→安装按钮(runtime=none 无依赖可装,显示配置提示); 否则→使用/卸载 */} - {pluginUnavailable ? ( - plugin?.runtime === 'none' ? ( - - 点击配置 Key - - ) : installing ? ( - - 安装中... - - ) : ( - - ) - ) : plugin ? ( - /* 已安装插件: 使用 + 卸载 */ -
- - {plugin?.runtime !== 'none' && (uninstalling ? ( - - ) : ( - - ))} -
- ) : ( - - )}
- {/* 数据集标签(静态): 该源适配的数据集, 路由选择在下方能力卡片 */} - - {/* 未安装插件显示安装命令提示 */} - {pluginUnavailable && plugin?.install_hint && ( -
{plugin.install_hint}
- )} + + {/* 能力芯片: 服务中 / 已适配 / 档位锁定 */} +
+ +
+ + {/* 底部: 状态提示 + 操作 */} +
+ + {servingCount > 0 + ? `服务中 ${servingCount} 项能力` + : pluginUnavailable + ? (plugin?.runtime === 'none' ? '点击配置 Key' : (plugin?.install_hint || plugin?.status || '')) + : ''} + +
+ {pluginUnavailable ? ( + plugin?.runtime !== 'none' && ( + installing ? ( + + 安装中... + + ) : ( + + ) + ) + ) : ( + <> + + {plugin && plugin?.runtime !== 'none' && ( + uninstalling ? ( + + ) : ( + + ) + )} + + )} +
+
) })} @@ -768,19 +909,16 @@ export function SettingsDataSourcesPanel() {
)} -
- 单击查看各源能力 +
+ 芯片: 高亮=服务中 · 灰=已适配 · =需更高档位 · - 能力卡片上点标签,单独选择每个数据集的提供方 - · - 点「使用」一键套用该源全部能力 - · - 未单独设置的由 TickFlow 提供 + 单击卡片查看介绍与配置, 点「套用」让该源接管其适配的全部能力
+ - {/* ===== 下方: 编辑区 ===== */} + {/* ===== 下方: 选中源的介绍 + 配置 (单一卡片布局) ===== */} {selected === 'tickflow' ? ( - 0} - onSwitch={() => switchProvider.mutate('tickflow')} - switching={switchProvider.isPending} - route={routeCtx} - /> - ) : selected === '__new__' || customList.some(c => c.name === selected) ? ( + 0} matrix={matrix.data} /> + ) : selected === '__new__' || customNames.has(selected) ? ( selected === '__new__' ? ( setSelected('tickflow')} onSaved={() => { - qc.invalidateQueries({ queryKey: QK.dataSources }) - // 数据集声明变化 → 能力增广与实时模式立即刷新 - qc.invalidateQueries({ queryKey: QK.preferences }) - qc.invalidateQueries({ queryKey: QK.capabilities }) - qc.invalidateQueries({ queryKey: QK.quoteStatus }) + invalidateSources() setSelected('tickflow') }} activeName={activeName} onActivate={(name) => switchProvider.mutate(name)} /> ) : ( - /* 自定义源详情: 能力卡片(路由) + 编辑器 */ -
-
-
- -

- {selectedCustom?.display_name || selected} · 数据集能力与路由 -

-
- -
- setSelected('tickflow')} - onSaved={() => { - qc.invalidateQueries({ queryKey: QK.dataSources }) - // 数据集声明变化 → 能力增广与实时模式立即刷新 - qc.invalidateQueries({ queryKey: QK.preferences }) - qc.invalidateQueries({ queryKey: QK.capabilities }) - qc.invalidateQueries({ queryKey: QK.quoteStatus }) - // 强制清除该源的详情缓存, 下次编辑重新拉取最新配置 - qc.removeQueries({ queryKey: ['data-source-detail', selected] }) - }} - activeName={activeName} - onActivate={(name) => switchProvider.mutate(name)} - onDelete={selectedCustom ? () => setConfirmDelete(selected) : undefined} - /> -
+ setSelected('tickflow')} + onSaved={() => { + invalidateSources() + // 强制清除该源的详情缓存, 下次编辑重新拉取最新配置 + qc.removeQueries({ queryKey: ['data-source-detail', selected] }) + }} + activeName={activeName} + onActivate={(name) => switchProvider.mutate(name)} + onDelete={selectedCustom ? () => setConfirmDelete(selected) : undefined} + /> ) ) : pluginList.find(x => x.name === selected) ? ( - /* 选中插件: 信息 + 能力卡片(路由) + Key 配置 */ x.name === selected)!} isActive={servingDatasets(selected).length > 0} - onSwitch={() => switchProvider.mutate(selected)} - switching={switchProvider.isPending} - route={routeCtx} + matrixCaps={matrixCaps} + servingSet={servingSetOf(selected)} /> ) : null} @@ -900,123 +1003,221 @@ export function SettingsDataSourcesPanel() { ) } -function PluginDetail({ plugin, isActive, onSwitch, switching, route }: { +/** 插件详情: 介绍 + 适配能力 + Key 配置合并为单一卡片 */ +function PluginDetail({ plugin, isActive, matrixCaps, servingSet }: { plugin: PluginDataSourceItem isActive: boolean - onSwitch: () => void - switching: boolean - route: RouteCtx + matrixCaps: CapabilityRoute[] + servingSet: Set }) { + const declared = new Set(plugin.datasets) return ( -
-
-
-
- +
+ {/* 介绍 */} +
+
+ +
+
+
+

{plugin.display_name}

+ 插件 · {plugin.runtime} + 第三方 + {isActive && ( + + 服务中 + + )}
-
-
-

{plugin.display_name}

- 插件 · {plugin.runtime} -
- {plugin.description &&

{plugin.description}

} + {plugin.description &&

{plugin.description}

} +
+ declared.has(c.id))} + servingSet={servingSet} + isTickFlow={false} + />
+
- {/* 本源参与的能力: 标签选择提供方, 即刻生效 */} - + {/* 主体: 操作 + Key 配置 (左) | 能力适配表 (右), 布局对齐 TickFlow 详情 */} +
+
+ {/* 独立状态行仅用于无 Key 配置区的插件; 有 Key 区时「状态」行已展示, 避免重复 */} + {!plugin.available && !plugin.api_key_env && ( +
+ {plugin.status} +
+ )} -
- {isActive ? ( - - 服务中 - - ) : plugin.available ? ( - - ) : ( - {plugin.status} + {/* API Key 配置 (声明了 api_key_env 的插件) */} + {plugin.api_key_env && ( +
+ +
)}
-
- {/* API Key 配置 (声明了 api_key_env 的插件) */} - {plugin.api_key_env && } -
- ) -} - -function TickFlowDetail({ active, onSwitch, switching, route }: { - active: boolean - onSwitch: () => void - switching: boolean - route: RouteCtx -}) { - return ( -
-
-
-
- -
-
-
-

TickFlow

- 第三方 - - {active && ( - - 服务中 - - )} -
-

- 默认数据源,具备全部能力(日K · 除权 · 分钟 · 实时 · 财务)。在下方每个能力卡片上点标签,可单独选择该数据集由哪个数据源提供。 -

+ {/* 能力适配表: 全部能力 × 该源适配状态 (样式对齐 TickFlow 能力档位表) */} +
+
+ {matrixCaps.map(cap => { + const Icon = CAP_ICON[cap.id] ?? Database + const serving = servingSet.has(cap.id) + const declaredCap = declared.has(cap.id) + return ( +
+ +
+ {cap.label} + {cap.desc} +
+ {serving ? ( + + 服务中 + + ) : declaredCap ? ( + + {plugin.available ? '已适配' : '未就绪'} + + ) : ( + — + )} +
+ ) + })}
+
+
+ ) +} - {/* 全部能力的提供方标签选择 + 恢复默认 */} - - - {!active && ( - - )} -
+/** TickFlow 详情: 介绍 + 能力档位表 + Key/可用功能左右两栏。 + * 检测档位集群 (档位徽章 + ? 说明 + 重新检测 + 可用功能悬停) 挂在 API Key 标题行右侧。 */ +function TickFlowDetail({ active, matrix }: { active: boolean; matrix?: CapabilityMatrix }) { + const caps = matrix?.capabilities ?? [] + const tier = matrix?.tickflow_tier + const { data: tfCaps } = useCapabilities() + const capEntries = tfCaps ? Object.entries(tfCaps.capabilities) : [] + const invalidate = useInvalidateTierRelated() + const redetect = useMutation({ + mutationFn: () => api.redetectCapabilities(), + onSuccess: () => invalidate(), + }) + // 检测档位集群: API Key 标题行右侧 (检测档位徽章 + 档位说明 + 重检测 + 可用功能悬停) + const tierCluster = tier ? ( +
+ 检测档位 + + + + {/* 可用功能: 收进悬停浮层, 不占版面 (能力清单 + 限频)。图标在标题行右侧, 浮层向左展开。 + 外层 top-full + pt-1.5: 间隙用内边距做, hover 区与图标无缝衔接 (mt 间隙会断 hover 链); + 不设 pointer-events-none, 否则鼠标移不进浮层、列表无法滚动。 */} +
+ + + +
+
+
+ 可用功能 + {capEntries.length} 项 +
+ {capEntries.length > 0 ? ( +
+ {capEntries.map(([cap, lim]) => ( +
+ + {CAP_LABELS[cap]?.name ?? cap} + + + {lim.rpm ? `${lim.rpm}/min` : lim.subscribe ? `${lim.subscribe} 订阅` : '—'} + {lim.batch ? ` · ${lim.batch}/次` : ''} + +
+ ))} +
+ ) : ( +
+ 暂无 — 配置 API Key 后自动检测 +
+ )} +
+ 根据 API Key 自动检测 +
+
+
+
+
+ ) : null + return ( +
+ {/* 介绍 */} +
+
+ +
+
+
+

TickFlow

+ 第三方 + + {active && ( + + 服务中 + + )} +
+

+ 默认数据源,每个能力所需订阅档位见下表 — 当前档位未解锁的能力不会出现在上方「能力路由」的选项里。 + 未单独设置的能力默认由 TickFlow 提供;也可在数据源区接入插件替换任意能力。 +

+
+
- {/* TickFlow API Key 配置 + 订阅档位 + 可用功能 (原 account tab 内容) */} - -
+ {/* 主体: API Key 配置 (左) | 能力档位表 (右) */} +
+
+ +
+
+ {/* 能力档位表: 各能力所需档位 + 当前档位可用性 */} + {caps.length > 0 && ( +
+ {caps.map(cap => { + const Icon = CAP_ICON[cap.id] ?? Database + return ( +
+ +
+ {cap.label} + {cap.desc} +
+ + {cap.tf_available ? ( + + 可用 + + ) : ( + 未解锁 + )} +
+ ) + })} +
+ )} +
+
+
) } diff --git a/frontend/src/pages/settings/Keys.tsx b/frontend/src/pages/settings/Keys.tsx index e4cd10a..f390d9a 100644 --- a/frontend/src/pages/settings/Keys.tsx +++ b/frontend/src/pages/settings/Keys.tsx @@ -8,8 +8,6 @@ import { Trash2, CheckCircle2, AlertCircle, - RefreshCw, - Activity, ExternalLink, Loader2, Save, @@ -17,17 +15,50 @@ import { HelpCircle, } from 'lucide-react' import { api } from '@/lib/api' -import { useCapabilities, useSettings } from '@/lib/useSharedQueries' +import { useSettings } from '@/lib/useSharedQueries' import { QK } from '@/lib/queryKeys' -import { CAP_LABELS, tierTextStyle, tierStyle, tierBaseName, ALL_TIERS, TierTag } from '@/lib/capability-labels' +import { tierStyle, tierBaseName, ALL_TIERS, TierTag } from '@/lib/capability-labels' -// ===== TickFlow Key 配置主体 (可嵌入 DataSources 的 TickFlow 详情区) ===== +// ===== TickFlow 详情内嵌区块 (组合进数据源页的单一详情卡, 不再各自成卡) ===== +// 档位变化会重塑能力矩阵候选 (按档位过滤) → Key/档位相关写操作统一连带失效。 -export function TickFlowKeyConfig() { +/** 区块小标题 (详情卡内部的分节, 区别于页面级 section) */ +function SectionHeading({ icon: Icon, title, badge, right }: { + icon: React.ComponentType<{ className?: string }> + title: string + badge?: string + right?: React.ReactNode +}) { + return ( +
+
+ +

{title}

+ {badge && ( + {badge} + )} +
+ {right} +
+ ) +} + +/** Key/档位/能力矩阵/侧栏状态的统一连带失效 (档位变化重塑矩阵候选) */ +export function useInvalidateTierRelated() { const qc = useQueryClient() + return () => { + qc.invalidateQueries({ queryKey: QK.settings }) + qc.invalidateQueries({ queryKey: QK.capabilities }) + qc.invalidateQueries({ queryKey: QK.capabilityMatrix }) + // 档位变化会改变实时行情模式(none/watchlist/full_market), 立即刷新侧边栏状态 + qc.invalidateQueries({ queryKey: QK.quoteStatus }) + } +} +/** API Key 配置区块: 状态 + 输入 + 保存并检测 (先探后存) */ +export function TickFlowKeySection({ right }: { right?: React.ReactNode }) { const settings = useSettings() - const caps = useCapabilities() + const invalidate = useInvalidateTierRelated() const [keyInput, setKeyInput] = useState('') const [revealing, setRevealing] = useState(false) @@ -38,10 +69,7 @@ export function TickFlowKeyConfig() { mutationFn: () => api.saveTickflowKey(keyInput.trim()), onSuccess: (data) => { setKeyInput('') - qc.invalidateQueries({ queryKey: QK.settings }) - qc.invalidateQueries({ queryKey: QK.capabilities }) - // 档位变化会改变实时行情模式(none/watchlist/full_market), 立即刷新侧边栏状态 - qc.invalidateQueries({ queryKey: QK.quoteStatus }) + invalidate() if (data.ok) { setSaved(true) setTimeout(() => setSaved(false), 2000) @@ -52,266 +80,148 @@ export function TickFlowKeyConfig() { const clear = useMutation({ mutationFn: () => api.clearTickflowKey(), - onSuccess: () => { - qc.invalidateQueries({ queryKey: QK.settings }) - qc.invalidateQueries({ queryKey: QK.capabilities }) - qc.invalidateQueries({ queryKey: QK.quoteStatus }) - }, + onSuccess: () => invalidate(), }) const redetect = useMutation({ - mutationFn: api.redetectCapabilities, - onSuccess: () => { - qc.invalidateQueries({ queryKey: QK.settings }) - qc.invalidateQueries({ queryKey: QK.capabilities }) - qc.invalidateQueries({ queryKey: QK.quoteStatus }) - }, + mutationFn: () => api.redetectCapabilities(), + onSuccess: () => invalidate(), }) const mode = settings.data?.mode const masked = settings.data?.tickflow_api_key_masked - const capCount = caps.data ? Object.keys(caps.data.capabilities).length : 0 return ( - <> -
- {/* ========== 左列: Key 配置 ========== */} -
- -

- 在{' '} - - tickflow.org - - {' '} - 注册获取。API Key 存放为本地文件,不会上传任何第三方,请妥善保管。 -

+
+ - {/* 当前状态 */} -
-
-
状态
-
- {mode === 'api_key' ? ( - <> - - 已配置 - {masked} - - ) : mode === 'free' ? ( - <> - - 免费 Key - {masked} - - ) : ( - <> - - 未配置 - - )} -
-
- {(mode === 'api_key' || mode === 'free') && ( - - )} -
+

+ 在{' '} + + tickflow.org + + {' '} + 注册获取。API Key 存放为本地文件,不会上传任何第三方,请妥善保管。 +

- {/* 输入 */} -
{ - e.preventDefault() - if (keyInput.trim()) save.mutate() - }} - className="space-y-2" - > -
- { setKeyInput(e.target.value); if (saved) setSaved(false) }} - className="w-full px-3 py-2 pr-9 rounded-input bg-base border border-border text-sm font-mono focus:outline-none focus:border-accent transition-colors duration-150 ease-smooth" - /> - -
- - - {/* 检测中提示 —— 成功/失败后自动消失 */} - {save.isPending && ( -
- - - 验证通过前请不要离开当前页面 · 如遇网络问题请点击 - - -
- )} -
- - {save.isError && ( -
- 保存失败:{String((save.error as any).message)} -
- )} - {/* 无效 key —— 先探后存:探测失败(key 无效/乱填)时不存储,提示用户 */} - {save.data && !save.data.ok && ( -
- - {save.data.reason === 'invalid' - ? 'Key 无效或已过期,请检查后重试(未保存该 Key)' - : save.data.error ?? '保存失败'} -
- )} - {save.data?.ok && ( -
- - 保存成功 — 档位 {save.data.tier_label} - {save.data.mode === 'free' && '(免费档 · 历史日K + 自选实时监控)'} -
- )} - -
- - {/* ========== 右列: 档位 + 能力 ========== */} -
- redetect.mutate()} - disabled={redetect.isPending} - className="inline-flex items-center gap-1.5 px-2.5 py-1 rounded-btn bg-elevated hover:bg-elevated/80 text-xs text-secondary transition-colors duration-150 ease-smooth disabled:opacity-50" - > - - 重新检测 - - } - > - {caps.data ? ( + {/* 当前状态 */} +
+
+
状态
+
+ {mode === 'api_key' ? ( <> -
-
- {caps.data.label} -
- -
-
- 根据 API Key 自动检测 · 拥有"代表性 capability"任一即认为该档 -
- - {settings.data?.missing_caps && settings.data.missing_caps.length > 0 && ( -
-
- 本档应有但未探测到({settings.data.missing_caps.length} 项) -
-
- {settings.data.missing_caps.map((c) => ( -
- {CAP_LABELS[c]?.name ?? c} -
- ))} -
-
- )} + + 已配置 + {masked} + + ) : mode === 'free' ? ( + <> + + 免费 Key + {masked} ) : ( -
加载中…
+ <> + + 未配置 + )} - - - - {caps.data && ( - -
- {Object.entries(caps.data.capabilities).map(([cap, lim]) => { - const meta = CAP_LABELS[cap] - return ( -
-
-
- {meta?.name ?? cap} -
- {meta?.hint && ( -
- {meta.hint} -
- )} -
-
-
- {lim.rpm ? `${lim.rpm}/min` : lim.subscribe ? `${lim.subscribe} 订阅` : '—'} -
- {lim.batch && ( -
{lim.batch} 只/次
- )} -
-
- ) - })} -
- - )} - - {settings.data?.probe_log && settings.data.probe_log.length > 0 && ( -
- - 查看检测日志 - -
- {settings.data.probe_log.map((line, i) => ( -
{line}
- ))} -
-
- )} - +
+ {(mode === 'api_key' || mode === 'free') && ( + + )}
+ {/* 输入 + 保存: 左右一行 */} +
{ + e.preventDefault() + if (keyInput.trim()) save.mutate() + }} + className="flex items-center gap-2" + > +
+ { setKeyInput(e.target.value); if (saved) setSaved(false) }} + className="w-full px-3 py-2 pr-9 rounded-input bg-base border border-border text-sm font-mono focus:outline-none focus:border-accent transition-colors duration-150 ease-smooth" + /> + +
+ +
+ + {/* 检测中提示 —— 成功/失败后自动消失 */} + {save.isPending && ( +
+ + + 验证通过前请不要离开当前页面 · 如遇网络问题请点击 + + +
+ )} + + {save.isError && ( +
+ 保存失败:{String((save.error as any).message)} +
+ )} + {/* 无效 key —— 先探后存:探测失败(key 无效/乱填)时不存储,提示用户 */} + {save.data && !save.data.ok && ( +
+ + {save.data.reason === 'invalid' + ? 'Key 无效或已过期,请检查后重试(未保存该 Key)' + : save.data.error ?? '保存失败'} +
+ )} + {save.data?.ok && ( +
+ + 保存成功 — 档位 {save.data.tier_label} + {save.data.mode === 'free' && '(免费档 · 历史日K + 自选实时监控)'} +
+ )} + {/* 确认清除 Key 弹窗 */} {confirmClear && (
@@ -342,15 +252,13 @@ export function TickFlowKeyConfig() {
)} - +
) } -// ===== 通用卡片 ===== - // ===== 档位说明弹窗 ===== -function TierHelpPopover({ currentLabel }: { currentLabel: string }) { +export function TierHelpPopover({ currentLabel }: { currentLabel: string }) { const [open, setOpen] = useState(false) const currentBase = tierBaseName(currentLabel) @@ -412,32 +320,3 @@ function TierHelpPopover({ currentLabel }: { currentLabel: string }) {
) } - - -interface CardProps { - icon: React.ComponentType<{ className?: string }> - title: string - badge?: string - right?: React.ReactNode - children: React.ReactNode -} - -function Card({ icon: Icon, title, badge, right, children }: CardProps) { - return ( -
-
-
- -

{title}

- {badge && ( - - {badge} - - )} -
- {right} -
- {children} -
- ) -} diff --git a/frontend/src/pages/settings/Monitoring.tsx b/frontend/src/pages/settings/Monitoring.tsx index 94607b1..44b6b80 100644 --- a/frontend/src/pages/settings/Monitoring.tsx +++ b/frontend/src/pages/settings/Monitoring.tsx @@ -1,5 +1,4 @@ -import { useState, useCallback, useEffect, useRef } from 'react' -import { Link } from 'react-router-dom' +import { useState, useCallback, useEffect, createContext, useContext } from 'react' import { useQueryClient, useMutation, useQuery } from '@tanstack/react-query' import { Activity, @@ -19,9 +18,14 @@ import { import { useUpdateQuoteInterval, useToggleRealtimeQuotes } from '@/lib/useSharedMutations' import { api } from '@/lib/api' import { QK } from '@/lib/queryKeys' +import { useCardFlash, cardFlashCls } from '@/lib/useCardFlash' import { toast } from '@/components/Toast' import { DepthConfigContent } from '@/components/data/DepthConfigCard' +// 卡片定位锚点: highlight= 时该卡片滚动到视口中央并闪烁高亮。 +// 其他页面用 /settings?tab=monitoring&highlight= 精确引导用户到某张卡片。 +const HighlightContext = createContext('') + // 页面 → 显示名 const PAGE_LABELS: Record = { 'overview-market': '看板', @@ -46,17 +50,28 @@ export function SettingsMonitoringPanel({ highlight }: { highlight?: string } = const { data: intervalData } = useQuoteInterval() const updateInterval = useUpdateQuoteInterval() const toggleQuote = useToggleRealtimeQuotes() - // 实时模式以 quote_status 为准 (数据源无关): watchlist=自选实时 / full_market=全市场 / none=不可用 - const quoteMode = quoteStatus?.mode ?? 'none' - const isWatchlistMode = quoteMode === 'watchlist' + // 实时模式以 quote_status 为准 (数据源无关): full_market=全市场 / none=不可用 const realtimeEnabled = prefs?.realtime_quotes_enabled ?? false // 分时图实时刷新间隔 (秒), 与后端 [3,60] clamp 对齐; 默认 6 const intradayInterval = prefs?.minute_intraday_refresh_interval ?? 6 // 滑块本地草稿: 拖动时即时反馈, 停顿 2s 后落库 (与行情轮询滑块一致) const [intradayIntervalDraft, setIntradayIntervalDraft] = useState(intradayInterval) + // 盘中分钟增量 (Expert 专有): 间隔 (秒), 与后端 [3,300] clamp 对齐; 默认 6 + const minuteRefreshInterval = prefs?.minute_refresh_interval ?? 6 + const [minuteRefreshIntervalDraft, setMinuteRefreshIntervalDraft] = useState(minuteRefreshInterval) + // 盘中增量服务状态 (15s 轮询; 无服务时 available=false) + const refreshStatus = useQuery({ + queryKey: ['minute-refresh-status'], + queryFn: api.minuteRefreshStatus, + refetchInterval: 15000, + }) const refreshPages = prefs?.sse_refresh_pages ?? {} const limitLadderMonitor = prefs?.limit_ladder_monitor_enabled ?? false const hasDepth = !!caps?.capabilities?.['depth5.batch'] + // 全量分钟 = intraday.universe 能力 (TickFlow Expert 专有): 标的池单请求拉全市场当日分钟, + // 修复轮的 intraday.batch 与其同档, 见后端 minute_refresh 服务 + const hasFullMinuteCap = !!caps?.capabilities?.['intraday.universe'] + const rs = refreshStatus.data // 新建监控规则时默认勾选的推送渠道 (全局默认值数组, 单条规则可独立修改) const webhookDefaultChannels = prefs?.webhook_default_channels ?? [] const sidebarIndexSymbols = prefs?.sidebar_index_symbols ?? SIDEBAR_INDEX_OPTIONS.map(i => i.symbol) @@ -103,15 +118,6 @@ export function SettingsMonitoringPanel({ highlight }: { highlight?: string } = setBotIdDraft(wecomBotId) setBotSecretDraft(wecomBotSecret) }, [wecomBotId, wecomBotSecret]) - const watchlistSymbols = prefs?.realtime_watchlist_symbols ?? [] - const watchlist = useQuery({ - queryKey: QK.watchlist, - queryFn: () => api.watchlistList(), - enabled: isWatchlistMode && watchlistSymbols.length > 0, - }) - const watchlistNameBySymbol = new Map( - (watchlist.data?.symbols ?? []).map(row => [row.symbol, row.name] as const), - ) const save = useCallback(async (cfg: Record) => { try { @@ -262,27 +268,27 @@ export function SettingsMonitoringPanel({ highlight }: { highlight?: string } = return () => window.clearTimeout(t) }, [intradayIntervalDraft, intradayInterval, save]) - // highlight=depth-fix 时闪烁高亮连板梯队修正卡片 - const [flash, setFlash] = useState(false) - const flashedRef = useRef(false) + // 盘中增量间隔: 服务端值变化时同步本地草稿 useEffect(() => { - if (highlight === 'depth-fix' && !flashedRef.current) { - flashedRef.current = true - // 延迟一帧确保 DOM 已渲染, 再触发闪烁 - requestAnimationFrame(() => { - setFlash(true) - const t = setTimeout(() => setFlash(false), 2000) - return () => clearTimeout(t) - }) - } - }, [highlight]) + setMinuteRefreshIntervalDraft(minuteRefreshInterval) + }, [minuteRefreshInterval]) + + // 盘中增量间隔: 草稿与已保存值不同时, 2s 防抖落库 + useEffect(() => { + if (minuteRefreshIntervalDraft === minuteRefreshInterval) return + const t = window.setTimeout(() => { + save({ minute_refresh_interval: minuteRefreshIntervalDraft }) + }, 2000) + return () => window.clearTimeout(t) + }, [minuteRefreshIntervalDraft, minuteRefreshInterval, save]) return ( +
{/* ========== 左列 ========== */}
{/* 行情状态 — 开关 + 间隔 */} - +
轮询间隔
- {isWatchlistMode ? '每轮拉取自选股实时行情的时间间隔' : '每轮拉取全市场行情的时间间隔'} + 每轮拉取全市场行情的时间间隔
@@ -325,43 +331,6 @@ export function SettingsMonitoringPanel({ highlight }: { highlight?: string } =
- {isWatchlistMode && ( - -
- 自选实时模式下自动监控「自选」页面前 5 个标的，最低 6 秒刷新。 -
- {watchlistSymbols.length > 0 ? ( -
- {watchlistSymbols.map(symbol => { - const name = watchlistNameBySymbol.get(symbol) - return ( -
-
- {symbol} - {name && {name}} -
- 自选页 -
- ) - })} -
- ) : ( -
- 自选列表为空，开启自选实时前请先添加自选股。 -
- )} -
- 当前 {watchlistSymbols.length}/5 只 - - 管理自选 - -
- - )} - {!isWatchlistMode && (

选择哪些页面跟随 SSE 实时刷新数据。关闭的页面不会被推送， @@ -379,10 +348,9 @@ export function SettingsMonitoringPanel({ highlight }: { highlight?: string } = ))}

- )} {/* 自选列表分时图实时刷新 (默认关闭, 开启后盘中按设定间隔轮询刷新分时数据) */} - + - {!isWatchlistMode && (

选择实时行情开启时，左侧菜单底部显示哪些指数点位和涨跌幅。 @@ -443,19 +410,15 @@ export function SettingsMonitoringPanel({ highlight }: { highlight?: string } = />

- )}
{/* ========== 右列 ========== */}
- {/* 连板梯队降级修正 (移至右列顶部) */} -
+ {/* 连板梯队降级修正 (右列顶部) */} )} -
+ + {/* 全量分钟 (TickFlow Expert 专有): 盘中全市场分钟落盘, intraday.universe 单请求增量 */} + + save({ minute_refresh_enabled: v })} + disabled={!hasFullMinuteCap || !!rs?.custom_provider_active} + /> +
+
+
+
刷新间隔
+
+ 交易时段内全市场分钟K增量落盘的间隔; 稳态单请求增量, 冷启动/断档自动全天回补 +
+
+ + {minuteRefreshIntervalDraft >= 60 && minuteRefreshIntervalDraft % 60 === 0 ? `${minuteRefreshIntervalDraft / 60}m` : `${minuteRefreshIntervalDraft}s`} + +
+
+ setMinuteRefreshIntervalDraft(parseInt(e.target.value, 10))} + className="flex-1 h-1 accent-accent cursor-pointer disabled:opacity-40 disabled:cursor-not-allowed" + /> + + {minuteRefreshIntervalDraft !== minuteRefreshInterval ? '2秒后保存' : '3s — 300s'} + +
+ {rs?.available && rs.rounds != null && rs.rounds > 0 && ( +
+ 已 {rs.rounds} 轮 · 最近 {rs.last_symbols} 标的 / {rs.last_rows} 行 / {rs.last_requests} 请求 + {rs.last_round_ms != null ? ` · ${(rs.last_round_ms / 1000).toFixed(1)}s` : ''} + {rs.last_error ? ` · ${rs.last_error}` : ''} +
+ )} +
+ {/* 推送通知 — 监控告警的外部推送渠道 (全局配置)。 飞书 / 企业微信。 每个渠道合并成一行: 勾选=新建规则默认推送, 点行展开地址配置。 */} - +

监控规则命中后,可把告警推送到外部。勾选渠道作为新建规则的默认推送, 单条规则仍可在编辑页独立修改。 @@ -764,6 +777,7 @@ export function SettingsMonitoringPanel({ highlight }: { highlight?: string } =

+ ) } @@ -822,8 +836,10 @@ interface CardProps { children: React.ReactNode } -function Card({ icon: Icon, title, badge, right, children }: CardProps) { - return ( +function Card({ icon: Icon, title, badge, right, children, anchor }: CardProps & { anchor?: string }) { + const highlight = useContext(HighlightContext) + const { ref, flash } = useCardFlash(anchor ? highlight : undefined, anchor ?? '') + const inner = (
@@ -840,4 +856,10 @@ function Card({ icon: Icon, title, badge, right, children }: CardProps) { {children}
) + if (!anchor) return inner + return ( +
+ {inner} +
+ ) } diff --git a/tiers.yaml b/tiers.yaml index 46139b0..7ba410c 100644 --- a/tiers.yaml +++ b/tiers.yaml @@ -66,6 +66,7 @@ expert: kline.minute.by_symbol: { rpm: 120, batch: 1 } intraday: { rpm: 120, batch: 1 } intraday.batch: { rpm: 60, batch: 200 } + intraday.universe: { rpm: 20 } # 全量分钟增量: 标的池单请求拉全市场; rpm=20 对应服务 3s 下限 depth5: { rpm: 120, batch: 1 } # 按标的查(单只):官方 120rpm/1 depth5.batch: { rpm: 60, batch: 200 } # 批量查(新增):官方 60rpm/200 adj_factor: { rpm: 120, batch: 200 }