mirror of
https://ghfast.top/https://github.com/aeroxw/tick-stock-panel.git
synced 2026-09-12 15:34:16 +08:00
Merge branch 'feat/capability-routing' (v0.2.2): 能力路由矩阵、fuyao 插件、分钟策略回测、盘中/复盘增强、首用引导改版
This commit is contained in:
shy3130
@@ -106,6 +106,7 @@
|
||||
|
||||
- 窗口、前 N 日和批次回算均按实际交易日，不得用自然日直接替代。
|
||||
- A 股交易时段统一按北京时间处理；服务器时区不能成为业务逻辑的隐式输入。
|
||||
- 分钟 K 的 `datetime` 统一为北京时间墙钟（naive，如 `09:35:00`）；数据源入口（`kline_sync` 的 `_normalize_minute` / `_try_custom_minute`）强制归一，禁止 UTC 口径入库或下发。
|
||||
- 日线、分钟线和实时快照必须明确交易日期归属，尤其注意午休、收盘后和跨日重启。
|
||||
- 分钟 K 的股票、ETF、指数分开存储和路由，不得仅凭代码格式猜测资产类型。
|
||||
|
||||
@@ -129,6 +130,16 @@
|
||||
- 插件加载失败、字段缺失和空数据必须有隔离测试，不能导致应用启动失败或其他数据源不可用。
|
||||
- 新增数据源适配应同步更新 `docs/custom-data-source.md` 或 `docs/plugin-development.md` 中对应契约。
|
||||
|
||||
### 能力路由矩阵契约
|
||||
|
||||
能力矩阵（`backend/app/data_providers/capabilities.py` 注册表 + `/api/settings/capability-matrix`）是能力路由的单一权威，遵循以下不变量：
|
||||
|
||||
- 注册表集中声明每个能力的展示元数据、路由偏好字段与 TickFlow 档位要求；前端不硬编码能力清单。新增能力按既有模式扩展：注册表 + preferences getter + capability-matrix 注入 + 矩阵测试。
|
||||
- 各页面能力门控统一以矩阵的 `usable` 为准（生效源当前能否真正提供该能力），不是 TickFlow 套餐视角；缺能力提示统一引导到数据源配置。
|
||||
- 能力层中立：通用界面（侧栏徽章、能力路由卡、各页门控提示）不得出现 TickFlow 档位/订阅词汇；档位信息只在 TickFlow 专属详情卡展示。provider 名称作为路由事实可以出现。
|
||||
- 每个能力独立路由，禁止跟随/派生特殊值（`same_as_daily` 已下线）；存量非法偏好值由 preferences getter 回退默认自愈，不做迁移。
|
||||
- 边界注记：分时监控由分钟能力兜底（`intraday_monitor_support`），不单设分时能力；`depth5` 已进矩阵但插件数据集白名单暂未开放，当前仅 TickFlow 提供。
|
||||
|
||||
## 5. 领域专项要求
|
||||
|
||||
### 5.1 策略
|
||||
|
||||
@@ -1,48 +1,38 @@
|
||||
|
||||
<div align="center">
|
||||
|
||||
# 📈 A股智能量化工作台
|
||||
|
||||
[](https://github.com/shy3130/tick-stock-panel)
|
||||
|
||||
# TSP--A股智能量化工作台
|
||||
|
||||
[](https://github.com/shy3130/tick-stock-panel)
|
||||
|
||||
**自托管、零运维的 A 股「选股 + 监控 + 回测」量化工作台**
|
||||
|
||||
**面向个人散户与量化爱好者而生**
|
||||
**多数据源能力路由 · 分钟级策略执行 · 全时段异动监控 · AI 辅助研究**
|
||||
|
||||
[](./LICENSE)
|
||||
[](https://www.python.org/)
|
||||
[](https://react.dev/)
|
||||
[](https://tickflow.org/auth/register?ref=V3KDKGXPEA)
|
||||
[](https://pola.rs/)
|
||||
[](./docs/custom-data-source.md)
|
||||
[](./Dockerfile)
|
||||
[](https://github.com/shy3130/tick-stock-panel/actions/workflows/docker.yml)
|
||||
[](https://github.com/shy3130/tick-stock-panel/stargazers)
|
||||
|
||||
</div>
|
||||
|
||||
<div align="center">
|
||||
|
||||
|
||||
|
||||
**[快速开始](#-快速开始)** · **[核心功能](#-核心功能)** · **[配置](#️-配置)** · **[完整文档](#-完整文档)**
|
||||
**[快速开始](#-快速开始)** · **[核心功能](#-核心功能)** · **[技术架构](#️-技术架构)** · **[配置](#️-配置)** · **[完整文档](#-完整文档)**
|
||||
|
||||
</div>
|
||||
|
||||
|
||||
---
|
||||
|
||||
**本项目以个人开源为主进行开发维护，数据源插件化，可任意接入第三方数据源。仅供学习研究使用，严禁商业用途。**
|
||||
|
||||
|
||||
**本项目个人开源，数据源插件化，可任意接入第三方数据源。仅供学习研究使用，严禁商业用途。**
|
||||
|
||||
|
||||
|
||||
|
||||
> ⚠️ 小白请绕路，本开源项目谨作为本地量化提供解决思路Demo，不作为投资软件或者看盘软件。
|
||||
> ⚠️ 小白请绕路，本开源项目谨作为本地量化提供解决思路与方案，不作为投资软件或者看盘软件。
|
||||
>
|
||||
> **明确不做**:不对标同花顺 / 通达信,不内置「AI 荐股 / 涨停预测」。
|
||||
|
||||
有问题可以邮件415333856@qq.com。
|
||||
有任何项目问题或商务合作 / 广告投放等合作意向,可邮件联系 415333856@qq.com。
|
||||
|
||||
觉得有用可以点个 Star
|
||||
|
||||
@@ -50,59 +40,60 @@
|
||||
|
||||
## ✨ 核心功能
|
||||
|
||||
| 模块 | 一句话 | 详见 |
|
||||
| 模块 | 一句话 | 详见 |
|
||||
| :--------------- | :--------------------------------------------------------------------- | :-------------------------------- |
|
||||
| 🔍 **选股引擎** | 18 个内置策略 + 自定义信号 + AI 生成 + 代码迁移,Polars 毫秒级扫全 A 股 | [strategy.md](./docs/strategy.md) |
|
||||
| 📊 **指标流水线** | MA/EMA/MACD/RSI/KDJ/布林/量比等,一次扫表落盘 enriched Parquet | [features.md](./docs/features.md) |
|
||||
| 🧪 **回测研究** | 因子/策略回测 + 财务快照因子(点时口径),T+1/费用/滑点约束,SSE 持久任务 | [features.md](./docs/features.md) |
|
||||
| 🔀 **能力路由** | 多数据集(日K/除权/实时/分钟/盘口/财务,持续扩展)按源能力独立路由,任选组合 | [custom-data-source.md](./docs/custom-data-source.md) |
|
||||
| 🔍 **选股引擎** | 18 个内置策略 + 分钟策略 + 自定义信号 + AI 生成,Polars 毫秒级扫全 A 股 | [strategy.md](./docs/strategy.md) |
|
||||
| 📊 **指标流水线** | MA/EMA/MACD/RSI/KDJ/布林/量比等 68 列指标与信号,一次扫表落盘 enriched Parquet | [features.md](./docs/features.md) |
|
||||
| 🧪 **回测研究** | 因子/策略/分钟回测 + 财务快照因子(点时口径),T+1/费用/滑点约束,结果可导出 | [features.md](./docs/features.md) |
|
||||
| ⛏️ **因子挖掘** | 嵌套样本外搜索多因子排名组合,与自有策略对照,候选库显式发布、永不自动上线 | [mining.md](./docs/mining.md) |
|
||||
| 🌡️ **市场环境** | 情绪周期 6 阶段(连板梯队驱动)+ 概念/行业主线排名,与 5 档环境分并存 | [market-phase.md](./docs/market-phase.md) |
|
||||
| 🚨 **异动监控** | 交易所异动规则口径(3/10/30 日偏离值),盘中实时接近度,系统告警与推送接入 | — |
|
||||
| 🚨 **异动监控** | 竞价/盘中/偏移三类异动一页覆盖:同花顺风向标 + 当日信号聚合 + 交易所偏离值口径 | — |
|
||||
| 📡 **监控中心** | 四类监控(策略/个股信号/价格/异动),多条件 AND/OR + 语音播报 + 飞书推送 | [features.md](./docs/features.md) |
|
||||
| 📈 **个股分析** | 9 类关键价位 + AI 四维分析(技术/基本面/财务/消息面) | [features.md](./docs/features.md) |
|
||||
| 🏆 **连板梯队** | 连板层级统计 + 概念涨幅轮动 + 盘后 AI 复盘 + 炸板/翘板预警 | [features.md](./docs/features.md) |
|
||||
| 🧰 **数据扩展** | 数据源插件化(stock-sdk 示例 + YAML 自定义源),扩展字段配成一级页面同台分析 | [custom-data-source.md](./docs/custom-data-source.md) |
|
||||
|
||||
|
||||
|
||||
|
||||
| 🏆 **连板梯队** | 连板层级统计 + 概念涨幅轮动 + 盘后 AI 复盘(龙虎榜/盘前风向标注入) + 炸板/翘板预警 | [features.md](./docs/features.md) |
|
||||
| 🧰 **数据扩展** | 数据源插件化(TickFlow/fuyao/stock-sdk + YAML 自定义源),扩展字段配成一级页面同台分析 | [custom-data-source.md](./docs/custom-data-source.md) |
|
||||
|
||||
<details>
|
||||
<summary><b>📦 主要页面与功能</b></summary>
|
||||
|
||||
**📊 行情总览**
|
||||
- **看板** Dashboard — 市场情绪评分 + 涨跌/成交额榜单 + 概念领涨领跌 + 大盘异动事件流,一日全貌
|
||||
- **自选** Watchlist — 自选股池,多分组管理(M:N),表格/卡片双视图,换手/量比/RSI 等实时指标
|
||||
- **看板** Dashboard — 市场情绪评分 + 涨跌/成交额榜单 + 概念/行业领涨领跌(点击板块直达成分股,领涨股带涨跌幅) + 大盘异动事件流,一日全貌
|
||||
- **自选** Watchlist — 自选股池,多分组管理(M:N),表格/卡片双视图,换手/量比/RSI 等实时指标,按档位分流实时刷新
|
||||
- **指数** Indices — 沪深指数浏览与同步
|
||||
|
||||
**🔍 选股与回测**
|
||||
- **策略** Screener — Polars 毫秒级扫描全 A 股,18 个内置策略卡片 + 自定义条件
|
||||
- **回测** Backtest — 两种模式:
|
||||
- **因子回测** — IC/IR、分层收益、多空组合,先筛掉无效指标
|
||||
- **策略回测** — 净值曲线、回撤、夏普、胜率,支持 T+1/手续费/滑点/止损,SSE 流式进度
|
||||
- **策略** Screener — Polars 毫秒级扫描全 A 股,日线/分钟策略统一单池,按策略声明周期自动路由执行
|
||||
- **回测** Backtest — 四种研究视图:
|
||||
- **因子回测** — IC/IR、分层收益、多空组合,62+ 因子目录先筛掉无效指标
|
||||
- **策略回测** — 净值曲线、回撤、夏普、胜率,T+1/手续费/滑点/止损,SSE 流式进度
|
||||
- **分钟策略回测** — 逐交易日回放信号、分钟收盘入场,分钟级成交明细
|
||||
- **验证** — 参数敏感性与滚动样本外
|
||||
- 研究闭环:结果导出 CSV(概要/净值/交易明细/分标的统计) → 保存候选 → **一键载入复测**
|
||||
- **挖掘** Mining — 嵌套样本外因子与策略挖掘:训练区间因子方向重估 + 相关性去重 + 多因子排名组合搜索,自有策略作对照轨;候选入库,显式确认后才发布,永不自动上线
|
||||
|
||||
**📈 个股与板块分析**
|
||||
- **个股分析** Stock Analysis (Beta) — 日K + 9 类关键价位 + AI 四维分析(技术/基本面/财务/消息面)
|
||||
- **财务分析** Financials — 利润表/资负表/现金流/关键指标 + AI 解读
|
||||
- **概念分析** Concept Analysis — ths 概念涨幅轮动矩阵 + 领涨/领跌主线 + 个股穿透
|
||||
- **行业分析** Industry Analysis — 行业分层涨幅轮动 + 领涨/领跌主线 + 成分股
|
||||
- **财务分析** Financials — 利润表/资负表/现金流/关键指标(多源并集合并,fuyao 财务四表适配) + AI 解读
|
||||
- **概念分析 / 行业分析** — ths 维度涨幅轮动矩阵 + 领涨/领跌主线 + 个股穿透
|
||||
- **市场环境** Regime — 情绪周期 6 阶段(冰点/启动/主升/高潮/退潮/修复,连板梯队驱动,EMA 平滑 + 2 日确认)+ 概念/行业主线排名,与 5 档环境分并存
|
||||
- **连板梯队** Limit Up Ladder — 连板层级统计 + 概念/行业分布 + 封单监控(可切换连跌梯队)
|
||||
|
||||
**🔔 监控与复盘**
|
||||
- **监控中心** Monitor — 策略/个股信号/价格/异动四类规则,支持自选分组作用域,盘中实时弹窗 + 语音播报(播报个股名称与信号) + 触发记录持久化
|
||||
- **异动监控** Abnormal Moves — 按交易所异动规则口径(3日 ±20%/±30%/±40% · 10日 +100% · 30日 +200%)实时计算个股偏离值接近度,盯住异动边缘名单;触发记录 + 站内通知/飞书·企微推送
|
||||
- **复盘** Review (Beta) — 盘后 AI 自动生成市场复盘,可定时执行、推送飞书、下载 Markdown
|
||||
- **异动监控** Abnormal Moves — 按交易时间线三 tab:
|
||||
- **竞价异动** — 同花顺盘前风向标(含当日/次日真实收益对照、追高风险标记)+ 全市场竞价扫描(待采集任务)
|
||||
- **盘中异动** — 涨停/炸板/翘板/跌停/新高/新低/放量当日信号聚合,零新增采集
|
||||
- **偏移异动** — 交易所异动偏离值口径(主板 3 日 ±20%、创业板/科创板 ±30%、北交所 ±40%;10 日 +100%/−50%、30 日 +200%/−70%),实时接近度
|
||||
- **复盘** Review (Beta) — 盘后 AI 自动生成市场复盘,注入龙虎榜资金动向与盘前风向标对照;可定时执行、推送飞书、下载 Markdown
|
||||
|
||||
**🗄️ 数据与扩展**
|
||||
- **数据** Data — 本地数据画像与同步状态(维表/日K/除权/Enriched/指数/ETF/分钟K/财务),盘后管道与历史扩展
|
||||
- **扩展分析** (动态菜单) — 把任意第三方/扩展数据字段配成一级菜单,与内置数据同台分析
|
||||
- **设置** Settings — 数据源与能力检测、AI 接口、实时监控、扩展页面、信号库、菜单与系统设置
|
||||
- **设置** Settings — 数据源与能力检测(能力路由矩阵、档位徽章)、AI 接口、实时监控、扩展页面、信号库、菜单与系统设置
|
||||
|
||||
</details>
|
||||
|
||||
|
||||
---
|
||||
|
||||
## 📸 界面预览
|
||||
@@ -148,11 +139,116 @@
|
||||
|
||||
</div>
|
||||
|
||||
|
||||
---
|
||||
|
||||
## 🏗️ 技术架构
|
||||
|
||||
### 分层总览
|
||||
|
||||
```mermaid
|
||||
flowchart TB
|
||||
subgraph DATA["数据源层 · 插件化"]
|
||||
direction LR
|
||||
D1["TickFlow SDK"] ~~~ D2["fuyao<br/>同花顺 REST"] ~~~ D3["stock-sdk"] ~~~ D4["YAML 自定义源"] ~~~ D5["+ 更多插件…"]
|
||||
end
|
||||
|
||||
subgraph ROUTE["能力路由层"]
|
||||
direction LR
|
||||
R(["多数据集 · 按能力独立路由 · 档位探测"])
|
||||
end
|
||||
|
||||
subgraph STORE["存储层"]
|
||||
direction LR
|
||||
ST1[("Parquet 分区表")] ~~~ ST2[("DuckDB")] ~~~ ST3[("JSON 按日缓存")]
|
||||
end
|
||||
|
||||
subgraph CALC["计算层 · Polars"]
|
||||
direction LR
|
||||
C1["指标流水线"] ~~~ C2["复权与信号"]
|
||||
end
|
||||
|
||||
subgraph RES["研究层"]
|
||||
direction LR
|
||||
G1["因子引擎"] ~~~ G2["回测引擎"] ~~~ G3["因子挖掘"]
|
||||
end
|
||||
|
||||
subgraph SVC["应用层 · FastAPI"]
|
||||
direction LR
|
||||
S1["REST · SSE"] ~~~ S2["监控引擎"] ~~~ S3["任务调度"] ~~~ S4["交易日探针"]
|
||||
end
|
||||
|
||||
subgraph FE["呈现层 · React 18"]
|
||||
direction LR
|
||||
F1["功能页面"] ~~~ F2["图表可视化"] ~~~ F3["实时推送"]
|
||||
end
|
||||
|
||||
subgraph EXT["二次开发与扩展 · 贯穿各层的插槽"]
|
||||
direction LR
|
||||
X1["自定义策略"] ~~~ X2["自定义信号"] ~~~ X3["扩展分析页面"] ~~~ X4["AI 接口"]
|
||||
end
|
||||
|
||||
DATA --- ROUTE
|
||||
ROUTE --- STORE
|
||||
STORE --- CALC
|
||||
CALC --- RES
|
||||
RES --- SVC
|
||||
SVC --- FE
|
||||
FE ~~~ EXT
|
||||
|
||||
classDef fe fill:#eef2ff,stroke:#6366f1,color:#312e81
|
||||
classDef svc fill:#ecfeff,stroke:#06b6d4,color:#164e63
|
||||
classDef res fill:#fff7ed,stroke:#f97316,color:#7c2d12
|
||||
classDef calc fill:#f0f9ff,stroke:#0ea5e9,color:#0c4a6e
|
||||
classDef store fill:#ecfdf5,stroke:#10b981,color:#064e3b
|
||||
classDef data fill:#fdf2f8,stroke:#ec4899,color:#831843
|
||||
classDef pluginSlot fill:#fdf2f8,stroke:#ec4899,color:#831843,stroke-dasharray:5 4
|
||||
classDef route fill:#faf5ff,stroke:#8b5cf6,color:#6b21a8,stroke-width:2px
|
||||
classDef ext fill:#f8fafc,stroke:#94a3b8,color:#334155,stroke-dasharray:5 4
|
||||
|
||||
class F1,F2,F3 fe
|
||||
class S1,S2,S3,S4 svc
|
||||
class G1,G2,G3 res
|
||||
class C1,C2 calc
|
||||
class ST1,ST2,ST3 store
|
||||
class D1,D2,D3,D4 data
|
||||
class D5 pluginSlot
|
||||
class R route
|
||||
class X1,X2,X3,X4 ext
|
||||
|
||||
style FE fill:#f5f3ff,stroke:#c7d2fe,color:#3730a3
|
||||
style SVC fill:#ecfeff,stroke:#a5f3fc,color:#155e75
|
||||
style RES fill:#fff7ed,stroke:#fed7aa,color:#9a3412
|
||||
style CALC fill:#f0f9ff,stroke:#bae6fd,color:#075985
|
||||
style STORE fill:#ecfdf5,stroke:#a7f3d0,color:#065f46
|
||||
style ROUTE fill:#faf5ff,stroke:#ddd6fe,color:#6b21a8
|
||||
style DATA fill:#fdf2f8,stroke:#fbcfe8,color:#9d174d
|
||||
style EXT fill:#f8fafc,stroke:#94a3b8,color:#334155,stroke-dasharray:7 5
|
||||
```
|
||||
|
||||
### 关键机制
|
||||
|
||||
| 机制 | 说明 |
|
||||
| :--- | :--- |
|
||||
| **能力路由矩阵** | 各数据集按源声明能力独立路由,注册表集中定义、可扩展:TICKFLOW 档位探测(None/Free/Starter/Pro/Expert)+ 插件源能力声明,fail-closed(声明 `pct_unit` 未声明即拒)。同一数据集可随时换源,指标与回测口径不变 |
|
||||
| **交易日探针** | fuyao 交易日历(确定性,含调休)→ tickflow 全市场行情时间戳探针(OR 语义)→ 工作日兜底;节假日自动停掉实时轮询与分钟增量,零无效请求 |
|
||||
| **财务多源合并** | 按 `(symbol, period_end)` 报告期累积,多源取并集、逐列按公告日取最新(PIT);公告前一律空值,绝不填 0 |
|
||||
| **非路由数据集直连** | 龙虎榜/盘前风向标/交易日历等 fuyao 专有能力不进路由矩阵,由独立服务直连消费——按日 JSON 缓存(历史不可变)、交易日回退、四态降级 |
|
||||
| **回测执行隔离** | 回测在 spawn worker 子进程运行,持久 run ID,刷新/切页重连不丢任务;子进程结果消息经锁保护回传 |
|
||||
| **分层缓存** | enriched 读取时现算指标(存储仅 15 列基础数据,现算 68 列指标与信号)+ 进程内快照缓存;扩展字段按日分区快照,页面即配即用 |
|
||||
|
||||
### 技术栈
|
||||
|
||||
| 层 | 选型 |
|
||||
| :----------- | :------------------------------------------------------------------------------------------------ |
|
||||
| **后端** | FastAPI · Pydantic v2 · APScheduler · sse-starlette |
|
||||
| **数据** | Polars(计算)· DuckDB(查询)· Parquet(存储) |
|
||||
| **回测** | 自研仓位模拟引擎(T+1/费用/滑点/分钟回放)· vectorbt(部分路径) |
|
||||
| **数据源** | [TickFlow](https://tickflow.org/auth/register?ref=V3KDKGXPEA) 官方 SDK · fuyao(同花顺 REST) · 插件化扩展(stock-sdk 示例插件 · YAML 自定义源) |
|
||||
| **AI**(可选) | OpenAI 兼容接口(DeepSeek / 通义 / Ollama 等) |
|
||||
| **前端** | React 18 · Vite · TypeScript · Tailwind · Tanstack Query · [Lightweight Charts](https://www.tradingview.com/lightweight-charts/)(TradingView 开源) · ECharts · dnd-kit |
|
||||
| **部署** | Docker 两阶段构建,前端 dist 拷进后端镜像,**单容器** |
|
||||
|
||||
---
|
||||
|
||||
## 🚀 快速开始
|
||||
|
||||
@@ -195,11 +291,11 @@ CODEX_CLI_VERSION=0.144.3 docker compose up --build
|
||||
|
||||
### 跑起来后的第一次使用
|
||||
|
||||
1. **设置 → 凭据与能力** → 点 **重新检测**,确认档位标签
|
||||
1. **设置 → 凭据与能力** → 点 **重新检测**,确认档位标签与能力路由矩阵
|
||||
2. **设置** → **立即跑盘后管道**:拉日 K + 计算 enriched 表(None / Free 走 free-api,当日数据盘后 1-2 小时可用)
|
||||
3. **自选**页加标的 → **选股**页点策略卡片扫描 / 配自定义信号
|
||||
4. **回测**页选策略 + 区间 → 看净值 / 夏普 / 交易明细(SSE 实时进度)
|
||||
5. **监控中心**配规则,盘中实时弹窗 + 持久化记录
|
||||
4. **回测**页选策略 + 区间 → 看净值 / 夏普 / 交易明细(SSE 实时进度),结果可导出 CSV、存候选一键复测
|
||||
5. **监控中心**配规则,盘中实时弹窗 + 持久化记录;**异动监控**覆盖竞价/盘中/偏移全时段
|
||||
|
||||
---
|
||||
|
||||
@@ -217,30 +313,17 @@ PORT=3018 # 服务端口
|
||||
|
||||
---
|
||||
|
||||
## 🏗️ 技术栈
|
||||
|
||||
| 层 | 选型 |
|
||||
| :----------- | :------------------------------------------------------------------------------------------------ |
|
||||
| **后端** | FastAPI · Pydantic v2 · APScheduler · sse-starlette |
|
||||
| **数据** | Polars(计算)· DuckDB(查询)· Parquet(存储) |
|
||||
| **回测** | vectorbt(全项目唯一 pandas 边界) |
|
||||
| **数据源** | [TickFlow](https://tickflow.org/auth/register?ref=V3KDKGXPEA) 官方 SDK · 插件化扩展(stock-sdk 示例插件 · YAML 自定义源) |
|
||||
| **AI**(可选) | OpenAI 兼容接口(DeepSeek / 通义 / Ollama 等) |
|
||||
| **前端** | React 18 · Vite · TypeScript · Tailwind · Tanstack Query · Lightweight Charts · ECharts · dnd-kit |
|
||||
| **部署** | Docker 两阶段构建,前端 dist 拷进后端镜像,**单容器** |
|
||||
|
||||
---
|
||||
|
||||
## 🗺️ 路线图
|
||||
|
||||
| Phase | 内容 | 状态 |
|
||||
| :----- | :----------------------------------------------------------------- | :--- |
|
||||
| 0-1 | 仓库骨架 · FastAPI 壳 · 能力探测 · K 线同步与分析页 | ✅ |
|
||||
| 2-3 | Polars enriched 流水线 · Screener · vectorbt 回测(T+1/手续费/止损) | ✅ |
|
||||
| 2-3 | Polars enriched 流水线 · Screener · 回测引擎(T+1/手续费/止损) | ✅ |
|
||||
| 4-5 | 监控引擎 · 四类监控规则 · 实时 SSE 推送 · 持久化记录 | ✅ |
|
||||
| 6 | 个股分析(专用日 K + 9 类关键价位 + AI 四维分析) | ✅ |
|
||||
| **v0.2** | 因子挖掘全链路 · 市场阶段与主线识别 · 异动监控 · 数据源插件化 | ✅ |
|
||||
| **v2** | Webhook 推送· 板块异动 · 早晚报 · 更多扩展 | 🚧 |
|
||||
| **v0.3** | 能力路由矩阵 · fuyao 数据源(财务/龙虎榜/风向标) · 分钟策略与回测 · 交易日探针 · 全时段异动中心 · 回测导出与候选复测 | ✅ |
|
||||
| **v2** | Webhook 推送· 板块异动 · 早晚报 · 全市场竞价采集 · 更多扩展 | 🚧 |
|
||||
|
||||
---
|
||||
|
||||
@@ -251,11 +334,11 @@ PORT=3018 # 服务端口
|
||||
| [docs/deployment.md](./docs/deployment.md) | 部署方式(Dev / Docker / GH Actions)、老 CPU 兼容、更新代码、访问密码 |
|
||||
| [docs/configuration.md](./docs/configuration.md) | 所有 `.env` 配置项详解(数据源、AI、服务、密码、数据目录) |
|
||||
| [docs/features.md](./docs/features.md) | 各功能模块详细说明(选股/指标/回测/监控/个股分析/数据扩展) |
|
||||
| [docs/custom-data-source.md](./docs/custom-data-source.md) | 自定义数据源接入、YAML 配置与 mock 联调示例 |
|
||||
| [docs/custom-data-source.md](./docs/custom-data-source.md) | 自定义数据源接入、能力路由契约、YAML 配置与 mock 联调示例 |
|
||||
| [docs/strategy.md](./docs/strategy.md) | 策略体系(18 内置策略 + 三种扩展方式 + 文件结构) |
|
||||
| [docs/mining.md](./docs/mining.md) | 因子与策略挖掘口径、防泄漏、任务隔离和发布边界 |
|
||||
| [docs/market-phase.md](./docs/market-phase.md) | 市场情绪周期 6 阶段与概念/行业主线识别的口径与设计 |
|
||||
| [docs/plugin-development.md](./docs/plugin-development.md) | 数据源插件开发规范(以 stock-sdk 为参考实现) |
|
||||
| [docs/plugin-development.md](./docs/plugin-development.md) | 数据源插件开发规范(以 stock-sdk / fuyao 为参考实现) |
|
||||
| [docs/secondary-development.md](./docs/secondary-development.md) | 代码二次开发、前端插槽、后端策略接口与 AI 开发模板 |
|
||||
| [backend/app/strategy/prompts/strategy-guide.md](./backend/app/strategy/prompts/strategy-guide.md) | 策略开发完整规范(AI 生成与手写) |
|
||||
|
||||
@@ -265,15 +348,42 @@ fork同时请点个star哦,欢迎 Issue 和 PR。
|
||||
|
||||
## 💬 交流群
|
||||
|
||||
欢迎加入交流群,讨论交流。
|
||||
欢迎加入交流群,一起讨论交流。作者个人维护的部分个性化接口,统一公布在群公告中,供大家免费使用。
|
||||
|
||||
<img src="./community-qr-code.jpg" alt="交流群二维码" width="240" />
|
||||
|
||||
---
|
||||
|
||||
## ❤️ 支持项目
|
||||
|
||||
<div align="center">
|
||||
|
||||
如果这个项目对你有帮助,欢迎请作者喝杯咖啡 ☕
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<td width="50%" align="center"><b>微信赞赏</b></td>
|
||||
<td width="50%" align="center"><b>支付宝</b></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td width="50%" align="center"><img src="./assets/support/wechat-appreciation.jpg" alt="微信赞赏码 · 感谢道友支持 愿一路长红" height="280" /></td>
|
||||
<td width="50%" align="center"><img src="./assets/support/alipay.jpg" alt="支付宝收款码 · 打开支付宝扫一扫" height="280" /></td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
愿道友一路长红 📈
|
||||
|
||||
</div>
|
||||
|
||||
> 打赏完全自愿,金额不限;不用于购买任何功能、数据权限、投资建议
|
||||
>
|
||||
> 作者精力有限,优先响应赞助回馈,希望理解
|
||||
|
||||
---
|
||||
|
||||
## ⚠️ 免责声明
|
||||
|
||||
本项目仅供**学习与量化研究**,**不构成任何投资建议**。回测结果不代表未来收益。A 股有风险,入市需谨慎。数据准确性以数据源 TickFlow 官方为准。
|
||||
本项目仅供**学习与量化研究**,**不构成任何投资建议**。回测结果不代表未来收益。A 股有风险,入市需谨慎。数据准确性以数据源官方为准。
|
||||
|
||||
## 📄 License
|
||||
|
||||
@@ -281,6 +391,8 @@ fork同时请点个star哦,欢迎 Issue 和 PR。
|
||||
|
||||
本项目依赖 [TickFlow](https://tickflow.org/auth/register?ref=V3KDKGXPEA) 提供数据服务,使用前请遵守其服务条款
|
||||
|
||||
内置数据源插件 [fuyao](https://fuyao.aicubes.cn/docs/api-reference/) 提供同花顺 REST 数据接口(行情 / 财务 / 龙虎榜 / 盘前风向标 / 交易日历等),需自备 API Key,使用前请遵守其服务条款
|
||||
|
||||
数据源插件 [stock-sdk](https://stock-sdk.linkdiary.cn) 遵循其各自的 ISC 协议。
|
||||
|
||||
## 社区
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 170 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 71 KiB |
@@ -2,7 +2,7 @@
|
||||
|
||||
import sys
|
||||
|
||||
__version__ = "0.2.1"
|
||||
__version__ = "0.2.2"
|
||||
|
||||
# Windows 默认 stdout/stderr 编码为 GBK(cp936),TickFlow SDK 内部输出含 emoji 的
|
||||
# 指数/标的名称(如 \U0001f193)时会抛 UnicodeEncodeError,导致请求失败。
|
||||
|
||||
@@ -1,13 +1,27 @@
|
||||
"""异动边缘监控 API — 按交易所异动规则口径统计接近触发的个股。"""
|
||||
"""异动监控 API — 竞价/盘中/偏移三类异动。
|
||||
|
||||
- /intraday: 盘中量价信号聚合 (enriched 当日信号列, 零新增采集)
|
||||
- /overview: 偏移异动边缘总览 (交易所异动规则口径的接近度)
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from fastapi import APIRouter, Query, Request
|
||||
|
||||
from app.services.abnormal_moves import build_overview
|
||||
from app.services.abnormal_moves import build_intraday, build_overview
|
||||
|
||||
router = APIRouter(prefix="/api/abnormal", tags=["abnormal"])
|
||||
|
||||
|
||||
@router.get("/intraday")
|
||||
def abnormal_intraday(
|
||||
request: Request,
|
||||
limit: int = Query(500, ge=1, le=2000),
|
||||
):
|
||||
"""盘中异动: 涨停/炸板/跌停翘板/跌停/新高/新低/放量 信号命中行。"""
|
||||
repo = request.app.state.repo
|
||||
return build_intraday(repo, limit=limit)
|
||||
|
||||
|
||||
@router.get("/overview")
|
||||
def abnormal_overview(
|
||||
request: Request,
|
||||
|
||||
@@ -346,6 +346,33 @@ class StrategyBacktestRequest(BaseModel):
|
||||
regime_filter: dict | None = None
|
||||
|
||||
|
||||
def _guard_minute_strategy_backtest(
|
||||
request: Request, strategy_id: str, start: date, asset_type: str,
|
||||
) -> None:
|
||||
"""分钟策略回测入口守卫: 仅 A 股 + 本地分钟K覆盖检查 (fail-fast)。"""
|
||||
engine = getattr(request.app.state, "strategy_engine", None)
|
||||
if engine is None:
|
||||
return
|
||||
try:
|
||||
s = engine.get(strategy_id)
|
||||
except ValueError:
|
||||
return
|
||||
if s is None or s.execution_backend != "minute_filter":
|
||||
return
|
||||
if asset_type != "stock":
|
||||
raise HTTPException(400, detail="分钟策略回测当前仅支持 A 股 (stock)")
|
||||
earliest = request.app.state.repo.earliest_minute_date()
|
||||
if earliest is None or start < earliest:
|
||||
have = f"最早到 {earliest}, " if earliest else ""
|
||||
raise HTTPException(
|
||||
400,
|
||||
detail=(
|
||||
f"本地分钟K{have}无法覆盖回测起始日 {start}。"
|
||||
"请先用「扩展分钟K历史」拉取更多数据, 或缩小回测区间"
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
@router.post("/strategy/run")
|
||||
def strategy_run(req: StrategyBacktestRequest, request: Request):
|
||||
"""策略回测 — 复用 StrategyDef 体系做全周期回测。"""
|
||||
@@ -355,6 +382,7 @@ def strategy_run(req: StrategyBacktestRequest, request: Request):
|
||||
end = req.end or date.today()
|
||||
start = _resolve_start(req, end, FACTOR_DEFAULT_DAYS)
|
||||
_guard_server_backtest_range(start, end)
|
||||
_guard_minute_strategy_backtest(request, req.strategy_id, start, req.asset_type)
|
||||
|
||||
cfg = StrategyBacktestConfig(
|
||||
strategy_id=req.strategy_id,
|
||||
@@ -505,6 +533,7 @@ async def strategy_stream(
|
||||
# 空 start = 全部历史: 用本地最早日K日期, 查不到再回退到默认窗口
|
||||
earliest = request.app.state.repo.earliest_daily_date()
|
||||
start_date = earliest or (end_date - timedelta(days=FACTOR_DEFAULT_DAYS))
|
||||
_guard_minute_strategy_backtest(request, strategy_id, start_date, asset_type)
|
||||
|
||||
# 服务端范围保护
|
||||
guard_violated = False
|
||||
|
||||
@@ -7,6 +7,7 @@ import math
|
||||
import re
|
||||
import shutil
|
||||
import tempfile
|
||||
import time
|
||||
from datetime import date, datetime
|
||||
from pathlib import Path
|
||||
from typing import Literal
|
||||
@@ -485,6 +486,198 @@ def dimension_members(
|
||||
}
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 板块分时 (dimension intraday)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# 点击触发 + 60s 进程内缓存: 分钟分区是滚动底座 (minute_refresh / 盘后分钟同步),
|
||||
# 不做后台预计算 — 板块基数大而单次聚合仅几十毫秒。
|
||||
_DIMENSION_INTRADAY_CACHE: dict[tuple[str, str, str, str | None], tuple[float, dict]] = {}
|
||||
_DIMENSION_INTRADAY_CACHE_TTL_S = 60.0
|
||||
# 成分股网格化 ffill 上限: 超大板块退化为逐时间戳可得均值 (内存保护)。
|
||||
_DIMENSION_INTRADAY_FFILL_CAP = 2000
|
||||
|
||||
|
||||
def _bare_symbol_expr(col: str = "symbol") -> pl.Expr:
|
||||
"""'000001.SZ' → '000001'; 已是裸代码则原样。"""
|
||||
return pl.col(col).cast(pl.String).str.strip_chars().str.split(".").list.first()
|
||||
|
||||
|
||||
def _dimension_member_bares(matched: pl.DataFrame, config: ExtConfig) -> list[str]:
|
||||
"""成分股裸代码集合 (symbol 列优先级与 dimension-members 端点一致)。"""
|
||||
if matched.is_empty():
|
||||
return []
|
||||
symbol_columns = ["symbol", "code", "股票代码", "代码"]
|
||||
for mapping in (config.symbol_map, config.code_map):
|
||||
if isinstance(mapping, dict) and mapping.get("type") == "mapped" and mapping.get("col"):
|
||||
symbol_columns.append(str(mapping["col"]))
|
||||
cols = [c for c in dict.fromkeys(symbol_columns) if c in matched.columns]
|
||||
if not cols:
|
||||
return []
|
||||
coalesced = pl.coalesce(
|
||||
[pl.col(c).cast(pl.String).str.strip_chars().str.split(".").list.first() for c in cols]
|
||||
)
|
||||
series = matched.select(coalesced.alias("_bare")).to_series()
|
||||
return sorted({s for s in series.to_list() if s})
|
||||
|
||||
|
||||
def _prev_daily_close(data_dir: Path, target_date: str) -> pl.DataFrame | None:
|
||||
"""目标日前最近一个日K分区的收盘价 → (_bare, prev_close); 无则 None。"""
|
||||
daily = data_dir / "kline_daily"
|
||||
if not daily.exists():
|
||||
return None
|
||||
dates = sorted(
|
||||
d.name[5:]
|
||||
for d in daily.iterdir()
|
||||
if d.is_dir() and d.name.startswith("date=") and (d / "part.parquet").exists()
|
||||
)
|
||||
prevs = [d for d in dates if d < target_date]
|
||||
if not prevs:
|
||||
return None
|
||||
path = daily / f"date={prevs[-1]}" / "part.parquet"
|
||||
if not path.exists():
|
||||
return None
|
||||
df = pl.read_parquet(path, columns=["symbol", "close"])
|
||||
return (
|
||||
df.with_columns(_bare_symbol_expr().alias("_bare"))
|
||||
.select([pl.col("_bare"), pl.col("close").cast(pl.Float64).alias("prev_close")])
|
||||
.unique(subset=["_bare"], keep="last")
|
||||
)
|
||||
|
||||
|
||||
def _dimension_intraday_compute(
|
||||
config: ExtConfig,
|
||||
data_dir: Path,
|
||||
field: str,
|
||||
value: str,
|
||||
snapshot_date: str | None,
|
||||
) -> dict:
|
||||
"""板块等权分时: 成分股当日分钟K逐分钟平均涨跌幅 + 全市场对照线。
|
||||
|
||||
口径: pct = 分钟close / ref − 1 (小数制, 与快照涨跌幅契约一致, 前端 ×100 显示),
|
||||
ref 优先前一交易日日K收盘 (prev_close, 开盘跳空体现在曲线起点);
|
||||
日K缺失的标的退化为当日首根分钟close (混合基准)。
|
||||
停牌/无成交分钟按成分股 forward-fill 后再平均, 全市场线取逐时间戳可得均值。
|
||||
"""
|
||||
minute_dir = data_dir / "kline_minute"
|
||||
partitions: list[str] = []
|
||||
if minute_dir.exists():
|
||||
partitions = sorted(
|
||||
d.name[5:]
|
||||
for d in minute_dir.iterdir()
|
||||
if d.is_dir() and d.name.startswith("date=") and (d / "part.parquet").exists()
|
||||
)
|
||||
if snapshot_date:
|
||||
target = snapshot_date if snapshot_date in partitions else None
|
||||
else:
|
||||
target = partitions[-1] if partitions else None
|
||||
if not target:
|
||||
return {"status": "no_data", "reason": "minute_missing", "date": snapshot_date, "points": []}
|
||||
|
||||
ext_df, _active = _read_ext_dataframe(config, data_dir)
|
||||
if ext_df.is_empty() or field not in ext_df.columns:
|
||||
return {"status": "empty", "reason": "no_members", "date": target, "points": []}
|
||||
member_bares = _dimension_member_bares(_filter_dimension_member_rows(ext_df, field, value), config)
|
||||
if not member_bares:
|
||||
return {"status": "empty", "reason": "no_members", "date": target, "points": []}
|
||||
|
||||
try:
|
||||
bars = pl.read_parquet(
|
||||
minute_dir / f"date={target}" / "part.parquet",
|
||||
columns=["symbol", "datetime", "close"],
|
||||
)
|
||||
except Exception as exc: # noqa: BLE001
|
||||
logger.warning("dimension-intraday read minute partition failed: %s", exc)
|
||||
return {"status": "no_data", "reason": "minute_schema", "date": target, "points": []}
|
||||
bars = bars.drop_nulls(subset=["datetime", "close"])
|
||||
if bars.is_empty():
|
||||
return {"status": "no_data", "reason": "minute_empty", "date": target, "points": []}
|
||||
bars = bars.with_columns(_bare_symbol_expr().alias("_bare"))
|
||||
|
||||
prev = _prev_daily_close(data_dir, target)
|
||||
joined = bars.join(prev, on="_bare", how="left") if prev is not None else bars.with_columns(
|
||||
pl.lit(None, dtype=pl.Float64).alias("prev_close")
|
||||
)
|
||||
refs = joined.group_by("_bare").agg(
|
||||
pl.col("prev_close").first().alias("_prev"),
|
||||
pl.col("close").sort_by("datetime").first().alias("_first"),
|
||||
).with_columns(pl.coalesce(["_prev", "_first"]).alias("_ref"))
|
||||
n_prev = refs["_prev"].is_not_null().sum()
|
||||
basis = "prev_close" if n_prev == refs.height else ("first_close" if n_prev == 0 else "mixed")
|
||||
joined = (
|
||||
joined.join(refs.select(["_bare", "_ref"]), on="_bare", how="left")
|
||||
.with_columns((pl.col("close") / pl.col("_ref") - 1.0).alias("_pct"))
|
||||
)
|
||||
|
||||
market = joined.group_by("datetime").agg(pl.col("_pct").mean().alias("_market"))
|
||||
|
||||
member_bars = joined.filter(pl.col("_bare").is_in(member_bares))
|
||||
members_with_minute = member_bars["_bare"].n_unique() if not member_bars.is_empty() else 0
|
||||
if members_with_minute == 0:
|
||||
return {
|
||||
"status": "empty", "reason": "no_member_bars", "date": target,
|
||||
"member_count": len(member_bares), "members_with_minute": 0, "points": [],
|
||||
}
|
||||
if members_with_minute <= _DIMENSION_INTRADAY_FFILL_CAP:
|
||||
# 网格化 (成分股 × 全时间轴) + 逐股 ffill: 停牌分钟冻结在最后价而非退出均值
|
||||
grid = (
|
||||
member_bars.select(pl.col("_bare").unique())
|
||||
.join(joined.select(pl.col("datetime").unique()), how="cross")
|
||||
)
|
||||
member_bars = (
|
||||
grid.join(member_bars.select(["_bare", "datetime", "_pct"]), on=["_bare", "datetime"], how="left")
|
||||
.sort(["_bare", "datetime"])
|
||||
.with_columns(pl.col("_pct").forward_fill().over("_bare"))
|
||||
)
|
||||
sector = member_bars.group_by("datetime").agg(pl.col("_pct").mean().alias("_sector"))
|
||||
|
||||
combined = market.join(sector, on="datetime", how="left").sort("datetime")
|
||||
|
||||
def _r4(v) -> float | None:
|
||||
return round(float(v), 4) if v is not None and not (isinstance(v, float) and math.isnan(v)) else None
|
||||
|
||||
points = [
|
||||
{
|
||||
"time": row["datetime"].strftime("%H:%M"),
|
||||
"sector": _r4(row["_sector"]),
|
||||
"market": _r4(row["_market"]),
|
||||
}
|
||||
for row in combined.iter_rows(named=True)
|
||||
]
|
||||
return {
|
||||
"status": "ok",
|
||||
"date": target,
|
||||
"basis": basis,
|
||||
"member_count": len(member_bares),
|
||||
"members_with_minute": members_with_minute,
|
||||
"points": points,
|
||||
}
|
||||
|
||||
|
||||
@router.get("/{config_id}/dimension-intraday")
|
||||
def dimension_intraday(
|
||||
request: Request,
|
||||
config_id: str,
|
||||
field: str = Query(..., min_length=1),
|
||||
value: str = Query(..., min_length=1),
|
||||
snapshot_date: str | None = Query(None, alias="date"),
|
||||
):
|
||||
"""板块分时走势 (等权): 成分股 × 当日分钟K聚合; 60s 缓存, 点击触发不预计算。"""
|
||||
config = _store(request).get(config_id)
|
||||
if not config:
|
||||
raise HTTPException(404, f"配置 '{config_id}' 不存在")
|
||||
|
||||
cache_key = (config_id, field, value.strip(), snapshot_date)
|
||||
now = time.monotonic()
|
||||
hit = _DIMENSION_INTRADAY_CACHE.get(cache_key)
|
||||
if hit is not None and now - hit[0] < _DIMENSION_INTRADAY_CACHE_TTL_S:
|
||||
return hit[1]
|
||||
|
||||
payload = _dimension_intraday_compute(config, _data_dir(request), field, value, snapshot_date)
|
||||
_DIMENSION_INTRADAY_CACHE[cache_key] = (now, payload)
|
||||
return payload
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# 文件上传
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
@@ -10,7 +10,7 @@ from typing import Optional
|
||||
from fastapi import APIRouter, HTTPException, Query, Request
|
||||
|
||||
from app.indicators.pipeline import compute_enriched, compute_enriched_single
|
||||
from app.market_time import cn_now, cn_today
|
||||
from app.market_time import cn_now, cn_today, in_continuous_session
|
||||
from app.price_limits import is_risk_warning_name, price_limit_pct
|
||||
from app.db_safe import is_valid_ext_ident
|
||||
from app.services import kline_sync
|
||||
@@ -780,11 +780,15 @@ def get_minute(
|
||||
request: Request,
|
||||
symbol: str = Query(..., description="标的代码"),
|
||||
trade_date: date | None = Query(None, alias="date", description="交易日期, 默认最新"),
|
||||
live: bool = Query(False, description="当日盘中跳过本地优先, 直接实时拉取(个股详情分时轮询用)"),
|
||||
):
|
||||
"""读取某只股票某天的分钟 K 线。
|
||||
|
||||
- 本地有完整数据(240条) → 直接返回
|
||||
- 本地无数据或不完整 → 从 TickFlow 实时拉取返回（不写入）
|
||||
- live=true 且当日连续竞价时段 → 跳过本地优先直接实时拉取:
|
||||
盘中分钟增量落盘的本地分区按 ≥60s 轮次更新, 90% 完整度启发式会让
|
||||
详情分时图停在上一增量轮, 与行情列表的节奏脱节
|
||||
"""
|
||||
repo = request.app.state.repo
|
||||
asset_type = repo.resolve_asset_type(symbol)
|
||||
@@ -834,6 +838,19 @@ def get_minute(
|
||||
price_limit = _get_price_limit_info(
|
||||
repo, symbol, trade_date, asset_type, stock_name,
|
||||
)
|
||||
|
||||
if live and trade_date == cn_today() and in_continuous_session():
|
||||
# 详情分时轮询: 当日盘中实时拉取最新一根K, 不落盘; 拉空(源侧延迟/
|
||||
# 时段边界)则落回下方本地优先路径。
|
||||
live_df = kline_sync.fetch_minute_single(symbol, trade_date, asset_type=asset_type)
|
||||
if not live_df.is_empty():
|
||||
return {
|
||||
"symbol": symbol, "name": stock_name, "stock_info": stock_info,
|
||||
"date": str(trade_date), "rows": live_df.to_dicts(),
|
||||
"source": "live", "asset_type": asset_type,
|
||||
"price_limit": price_limit, "prev_close": prev_close,
|
||||
}
|
||||
|
||||
df = repo.get_minute(symbol, trade_date, asset_type=asset_type)
|
||||
|
||||
# 完整交易日应有 240 条分钟K；如果是今天(盘中)，期望条数按已交易分钟估算
|
||||
@@ -1078,9 +1095,11 @@ async def clear_minute(request: Request):
|
||||
removed = 0
|
||||
if minute_dir.exists():
|
||||
try:
|
||||
result = repo.db.execute("SELECT COUNT(*) AS cnt FROM kline_minute").fetchone()
|
||||
# execute_one (cursor+close): 直连 db.execute 的未消费结果集会在 Windows 上
|
||||
# 钉住分区句柄, 导致下方 rmtree 静默删不掉被钉文件
|
||||
result = repo.execute_one("SELECT COUNT(*) AS cnt FROM kline_minute")
|
||||
removed = result[0] if result else 0
|
||||
except Exception: # noqa: BLE001
|
||||
except Exception:
|
||||
pass
|
||||
# 仅删 kline_minute 目录, 绝不触碰其他目录
|
||||
shutil.rmtree(minute_dir, ignore_errors=True)
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
"""AI 大盘复盘 API — 流式复盘 + 报告持久化。
|
||||
"""AI 大盘复盘 API — 流式复盘 + 报告持久化 + 龙虎榜 + 盘前风向标。
|
||||
|
||||
路由前缀: /api/market-recap
|
||||
|
||||
@@ -7,16 +7,19 @@
|
||||
GET /reports 历史复盘列表
|
||||
POST /reports 保存一条复盘报告
|
||||
DELETE /reports/{report_id} 删除一条复盘报告
|
||||
GET /dragon-tiger 龙虎榜三榜 (fuyao 专有, 历史按日缓存)
|
||||
GET /auction-benchmark 盘前风向标 (fuyao 专有, 含当日/次日真实收益)
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from datetime import date as date_cls
|
||||
|
||||
from fastapi import APIRouter, HTTPException, Request
|
||||
from fastapi import APIRouter, HTTPException, Query, Request
|
||||
from fastapi.responses import StreamingResponse
|
||||
from pydantic import BaseModel
|
||||
|
||||
from app.services import market_recap_reports
|
||||
from app.services import auction_benchmark, dragon_tiger, market_recap_reports
|
||||
from app.services.market_recap import recap_market_stream
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -30,6 +33,46 @@ class AnalyzeRequest(BaseModel):
|
||||
focus: str = "" # 可选:用户追加的复盘关注点
|
||||
|
||||
|
||||
@router.get("/dragon-tiger")
|
||||
def get_dragon_tiger(
|
||||
request: Request,
|
||||
date: str | None = Query(default=None, description="复盘目标日 YYYY-MM-DD, 缺省取最近已发布交易日"),
|
||||
):
|
||||
"""龙虎榜三榜 (全部/机构/游资)。fuyao 专有, 未配置时 state=source_unavailable。
|
||||
|
||||
非交易日/当日未发布由服务层自动回退到上一交易日 (state=fallback_prev)。
|
||||
"""
|
||||
target = None
|
||||
if date:
|
||||
try:
|
||||
target = date_cls.fromisoformat(date)
|
||||
except ValueError:
|
||||
raise HTTPException(400, f"date 格式应为 YYYY-MM-DD, 收到: {date}")
|
||||
return dragon_tiger.get_dragon_tiger(
|
||||
request.app.state.repo.store.data_dir, target
|
||||
)
|
||||
|
||||
|
||||
@router.get("/auction-benchmark")
|
||||
def get_auction_benchmark(
|
||||
request: Request,
|
||||
date: str | None = Query(default=None, description="复盘目标日 YYYY-MM-DD, 缺省取最近交易日"),
|
||||
):
|
||||
"""盘前风向标 (同花顺竞价筛选名单 + 当日/次日真实收益)。
|
||||
|
||||
fuyao 专有, 未配置时 state=source_unavailable; 非交易日由服务层自动回退。
|
||||
"""
|
||||
target = None
|
||||
if date:
|
||||
try:
|
||||
target = date_cls.fromisoformat(date)
|
||||
except ValueError:
|
||||
raise HTTPException(400, f"date 格式应为 YYYY-MM-DD, 收到: {date}")
|
||||
return auction_benchmark.get_auction_benchmark(
|
||||
request.app.state.repo.store.data_dir, target
|
||||
)
|
||||
|
||||
|
||||
@router.post("/analyze")
|
||||
async def analyze_market(request: Request, req: AnalyzeRequest):
|
||||
"""AI 大盘复盘 — NDJSON 流式返回。
|
||||
|
||||
@@ -107,6 +107,11 @@ class RuleModel(BaseModel):
|
||||
# ladder 专属 (连板梯队封单监控)
|
||||
metric: str = "sealed_vol" # sealed_vol=封单量(手) | sealed_amount=封单额(元)
|
||||
threshold: float = 0 # 封单 <= 此值时报警 (原始单位: 量=手, 额=元)
|
||||
# volume_delta 专属 (轮询放量监控): 相邻两次全市场快照的成交量增量
|
||||
threshold_volume: float = 9000 # 单轮增量 >= 此值(手)时报警
|
||||
threshold_amount: float = 1e6 # metric=amount 时: 单轮增量 >= 此值(元)时报警
|
||||
# 基础过滤 (与策略 basic_filter 语义对齐): 值为 null 表示不过滤
|
||||
basic_filter: dict = {}
|
||||
|
||||
|
||||
# ── 字段选项 ─────────────────────────────────────────────
|
||||
@@ -160,6 +165,7 @@ def get_options(request: Request):
|
||||
{"key": "strategy", "label": "策略监控"},
|
||||
{"key": "abnormal", "label": "异动监控"},
|
||||
{"key": "sector", "label": "板块监控"},
|
||||
{"key": "volume_delta", "label": "轮询放量"},
|
||||
],
|
||||
"scopes": [
|
||||
{"key": "symbols", "label": "指定标的"},
|
||||
|
||||
@@ -317,6 +317,9 @@ def run_preset(req: PresetRequest, request: Request):
|
||||
raise HTTPException(status_code=status_code, detail=str(e)) from e
|
||||
|
||||
safe_data = _safe(asdict(result))
|
||||
# 分钟周期结果不写入盘后缓存 (strategy_cache 是日线语义, as_of/updated_at
|
||||
# 混入分钟结果会污染页面秒加载路径)。
|
||||
if req.timeframe == "1d":
|
||||
_update_cache_strategy(data_dir, str(as_of), req.strategy_id, safe_data)
|
||||
|
||||
return _result_with_ext(safe_data, ext_values)
|
||||
@@ -583,8 +586,8 @@ def run_all(request: Request, body: Optional[dict] = None):
|
||||
elapsed = (time.perf_counter() - t_total) * 1000
|
||||
logger.info("run_all: total took %.1fms (%d strategies)", elapsed, len(all_ids))
|
||||
|
||||
# 写入策略缓存 (供页面秒加载)
|
||||
if results:
|
||||
# 写入策略缓存 (供页面秒加载); 分钟周期结果不落盘 (日线语义缓存)
|
||||
if results and timeframe == "1d":
|
||||
try:
|
||||
strategy_cache.write_cache(data_dir, str(as_of), results)
|
||||
except Exception: # noqa: BLE001
|
||||
|
||||
+62
-22
@@ -386,6 +386,21 @@ def _realtime_allowed() -> bool:
|
||||
return QuoteService.is_realtime_allowed()
|
||||
|
||||
|
||||
def _minute_history_days() -> int | None:
|
||||
"""当前分钟源的 1 分钟历史深度(交易日); None = 深历史(tickflow 基准)。
|
||||
|
||||
provider 可选类属性 minute_history_days 声明 (如 stock-sdk = 5,
|
||||
免费分时接口只保留最近 5 个交易日); 未声明或走 tickflow 时视为深历史。
|
||||
前端分时档位/默认值据此收窄。
|
||||
"""
|
||||
from app.services import kline_sync, preferences
|
||||
provider_name = preferences.get_minute_data_provider()
|
||||
provider, fallback, _err = kline_sync._resolve_minute_provider(provider_name)
|
||||
if fallback or provider is None:
|
||||
return None
|
||||
return getattr(provider, "minute_history_days", None)
|
||||
|
||||
|
||||
class MinuteSyncPrefs(BaseModel):
|
||||
minute_sync_enabled: bool
|
||||
minute_sync_days: int = 5
|
||||
@@ -397,6 +412,7 @@ class DataProvidersIn(BaseModel):
|
||||
daily_data_provider: str | None = None
|
||||
adj_factor_provider: str | None = None
|
||||
minute_data_provider: str | None = None
|
||||
depth5_data_provider: str | None = None
|
||||
realtime_data_provider: str | None = None
|
||||
financial_data_provider: str | None = None
|
||||
|
||||
@@ -478,14 +494,17 @@ def get_preferences() -> dict:
|
||||
"minute_sync_enabled": preferences.get_minute_sync_enabled(),
|
||||
"minute_sync_days": preferences.get_minute_sync_days(),
|
||||
"minute_sync_segment_days": preferences.get_minute_sync_segment_days(),
|
||||
"minute_refresh_enabled": preferences.get_minute_refresh_enabled(),
|
||||
"minute_refresh_interval": preferences.get_minute_refresh_interval(),
|
||||
"daily_data_provider": preferences.get_daily_data_provider(),
|
||||
"adj_factor_provider": preferences.get_adj_factor_provider(),
|
||||
"minute_data_provider": preferences.get_minute_data_provider(),
|
||||
"minute_history_days": _minute_history_days(),
|
||||
"depth5_data_provider": preferences.get_depth5_data_provider(),
|
||||
"realtime_data_provider": preferences.get_realtime_data_provider(),
|
||||
"financial_data_provider": preferences.get_financial_provider(),
|
||||
"data_source_job_timeout_s": preferences.get_data_source_job_timeout_s(),
|
||||
"data_source_long_job_timeout_s": preferences.get_data_source_long_job_timeout_s(),
|
||||
"realtime_watchlist_symbols": preferences.get_realtime_watchlist_symbols(),
|
||||
**preferences.get_realtime_quote_scope(),
|
||||
"pipeline_pull_a_share": preferences.get_pipeline_pull_a_share(),
|
||||
"pipeline_pull_etf": preferences.get_pipeline_pull_etf(),
|
||||
@@ -541,6 +560,31 @@ def list_data_sources() -> dict:
|
||||
}
|
||||
|
||||
|
||||
@router.get("/capability-matrix")
|
||||
def get_capability_matrix() -> dict:
|
||||
"""能力 x 源路由矩阵: 能力注册表 + 各源能力声明 + 当前路由偏好, 设置页一次拉全。
|
||||
|
||||
偏好值经 preferences getters 注入 (自带合法源校验, 非法值回退默认),
|
||||
TickFlow 当前档位由 tickflow policy 注入 (候选按档位过滤),
|
||||
组装逻辑在 data_providers.capabilities, 本层保持薄。
|
||||
"""
|
||||
from app.data_providers.capabilities import build_capability_matrix
|
||||
from app.services import preferences
|
||||
from app.tickflow import policy
|
||||
|
||||
return build_capability_matrix(
|
||||
{
|
||||
"realtime_data_provider": preferences.get_realtime_data_provider(),
|
||||
"daily_data_provider": preferences.get_daily_data_provider(),
|
||||
"minute_data_provider": preferences.get_minute_data_provider(),
|
||||
"depth5_data_provider": preferences.get_depth5_data_provider(),
|
||||
"adj_factor_provider": preferences.get_adj_factor_provider(),
|
||||
"financial_data_provider": preferences.get_financial_provider(),
|
||||
},
|
||||
tickflow_tier=policy.base_tier_name(),
|
||||
)
|
||||
|
||||
|
||||
@router.post("/plugin-key")
|
||||
def save_plugin_key(req: PluginKeyIn) -> dict:
|
||||
"""保存插件 API Key(先探后存, 对齐 /tickflow-key 语义)。
|
||||
@@ -687,9 +731,8 @@ def delete_data_source(name: str, request: Request) -> dict:
|
||||
updates["realtime_data_provider"] = "tickflow"
|
||||
if preferences.get_financial_provider() == name:
|
||||
updates["financial_data_provider"] = "tickflow"
|
||||
adj = preferences.get_adj_factor_provider()
|
||||
if adj == name:
|
||||
updates["adj_factor_provider"] = "same_as_daily"
|
||||
if preferences.get_adj_factor_provider() == name:
|
||||
updates["adj_factor_provider"] = "tickflow"
|
||||
if updates:
|
||||
preferences.save(updates)
|
||||
# 删除源可能触发偏好回退 tickflow, 同步刷新能力快照
|
||||
@@ -735,6 +778,7 @@ def update_data_providers(req: DataProvidersIn, request: Request) -> dict:
|
||||
"daily_data_provider": preferences.get_daily_data_provider(),
|
||||
"adj_factor_provider": preferences.get_adj_factor_provider(),
|
||||
"minute_data_provider": preferences.get_minute_data_provider(),
|
||||
"depth5_data_provider": preferences.get_depth5_data_provider(),
|
||||
"realtime_data_provider": preferences.get_realtime_data_provider(),
|
||||
"financial_data_provider": preferences.get_financial_provider(),
|
||||
}
|
||||
@@ -841,6 +885,15 @@ def update_minute_sync(req: MinuteSyncPrefs) -> dict:
|
||||
}
|
||||
|
||||
|
||||
@router.get("/minute-refresh/status")
|
||||
def minute_refresh_status(request: Request) -> dict:
|
||||
"""盘中分钟增量刷新服务状态 (开关/能力门控/最近一轮/下一轮)。"""
|
||||
svc = getattr(request.app.state, "minute_refresh", None)
|
||||
if svc is None:
|
||||
return {"available": False}
|
||||
return {"available": True, **svc.status()}
|
||||
|
||||
|
||||
class RealtimeQuotesPrefs(BaseModel):
|
||||
realtime_quotes_enabled: bool
|
||||
|
||||
@@ -857,8 +910,8 @@ class RealtimeQuoteScopePrefs(BaseModel):
|
||||
def update_realtime_quotes(req: RealtimeQuotesPrefs, request: Request) -> dict:
|
||||
"""保存全局实时行情开关。
|
||||
|
||||
none 档无实时行情权限；free 档开启自选股实时；starter+ 开启全市场实时。
|
||||
前端据此把开关置灰 / 回弹。
|
||||
无实时能力的档位(TickFlow none/free)开关回弹强制关闭;
|
||||
starter+ 或自定义实时源(如 fuyao)为全市场实时。前端据此把开关置灰 / 回弹。
|
||||
"""
|
||||
from app.services import preferences
|
||||
qs = getattr(request.app.state, "quote_service", None)
|
||||
@@ -912,10 +965,6 @@ def update_realtime_quotes(req: RealtimeQuotesPrefs, request: Request) -> dict:
|
||||
+ f"（任务 {job_id}）"
|
||||
)
|
||||
raise HTTPException(status_code=409, detail=detail)
|
||||
if req.realtime_quotes_enabled and qs and qs.realtime_mode() == "watchlist" and not preferences.get_realtime_watchlist_symbols():
|
||||
preferences.save({"realtime_quotes_enabled": False})
|
||||
_sync_depth_polling(False)
|
||||
return {"realtime_quotes_enabled": False, "realtime_allowed": True, "mode": "watchlist", "error": "watchlist_empty"}
|
||||
|
||||
preferences.save({"realtime_quotes_enabled": req.realtime_quotes_enabled})
|
||||
if qs:
|
||||
@@ -936,18 +985,6 @@ def update_realtime_quote_scope(req: RealtimeQuoteScopePrefs) -> dict:
|
||||
return preferences.set_realtime_quote_scope(cfg)
|
||||
|
||||
|
||||
class RealtimeWatchlistPrefs(BaseModel):
|
||||
symbols: list[str] = []
|
||||
|
||||
|
||||
@router.put("/preferences/realtime-watchlist")
|
||||
def update_realtime_watchlist(req: RealtimeWatchlistPrefs) -> dict:
|
||||
"""兼容旧入口；Free 实时标的由自选页前 5 个决定。"""
|
||||
from app.services import preferences
|
||||
symbols = preferences.set_realtime_watchlist_symbols(req.symbols)
|
||||
return {"realtime_watchlist_symbols": symbols}
|
||||
|
||||
|
||||
class IndicesNavPinnedPrefs(BaseModel):
|
||||
indices_nav_pinned: bool
|
||||
|
||||
@@ -981,6 +1018,9 @@ class RealtimeMonitorConfigIn(BaseModel):
|
||||
screener_auto_run: bool | None = None
|
||||
minute_intraday_refresh: bool | None = None
|
||||
minute_intraday_refresh_interval: int | None = None
|
||||
# 盘中分钟增量落盘 (Expert 专有) — 交易时段常驻服务, 归实时监控配置
|
||||
minute_refresh_enabled: bool | None = None
|
||||
minute_refresh_interval: int | None = None
|
||||
monitor_ext_fields: dict | None = None
|
||||
|
||||
|
||||
|
||||
@@ -813,6 +813,14 @@ class BacktestEngine:
|
||||
matrix, raw_candidates, config, progress_cb, cancel_event,
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _resolve_entry_prices(matrix: "MarketMatrix", config: "MatcherConfig") -> np.ndarray:
|
||||
"""入场价矩阵: 分钟策略的逐格覆盖有限值处优先, 否则按 open/close 惯例。"""
|
||||
base = matrix.open if config.entry_fill == "open_t+1" else matrix.close
|
||||
if matrix.entry_price is None:
|
||||
return base
|
||||
return np.where(np.isfinite(matrix.entry_price), matrix.entry_price, base)
|
||||
|
||||
def _simulate_independent_matrix(
|
||||
self,
|
||||
matrix: MarketMatrix,
|
||||
@@ -823,7 +831,7 @@ class BacktestEngine:
|
||||
options: SimulationOptions | None = None,
|
||||
) -> SimResult:
|
||||
options = options or SimulationOptions()
|
||||
entry_prices = matrix.open if config.entry_fill == "open_t+1" else matrix.close
|
||||
entry_prices = self._resolve_entry_prices(matrix, config)
|
||||
exit_prices = matrix.open if config.exit_fill == "open_t+1" else matrix.close
|
||||
buy_cost_pct = config.buy_cost_pct()
|
||||
sell_cost_pct = config.sell_cost_pct()
|
||||
@@ -1722,7 +1730,7 @@ class BacktestEngine:
|
||||
) -> SimResult:
|
||||
options = options or SimulationOptions()
|
||||
time_count, asset_count = matrix.shape
|
||||
entry_prices = matrix.open if config.entry_fill == "open_t+1" else matrix.close
|
||||
entry_prices = self._resolve_entry_prices(matrix, config)
|
||||
exit_prices = matrix.open if config.exit_fill == "open_t+1" else matrix.close
|
||||
buy_cost_pct = config.buy_cost_pct()
|
||||
sell_cost_pct = config.sell_cost_pct()
|
||||
|
||||
@@ -551,6 +551,9 @@ class MarketMatrix:
|
||||
exit_signal_code: np.ndarray
|
||||
entry_signal_ids: tuple[str, ...]
|
||||
exit_signal_ids: tuple[str, ...]
|
||||
# 逐格入场价覆盖 (time x asset, NaN=回退 open/close 惯例)。分钟策略回测用:
|
||||
# 信号在盘中第 m 根触发, 入场价 = 触发分钟收盘价, 而非当日开盘/收盘。
|
||||
entry_price: np.ndarray | None = None
|
||||
|
||||
@property
|
||||
def shape(self) -> tuple[int, int]:
|
||||
@@ -2302,11 +2305,14 @@ def build_market_matrix_from_signals(
|
||||
exit_delay_bars: int = 0,
|
||||
reference_price: np.ndarray | None = None,
|
||||
minute_exit_trigger: bool = False,
|
||||
entry_price_override: np.ndarray | None = None,
|
||||
) -> MarketMatrix:
|
||||
"""Combine base data and strategy signals into the matcher input matrix."""
|
||||
if entry_delay_bars not in (0, 1) or exit_delay_bars not in (0, 1):
|
||||
raise ValueError("phase-two MarketMatrix supports only zero or one bar delay")
|
||||
validate_signal_matrix(signals, market.shape)
|
||||
if entry_price_override is not None and entry_price_override.shape != market.shape:
|
||||
raise ValueError("entry_price_override shape does not match MarketDataMatrix")
|
||||
|
||||
present = _present_matrix(market.open, market.high, market.low, market.close, market.volume)
|
||||
entry, entry_signal_time, entry_signal_code = _delay_signal_matrix(
|
||||
@@ -2379,6 +2385,11 @@ def build_market_matrix_from_signals(
|
||||
exit_signal_code=exit_signal_code,
|
||||
entry_signal_ids=signals.entry_signal_ids,
|
||||
exit_signal_ids=signals.exit_signal_ids,
|
||||
entry_price=(
|
||||
np.array(entry_price_override, dtype=np.float32, copy=True)
|
||||
if entry_price_override is not None
|
||||
else None
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,286 @@
|
||||
"""分钟策略回测回放器 — 逐交易日回放 filter_minute_history 产生入场信号。
|
||||
|
||||
与实盘选股 (ScreenerService 1m context) 走同一条 StrategyEngine.run 执行路径,
|
||||
消除回测/实盘偏差。语义铁律:
|
||||
|
||||
- 分钟侧: 传入当日全量分钟分区, 策略函数自身因果 (第 m 根只用 <=m 的K线);
|
||||
- 日线侧: T 日的日线条件窗口只含 T-1 及更早的完成态日K — 与实盘盘中行为一致
|
||||
(当日成形K不进窗口), 杜绝未来函数;
|
||||
- 按交易日精确对日: 缺分钟分区的日子显式跳过, 不做"回退最近分区" (那是实盘语义)。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import time
|
||||
from collections.abc import Callable
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import date, timedelta
|
||||
|
||||
import polars as pl
|
||||
|
||||
from app.price_limits import is_risk_warning_name, price_limit_pct
|
||||
from app.strategy.engine import StrategyDataContext, StrategyDef, StrategyEngine
|
||||
|
||||
# 日线面板列: 基础行情 + 涨停/炸板信号 (策略日线窗口契约) + 基础过滤/展示列。
|
||||
# raw_close 用于涨停价计算 (分钟价是未复权真实价, 涨停规则定义在原始价上)。
|
||||
MINUTE_DAILY_PANEL_COLUMNS = frozenset({
|
||||
"open", "high", "low", "close", "volume", "amount",
|
||||
"raw_close", "raw_high", "raw_low",
|
||||
"turnover_rate",
|
||||
"signal_limit_up", "signal_limit_down", "signal_broken_limit_up",
|
||||
})
|
||||
MINUTE_INSTRUMENT_COLUMNS = frozenset({"name", "total_shares", "float_shares"})
|
||||
|
||||
|
||||
def minute_replay_feature_plan(daily_bars: int):
|
||||
"""分钟回测的日线面板加载计划。
|
||||
|
||||
execution_backend 用 polars_expr 走"按需计算信号"路径: enriched 分区只落
|
||||
基础列, 涨停/炸板信号由 load_panel_for_backtest 的 compute_limit_signals
|
||||
按 signal_columns 需求现算 (matrix_native 路径会跳过通用信号计算)。
|
||||
"""
|
||||
# 函数级导入规避与 strategy.py 的循环依赖 (strategy 顶层导入本模块)。
|
||||
from app.backtest.strategy import ResolvedFeaturePlan
|
||||
|
||||
return ResolvedFeaturePlan(
|
||||
base_columns=MINUTE_DAILY_PANEL_COLUMNS,
|
||||
intermediate_columns=frozenset(),
|
||||
indicator_columns=frozenset(),
|
||||
signal_columns=frozenset({
|
||||
"signal_limit_up", "signal_limit_down", "signal_broken_limit_up",
|
||||
}),
|
||||
matrix_columns=frozenset(),
|
||||
instrument_columns=MINUTE_INSTRUMENT_COLUMNS,
|
||||
warmup_bars=max(daily_bars, 1),
|
||||
full_feature_fallback=False,
|
||||
execution_backend="polars_expr",
|
||||
)
|
||||
|
||||
|
||||
def minute_panel_start(start: date, daily_bars: int) -> date:
|
||||
"""日线面板加载起点: 覆盖首个回测日的 daily_bars 交易日窗口。
|
||||
|
||||
N 个交易日约需 N*2 自然日 (周末/节假日), 再留 warmup 余量。
|
||||
"""
|
||||
calendar_days = max(daily_bars, 1) * 2 + 30
|
||||
return start - timedelta(days=calendar_days)
|
||||
|
||||
|
||||
def _trigger_hhmm(value) -> str:
|
||||
"""从 last_datetime 提取北京时间 "HH:MM" 触发分钟。
|
||||
|
||||
分区 datetime 为 UTC 存储 (tz-aware 或 naive-UTC), 统一折算到北京时区。
|
||||
"""
|
||||
from app.market_time import CN_TZ
|
||||
|
||||
if hasattr(value, "astimezone"):
|
||||
if value.tzinfo is None:
|
||||
from datetime import timezone
|
||||
|
||||
value = value.replace(tzinfo=timezone.utc)
|
||||
return value.astimezone(CN_TZ).strftime("%H:%M")
|
||||
text = str(value or "")
|
||||
if len(text) >= 16 and text[13] == ":":
|
||||
return text[11:16]
|
||||
return text[-5:] if text else ""
|
||||
|
||||
|
||||
def _scalar_limit_up_price(prev_close: float, limit_pct: float) -> float:
|
||||
"""与 polars_limit_price 同口径的标量涨停价 (整数分半进位)。"""
|
||||
cents = int(prev_close * 100 + 0.5)
|
||||
numerator = round((1 + limit_pct) * 100)
|
||||
return ((cents * numerator + 50) // 100) / 100
|
||||
|
||||
|
||||
@dataclass
|
||||
class MinuteReplayHit:
|
||||
"""一个盘中入场信号: 触发分钟收盘买入。"""
|
||||
|
||||
trade_date: date
|
||||
symbol: str
|
||||
# 已按当日 复权close/原始close 比例折算到复权价系的入场价, 与日线出场价同尺度。
|
||||
entry_price: float
|
||||
trigger_time: str # "HH:MM" — 触发分钟K的时间戳
|
||||
score: float = 0.0
|
||||
|
||||
|
||||
@dataclass
|
||||
class MinuteReplayResult:
|
||||
hits: list[MinuteReplayHit] = field(default_factory=list)
|
||||
skipped_days: list[date] = field(default_factory=list)
|
||||
replayed_days: int = 0
|
||||
strategy_matches: int = 0
|
||||
buy_limit_up: int = 0
|
||||
elapsed_ms: float = 0.0
|
||||
|
||||
|
||||
class MinuteSignalReplayer:
|
||||
"""逐交易日回放分钟策略, 产出与实盘选股同源的入场命中。"""
|
||||
|
||||
def __init__(self, engine, strategy_engine: StrategyEngine) -> None:
|
||||
# engine: BacktestEngine — 只用其 repo (分钟分区读取)。
|
||||
self.engine = engine
|
||||
self.strategy_engine = strategy_engine
|
||||
|
||||
def replay(
|
||||
self,
|
||||
strategy: StrategyDef,
|
||||
*,
|
||||
panel: pl.DataFrame,
|
||||
start: date,
|
||||
end: date,
|
||||
params: dict,
|
||||
overrides: dict,
|
||||
pool: list[str] | None = None,
|
||||
symbols: list[str] | None = None,
|
||||
progress_cb: Callable[[dict], None] | None = None,
|
||||
cancel_event=None,
|
||||
) -> MinuteReplayResult:
|
||||
t0 = time.perf_counter()
|
||||
result = MinuteReplayResult()
|
||||
repo = self.engine.repo
|
||||
if panel.is_empty():
|
||||
return result
|
||||
|
||||
universe = symbols if symbols else panel.get_column("symbol").unique().to_list()
|
||||
daily_bars = int(strategy.minute_daily_bars or 0)
|
||||
|
||||
# 面板交易日序列 (升序) — 日线窗口切片与缺分区日判定的基准。
|
||||
panel_dates = panel.get_column("date").unique().sort().to_list()
|
||||
date_to_window: dict[date, tuple[date, date]] = {}
|
||||
for i, day in enumerate(panel_dates):
|
||||
window_start = panel_dates[max(0, i - daily_bars)] if daily_bars > 0 else day
|
||||
date_to_window[day] = (window_start, day)
|
||||
|
||||
# 逐分区日回放: 只回放 [start, end] 内有分钟分区的交易日。
|
||||
minute_days = repo.list_minute_dates(start, end, "stock")
|
||||
minute_day_set = set(minute_days)
|
||||
replay_days = [day for day in panel_dates if start <= day <= end]
|
||||
result.skipped_days = [day for day in replay_days if day not in minute_day_set]
|
||||
total = len(minute_days)
|
||||
|
||||
# 逐标的的 T-1 原始收盘/复权收盘查表 (涨停价与复权折算用)。
|
||||
prev_raw_close: dict[str, float] = {}
|
||||
prev_name: dict[str, str] = {}
|
||||
adj_factor: dict[str, float] = {}
|
||||
|
||||
for i, day in enumerate(minute_days):
|
||||
if cancel_event is not None and cancel_event.is_set():
|
||||
break
|
||||
if progress_cb is not None:
|
||||
progress_cb({
|
||||
"day": i + 1,
|
||||
"total": max(total, 1),
|
||||
"date": str(day),
|
||||
})
|
||||
|
||||
history = repo.get_minute_by_dates(universe, [day], "stock")
|
||||
if history.is_empty():
|
||||
result.skipped_days.append(day)
|
||||
continue
|
||||
|
||||
# 日线窗口: 截至 T-1 的完成态日K (index of last panel date < day)。
|
||||
prior = [d for d in panel_dates if d < day]
|
||||
if not prior:
|
||||
# 面板起点之前的分区日 (窗口数据不足), 策略按数据不足自然不命中。
|
||||
daily_history = pl.DataFrame()
|
||||
current = pl.DataFrame()
|
||||
else:
|
||||
last_prior = prior[-1]
|
||||
window_start, _ = date_to_window[last_prior]
|
||||
daily_history = panel.filter(
|
||||
(pl.col("date") >= window_start) & (pl.col("date") <= last_prior)
|
||||
) if daily_bars > 0 else pl.DataFrame()
|
||||
current = panel.filter(pl.col("date") == last_prior)
|
||||
|
||||
# T-1 收盘/名称 + T 日复权因子查表。
|
||||
_refresh_day_lookups(prev_raw_close, prev_name, current, prior)
|
||||
day_rows = panel.filter(pl.col("date") == day).select(
|
||||
"symbol", "close", "raw_close",
|
||||
)
|
||||
adj_factor.clear()
|
||||
adj_factor.update(_adj_factors(day_rows))
|
||||
|
||||
context = StrategyDataContext(
|
||||
asset_type="stock",
|
||||
timeframe="1m",
|
||||
as_of=day,
|
||||
current=current if not current.is_empty() else None,
|
||||
history=history,
|
||||
daily_history=daily_history if not daily_history.is_empty() else None,
|
||||
)
|
||||
try:
|
||||
run_result = self.strategy_engine.run(
|
||||
strategy.meta.get("id", ""),
|
||||
context,
|
||||
pool,
|
||||
params,
|
||||
overrides,
|
||||
)
|
||||
except ValueError:
|
||||
# 单日执行失败 (如窗口缺列) 记为跳过, 不中断整个回放。
|
||||
result.skipped_days.append(day)
|
||||
continue
|
||||
|
||||
result.replayed_days += 1
|
||||
result.strategy_matches += len(run_result.rows)
|
||||
for row in run_result.rows:
|
||||
symbol = row.get("symbol")
|
||||
close = row.get("close")
|
||||
if not symbol or close is None or float(close) <= 0:
|
||||
continue
|
||||
raw_close = float(close)
|
||||
name = prev_name.get(str(symbol), "")
|
||||
prev = prev_raw_close.get(str(symbol))
|
||||
# 涨停拒买: 触发分钟收盘已达当日涨停价 (按 T-1 原始收盘 + 板块规则)。
|
||||
if prev is not None and prev > 0:
|
||||
limit_up = _scalar_limit_up_price(
|
||||
prev, price_limit_pct(str(symbol), day, is_risk_warning=is_risk_warning_name(name)),
|
||||
)
|
||||
if raw_close >= limit_up - 1e-9:
|
||||
result.buy_limit_up += 1
|
||||
continue
|
||||
trigger = row.get("last_datetime")
|
||||
trigger_time = _trigger_hhmm(trigger)
|
||||
result.hits.append(MinuteReplayHit(
|
||||
trade_date=day,
|
||||
symbol=str(symbol),
|
||||
entry_price=raw_close * adj_factor.get(str(symbol), 1.0),
|
||||
trigger_time=trigger_time,
|
||||
score=float(run_result.scores.get(str(symbol), 0.0) or 0.0),
|
||||
))
|
||||
|
||||
result.elapsed_ms = round((time.perf_counter() - t0) * 1000, 1)
|
||||
return result
|
||||
|
||||
|
||||
def _refresh_day_lookups(
|
||||
prev_raw_close: dict[str, float],
|
||||
prev_name: dict[str, str],
|
||||
prior_snapshot: pl.DataFrame,
|
||||
prior: list[date],
|
||||
) -> None:
|
||||
"""从 T-1 快照刷新逐标的原始收盘与名称查表 (涨停价/ST 判定用)。"""
|
||||
if prior_snapshot.is_empty():
|
||||
return
|
||||
frame = prior_snapshot
|
||||
if "raw_close" not in frame.columns:
|
||||
frame = frame.with_columns(pl.col("close").alias("raw_close"))
|
||||
if "name" not in frame.columns:
|
||||
frame = frame.with_columns(pl.lit("").alias("name"))
|
||||
prev_raw_close.clear()
|
||||
prev_name.clear()
|
||||
for symbol, raw_close, name in frame.select("symbol", "raw_close", "name").iter_rows():
|
||||
prev_raw_close[str(symbol)] = float(raw_close) if raw_close is not None else 0.0
|
||||
prev_name[str(symbol)] = str(name or "")
|
||||
|
||||
|
||||
def _adj_factors(day_rows: pl.DataFrame) -> dict[str, float]:
|
||||
"""T 日 复权close/原始close 比例: 把分钟原始价折算到复权价系。"""
|
||||
factors: dict[str, float] = {}
|
||||
if day_rows.is_empty() or "raw_close" not in day_rows.columns:
|
||||
return factors
|
||||
for symbol, close, raw_close in day_rows.select("symbol", "close", "raw_close").iter_rows():
|
||||
if close and raw_close and float(raw_close) > 0:
|
||||
factors[str(symbol)] = float(close) / float(raw_close)
|
||||
return factors
|
||||
@@ -28,13 +28,20 @@ from app.backtest.matrix import (
|
||||
MatrixPipelineConfig,
|
||||
MatrixPrewarmCancelledError,
|
||||
MatrixStrategyPipeline,
|
||||
SignalMatrix,
|
||||
apply_time_masks,
|
||||
build_market_data_matrix,
|
||||
build_market_matrix,
|
||||
build_market_matrix_from_signals,
|
||||
rolling_mean,
|
||||
slice_market_data_matrix,
|
||||
slice_signal_matrix,
|
||||
)
|
||||
from app.backtest.minute_replay import (
|
||||
MinuteSignalReplayer,
|
||||
minute_panel_start,
|
||||
minute_replay_feature_plan,
|
||||
)
|
||||
from app.backtest.minute_trigger import unsupported_minute_exit_signals
|
||||
from app.config import settings
|
||||
from app.indicators.pipeline import (
|
||||
@@ -995,7 +1002,7 @@ class StrategyBacktestService:
|
||||
s,
|
||||
StrategyDataContext(
|
||||
asset_type=config.asset_type,
|
||||
timeframe="1d",
|
||||
timeframe="1m" if s.execution_backend == "minute_filter" else "1d",
|
||||
as_of=config.end,
|
||||
),
|
||||
)
|
||||
@@ -1044,6 +1051,26 @@ class StrategyBacktestService:
|
||||
overrides.get("score_max"),
|
||||
)
|
||||
|
||||
if s.execution_backend == "minute_filter":
|
||||
# 分钟策略回测: 逐交易日回放 filter_minute_history (与实盘选股同源),
|
||||
# 信号分钟收盘价入场, 之后复用日K矩阵模拟的离场与组合管理。
|
||||
return self._run_minute_backtest(
|
||||
config, s, params, overrides,
|
||||
stop_loss=stop_loss,
|
||||
take_profit=take_profit,
|
||||
trailing_stop=trailing_stop,
|
||||
trailing_take_profit_activate=trailing_take_profit_activate,
|
||||
trailing_take_profit_drawdown=trailing_take_profit_drawdown,
|
||||
max_hold_days=max_hold_days,
|
||||
score_min=score_min,
|
||||
score_max=score_max,
|
||||
progress_cb=progress_cb,
|
||||
cancel_event=cancel_event,
|
||||
result_policy=result_policy,
|
||||
run_id=run_id,
|
||||
t0=t0,
|
||||
)
|
||||
|
||||
try:
|
||||
if s.execution_backend == "composite":
|
||||
# composite 回测: 子策略必须全为 matrix_native(否则 fail-closed),
|
||||
@@ -1629,6 +1656,300 @@ class StrategyBacktestService:
|
||||
elapsed_ms=round(elapsed, 1),
|
||||
)
|
||||
|
||||
# ── 分钟策略回测: 逐日回放入场 + 日K矩阵离场 ──
|
||||
|
||||
def _run_minute_backtest(
|
||||
self,
|
||||
config: StrategyBacktestConfig,
|
||||
s: StrategyDef,
|
||||
params: dict,
|
||||
overrides: dict,
|
||||
*,
|
||||
stop_loss,
|
||||
take_profit,
|
||||
trailing_stop,
|
||||
trailing_take_profit_activate,
|
||||
trailing_take_profit_drawdown,
|
||||
max_hold_days,
|
||||
score_min,
|
||||
score_max,
|
||||
progress_cb,
|
||||
cancel_event,
|
||||
result_policy: BacktestResultPolicy,
|
||||
run_id: str,
|
||||
t0: float,
|
||||
) -> StrategyBacktestResult:
|
||||
def _err(msg: str) -> StrategyBacktestResult:
|
||||
return StrategyBacktestResult(
|
||||
run_id=run_id,
|
||||
config=self._config_to_dict(config),
|
||||
error=msg,
|
||||
elapsed_ms=(time.perf_counter() - t0) * 1000,
|
||||
)
|
||||
|
||||
if config.asset_type != "stock":
|
||||
return _err("分钟策略回测当前仅支持 A 股 (stock)")
|
||||
if config.exit_fill == "signal_next_minute":
|
||||
return _err("分钟策略回测暂不支持「信号触发卖出」离场口径")
|
||||
|
||||
minute_days = self.engine.repo.list_minute_dates(config.start, config.end, "stock")
|
||||
if not minute_days:
|
||||
earliest = self.engine.repo.earliest_minute_date()
|
||||
hint = f"本地分钟K最早到 {earliest}, " if earliest else "本地无分钟K数据, "
|
||||
return _err(
|
||||
f"回测区间内无分钟K数据: {hint}请先用「扩展分钟K历史」拉取, 或开启盘中分钟增量"
|
||||
)
|
||||
|
||||
# 日线面板一次加载: 覆盖首个回测日的日线窗口 + 模拟区间 (含 full 模式尾部)。
|
||||
daily_bars = int(s.minute_daily_bars or 0)
|
||||
feature_plan = minute_replay_feature_plan(daily_bars)
|
||||
load_start = minute_panel_start(config.start, daily_bars)
|
||||
full_horizon_days = int(max_hold_days or config.holding_days or 5)
|
||||
load_end = config.end
|
||||
if config.mode == "full":
|
||||
load_end = config.end + timedelta(days=(full_horizon_days + 5) * 2)
|
||||
sim_end = load_end if config.mode == "full" else config.end
|
||||
|
||||
timing_ms: dict[str, float] = {}
|
||||
t_load = time.perf_counter()
|
||||
try:
|
||||
panel = self.engine.load_panel_for_backtest(
|
||||
config.symbols,
|
||||
load_start,
|
||||
load_end,
|
||||
feature_plan,
|
||||
asset_type="stock",
|
||||
)
|
||||
except (ValueError, OSError, pl.exceptions.PolarsError) as e:
|
||||
return _err(f"回测特征准备失败: {e}")
|
||||
timing_ms["load_panel"] = round((time.perf_counter() - t_load) * 1000, 1)
|
||||
if panel.is_empty():
|
||||
return _err("无日线数据, 请检查日期范围或先运行盘后管道")
|
||||
|
||||
replayer = MinuteSignalReplayer(self.engine, self.strategy_engine)
|
||||
replay = replayer.replay(
|
||||
s,
|
||||
panel=panel,
|
||||
start=config.start,
|
||||
end=config.end,
|
||||
params=params,
|
||||
overrides=overrides,
|
||||
symbols=config.symbols,
|
||||
progress_cb=progress_cb,
|
||||
cancel_event=cancel_event,
|
||||
)
|
||||
timing_ms["minute_replay"] = replay.elapsed_ms
|
||||
if cancel_event is not None and cancel_event.is_set():
|
||||
return StrategyBacktestResult(
|
||||
run_id=run_id,
|
||||
config=self._config_to_dict(config),
|
||||
error="cancelled",
|
||||
elapsed_ms=round((time.perf_counter() - t0) * 1000, 1),
|
||||
)
|
||||
if not replay.hits:
|
||||
skipped_hint = (
|
||||
f" (区间内 {len(replay.skipped_days)} 个交易日缺分钟K分区被跳过)"
|
||||
if replay.skipped_days else ""
|
||||
)
|
||||
return _err("在指定区间内未产生买入信号" + skipped_hint)
|
||||
|
||||
# 日频信号网格: 正式区间面板 → time x asset 矩阵, 命中格写入入场价覆盖。
|
||||
sim_panel = panel.filter(
|
||||
(pl.col("date") >= config.start) & (pl.col("date") <= sim_end)
|
||||
)
|
||||
if sim_panel.is_empty():
|
||||
return _err("正式回测区间内无数据")
|
||||
axis_dates = sim_panel.get_column("date").unique().sort().to_list()
|
||||
# 轴顺序必须与 build_market_data_matrix 的 _encode_axes 一致 (unique().sort()),
|
||||
# 否则 (time, asset) 下标指向错误的标的。
|
||||
axis_symbols = sim_panel.get_column("symbol").cast(pl.Utf8).unique().sort().to_list()
|
||||
time_index = {day: i for i, day in enumerate(axis_dates)}
|
||||
asset_index = {sym: i for i, sym in enumerate(axis_symbols)}
|
||||
shape = (len(axis_dates), len(axis_symbols))
|
||||
|
||||
entry = np.zeros(shape, dtype=np.uint8)
|
||||
score = np.zeros(shape, dtype=np.float32)
|
||||
entry_price_override = np.full(shape, np.nan, dtype=np.float32)
|
||||
trigger_times: dict[tuple[str, date], str] = {}
|
||||
dropped_axis_hits = 0
|
||||
for hit in replay.hits:
|
||||
time_id = time_index.get(hit.trade_date)
|
||||
asset_id = asset_index.get(hit.symbol)
|
||||
if time_id is None or asset_id is None:
|
||||
dropped_axis_hits += 1
|
||||
continue
|
||||
entry[time_id, asset_id] = 1
|
||||
score[time_id, asset_id] = hit.score
|
||||
entry_price_override[time_id, asset_id] = hit.entry_price
|
||||
trigger_times[(hit.symbol, hit.trade_date)] = hit.trigger_time
|
||||
raw_candidates = int(entry.sum())
|
||||
entry.setflags(write=False)
|
||||
score.setflags(write=False)
|
||||
entry_price_override.setflags(write=False)
|
||||
exit_mask = np.zeros(shape, dtype=np.uint8)
|
||||
exit_mask.setflags(write=False)
|
||||
codes = np.zeros(shape, dtype=np.int16)
|
||||
codes.setflags(write=False)
|
||||
signals = SignalMatrix(
|
||||
entry=entry,
|
||||
exit=exit_mask,
|
||||
score=score,
|
||||
entry_signal_code=codes,
|
||||
exit_signal_code=codes,
|
||||
entry_signal_ids=(),
|
||||
exit_signal_ids=(),
|
||||
)
|
||||
|
||||
matcher_config = MatcherConfig(
|
||||
matching=config.matching,
|
||||
entry_fill="close_t",
|
||||
exit_fill=config.exit_fill,
|
||||
fees_pct=config.fees_pct,
|
||||
commission_pct=config.commission_pct,
|
||||
stamp_tax_pct=config.stamp_tax_pct,
|
||||
slippage_bps=config.slippage_bps,
|
||||
stop_loss_pct=stop_loss,
|
||||
take_profit_pct=take_profit,
|
||||
trailing_stop_pct=trailing_stop,
|
||||
trailing_take_profit_activate_pct=trailing_take_profit_activate,
|
||||
trailing_take_profit_drawdown_pct=trailing_take_profit_drawdown,
|
||||
max_hold_days=max_hold_days,
|
||||
max_positions=config.max_positions,
|
||||
max_exposure_pct=config.max_exposure_pct,
|
||||
score_min=score_min,
|
||||
score_max=score_max,
|
||||
initial_capital=config.initial_capital,
|
||||
position_sizing=config.position_sizing,
|
||||
# 分钟策略的成交价由 entry_price_override 提供 (触发分钟收盘),
|
||||
# 不再叠加日线口径的分钟成交细化。
|
||||
minute_fill=False,
|
||||
)
|
||||
|
||||
t_matrix = time.perf_counter()
|
||||
market_data = build_market_data_matrix(sim_panel)
|
||||
market_matrix = build_market_matrix_from_signals(
|
||||
market_data,
|
||||
signals,
|
||||
# 入场即信号日盘中 (分钟价覆盖), 离场沿用日K口径。
|
||||
entry_delay_bars=0,
|
||||
exit_delay_bars=1 if matcher_config.exit_fill == "open_t+1" else 0,
|
||||
entry_price_override=entry_price_override,
|
||||
)
|
||||
timing_ms["matrix_build"] = round((time.perf_counter() - t_matrix) * 1000, 1)
|
||||
del sim_panel, market_data
|
||||
|
||||
t_sim = time.perf_counter()
|
||||
if config.mode == "full":
|
||||
result = self.engine.simulate_independent_market_matrix(
|
||||
market_matrix,
|
||||
raw_candidates,
|
||||
matcher_config,
|
||||
progress_cb,
|
||||
cancel_event,
|
||||
result_policy.simulation_options(),
|
||||
)
|
||||
else:
|
||||
result = self.engine.simulate_market_matrix(
|
||||
market_matrix,
|
||||
matcher_config,
|
||||
progress_cb,
|
||||
cancel_event,
|
||||
result_policy.simulation_options(),
|
||||
)
|
||||
timing_ms["simulate"] = round((time.perf_counter() - t_sim) * 1000, 1)
|
||||
timing_ms["statistics"] = float(result.stats.pop("statistics_ms", 0.0))
|
||||
|
||||
if cancel_event is not None and cancel_event.is_set():
|
||||
return StrategyBacktestResult(
|
||||
run_id=run_id,
|
||||
config=self._config_to_dict(config),
|
||||
error="cancelled",
|
||||
elapsed_ms=round((time.perf_counter() - t0) * 1000, 1),
|
||||
)
|
||||
if result.stats.get("error"):
|
||||
return _err(result.stats["error"])
|
||||
|
||||
execution = result.stats.get("execution") or {}
|
||||
execution["buy_limit_up"] = int(execution.get("buy_limit_up", 0)) + replay.buy_limit_up
|
||||
result.stats["execution"] = execution
|
||||
timing_ms["total"] = round((time.perf_counter() - t0) * 1000, 1)
|
||||
result.stats["timing_ms"] = timing_ms
|
||||
result.stats["panel_rows"] = int(len(axis_dates) * len(axis_symbols))
|
||||
result.stats["panel_columns"] = 0
|
||||
result.stats["feature_columns"] = 0
|
||||
result.stats["execution_backend"] = s.execution_backend
|
||||
result.stats["selection"] = {
|
||||
"strategy_matches": replay.strategy_matches,
|
||||
"entry_candidates": raw_candidates,
|
||||
"entry_trigger_filtered": max(replay.strategy_matches - raw_candidates, 0),
|
||||
"entry_trigger_enabled": False,
|
||||
}
|
||||
result.stats["minute_replay"] = {
|
||||
"replayed_days": replay.replayed_days,
|
||||
"skipped_days": [str(day) for day in replay.skipped_days[:50]],
|
||||
"skipped_day_count": len(replay.skipped_days),
|
||||
"dropped_axis_hits": dropped_axis_hits,
|
||||
}
|
||||
|
||||
benchmark_curve = (
|
||||
self._build_benchmark_curve(config.start, config.end)
|
||||
if result_policy.include_benchmark
|
||||
else []
|
||||
)
|
||||
strategy_info = {
|
||||
"id": s.meta.get("id", config.strategy_id),
|
||||
"name": s.meta.get("name", config.strategy_id),
|
||||
"description": s.meta.get("description", ""),
|
||||
"entry_signals": [],
|
||||
"exit_signals": [],
|
||||
"stop_loss": stop_loss,
|
||||
"take_profit": take_profit,
|
||||
"trailing_stop": trailing_stop,
|
||||
"trailing_take_profit_activate": trailing_take_profit_activate,
|
||||
"trailing_take_profit_drawdown": trailing_take_profit_drawdown,
|
||||
"max_hold_days": max_hold_days,
|
||||
"full_horizon_days": full_horizon_days,
|
||||
"score_min": score_min,
|
||||
"score_max": score_max,
|
||||
"source": s.source,
|
||||
"execution_backend": s.execution_backend,
|
||||
} if result_policy.include_strategy_info else {}
|
||||
|
||||
trades = (
|
||||
[self._trade_to_dict(t) for t in result.trades]
|
||||
if result_policy.include_trades
|
||||
else []
|
||||
)
|
||||
# 入场时间戳补分钟: 交易记录携带触发分钟 (HH:MM), 与日线回测的纯日期区分。
|
||||
for trade in trades:
|
||||
entry_text = str(trade.get("entry_date") or "")
|
||||
try:
|
||||
key = (str(trade.get("symbol")), date.fromisoformat(entry_text[:10]))
|
||||
except ValueError:
|
||||
continue
|
||||
trigger = trigger_times.get(key)
|
||||
if trigger:
|
||||
trade["entry_date"] = f"{entry_text[:10]} {trigger}"
|
||||
|
||||
selected_stats = result_policy.select_stats(result.stats)
|
||||
elapsed = (time.perf_counter() - t0) * 1000
|
||||
return StrategyBacktestResult(
|
||||
run_id=run_id,
|
||||
config=self._config_to_dict(config),
|
||||
stats=selected_stats,
|
||||
equity_curve=result.equity_curve if result_policy.include_curves else [],
|
||||
drawdown_curve=result.drawdown_curve if result_policy.include_curves else [],
|
||||
benchmark_curve=benchmark_curve,
|
||||
trades=trades,
|
||||
per_symbol_stats=(
|
||||
result.per_symbol_stats
|
||||
if result_policy.include_per_symbol_stats
|
||||
else []
|
||||
),
|
||||
strategy_info=strategy_info,
|
||||
elapsed_ms=round(elapsed, 1),
|
||||
)
|
||||
|
||||
# ── 全量模拟 (选股能力统计, 不建组合不算净值) ──
|
||||
|
||||
def _run_full_simulation(
|
||||
|
||||
@@ -257,9 +257,10 @@ def _worker_entry(task: dict[str, Any], event_queue, cancel_event) -> None:
|
||||
if store is not None:
|
||||
with suppress(Exception):
|
||||
store.db.close()
|
||||
# 终态消息已入队: 显式冲刷队列后立即退出。大数据量任务跳过解释器
|
||||
# teardown (GC、DuckDB 线程 join、DLL 卸载), 否则收尾可达数十秒,
|
||||
# 会撞上父进程 10s 退出预算。close+join_thread 保证消息完整落管。
|
||||
# 终态消息已入队: 显式冲刷队列后立即退出。put 只是入队, 实际写管道的
|
||||
# 是后台 feeder 线程, close+join_thread 保证消息完整落管 (否则父进程误判
|
||||
# "exited without result"); 大数据量任务再跳过解释器 teardown (GC、DuckDB
|
||||
# 线程 join、DLL 卸载), 否则收尾可达数十秒, 撞上父进程 10s 退出预算。
|
||||
with suppress(Exception):
|
||||
event_queue.close()
|
||||
event_queue.join_thread()
|
||||
@@ -320,6 +321,21 @@ def run_worker_task(
|
||||
elif message_type == "error":
|
||||
failure = message
|
||||
|
||||
# 子进程退出后, 队列读线程可能尚未把管道尾部的 result/error 搬进本地缓冲
|
||||
# (0.1s 轮询在系统高负载下会先看到 Empty+进程已死)。join 后做一次兜底排空,
|
||||
# 只要消息完整刷入过管道就一定能取到。
|
||||
if result is None and failure is None:
|
||||
for _ in range(2):
|
||||
try:
|
||||
message = events.get(timeout=1.0)
|
||||
except queue.Empty:
|
||||
break
|
||||
message_type = message.get("type")
|
||||
if message_type == "result":
|
||||
result = message["payload"]
|
||||
elif message_type == "error":
|
||||
failure = message
|
||||
|
||||
process.join(timeout=10.0)
|
||||
worker_exit_forcibly = False
|
||||
if process.is_alive():
|
||||
|
||||
@@ -0,0 +1,201 @@
|
||||
"""能力注册表与能力路由矩阵 — 数据集维度的单一权威定义。
|
||||
|
||||
能力 (capability) = 一个标准化数据集 (CONTRIBUTING「数据源插件化要求」):
|
||||
daily / adj_factor / realtime / minute / depth5 / financial (注册表顺序即设置页卡片顺序)。注册表集中声明每个
|
||||
能力的展示元数据、路由偏好字段与 TickFlow 档位要求, 前端设置页不再各自硬编码。
|
||||
depth5 目前仅 TickFlow 供 (插件数据集白名单未开放, 见 loader), 仍进矩阵是为了
|
||||
可用性门控诚实: 五档不可用时连板梯队封单/看板封单缺数据应有提示。
|
||||
|
||||
build_capability_matrix 把注册表、插件/自定义源的能力声明 (datasets) 和当前
|
||||
路由偏好合并为一个矩阵, 供设置页一次拉全。当前偏好由 API 层注入
|
||||
(preferences getters 自带合法源校验), 本模块不反向依赖 services 层。
|
||||
|
||||
候选契约: 每个能力的 candidates 只包含「当前确实可提供该能力」的源 —
|
||||
TickFlow 按当前订阅档位过滤 (日K全档位, 其余按注册表 tf_tier 门槛),
|
||||
未就绪的插件/自定义源 (依赖未装/Key 未配) 放入 pending 并携带原因,
|
||||
供前端置灰提示。其他页面可以把 candidates 直接当作可用提供方名单。
|
||||
|
||||
usable 契约: 每个能力额外给出 usable = 生效源当前能否真正提供该能力
|
||||
(生效源在 candidates 中)。各页面的能力门控 (缺能力提示 → 数据源配置)
|
||||
统一以 usable 为准, 而不是 TickFlow 套餐视角 — 路由到可用插件时同样可用,
|
||||
路由到 TickFlow 但档位不足时同样不可用。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from app.data_providers import custom as custom_sources
|
||||
|
||||
CAPABILITY_REGISTRY: list[dict] = [
|
||||
{
|
||||
"id": "daily",
|
||||
"label": "日K",
|
||||
"desc": "历史K线与实时覆写",
|
||||
"field": "daily_data_provider",
|
||||
"default": "tickflow",
|
||||
"tf_tier": "none",
|
||||
},
|
||||
{
|
||||
"id": "adj_factor",
|
||||
"label": "除权因子",
|
||||
"desc": "前复权计算基准",
|
||||
"field": "adj_factor_provider",
|
||||
"default": "tickflow",
|
||||
"tf_tier": "starter",
|
||||
# 独立路由 (曾经的「跟随日K」特殊值已下线: 每个能力单独配置,
|
||||
# 复权口径一致性改由未来的一致性警示保障, 不做路由耦合)
|
||||
},
|
||||
{
|
||||
"id": "realtime",
|
||||
"label": "实时行情",
|
||||
"desc": "全市场实时快照",
|
||||
"field": "realtime_data_provider",
|
||||
"default": "tickflow",
|
||||
"tf_tier": "starter",
|
||||
},
|
||||
{
|
||||
"id": "minute",
|
||||
"label": "分钟K",
|
||||
"desc": "分时图与分钟回测",
|
||||
"field": "minute_data_provider",
|
||||
"default": "tickflow",
|
||||
"tf_tier": "pro",
|
||||
},
|
||||
{
|
||||
"id": "depth5",
|
||||
"label": "五档盘口",
|
||||
"desc": "连板梯队封单与盘口深度",
|
||||
"field": "depth5_data_provider",
|
||||
"default": "tickflow",
|
||||
"tf_tier": "pro",
|
||||
# 插件契约暂未开放 depth5 数据集 (loader 白名单), 当前仅 TickFlow 供
|
||||
},
|
||||
{
|
||||
"id": "financial",
|
||||
"label": "财务数据",
|
||||
"desc": "财务指标与三大报表",
|
||||
"field": "financial_data_provider",
|
||||
"default": "tickflow",
|
||||
"tf_tier": "expert",
|
||||
},
|
||||
{
|
||||
"id": "full_minute",
|
||||
"label": "全量分钟",
|
||||
"desc": "盘中全市场当日分钟落盘 (冷启动全天 + 标的池增量)",
|
||||
"field": None,
|
||||
"default": "tickflow",
|
||||
"tf_tier": "expert",
|
||||
# intraday.universe 能力 (TickFlow Expert 专有): 插件契约不开放此数据集,
|
||||
# 生效源恒为 TickFlow — 不可路由, 无对应 provider 偏好字段
|
||||
},
|
||||
]
|
||||
|
||||
_TICKFLOW_CANDIDATE = {
|
||||
"name": "tickflow",
|
||||
"display": "TickFlow",
|
||||
"kind": "builtin",
|
||||
"available": True,
|
||||
"status": "ok",
|
||||
"note": None,
|
||||
}
|
||||
|
||||
# 档位排序: none 最低 (无 Key/无效 Key, 仅免费通道历史日K), 未知档按 none 处理 (fail-closed)
|
||||
_TIER_RANK = {"none": -1, "free": 0, "starter": 1, "pro": 2, "expert": 3}
|
||||
|
||||
|
||||
def _tier_base(tier: str) -> str:
|
||||
"""归一化档位输入为基础名: "Pro +" -> "pro"; 空值归为 none。"""
|
||||
text = str(tier or "").strip().lower()
|
||||
if not text:
|
||||
return "none"
|
||||
return text.split()[0].split("+")[0]
|
||||
|
||||
|
||||
def _declared_sources() -> list[dict]:
|
||||
"""插件 + 自定义源 → 统一能力声明视图。未注册 (hidden/加载失败) 的源不会出现。"""
|
||||
rows: list[dict] = []
|
||||
for plugin in custom_sources.list_plugins():
|
||||
rows.append({
|
||||
"name": plugin["name"],
|
||||
"display": plugin.get("display_name") or plugin["name"],
|
||||
"datasets": set(plugin.get("datasets") or []),
|
||||
"available": bool(plugin.get("available")),
|
||||
"status": str(plugin.get("status") or ""),
|
||||
"kind": "plugin",
|
||||
})
|
||||
for source in custom_sources.list_sources():
|
||||
rows.append({
|
||||
"name": source["name"],
|
||||
"display": source.get("display_name") or source["name"],
|
||||
"datasets": set(source.get("datasets") or []),
|
||||
# 自定义源注册即已通过加载校验, 视为可用
|
||||
"available": True,
|
||||
"status": "ok",
|
||||
"kind": "custom",
|
||||
})
|
||||
return rows
|
||||
|
||||
|
||||
def _display_of(sources: list[dict], name: str) -> str:
|
||||
if name == "tickflow":
|
||||
return "TickFlow"
|
||||
for s in sources:
|
||||
if s["name"] == name:
|
||||
return s["display"]
|
||||
return name
|
||||
|
||||
|
||||
def build_capability_matrix(current: dict[str, str], tickflow_tier: str = "none") -> dict:
|
||||
"""注册表 + 源能力声明 + 当前偏好 → 能力路由矩阵。
|
||||
|
||||
current 为 {偏好字段: 当前值}, 由 API 层经 preferences getters 注入;
|
||||
getters 已把非法值 (未注册源) 回退为默认, 这里直接信任。effective
|
||||
即当前值本身 (每个能力独立路由, 无跟随/派生特殊值)。
|
||||
|
||||
tickflow_tier 为 TickFlow 当前档位基础名 (none/free/starter/pro/expert),
|
||||
由 API 层从 tickflow policy 注入。当前档位不提供的能力里 TickFlow
|
||||
不进候选, 但偏好仍指向 tickflow 时以 tf_available=False 标记,
|
||||
供前端提示「档位不足」。未知档按 none 处理。
|
||||
"""
|
||||
tier_base = _tier_base(tickflow_tier)
|
||||
tier_rank = _TIER_RANK.get(tier_base, -1)
|
||||
sources = _declared_sources()
|
||||
|
||||
capabilities = []
|
||||
for cap in CAPABILITY_REGISTRY:
|
||||
# field=None → 不可路由能力 (仅 TickFlow 提供, 无路由偏好), 生效源恒为默认
|
||||
effective = current.get(cap["field"], cap["default"]) if cap["field"] else cap["default"]
|
||||
tf_available = tier_rank >= _TIER_RANK[cap["tf_tier"]]
|
||||
candidates: list[dict] = []
|
||||
pending: list[dict] = []
|
||||
if tf_available:
|
||||
candidates.append(dict(_TICKFLOW_CANDIDATE))
|
||||
for s in sources:
|
||||
if cap["id"] not in s["datasets"]:
|
||||
continue
|
||||
entry = {
|
||||
"name": s["name"],
|
||||
"display": s["display"],
|
||||
"kind": s["kind"],
|
||||
"available": s["available"],
|
||||
"status": s["status"],
|
||||
"note": None if s["available"] else (s["status"] or "不可用"),
|
||||
}
|
||||
(candidates if s["available"] else pending).append(entry)
|
||||
usable = any(c["name"] == effective for c in candidates)
|
||||
capabilities.append({
|
||||
"id": cap["id"],
|
||||
"label": cap["label"],
|
||||
"desc": cap["desc"],
|
||||
"field": cap["field"],
|
||||
"default": cap["default"],
|
||||
"tf_tier": cap["tf_tier"],
|
||||
"tf_available": tf_available,
|
||||
"usable": usable,
|
||||
"current": effective,
|
||||
"current_display": _display_of(sources, effective),
|
||||
"effective": effective,
|
||||
"effective_display": _display_of(sources, effective),
|
||||
"candidates": candidates,
|
||||
"pending": pending,
|
||||
})
|
||||
return {"tickflow_tier": tier_base, "capabilities": capabilities}
|
||||
@@ -37,6 +37,9 @@ class DatasetConfig:
|
||||
end_param: str = "end_time"
|
||||
asset_type_param: str | None = None
|
||||
freq_param: str | None = None
|
||||
# realtime 比例字段(change_pct/amplitude/turnover_rate)的单位声明:
|
||||
# "percent"(返回 3.66 表示 3.66%)或 "decimal"(返回 0.0366 表示 3.66%)。
|
||||
pct_unit: str | None = None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
@@ -75,6 +78,10 @@ def _dataset_from_dict(raw: dict[str, Any]) -> DatasetConfig:
|
||||
if not 0 < timeout <= MAX_TIMEOUT:
|
||||
raise ValueError(f"timeout must be between 0 and {MAX_TIMEOUT:g} seconds")
|
||||
|
||||
pct_unit = str(raw.get("pct_unit") or "").strip().lower() or None
|
||||
if pct_unit not in (None, "percent", "decimal"):
|
||||
raise ValueError(f"pct_unit must be 'percent' or 'decimal', got {pct_unit!r}")
|
||||
|
||||
return DatasetConfig(
|
||||
url=str(raw.get("url", "") or ""),
|
||||
method=str(raw.get("method", "GET") or "GET").upper(),
|
||||
@@ -91,6 +98,7 @@ def _dataset_from_dict(raw: dict[str, Any]) -> DatasetConfig:
|
||||
end_param=str(raw.get("end_param", "end_time") or "end_time").strip() or "end_time",
|
||||
asset_type_param=(str(raw.get("asset_type_param") or "").strip() or None),
|
||||
freq_param=(str(raw.get("freq_param") or "").strip() or None),
|
||||
pct_unit=pct_unit,
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -11,6 +11,7 @@ from pathlib import Path
|
||||
|
||||
import yaml
|
||||
|
||||
from app import secrets_store
|
||||
from app.config import settings
|
||||
from app.data_providers.custom.config import (
|
||||
DEFAULT_TIMEOUT,
|
||||
@@ -89,6 +90,19 @@ def list_plugins() -> list[dict]:
|
||||
return list(_PLUGIN_STATUS.values())
|
||||
|
||||
|
||||
def _plugin_key_masked(name: str, api_key_env: str) -> str:
|
||||
"""插件当前生效 Key 的脱敏串 (secrets.json 优先, .env 兜底), 未配置返回空。
|
||||
|
||||
与 TickFlow Key 的展示契约一致 (settings API 的 tickflow_api_key_masked):
|
||||
完整 Key 永不出后端, 只出 mask() 结果, 供设置页常驻显示。
|
||||
"""
|
||||
env = str(api_key_env or "").strip()
|
||||
if not env:
|
||||
return ""
|
||||
key = secrets_store.get_env_backed_secret(f"{name.lower()}_api_key", env)
|
||||
return secrets_store.mask(key) if key else ""
|
||||
|
||||
|
||||
def plugin_manifest(name: str) -> dict | None:
|
||||
"""读取指定插件的 plugin.yaml 清单。"""
|
||||
plugin_dir = plugins_dir() / (name or "")
|
||||
@@ -353,6 +367,7 @@ def _config_to_dict(config: CustomSourceConfig) -> dict:
|
||||
} if ds_name != "realtime" else {}),
|
||||
**({"asset_type_param": ds.asset_type_param} if ds_name == "minute" and ds.asset_type_param else {}),
|
||||
**({"freq_param": ds.freq_param} if ds_name == "minute" and ds.freq_param else {}),
|
||||
**({"pct_unit": ds.pct_unit} if ds_name == "realtime" and ds.pct_unit else {}),
|
||||
}
|
||||
return out
|
||||
|
||||
@@ -476,6 +491,13 @@ def _sanitize_dataset(ds_name: str, ds_cfg: dict) -> dict:
|
||||
out["start_param"] = start_param
|
||||
if end_param:
|
||||
out["end_param"] = end_param
|
||||
pct_unit = str(ds_cfg.get("pct_unit") or "").strip().lower()
|
||||
if pct_unit:
|
||||
if ds_name != "realtime":
|
||||
raise ValueError(f"{ds_name}: pct_unit 仅用于 realtime 数据集")
|
||||
if pct_unit not in ("percent", "decimal"):
|
||||
raise ValueError(f"{ds_name}: pct_unit 必须是 percent 或 decimal")
|
||||
out["pct_unit"] = pct_unit
|
||||
if ds_name == "minute":
|
||||
asset_type_param = str(ds_cfg.get("asset_type_param") or "").strip()
|
||||
freq_param = str(ds_cfg.get("freq_param") or "").strip()
|
||||
@@ -552,7 +574,9 @@ def _register_one_plugin(manifest: dict) -> None:
|
||||
"status": reason,
|
||||
"description": manifest.get("description", ""),
|
||||
"install_hint": manifest.get("install_hint", ""),
|
||||
"homepage": manifest.get("homepage", ""),
|
||||
"api_key_env": manifest.get("api_key_env", ""),
|
||||
"api_key_masked": _plugin_key_masked(name, manifest.get("api_key_env", "")),
|
||||
}
|
||||
if not available:
|
||||
return # 依赖没装: 不注册, 但状态已记录供 UI 显示
|
||||
|
||||
@@ -34,29 +34,56 @@ _REQUIRED = {
|
||||
"financial": {"symbol"},
|
||||
}
|
||||
|
||||
# 小数制下 change_pct/amplitude/turnover_rate 的物理上限: A股最大涨跌停 30% (+容差)。
|
||||
# 小数制下 change_pct 的物理上限: A股最大涨跌停 30% (+容差)。
|
||||
# 中位数口径下小数制批次不可能超过该值, 百分制批次(典型中位数 0.5~3)必然超过。
|
||||
# 仅对 change_pct 有效——amplitude/turnover_rate 的两种单位在数值区间上重叠
|
||||
# (百分制 0.05 = 0.05% 与小数制 0.05 = 5%), 无物理依据可判。
|
||||
_PCT_FRACTION_MAX = 0.31
|
||||
|
||||
_PCT_COLUMNS = ("change_pct", "amplitude", "turnover_rate")
|
||||
|
||||
def _normalize_pct_units(df: pl.DataFrame) -> pl.DataFrame:
|
||||
"""百分制源自适应归一为小数制 (契约: change_pct/amplitude/turnover_rate 为小数,
|
||||
0.0366 = 3.66%)。不少第三方接口(如 a-stock-data)直接返回 3.66 表示 3.66%,
|
||||
若不归一, 下游(行业/概念统计、前端 x100 展示)会整体放大 100 倍。
|
||||
|
||||
截面判定: 样本 >= 5 用 |值| 中位数(对个别无涨跌幅限制新股免疫),
|
||||
小样本退用最大值。整批同除 100, 避免逐值阈值在 0.3~1 区间的歧义。
|
||||
def _normalize_pct_units(
|
||||
df: pl.DataFrame,
|
||||
pct_unit: str | None = None,
|
||||
transformed_cols: frozenset[str] = frozenset(),
|
||||
) -> pl.DataFrame:
|
||||
"""比例字段单位归一为契约小数制 (change_pct/amplitude/turnover_rate,
|
||||
0.0366 = 3.66%, CONTRIBUTING §3.1)。单位只认显式声明, 不靠数值猜:
|
||||
|
||||
- pct_unit="percent" → 三列无条件 /100 (声明即契约, 即使数值看着像小数制);
|
||||
- pct_unit="decimal" → 原样透传 (即使数值看着像百分制也不动);
|
||||
- 未声明 → change_pct 保留截面中位数判定(涨跌停 30% 上限使其物理可判:
|
||||
样本 >= 5 用 |值| 中位数, 小样本退用最大值, 整批同除 100);
|
||||
amplitude/turnover_rate 置 None 交下游重算(enriched 管道按
|
||||
high/low/prev_close 与股本口径重算), 除非该列已被 transforms 显式
|
||||
处理过(视为用户已接管单位, 原样透传)。
|
||||
"""
|
||||
for col in ("change_pct", "amplitude", "turnover_rate"):
|
||||
dropped_undeclared = False
|
||||
for col in _PCT_COLUMNS:
|
||||
if col not in df.columns:
|
||||
continue
|
||||
df = df.with_columns(pl.col(col).cast(pl.Float64, strict=False).alias(col))
|
||||
if pct_unit == "percent":
|
||||
df = df.with_columns((pl.col(col) / 100).alias(col))
|
||||
elif pct_unit == "decimal" or col in transformed_cols:
|
||||
continue
|
||||
elif col == "change_pct":
|
||||
vals = df[col].drop_nulls().abs()
|
||||
if vals.is_empty():
|
||||
continue
|
||||
stat = vals.median() if vals.len() >= 5 else vals.max()
|
||||
if stat > _PCT_FRACTION_MAX:
|
||||
df = df.with_columns((pl.col(col) / 100).alias(col))
|
||||
else:
|
||||
df = df.with_columns(pl.lit(None, dtype=pl.Float64).alias(col))
|
||||
dropped_undeclared = True
|
||||
if dropped_undeclared:
|
||||
logger.warning(
|
||||
"自定义源 realtime 未声明 pct_unit: amplitude/turnover_rate 的单位"
|
||||
"无法从数值判定, 已置 None 交由下游按股本/价格口径重算;"
|
||||
"请在 realtime 数据集配置中显式声明 pct_unit: percent 或 decimal"
|
||||
)
|
||||
return df
|
||||
|
||||
|
||||
@@ -82,6 +109,11 @@ class GenericHTTPProvider:
|
||||
missing = sorted(required - mapped)
|
||||
if missing:
|
||||
errors.append(f"{dataset}: missing mapped fields: {', '.join(missing)}")
|
||||
if cfg.pct_unit is not None:
|
||||
if dataset != "realtime":
|
||||
errors.append(f"{dataset}: pct_unit 仅用于 realtime 数据集")
|
||||
elif cfg.pct_unit not in ("percent", "decimal"):
|
||||
errors.append(f"{dataset}: pct_unit 必须是 percent 或 decimal")
|
||||
if dataset != "realtime":
|
||||
request_params = [cfg.symbols_param, cfg.start_param, cfg.end_param]
|
||||
if dataset == "minute":
|
||||
@@ -146,8 +178,13 @@ class GenericHTTPProvider:
|
||||
cfg = self._dataset("realtime")
|
||||
rows = self._request_rows(cfg)
|
||||
df = self._mapped_frame(cfg, rows)
|
||||
# 百分制源(返回 3.66 表示 3.66%)截面归一为契约小数制
|
||||
df = _normalize_pct_units(df)
|
||||
# 单位归一: 显式 pct_unit 声明优先; 未声明时 amplitude/turnover_rate
|
||||
# fail-closed 置 None(交下游重算), change_pct 保留截面判定
|
||||
df = _normalize_pct_units(
|
||||
df,
|
||||
pct_unit=cfg.pct_unit,
|
||||
transformed_cols=frozenset(cfg.transforms) & set(_PCT_COLUMNS),
|
||||
)
|
||||
if df.is_empty():
|
||||
return []
|
||||
return df.to_dicts()
|
||||
|
||||
@@ -303,8 +303,6 @@ def run_now(
|
||||
written_adj = 0
|
||||
affected_symbols: list[str] = []
|
||||
adj_provider = _prefs.get_adj_factor_provider()
|
||||
if adj_provider == "same_as_daily":
|
||||
adj_provider = _prefs.get_daily_data_provider()
|
||||
can_sync_adj = capset.has(Cap.ADJ_FACTOR) or adj_provider != "tickflow"
|
||||
if can_sync_adj:
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
@@ -173,6 +173,16 @@ async def _application_lifespan(app: FastAPI):
|
||||
except Exception as e: # noqa: BLE001
|
||||
logger.warning("depth_service init failed: %s", e)
|
||||
|
||||
# 盘中分钟增量刷新 (Expert 专有): 线程常驻, 开关/时段/能力门控在循环内每轮判断
|
||||
try:
|
||||
from app.services.minute_refresh import MinuteRefreshService
|
||||
minute_refresh = MinuteRefreshService(repo)
|
||||
minute_refresh.set_app_state(app.state)
|
||||
app.state.minute_refresh = minute_refresh
|
||||
minute_refresh.start()
|
||||
except Exception as e:
|
||||
logger.warning("minute_refresh init failed: %s", e)
|
||||
|
||||
# 停机缺口自检: 延迟后台扫描, 发现最近交易日的盘中快照/缺口时自动创建
|
||||
# 修复任务 (盘中停机→次日开实时场景, 不修则坏数据被"只刷今天"分支永久留存)
|
||||
try:
|
||||
@@ -356,6 +366,9 @@ async def _application_lifespan(app: FastAPI):
|
||||
wbot = getattr(app.state, "wecom_bot_service", None)
|
||||
if wbot:
|
||||
wbot.stop()
|
||||
mrs = getattr(app.state, "minute_refresh", None)
|
||||
if mrs:
|
||||
mrs.stop()
|
||||
logger.info("shutdown")
|
||||
|
||||
|
||||
|
||||
@@ -28,6 +28,15 @@ def cn_today() -> date:
|
||||
return datetime.now(CN_TZ).date()
|
||||
|
||||
|
||||
def in_continuous_session(now: datetime | None = None) -> bool:
|
||||
"""A股连续竞价时段 (北京时间): 9:30-11:30 / 13:00-15:00, 仅工作日。"""
|
||||
now = now or cn_now()
|
||||
return now.weekday() < 5 and (
|
||||
_MORNING_START <= now.time() <= _MORNING_END
|
||||
or _AFTERNOON_START <= now.time() <= _AFTERNOON_END
|
||||
)
|
||||
|
||||
|
||||
def trading_minutes_elapsed_from_dt(dt: datetime) -> float:
|
||||
"""根据北京时间 datetime 计算当日已交易分钟数。
|
||||
|
||||
|
||||
@@ -1,12 +1,17 @@
|
||||
"""扶摇(同花顺金融数据 API) HTTP 客户端。
|
||||
|
||||
职责: 认证、统一信封解包、分页拉取快照。不知道 provider / services 层。
|
||||
职责: 认证、统一信封解包、快照分页、单标的日K、市场 dump 下载。不知道 provider / services 层。
|
||||
文档: https://fuyao.aicubes.cn/docs — REST + X-api-key, 响应信封 {code, message, data}。
|
||||
|
||||
时间字段口径: 所有 *ms 字段(含 start/end 入参与 date_ms/ex_date_ms 出参)均为
|
||||
北京时间零点对应的 epoch ms(= UTC 前一日 16:00), 由 provider 层统一 +8h 换算。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
import httpx
|
||||
|
||||
@@ -14,8 +19,9 @@ logger = logging.getLogger(__name__)
|
||||
|
||||
BASE_URL = "https://fuyao.aicubes.cn"
|
||||
|
||||
# A 股约 5400 只, 500/页约 11 页; 50 页上限防御 count 异常导致的死循环。
|
||||
_SNAPSHOT_PAGE_SIZE = 500
|
||||
# 单页 6000 覆盖全市场(实测 ~5600 含北交所, 2026-08 服务端不截断 limit=6000),
|
||||
# 一次请求拉完; 分页循环兜底未来标的扩容或服务端改为截断的场景。
|
||||
_SNAPSHOT_PAGE_SIZE = 6000
|
||||
_SNAPSHOT_MAX_PAGES = 50
|
||||
_PAGE_INTERVAL_S = 0.15 # 页间隔, 降低触发限频 (code=4001) 的概率
|
||||
|
||||
@@ -59,7 +65,9 @@ class FuyaoClient:
|
||||
return payload.get("data") or {}
|
||||
|
||||
# ---- 快照 ----
|
||||
def snapshot_page(self, limit: int = _SNAPSHOT_PAGE_SIZE, offset: int = 0) -> tuple[list[dict], int]:
|
||||
def snapshot_page(
|
||||
self, limit: int = _SNAPSHOT_PAGE_SIZE, offset: int = 0
|
||||
) -> tuple[list[dict], int]:
|
||||
"""拉取一页 A 股全市场快照。返回 (rows, total), total 为全市场总数。
|
||||
|
||||
实测响应(2026-08): data={timestamp, total, item}; 官方文档示例为
|
||||
@@ -106,3 +114,167 @@ class FuyaoClient:
|
||||
if not out:
|
||||
raise FuyaoError("全市场快照为空")
|
||||
return out, server_ts
|
||||
|
||||
# ---- 历史日K ----
|
||||
def historical_kline(
|
||||
self, thscode: str, start_ms: int, end_ms: int, adjust: str = "none"
|
||||
) -> list[dict]:
|
||||
"""单标的日K(interval=1d 固定)。单次窗口 ≤10 年, 超出由调用方分片。
|
||||
|
||||
adjust 必须显式传 "none" 取原始价 — 服务端默认是 forward(前复权),
|
||||
官方前复权序列事件间存在逐日漂移, 项目内禁止使用。
|
||||
返回 data.item 原始行: {date_ms, open_price, high_price, low_price,
|
||||
close_price, volume(股), turnover(元)}。
|
||||
"""
|
||||
data = self._get(
|
||||
"/api/a-share/prices/historical",
|
||||
{
|
||||
"thscode": thscode,
|
||||
"interval": "1d",
|
||||
"adjust": adjust,
|
||||
"start": int(start_ms),
|
||||
"end": int(end_ms),
|
||||
},
|
||||
)
|
||||
rows = data.get("item")
|
||||
return rows if isinstance(rows, list) else []
|
||||
|
||||
# ---- 财务 ----
|
||||
# 端点均单标的(thscode 不接受逗号)。取数模式二选一: limit=最近N期 或 start/end 区间,
|
||||
# 这里只用 limit。period=quarterly 覆盖每个季度末(含年报期), 与项目"各报告期累积"口径一致。
|
||||
_STATEMENT_ENDPOINTS = {
|
||||
"income": "income-statements",
|
||||
"balance_sheet": "balance-sheets",
|
||||
"cash_flow": "cash-flow-statements",
|
||||
}
|
||||
|
||||
def financial_statements(
|
||||
self, stmt: str, thscode: str, limit: int = 1
|
||||
) -> list[dict]:
|
||||
"""单标的财务报表多期序列。stmt: income | balance_sheet | cash_flow。
|
||||
|
||||
返回 data.item 原始行: 共有元数据(thscode/period/fiscal_year/fiscal_period/
|
||||
report_date_ms/period_end_ms/currency) + 各表字段。行内 null 表示该期未披露。
|
||||
"""
|
||||
endpoint = self._STATEMENT_ENDPOINTS.get(stmt)
|
||||
if endpoint is None:
|
||||
raise FuyaoError(f"未知财务报表类型: {stmt}")
|
||||
data = self._get(
|
||||
f"/api/a-share/financials/{endpoint}",
|
||||
{"thscode": thscode, "period": "quarterly", "limit": max(1, min(20, limit))},
|
||||
)
|
||||
rows = data.get("item")
|
||||
return rows if isinstance(rows, list) else []
|
||||
|
||||
def financial_indicators(self, thscode: str, report: str) -> list[dict]:
|
||||
"""单标的单报告期财务指标(report 格式 yyyy-N, N=1..4 对应一季报..年报)。
|
||||
|
||||
返回 data.abilities 原始列表 [{ability, indicators: [{index_id, value}]}];
|
||||
value 为保留原始精度的数值字符串(百分制指标即百分点数), 缺失为 null。
|
||||
未披露报告期实测返回 code=5003(文档写 3002, 以实测为准) → 经 _get 抛 FuyaoError,
|
||||
由调用方按"该期无数据"处理。
|
||||
"""
|
||||
data = self._get(
|
||||
"/api/a-share/financials/indicators",
|
||||
{"thscode": thscode, "report": report},
|
||||
)
|
||||
abilities = data.get("abilities")
|
||||
return abilities if isinstance(abilities, list) else []
|
||||
|
||||
def valuations_snapshot(self, thscodes: list[str]) -> list[dict]:
|
||||
"""批量估值快照(pe_ttm/pe_mrq/pb_mrq/ps_ttm/pcf_ttm), 数值为最新口径。
|
||||
|
||||
服务端单次上限 100 只(超出 code=1003), 分批由调用方负责。
|
||||
返回 data.item 原始行。
|
||||
"""
|
||||
data = self._get(
|
||||
"/api/a-share/valuations/snapshot",
|
||||
{"thscodes": ",".join(thscodes[:100])},
|
||||
)
|
||||
rows = data.get("item")
|
||||
return rows if isinstance(rows, list) else []
|
||||
|
||||
def price_snapshot_batch(self, thscodes: list[str]) -> list[dict]:
|
||||
"""按 thscodes 批量行情快照(最新价等), 用于估值推导的分母。
|
||||
|
||||
与全市场分页快照同一端点; thscodes 显式传入时不分页。
|
||||
返回 data.item 原始行。
|
||||
"""
|
||||
data = self._get(
|
||||
"/api/a-share/prices/snapshot",
|
||||
{"thscodes": ",".join(thscodes[:100])},
|
||||
)
|
||||
rows = data.get("item")
|
||||
return rows if isinstance(rows, list) else []
|
||||
|
||||
def trading_days(self) -> list[dict]:
|
||||
"""近一年 A 股交易日序列 (固定窗口 [今日-1年, 今日], 无入参)。
|
||||
|
||||
返回 data.item 原始行: {date_ms(上海零点), date(yyyyMMdd)}。
|
||||
供交易日探针判定「今天在列表内 ⇔ 交易日」。
|
||||
"""
|
||||
data = self._get("/api/a-share/calendar/trading-days", {})
|
||||
rows = data.get("item")
|
||||
return rows if isinstance(rows, list) else []
|
||||
|
||||
def dragon_tiger_list(self, board_type: str = "all", date: str | None = None) -> dict:
|
||||
"""龙虎榜榜单 (特色数据)。board_type: all | org | hot_money。
|
||||
|
||||
返回 data 原始容器: {trade_date, count, stock_count, stock_items[],
|
||||
hot_money_items[]}。省略 date 时服务端自动取最近已发布交易日;
|
||||
显式传非交易日返回 code=1002 (由调用方做交易日回退)。
|
||||
实测字段(2026-08): stock_items 12 个基础字段, org 榜额外带 4 个机构字段;
|
||||
文档中的 limit_reason / amount 实际不返回。
|
||||
"""
|
||||
params: dict = {"board_type": board_type}
|
||||
if date:
|
||||
params["date"] = date
|
||||
return self._get("/api/a-share/special-data/dragon-tiger-list", params)
|
||||
|
||||
def short_term_benchmark(self, date: str | None = None) -> dict:
|
||||
"""短线风向标竞价基准 (同花顺竞价筛选, 每日约 5~6 只)。
|
||||
|
||||
返回 data 原始容器: {date, date_ms, item[]}。item 行:
|
||||
{thscode, ticker, name, auction_pct, tags[]}。支持一年内历史日期;
|
||||
显式传非交易日返回 code=1002 (由调用方做交易日回退)。
|
||||
"""
|
||||
params: dict = {}
|
||||
if date:
|
||||
params["date"] = date
|
||||
return self._get("/api/a-share/auction/short-term-benchmark", params)
|
||||
|
||||
# ---- 市场 dump ----
|
||||
def dump_download_url(self, dump_kind: str) -> dict:
|
||||
"""获取 dump 预签名下载信息(约 300s 有效)。
|
||||
|
||||
dump_kind: adjustment-factors | daily-k-10d | daily-k。
|
||||
返回 {presigned_url, presigned_url_expires_at, expires_in_seconds};
|
||||
release 版本号(如 20260828)嵌在 presigned_url 的 releases/<date>/ 路径中,
|
||||
供调用方做缓存版本管理。
|
||||
"""
|
||||
return self._get(f"/api/dump/market-dumps/{dump_kind}/download-url", {})
|
||||
|
||||
def download_dump(self, dump_kind: str, dest: Path) -> Path:
|
||||
"""下载 dump 到 dest(先写 .part 临时文件, 成功后原子改名)。失败抛 FuyaoError。
|
||||
|
||||
预签名 URL 指向对象存储, 请求不得携带 X-api-key 头 → 用独立裸请求,
|
||||
不经过持有认证头的 self._http。
|
||||
"""
|
||||
url = str(self.dump_download_url(dump_kind).get("presigned_url") or "")
|
||||
if not url:
|
||||
raise FuyaoError(f"dump {dump_kind} 未返回预签名 URL")
|
||||
dest.parent.mkdir(parents=True, exist_ok=True)
|
||||
tmp = dest.with_name(dest.name + ".part")
|
||||
try:
|
||||
with httpx.stream("GET", url, timeout=120.0, follow_redirects=True) as resp:
|
||||
if resp.status_code != 200:
|
||||
raise FuyaoError(f"dump {dump_kind} 下载失败 HTTP {resp.status_code}")
|
||||
with open(tmp, "wb") as fh:
|
||||
for chunk in resp.iter_bytes(1 << 20):
|
||||
fh.write(chunk)
|
||||
tmp.replace(dest)
|
||||
except httpx.HTTPError as e:
|
||||
raise FuyaoError(f"dump {dump_kind} 下载网络失败: {e}") from e
|
||||
finally:
|
||||
tmp.unlink(missing_ok=True)
|
||||
return dest
|
||||
|
||||
@@ -7,8 +7,8 @@ display_name: "fuyao"
|
||||
runtime: none
|
||||
entry: app.plugins.fuyao.provider:FuyaoProvider
|
||||
check: app.plugins.fuyao.provider:availability
|
||||
datasets: [realtime]
|
||||
datasets: [realtime, daily, adj_factor, financial]
|
||||
api_key_env: FUYAO_API_KEY # 声明后设置页提供 Key 输入框(先探后存, secrets.json 优先)
|
||||
hidden: true # 优化完成前不在数据源页展示; 删除此行即可恢复
|
||||
description: "同花顺官方 REST 数据 API。当前提供 A 股全市场实时快照(分页拉取);日K/分钟/财务未接入,自动回退 TickFlow。"
|
||||
description: "同花顺官方 REST 数据 API。A 股实时快照、日K(原始价: 近端走 10d dump, 深窗口走 10 年全量 dump 一次下载秒级筛选, 兜底单标的接口)、除权因子(事件 dump + 本地日K dump 配价推导, 涨跌停自检, 秒级)、财务(利润表/资产负债表/现金流量表/指标, 字段映射至项目口径, bps 由估值反推; 股本无上游接口, 指标历史期建议切回 TickFlow 补齐);分钟未接入,自动回退 TickFlow。"
|
||||
install_hint: "点击卡片中的输入框配置 API Key(https://fuyao.aicubes.cn 申请),或在 .env 中配置 FUYAO_API_KEY"
|
||||
homepage: "https://fuyao.aicubes.cn"
|
||||
|
||||
@@ -3,35 +3,77 @@
|
||||
方法签名对齐 custom.GenericHTTPProvider(service 分流点按这套签名调用),
|
||||
注入 custom loader 注册表后, 各 service 无需改动即可路由到本 provider。
|
||||
|
||||
当前实现数据集: realtime (A 股全市场快照, 分页)。
|
||||
未声明 daily / minute / financial → provider_has_dataset 为 False, 自动回退 tickflow。
|
||||
实现数据集:
|
||||
- realtime A 股全市场快照 (分页)
|
||||
- daily A 股日K, 原始价; 近端窗口走 daily-k-10d 全市场 dump(1 次请求),
|
||||
深窗口走单标的 historical 接口(≤10 年/次自动分片)
|
||||
- adj_factor A 股除权因子; adjustment-factors 事件 dump + 自家原始日K前收盘,
|
||||
按交易所公式推导单事件比值, 涨跌停自检
|
||||
- financial 财务五表(股本除外): 三表多期序列 + 指标单期, 字段映射为 TickFlow
|
||||
canonical 列名, 扶摇独有字段原名透传为扩展列; bps 由估值 pb_mrq
|
||||
反推; shares 无上游接口恒空
|
||||
未声明 minute → provider_has_dataset 为 False, 自动回退 tickflow。
|
||||
|
||||
单位口径 (CONTRIBUTING §3.1, 不可凭字段名推断):
|
||||
单位与口径 (CONTRIBUTING §3.1, 不可凭字段名推断):
|
||||
- 扶摇 price_change_ratio_pct 为百分数数值 (1.74 = +1.74%), 本项目 realtime
|
||||
change_pct 契约为小数制 (0.0174 = 1.74%) → 此处显式 / 100。
|
||||
- volume 单位股、turnover 单位元, 与内部契约一致, 直接透传。
|
||||
- 扶摇 volume 单位为股, 本项目日K/实时契约均为手 → 统一 floor(股/100)。
|
||||
- turnover 单位元, 与内部一致, 直接透传。
|
||||
- 日K取数 adjust=none 锁定: 官方 forward 序列事件间有逐日漂移(2026-08 实测),
|
||||
项目内前复权一律由 indicators.pipeline 用本地因子计算。
|
||||
- ex_factor 为单事件比值(非累积), 累积链由 pipeline._apply_adj_factor 构建。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import calendar
|
||||
import contextlib
|
||||
import logging
|
||||
import math
|
||||
import re
|
||||
import time
|
||||
from collections.abc import Callable
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import UTC, date, datetime, timedelta
|
||||
from pathlib import Path
|
||||
|
||||
import polars as pl
|
||||
|
||||
from app.data_providers.normalizer import DAILY_COLS, normalize_daily
|
||||
from app.indicators.pipeline import filter_halt_days
|
||||
from app.plugins.fuyao import client as fuyao_client
|
||||
from app.plugins.fuyao.client import FuyaoClient, FuyaoError
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# 只声明真实提供的数据集; 其余数据集 provider_has_dataset 返回 False → 回退 tickflow
|
||||
_DATASETS = ("realtime",)
|
||||
_DATASETS = ("realtime", "daily", "adj_factor", "financial")
|
||||
|
||||
API_KEY_ENV = "FUYAO_API_KEY"
|
||||
SECRETS_FIELD = "fuyao_api_key" # UI 配置的 Key 存 secrets.json, 优先级高于 .env
|
||||
|
||||
# 扶摇 *ms 时间字段为北京时间零点(= UTC 前一日 16:00), +8h 后按 UTC 解析即得交易日
|
||||
_SH_MS = 28_800_000
|
||||
_HIST_MAX_SPAN_MS = 3650 * 86_400_000 # historical 单次窗口上限 10 年, 超出由本层分片
|
||||
_HIST_INTERVAL_S = 0.12 # 单标的请求节流(实测 200+ 连发未触发 4001 限频)
|
||||
_FINANCIAL_HISTORY_PERIODS = 8 # 财务首装全量历史: 最近 8 期季报(约 2 年)
|
||||
_VALUATION_BATCH = 100 # 估值/价格快照端点单次上限 100 只
|
||||
# 项目财务表名 → 扶摇报表端点名
|
||||
_STATEMENT_ENDPOINTS = {
|
||||
"income": "income-statements",
|
||||
"balance_sheet": "balance-sheets",
|
||||
"cash_flow": "cash-flow-statements",
|
||||
}
|
||||
_ADJ_DUMP_KIND = "adjustment-factors"
|
||||
_DAILY10_DUMP_KIND = "daily-k-10d"
|
||||
_DAILY_DUMP_KIND = "daily-k" # 10 年全量日K dump(约 172MB), 深窗口一次下载覆盖全市场
|
||||
_RECENT_DUMP_DAYS = 12 # 窗口跨度 ≤ 此天数时优先走 10d dump(覆盖 ≈10 个交易日)
|
||||
_PREV_CLOSE_BACKDAYS = 30 # 推导因子时向前找"除权日前收盘"的回看天数(容忍长期停牌)
|
||||
|
||||
|
||||
def get_api_key() -> str:
|
||||
from app import secrets_store
|
||||
|
||||
return secrets_store.get_env_backed_secret(SECRETS_FIELD, API_KEY_ENV)
|
||||
|
||||
|
||||
@@ -39,7 +81,8 @@ def availability() -> tuple[bool, str]:
|
||||
"""loader 启动自检: API Key 已配置(secrets.json 或 .env)才注册为可切换数据源。不抛异常。"""
|
||||
if get_api_key():
|
||||
return True, "ok"
|
||||
return False, f"未配置 {API_KEY_ENV}(可在设置页数据源卡片中直接填写)"
|
||||
# 状态行会拼在「未配置」标签之后, 文案不再重复"未配置"字样
|
||||
return False, f"缺少 API Key(可在下方输入框直接填写,或配置环境变量 {API_KEY_ENV})"
|
||||
|
||||
|
||||
def probe_api_key(api_key: str) -> tuple[bool, str]:
|
||||
@@ -85,6 +128,128 @@ def _first(row: dict, *names: str):
|
||||
return None
|
||||
|
||||
|
||||
def _date_of_ms(value) -> date | None:
|
||||
"""扶摇 *ms(上海零点) → 交易日。None/非法值返回 None, 不伪造。"""
|
||||
if value is None:
|
||||
return None
|
||||
try:
|
||||
ms = int(value)
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
return datetime.fromtimestamp((ms + _SH_MS) // 1000, tz=UTC).date()
|
||||
|
||||
|
||||
def _ms_of_date(d: date) -> int:
|
||||
"""交易日 → 扶摇 start/end 入参口径的 ms(该日上海零点的 epoch ms, 不依赖本机时区)。"""
|
||||
return (calendar.timegm(d.timetuple()) - 28_800) * 1000
|
||||
|
||||
|
||||
def _iso_of_ms(value) -> str | None:
|
||||
"""扶摇 *ms → ISO 日期字符串(项目财务表 period_end/announce_date 的存储口径)。"""
|
||||
d = _date_of_ms(value)
|
||||
return d.isoformat() if d is not None else None
|
||||
|
||||
|
||||
def _report_quarter(fiscal_period) -> int | None:
|
||||
"""fiscal_period(Q1..Q4/FY) → 指标接口 report 参数的季号 N(1..4)。"""
|
||||
if not fiscal_period:
|
||||
return None
|
||||
text = str(fiscal_period).strip().upper()
|
||||
if text == "FY":
|
||||
return 4
|
||||
try:
|
||||
return int(text.lstrip("Q"))
|
||||
except ValueError:
|
||||
return None
|
||||
|
||||
|
||||
def _ref_price(
|
||||
prev_close: float, dividend: float, bonus: float, allot: float, allot_price: float
|
||||
) -> float | None:
|
||||
"""交易所除权参考价: (P - D + AR·AP) / (1 + S + AR), 四舍五入(half-up)保留 2 位。
|
||||
|
||||
half-up 是交易所口径; 银行家舍入会让约半数事件在第 2 位小数上偏离(对拍实证)。
|
||||
"""
|
||||
denom = 1.0 + bonus + allot
|
||||
if denom <= 0:
|
||||
return None
|
||||
x = (prev_close - dividend + allot * allot_price) / denom
|
||||
return math.floor(x * 100 + 0.5) / 100
|
||||
|
||||
|
||||
def _price_limit(symbol: str) -> float:
|
||||
"""按代码前缀给涨跌停幅度(自检容差用): 创业板/科创板 20%, 北交所 30%, 主板 10%。"""
|
||||
code = symbol.split(".")[0]
|
||||
if code.startswith(("300", "301", "688", "689")):
|
||||
return 0.20
|
||||
if code.startswith(("8", "4", "92")):
|
||||
return 0.30
|
||||
return 0.10
|
||||
|
||||
|
||||
def _release_of(url: str) -> str:
|
||||
"""从预签名 URL 提取 release 版本号(releases/<date>/ 路径), 提不到返回 unknown。"""
|
||||
m = re.search(r"releases/(\d+)/", url or "")
|
||||
return m.group(1) if m else "unknown"
|
||||
|
||||
|
||||
def _cache_dir() -> Path:
|
||||
from app.config import settings
|
||||
|
||||
d = settings.data_dir / "cache" / "fuyao"
|
||||
d.mkdir(parents=True, exist_ok=True)
|
||||
return d
|
||||
|
||||
|
||||
def _kline_rows(symbol: str, bars: list[dict]) -> list[dict]:
|
||||
"""historical/dump 原始行(价格元, volume 股) → 内部行(volume 手)。"""
|
||||
out = []
|
||||
for b in bars:
|
||||
v = _to_float(b.get("volume"))
|
||||
out.append(
|
||||
{
|
||||
"symbol": symbol,
|
||||
"date": _date_of_ms(b.get("date_ms")),
|
||||
"open": _to_float(b.get("open_price")),
|
||||
"high": _to_float(b.get("high_price")),
|
||||
"low": _to_float(b.get("low_price")),
|
||||
"close": _to_float(b.get("close_price")),
|
||||
"volume": math.floor(v / 100.0) if v is not None else None,
|
||||
"amount": _to_float(b.get("turnover")),
|
||||
}
|
||||
)
|
||||
return out
|
||||
|
||||
|
||||
def _tail_ok(end_d: date, covered_max: date) -> bool:
|
||||
"""请求终点是否被覆盖到 covered_max: 周末/节假日的自然缺口(≤3 天)不算缺失。"""
|
||||
if end_d <= covered_max:
|
||||
return True
|
||||
return (end_d - covered_max).days <= 3 and end_d.weekday() >= 5
|
||||
|
||||
|
||||
def _dump_covers(dump: pl.DataFrame, start_d: date, end_d: date) -> bool:
|
||||
"""dump 日期范围是否覆盖请求窗口: 起点必须落在 dump 内; 终点允许周末自然缺口。"""
|
||||
if dump.is_empty() or "date_ms" not in dump.columns:
|
||||
return False
|
||||
dates = pl.from_epoch(dump["date_ms"].cast(pl.Int64) + _SH_MS, time_unit="ms").dt.date()
|
||||
dmin, dmax = dates.min(), dates.max()
|
||||
return start_d >= dmin and _tail_ok(end_d, dmax)
|
||||
|
||||
|
||||
def _dump_date_range(path: Path) -> tuple[date | None, date | None]:
|
||||
"""lazy 读 parquet 的 date_ms 边界(走元数据/少量行组, 不整读大文件)。"""
|
||||
row = (
|
||||
pl.scan_parquet(path)
|
||||
.select(
|
||||
pl.from_epoch(pl.col("date_ms").min() + _SH_MS, time_unit="ms").dt.date().alias("dmin"),
|
||||
pl.from_epoch(pl.col("date_ms").max() + _SH_MS, time_unit="ms").dt.date().alias("dmax"),
|
||||
)
|
||||
.collect()
|
||||
)
|
||||
return row["dmin"][0], row["dmax"][0]
|
||||
|
||||
|
||||
def _map_snapshot_row(row: dict, fetched_ms: int) -> dict | None:
|
||||
"""扶摇快照行 → 内部 realtime record。字段缺失时按依赖推导, 不伪造数据。
|
||||
|
||||
@@ -108,6 +273,8 @@ def _map_snapshot_row(row: dict, fetched_ms: int) -> dict | None:
|
||||
# 与 quote_service 的推导同口径: 小数制, 不乘 100
|
||||
change_pct = change_amount / prev
|
||||
|
||||
volume = _to_float(row.get("volume"))
|
||||
|
||||
return {
|
||||
"symbol": symbol,
|
||||
"name": row.get("name"), # 快照无名称, 由下游维表关联
|
||||
@@ -116,7 +283,7 @@ def _map_snapshot_row(row: dict, fetched_ms: int) -> dict | None:
|
||||
"open": _to_float(row.get("open_price")),
|
||||
"high": _to_float(_first(row, "high_price", "highest_price")),
|
||||
"low": _to_float(_first(row, "low_price", "lowest_price")),
|
||||
"volume": _to_float(row.get("volume")),
|
||||
"volume": math.floor(volume / 100.0) if volume is not None else None, # 股 → 手
|
||||
"amount": _to_float(row.get("turnover")),
|
||||
"change_pct": change_pct,
|
||||
"change_amount": change_amount,
|
||||
@@ -136,18 +303,76 @@ class FuyaoProvider:
|
||||
def __init__(self) -> None:
|
||||
self.config = _FuyaoConfig()
|
||||
self._client: FuyaoClient | None = None
|
||||
self._dump_memo: dict[str, pl.DataFrame] = {}
|
||||
self._dump_path_memo: dict[str, Path] = {}
|
||||
|
||||
def close(self) -> None: # loader.load_all 重建注册表时会对每个 provider 调 close
|
||||
if self._client is not None:
|
||||
with contextlib.suppress(Exception):
|
||||
self._client.close()
|
||||
self._client = None
|
||||
self._dump_memo.clear()
|
||||
self._dump_path_memo.clear()
|
||||
|
||||
def _get_client(self) -> FuyaoClient:
|
||||
if self._client is None:
|
||||
self._client = fuyao_client.FuyaoClient(api_key=get_api_key())
|
||||
return self._client
|
||||
|
||||
# ---- dump 缓存 ----
|
||||
def _ensure_dump_path(self, dump_kind: str, cache_prefix: str) -> Path:
|
||||
"""确保最新 release 的 dump 已落盘, 返回缓存路径(大文件不整读进内存)。
|
||||
|
||||
release 号取自预签名 URL 的 releases/<date>/ 路径; 新 release 落盘后清理旧版缓存。
|
||||
"""
|
||||
memo = self._dump_path_memo.get(dump_kind)
|
||||
if memo is not None and memo.exists():
|
||||
return memo
|
||||
client = self._get_client()
|
||||
info = client.dump_download_url(dump_kind)
|
||||
release = _release_of(str(info.get("presigned_url") or ""))
|
||||
dest = _cache_dir() / f"{cache_prefix}__{release}.parquet"
|
||||
if not dest.exists():
|
||||
client.download_dump(dump_kind, dest)
|
||||
for old in dest.parent.glob(f"{cache_prefix}__*.parquet"):
|
||||
if old.name != dest.name:
|
||||
old.unlink(missing_ok=True)
|
||||
logger.info("扶摇 dump %s(release %s)已下载: %s", dump_kind, release, dest.name)
|
||||
self._dump_path_memo[dump_kind] = dest
|
||||
return dest
|
||||
|
||||
def _ensure_dump(self, dump_kind: str, cache_prefix: str) -> pl.DataFrame:
|
||||
"""小体量 dump(快照 10d / 因子)整读 + 进程内 memo, 避免重复打接口/读盘。"""
|
||||
memo = self._dump_memo.get(dump_kind)
|
||||
if memo is not None:
|
||||
return memo
|
||||
df = pl.read_parquet(self._ensure_dump_path(dump_kind, cache_prefix))
|
||||
self._dump_memo[dump_kind] = df
|
||||
return df
|
||||
|
||||
def _ensure_daily_big_dump(self, start_d: date) -> Path | None:
|
||||
"""10 年全量日K dump(约 172MB)。只要求覆盖窗口起点; 末端缺口由 10d dump 补。
|
||||
|
||||
已有缓存覆盖起点就直接复用, 不追新 release(避免深窗口高频触发时日日重下
|
||||
172MB — 旧 release 的中段历史不会变, 尾部新鲜度交给 1MB 的 10d dump)。
|
||||
"""
|
||||
for f in sorted(_cache_dir().glob("daily_k__*.parquet"), reverse=True):
|
||||
try:
|
||||
dmin, _ = _dump_date_range(f)
|
||||
except Exception: # 缓存损坏不致命, 换下一个/重拉
|
||||
continue
|
||||
if dmin is not None and start_d >= dmin:
|
||||
return f
|
||||
try:
|
||||
path = self._ensure_dump_path(_DAILY_DUMP_KIND, "daily_k")
|
||||
except FuyaoError as e:
|
||||
logger.warning("扶摇 10 年 dump 不可用, 回退单标的接口: %s", e)
|
||||
return None
|
||||
dmin, _ = _dump_date_range(path)
|
||||
if dmin is None or start_d < dmin:
|
||||
return None # 窗口比 10 年更早 → 单标的兜底
|
||||
return path
|
||||
|
||||
# ---- realtime ----
|
||||
def get_realtime(self) -> list[dict]:
|
||||
"""全市场实时快照 → 内部 realtime records。失败软返回空列表(不阻断轮询)。"""
|
||||
@@ -175,11 +400,661 @@ class FuyaoProvider:
|
||||
logger.info("扶摇实时行情拉取完成: %d 条(丢弃 %d 行)", len(records), dropped)
|
||||
return records
|
||||
|
||||
# ---- daily ----
|
||||
def get_daily(
|
||||
self,
|
||||
symbols: list[str],
|
||||
start_time: datetime | None,
|
||||
end_time: datetime | None,
|
||||
asset_type: str = "stock",
|
||||
on_chunk_done: Callable[[int, int], None] | None = None,
|
||||
) -> pl.DataFrame:
|
||||
"""A 股日K → 内部契约。原始价(adjust=none 锁定)、volume 股→手(floor /100)。
|
||||
|
||||
取数三档(逐级降级, 全部失败才空手而归):
|
||||
- 近端窗口(跨度 ≤12 天): daily-k-10d dump(约 1MB), 1 次请求覆盖全部标的;
|
||||
- 深窗口: daily-k 10 年全量 dump(约 172MB, 缓存覆盖起点即复用, 不追新 release),
|
||||
末端缺口由 10d dump 补尾 — 全市场深回填从"逐票 34 分钟"降为"一次下载+秒级筛选";
|
||||
- 兜底: 单标的 historical 接口(窗口早于 dump 覆盖 / dump 不可用; 10 年自动分片,
|
||||
逐标的节流 + 进度回调)。
|
||||
"""
|
||||
if not symbols or asset_type != "stock":
|
||||
return pl.DataFrame()
|
||||
end_dt = end_time or datetime.now()
|
||||
start_dt = start_time or (end_dt - timedelta(days=365))
|
||||
start_d, end_d = start_dt.date(), end_dt.date()
|
||||
|
||||
if (end_d - start_d).days <= _RECENT_DUMP_DAYS:
|
||||
try:
|
||||
dump = self._ensure_dump(_DAILY10_DUMP_KIND, "daily_k_10d")
|
||||
if _dump_covers(dump, start_d, end_d):
|
||||
df = self._daily_from_dump(dump, set(symbols), start_d, end_d)
|
||||
if on_chunk_done:
|
||||
on_chunk_done(1, 1)
|
||||
logger.info("扶摇日K(10d dump)完成: %d 行 [%s ~ %s]", df.height, start_d, end_d)
|
||||
return df
|
||||
logger.info("扶摇 10d dump 未覆盖窗口 [%s ~ %s], 尝试 10 年 dump", start_d, end_d)
|
||||
except FuyaoError as e:
|
||||
logger.warning("扶摇日K 10d dump 不可用, 尝试 10 年 dump: %s", e)
|
||||
|
||||
df = self._daily_from_big_dump(set(symbols), start_d, end_d)
|
||||
if df is not None:
|
||||
if on_chunk_done:
|
||||
on_chunk_done(1, 1)
|
||||
logger.info("扶摇日K(10 年 dump)完成: %d 行 [%s ~ %s]", df.height, start_d, end_d)
|
||||
return df
|
||||
df = self._daily_from_api(symbols, start_d, end_d, on_chunk_done)
|
||||
logger.info("扶摇日K(单标的接口)完成: %d 行 [%s ~ %s]", df.height, start_d, end_d)
|
||||
return df
|
||||
|
||||
def _daily_from_dump(
|
||||
self, dump: pl.DataFrame, symset: set[str], start_d: date, end_d: date
|
||||
) -> pl.DataFrame:
|
||||
df = dump.with_columns(
|
||||
pl.from_epoch(pl.col("date_ms") + _SH_MS, time_unit="ms").dt.date().alias("date")
|
||||
)
|
||||
return self._map_daily_dump(df, symset, start_d, end_d)
|
||||
|
||||
def _map_daily_dump(
|
||||
self, df: pl.DataFrame, symset: set[str], start_d: date, end_d: date
|
||||
) -> pl.DataFrame:
|
||||
"""dump 原始行 → 内部日K契约(股→手、字段重命名、停牌过滤)。"""
|
||||
if "adjusted" in df.columns:
|
||||
df = df.filter(pl.col("adjusted") == "none") # 防御: dump 变为复权口径时拒绝落库
|
||||
out = (
|
||||
df.filter(
|
||||
(pl.col("date") >= start_d)
|
||||
& (pl.col("date") <= end_d)
|
||||
& pl.col("thscode").is_in(sorted(symset))
|
||||
)
|
||||
.with_columns(
|
||||
(pl.col("volume") / 100.0).floor().alias("volume"), # 股 → 手
|
||||
pl.col("thscode").alias("symbol"),
|
||||
)
|
||||
.rename(
|
||||
{
|
||||
"open_price": "open",
|
||||
"high_price": "high",
|
||||
"low_price": "low",
|
||||
"close_price": "close",
|
||||
"turnover": "amount",
|
||||
}
|
||||
)
|
||||
)
|
||||
out = filter_halt_days(out)
|
||||
cols = [c for c in DAILY_COLS if c in out.columns]
|
||||
return out.select(cols).sort(["symbol", "date"]) if not out.is_empty() else out.select(cols)
|
||||
|
||||
def _daily_from_big_dump(
|
||||
self, symset: set[str], start_d: date, end_d: date
|
||||
) -> pl.DataFrame | None:
|
||||
"""深窗口主路径: 10 年全量 dump(lazy 按需筛) + 必要时 10d dump 补尾。
|
||||
|
||||
覆盖不了(窗口早于 10 年 / dump 拉取失败)返回 None, 由调用方走单标的接口。
|
||||
"""
|
||||
path = self._ensure_daily_big_dump(start_d)
|
||||
if path is None:
|
||||
return None
|
||||
_, dmax = _dump_date_range(path)
|
||||
big_hi = min(end_d, dmax)
|
||||
# 窗口/标的过滤下推到 lazy 计划, 只物化需要的行(全量 10 年 ≈ 13.6M 行)
|
||||
window = (
|
||||
pl.scan_parquet(path)
|
||||
.with_columns(
|
||||
pl.from_epoch(pl.col("date_ms") + _SH_MS, time_unit="ms").dt.date().alias("date")
|
||||
)
|
||||
.filter(
|
||||
(pl.col("date") >= start_d)
|
||||
& (pl.col("date") <= big_hi)
|
||||
& pl.col("thscode").is_in(sorted(symset))
|
||||
)
|
||||
.collect()
|
||||
)
|
||||
parts = [self._map_daily_dump(window, symset, start_d, big_hi)]
|
||||
if not _tail_ok(end_d, dmax):
|
||||
# 末端缺口(如 10 年 dump 是旧 release, end 是最近交易日): 10d dump 补尾
|
||||
try:
|
||||
ten = self._ensure_dump(_DAILY10_DUMP_KIND, "daily_k_10d")
|
||||
ten_dates = pl.from_epoch(
|
||||
ten["date_ms"].cast(pl.Int64) + _SH_MS, time_unit="ms"
|
||||
).dt.date()
|
||||
ten_min, ten_max = ten_dates.min(), ten_dates.max()
|
||||
if (
|
||||
ten_min is not None
|
||||
and ten_min <= dmax + timedelta(days=1)
|
||||
and _tail_ok(end_d, ten_max)
|
||||
):
|
||||
tail_start = max(start_d, dmax + timedelta(days=1))
|
||||
parts.append(self._daily_from_dump(ten, symset, tail_start, end_d))
|
||||
else:
|
||||
return None # 中段或尾部仍有缺口 → 单标的兜底, 不交缺口数据
|
||||
except FuyaoError as e:
|
||||
logger.warning("扶摇 10d dump 补尾失败: %s", e)
|
||||
return None
|
||||
non_empty = [p for p in parts if not p.is_empty()]
|
||||
if not non_empty:
|
||||
return pl.DataFrame()
|
||||
out = pl.concat(non_empty, how="vertical_relaxed")
|
||||
return out.unique(subset=["symbol", "date"], keep="last").sort(["symbol", "date"])
|
||||
|
||||
def _daily_from_api(
|
||||
self,
|
||||
symbols: list[str],
|
||||
start_d: date,
|
||||
end_d: date,
|
||||
on_chunk_done: Callable[[int, int], None] | None,
|
||||
) -> pl.DataFrame:
|
||||
frames: list[pl.DataFrame] = []
|
||||
for i, sym in enumerate(symbols):
|
||||
rows = self._historical_bars(sym, start_d, end_d)
|
||||
time.sleep(_HIST_INTERVAL_S)
|
||||
if rows:
|
||||
df = normalize_daily(_kline_rows(sym, rows), default_symbol=sym, source=self.name)
|
||||
if not df.is_empty():
|
||||
frames.append(df)
|
||||
if on_chunk_done:
|
||||
on_chunk_done(i + 1, len(symbols))
|
||||
return pl.concat(frames, how="diagonal_relaxed") if frames else pl.DataFrame()
|
||||
|
||||
def _historical_bars(self, symbol: str, start_d: date, end_d: date) -> list[dict]:
|
||||
"""按 ≤10 年窗口分片拉取单标的原始日K。中途失败软返回已得行, 不抛出。"""
|
||||
out: list[dict] = []
|
||||
s = _ms_of_date(start_d)
|
||||
e = _ms_of_date(end_d)
|
||||
while s <= e:
|
||||
chunk_end = min(e, s + _HIST_MAX_SPAN_MS - 1)
|
||||
try:
|
||||
out.extend(self._get_client().historical_kline(symbol, s, chunk_end, adjust="none"))
|
||||
except FuyaoError as err:
|
||||
logger.warning("扶摇日K拉取失败 %s [%s ~ %s]: %s", symbol, start_d, end_d, err)
|
||||
break
|
||||
if s + _HIST_MAX_SPAN_MS <= e:
|
||||
time.sleep(_HIST_INTERVAL_S)
|
||||
s = chunk_end + 1
|
||||
return out
|
||||
|
||||
# ---- adj_factor ----
|
||||
def get_adj_factors(
|
||||
self,
|
||||
symbols: list[str],
|
||||
start_time: datetime | None,
|
||||
end_time: datetime | None,
|
||||
asset_type: str = "stock",
|
||||
on_chunk_done: Callable[[int, int], None] | None = None,
|
||||
) -> pl.DataFrame:
|
||||
"""A 股除权因子 → 内部契约(symbol/trade_date/ex_factor, 单事件比值非累积)。
|
||||
|
||||
数据链: adjustment-factors 全量事件 dump → 清洗(时区 +8h / 滤全零 / 同日成分合并)
|
||||
→ 前收盘价从本地日K dump 一次取齐(缺价标的回退单标的接口) → 交易所公式推导
|
||||
→ 涨跌停自检剔除异常。
|
||||
未来已公告事件无前收盘, 留给滚动增量窗口(15 天)自然补上。
|
||||
"""
|
||||
schema = {"symbol": pl.String, "trade_date": pl.Date, "ex_factor": pl.Float64}
|
||||
if not symbols or asset_type != "stock":
|
||||
return pl.DataFrame(schema=schema)
|
||||
try:
|
||||
events = self._load_adj_events(set(symbols), start_time, end_time)
|
||||
except FuyaoError as e:
|
||||
logger.warning("扶摇除权因子 dump 加载失败: %s", e)
|
||||
return pl.DataFrame(schema=schema)
|
||||
if events.is_empty():
|
||||
return pl.DataFrame(schema=schema)
|
||||
|
||||
out_rows: list[dict] = []
|
||||
syms = sorted(events["symbol"].unique().to_list())
|
||||
# 前收盘价优先从本地日K dump 一次取齐(全市场配价从逐标的 ~13 分钟降为秒级),
|
||||
# 大 dump 不可用或个别标的缺价时回退单标的接口(节流保留)。
|
||||
bounds = events.group_by("symbol").agg(
|
||||
pl.col("ex_date").min().alias("first_ex"),
|
||||
pl.col("ex_date").max().alias("last_ex"),
|
||||
)
|
||||
closes_by_sym = self._closes_from_dumps(bounds)
|
||||
fallbacks = 0
|
||||
for i, sym in enumerate(syms):
|
||||
evs = events.filter(pl.col("symbol") == sym).sort("ex_date")
|
||||
first_ex: date = evs["ex_date"][0]
|
||||
last_ex: date = evs["ex_date"][-1]
|
||||
closes = (closes_by_sym or {}).get(sym)
|
||||
if not closes or min(closes) > first_ex:
|
||||
# dump 无该标的 / 覆盖不到首个事件前 → 单标的接口兜底
|
||||
closes = self._fetch_closes(
|
||||
sym, first_ex - timedelta(days=_PREV_CLOSE_BACKDAYS), last_ex
|
||||
)
|
||||
time.sleep(_HIST_INTERVAL_S)
|
||||
fallbacks += 1
|
||||
if not closes:
|
||||
logger.warning("扶摇除权因子: %s 原始日K为空, 跳过其 %d 个事件", sym, evs.height)
|
||||
continue
|
||||
days = sorted(closes)
|
||||
for ev in evs.iter_rows(named=True):
|
||||
exd: date = ev["ex_date"]
|
||||
prev_days = [d for d in days if d < exd]
|
||||
if not prev_days:
|
||||
continue # 日K窗口未覆盖的远古事件(如 10 年分片边界之外)
|
||||
p = closes[prev_days[-1]]
|
||||
ref = _ref_price(p, ev["dividend"], ev["bonus"], ev["allot"], ev["allot_price"])
|
||||
if ref is None or ref <= 0:
|
||||
logger.warning(
|
||||
"扶摇除权因子: %s %s 参考价无法计算(P=%s D=%s S=%s AR=%s AP=%s), 跳过",
|
||||
sym,
|
||||
exd,
|
||||
p,
|
||||
ev["dividend"],
|
||||
ev["bonus"],
|
||||
ev["allot"],
|
||||
ev["allot_price"],
|
||||
)
|
||||
continue
|
||||
factor = p / ref
|
||||
ex_days = [d for d in days if d >= exd]
|
||||
if ex_days: # 涨跌停自检: 错误因子会令复权后除权日涨跌幅超出板限
|
||||
ret = closes[ex_days[0]] / (p / factor) - 1.0
|
||||
if abs(ret) > _price_limit(sym) + 0.02:
|
||||
logger.warning(
|
||||
"扶摇除权因子自检剔除 %s %s: 复权后除权日涨跌幅 %.1f%% 超出涨跌停",
|
||||
sym,
|
||||
exd,
|
||||
ret * 100,
|
||||
)
|
||||
continue
|
||||
out_rows.append({"symbol": sym, "trade_date": exd, "ex_factor": factor})
|
||||
if on_chunk_done:
|
||||
on_chunk_done(i + 1, len(syms))
|
||||
if closes_by_sym is not None:
|
||||
logger.info(
|
||||
"扶摇除权因子: 本地 dump 配价 %d/%d 标的, 回退接口 %d 标的",
|
||||
len(closes_by_sym),
|
||||
len(syms),
|
||||
fallbacks,
|
||||
)
|
||||
if not out_rows:
|
||||
return pl.DataFrame(schema=schema)
|
||||
return (
|
||||
pl.DataFrame(out_rows, schema=schema)
|
||||
.unique(subset=["symbol", "trade_date"], keep="last")
|
||||
.sort(["symbol", "trade_date"])
|
||||
)
|
||||
|
||||
def _load_adj_events(
|
||||
self,
|
||||
symset: set[str],
|
||||
start_time: datetime | None,
|
||||
end_time: datetime | None,
|
||||
) -> pl.DataFrame:
|
||||
"""事件 dump → 清洗 → 窗口过滤。返回 symbol/ex_date/dividend/bonus/allot/allot_price。
|
||||
|
||||
清洗规则(对拍实证, 见 2026-08 验证记录):
|
||||
- ex_date_ms 为上海零点戳, +8h 转日期;
|
||||
- 全零事件行(疑似特殊事件, dump 未给成分)过滤;
|
||||
- 同日拆行(如分红/送转各一行)按成分合并后推导, 顺序不可反;
|
||||
- 配股但配股价缺失 → 无法推导, 过滤。
|
||||
"""
|
||||
today = datetime.now().date()
|
||||
df = self._ensure_dump(_ADJ_DUMP_KIND, "adj_factors").rename({"thscode": "symbol"})
|
||||
df = (
|
||||
df.with_columns(
|
||||
pl.from_epoch(pl.col("ex_date_ms") + _SH_MS, time_unit="ms")
|
||||
.dt.date()
|
||||
.alias("ex_date"),
|
||||
pl.col("dividend_per_share").fill_null(0.0),
|
||||
pl.col("per_share_bonus").fill_null(0.0),
|
||||
pl.col("allotment_ratio").fill_null(0.0),
|
||||
pl.col("allotment_price").fill_null(0.0),
|
||||
)
|
||||
.filter(
|
||||
(pl.col("ex_date") <= today) # 未来已公告事件无前收盘, 留给滚动增量
|
||||
& (
|
||||
(pl.col("dividend_per_share") != 0)
|
||||
| (pl.col("per_share_bonus") != 0)
|
||||
| (pl.col("allotment_ratio") != 0)
|
||||
)
|
||||
)
|
||||
.group_by("symbol", "ex_date")
|
||||
.agg(
|
||||
pl.col("dividend_per_share").sum().alias("dividend"),
|
||||
pl.col("per_share_bonus").sum().alias("bonus"),
|
||||
pl.col("allotment_ratio").sum().alias("allot"),
|
||||
pl.col("allotment_price").max().alias("allot_price"), # 同日拆行共享配股价
|
||||
)
|
||||
.filter(~((pl.col("allot") > 0) & (pl.col("allot_price") <= 0)))
|
||||
.filter(pl.col("symbol").is_in(sorted(symset)))
|
||||
)
|
||||
if start_time is not None:
|
||||
df = df.filter(pl.col("ex_date") >= start_time.date())
|
||||
if end_time is not None:
|
||||
df = df.filter(pl.col("ex_date") <= end_time.date())
|
||||
return df.select("symbol", "ex_date", "dividend", "bonus", "allot", "allot_price")
|
||||
|
||||
# ---- financial ----
|
||||
# 字段映射: 扶摇原始字段 → 项目 canonical 列名(TickFlow 口径, 前端财务页与回测
|
||||
# FUNDAMENTAL_FACTORS 按此消费)。映射表之外的扶摇独有字段以原名透传为扩展列。
|
||||
_INCOME_FIELD_MAP = {
|
||||
"operating_income": "revenue",
|
||||
"operating_costs": "operating_cost",
|
||||
"sales_fee": "selling_expense",
|
||||
"manage_fee": "admin_expense",
|
||||
"research_and_development_expenses": "rd_expense",
|
||||
"operating_profit": "operating_profit",
|
||||
"interest_expenses": "financial_expense", # 近似口径: 扶摇只给利息费用
|
||||
"profit_total": "total_profit",
|
||||
"income_tax_expense": "income_tax",
|
||||
"net_profit": "net_income",
|
||||
"parent_holder_net_profit": "net_income_attributable",
|
||||
"basic_eps": "basic_eps",
|
||||
}
|
||||
_BALANCE_FIELD_MAP = {
|
||||
"assets_total": "total_assets",
|
||||
"total_current_assets": "total_current_assets",
|
||||
"non_current_nets_total": "total_non_current_assets",
|
||||
"cash": "cash_and_equivalents",
|
||||
"accounts_receivable": "accounts_receivable",
|
||||
"total_debt": "total_liabilities",
|
||||
"holder_equity_total": "total_equity",
|
||||
}
|
||||
_CASHFLOW_FIELD_MAP = {
|
||||
"act_cash_flow_net": "net_operating_cash_flow",
|
||||
"invest_cash_flow_net": "net_investing_cash_flow",
|
||||
"financing_cash_flow_net": "net_financing_cash_flow",
|
||||
"pay_fixed_assets_etc_cash": "capex",
|
||||
"cash_equivalents_net_addition": "net_cash_change",
|
||||
}
|
||||
# 官方指标 index_id → canonical。归母净利同比近似 tickflow net_income_yoy;
|
||||
# 实测 index_id 与文档有出入(calculate_ 前缀等), 以实测为准。
|
||||
_METRICS_FIELD_MAP = {
|
||||
"index_weighted_avg_roe": "roe",
|
||||
"total_assets_net_ratio": "roa",
|
||||
"sale_gross_margin": "gross_margin",
|
||||
"sale_net_interest_ratio": "net_margin",
|
||||
"assets_debt_ratio": "debt_to_asset_ratio",
|
||||
"calculate_operating_income_yoy_growth_ratio": "revenue_yoy",
|
||||
"calculate_parent_holder_net_profit_yoy_growth_ratio": "net_income_yoy",
|
||||
"operating_cash_flow_net_divide_income": "operating_cash_to_revenue",
|
||||
"inventory_turnover_ratio": "inventory_turnover",
|
||||
}
|
||||
|
||||
def get_financials(
|
||||
self,
|
||||
table: str,
|
||||
symbols: list[str],
|
||||
latest_only: bool = True,
|
||||
) -> pl.DataFrame:
|
||||
"""拉取财务数据, 映射为 canonical 列(symbol/period_end/announce_date/指标)。
|
||||
|
||||
- 三大报表: 单股单请求, latest_only 决定最近 1 期还是 8 期季报;
|
||||
- metrics: 指标接口为单股单期, 恒只拉最新一期(bps 由估值快照 pb_mrq 反推,
|
||||
eps_basic 顺带取自利润表); 历史各期建议切回 TickFlow 同步补齐 —
|
||||
报告期合并写入会让两源数据共存, 互不覆盖;
|
||||
- shares: 扶摇无股本接口, 恒返回空(已有存量靠合并写入保留)。
|
||||
"""
|
||||
if table == "shares":
|
||||
logger.info("扶摇无股本接口, shares 表跳过 (已有数据保留)")
|
||||
return pl.DataFrame()
|
||||
if table in _STATEMENT_ENDPOINTS:
|
||||
field_map = {
|
||||
"income": self._INCOME_FIELD_MAP,
|
||||
"balance_sheet": self._BALANCE_FIELD_MAP,
|
||||
"cash_flow": self._CASHFLOW_FIELD_MAP,
|
||||
}[table]
|
||||
return self._financial_statements(table, field_map, symbols, latest_only)
|
||||
if table == "metrics":
|
||||
return self._financial_metrics(symbols)
|
||||
return pl.DataFrame()
|
||||
|
||||
def _financial_statements(
|
||||
self,
|
||||
stmt: str,
|
||||
field_map: dict[str, str],
|
||||
symbols: list[str],
|
||||
latest_only: bool,
|
||||
) -> pl.DataFrame:
|
||||
client = self._get_client()
|
||||
limit = 1 if latest_only else _FINANCIAL_HISTORY_PERIODS
|
||||
rows_out: list[dict] = []
|
||||
for i, sym in enumerate(symbols):
|
||||
if i:
|
||||
time.sleep(_HIST_INTERVAL_S)
|
||||
try:
|
||||
rows = client.financial_statements(stmt, sym, limit=limit)
|
||||
except FuyaoError as e:
|
||||
logger.warning("扶摇财务 %s %s 失败: %s", stmt, sym, e)
|
||||
continue
|
||||
for r in rows:
|
||||
row: dict = {
|
||||
"symbol": sym,
|
||||
"period_end": _iso_of_ms(r.get("period_end_ms")),
|
||||
"announce_date": _iso_of_ms(r.get("report_date_ms")),
|
||||
}
|
||||
for src, dst in field_map.items():
|
||||
row[dst] = _to_float(r.get(src))
|
||||
# 扶摇独有字段以原名透传为扩展列 (canonical 之外的增量信息)
|
||||
for src, value in r.items():
|
||||
if src not in field_map and src not in row and isinstance(value, (int, float)):
|
||||
row[src] = value
|
||||
rows_out.append(row)
|
||||
return pl.DataFrame(rows_out) if rows_out else pl.DataFrame()
|
||||
|
||||
def _financial_metrics(self, symbols: list[str]) -> pl.DataFrame:
|
||||
client = self._get_client()
|
||||
# 指标接口按 report(yyyy-N) 单期查询 → 先用利润表 limit=1 反查每股最新披露期
|
||||
latest: dict[str, dict] = {}
|
||||
for i, sym in enumerate(symbols):
|
||||
if i:
|
||||
time.sleep(_HIST_INTERVAL_S)
|
||||
try:
|
||||
rows = client.financial_statements("income", sym, limit=1)
|
||||
except FuyaoError as e:
|
||||
logger.warning("扶摇财务 income %s 失败: %s", sym, e)
|
||||
continue
|
||||
if rows:
|
||||
latest[sym] = rows[0]
|
||||
if not latest:
|
||||
return pl.DataFrame()
|
||||
bps_by_sym = self._derive_bps(sorted(latest))
|
||||
rows_out: list[dict] = []
|
||||
for sym, r in latest.items():
|
||||
quarter = _report_quarter(r.get("fiscal_period"))
|
||||
report = f"{r.get('fiscal_year')}-{quarter}" if quarter else None
|
||||
row: dict = {
|
||||
"symbol": sym,
|
||||
"period_end": _iso_of_ms(r.get("period_end_ms")),
|
||||
"announce_date": _iso_of_ms(r.get("report_date_ms")),
|
||||
"eps_basic": _to_float(r.get("basic_eps")),
|
||||
"bps": bps_by_sym.get(sym),
|
||||
}
|
||||
if report:
|
||||
try:
|
||||
abilities = client.financial_indicators(sym, report)
|
||||
except FuyaoError as e:
|
||||
logger.warning("扶摇指标 %s %s 失败: %s", sym, report, e)
|
||||
abilities = []
|
||||
for ability in abilities:
|
||||
for ind in ability.get("indicators") or []:
|
||||
index_id = ind.get("index_id")
|
||||
if not index_id:
|
||||
continue
|
||||
value = _to_float(ind.get("value"))
|
||||
if value is not None:
|
||||
row[self._METRICS_FIELD_MAP.get(index_id, index_id)] = value
|
||||
rows_out.append(row)
|
||||
return pl.DataFrame(rows_out) if rows_out else pl.DataFrame()
|
||||
|
||||
def trading_days(self) -> set:
|
||||
"""近一年交易日集合 (供交易日探针)。失败抛 FuyaoError, 由探针兜为未知。"""
|
||||
rows = self._get_client().trading_days()
|
||||
return {
|
||||
d
|
||||
for d in (_date_of_ms(r.get("date_ms")) for r in rows)
|
||||
if d is not None
|
||||
}
|
||||
|
||||
def dragon_tiger(self, board_type: str = "all", date: str | None = None) -> dict:
|
||||
"""龙虎榜单榜 (复盘页卡片 + AI 复盘上下文)。返回原始 data 容器。
|
||||
|
||||
非路由数据集 (tickflow 无对应能力), 不进 plugin.yaml datasets,
|
||||
由 services.dragon_tiger 统一做三榜聚合/缓存/交易日回退。
|
||||
"""
|
||||
return self._get_client().dragon_tiger_list(board_type, date)
|
||||
|
||||
def short_term_benchmark(self, date: str | None = None) -> dict:
|
||||
"""短线风向标竞价基准 (复盘页卡片 + AI 复盘上下文)。返回原始 data 容器。
|
||||
|
||||
非路由数据集 (tickflow 无对应能力), 不进 plugin.yaml datasets,
|
||||
由 services.auction_benchmark 统一做按日缓存/收益enrich/交易日回退。
|
||||
"""
|
||||
return self._get_client().short_term_benchmark(date)
|
||||
|
||||
def _derive_bps(self, symbols: list[str]) -> dict[str, float]:
|
||||
"""估值快照 pb_mrq 与行情快照最新价同源同刻 → bps = price / pb_mrq。
|
||||
|
||||
与财报口径 bps 可能差几个百分点(上游权益基准不完全透明), 用于补齐
|
||||
metrics.bps 使回测 pb_latest 因子可用。接口失败只影响 bps 列, 不致命。
|
||||
"""
|
||||
client = self._get_client()
|
||||
pb: dict[str, float] = {}
|
||||
price: dict[str, float] = {}
|
||||
for i in range(0, len(symbols), _VALUATION_BATCH):
|
||||
if i:
|
||||
time.sleep(_HIST_INTERVAL_S)
|
||||
batch = symbols[i : i + _VALUATION_BATCH]
|
||||
for fetch, store in (
|
||||
(client.valuations_snapshot, pb),
|
||||
(client.price_snapshot_batch, price),
|
||||
):
|
||||
time.sleep(_HIST_INTERVAL_S)
|
||||
try:
|
||||
for r in fetch(batch):
|
||||
value = _to_float(r.get("pb_mrq" if store is pb else "last_price"))
|
||||
code = r.get("thscode")
|
||||
if code and value is not None and value != 0:
|
||||
store[code] = value
|
||||
except FuyaoError as e:
|
||||
logger.warning("扶摇 bps 推导快照失败(%d 只): %s", len(batch), e)
|
||||
return {
|
||||
sym: price[sym] / pb[sym]
|
||||
for sym in symbols
|
||||
if sym in pb and pb[sym] and sym in price
|
||||
}
|
||||
|
||||
def _fetch_closes(self, symbol: str, start_d: date, end_d: date) -> dict[date, float]:
|
||||
rows = self._historical_bars(symbol, start_d, end_d)
|
||||
out: dict[date, float] = {}
|
||||
for r in rows:
|
||||
d = _date_of_ms(r.get("date_ms"))
|
||||
c = _to_float(r.get("close_price"))
|
||||
if d is not None and c is not None:
|
||||
out[d] = c
|
||||
return out
|
||||
|
||||
def _closes_from_dumps(
|
||||
self, bounds: pl.DataFrame
|
||||
) -> dict[str, dict[date, float]] | None:
|
||||
"""从本地日K dump 一次取齐全部事件标的的收盘价。
|
||||
|
||||
每标的开窗 [first_ex-30d, last_ex](与单标的接口同窗): 10 年大 dump 为
|
||||
主体, 10d dump 叠加补末端新鲜度(除权日当天的自检需要 ex 日收盘)。
|
||||
返回 symbol → {date: close}; 大 dump 不可用/窗口早于其覆盖/读盘失败
|
||||
→ None, 由调用方整轮回退单标的接口。
|
||||
"""
|
||||
try:
|
||||
lo = bounds["first_ex"].min() - timedelta(days=_PREV_CLOSE_BACKDAYS)
|
||||
path = self._ensure_daily_big_dump(lo)
|
||||
if path is None:
|
||||
return None
|
||||
sources = [self._closes_scan(pl.scan_parquet(path))]
|
||||
ten = self._dump_memo.get(_DAILY10_DUMP_KIND)
|
||||
if ten is not None:
|
||||
sources.append(self._closes_scan(ten.lazy()))
|
||||
else:
|
||||
try:
|
||||
p10 = self._ensure_dump_path(_DAILY10_DUMP_KIND, "daily_k_10d")
|
||||
except FuyaoError as e:
|
||||
logger.info("扶摇 10d dump 不可用, 配价仅用 10 年 dump: %s", e)
|
||||
else:
|
||||
sources.append(self._closes_scan(pl.scan_parquet(p10)))
|
||||
win = bounds.select(
|
||||
"symbol",
|
||||
(pl.col("first_ex") - pl.duration(days=_PREV_CLOSE_BACKDAYS)).alias("lo"),
|
||||
"last_ex",
|
||||
)
|
||||
# concat 顺序 = 叠加优先级: 同 (symbol, date) 时 10d dump(更新鲜)覆盖大 dump
|
||||
df = (
|
||||
pl.concat(sources, how="vertical_relaxed")
|
||||
.join(win.lazy(), on="symbol", how="inner")
|
||||
.filter(
|
||||
(pl.col("date") >= pl.col("lo")) & (pl.col("date") <= pl.col("last_ex"))
|
||||
)
|
||||
.unique(subset=["symbol", "date"], keep="last", maintain_order=True)
|
||||
.collect()
|
||||
)
|
||||
return {
|
||||
f["symbol"][0]: dict(
|
||||
zip(f["date"].to_list(), f["close"].to_list(), strict=True)
|
||||
)
|
||||
for f in df.partition_by("symbol")
|
||||
}
|
||||
except Exception as e: # 缓存损坏等不致命: 回退逐标的接口
|
||||
logger.warning("扶摇除权因子本地配价失败, 回退单标的接口: %s", e)
|
||||
return None
|
||||
|
||||
@staticmethod
|
||||
def _closes_scan(lf: pl.LazyFrame) -> pl.LazyFrame:
|
||||
"""dump 行 → (symbol, date, close) lazy 投影; adjusted 列存在时锁 none。"""
|
||||
if "adjusted" in lf.collect_schema().names():
|
||||
lf = lf.filter(pl.col("adjusted") == "none")
|
||||
return lf.select(
|
||||
pl.col("thscode").alias("symbol"),
|
||||
pl.from_epoch(pl.col("date_ms").cast(pl.Int64) + _SH_MS, time_unit="ms")
|
||||
.dt.date()
|
||||
.alias("date"),
|
||||
pl.col("close_price").cast(pl.Float64).alias("close"),
|
||||
).filter(pl.col("close").is_not_null())
|
||||
|
||||
# ---- 测试(设置页试拉) ----
|
||||
def test_dataset(self, dataset: str, symbols: list[str] | None = None) -> dict:
|
||||
if dataset in ("daily", "adj_factor"):
|
||||
syms = [s for s in (symbols or [])][:3] or ["000001.SZ"]
|
||||
try:
|
||||
if dataset == "daily":
|
||||
df = self.get_daily(syms, datetime.now() - timedelta(days=30), datetime.now())
|
||||
else:
|
||||
df = self.get_adj_factors(
|
||||
syms, datetime.now() - timedelta(days=365), datetime.now()
|
||||
)
|
||||
except FuyaoError as e:
|
||||
return {"provider": self.name, "dataset": dataset, "rows": 0, "error": str(e)}
|
||||
head = df.head(5).to_dicts()
|
||||
for row in head: # date/datetime → ISO 字符串, 保证 JSON 可序列化
|
||||
for k, v in list(row.items()):
|
||||
if isinstance(v, (date, datetime)):
|
||||
row[k] = v.isoformat()
|
||||
return {
|
||||
"provider": self.name,
|
||||
"dataset": dataset,
|
||||
"rows": df.height,
|
||||
"columns": df.columns,
|
||||
"preview": head,
|
||||
}
|
||||
if dataset == "financial":
|
||||
syms = [s for s in (symbols or [])][:1] or ["600519.SH"]
|
||||
try:
|
||||
df = self.get_financials("metrics", syms, latest_only=True)
|
||||
except FuyaoError as e:
|
||||
return {"provider": self.name, "dataset": dataset, "rows": 0, "error": str(e)}
|
||||
head = df.head(5).to_dicts()
|
||||
return {
|
||||
"provider": self.name,
|
||||
"dataset": dataset,
|
||||
"rows": df.height,
|
||||
"columns": df.columns,
|
||||
"preview": head,
|
||||
}
|
||||
if dataset != "realtime":
|
||||
return {"provider": self.name, "dataset": dataset, "rows": 0,
|
||||
"error": f"扶摇插件未接入 {dataset} 数据集(自动回退 TickFlow)"}
|
||||
return {
|
||||
"provider": self.name,
|
||||
"dataset": dataset,
|
||||
"rows": 0,
|
||||
"error": f"扶摇插件未接入 {dataset} 数据集(自动回退 TickFlow)",
|
||||
}
|
||||
try:
|
||||
rows, count = self._get_client().snapshot_page(limit=5)
|
||||
except FuyaoError as e:
|
||||
|
||||
@@ -12,7 +12,8 @@ from __future__ import annotations
|
||||
import logging
|
||||
from collections.abc import Callable
|
||||
from dataclasses import dataclass, field
|
||||
from datetime import datetime
|
||||
from datetime import datetime, timedelta
|
||||
from datetime import time as dtime
|
||||
|
||||
import polars as pl
|
||||
|
||||
@@ -51,6 +52,9 @@ class StockSDKProvider:
|
||||
|
||||
name = "stocksdk"
|
||||
builtin = True
|
||||
# 分钟历史深度能力(可选声明, 未声明视为深历史): stock-sdk 免费分时接口
|
||||
# 只保留最近 5 个交易日的 1 分钟数据, 分时档位/默认值据此收窄。
|
||||
minute_history_days = 5
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.config = _StockSDKConfig()
|
||||
@@ -145,15 +149,47 @@ class StockSDKProvider:
|
||||
return pl.DataFrame()
|
||||
period = "".join(ch for ch in str(freq) if ch.isdigit()) or "1"
|
||||
logger.info("stock-sdk minute 拉取开始(%d symbols, period=%s)", len(symbols), period)
|
||||
|
||||
# 上游区间查询的分钟 open 为日级常量(伪值), 单日查询才给最新交易日真实
|
||||
# 分钟 open → 末尾 3 个自然日逐日单拉(跳过周末, 覆盖周五收盘后场景),
|
||||
# 其余历史段仍走单个区间任务控制桥接成本。伪 open 由 _null_degenerate_opens
|
||||
# 在 _minute_df 内置 null。
|
||||
windows: list[tuple[datetime | None, datetime | None]] = []
|
||||
if (
|
||||
start_time is not None
|
||||
and end_time is not None
|
||||
and end_time.date() > start_time.date()
|
||||
):
|
||||
tail_start = end_time.date() - timedelta(days=3)
|
||||
if tail_start > start_time.date():
|
||||
head_end = datetime.combine(tail_start - timedelta(days=1), dtime.max)
|
||||
windows.append((start_time, head_end))
|
||||
else:
|
||||
tail_start = start_time.date()
|
||||
day = tail_start
|
||||
while day <= end_time.date():
|
||||
if day.weekday() < 5:
|
||||
windows.append((
|
||||
datetime.combine(day, dtime.min),
|
||||
datetime.combine(day, dtime.max),
|
||||
))
|
||||
day += timedelta(days=1)
|
||||
else:
|
||||
windows.append((start_time, end_time))
|
||||
|
||||
frames: list[pl.DataFrame] = []
|
||||
chunks = chunked(symbols, _BATCH)
|
||||
for i, chunk in enumerate(chunks):
|
||||
total = len(chunks) * len(windows)
|
||||
step = 0
|
||||
for win_start, win_end in windows:
|
||||
for chunk in chunks:
|
||||
step += 1
|
||||
job = {
|
||||
"op": "minute",
|
||||
"symbols": chunk,
|
||||
"period": period,
|
||||
"start": _yyyymmdd(start_time),
|
||||
"end": _yyyymmdd(end_time),
|
||||
"start": _yyyymmdd(win_start),
|
||||
"end": _yyyymmdd(win_end),
|
||||
}
|
||||
try:
|
||||
result = bridge.run_job(job, timeout=180)
|
||||
@@ -165,9 +201,44 @@ class StockSDKProvider:
|
||||
if not df.is_empty():
|
||||
frames.append(df)
|
||||
if on_chunk_done:
|
||||
on_chunk_done(i + 1, len(chunks))
|
||||
on_chunk_done(step, total)
|
||||
# 末窗口(最新一日)的真实 open 与首窗口可能重叠同日(时区/边界), keep="last"
|
||||
# 由上层 _write_minute_partition 的 unique 处理; 这里仅拼接。
|
||||
return pl.concat(frames, how="diagonal_relaxed") if frames else pl.DataFrame()
|
||||
|
||||
@staticmethod
|
||||
def _null_degenerate_opens(df: pl.DataFrame) -> pl.DataFrame:
|
||||
"""把"日级常量"的伪分钟 open 置 null。
|
||||
|
||||
stock-sdk 上游对历史日的分钟 open 只给全天常量(如涨跌停价/日开),
|
||||
并非真实分钟开盘价; 只有最新交易日在单日查询下给真实值。伪 open 入库
|
||||
会让分钟K的 close-vs-open 口径全偏(如分时量恒红), 故按日检测:
|
||||
同日 rows>10 且 open 唯一值<=3 而 close 唯一值>10 → open 判定非分钟级,
|
||||
置 null(fail-closed, 不伪造 prev_close 替代)。
|
||||
"""
|
||||
if df.is_empty() or "open" not in df.columns or "datetime" not in df.columns:
|
||||
return df
|
||||
stats = df.group_by(pl.col("datetime").dt.date()).agg(
|
||||
pl.len().alias("n"),
|
||||
pl.col("open").n_unique().alias("uo"),
|
||||
pl.col("close").n_unique().alias("uc"),
|
||||
)
|
||||
fake_dates = stats.filter(
|
||||
(pl.col("n") > 10) & (pl.col("uo") <= 3) & (pl.col("uc") > 10)
|
||||
)["datetime"]
|
||||
if fake_dates.is_empty():
|
||||
return df
|
||||
logger.warning(
|
||||
"stock-sdk minute open 为日级常量, 置 null: %s %s",
|
||||
df["symbol"][0] if "symbol" in df.columns else "?", fake_dates.to_list(),
|
||||
)
|
||||
return df.with_columns(
|
||||
pl.when(pl.col("datetime").dt.date().is_in(fake_dates))
|
||||
.then(None)
|
||||
.otherwise(pl.col("open"))
|
||||
.alias("open")
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _minute_df(rows: list[dict], symbol: str) -> pl.DataFrame:
|
||||
if not rows:
|
||||
@@ -192,6 +263,7 @@ class StockSDKProvider:
|
||||
for col in ("open", "high", "low", "close", "volume", "amount"):
|
||||
if col in df.columns:
|
||||
df = df.with_columns(pl.col(col).cast(pl.Float64, strict=False))
|
||||
df = StockSDKProvider._null_degenerate_opens(df)
|
||||
keep = [c for c in _MINUTE_CANONICAL if c in df.columns]
|
||||
return df.select(keep) if "datetime" in keep else pl.DataFrame()
|
||||
|
||||
|
||||
@@ -223,3 +223,60 @@ def build_overview(
|
||||
"counts": counts,
|
||||
"rows": out_rows[:limit],
|
||||
}
|
||||
|
||||
|
||||
# ================================================================
|
||||
# 盘中异动 (量价信号聚合, 异动监控「盘中」tab)
|
||||
#
|
||||
# 数据源: enriched 最新快照的当日消息号列 (零新增采集):
|
||||
# 涨停/跌停/跌停翘板/炸板/放量(量比≥2)/创60日新高/新低。
|
||||
# 行序 = 信号优先级 (涨停 > 炸板 > 翘板 > 跌停 > 新高 > 新低 > 放量),
|
||||
# 同级按 |今日涨跌| 降序; counts 供前端筛选 chips 展示各类型数量。
|
||||
# ================================================================
|
||||
|
||||
_INTRADAY_SIGNALS: tuple[tuple[str, str], ...] = (
|
||||
("signal_limit_up", "limit_up"),
|
||||
("signal_broken_limit_up", "broken"),
|
||||
("signal_limit_down_recovery", "recovery"),
|
||||
("signal_limit_down", "limit_down"),
|
||||
("signal_n_day_high", "new_high"),
|
||||
("signal_n_day_low", "new_low"),
|
||||
("signal_volume_surge", "volume_surge"),
|
||||
)
|
||||
_INTRADAY_PRIORITY = {key: i for i, (_, key) in enumerate(_INTRADAY_SIGNALS)}
|
||||
_INTRADAY_COLS = ("symbol", "name", "close", "change_pct", "amplitude",
|
||||
"vol_ratio_5d", "turnover_rate", "consecutive_limit_ups")
|
||||
|
||||
|
||||
def build_intraday(repo: Any, limit: int = 500) -> dict[str, Any]:
|
||||
"""enriched 最新快照 → 当日异动信号命中行 (含各类型计数)。"""
|
||||
df, cache_date = repo.get_enriched_latest()
|
||||
empty = {"cache_date": cache_date.isoformat() if cache_date else None,
|
||||
"counts": {}, "rows": []}
|
||||
if df.is_empty() or "symbol" not in df.columns:
|
||||
return empty
|
||||
present = [(c, k) for c, k in _INTRADAY_SIGNALS if c in df.columns]
|
||||
if not present:
|
||||
return empty
|
||||
|
||||
hits = df.filter(pl.any_horizontal([pl.col(c).fill_null(False) for c, _ in present]))
|
||||
if hits.is_empty():
|
||||
return empty
|
||||
counts = {k: int(hits[c].fill_null(False).sum()) for c, k in present}
|
||||
|
||||
sig_cols = {k: hits[c].fill_null(False).to_list() for c, k in present}
|
||||
base_cols = [c for c in _INTRADAY_COLS if c in hits.columns]
|
||||
base = hits.select(base_cols).to_dicts()
|
||||
rows: list[dict[str, Any]] = []
|
||||
for i, r in enumerate(base):
|
||||
signals = [k for k, flags in sig_cols.items() if flags[i]]
|
||||
rows.append({
|
||||
**{c: r.get(c) for c in base_cols},
|
||||
"signals": signals,
|
||||
"_prio": min((_INTRADAY_PRIORITY[s] for s in signals), default=99),
|
||||
})
|
||||
rows.sort(key=lambda r: (r["_prio"], -abs(r.get("change_pct") or 0.0)))
|
||||
for r in rows:
|
||||
r.pop("_prio", None)
|
||||
return {"cache_date": cache_date.isoformat() if cache_date else None,
|
||||
"counts": counts, "rows": rows[:limit]}
|
||||
|
||||
@@ -0,0 +1,288 @@
|
||||
"""短线风向标服务 (fuyao 专有) — 复盘页卡片 + AI 复盘上下文。
|
||||
|
||||
非路由数据集: tickflow 无对应能力, 直接经 custom_sources 调 fuyao provider;
|
||||
fuyao 未配置时返回 source_unavailable 状态, 前端降级提示。
|
||||
|
||||
数据契约 (实测 2026-08-30, 60 交易日回测 353 样本):
|
||||
- 每日 5~6 只, 服务端筛选的竞价异动股, 附概念标签
|
||||
- 名单当日 (开盘买→收盘卖) 均值 +0.54% vs 全市场 +0.10%, 有真实当日选股能力;
|
||||
但高开≥5% 子集当日 -1.97% (追高陷阱) → 前端对高开子集标「追高风险」
|
||||
- 次日无显著优势 (+0.08%), 定位为「当日观察名单」而非隔夜轮动信号
|
||||
|
||||
缓存策略 (与 dragon_tiger 同模式):
|
||||
- 历史名单不可变 → 按日落 JSON 缓存 (data/auction_benchmark/date=YYYY-MM-DD.json),
|
||||
缓存命中不触发插件注册表加载
|
||||
- 收益 enrich (当日oc/全天/次日) 不落缓存 — 次日数据晚到, 读取时现算
|
||||
- 当日名单不缓存 (竞价阶段名单可能变动, 以现拉为准)
|
||||
- 显式日期失败 → 回退上一交易日一次 (state=fallback_prev)
|
||||
|
||||
日期解析: 接口对显式非交易日报 code=1002, 本层用本地 kline_daily 分区日期
|
||||
把目标日回退到「≤目标日的最近交易日」, 规避报错。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import json
|
||||
import logging
|
||||
import re
|
||||
from datetime import date as date_cls
|
||||
from pathlib import Path
|
||||
|
||||
import polars as pl
|
||||
|
||||
from app.market_time import cn_today
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_DATE_DIR_RE = re.compile(r"^date=(\d{4}-\d{2}-\d{2})$")
|
||||
|
||||
|
||||
def _local_trading_days(data_dir: Path) -> list[date_cls]:
|
||||
"""本地日K分区日期 = 已知交易日集合 (升序)。扫描失败返回空。"""
|
||||
root = data_dir / "kline_daily"
|
||||
out: list[date_cls] = []
|
||||
try:
|
||||
for d in root.iterdir():
|
||||
m = _DATE_DIR_RE.match(d.name)
|
||||
if d.is_dir() and m:
|
||||
try:
|
||||
out.append(date_cls.fromisoformat(m.group(1)))
|
||||
except ValueError:
|
||||
continue
|
||||
except OSError:
|
||||
return []
|
||||
return sorted(out)
|
||||
|
||||
|
||||
def resolve_trade_date(data_dir: Path, target: date_cls | None) -> date_cls | None:
|
||||
"""目标日 → ≤目标日的最近本地交易日。None → None (由 fuyao 取当日)。"""
|
||||
if target is None:
|
||||
return None
|
||||
days = _local_trading_days(data_dir)
|
||||
if not days:
|
||||
return target
|
||||
candidates = [d for d in days if d <= target]
|
||||
return max(candidates) if candidates else target
|
||||
|
||||
|
||||
def _prev_trading_day(data_dir: Path, d: date_cls) -> date_cls | None:
|
||||
days = _local_trading_days(data_dir)
|
||||
earlier = [x for x in days if x < d]
|
||||
return max(earlier) if earlier else None
|
||||
|
||||
|
||||
def _next_trading_day(data_dir: Path, d: date_cls) -> date_cls | None:
|
||||
days = _local_trading_days(data_dir)
|
||||
later = [x for x in days if x > d]
|
||||
return min(later) if later else None
|
||||
|
||||
|
||||
def _provider():
|
||||
from app.data_providers import custom as custom_sources
|
||||
|
||||
if not custom_sources.is_custom_provider("fuyao"):
|
||||
return None
|
||||
return custom_sources.get_provider("fuyao")
|
||||
|
||||
|
||||
def _cache_path(data_dir: Path, d: date_cls) -> Path:
|
||||
return data_dir / "auction_benchmark" / f"date={d.isoformat()}.json"
|
||||
|
||||
|
||||
def _load_cache(path: Path) -> dict | None:
|
||||
try:
|
||||
return json.loads(path.read_text(encoding="utf-8"))
|
||||
except (OSError, ValueError):
|
||||
return None
|
||||
|
||||
|
||||
def _store_cache(path: Path, payload: dict) -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
tmp = path.with_name(path.name + ".part")
|
||||
tmp.write_text(json.dumps(payload, ensure_ascii=False), encoding="utf-8")
|
||||
tmp.replace(path)
|
||||
|
||||
|
||||
def _raw_items(data: dict) -> list[dict]:
|
||||
items = data.get("item")
|
||||
return [r for r in items if isinstance(r, dict)] if isinstance(items, list) else []
|
||||
|
||||
|
||||
def _read_kline_closes(data_dir: Path, d: date_cls) -> dict[str, float]:
|
||||
"""某交易日全市场 {symbol: close}; 无分区/读失败返回空。"""
|
||||
root = data_dir / "kline_daily" / f"date={d.isoformat()}"
|
||||
try:
|
||||
files = sorted(root.glob("*.parquet"))
|
||||
if not files:
|
||||
return {}
|
||||
df = pl.concat([pl.read_parquet(f, columns=["symbol", "close"]) for f in files])
|
||||
return dict(zip(df["symbol"].to_list(), df["close"].to_list()))
|
||||
except (OSError, pl.exceptions.PolarsError):
|
||||
return {}
|
||||
|
||||
|
||||
def _enrich(data_dir: Path, trade_date: date_cls, items: list[dict]) -> list[dict]:
|
||||
"""名单叠加真实收益: day0_oc (开盘买→收盘卖) / day0_pct (全天) / d1_pct (次日)。
|
||||
|
||||
用相邻 kline_daily 分区现算, 不落缓存 — 次日分区晚到时先给 None, 到了自然补上。
|
||||
"""
|
||||
root = data_dir / "kline_daily" / f"date={trade_date.isoformat()}"
|
||||
day0: dict[str, tuple[float, float]] = {} # symbol -> (open, close)
|
||||
try:
|
||||
files = sorted(root.glob("*.parquet"))
|
||||
if files:
|
||||
df = pl.concat([pl.read_parquet(f, columns=["symbol", "open", "close"]) for f in files])
|
||||
day0 = dict(zip(df["symbol"].to_list(), zip(df["open"].to_list(), df["close"].to_list())))
|
||||
except (OSError, pl.exceptions.PolarsError):
|
||||
day0 = {}
|
||||
|
||||
prev = _prev_trading_day(data_dir, trade_date)
|
||||
prev_close = _read_kline_closes(data_dir, prev) if prev else {}
|
||||
nxt = _next_trading_day(data_dir, trade_date)
|
||||
next_close = _read_kline_closes(data_dir, nxt) if nxt else {}
|
||||
|
||||
out: list[dict] = []
|
||||
for r in items:
|
||||
sym = str(r.get("thscode") or "")
|
||||
oc = pct = d1 = None
|
||||
if sym in day0:
|
||||
o, c = day0[sym]
|
||||
if o and o > 0 and c is not None:
|
||||
oc = c / o - 1.0
|
||||
pc = prev_close.get(sym)
|
||||
if pc and pc > 0 and c is not None:
|
||||
pct = c / pc - 1.0
|
||||
nc = next_close.get(sym)
|
||||
if c and c > 0 and nc is not None:
|
||||
d1 = nc / c - 1.0
|
||||
out.append({
|
||||
"thscode": sym,
|
||||
"ticker": r.get("ticker"),
|
||||
"name": r.get("name"),
|
||||
"auction_pct": r.get("auction_pct"),
|
||||
"tags": [str(t) for t in (r.get("tags") or [])],
|
||||
"day0_oc": oc,
|
||||
"day0_pct": pct,
|
||||
"d1_pct": d1,
|
||||
})
|
||||
return out
|
||||
|
||||
|
||||
def _base_payload(data_dir: Path, trade_date: date_cls, data: dict) -> dict:
|
||||
return {
|
||||
"state": "ok",
|
||||
"requested_date": trade_date.isoformat(),
|
||||
"trade_date": str(data.get("date") or trade_date.isoformat()),
|
||||
"count": len(_raw_items(data)),
|
||||
"raw_items": _raw_items(data),
|
||||
}
|
||||
|
||||
|
||||
def get_auction_benchmark(data_dir: Path, target: date_cls | None = None) -> dict:
|
||||
"""取短线风向标名单 (含当日/次日真实收益)。返回给前端的统一容器。
|
||||
|
||||
state: ok (正常) | fallback_prev (目标日拉取失败, 已回退上一期)
|
||||
| source_unavailable (未配置 fuyao) | no_data (拉取失败)
|
||||
"""
|
||||
days = _local_trading_days(data_dir)
|
||||
if target is None:
|
||||
target = max(days) if days else None
|
||||
trade_date = resolve_trade_date(data_dir, target)
|
||||
today = cn_today()
|
||||
|
||||
# 历史日缓存优先 (纯本地, 不触发插件注册表加载)
|
||||
if trade_date is not None and trade_date < today:
|
||||
cached = _load_cache(_cache_path(data_dir, trade_date))
|
||||
if cached is not None:
|
||||
return _respond(data_dir, trade_date, cached, cached.get("state") or "ok")
|
||||
|
||||
provider = _provider()
|
||||
if provider is None:
|
||||
return {"state": "source_unavailable"}
|
||||
|
||||
from app.plugins.fuyao.client import FuyaoError
|
||||
|
||||
explicit = trade_date.isoformat() if trade_date is not None else None
|
||||
try:
|
||||
data = provider.short_term_benchmark(explicit)
|
||||
try:
|
||||
actual = date_cls.fromisoformat(str(data.get("date")))
|
||||
except ValueError:
|
||||
actual = trade_date
|
||||
base = _base_payload(data_dir, actual, data)
|
||||
# 历史日不可变 → 落缓存; 当日不缓存 (竞价阶段名单可能变动)
|
||||
if actual is not None and actual < today:
|
||||
with contextlib.suppress(OSError):
|
||||
_store_cache(_cache_path(data_dir, actual), base)
|
||||
return _respond(data_dir, actual, base, "ok")
|
||||
except FuyaoError as e:
|
||||
# 显式日期失败 (非交易日/边界日) → 回退上一交易日一次
|
||||
if trade_date is not None:
|
||||
prev = _prev_trading_day(data_dir, trade_date)
|
||||
if prev is not None:
|
||||
try:
|
||||
cached_prev = _load_cache(_cache_path(data_dir, prev))
|
||||
if cached_prev is not None:
|
||||
return _respond(data_dir, prev, cached_prev, "fallback_prev",
|
||||
requested=explicit)
|
||||
data_prev = provider.short_term_benchmark(prev.isoformat())
|
||||
base = _base_payload(data_dir, prev, data_prev)
|
||||
with contextlib.suppress(OSError):
|
||||
_store_cache(_cache_path(data_dir, prev), base)
|
||||
return _respond(data_dir, prev, base, "fallback_prev",
|
||||
requested=explicit)
|
||||
except FuyaoError:
|
||||
pass
|
||||
logger.warning("短线风向标拉取失败: %s", e)
|
||||
return {"state": "no_data", "message": str(e)}
|
||||
|
||||
|
||||
def _respond(
|
||||
data_dir: Path,
|
||||
trade_date: date_cls | None,
|
||||
base: dict,
|
||||
state: str,
|
||||
requested: str | None = None,
|
||||
) -> dict:
|
||||
"""缓存/现拉的原始容器 → 叠加收益 enrich 后的前端容器。"""
|
||||
if trade_date is None:
|
||||
return {**base, "state": state}
|
||||
payload = {
|
||||
"state": state,
|
||||
"requested_date": requested if requested is not None else base.get("requested_date"),
|
||||
"trade_date": base.get("trade_date") or trade_date.isoformat(),
|
||||
"count": base.get("count") or 0,
|
||||
"items": _enrich(data_dir, trade_date, base.get("raw_items") or []),
|
||||
}
|
||||
return payload
|
||||
|
||||
|
||||
def build_recap_context(data_dir: Path) -> str:
|
||||
"""AI 复盘的盘前风向标摘要段 (纯文本, 失败返回空串不影响复盘)。"""
|
||||
try:
|
||||
payload = get_auction_benchmark(data_dir, None)
|
||||
if payload.get("state") not in ("ok", "fallback_prev"):
|
||||
return ""
|
||||
items = payload.get("items") or []
|
||||
if not items:
|
||||
return ""
|
||||
trade_date = payload.get("trade_date") or ""
|
||||
lines = [f"(数据日期: {trade_date})"]
|
||||
segs = []
|
||||
for i in items:
|
||||
seg = (f"{i.get('name')}({i.get('thscode')}) 竞价{(i.get('auction_pct') or 0):+.2f}%"
|
||||
f"[{'·'.join(i.get('tags') or [])}]")
|
||||
if i.get("day0_oc") is not None:
|
||||
seg += f" → 当日开盘买{i['day0_oc']*100:+.2f}%"
|
||||
if i.get("d1_pct") is not None:
|
||||
seg += f", 次日{i['d1_pct']*100:+.2f}%"
|
||||
segs.append(seg)
|
||||
lines.append("盘前风向标名单: " + "; ".join(segs))
|
||||
ocs = [i["day0_oc"] for i in items if i.get("day0_oc") is not None]
|
||||
if ocs:
|
||||
lines.append(f"名单当日(开盘买→收盘卖)均值 {sum(ocs)/len(ocs)*100:+.2f}%")
|
||||
return "\n".join(lines)
|
||||
except Exception as e: # noqa: BLE001 — 摘要失败不影响复盘主流程
|
||||
logger.debug("盘前风向标复盘摘要构建失败: %s", e)
|
||||
return ""
|
||||
@@ -0,0 +1,227 @@
|
||||
"""龙虎榜数据服务 (fuyao 专有) — 复盘页卡片 + AI 复盘上下文。
|
||||
|
||||
非路由数据集: tickflow 无对应能力, 直接经 custom_sources 调 fuyao provider;
|
||||
fuyao 未配置时返回 source_unavailable 状态, 前端降级提示。
|
||||
|
||||
数据契约 (实测 2026-08, 文档的 limit_reason/amount 实际不返回):
|
||||
- 三榜一次取齐: all/org 为股票表 (org 额外带机构 4 字段), hot_money 为席位表
|
||||
- 历史榜单不可变 → 按日落 JSON 缓存 (data/dragon_tiger/date=YYYY-MM-DD.json)
|
||||
- 当日榜单盘中未发布 → 自动回退上一交易日并标记 state=fallback_prev
|
||||
|
||||
日期解析: 接口对显式非交易日报 code=1002, 本层用本地 kline_daily 分区日期
|
||||
把目标日回退到「≤目标日的最近交易日」, 规避报错。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import json
|
||||
import logging
|
||||
import re
|
||||
from datetime import date as date_cls
|
||||
from pathlib import Path
|
||||
|
||||
from app.market_time import cn_today
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_BOARDS = ("all", "org", "hot_money")
|
||||
_DATE_DIR_RE = re.compile(r"^date=(\d{4}-\d{2}-\d{2})$")
|
||||
|
||||
|
||||
def _local_trading_days(data_dir: Path) -> list[date_cls]:
|
||||
"""本地日K分区日期 = 已知交易日集合 (升序)。扫描失败返回空。"""
|
||||
root = data_dir / "kline_daily"
|
||||
out: list[date_cls] = []
|
||||
try:
|
||||
for d in root.iterdir():
|
||||
m = _DATE_DIR_RE.match(d.name)
|
||||
if d.is_dir() and m:
|
||||
try:
|
||||
out.append(date_cls.fromisoformat(m.group(1)))
|
||||
except ValueError:
|
||||
continue
|
||||
except OSError:
|
||||
return []
|
||||
return sorted(out)
|
||||
|
||||
|
||||
def resolve_trade_date(data_dir: Path, target: date_cls | None) -> date_cls | None:
|
||||
"""目标日 → ≤目标日的最近本地交易日。None 或本地无更早分区 → None (由 fuyao 默认)。
|
||||
|
||||
目标日早于全部本地分区 (极老的历史复盘) → 原样返回, 由调用方试显式日期,
|
||||
失败如实报 no_data, 不静默给错日期的数据。
|
||||
"""
|
||||
if target is None:
|
||||
return None
|
||||
days = _local_trading_days(data_dir)
|
||||
if not days:
|
||||
return target
|
||||
candidates = [d for d in days if d <= target]
|
||||
return max(candidates) if candidates else target
|
||||
|
||||
|
||||
def _prev_trading_day(data_dir: Path, d: date_cls) -> date_cls | None:
|
||||
days = _local_trading_days(data_dir)
|
||||
earlier = [x for x in days if x < d]
|
||||
return max(earlier) if earlier else None
|
||||
|
||||
|
||||
def _provider():
|
||||
from app.data_providers import custom as custom_sources
|
||||
|
||||
if not custom_sources.is_custom_provider("fuyao"):
|
||||
return None
|
||||
return custom_sources.get_provider("fuyao")
|
||||
|
||||
|
||||
def _fetch_boards(provider, d: date_cls) -> dict:
|
||||
"""三榜取齐。任一榜失败抛 FuyaoError (整日失败, 不留半套缓存)。"""
|
||||
return {
|
||||
bt: _boards_of(provider, bt, d.isoformat()) for bt in _BOARDS
|
||||
}
|
||||
|
||||
|
||||
def _boards_of(provider, board_type: str, iso: str | None) -> dict:
|
||||
data = provider.dragon_tiger(board_type, iso)
|
||||
return {
|
||||
"trade_date": data.get("trade_date"),
|
||||
"stock_count": data.get("stock_count"),
|
||||
"count": data.get("count"),
|
||||
"stock_items": data.get("stock_items") or [],
|
||||
"hot_money_items": data.get("hot_money_items") or [],
|
||||
}
|
||||
|
||||
|
||||
def _cache_path(data_dir: Path, d: date_cls) -> Path:
|
||||
return data_dir / "dragon_tiger" / f"date={d.isoformat()}.json"
|
||||
|
||||
|
||||
def _load_cache(path: Path) -> dict | None:
|
||||
try:
|
||||
return json.loads(path.read_text(encoding="utf-8"))
|
||||
except (OSError, ValueError):
|
||||
return None
|
||||
|
||||
|
||||
def _store_cache(path: Path, payload: dict) -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
tmp = path.with_name(path.name + ".part")
|
||||
tmp.write_text(json.dumps(payload, ensure_ascii=False), encoding="utf-8")
|
||||
tmp.replace(path)
|
||||
|
||||
|
||||
def get_dragon_tiger(data_dir: Path, target: date_cls | None = None) -> dict:
|
||||
"""取龙虎榜 (三榜)。返回给前端的统一容器。
|
||||
|
||||
state: ok (正常) | fallback_prev (当日未发布, 已回退上一期并带上一期数据)
|
||||
| source_unavailable (未配置 fuyao) | no_data (拉取失败)
|
||||
"""
|
||||
# 无目标日 → 最近本地交易日 (走缓存判定; 本地无任何分区才让 fuyao 自取默认)
|
||||
if target is None:
|
||||
days = _local_trading_days(data_dir)
|
||||
target = max(days) if days else None
|
||||
trade_date = resolve_trade_date(data_dir, target)
|
||||
today = cn_today()
|
||||
|
||||
# 历史日缓存优先 (纯本地, 不触发插件注册表加载)
|
||||
if trade_date is not None and trade_date < today:
|
||||
cached = _load_cache(_cache_path(data_dir, trade_date))
|
||||
if cached is not None:
|
||||
return cached
|
||||
|
||||
provider = _provider()
|
||||
if provider is None:
|
||||
return {"state": "source_unavailable"}
|
||||
|
||||
from app.plugins.fuyao.client import FuyaoError
|
||||
|
||||
# 日期未知 (target=None / 本地无分区) → 省略 date 让 fuyao 取最近已发布交易日
|
||||
explicit = trade_date.isoformat() if trade_date is not None else None
|
||||
try:
|
||||
raw = {bt: _boards_of(provider, bt, explicit) for bt in _BOARDS}
|
||||
try:
|
||||
actual = date_cls.fromisoformat(str(raw["all"].get("trade_date")))
|
||||
except ValueError:
|
||||
actual = None
|
||||
payload = {
|
||||
"state": "ok",
|
||||
"requested_date": explicit,
|
||||
"trade_date": raw["all"].get("trade_date"),
|
||||
**raw,
|
||||
}
|
||||
# 历史日不可变 → 落缓存; 当日不缓存 (盘中 fallback / 盘后补充都以现拉为准)
|
||||
if actual is not None and actual < today:
|
||||
with contextlib.suppress(OSError):
|
||||
_store_cache(_cache_path(data_dir, actual), payload)
|
||||
return payload
|
||||
except FuyaoError as e:
|
||||
# 显式日期失败 (当日未发布 / 边界日) → 回退上一交易日一次
|
||||
if trade_date is not None:
|
||||
prev = _prev_trading_day(data_dir, trade_date)
|
||||
if prev is not None:
|
||||
try:
|
||||
cached_prev = _load_cache(_cache_path(data_dir, prev))
|
||||
if cached_prev is not None:
|
||||
base = dict(cached_prev)
|
||||
base.pop("state", None)
|
||||
return {**base, "state": "fallback_prev",
|
||||
"requested_date": explicit}
|
||||
raw_prev = _fetch_boards(provider, prev)
|
||||
base = {
|
||||
"requested_date": prev.isoformat(),
|
||||
"trade_date": prev.isoformat(),
|
||||
**raw_prev,
|
||||
}
|
||||
with contextlib.suppress(OSError):
|
||||
_store_cache(_cache_path(data_dir, prev), {**base, "state": "ok"})
|
||||
return {**base, "state": "fallback_prev",
|
||||
"requested_date": explicit}
|
||||
except FuyaoError:
|
||||
pass
|
||||
logger.warning("龙虎榜拉取失败: %s", e)
|
||||
return {"state": "no_data", "message": str(e)}
|
||||
|
||||
|
||||
def build_recap_context(data_dir: Path) -> str:
|
||||
"""AI 复盘的龙虎榜摘要段 (纯文本, 失败返回空串不影响复盘)。"""
|
||||
try:
|
||||
payload = get_dragon_tiger(data_dir, None)
|
||||
if payload.get("state") not in ("ok", "fallback_prev"):
|
||||
return ""
|
||||
items = payload.get("all", {}).get("stock_items") or []
|
||||
org_items = payload.get("org", {}).get("stock_items") or []
|
||||
hm_items = payload.get("hot_money", {}).get("hot_money_items") or []
|
||||
trade_date = payload.get("trade_date") or ""
|
||||
lines = [f"(数据日期: {trade_date})"]
|
||||
|
||||
top_buy = sorted(
|
||||
[i for i in items if (i.get("net_value") or 0) > 0],
|
||||
key=lambda x: x.get("net_value") or 0, reverse=True,
|
||||
)[:5]
|
||||
if top_buy:
|
||||
lines.append("净买入居前: " + "; ".join(
|
||||
f"{i.get('name')}({i.get('thscode')}) 净买{float(i.get('net_value') or 0)/1e8:.2f}亿"
|
||||
f" 涨跌{(i.get('change') or 0)*100:.1f}%"
|
||||
+ (" [3日榜]" if i.get("range_days") == 3 else "")
|
||||
for i in top_buy))
|
||||
top_sell = sorted(
|
||||
[i for i in items if (i.get("net_value") or 0) < 0],
|
||||
key=lambda x: x.get("net_value") or 0,
|
||||
)[:5]
|
||||
if top_sell:
|
||||
lines.append("净卖出居前: " + "; ".join(
|
||||
f"{i.get('name')} 净卖{abs(float(i.get('net_value') or 0))/1e8:.2f}亿"
|
||||
for i in top_sell))
|
||||
if org_items:
|
||||
lines.append("机构净买居前: " + "; ".join(
|
||||
f"{i.get('name')} 机构净买{float(i.get('org_net_value') or 0)/1e8:.2f}亿"
|
||||
for i in sorted(org_items, key=lambda x: x.get("org_net_value") or 0, reverse=True)[:5]))
|
||||
if hm_items:
|
||||
lines.append("活跃游资: " + "; ".join(
|
||||
f"{h.get('name')} 净买{float(h.get('buying') or 0)/1e8:.2f}亿"
|
||||
for h in sorted(hm_items, key=lambda x: x.get('buying') or 0, reverse=True)[:5]))
|
||||
return "\n".join(lines)
|
||||
except Exception as e: # noqa: BLE001 — 摘要失败不影响复盘主流程
|
||||
logger.debug("龙虎榜复盘摘要构建失败: %s", e)
|
||||
return ""
|
||||
@@ -167,8 +167,6 @@ def run_extend_history(
|
||||
|
||||
from app.services import preferences as _prefs
|
||||
adj_provider = _prefs.get_adj_factor_provider()
|
||||
if adj_provider == "same_as_daily":
|
||||
adj_provider = _prefs.get_daily_data_provider()
|
||||
can_sync_adj = capset.has(Cap.ADJ_FACTOR) or adj_provider != "tickflow"
|
||||
if can_sync_adj:
|
||||
emit("extend_history", 48, f"获取除权因子 [{adj_start_str} ~ {adj_end_str}]…")
|
||||
|
||||
@@ -155,6 +155,13 @@ def _sync_table(
|
||||
|
||||
|
||||
def _merge_report_history(*frames: pl.DataFrame) -> pl.DataFrame:
|
||||
"""按 (symbol, period_end) 合并各报告期, 同期多行逐列取最新非空值。
|
||||
|
||||
语义(区分"覆盖"与"填空"): 每列独立取 announce_date 最新的非空值 —
|
||||
新同步行有值则覆盖旧值, 新行缺的列(如 fuyao 不提供的字段)由旧行补齐,
|
||||
实现多数据源并集共存。历史报告期不可变, 合并不会引入过期数据。
|
||||
无 announce_date 的帧按输入顺序, 后写优先(与旧行为 keep="last" 一致)。
|
||||
"""
|
||||
valid = [
|
||||
frame
|
||||
for frame in frames
|
||||
@@ -166,11 +173,15 @@ def _merge_report_history(*frames: pl.DataFrame) -> pl.DataFrame:
|
||||
pl.concat(valid, how="diagonal_relaxed")
|
||||
.filter(pl.col("symbol").is_not_null() & pl.col("period_end").is_not_null())
|
||||
)
|
||||
# 同一 (symbol, period_end) 多条时保留 announce_date 最新一条 (业绩修正以最新公告为准)。
|
||||
if "announce_date" in merged.columns:
|
||||
merged = merged.sort(["symbol", "period_end", "announce_date"], nulls_last=True)
|
||||
return merged.unique(subset=["symbol", "period_end"], keep="last").sort(
|
||||
["symbol", "period_end"]
|
||||
sort_keys = ["symbol", "period_end"] + (
|
||||
["announce_date"] if "announce_date" in merged.columns else []
|
||||
)
|
||||
merged = merged.sort(sort_keys, nulls_last=True)
|
||||
value_cols = [c for c in merged.columns if c not in ("symbol", "period_end")]
|
||||
return (
|
||||
merged.group_by("symbol", "period_end")
|
||||
.agg([pl.col(c).drop_nulls().last() for c in value_cols])
|
||||
.sort(["symbol", "period_end"])
|
||||
)
|
||||
|
||||
|
||||
|
||||
@@ -20,7 +20,7 @@ from app.services import preferences
|
||||
from app.tickflow.capabilities import Cap, CapabilitySet
|
||||
from app.tickflow.client import get_client
|
||||
from app.tickflow.rate_limits import chunked, resolve_limit, sleep_between_batches
|
||||
from app.tickflow.repository import KlineRepository
|
||||
from app.tickflow.repository import KlineRepository, replace_with_retry
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -32,11 +32,12 @@ def _atomic_write_parquet(df: pl.DataFrame, out) -> None:
|
||||
单文件、每次「读→concat→原地写」, 直接 write_parquet(out) 在进程被 kill
|
||||
(dev.sh 清端口用 kill -9)、reap 超时或断电时会留下半截文件, 之后复权视图
|
||||
scan_parquet 整条链路报错、enriched 全市场重算不出。临时文件后缀 .tmp 不匹配
|
||||
*.parquet glob, 不会被扫描误读。
|
||||
*.parquet glob, 不会被扫描误读。Windows 下目标正被并发读取时由
|
||||
replace_with_retry 短退避穿过。
|
||||
"""
|
||||
tmp = out.with_name(out.name + ".tmp")
|
||||
df.write_parquet(tmp)
|
||||
tmp.replace(out) # 同目录 rename, POSIX/NTFS 均为原子操作
|
||||
replace_with_retry(tmp, out)
|
||||
|
||||
|
||||
# 标准列(无论 SDK 返回什么形状,我们把它规范成这套)
|
||||
@@ -335,8 +336,6 @@ def sync_adj_factor(symbols: list[str], repo: KlineRepository,
|
||||
return 0, []
|
||||
|
||||
provider_name = preferences.get_adj_factor_provider()
|
||||
if provider_name == "same_as_daily":
|
||||
provider_name = preferences.get_daily_data_provider()
|
||||
if provider_name != "tickflow":
|
||||
from app.data_providers import custom as custom_sources
|
||||
if custom_sources.provider_has_dataset(provider_name, "adj_factor"):
|
||||
@@ -446,8 +445,80 @@ CANONICAL_MINUTE_COLS = [
|
||||
]
|
||||
|
||||
|
||||
# 北京墙钟特征时段(含集合竞价 09:15 与收盘 15:00): 上午 09-11, 下午 13-15
|
||||
_BJ_HOURS = [9, 10, 11, 13, 14, 15]
|
||||
# 上述时段 -8h 的 UTC 墙钟特征: 上午 01-03, 下午 05-07
|
||||
_UTC_SHIFTED_HOURS = [1, 2, 3, 5, 6, 7]
|
||||
|
||||
|
||||
def _enforce_minute_beijing_wallclock(df: pl.DataFrame, *, source: str) -> pl.DataFrame:
|
||||
"""分钟 K datetime 时区契约守卫: 统一为北京墙钟 (naive)。
|
||||
|
||||
契约 (CONTRIBUTING §3.3): kline_minute.datetime 必须是北京时间墙钟, 如 09:35:00。
|
||||
在两个源头入口强制 —— _normalize_minute (TickFlow 帧) 与 _try_custom_minute
|
||||
(插件/自定义源帧); 落盘 (_write_minute_partition) 与内存消费 (监控/补拉/脉冲)
|
||||
均在其下游, 这里收口即全覆盖:
|
||||
- tz-aware → 转 Asia/Shanghai 后去时区;
|
||||
- naive 且时刻落在 A 股交易时段 → 直通 (已是北京墙钟);
|
||||
- naive 且整体呈"交易时段 -8h"的 UTC 特征 → 自动 +8 纠偏并记日志;
|
||||
- 无法识别的口径 → fail-closed 抛 ValueError, 不让脏时间入库或下发。
|
||||
幂等: 纠偏后的帧再过守卫直通, 不会二次改写。
|
||||
"""
|
||||
if df.is_empty() or "datetime" not in df.columns:
|
||||
return df
|
||||
dtype = df.schema["datetime"]
|
||||
if not isinstance(dtype, pl.Datetime):
|
||||
# trade_time 等字符串路径: 先解析成 Datetime (失败置 null), 再做时段分类
|
||||
if dtype == pl.Utf8:
|
||||
df = df.with_columns(pl.col("datetime").str.to_datetime(strict=False))
|
||||
else:
|
||||
df = df.with_columns(pl.col("datetime").cast(pl.Datetime("us"), strict=False))
|
||||
dtype = df.schema["datetime"]
|
||||
if not isinstance(dtype, pl.Datetime):
|
||||
return df # 仍非 Datetime: 维持原行为交由下游处理
|
||||
if isinstance(dtype, pl.Datetime) and dtype.time_zone is not None:
|
||||
df = df.with_columns(
|
||||
pl.col("datetime")
|
||||
.dt.convert_time_zone("Asia/Shanghai")
|
||||
.dt.replace_time_zone(None)
|
||||
.cast(pl.Datetime("us"))
|
||||
)
|
||||
logger.info("minute datetime tz-aware input converted to Beijing wallclock (source=%s)", source)
|
||||
return df
|
||||
|
||||
hour = pl.col("datetime").dt.hour()
|
||||
beijing = int(df.select(hour.is_in(_BJ_HOURS).sum()).item() or 0)
|
||||
utc_shifted = int(df.select(hour.is_in(_UTC_SHIFTED_HOURS).sum()).item() or 0)
|
||||
if beijing == 0 and utc_shifted == 0:
|
||||
if df["datetime"].null_count() == df.height:
|
||||
return df # 全 null: 维持原行为, 由下游落盘过滤
|
||||
raise ValueError(
|
||||
f"minute datetime 口径无法识别 (source={source}, rows={df.height}, "
|
||||
f"sample={df['datetime'].drop_nulls().head(2).to_list()}): "
|
||||
"契约要求北京墙钟 (09:30-15:00), 既非交易时段也非 UTC 平移特征"
|
||||
)
|
||||
if utc_shifted > beijing:
|
||||
if beijing:
|
||||
logger.warning(
|
||||
"minute datetime mixed convention, shifting all by +8h per UTC majority "
|
||||
"(source=%s, utc=%d, beijing=%d)", source, utc_shifted, beijing,
|
||||
)
|
||||
else:
|
||||
logger.info(
|
||||
"minute datetime UTC wallclock detected, shifted +8h to Beijing "
|
||||
"(source=%s, rows=%d)", source, utc_shifted,
|
||||
)
|
||||
return df.with_columns(pl.col("datetime") + pl.duration(hours=8))
|
||||
if utc_shifted:
|
||||
logger.warning(
|
||||
"minute datetime has %d UTC-like rows among %d Beijing rows, left as-is "
|
||||
"(source=%s)", utc_shifted, beijing, source,
|
||||
)
|
||||
return df
|
||||
|
||||
|
||||
def _normalize_minute(df_in, default_symbol: str | None = None) -> pl.DataFrame:
|
||||
"""把 SDK 返回的分钟 K 数据规范成 canonical 列。"""
|
||||
"""把 SDK 返回的分钟 K 数据规范成 canonical 列 (datetime 收口为北京墙钟)。"""
|
||||
if df_in is None or len(df_in) == 0:
|
||||
return pl.DataFrame()
|
||||
|
||||
@@ -465,8 +536,14 @@ def _normalize_minute(df_in, default_symbol: str | None = None) -> pl.DataFrame:
|
||||
|
||||
# datetime 列:优先用 timestamp(毫秒精度),其次 trade_time
|
||||
if "timestamp" in df.columns:
|
||||
# TickFlow 毫秒时间戳为 UTC 基准; 契约要求北京墙钟 naive
|
||||
# (与 stock-sdk provider 归一口径一致, 见 CONTRIBUTING §3.3)
|
||||
df = df.with_columns(
|
||||
pl.from_epoch("timestamp", time_unit="ms").alias("datetime"),
|
||||
pl.from_epoch(pl.col("timestamp").cast(pl.Int64), time_unit="ms")
|
||||
.dt.replace_time_zone("UTC")
|
||||
.dt.convert_time_zone("Asia/Shanghai")
|
||||
.dt.replace_time_zone(None)
|
||||
.alias("datetime")
|
||||
).drop("timestamp")
|
||||
for drop_col in ("trade_time", "trade_date"):
|
||||
if drop_col in df.columns:
|
||||
@@ -478,6 +555,10 @@ def _normalize_minute(df_in, default_symbol: str | None = None) -> pl.DataFrame:
|
||||
elif "trade_date" in df.columns:
|
||||
df = df.rename({"trade_date": "datetime"})
|
||||
|
||||
if "datetime" in df.columns:
|
||||
# 时区契约守卫: 须在下方通用 us-cast 之前, 避免带时区列被静默剥成 UTC-naive
|
||||
df = _enforce_minute_beijing_wallclock(df, source="tickflow")
|
||||
|
||||
if "symbol" not in df.columns and default_symbol is not None:
|
||||
df = df.with_columns(pl.lit(default_symbol).alias("symbol"))
|
||||
|
||||
@@ -606,11 +687,18 @@ def _try_custom_minute(
|
||||
symbols, start_time=start_time, end_time=end_time,
|
||||
asset_type=asset_type, freq=freq, on_chunk_done=wrapped_cb,
|
||||
)
|
||||
return (df, False)
|
||||
except Exception as e: # noqa: BLE001
|
||||
except Exception as e:
|
||||
logger.warning("custom minute provider %s call failed, falling back to TickFlow: %s",
|
||||
provider_name, e)
|
||||
return (None, True)
|
||||
try:
|
||||
# 时区契约守卫: 插件/自定义源帧同样收口为北京墙钟 (CONTRIBUTING §3.3)
|
||||
df = _enforce_minute_beijing_wallclock(df, source=provider_name)
|
||||
except Exception as e:
|
||||
logger.warning("custom minute provider %s datetime 契约校验失败, falling back to TickFlow: %s",
|
||||
provider_name, e)
|
||||
return (None, True)
|
||||
return (df, False)
|
||||
|
||||
|
||||
def sync_minute_batch(
|
||||
@@ -828,6 +916,110 @@ def fetch_intraday_monitor_batch(
|
||||
return pl.concat(frames, how="diagonal_relaxed") if frames else pl.DataFrame()
|
||||
|
||||
|
||||
def fetch_intraday_full_market_burst(
|
||||
symbols: list[str],
|
||||
capset: CapabilitySet | None,
|
||||
*,
|
||||
count: int = 300,
|
||||
) -> tuple[pl.DataFrame, int]:
|
||||
"""全市场当日分钟K并发脉冲拉取 (盘中增量刷新专用, 不落盘)。
|
||||
|
||||
与 fetch_intraday_monitor_batch 的区别:
|
||||
- 监控路径每轮只拉少量标的 (≤ batch 上限, 单请求);
|
||||
本函数按 batch_size 把全市场切块后用线程池一次全部打出
|
||||
(5546/200 = 28 并发), 配合 >=60s 的固定轮节奏, 任何 60s
|
||||
滑动窗口至多一个脉冲 (28 < 48 安全 rpm)。
|
||||
- 单块失败不拖垮整轮: 成功块照常返回落盘; 失败块立即单独重试一次,
|
||||
仍失败则跳过该块 (本函数每轮拉全天, 下一轮天然自愈)。失败块过多
|
||||
(>4, 系统性故障/限流风暴) 时跳过重试, 避免向已过载的服务端加压。
|
||||
|
||||
限流口径: 只用 intraday.batch 独立池 (Cap.INTRADAY_BATCH, Expert 专有),
|
||||
不与 kline.minute.batch (盘后分钟同步) 共享配额。
|
||||
返回 (当日全市场分钟K, 请求数)。
|
||||
"""
|
||||
if not symbols:
|
||||
return (pl.DataFrame(), 0)
|
||||
limits = capset.limits(Cap.INTRADAY_BATCH) if capset and capset.has(Cap.INTRADAY_BATCH) else None
|
||||
batch_size = max(1, int(limits.batch) if limits and limits.batch else 200)
|
||||
chunks = list(chunked(symbols, batch_size))
|
||||
if not chunks:
|
||||
return (pl.DataFrame(), 0)
|
||||
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
|
||||
tf = get_client()
|
||||
|
||||
def _fetch(chunk: list[str]) -> tuple[list[pl.DataFrame], Exception | None]:
|
||||
# 单块独立容错: 异常作为返回值上交而不是抛出, 避免一个块把整轮
|
||||
# 已成功的数据一起拖垮 (pool.map 迭代中抛异常会废弃全部已收 frames)
|
||||
try:
|
||||
raw = tf.klines.intraday_batch(
|
||||
chunk, count=count, as_dataframe=True, show_progress=False,
|
||||
batch_size=len(chunk),
|
||||
)
|
||||
return (_normalize_intraday_raw(raw), None)
|
||||
except Exception as e:
|
||||
return ([], e)
|
||||
|
||||
frames: list[pl.DataFrame] = []
|
||||
failed: list[list[str]] = []
|
||||
with ThreadPoolExecutor(max_workers=min(len(chunks), 32)) as pool:
|
||||
for chunk, (sub, err) in zip(chunks, pool.map(_fetch, chunks), strict=True):
|
||||
if err is not None:
|
||||
failed.append(chunk)
|
||||
else:
|
||||
frames.extend(sub)
|
||||
|
||||
requests = len(chunks)
|
||||
if failed:
|
||||
if len(failed) > 4:
|
||||
missed = sum(len(chunk) for chunk in failed)
|
||||
logger.warning(
|
||||
"intraday burst: %d/%d chunks failed (%d symbols), systemic — skip retry, next round re-pulls full day",
|
||||
len(failed), len(chunks), missed,
|
||||
)
|
||||
else:
|
||||
logger.warning("intraday burst: %d/%d chunks failed, retrying once", len(failed), len(chunks))
|
||||
for chunk in failed:
|
||||
sub, err = _fetch(chunk)
|
||||
requests += 1
|
||||
if err is None:
|
||||
frames.extend(sub)
|
||||
else:
|
||||
logger.warning(
|
||||
"intraday burst: chunk retry still failed, skip %d symbols this round: %s",
|
||||
len(chunk), err,
|
||||
)
|
||||
if not frames:
|
||||
return (pl.DataFrame(), requests)
|
||||
return (pl.concat(frames, how="diagonal_relaxed"), requests)
|
||||
|
||||
|
||||
def fetch_intraday_universe_increment(
|
||||
universe: str = "CN_Equity_A",
|
||||
*,
|
||||
count: int = 3,
|
||||
) -> tuple[pl.DataFrame, int]:
|
||||
"""全市场当日分钟K增量拉取 (盘中稳态轮专用, 不落盘)。
|
||||
|
||||
/v1/klines/intraday/universe: 传 universe ID 一次请求返回全市场每只标的
|
||||
最新 count 根分钟K (服务端实测上限 3 根/标的), 替代稳态场景下 28 块并发
|
||||
的 intraday.batch 脉冲 (请求量 28→1, 传输量 ~40 倍降)。缺口回补
|
||||
(冷启动/长时间断档/全天修复) 仍走 fetch_intraday_full_market_burst。
|
||||
返回 (增量分钟K, 请求数); 拉取失败返回空 df 由调用方按失败轮处理。
|
||||
"""
|
||||
tf = get_client()
|
||||
try:
|
||||
raw = tf.klines.intraday_universe(universe, count=count, as_dataframe=True)
|
||||
except Exception as e:
|
||||
logger.warning("intraday universe fetch failed (%s): %s", universe, e)
|
||||
return (pl.DataFrame(), 0)
|
||||
frames = _normalize_intraday_raw(raw)
|
||||
if not frames:
|
||||
return (pl.DataFrame(), 0)
|
||||
return (pl.concat(frames, how="diagonal_relaxed"), 1)
|
||||
|
||||
|
||||
def fetch_minute_single(
|
||||
symbol: str,
|
||||
trade_date: date,
|
||||
|
||||
@@ -258,10 +258,12 @@ def _dimension_rank(rows: list[dict], repo, kind: str, limit: int = 5, level: in
|
||||
|
||||
store = ExtConfigStore(repo.store.data_dir)
|
||||
groups: dict[str, dict[str, dict]] = {}
|
||||
group_source: dict[str, str] = {} # 组名 → 首个命中的扩展字段 "configId.field" (看板成分股弹窗用)
|
||||
for config in store.load_all():
|
||||
field = _dimension_field(config, kind)
|
||||
if not field:
|
||||
continue
|
||||
source_field = f"{config.id}.{field}"
|
||||
for ext_row in _read_ext_rows(repo.store.data_dir, config, field):
|
||||
quote = None
|
||||
for key in _symbol_keys(ext_row, config):
|
||||
@@ -277,6 +279,7 @@ def _dimension_rank(rows: list[dict], repo, kind: str, limit: int = 5, level: in
|
||||
parts = value.split("-")
|
||||
value = parts[level - 1] if level <= len(parts) else parts[-1]
|
||||
groups.setdefault(value, {})[symbol] = quote
|
||||
group_source.setdefault(value, source_field)
|
||||
|
||||
items = []
|
||||
for name, by_symbol in groups.items():
|
||||
@@ -293,6 +296,7 @@ def _dimension_rank(rows: list[dict], repo, kind: str, limit: int = 5, level: in
|
||||
"up_count": sum(1 for v in changes if v > 0),
|
||||
"down_count": sum(1 for v in changes if v < 0),
|
||||
"amount": sum(_finite(s.get("amount")) or 0 for s in stocks),
|
||||
"source_field": group_source.get(name),
|
||||
"leader": {
|
||||
"symbol": leader.get("symbol"),
|
||||
"name": leader.get("name"),
|
||||
|
||||
@@ -179,8 +179,9 @@ def _build_emotion_block(overview: dict) -> str:
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def _build_user_prompt(overview: dict, news: list[dict], focus: str) -> str:
|
||||
"""构建用户消息:复盘日期 + 市场数据精简切片 + 新闻 + 关注点。"""
|
||||
def _build_user_prompt(overview: dict, news: list[dict], focus: str, lhb_context: str = "",
|
||||
bench_context: str = "") -> str:
|
||||
"""构建用户消息:复盘日期 + 市场数据精简切片 + 龙虎榜(可选) + 盘前风向标(可选) + 新闻 + 关注点。"""
|
||||
as_of = overview.get("as_of") or "今日"
|
||||
|
||||
parts: list[str] = [
|
||||
@@ -202,6 +203,22 @@ def _build_user_prompt(overview: dict, news: list[dict], focus: str) -> str:
|
||||
_build_sector_block(overview.get("industry_rank"), "行业"),
|
||||
]
|
||||
|
||||
# 龙虎榜资金动向 (fuyao 数据源, 摘要自带数据日期; 无数据源/失败时为空不占段)
|
||||
if lhb_context:
|
||||
parts.extend([
|
||||
"",
|
||||
"## 龙虎榜资金动向",
|
||||
lhb_context,
|
||||
])
|
||||
|
||||
# 盘前风向标 (fuyao 竞价筛选名单 + 当日实际表现对照; 失败为空不占段)
|
||||
if bench_context:
|
||||
parts.extend([
|
||||
"",
|
||||
"## 盘前风向标(竞价)",
|
||||
bench_context,
|
||||
])
|
||||
|
||||
if news:
|
||||
news_lines = []
|
||||
for i, n in enumerate(news[:8], 1):
|
||||
@@ -300,7 +317,16 @@ async def recap_market_stream(
|
||||
try:
|
||||
from app.services.ai_provider import stream_ai_text
|
||||
|
||||
user_prompt = _build_user_prompt(overview, news or [], focus)
|
||||
# 龙虎榜摘要 (fuyao 专有): 拉取失败/未配置 → 空串, 复盘主流程不受影响
|
||||
from app.services import dragon_tiger as dragon_tiger_svc
|
||||
|
||||
lhb_ctx = dragon_tiger_svc.build_recap_context(repo.store.data_dir)
|
||||
|
||||
# 盘前风向标摘要 (fuyao 专有): 失败/未配置 → 空串
|
||||
from app.services import auction_benchmark as auction_benchmark_svc
|
||||
|
||||
bench_ctx = auction_benchmark_svc.build_recap_context(repo.store.data_dir)
|
||||
user_prompt = _build_user_prompt(overview, news or [], focus, lhb_ctx, bench_ctx)
|
||||
got_content = False
|
||||
async for delta in stream_ai_text(
|
||||
[
|
||||
|
||||
@@ -0,0 +1,305 @@
|
||||
"""盘中分钟K增量落盘服务 (Expert 专有)。
|
||||
|
||||
两段式拉取 (见 feat/minute-strategy 方案):
|
||||
- 全天修复轮: intraday.batch (日内分时批量) 并发脉冲一次拉全市场当日全部
|
||||
分钟K — 冷启动 (如 10 点才开服务, 补 9:30 起缺口) / 覆盖滞后超阈值 /
|
||||
连续空轮自愈时触发。
|
||||
- 稳态增量轮: intraday.universe 传 CN_Equity_A 标的池, 单请求返回全市场
|
||||
每只最新 3 根 (服务端上限), 靠 _write_minute_partition 的
|
||||
unique(symbol,datetime) 幂等合并滚出全天。
|
||||
|
||||
单轮合并写入当日 kline_minute 分区, 供分钟策略 (minute_filter) 读到新鲜数据。
|
||||
|
||||
设计约束:
|
||||
- Expert 专有: 能力门控 Cap.INTRADAY_UNIVERSE (全量分钟) — 仅 TickFlow
|
||||
Expert 档具备, 天然排他 (自定义分钟源无此能力, 且服务本就让位插件)。
|
||||
- 修复轮并发脉冲: 全市场按 batch_size 分块 (5546/200 = 28 块) 一次打出。
|
||||
任何 60s 滑动窗口至多一个脉冲 (28 < 48 安全 rpm); 单块失败不拖垮整轮,
|
||||
失败块单独重试一次 (见 fetch_intraday_full_market_burst)。
|
||||
- 稳态轮单请求: 无脉冲并发, 间隔可低至 3s; 实际节奏 = max(间隔, 单轮完成),
|
||||
服务端响应 ~5s 时自动退化为响应节奏, 不会重叠请求。
|
||||
- 固定节奏: 默认 6s 一轮 (clamp [3, 300]), 不补跑 (missed 轮次直接跳过)。
|
||||
- 仅连续竞价时段运行 (9:30-11:30 / 13:00-15:00), 午休/收盘自动暂停与恢复;
|
||||
午休后恢复因覆盖滞后会多跑一次修复轮, 幂等无害。
|
||||
- 不与其他分钟能力冲突: 与 盘后分钟同步 (kline.minute.batch) / 分时监控路径
|
||||
分属不同限流池; 落盘走 _write_minute_partition 的 unique(symbol,datetime)
|
||||
合并, 与盘后同步写同一分区安全幂等。
|
||||
- 数据源插件化让位: 配置了自定义分钟源 (minute_data_provider != tickflow) 时
|
||||
服务不启动 — 盘中增量交由插件自管, 本服务不抢占。
|
||||
|
||||
分层: 本模块只做调度/落盘/状态; TickFlow SDK 调用全部在 kline_sync 边界层
|
||||
(fetch_intraday_full_market_burst / fetch_intraday_universe_increment),
|
||||
保持插件化边界不泄漏。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import contextlib
|
||||
import threading
|
||||
import time
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Any
|
||||
|
||||
import polars as pl
|
||||
|
||||
from app.market_time import cn_now, cn_today, in_continuous_session
|
||||
from app.services import preferences
|
||||
|
||||
# 轮询间隔允许范围 (秒): 稳态轮单请求无并发脉冲, 下限 3s; 上限防误配。
|
||||
REFRESH_INTERVAL_MIN = 3
|
||||
REFRESH_INTERVAL_MAX = 300
|
||||
# 等待步长 (秒): 循环小步睡眠, 便于快速停止与偏好热生效。
|
||||
_LOOP_STEP_S = 2.0
|
||||
# 当日覆盖滞后超过该分钟数 (≈ universe 单请求 3 根余量) → 触发全天修复轮。
|
||||
_REPAIR_LAG_MINUTES = 3.0
|
||||
# 连续空轮达到该次数 → 强制全天修复轮 (自愈 universe 端点持续异常)。
|
||||
_EMPTY_ROUNDS_TO_REPAIR = 2
|
||||
|
||||
|
||||
def _in_continuous_session(now=None) -> bool:
|
||||
"""A股连续竞价时段 (北京时间): 9:30-11:30 / 13:00-15:00, 仅工作日。"""
|
||||
return in_continuous_session(now)
|
||||
|
||||
|
||||
@dataclass
|
||||
class _RefreshState:
|
||||
"""服务运行状态 (status() 的内存镜像, 循环线程内更新)。"""
|
||||
|
||||
rounds: int = 0
|
||||
last_round_at: float | None = None # epoch 秒
|
||||
last_round_ms: float | None = None # 单轮耗时
|
||||
last_rows: int = 0 # 上轮写入行数 (合并后)
|
||||
last_symbols: int = 0 # 上轮覆盖标的数
|
||||
last_requests: int = 0 # 上轮请求数 (增量恒 1, 修复=分块数+重试)
|
||||
last_mode: str | None = None # 上轮模式: "increment" / "full"
|
||||
last_error: str | None = None
|
||||
next_round_at: float | None = None # epoch 秒
|
||||
extra: dict[str, Any] = field(default_factory=dict)
|
||||
|
||||
|
||||
class MinuteRefreshService:
|
||||
"""盘中分钟增量刷新: 单实例挂 app.state.minute_refresh, 后台守护线程。"""
|
||||
|
||||
def __init__(self, repo) -> None:
|
||||
self._repo = repo
|
||||
self._app_state: Any | None = None
|
||||
self._thread: threading.Thread | None = None
|
||||
self._stop = threading.Event()
|
||||
self._state = _RefreshState()
|
||||
self._round_lock = threading.Lock() # 同时只允许一轮 (手动触发与定时轮互斥)
|
||||
self._empty_rounds = 0 # 连续空轮计数 (escalate 到全天修复)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 生命周期
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def set_repo(self, repo) -> None:
|
||||
self._repo = repo
|
||||
|
||||
def set_app_state(self, app_state: Any) -> None:
|
||||
self._app_state = app_state
|
||||
|
||||
def start(self) -> bool:
|
||||
"""启动后台线程 (幂等)。开关/时段/能力判断都在循环内每轮做, 热生效。"""
|
||||
if self._thread is not None and self._thread.is_alive():
|
||||
return True
|
||||
self._stop.clear()
|
||||
self._thread = threading.Thread(
|
||||
target=self._loop, name="minute-refresh", daemon=True,
|
||||
)
|
||||
self._thread.start()
|
||||
return True
|
||||
|
||||
def stop(self) -> None:
|
||||
self._stop.set()
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 门控
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def capability_ok(self) -> bool:
|
||||
"""Cap.INTRADAY_UNIVERSE (全量分钟) 存在。能力探测结果缓存在 app.state。
|
||||
|
||||
门控挂在稳态增量的主能力上; 全天修复轮用的 intraday.batch 与其
|
||||
同属 Expert 档 (tiers.yaml), 目前两者必然同时持有。
|
||||
"""
|
||||
capset = getattr(self._app_state, "capabilities", None) if self._app_state else None
|
||||
if capset is None:
|
||||
return False
|
||||
try:
|
||||
from app.tickflow.capabilities import Cap
|
||||
|
||||
return capset.has(Cap.INTRADAY_UNIVERSE)
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
def custom_provider_active(self) -> bool:
|
||||
"""配置了自定义分钟源 → 让位插件, 本服务不启动。"""
|
||||
try:
|
||||
return preferences.get_minute_data_provider() != "tickflow"
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
def _gate_reason(self) -> str | None:
|
||||
"""返回本轮不执行的原因 (None = 放行)。"""
|
||||
if not preferences.get_minute_refresh_enabled():
|
||||
return "disabled"
|
||||
if self.custom_provider_active():
|
||||
return "custom_minute_provider"
|
||||
if not self.capability_ok():
|
||||
return "capability"
|
||||
if not _in_continuous_session():
|
||||
return "outside_trading_hours"
|
||||
# 节假日 (工作日但休市): 周几门控覆盖不到, 由交易日探针剔除。
|
||||
# 未知 (None) 维持现状 — 空轮升级机制兜底 (探针失灵时的第二道防线)。
|
||||
from app.services import trading_day
|
||||
|
||||
if trading_day.is_trading_day() is False:
|
||||
return "holiday"
|
||||
return None
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 主循环
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _loop(self) -> None:
|
||||
while not self._stop.is_set():
|
||||
try:
|
||||
reason = self._gate_reason()
|
||||
if reason is None:
|
||||
interval = preferences.get_minute_refresh_interval()
|
||||
started = time.time()
|
||||
self._run_round()
|
||||
# 固定节奏: 下一轮 = max(本轮起点+间隔, 本轮完成), 不补跑
|
||||
finish = time.time()
|
||||
self._state.next_round_at = max(started + interval, finish)
|
||||
# 等到下一轮 (小步睡眠保持可停/偏好热切换)
|
||||
while not self._stop.is_set():
|
||||
now = time.time()
|
||||
gate = self._gate_reason()
|
||||
if gate is not None:
|
||||
self._state.next_round_at = None
|
||||
break # 门控关闭 → 回外层等待重评估
|
||||
if now >= self._state.next_round_at:
|
||||
break
|
||||
self._stop.wait(min(_LOOP_STEP_S, max(0.0, self._state.next_round_at - now)))
|
||||
continue
|
||||
except Exception as e:
|
||||
self._state.last_error = f"round failed: {e}"
|
||||
self._stop.wait(_LOOP_STEP_S)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 单轮
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _today_coverage_lag_minutes(self) -> float | None:
|
||||
"""当日分区最新K距现在的分钟数; None = 当日无数据。
|
||||
|
||||
覆盖度探测只读当日分区文件 (毫秒级); 任何异常按无数据处理 →
|
||||
本轮走全天修复, 不会因探测失败而丢增量。
|
||||
"""
|
||||
with contextlib.suppress(Exception):
|
||||
part = (
|
||||
self._repo.store.data_dir / "kline_minute"
|
||||
/ f"date={cn_today().isoformat()}" / "part.parquet"
|
||||
)
|
||||
if not part.exists():
|
||||
return None
|
||||
mx = pl.read_parquet(part, columns=["datetime"])["datetime"].max()
|
||||
if mx is None:
|
||||
return None
|
||||
# 分区 datetime 为北京墙钟 naive, cn_now 带时区 → 剥齐再比
|
||||
return (cn_now().replace(tzinfo=None) - mx).total_seconds() / 60
|
||||
return None
|
||||
|
||||
def _select_mode(self) -> str:
|
||||
"""选轮次模式: 稳态增量 (universe 单请求) vs 全天修复 (burst 脉冲)。
|
||||
|
||||
当日已有数据且覆盖滞后 ≤ _REPAIR_LAG_MINUTES (≈ 3 根余量) → 增量;
|
||||
冷启动 / 断档超阈值 / 连续空轮 → 全天修复。
|
||||
"""
|
||||
if self._empty_rounds >= _EMPTY_ROUNDS_TO_REPAIR:
|
||||
return "full"
|
||||
lag = self._today_coverage_lag_minutes()
|
||||
if lag is None or lag > _REPAIR_LAG_MINUTES:
|
||||
return "full"
|
||||
return "increment"
|
||||
|
||||
def _run_round(self) -> None:
|
||||
from app.services import kline_sync
|
||||
|
||||
t0 = time.perf_counter()
|
||||
mode = self._select_mode()
|
||||
with self._round_lock:
|
||||
if mode == "increment":
|
||||
df, requests = kline_sync.fetch_intraday_universe_increment()
|
||||
self._state.last_symbols = (
|
||||
df["symbol"].n_unique() if not df.is_empty() else 0
|
||||
)
|
||||
else:
|
||||
symbols = self._universe()
|
||||
self._state.last_symbols = len(symbols)
|
||||
if not symbols:
|
||||
self._state.last_error = "empty universe (instruments 未加载)"
|
||||
return
|
||||
capset = getattr(self._app_state, "capabilities", None) if self._app_state else None
|
||||
df, requests = kline_sync.fetch_intraday_full_market_burst(symbols, capset)
|
||||
self._state.last_requests = requests
|
||||
if df.is_empty():
|
||||
self._empty_rounds += 1
|
||||
self._state.last_error = f"intraday {mode} returned no data"
|
||||
return
|
||||
self._empty_rounds = 0
|
||||
written = kline_sync._write_minute_partition(
|
||||
df, self._repo.store.data_dir / "kline_minute",
|
||||
)
|
||||
|
||||
self._state.rounds += 1
|
||||
self._state.last_round_at = time.time()
|
||||
self._state.last_round_ms = (time.perf_counter() - t0) * 1000
|
||||
self._state.last_rows = written
|
||||
self._state.last_mode = mode
|
||||
self._state.last_error = None
|
||||
|
||||
def _universe(self) -> list[str]:
|
||||
"""全市场 A 股标的 (instruments 维表, 与盘后分钟同步同一来源)。"""
|
||||
inst = self._repo.get_instruments()
|
||||
if inst.is_empty() or "symbol" not in inst.columns:
|
||||
return []
|
||||
return inst["symbol"].cast(pl.Utf8).unique().sort().to_list()
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# 状态
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def status(self) -> dict[str, Any]:
|
||||
import contextlib
|
||||
|
||||
with contextlib.suppress(Exception):
|
||||
enabled = preferences.get_minute_refresh_enabled()
|
||||
running = self._thread is not None and self._thread.is_alive()
|
||||
gate = self._gate_reason()
|
||||
return {
|
||||
"enabled": enabled,
|
||||
"running": running,
|
||||
"interval_seconds": preferences.get_minute_refresh_interval(),
|
||||
"capability_ok": self.capability_ok(),
|
||||
"custom_provider_active": self.custom_provider_active(),
|
||||
"in_trading_hours": _in_continuous_session(),
|
||||
"gate_reason": gate if (enabled and running) else (gate or "disabled"),
|
||||
"rounds": self._state.rounds,
|
||||
"last_round_at": self._state.last_round_at,
|
||||
"last_round_ms": self._state.last_round_ms,
|
||||
"last_rows": self._state.last_rows,
|
||||
"last_symbols": self._state.last_symbols,
|
||||
"last_requests": self._state.last_requests,
|
||||
"last_mode": self._state.last_mode,
|
||||
"next_round_at": self._state.next_round_at,
|
||||
"last_error": self._state.last_error,
|
||||
}
|
||||
|
||||
def trigger_manual_round(self) -> dict[str, Any]:
|
||||
"""手动触发一轮 (无视时段门控, 但仍受能力/插件门控); 供状态页「立即刷新」。"""
|
||||
if self.custom_provider_active() or not self.capability_ok():
|
||||
return {"ok": False, "reason": self._gate_reason() or "capability"}
|
||||
threading.Thread(target=self._run_round, daemon=True, name="minute-refresh-manual").start()
|
||||
return {"ok": True}
|
||||
@@ -2,8 +2,12 @@
|
||||
|
||||
设计:
|
||||
- job_store/ 文件夹,每个 job 一个 {id}.json,最多保留 max_jobs 个文件
|
||||
- running/pending 状态的 job 仅存内存(高频读写)
|
||||
- succeeded/failed 后写入独立文件并从内存释放
|
||||
- create()/start() 即落盘 pending/running 快照(进度只更新内存) ——
|
||||
进程意外死亡(uvicorn --reload 热重载 / 被 kill)时记录不蒸发
|
||||
- succeeded/failed 后写入终态并从内存释放
|
||||
- 实例化时扫描磁盘,把上个进程遗留的 pending/running 孤儿记录补标为
|
||||
failed(中断);finished_at 取文件 mtime(最后已知存活时刻),不用下次
|
||||
开机时间虚增时长
|
||||
- 列表查询 = 内存中的活跃 job + 磁盘文件扫描,按时间排序
|
||||
- 单个查询 = 内存优先,没有则读磁盘
|
||||
- 创建新 job 前检查文件数量,>= max_jobs 时删除最老的文件
|
||||
@@ -15,7 +19,7 @@ import logging
|
||||
import os
|
||||
import threading
|
||||
import uuid
|
||||
from datetime import datetime
|
||||
from datetime import UTC, datetime
|
||||
from pathlib import Path
|
||||
from typing import Any, Literal
|
||||
|
||||
@@ -105,11 +109,12 @@ class JobStore:
|
||||
self._active_id: str | None = None
|
||||
self._lock = threading.Lock()
|
||||
self._store_dir.mkdir(parents=True, exist_ok=True)
|
||||
self._reap_orphans()
|
||||
|
||||
# ===== persistence =====
|
||||
|
||||
def _write_file(self, job: dict[str, Any]) -> None:
|
||||
"""将终态 job 写入独立 JSON 文件。"""
|
||||
"""将 job 快照写入独立 JSON 文件(create/start/终态均落盘)。"""
|
||||
path = self._store_dir / f"{job['id']}.json"
|
||||
try:
|
||||
path.write_text(
|
||||
@@ -154,6 +159,42 @@ class JobStore:
|
||||
jobs.sort(key=lambda j: j.get("started_at") or "", reverse=True)
|
||||
return jobs
|
||||
|
||||
def _reap_orphans(self) -> None:
|
||||
"""启动补录: 把上个进程遗留的 pending/running 记录标为中断。
|
||||
|
||||
单例在进程启动时实例化,此时磁盘上的 pending/running 必然来自已死
|
||||
亡的进程(uvicorn --reload 热重载 / 被 kill)—— 不补录则这些记录
|
||||
永远停留在「运行中」, 同步历史里既看不到结果也看不到失败, 即
|
||||
「数据在、记录丢」。finished_at 取文件 mtime(最后一次落盘 = 最后
|
||||
已知存活时刻), 避免拿下次开机时间虚增 duration。
|
||||
"""
|
||||
for f in self._store_dir.glob("*.json"):
|
||||
try:
|
||||
j = json.loads(f.read_text("utf-8"))
|
||||
except Exception:
|
||||
continue
|
||||
orig_status = j.get("status")
|
||||
if not j.get("id") or orig_status not in ("pending", "running"):
|
||||
continue
|
||||
j["status"] = "failed"
|
||||
j["error"] = "后端重启,任务中断(启动时补录)"
|
||||
try:
|
||||
mtime = datetime.fromtimestamp(f.stat().st_mtime, tz=UTC)
|
||||
end = mtime.strftime("%Y-%m-%dT%H:%M:%SZ")
|
||||
except Exception:
|
||||
end = datetime.utcnow().isoformat(timespec="seconds") + "Z"
|
||||
# mtime 早于 started_at(时钟回拨等)时夹住, 避免 duration 为负
|
||||
if j.get("started_at") and end < j["started_at"]:
|
||||
end = j["started_at"]
|
||||
j["finished_at"] = end
|
||||
j["duration_s"] = _duration_s(j)
|
||||
logger.warning(
|
||||
"job_store: 补录中断任务 %s (上个进程遗留 %s 记录)",
|
||||
j["id"],
|
||||
orig_status,
|
||||
)
|
||||
self._write_file(j)
|
||||
|
||||
# ===== lifecycle =====
|
||||
|
||||
def create(
|
||||
@@ -189,7 +230,7 @@ class JobStore:
|
||||
return self._active_id, False
|
||||
|
||||
job_id = uuid.uuid4().hex[:10]
|
||||
self._active_jobs[job_id] = {
|
||||
job = {
|
||||
"id": job_id,
|
||||
"status": "pending",
|
||||
"stage": "init",
|
||||
@@ -204,7 +245,11 @@ class JobStore:
|
||||
"error": None,
|
||||
"timeout_s": timeout_s,
|
||||
}
|
||||
self._active_jobs[job_id] = job
|
||||
self._active_id = job_id
|
||||
# pending 即落盘: 进程在 start() 前死亡时记录也不丢
|
||||
self._delete_oldest()
|
||||
self._write_file(job)
|
||||
_register_cancel_flag(job_id)
|
||||
return job_id, True
|
||||
|
||||
@@ -218,6 +263,8 @@ class JobStore:
|
||||
# 心跳基准初始化为启动时刻: start() 到首次 progress() 之间的
|
||||
# 初始化阶段(解析标的池等)同样计入停滞计时。
|
||||
j["last_progress_at"] = j["started_at"]
|
||||
# running 快照落盘: 进程死亡后由下次启动的 _reap_orphans 补录
|
||||
self._write_file(j)
|
||||
|
||||
def succeed(self, job_id: str, result: Any) -> None:
|
||||
with self._lock:
|
||||
|
||||
@@ -84,29 +84,6 @@ def get_realtime_quote_interval() -> float:
|
||||
return load().get("realtime_quote_interval", 6.0)
|
||||
|
||||
|
||||
def get_realtime_watchlist_symbols() -> list[str]:
|
||||
"""Free 档自选实时监控标的:直接取自选页前 5 个。"""
|
||||
try:
|
||||
from app.services import watchlist
|
||||
rows = watchlist.list_symbols()
|
||||
except Exception as e: # noqa: BLE001
|
||||
logger.warning("load watchlist for realtime failed: %s", e)
|
||||
return []
|
||||
out: list[str] = []
|
||||
for row in rows:
|
||||
symbol = str((row or {}).get("symbol") or "").strip().upper()
|
||||
if symbol and symbol not in out:
|
||||
out.append(symbol)
|
||||
if len(out) >= 5:
|
||||
break
|
||||
return out
|
||||
|
||||
|
||||
def set_realtime_watchlist_symbols(symbols: list[str]) -> list[str]: # noqa: ARG001
|
||||
"""兼容旧接口: Free 实时标的现在由自选页前 5 个决定。"""
|
||||
return get_realtime_watchlist_symbols()
|
||||
|
||||
|
||||
def set_realtime_quote_interval(interval: float) -> float:
|
||||
"""保存行情轮询间隔（不在此做 min/max 校验，由调用方按档位限制）。"""
|
||||
current = load()
|
||||
@@ -214,6 +191,27 @@ def get_minute_sync_segment_days() -> int:
|
||||
"""
|
||||
return max(5, min(30, load().get("minute_sync_segment_days", 20)))
|
||||
|
||||
# ===== 盘中分钟增量刷新 (Expert 专有) =====
|
||||
|
||||
# 稳态轮为 intraday.universe 单请求增量, 无脉冲并发, 间隔可低至 3s;
|
||||
# 全天修复轮 (intraday.batch 28 块爆发) 的 rpm 安全与间隔无关, 由轮次
|
||||
# 调度 max(间隔, 单轮完成) 天然防重叠。
|
||||
_MINUTE_REFRESH_INTERVAL_MIN = 3
|
||||
_MINUTE_REFRESH_INTERVAL_MAX = 300
|
||||
|
||||
|
||||
def get_minute_refresh_enabled() -> bool:
|
||||
"""盘中分钟K增量落盘开关。默认关闭; 能力门控 (Expert) 在服务层判断。"""
|
||||
return bool(load().get("minute_refresh_enabled", False))
|
||||
|
||||
|
||||
def get_minute_refresh_interval() -> int:
|
||||
"""盘中分钟增量刷新间隔(秒)。默认 6,范围 [3, 300]。"""
|
||||
return max(
|
||||
_MINUTE_REFRESH_INTERVAL_MIN,
|
||||
min(_MINUTE_REFRESH_INTERVAL_MAX, int(load().get("minute_refresh_interval", 6))),
|
||||
)
|
||||
|
||||
|
||||
# ===== 数据源选择 (默认 TickFlow；第一阶段仅日K切换入口) =====
|
||||
|
||||
@@ -260,10 +258,9 @@ def get_daily_data_provider() -> str:
|
||||
|
||||
|
||||
def get_adj_factor_provider() -> str:
|
||||
provider = str(load().get("adj_factor_provider", "same_as_daily") or "same_as_daily").lower()
|
||||
if provider == "same_as_daily":
|
||||
return provider
|
||||
return provider if provider in _allowed_data_providers() else "same_as_daily"
|
||||
# 「跟随日K」(same_as_daily) 特殊值已下线: 存量配置里的旧值按非法值回退 tickflow
|
||||
provider = str(load().get("adj_factor_provider", "tickflow") or "tickflow").lower()
|
||||
return provider if provider in _allowed_data_providers() else "tickflow"
|
||||
|
||||
|
||||
def get_minute_data_provider() -> str:
|
||||
@@ -271,6 +268,11 @@ def get_minute_data_provider() -> str:
|
||||
return provider if provider in _allowed_data_providers() else "tickflow"
|
||||
|
||||
|
||||
def get_depth5_data_provider() -> str:
|
||||
provider = str(load().get("depth5_data_provider", "tickflow") or "tickflow").lower()
|
||||
return provider if provider in _allowed_data_providers() else "tickflow"
|
||||
|
||||
|
||||
def get_realtime_data_provider() -> str:
|
||||
provider = str(load().get("realtime_data_provider", "tickflow") or "tickflow").lower()
|
||||
return provider if provider in _allowed_data_providers() else "tickflow"
|
||||
@@ -921,6 +923,13 @@ def set_realtime_monitor_config(cfg: dict) -> dict:
|
||||
updates["minute_intraday_refresh_interval"] = max(
|
||||
_INTRADAY_REFRESH_INTERVAL_MIN,
|
||||
min(_INTRADAY_REFRESH_INTERVAL_MAX, int(cfg["minute_intraday_refresh_interval"])))
|
||||
if "minute_refresh_enabled" in cfg:
|
||||
updates["minute_refresh_enabled"] = bool(cfg["minute_refresh_enabled"])
|
||||
if "minute_refresh_interval" in cfg:
|
||||
# clamp 到 [3, 300], 与 getter 一致, 防前端传越界值
|
||||
updates["minute_refresh_interval"] = max(
|
||||
_MINUTE_REFRESH_INTERVAL_MIN,
|
||||
min(_MINUTE_REFRESH_INTERVAL_MAX, int(cfg["minute_refresh_interval"])))
|
||||
if "monitor_ext_fields" in cfg:
|
||||
raw = cfg["monitor_ext_fields"] or {}
|
||||
updates["monitor_ext_fields"] = {
|
||||
@@ -942,6 +951,8 @@ def get_realtime_monitor_config() -> dict:
|
||||
"screener_auto_run": get_screener_auto_run(),
|
||||
"minute_intraday_refresh": get_minute_intraday_refresh(),
|
||||
"minute_intraday_refresh_interval": get_minute_intraday_refresh_interval(),
|
||||
"minute_refresh_enabled": get_minute_refresh_enabled(),
|
||||
"minute_refresh_interval": get_minute_refresh_interval(),
|
||||
"monitor_ext_fields": get_monitor_ext_fields(),
|
||||
}
|
||||
|
||||
|
||||
@@ -28,7 +28,7 @@ import threading
|
||||
import time
|
||||
from concurrent.futures import ThreadPoolExecutor
|
||||
from contextlib import contextmanager
|
||||
from datetime import date, time as dt_time
|
||||
from datetime import date, datetime, time as dt_time
|
||||
|
||||
import polars as pl
|
||||
|
||||
@@ -167,13 +167,15 @@ class QuoteService:
|
||||
|
||||
CORE_INDEX_SYMBOLS = ("000001.SH", "399001.SZ", "399006.SZ", "000680.SH")
|
||||
|
||||
# 档位 → 最小轮询间隔 (秒)
|
||||
# 档位 → 最小轮询间隔 (秒) — TickFlow 档位限速保护, 仅实时源为 tickflow 时适用
|
||||
TIER_MIN_INTERVAL = {
|
||||
"expert": 1.0,
|
||||
"pro": 3.0,
|
||||
"starter": 6.0,
|
||||
"free": 6.0,
|
||||
}
|
||||
# 插件/自定义源: 不受 TickFlow 档位保护约束, 通用下限 1s (默认间隔仍为 DEFAULT_INTERVAL)
|
||||
CUSTOM_PROVIDER_MIN_INTERVAL = 1.0
|
||||
DEFAULT_INTERVAL = 6.0
|
||||
MAX_INTERVAL = 60.0
|
||||
|
||||
@@ -218,6 +220,17 @@ class QuoteService:
|
||||
# 午休/收盘最终同步状态: 到边界后必须成功拉取一版行情, 再进入休盘态。
|
||||
self._final_sync_done: set[tuple[date, str]] = set()
|
||||
self._final_sync_failed: dict[tuple[date, str], str] = {}
|
||||
self._holiday_active = False # 交易日探针当前是否判休市 (日志去重)
|
||||
# 轮询放量 (volume_delta 规则): 上一轮全市场股票快照的 (累计成交量[手], 累计成交额[元])。
|
||||
# 每轮全量快照后更新 (含非连续竞价时段, 保证 13:00 恢复时 prev 是 12:59
|
||||
# 而非 11:30); 跨交易日清空; cur < prev (数据源重置) 时丢弃该轮差值。
|
||||
self._prev_stock_volume: dict[str, tuple[float, float]] | None = None
|
||||
self._prev_volume_fetched_at: float | None = None # epoch 毫秒
|
||||
self._prev_volume_date: date | None = None
|
||||
# 最近一轮的有效差值 (vol_delta[手], amt_delta[元]) - 仅连续竞价时段内、
|
||||
# prev 不早于本时段开盘时计算
|
||||
self._volume_delta: dict[str, tuple[float, float]] = {}
|
||||
self._volume_delta_span_s: float = 0.0
|
||||
|
||||
# ================================================================
|
||||
# 生命周期
|
||||
@@ -423,15 +436,18 @@ class QuoteService:
|
||||
|
||||
@classmethod
|
||||
def realtime_mode(cls) -> str:
|
||||
"""当前实时行情模式: none / watchlist / full_market。"""
|
||||
"""当前实时行情模式: none / full_market。
|
||||
|
||||
TickFlow 免费档不再提供"自选前 5 只"降级实时(自定义源 fuyao 的全市场
|
||||
快照已全面覆盖且免费); TickFlow 免费档 = 无实时, 接入自定义实时源
|
||||
(如 fuyao)或升级 TickFlow 后恢复全市场模式。
|
||||
"""
|
||||
from app.services import preferences
|
||||
if preferences.get_realtime_data_provider() != "tickflow":
|
||||
return "full_market"
|
||||
tier = cls._current_tier()
|
||||
if tier == "none":
|
||||
if tier in ("none", "free"):
|
||||
return "none"
|
||||
if tier == "free":
|
||||
return "watchlist"
|
||||
return "full_market"
|
||||
|
||||
@classmethod
|
||||
@@ -441,6 +457,11 @@ class QuoteService:
|
||||
|
||||
@classmethod
|
||||
def _tier_min_interval(cls) -> float:
|
||||
# 实时源路由到插件/自定义源时, TickFlow 档位限速不适用 (中立能力原则):
|
||||
# 下限放宽到通用 1s, 默认/已保存间隔不变
|
||||
from app.services import preferences
|
||||
if preferences.get_realtime_data_provider() != "tickflow":
|
||||
return cls.CUSTOM_PROVIDER_MIN_INTERVAL
|
||||
tier = cls._current_tier()
|
||||
return cls.TIER_MIN_INTERVAL.get(tier, cls.DEFAULT_INTERVAL)
|
||||
|
||||
@@ -494,7 +515,6 @@ class QuoteService:
|
||||
|
||||
def status(self) -> dict:
|
||||
"""返回行情服务状态。"""
|
||||
from app.services import preferences
|
||||
age = (time.perf_counter() - self._fetch_time) * 1000 if self._fetch_time else -1
|
||||
mode = self.realtime_mode()
|
||||
phase = self._market_phase()
|
||||
@@ -507,7 +527,6 @@ class QuoteService:
|
||||
"paused": self._paused,
|
||||
"mode": mode,
|
||||
"realtime_allowed": mode != "none",
|
||||
"watchlist_symbol_count": len(preferences.get_realtime_watchlist_symbols()),
|
||||
"interval_s": self._interval,
|
||||
"symbol_count": self._symbol_count,
|
||||
"index_symbol_count": self._index_symbol_count,
|
||||
@@ -561,14 +580,11 @@ class QuoteService:
|
||||
waited += 0.5
|
||||
|
||||
def _fetch_quotes(self, *, final: bool = False) -> bool:
|
||||
"""按当前档位拉取行情。加锁串行化 (后台轮询 vs 手动 refresh)。返回本轮是否成功更新。"""
|
||||
"""拉取行情。加锁串行化 (后台轮询 vs 手动 refresh)。返回本轮是否成功更新。"""
|
||||
with self._fetch_lock:
|
||||
before = self._fetched_at
|
||||
if final:
|
||||
logger.info("最终行情同步开始")
|
||||
if self.realtime_mode() == "watchlist":
|
||||
self._fetch_watchlist_quotes()
|
||||
else:
|
||||
self._fetch_full_market_quotes()
|
||||
return self._fetched_at > before
|
||||
|
||||
@@ -719,6 +735,9 @@ class QuoteService:
|
||||
_persist_last_fetch(fetched_at)
|
||||
logger.info("行情刷新: %d 只股票, %d 只ETF, %d 只指数, 耗时 %.0fms", len(stock_records), len(etf_records), len(index_records), fetch_ms)
|
||||
|
||||
# 轮询放量状态更新 (volume_delta 规则的差值来源)
|
||||
self._update_volume_delta(stock_records, fetched_at)
|
||||
|
||||
# ---- 写 kline_daily (不复权原始价格, 只有 OHLCV) ----
|
||||
daily_df = self._build_daily(stock_records)
|
||||
if not daily_df.is_empty() and self._repo:
|
||||
@@ -765,150 +784,11 @@ class QuoteService:
|
||||
# ---- 策略监控 + 告警评估 ----
|
||||
self._evaluate_monitors(daily_df, quote_extra)
|
||||
|
||||
def _fetch_watchlist_quotes(self) -> None:
|
||||
"""Free 档自选股实时: 按 capability batch 上限分批拉取。"""
|
||||
from app.services import preferences
|
||||
from app.tickflow.client import get_paid_realtime_client
|
||||
from app.tickflow.capabilities import Cap
|
||||
from app.tickflow.policy import detect_capabilities
|
||||
from app.tickflow.rate_limits import chunked, resolve_limit, sleep_between_batches
|
||||
|
||||
symbols = preferences.get_realtime_watchlist_symbols()
|
||||
# 指数监控规则标的并入轮询 (与股票共享 batch 额度)
|
||||
engine = getattr(self._app_state, "monitor_engine", None) if self._app_state else None
|
||||
if engine:
|
||||
for _r in list(engine.rules.values()):
|
||||
if _r.get("enabled", True) and _r.get("asset_type") == "index" and _r.get("scope") == "symbols":
|
||||
for _s in _r.get("symbols", []):
|
||||
if _s and _s not in symbols:
|
||||
symbols.append(_s)
|
||||
if not symbols:
|
||||
logger.info("自选实时未配置标的, 跳过行情拉取")
|
||||
return
|
||||
|
||||
tf = get_paid_realtime_client()
|
||||
if tf is None:
|
||||
logger.warning("自选实时拉取失败:未配置付费服务器 API Key")
|
||||
return
|
||||
|
||||
# 按 capability batch 上限分批: 股票+指数共享额度, 超过上限会导致整轮失败
|
||||
capset = detect_capabilities()
|
||||
lim = resolve_limit(capset, Cap.QUOTE_BY_SYMBOL, default_batch=5)
|
||||
batches = chunked(symbols, lim.batch)
|
||||
|
||||
t0 = time.perf_counter()
|
||||
now_ts = time.perf_counter()
|
||||
resp = []
|
||||
for i, batch in enumerate(batches):
|
||||
sleep_between_batches(i, lim.rpm)
|
||||
try:
|
||||
resp.extend(tf.quotes.get(symbols=batch) or [])
|
||||
except Exception as e: # noqa: BLE001
|
||||
logger.warning("自选实时批次 %d/%d 拉取失败: %s", i + 1, len(batches), e)
|
||||
|
||||
if not resp:
|
||||
logger.warning("自选实时行情数据为空")
|
||||
return
|
||||
|
||||
records = []
|
||||
for q in resp:
|
||||
ext = q.get("ext") or {}
|
||||
last_price = q.get("last_price")
|
||||
prev_close = q.get("prev_close")
|
||||
change_amount = ext.get("change_amount")
|
||||
change_pct = ext.get("change_pct")
|
||||
if change_amount is None and last_price is not None and prev_close is not None:
|
||||
change_amount = float(last_price) - float(prev_close)
|
||||
if change_pct is None and change_amount is not None and prev_close not in (None, 0):
|
||||
# 小数制, 与 ext.change_pct / enriched 口径一致 (不乘 100)
|
||||
change_pct = float(change_amount) / float(prev_close)
|
||||
records.append({
|
||||
"symbol": q.get("symbol"),
|
||||
"name": q.get("name") or ext.get("name"),
|
||||
"last_price": last_price,
|
||||
"prev_close": prev_close,
|
||||
"open": q.get("open"),
|
||||
"high": q.get("high"),
|
||||
"low": q.get("low"),
|
||||
"volume": q.get("volume"),
|
||||
"amount": q.get("amount"),
|
||||
"change_pct": change_pct,
|
||||
"change_amount": change_amount,
|
||||
"amplitude": ext.get("amplitude"),
|
||||
"turnover_rate": ext.get("turnover_rate"),
|
||||
"timestamp": q.get("timestamp"),
|
||||
"session": q.get("session"),
|
||||
})
|
||||
|
||||
index_set = self._repo.get_index_symbol_set() if self._repo else set()
|
||||
etf_set = self._repo.get_etf_symbol_set() if self._repo else set()
|
||||
index_records, etf_records, stock_records = self._split_records_by_asset(records, index_set, etf_set)
|
||||
|
||||
fetch_ms = (time.perf_counter() - t0) * 1000
|
||||
fetched_at = time.time() * 1000
|
||||
with self._lock:
|
||||
self._fetch_time = now_ts
|
||||
self._fetch_ms = fetch_ms
|
||||
self._fetched_at = fetched_at
|
||||
self._symbol_count = len(stock_records)
|
||||
self._index_symbol_count = len(index_records)
|
||||
self._etf_symbol_count = len(etf_records)
|
||||
self._index_quotes_cache = self._build_index_quotes(index_records) if index_records else None
|
||||
|
||||
_persist_last_fetch(fetched_at)
|
||||
logger.info("自选实时刷新: %d 只股票, %d 只ETF, %d 只指数, 耗时 %.0fms",
|
||||
len(stock_records), len(etf_records), len(index_records), fetch_ms)
|
||||
|
||||
daily_df = self._build_daily(stock_records)
|
||||
quote_extra = self._build_quote_extra(stock_records)
|
||||
if not daily_df.is_empty() and self._repo:
|
||||
try:
|
||||
self._repo.merge_live_daily_asset("stock", daily_df)
|
||||
except Exception as e: # noqa: BLE001
|
||||
logger.warning("自选实时日K写盘失败: %s", e)
|
||||
self._flush_live_enriched(daily_df, quote_extra, asset_type="stock", merge=True)
|
||||
|
||||
# ETF/指数进自选前5时按各自资产落盘, 不污染股票表
|
||||
etf_daily_df = self._build_daily(etf_records)
|
||||
if not etf_daily_df.is_empty() and self._repo:
|
||||
try:
|
||||
self._repo.merge_live_daily_asset("etf", etf_daily_df)
|
||||
except Exception as e: # noqa: BLE001
|
||||
logger.warning("自选实时 ETF 日K写盘失败: %s", e)
|
||||
self._flush_live_enriched(etf_daily_df, self._build_quote_extra(etf_records), asset_type="etf", merge=True)
|
||||
index_daily_df = self._build_daily(index_records)
|
||||
if not index_daily_df.is_empty() and self._repo:
|
||||
try:
|
||||
self._repo.merge_live_daily_asset("index", index_daily_df)
|
||||
except Exception as e: # noqa: BLE001
|
||||
logger.warning("自选实时指数日K写盘失败: %s", e)
|
||||
self._flush_live_enriched(index_daily_df, self._build_quote_extra(index_records), asset_type="index", merge=True)
|
||||
|
||||
self._broadcast_quote_updated()
|
||||
self._evaluate_monitors(daily_df, quote_extra)
|
||||
|
||||
# ================================================================
|
||||
# 工具
|
||||
# ================================================================
|
||||
|
||||
@staticmethod
|
||||
def _split_records_by_asset(
|
||||
records: list[dict], index_set: set[str], etf_set: set[str],
|
||||
) -> tuple[list[dict], list[dict], list[dict]]:
|
||||
"""把行情 records 按资产拆成 (index, etf, stock)。判定顺序与 resolve_asset_type 一致: 先 ETF 后指数。"""
|
||||
index_records: list[dict] = []
|
||||
etf_records: list[dict] = []
|
||||
stock_records: list[dict] = []
|
||||
for r in records:
|
||||
sym = r.get("symbol")
|
||||
if sym in etf_set:
|
||||
etf_records.append(r)
|
||||
elif sym in index_set:
|
||||
index_records.append(r)
|
||||
else:
|
||||
stock_records.append(r)
|
||||
return index_records, etf_records, stock_records
|
||||
|
||||
@staticmethod
|
||||
def _build_daily(records: list[dict]) -> pl.DataFrame:
|
||||
"""将 API records 转为日K格式 DataFrame (OHLCV + quote_ts, 写 kline_daily 用)。"""
|
||||
@@ -1041,8 +921,29 @@ class QuoteService:
|
||||
return (cn_today(), "close")
|
||||
return None
|
||||
|
||||
def _holiday_gate(self) -> bool:
|
||||
"""交易日探针门控: 确定休市 → False (停止轮询, 含 final 定版)。
|
||||
|
||||
探针未知 (None, 未配置 fuyao 且 tickflow 不可用/开盘缓冲窗内) → True,
|
||||
维持周几近似现状行为。探针是纯读, 不落盘; 休市结论带 TTL 定期复探,
|
||||
误判自愈。首次判定变化打一条日志, 避免每拍刷屏。
|
||||
"""
|
||||
from app.services import trading_day
|
||||
|
||||
holiday = trading_day.is_trading_day() is False
|
||||
if holiday != self._holiday_active:
|
||||
self._holiday_active = holiday
|
||||
if holiday:
|
||||
logger.info("交易日探针判定休市, 行情轮询暂停 (30 分钟复探)")
|
||||
return not holiday
|
||||
|
||||
def _should_poll_for_phase(self, phase: str) -> bool:
|
||||
"""是否处于会主动拉行情的阶段。final 阶段成功后即停止。"""
|
||||
"""是否处于会主动拉行情的阶段。final 阶段成功后即停止。
|
||||
|
||||
节假日 (工作日但休市) 由交易日探针剔除 — 周几门控覆盖不到的部分。
|
||||
"""
|
||||
if not self._holiday_gate():
|
||||
return False
|
||||
if phase in {"preopen", "morning", "pre_afternoon", "afternoon"}:
|
||||
return True
|
||||
key = self._final_sync_key(phase)
|
||||
@@ -1117,6 +1018,8 @@ class QuoteService:
|
||||
eval_df = enriched_today
|
||||
if engine.has_rule_type("ladder"):
|
||||
eval_df = self._inject_sealed_vol(enriched_today, enriched_date)
|
||||
if engine.has_rule_type("volume_delta"):
|
||||
eval_df = self._inject_volume_delta(eval_df)
|
||||
eval_df = self._inject_intraday_signals(eval_df, engine, "stock")
|
||||
rule_events = engine.evaluate(eval_df, asset_type="stock")
|
||||
if engine.consume_strategy_result_updates():
|
||||
@@ -1203,7 +1106,8 @@ class QuoteService:
|
||||
"window_change_pct", "coverage_ratio", "valid_count",
|
||||
"total_count", "up_count", "down_count", "leader",
|
||||
"abnormal_window", "abnormal_value", "abnormal_threshold",
|
||||
"abnormal_closeness",
|
||||
"abnormal_closeness", "volume_delta", "volume_delta_span",
|
||||
"volume_delta_amount",
|
||||
):
|
||||
if key in ev:
|
||||
alert[key] = ev[key]
|
||||
@@ -1352,6 +1256,88 @@ class QuoteService:
|
||||
)
|
||||
return self._intraday_signal_evaluator.inject(enriched, signals)
|
||||
|
||||
@staticmethod
|
||||
def _continuous_session_start_ms() -> float:
|
||||
"""当前连续竞价时段的起点 (北京时间 9:30 或 13:00) 的 epoch 毫秒。"""
|
||||
now = cn_now()
|
||||
start_time = dt_time(13, 0) if now.time() >= dt_time(13, 0) else dt_time(9, 30)
|
||||
return datetime.combine(now.date(), start_time, tzinfo=now.tzinfo).timestamp() * 1000.0
|
||||
|
||||
def _update_volume_delta(self, stock_records: list[dict], fetched_at_ms: float) -> None:
|
||||
"""全市场相邻两次快照的股票累计成交量差值 (手), 供 volume_delta 规则。
|
||||
|
||||
- prev 每轮都更新 (含非连续竞价时段); 差值只在连续竞价时段内计算
|
||||
- 开盘保护: prev 早于本时段起点 (9:30/13:00) 时本轮差值无效 -- 避免
|
||||
9:25 集合竞价撮合量 / 午休缺口被当成"突然放量"
|
||||
- cur < prev (数据源重置/口径跳变) 的个股丢弃差值; 跨交易日清空
|
||||
"""
|
||||
today = cn_today()
|
||||
if self._prev_volume_date != today:
|
||||
self._prev_stock_volume = None
|
||||
self._prev_volume_fetched_at = None
|
||||
self._prev_volume_date = today
|
||||
self._volume_delta = {}
|
||||
|
||||
cur: dict[str, tuple[float, float]] = {}
|
||||
for r in stock_records:
|
||||
sym = r.get("symbol")
|
||||
vol = r.get("volume")
|
||||
amt = r.get("amount")
|
||||
if not sym or not isinstance(vol, (int, float)):
|
||||
continue
|
||||
cur[str(sym)] = (
|
||||
float(vol),
|
||||
float(amt) if isinstance(amt, (int, float)) else 0.0,
|
||||
)
|
||||
|
||||
prev = self._prev_stock_volume
|
||||
prev_ts = self._prev_volume_fetched_at
|
||||
if (
|
||||
prev is not None
|
||||
and prev_ts is not None
|
||||
and self._is_continuous_trading()
|
||||
and prev_ts >= self._continuous_session_start_ms()
|
||||
):
|
||||
delta = {
|
||||
sym: (v - prev[sym][0], a - prev[sym][1])
|
||||
for sym, (v, a) in cur.items()
|
||||
if sym in prev and v >= prev[sym][0] and a >= prev[sym][1] and v - prev[sym][0] > 0
|
||||
}
|
||||
self._volume_delta = delta
|
||||
self._volume_delta_span_s = max((fetched_at_ms - prev_ts) / 1000.0, 0.001)
|
||||
else:
|
||||
self._volume_delta = {}
|
||||
|
||||
self._prev_stock_volume = cur
|
||||
self._prev_volume_fetched_at = fetched_at_ms
|
||||
|
||||
def _inject_volume_delta(self, enriched_today: pl.DataFrame) -> pl.DataFrame:
|
||||
"""把最近一轮快照差值作为临时列注入 enriched 副本。
|
||||
|
||||
_volume_delta (手) / _volume_delta_amount (元) / _volume_delta_span (秒, 快照间隔)。
|
||||
无有效差值 (首轮/开盘保护/暂停后恢复) 时返回原 df, 规则安全降级不触发。
|
||||
"""
|
||||
try:
|
||||
delta = self._volume_delta
|
||||
if not delta:
|
||||
return enriched_today
|
||||
span = self._volume_delta_span_s
|
||||
delta_df = pl.DataFrame({
|
||||
"symbol": list(delta.keys()),
|
||||
"_volume_delta": [v for v, _ in delta.values()],
|
||||
"_volume_delta_amount": [a for _, a in delta.values()],
|
||||
"_volume_delta_span": [span] * len(delta),
|
||||
})
|
||||
drop_cols = [
|
||||
c for c in ("_volume_delta", "_volume_delta_amount", "_volume_delta_span")
|
||||
if c in enriched_today.columns
|
||||
]
|
||||
df = enriched_today.drop(drop_cols) if drop_cols else enriched_today
|
||||
return df.join(delta_df, on="symbol", how="left")
|
||||
except Exception as e: # noqa: BLE001
|
||||
logger.debug("快照差值注入失败 (volume_delta 规则将不触发): %s", e)
|
||||
return enriched_today
|
||||
|
||||
def _inject_sealed_vol(self, enriched_today: pl.DataFrame, enriched_date) -> pl.DataFrame:
|
||||
"""从 depth_service 取封单量, 作为临时列 _sealed_vol 注入 enriched 副本。
|
||||
|
||||
@@ -1414,7 +1400,7 @@ class QuoteService:
|
||||
source_labels = {
|
||||
"strategy": "策略", "signal": "信号",
|
||||
"price": "价格", "market": "异动", "ladder": "连板梯队",
|
||||
"sector": "板块",
|
||||
"sector": "板块", "volume_delta": "放量",
|
||||
}
|
||||
rules = engine.rules if engine is not None else {}
|
||||
enqueued = 0
|
||||
|
||||
@@ -386,6 +386,27 @@ class ScreenerService:
|
||||
|
||||
if current is None:
|
||||
current = self._load_enriched_for_date(as_of)
|
||||
if timeframe == "1m":
|
||||
# 分钟策略数据源是本地当日分钟K分区 (单分区文件直读), 与日线
|
||||
# enriched 历史窗口无关, 不走 required_history_bars 日线路径。
|
||||
history = self._load_minute_history(as_of, current)
|
||||
# 策略声明 META["daily_history_bars"] 时额外装配日线 enriched 窗口,
|
||||
# 供分钟策略叠加日线维度条件 (如 N 日内涨停过)。
|
||||
daily_history = None
|
||||
if engine is not None:
|
||||
daily_bars = engine.minute_daily_history_bars(strategy_ids)
|
||||
if daily_bars > 0:
|
||||
daily_history = self._load_enriched_history(as_of, daily_bars)
|
||||
return StrategyDataContext(
|
||||
asset_type=self.asset_type,
|
||||
timeframe=timeframe,
|
||||
as_of=as_of,
|
||||
current=current,
|
||||
history=history,
|
||||
daily_history=daily_history,
|
||||
market=None,
|
||||
cache_key=cache_key,
|
||||
)
|
||||
history_bars = engine.required_history_bars(
|
||||
strategy_ids,
|
||||
params_map=params_map,
|
||||
@@ -404,6 +425,31 @@ class ScreenerService:
|
||||
cache_key=cache_key,
|
||||
)
|
||||
|
||||
def _load_minute_history(self, as_of: date, current: pl.DataFrame | None) -> pl.DataFrame:
|
||||
"""分钟策略数据源: 优先 as_of 当日分钟分区, 缺失时回退全市场最近分区。
|
||||
|
||||
只按日期直读单个分区文件 (get_minute_by_dates), 与全量 glob 扫描解耦,
|
||||
内存只随当日分区大小 (~67万行) 走。标的池限定为 enriched 快照 universe;
|
||||
分区与快照的日期差是允许的 (分钟分区可能比 enriched 更新, 行自带时间戳)。
|
||||
"""
|
||||
if self.asset_type != "stock":
|
||||
raise ValueError("分钟策略当前仅支持 A 股")
|
||||
symbols: list[str] = []
|
||||
if current is not None and not current.is_empty():
|
||||
symbols = current["symbol"].cast(pl.Utf8).unique().to_list()
|
||||
if not symbols:
|
||||
return pl.DataFrame()
|
||||
df = self.repo.get_minute_by_dates(symbols, [as_of])
|
||||
if df.is_empty():
|
||||
fallback = self.repo.latest_minute_date_global()
|
||||
if fallback is None:
|
||||
raise ValueError(
|
||||
"无分钟K数据 — 请先在 数据→分钟K 完成同步, 或开启盘中增量刷新"
|
||||
)
|
||||
if fallback != as_of:
|
||||
df = self.repo.get_minute_by_dates(symbols, [fallback])
|
||||
return df
|
||||
|
||||
def latest_date(self) -> date | None:
|
||||
if self.asset_type != "stock":
|
||||
_, d = self.repo.get_enriched_latest_asset(self.asset_type)
|
||||
|
||||
@@ -0,0 +1,139 @@
|
||||
"""交易日探针 (oracle) — 回答「今天是否 A 股交易日」。
|
||||
|
||||
消费方 (实时行情轮询 / 盘中分钟增量) 在周几+时段门控之后调用, 用于把
|
||||
「工作日但休市」的节假日从轮询窗口里剔除; 返回 None (未知) 时调用方
|
||||
维持现状行为 (周几近似 + 快照新鲜度判据兜底), 不引入新依赖。
|
||||
|
||||
探测链 (按确定性排序, 先到先得):
|
||||
1. fuyao 交易日历 (已配置 fuyao 时): GET /api/a-share/calendar/trading-days,
|
||||
今天在近一年交易日列表内 ⇔ 交易日。权威日历, 无时段依赖, 无开盘缓冲问题。
|
||||
2. tickflow 实时行情时间戳: 拉一篮流动性票快照 (单请求), max(timestamp)
|
||||
日期 == 今天 ⇔ 交易日。非交易日全市场戳停在上一交易日 (2026-08-29 周六
|
||||
实测 5551/5551, 含停牌股 — 戳是快照定版时刻, 非最后成交时刻);
|
||||
交易日集合竞价阶段 (9:15-9:30) 戳是否已翻新未实测 → 开盘缓冲窗内
|
||||
戳过期不作数, 保守视为未知。
|
||||
3. 均不可用 → None: 调用方按周几近似继续。
|
||||
|
||||
安全约束:
|
||||
- 周末直接返回 False (周几判断零成本, 不打任何请求)。
|
||||
- 探针是纯读: 只产出一个布尔判定, 不落盘、不进行情管道、不碰归属链路。
|
||||
- 只用于「降档」(休市不轮询); 休市结论 TTL 较短 (30 分钟) 定期复探,
|
||||
探针误判最坏损失一段快照且可自愈; 未知结论短 TTL (5 分钟) 防止
|
||||
轮询循环每拍重打失败的探测。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import threading
|
||||
import time
|
||||
from dataclasses import dataclass
|
||||
from datetime import datetime, time as dt_time
|
||||
|
||||
from app.market_time import CN_TZ, cn_now
|
||||
|
||||
# tickflow 戳探针的开盘缓冲窗: 此时刻之前戳仍是上一交易日属正常 (集合竞价),
|
||||
# 不据此判休市。周一实测竞价戳翻新时机后可收紧。仅上午首个窗口需要。
|
||||
_STALE_BUFFER_UNTIL = dt_time(9, 40)
|
||||
|
||||
# 一篮流动性票: 探 max(timestamp), 任一戳为今日即交易日 (OR 语义)。
|
||||
# 大盘蓝筹同日全部停牌 = 市场性事件, 与休市同处理无碍。
|
||||
_BASKET = ("000001.SZ", "600519.SH", "600036.SH", "601318.SH", "000651.SZ")
|
||||
|
||||
_TTL_TRADING_S = 3600.0 # 交易日结论每小时复探 (跨日天然失效)
|
||||
_TTL_HOLIDAY_S = 1800.0 # 休市结论 30 分钟复探, 误判自愈上限
|
||||
_TTL_UNKNOWN_S = 300.0 # 未知结论 5 分钟后重试探测
|
||||
|
||||
_CACHE_LOCK = threading.Lock()
|
||||
|
||||
|
||||
@dataclass
|
||||
class _Cache:
|
||||
day: object | None = None
|
||||
verdict: bool | None = None
|
||||
probed_at: float = 0.0
|
||||
|
||||
|
||||
_CACHE = _Cache()
|
||||
|
||||
|
||||
def reset_cache() -> None:
|
||||
"""清空探针缓存 (测试用)。"""
|
||||
with _CACHE_LOCK:
|
||||
_CACHE.day = None
|
||||
_CACHE.verdict = None
|
||||
_CACHE.probed_at = 0.0
|
||||
|
||||
|
||||
def _probe_fuyao(now: datetime) -> bool | None:
|
||||
"""fuyao 交易日历: 今天在列表内 ⇔ 交易日。未配置 fuyao / 失败 → None。"""
|
||||
try:
|
||||
from app.data_providers import custom as custom_sources
|
||||
|
||||
if not custom_sources.is_custom_provider("fuyao"):
|
||||
return None
|
||||
provider = custom_sources.get_provider("fuyao")
|
||||
days = provider.trading_days()
|
||||
return now.date() in days if days else None
|
||||
except Exception: # noqa: BLE001 — 探针失败按未知处理, 不上抛
|
||||
return None
|
||||
|
||||
|
||||
def _probe_tickflow(now: datetime) -> bool | None:
|
||||
"""tickflow 行情时间戳: max(timestamp) 日期 == 今天 ⇔ 交易日。
|
||||
|
||||
戳停在上一交易日: 开盘缓冲窗内 → None (可能是竞价未翻新), 窗后 → False。
|
||||
无实时权限 / 网络失败 / 无有效戳 → None。
|
||||
"""
|
||||
try:
|
||||
from app.tickflow.client import get_client
|
||||
|
||||
rows = get_client().quotes.get(symbols=list(_BASKET)) or []
|
||||
stamps = [r.get("timestamp") for r in rows if isinstance(r, dict)]
|
||||
valid = [int(t) for t in stamps if isinstance(t, (int, float)) and t]
|
||||
if not valid:
|
||||
return None
|
||||
latest_day = datetime.fromtimestamp(max(valid) / 1000, tz=CN_TZ).date()
|
||||
if latest_day == now.date():
|
||||
return True
|
||||
if now.time() < _STALE_BUFFER_UNTIL:
|
||||
return None
|
||||
return False
|
||||
except Exception: # noqa: BLE001 — 无权限/网络失败按未知处理
|
||||
return None
|
||||
|
||||
|
||||
def is_trading_day(now: datetime | None = None) -> bool | None:
|
||||
"""今天是否 A 股交易日。True=交易日, False=确定休市, None=未知 (维持周几近似)。
|
||||
|
||||
周末零成本直判; 工作日走探测链 (fuyao 日历 → tickflow 时间戳),
|
||||
结论按 TTL 缓存。线程安全: 实时行情与分钟增量两个线程共用。
|
||||
"""
|
||||
now = now or cn_now()
|
||||
if now.weekday() >= 5:
|
||||
return False
|
||||
|
||||
with _CACHE_LOCK:
|
||||
if (
|
||||
_CACHE.day == now.date()
|
||||
and _CACHE.verdict is not None
|
||||
and (time.monotonic() - _CACHE.probed_at) < _ttl_of(_CACHE.verdict)
|
||||
):
|
||||
return _CACHE.verdict
|
||||
|
||||
verdict = _probe_fuyao(now)
|
||||
if verdict is None:
|
||||
verdict = _probe_tickflow(now)
|
||||
|
||||
with _CACHE_LOCK:
|
||||
_CACHE.day = now.date()
|
||||
_CACHE.verdict = verdict
|
||||
_CACHE.probed_at = time.monotonic()
|
||||
return verdict
|
||||
|
||||
|
||||
def _ttl_of(verdict: bool | None) -> float:
|
||||
if verdict is True:
|
||||
return _TTL_TRADING_S
|
||||
if verdict is False:
|
||||
return _TTL_HOLIDAY_S
|
||||
return _TTL_UNKNOWN_S
|
||||
@@ -155,6 +155,9 @@ class StrategyDataContext:
|
||||
as_of: date
|
||||
current: pl.DataFrame | None = None
|
||||
history: pl.DataFrame | None = None
|
||||
# 仅 1m 分支: 策略声明 META["daily_history_bars"] 时注入的日线 enriched 窗口,
|
||||
# 供分钟策略叠加日线维度条件 (如 N 日内涨停过); 未声明时为 None。
|
||||
daily_history: pl.DataFrame | None = None
|
||||
market: Any | None = None
|
||||
cache_key: str | None = None
|
||||
|
||||
@@ -199,6 +202,11 @@ class StrategyDef:
|
||||
execution_backend: str = "polars_expr"
|
||||
matrix_strategy: Any | None = None
|
||||
composite: CompositeSpec | None = None # 仅 backend=="composite" 时非空
|
||||
# 仅 backend=="minute_filter" 时非空: 输入为当日分钟K窗口, 输出为命中标的行
|
||||
filter_minute_history_fn: Callable[[pl.DataFrame, dict], pl.DataFrame] | None = None
|
||||
# 仅 minute_filter: META["daily_history_bars"] 声明需要的日线历史窗口 (0=不需要;
|
||||
# >0 时 filter_minute_history 必须接受 daily 关键字, 引擎注入 context.daily_history)
|
||||
minute_daily_bars: int = 0
|
||||
|
||||
|
||||
@dataclass
|
||||
@@ -471,6 +479,7 @@ class StrategyEngine:
|
||||
|
||||
filter_fn = getattr(mod, "filter", None)
|
||||
filter_history_fn = getattr(mod, "filter_history", None)
|
||||
filter_minute_history_fn = getattr(mod, "filter_minute_history", None)
|
||||
execution_backend = str(
|
||||
getattr(
|
||||
mod,
|
||||
@@ -481,7 +490,7 @@ class StrategyEngine:
|
||||
),
|
||||
)
|
||||
)
|
||||
valid_backends = {"polars_expr", "matrix_native", "python_history_legacy", "composite"}
|
||||
valid_backends = {"polars_expr", "matrix_native", "python_history_legacy", "composite", "minute_filter"}
|
||||
if execution_backend not in valid_backends:
|
||||
raise ValueError(
|
||||
f"unsupported execution backend {execution_backend!r}; "
|
||||
@@ -490,6 +499,7 @@ class StrategyEngine:
|
||||
|
||||
matrix_strategy = getattr(mod, "MATRIX_STRATEGY", None)
|
||||
composite_spec: CompositeSpec | None = None
|
||||
minute_daily_bars = 0
|
||||
if execution_backend == "matrix_native":
|
||||
from app.backtest.matrix import MatrixStrategy
|
||||
|
||||
@@ -515,6 +525,39 @@ class StrategyEngine:
|
||||
"composite strategy must not declare filter, filter_history or MATRIX_STRATEGY"
|
||||
)
|
||||
composite_spec = _parse_composite_children(meta.get("children"))
|
||||
elif execution_backend == "minute_filter":
|
||||
# 分钟形态策略: 只声明 filter_minute_history; 数据源是本地当日分钟K分区
|
||||
# (由 ScreenerService.build_strategy_context 的 1m 分支注入), 因此 timeframes
|
||||
# 必须且只能是 ["1m"] — 混入 1d 会让日线 context 走错数据路径。
|
||||
if (
|
||||
filter_minute_history_fn is None
|
||||
or filter_fn is not None
|
||||
or filter_history_fn is not None
|
||||
or matrix_strategy is not None
|
||||
):
|
||||
raise ValueError(
|
||||
"minute_filter strategy must declare only filter_minute_history"
|
||||
)
|
||||
if meta.get("timeframes") != ["1m"]:
|
||||
raise ValueError(
|
||||
"minute_filter strategy must declare timeframes == ['1m']"
|
||||
)
|
||||
# 可选日线历史窗口: 声明 daily_history_bars 时 fn 必须接受 daily 关键字,
|
||||
# 引擎会把 context.daily_history (enriched 日线窗口) 注入进来。
|
||||
minute_daily_bars = int(meta.get("daily_history_bars") or 0)
|
||||
if minute_daily_bars < 0 or minute_daily_bars > 250:
|
||||
raise ValueError(
|
||||
"minute_filter daily_history_bars must be within [0, 250]"
|
||||
)
|
||||
if minute_daily_bars > 0:
|
||||
import inspect
|
||||
|
||||
sig = inspect.signature(filter_minute_history_fn)
|
||||
if "daily" not in sig.parameters:
|
||||
raise ValueError(
|
||||
"minute_filter daily_history_bars requires "
|
||||
"filter_minute_history to accept a 'daily' keyword"
|
||||
)
|
||||
elif filter_history_fn is None or filter_fn is not None:
|
||||
raise ValueError("python_history_legacy strategy must declare only filter_history")
|
||||
|
||||
@@ -538,6 +581,8 @@ class StrategyEngine:
|
||||
execution_backend=execution_backend,
|
||||
matrix_strategy=matrix_strategy,
|
||||
composite=composite_spec,
|
||||
filter_minute_history_fn=filter_minute_history_fn,
|
||||
minute_daily_bars=minute_daily_bars,
|
||||
)
|
||||
|
||||
def reload(self) -> None:
|
||||
@@ -652,6 +697,16 @@ class StrategyEngine:
|
||||
return None
|
||||
return max(0, int(value))
|
||||
|
||||
def minute_daily_history_bars(self, strategy_ids: list[str]) -> int:
|
||||
"""1m 分支需要的日线 enriched 窗口大小: 各 minute_filter 策略声明的
|
||||
META["daily_history_bars"] 取 max, 未声明 (纯分钟策略) 为 0。"""
|
||||
required = 0
|
||||
for strategy_id in strategy_ids:
|
||||
strategy = self.get(strategy_id)
|
||||
if strategy.execution_backend == "minute_filter":
|
||||
required = max(required, strategy.minute_daily_bars)
|
||||
return required
|
||||
|
||||
def required_history_bars(
|
||||
self,
|
||||
strategy_ids: list[str],
|
||||
@@ -885,7 +940,26 @@ class StrategyEngine:
|
||||
exit_signal_hits = self._collect_signal_hits(signal_df, exit_signals)
|
||||
|
||||
# 普通策略只读目标日期；历史策略读取调用方注入的历史窗口。
|
||||
if s.filter_history_fn:
|
||||
if s.execution_backend == "minute_filter":
|
||||
# 分钟策略: 读取调用方注入的当日分钟K窗口。无 date 列, 不按 as_of 过滤,
|
||||
# 每个命中行自带最后K线时间戳 (last_datetime)。
|
||||
if history is None:
|
||||
raise ValueError(f"strategy {strategy_id} requires minute history data")
|
||||
if history.is_empty():
|
||||
return StrategyResult(
|
||||
as_of=as_of,
|
||||
strategy_id=strategy_id,
|
||||
exit_signal_hits=exit_signal_hits,
|
||||
)
|
||||
if s.minute_daily_bars > 0:
|
||||
df = s.filter_minute_history_fn(history, params, daily=context.daily_history)
|
||||
else:
|
||||
df = s.filter_minute_history_fn(history, params)
|
||||
# 基础过滤/展示列 (name/total_shares/change_pct 等) 来自 enriched 快照,
|
||||
# 在命中结果上事后联表, 避免把 enriched 列铺到全市场分钟行上。
|
||||
if current is not None and not current.is_empty():
|
||||
df = self._join_basic_columns(df, current)
|
||||
elif s.filter_history_fn:
|
||||
if history is None:
|
||||
raise ValueError(f"strategy {strategy_id} requires history data")
|
||||
df = history
|
||||
@@ -945,7 +1019,9 @@ class StrategyEngine:
|
||||
# Stage 3: 评分
|
||||
df = self._apply_scoring(df, scoring, scoring_directions)
|
||||
entry_signal_hits = self._collect_signal_hits(df, entry_signals)
|
||||
if not entry_signals and (s.filter_history_fn or s.filter_fn):
|
||||
if not entry_signals and (
|
||||
s.filter_history_fn or s.filter_fn or s.execution_backend == "minute_filter"
|
||||
):
|
||||
entry_signal_hits = [
|
||||
{"symbol": str(symbol), "signals": []}
|
||||
for symbol in df["symbol"].cast(pl.Utf8).unique().to_list()
|
||||
@@ -1036,7 +1112,8 @@ class StrategyEngine:
|
||||
history_strats = [
|
||||
(sid, strategy)
|
||||
for sid, strategy in selected
|
||||
if strategy.filter_history_fn or strategy.execution_backend == "matrix_native"
|
||||
if strategy.filter_history_fn
|
||||
or strategy.execution_backend in ("matrix_native", "minute_filter")
|
||||
]
|
||||
shared_history = context.history
|
||||
if history_strats and shared_history is None:
|
||||
@@ -1466,6 +1543,25 @@ class StrategyEngine:
|
||||
return df.filter(expr)
|
||||
return df
|
||||
|
||||
# 分钟策略命中行需要从事后联表补齐的 enriched 列: 基础过滤引用 + 前端展示。
|
||||
# close 不在列 — 分钟策略输出的 close 是最后一根分钟K收盘价, 优先于日线快照。
|
||||
MINUTE_JOIN_COLUMNS: tuple[str, ...] = (
|
||||
"name", "total_shares", "float_shares", "amount",
|
||||
"turnover_rate", "change_pct", "pre_close",
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _join_basic_columns(df: pl.DataFrame, current: pl.DataFrame) -> pl.DataFrame:
|
||||
"""把 enriched 快照列按 symbol 联到分钟策略输出上, 只补 df 缺失的列。"""
|
||||
cols = [
|
||||
c for c in StrategyEngine.MINUTE_JOIN_COLUMNS
|
||||
if c in current.columns and c not in df.columns
|
||||
]
|
||||
if not cols:
|
||||
return df
|
||||
extra = current.select(["symbol", *cols]).unique(subset=["symbol"], keep="last")
|
||||
return df.join(extra, on="symbol", how="left")
|
||||
|
||||
# ================================================================
|
||||
# 内部: 评分
|
||||
# ================================================================
|
||||
|
||||
@@ -47,6 +47,7 @@ _SIGNAL_CN: dict[str, str] = {
|
||||
"close": "收盘价", "open": "开盘价", "high": "最高价", "low": "最低价",
|
||||
"change_pct": "涨跌幅", "change_amount": "涨跌额", "amplitude": "振幅",
|
||||
"turnover_rate": "换手率", "volume": "成交量", "amount": "成交额",
|
||||
"_volume_delta": "轮询成交量差值(手)", "_sealed_vol": "封单量(手)",
|
||||
# 均线
|
||||
"ma5": "MA5", "ma10": "MA10", "ma20": "MA20", "ma30": "MA30", "ma60": "MA60",
|
||||
"ema5": "EMA5", "ema10": "EMA10", "ema20": "EMA20",
|
||||
@@ -983,6 +984,9 @@ class MonitorRuleEngine:
|
||||
elif rtype == "ladder":
|
||||
# 连板梯队封单监控: 独立处理 (需带预警封单值, 走专属 message)
|
||||
return self._evaluate_ladder(scoped, rule, now)
|
||||
elif rtype == "volume_delta":
|
||||
# 轮询放量监控: 相邻两次全市场快照的成交量差值, 独立处理走专属 message
|
||||
return self._evaluate_volume_delta(scoped, rule, now)
|
||||
else:
|
||||
# signal / price / market: 通用条件匹配
|
||||
for sym, name, price, pct, hit_sigs in self._match_conditions(scoped, rule):
|
||||
@@ -1364,6 +1368,140 @@ class MonitorRuleEngine:
|
||||
results.append((sym, name, price, pct, hit_sigs))
|
||||
return results
|
||||
|
||||
@staticmethod
|
||||
def _volume_delta_basic_mask(df: pl.DataFrame, bf: dict, name_map: dict[str, str]) -> pl.Expr | None:
|
||||
"""轮询放量基础过滤掩码 (与策略 basic_filter 语义对齐, 字段缺失时该项跳过)。
|
||||
|
||||
支持: price_min/max (收盘价), market_cap_min (总市值=close x total_shares),
|
||||
float_cap_min/max (流通市值), amount_min (当日累计成交额), exclude_st (名称含 ST)。
|
||||
"""
|
||||
masks: list[pl.Expr] = []
|
||||
if bf.get("price_min") is not None:
|
||||
masks.append(pl.col("close") >= float(bf["price_min"]))
|
||||
if bf.get("price_max") is not None:
|
||||
masks.append(pl.col("close") <= float(bf["price_max"]))
|
||||
if bf.get("amount_min") is not None and "amount" in df.columns:
|
||||
masks.append(pl.col("amount") >= float(bf["amount_min"]))
|
||||
if bf.get("market_cap_min") is not None and "total_shares" in df.columns:
|
||||
masks.append((pl.col("close") * pl.col("total_shares")) >= float(bf["market_cap_min"]))
|
||||
if bf.get("float_cap_min") is not None and "float_shares" in df.columns:
|
||||
masks.append((pl.col("close") * pl.col("float_shares")) >= float(bf["float_cap_min"]))
|
||||
if bf.get("float_cap_max") is not None and "float_shares" in df.columns:
|
||||
masks.append((pl.col("close") * pl.col("float_shares")) <= float(bf["float_cap_max"]))
|
||||
if bf.get("exclude_st") and name_map:
|
||||
st_symbols = [
|
||||
sym for sym, name in name_map.items()
|
||||
if name and "ST" in str(name).upper()
|
||||
]
|
||||
if st_symbols:
|
||||
masks.append(~pl.col("symbol").is_in(st_symbols))
|
||||
if not masks:
|
||||
return None
|
||||
return pl.all_horizontal(masks)
|
||||
|
||||
def _evaluate_volume_delta(self, scoped: pl.DataFrame, rule: dict, now: float) -> list[dict]:
|
||||
"""评估轮询放量监控: 相邻两次全市场快照的成交量/成交额差值。
|
||||
|
||||
差值列 _volume_delta(手)/_volume_delta_amount(元)/间隔列 _volume_delta_span
|
||||
由 quote_service 评估前注入。metric=volume 按手数、amount 按金额比较阈值;
|
||||
basic_filter 先行过滤 (股价/市值/成交额/ST, 与策略 basic_filter 语义对齐)。
|
||||
命中 >5 只时合并为一条批量事件防刷屏。
|
||||
"""
|
||||
if "_volume_delta" not in scoped.columns:
|
||||
return [] # 无差值数据 (首轮/开盘保护/非全市场轮询), 安全降级
|
||||
|
||||
metric = rule.get("metric", "volume")
|
||||
if metric == "amount" and "_volume_delta_amount" in scoped.columns:
|
||||
cmp_col, threshold = "_volume_delta_amount", rule.get("threshold_amount", 1e6)
|
||||
th_text = f"{threshold / 1e4:,.0f} 万元"
|
||||
else:
|
||||
cmp_col, threshold = "_volume_delta", rule.get("threshold_volume", 9000)
|
||||
th_text = f"{threshold:,.0f} 手"
|
||||
|
||||
cooldown = rule.get("cooldown_seconds", 300)
|
||||
severity = rule.get("severity", "warn")
|
||||
span_s = 0.0
|
||||
if "_volume_delta_span" in scoped.columns and scoped.height > 0:
|
||||
v = scoped["_volume_delta_span"][0]
|
||||
span_s = float(v) if v is not None else 0.0
|
||||
span_text = f" (间隔 {span_s:.0f}s)" if span_s > 0 else ""
|
||||
|
||||
candidate = scoped
|
||||
bf = rule.get("basic_filter") or {}
|
||||
if bf:
|
||||
mask = self._volume_delta_basic_mask(candidate, bf, self._name_map)
|
||||
if mask is not None:
|
||||
candidate = candidate.filter(mask)
|
||||
|
||||
hit = candidate.filter(
|
||||
pl.col(cmp_col).is_not_null() & (pl.col(cmp_col) >= threshold)
|
||||
).sort(cmp_col, descending=True)
|
||||
if hit.is_empty():
|
||||
return []
|
||||
hit_rows = list(hit.iter_rows(named=True))
|
||||
|
||||
def _name_of(row: dict) -> str:
|
||||
sym = row.get("symbol", "")
|
||||
return row.get("name") or self._name_map.get(sym) or sym
|
||||
|
||||
def _fmt(v) -> str:
|
||||
if metric == "amount":
|
||||
return f"{v / 1e4:,.0f} 万元"
|
||||
return f"{v:,.0f} 手"
|
||||
|
||||
def _event(symbol: str, name: str, message: str, *, delta=None, price=None, pct=None) -> dict:
|
||||
ev = {
|
||||
"ts": int(now * 1000),
|
||||
"rule_id": rule["id"],
|
||||
"rule_name": rule.get("name", ""),
|
||||
"source": "volume_delta",
|
||||
"type": "轮询放量",
|
||||
"symbol": symbol,
|
||||
"name": name,
|
||||
"message": message,
|
||||
"price": price,
|
||||
"change_pct": pct,
|
||||
"signals": [],
|
||||
"severity": severity,
|
||||
"conditions": [],
|
||||
"logic": "and",
|
||||
"volume_delta": delta,
|
||||
"volume_delta_span": round(span_s, 1),
|
||||
}
|
||||
if metric == "amount":
|
||||
ev["volume_delta_amount"] = delta
|
||||
return ev
|
||||
|
||||
if len(hit_rows) > 5:
|
||||
top = "、".join(_name_of(r) for r in hit_rows[:8])
|
||||
suffix = "等" if len(hit_rows) > 8 else ""
|
||||
message = (
|
||||
f"放量 · 单轮增量 >= {th_text}{span_text} · "
|
||||
f"共 {len(hit_rows)} 只: {top}{suffix}"
|
||||
)
|
||||
key = (rule["id"], "_volume_delta_batch", "volume_delta")
|
||||
last = self._last_fire.get(key)
|
||||
if last is not None and (now - last) < cooldown:
|
||||
return []
|
||||
self._last_fire[key] = now
|
||||
return [_event("", "", message)]
|
||||
|
||||
events: list[dict] = []
|
||||
for row in hit_rows:
|
||||
sym = row.get("symbol", "")
|
||||
key = (rule["id"], sym, "volume_delta")
|
||||
last = self._last_fire.get(key)
|
||||
if last is not None and (now - last) < cooldown:
|
||||
continue
|
||||
self._last_fire[key] = now
|
||||
delta = row.get(cmp_col)
|
||||
message = f"放量 · 单轮增量 {_fmt(delta)} >= {th_text}{span_text}"
|
||||
events.append(_event(
|
||||
sym, _name_of(row), message,
|
||||
delta=delta, price=row.get("close"), pct=row.get("change_pct"),
|
||||
))
|
||||
return events
|
||||
|
||||
def _evaluate_ladder(self, scoped: pl.DataFrame, rule: dict, now: float) -> list[dict]:
|
||||
"""评估连板梯队封单监控规则。
|
||||
|
||||
|
||||
@@ -28,7 +28,7 @@ logger = logging.getLogger(__name__)
|
||||
|
||||
# ── 常量 ────────────────────────────────────────────────
|
||||
ID_RE = re.compile(r"^[a-z0-9_]{1,40}$")
|
||||
RULE_TYPES = {"strategy", "signal", "price", "market", "ladder", "sector", "abnormal"}
|
||||
RULE_TYPES = {"strategy", "signal", "price", "market", "ladder", "sector", "abnormal", "volume_delta"}
|
||||
SCOPES = {"symbols", "all", "sector", "watchlist_group"}
|
||||
LOGICS = {"and", "or"}
|
||||
DIRECTIONS = {"entry", "exit", "both"}
|
||||
@@ -45,6 +45,19 @@ SECTOR_WINDOWS = {1, 3, 5, 10, 15}
|
||||
# abnormal 规则 (异动边缘): 接近度方向 / 关注窗口
|
||||
ABNORMAL_DIRECTIONS = {"up", "down", "both"}
|
||||
ABNORMAL_WINDOWS = {"any", "3d", "10d", "30d"}
|
||||
# volume_delta 规则 (轮询放量): 阈值口径 (手数 / 成交额)
|
||||
VD_METRICS = {"volume", "amount"}
|
||||
# volume_delta 基础过滤默认值 (与策略 DEFAULT_BASIC_FILTER 核心子集对齐:
|
||||
# 价格 3-300 元, 总市值 >=10 亿, 当日成交额 >=2000 万, 剔除 ST)
|
||||
VD_BASIC_FILTER_DEFAULTS: dict = {
|
||||
"price_min": 3,
|
||||
"price_max": 300,
|
||||
"market_cap_min": 10e8,
|
||||
"float_cap_min": None,
|
||||
"float_cap_max": None,
|
||||
"amount_min": 0.2e8,
|
||||
"exclude_st": True,
|
||||
}
|
||||
|
||||
# 布尔信号列前缀 (op=truth 时 field 取这些)
|
||||
_SIGNAL_PREFIXES = ("signal_", "csg_")
|
||||
@@ -190,6 +203,39 @@ def validate(rule: dict) -> None:
|
||||
threshold_pct = rule.get("threshold_pct")
|
||||
if not isinstance(threshold_pct, (int, float)) or not 1 <= threshold_pct <= 150:
|
||||
raise ValueError("异动接近度阈值必须是 1 到 150 之间的百分比数字")
|
||||
elif rule.get("type") == "volume_delta":
|
||||
# 轮询放量监控: 相邻两次全市场快照的成交量/成交额差值, 不用 conditions
|
||||
if rule.get("asset_type", "stock") != "stock":
|
||||
raise ValueError("轮询放量监控仅支持个股 (依赖全市场股票快照)")
|
||||
if rule.get("scope", "all") == "sector":
|
||||
raise ValueError("轮询放量监控不支持板块作用域")
|
||||
if rule.get("metric", "volume") not in VD_METRICS:
|
||||
raise ValueError(f"metric 必须是 {VD_METRICS} 之一 (volume=手数, amount=金额)")
|
||||
if rule.get("metric", "volume") == "amount":
|
||||
thr = rule.get("threshold_amount")
|
||||
if isinstance(thr, bool) or not isinstance(thr, (int, float)) or not math.isfinite(thr) or thr < 1:
|
||||
raise ValueError("threshold_amount 必须是 >=1 的数字 (单轮成交额增量, 单位元)")
|
||||
else:
|
||||
thr = rule.get("threshold_volume")
|
||||
if isinstance(thr, bool) or not isinstance(thr, (int, float)) or not math.isfinite(thr) or thr < 1:
|
||||
raise ValueError("threshold_volume 必须是 >=1 的数字 (单轮成交量增量, 单位手)")
|
||||
bf = rule.get("basic_filter")
|
||||
if bf is not None:
|
||||
if not isinstance(bf, dict):
|
||||
raise ValueError("basic_filter 必须是对象")
|
||||
for key, value in bf.items():
|
||||
if key == "exclude_st":
|
||||
if not isinstance(value, bool):
|
||||
raise ValueError("basic_filter.exclude_st 必须是布尔值")
|
||||
elif key in ("price_min", "price_max", "market_cap_min", "float_cap_min",
|
||||
"float_cap_max", "amount_min"):
|
||||
if value is not None and (
|
||||
isinstance(value, bool) or not isinstance(value, (int, float))
|
||||
or not math.isfinite(value) or value <= 0
|
||||
):
|
||||
raise ValueError(f"basic_filter.{key} 必须是正数字或 null")
|
||||
else:
|
||||
raise ValueError(f"basic_filter 不支持字段: {key}")
|
||||
else:
|
||||
# 信号/价格/市场类型: 需要 conditions
|
||||
conds = rule.get("conditions")
|
||||
@@ -254,7 +300,7 @@ def normalize(rule: dict) -> dict:
|
||||
r.setdefault("enabled", True)
|
||||
r.setdefault("asset_type", "stock")
|
||||
# sector/abnormal 默认全市场 (sector 随后强制 all; abnormal 支持指定标的)
|
||||
r.setdefault("scope", "all" if r.get("type") in {"sector", "abnormal"} else "symbols")
|
||||
r.setdefault("scope", "all" if r.get("type") in {"sector", "abnormal", "volume_delta"} else "symbols")
|
||||
r.setdefault("symbols", [])
|
||||
r.setdefault("group_id", None)
|
||||
# watchlist_group 作用域: 成员动态来自分组, symbols 不参与; 其他作用域清掉残留 group_id
|
||||
@@ -290,6 +336,15 @@ def normalize(rule: dict) -> dict:
|
||||
# ladder 专属默认字段
|
||||
r.setdefault("metric", "sealed_vol")
|
||||
r.setdefault("threshold", 0)
|
||||
# volume_delta 专属默认字段 (轮询放量): 冷却期默认 300s 而非 3600s --
|
||||
# 持续放量会连续多轮达标, 1 小时只提醒一次太迟钝。
|
||||
if r.get("type") == "volume_delta":
|
||||
if r.get("cooldown_seconds") is None:
|
||||
r["cooldown_seconds"] = 300
|
||||
r["metric"] = r["metric"] if r.get("metric") in VD_METRICS else "volume"
|
||||
r.setdefault("threshold_volume", 9000)
|
||||
r.setdefault("threshold_amount", 1e6)
|
||||
r["basic_filter"] = {**VD_BASIC_FILTER_DEFAULTS, **(r.get("basic_filter") or {})}
|
||||
if r.get("type") == "sector":
|
||||
r["scope"] = "all"
|
||||
r["symbols"] = []
|
||||
|
||||
@@ -20,6 +20,7 @@ class Cap(StrEnum):
|
||||
KLINE_MINUTE_BATCH = "kline.minute.batch"
|
||||
INTRADAY = "intraday"
|
||||
INTRADAY_BATCH = "intraday.batch"
|
||||
INTRADAY_UNIVERSE = "intraday.universe"
|
||||
DEPTH5 = "depth5"
|
||||
DEPTH5_BATCH = "depth5.batch"
|
||||
WEBSOCKET = "websocket"
|
||||
|
||||
@@ -32,7 +32,8 @@ _CAPSET_CACHE_FILE = "capabilities.json"
|
||||
# v2: 拆分 depth5 → depth5(单只) + depth5.batch(批量)
|
||||
# v3: 探测补全 quote.batch(此前 tiers.yaml 声明了但 _probe_real 漏探测)
|
||||
# v5: Free 档补充付费服务器 quote.by_symbol(10rpm/5标的),用于自选股实时监控。
|
||||
_CACHE_SCHEMA_VERSION = 5
|
||||
# v6: 新增 intraday.universe(全量分钟) 探测。
|
||||
_CACHE_SCHEMA_VERSION = 6
|
||||
|
||||
# 探测用最小代价请求:挑流通性最好的 1 只标的试
|
||||
_PROBE_SYMBOL = "600000.SH" # 浦发银行,长期不会退市
|
||||
@@ -234,6 +235,11 @@ def _probe_real(tiers: dict) -> tuple[CapabilitySet, list[str], set[Cap]]:
|
||||
lambda: tf.klines.intraday_batch([_PROBE_SYMBOL], count=1, as_dataframe=False),
|
||||
defaults(Cap.INTRADAY_BATCH))
|
||||
|
||||
# intraday.universe — 全量分钟: 标的池单请求拉全市场最新 N 根 (Expert)
|
||||
try_call(Cap.INTRADAY_UNIVERSE,
|
||||
lambda: tf.klines.intraday_universe("CN_Equity_A", count=1, as_dataframe=False),
|
||||
defaults(Cap.INTRADAY_UNIVERSE))
|
||||
|
||||
# depth5 — 按标的查(单只)
|
||||
try_call(Cap.DEPTH5,
|
||||
lambda: tf.depth.get(_PROBE_SYMBOL),
|
||||
@@ -316,8 +322,6 @@ def _augment_custom_sources(capset: CapabilitySet) -> None:
|
||||
|
||||
daily_provider = preferences.get_daily_data_provider()
|
||||
adj_provider = preferences.get_adj_factor_provider()
|
||||
if adj_provider == "same_as_daily":
|
||||
adj_provider = daily_provider
|
||||
active_providers = {
|
||||
"daily": daily_provider,
|
||||
"adj_factor": adj_provider,
|
||||
@@ -466,6 +470,7 @@ _CAP_ALIASES: dict[Cap, str] = {
|
||||
Cap.KLINE_MINUTE_BY_SYMBOL: "分钟K",
|
||||
Cap.INTRADAY: "分时",
|
||||
Cap.INTRADAY_BATCH: "批量分时",
|
||||
Cap.INTRADAY_UNIVERSE: "全量分钟",
|
||||
Cap.DEPTH5: "五档",
|
||||
Cap.DEPTH5_BATCH: "批量五档",
|
||||
Cap.WEBSOCKET: "WS",
|
||||
|
||||
@@ -38,6 +38,33 @@ from app.parquet import scan_enriched_parquet
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def replace_with_retry(src: Path, dst: Path, *, attempts: int = 10, delay_s: float = 0.5) -> None:
|
||||
"""os.replace 的 Windows 读锁重试版。
|
||||
|
||||
分区 parquet 的读端 (polars scan_parquet / DuckDB read_parquet 视图) 在扫描进行
|
||||
期间持有句柄; Windows 不允许替换"仍被读端打开"的目标文件 (PermissionError,
|
||||
WinError 5), Linux 的 inode 交换语义则无此限制。读端扫描通常亚秒级完成,
|
||||
短退避重试即可穿过并发读窗口; attempts 次仍被占用则原样抛出, 由上层记录失败。
|
||||
"""
|
||||
last: PermissionError | None = None
|
||||
for i in range(attempts):
|
||||
try:
|
||||
src.replace(dst)
|
||||
if i:
|
||||
logger.info("parquet replace succeeded after %d blocked attempt(s): %s", i, dst)
|
||||
return
|
||||
except PermissionError as e:
|
||||
last = e
|
||||
if i == 0:
|
||||
logger.warning(
|
||||
"parquet replace blocked by concurrent reader, retrying (total <= %.1fs): %s",
|
||||
attempts * delay_s, dst,
|
||||
)
|
||||
if i < attempts - 1:
|
||||
time.sleep(delay_s)
|
||||
raise last # type: ignore[misc] # attempts >= 1 时 last 必已赋值
|
||||
|
||||
|
||||
def enriched_dirname(asset_type: str) -> str:
|
||||
"""asset_type → enriched parquet 目录名。ETF 走独立目录, 其余(stock)用日K enriched。"""
|
||||
return "kline_etf_enriched" if asset_type == "etf" else "kline_daily_enriched"
|
||||
@@ -1813,13 +1840,15 @@ class KlineRepository:
|
||||
# ================================================================
|
||||
|
||||
def latest_minute_date(self, symbol: str, asset_type: str = "stock") -> date | None:
|
||||
# 注意: 必须走 execute_one (cursor+close)。直连 self.db.execute(...).fetchone()
|
||||
# 的未消费结果集会把首个分区 parquet 的句柄钉在共享连接上, Windows 下阻塞
|
||||
# 同步写入的 os.replace → 个股分时"补齐数据"500。
|
||||
table = "kline_etf_minute" if asset_type == "etf" else "kline_minute"
|
||||
try:
|
||||
with self._lock:
|
||||
row = self.db.execute(
|
||||
row = self.execute_one(
|
||||
f"SELECT max(CAST(datetime AS DATE)) FROM {table} WHERE symbol = ?",
|
||||
[symbol],
|
||||
).fetchone()
|
||||
)
|
||||
if row and row[0]:
|
||||
return row[0] if isinstance(row[0], date) else date.fromisoformat(str(row[0]))
|
||||
except duckdb.CatalogException:
|
||||
@@ -1829,10 +1858,9 @@ class KlineRepository:
|
||||
def latest_minute_date_global(self) -> date | None:
|
||||
"""全市场最近分钟K日期 (不分 symbol)。用于非交易日回退到上一交易日。"""
|
||||
try:
|
||||
with self._lock:
|
||||
row = self.db.execute(
|
||||
row = self.execute_one(
|
||||
"SELECT max(CAST(datetime AS DATE)) FROM kline_minute",
|
||||
).fetchone()
|
||||
)
|
||||
if row and row[0]:
|
||||
return row[0] if isinstance(row[0], date) else date.fromisoformat(str(row[0]))
|
||||
except Exception: # noqa: BLE001
|
||||
@@ -1841,10 +1869,9 @@ class KlineRepository:
|
||||
def earliest_daily_date(self) -> date | None:
|
||||
"""本地日K数据的最早日期。"""
|
||||
try:
|
||||
with self._lock:
|
||||
res = self.db.execute(
|
||||
res = self.execute_one(
|
||||
"SELECT min(date) FROM kline_daily",
|
||||
).fetchone()
|
||||
)
|
||||
if res and res[0]:
|
||||
d = res[0]
|
||||
return d if isinstance(d, date) else date.fromisoformat(str(d))
|
||||
@@ -1855,10 +1882,9 @@ class KlineRepository:
|
||||
def earliest_minute_date(self) -> date | None:
|
||||
"""本地分钟K数据的最早日期。"""
|
||||
try:
|
||||
with self._lock:
|
||||
res = self.db.execute(
|
||||
res = self.execute_one(
|
||||
"SELECT min(CAST(datetime AS DATE)) FROM kline_minute",
|
||||
).fetchone()
|
||||
)
|
||||
if res and res[0]:
|
||||
d = res[0]
|
||||
return d if isinstance(d, date) else date.fromisoformat(str(d))
|
||||
@@ -1866,13 +1892,35 @@ class KlineRepository:
|
||||
return None
|
||||
return None
|
||||
|
||||
def list_minute_dates(self, start: date, end: date, asset_type: str = "stock") -> list[date]:
|
||||
"""枚举 [start, end] 内存在的分钟K分区日 (目录名直读, 零 parquet 扫描)。
|
||||
|
||||
分钟回测按交易日精确对日: 缺分区的日子由调用方显式跳过,
|
||||
不做"回退最近分区" (那是实盘选股的语义, 回放会串日)。
|
||||
"""
|
||||
dirname = "kline_minute" if asset_type == "stock" else f"kline_{asset_type}_minute"
|
||||
minute_dir = self.store.data_dir / dirname
|
||||
if not minute_dir.exists():
|
||||
return []
|
||||
out: list[date] = []
|
||||
for entry in minute_dir.iterdir():
|
||||
if not (entry.is_dir() and entry.name.startswith("date=")):
|
||||
continue
|
||||
try:
|
||||
day = date.fromisoformat(entry.name[5:])
|
||||
except ValueError:
|
||||
continue
|
||||
if start <= day <= end:
|
||||
out.append(day)
|
||||
out.sort()
|
||||
return out
|
||||
|
||||
def latest_daily_date(self) -> date | None:
|
||||
"""本地日K数据的最新日期。"""
|
||||
try:
|
||||
with self._lock:
|
||||
res = self.db.execute(
|
||||
res = self.execute_one(
|
||||
"SELECT max(date) FROM kline_daily",
|
||||
).fetchone()
|
||||
)
|
||||
if res and res[0]:
|
||||
d = res[0]
|
||||
return d if isinstance(d, date) else date.fromisoformat(str(d))
|
||||
@@ -1924,10 +1972,9 @@ class KlineRepository:
|
||||
|
||||
def _latest_enriched_date_duckdb(self) -> date | None:
|
||||
try:
|
||||
with self._lock:
|
||||
res = self.db.execute(
|
||||
res = self.execute_one(
|
||||
"SELECT max(date) FROM kline_enriched",
|
||||
).fetchone()
|
||||
)
|
||||
if res and res[0]:
|
||||
d = res[0]
|
||||
return d if isinstance(d, date) else date.fromisoformat(str(d))
|
||||
@@ -2095,10 +2142,11 @@ class KlineRepository:
|
||||
直接 write_parquet(out) 在进程被 kill (dev.sh 清端口用 kill -9)
|
||||
或断电时会留下半截文件, 之后 scan_parquet glob 整条链路报错。
|
||||
临时文件后缀 .tmp 不匹配 *.parquet glob, 不会被扫描误读。
|
||||
Windows 下目标正被并发读取时由 replace_with_retry 短退避穿过。
|
||||
"""
|
||||
tmp = out.with_name(out.name + ".tmp")
|
||||
df.write_parquet(tmp)
|
||||
tmp.replace(out) # 同目录 rename, POSIX/NTFS 均为原子操作
|
||||
replace_with_retry(tmp, out)
|
||||
|
||||
def _write_daily_partition(self, df: pl.DataFrame, table: str) -> None:
|
||||
"""按 date 分区写入 parquet，每个日期一个文件，支持 merge-upsert。"""
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[project]
|
||||
name = "tickflow-stock-panel-backend"
|
||||
version = "0.2.1"
|
||||
version = "0.2.2"
|
||||
description = "A 股选股 + 监控 + 回测面板 — TickFlow 适配"
|
||||
requires-python = ">=3.11"
|
||||
license = { text = "MIT" }
|
||||
|
||||
@@ -319,6 +319,7 @@ def test_builtin_matrix_strategies_use_their_declared_formula_modules():
|
||||
path for path in strategy_dir.glob("*.py") if path.name != "__init__.py"
|
||||
)
|
||||
|
||||
# 分钟形态策略 (minute_red_streak) 已迁至自定义策略目录, 内置策略全部 matrix 后端
|
||||
assert len(strategy_files) == 19
|
||||
for strategy_path in strategy_files:
|
||||
strategy = StrategyEngine._load_file(strategy_path)
|
||||
@@ -784,7 +785,10 @@ def test_registered_builtin_matrix_strategies_share_one_cache_profile():
|
||||
strategy_dirs=[REPO_ROOT / "backend" / "app" / "strategy" / "builtin"]
|
||||
)
|
||||
profile = build_matrix_cache_profile(engine, "stock")
|
||||
strategies = engine.strategy_definitions()
|
||||
strategies = tuple(
|
||||
s for s in engine.strategy_definitions()
|
||||
if s.execution_backend != "minute_filter"
|
||||
)
|
||||
|
||||
assert len(strategies) == 19
|
||||
assert all(strategy.execution_backend == "matrix_native" for strategy in strategies)
|
||||
|
||||
@@ -0,0 +1,330 @@
|
||||
"""分钟策略回测端到端集成测试 (三期 v1)。
|
||||
|
||||
用合成的分钟K分区 + 合成日线面板 + 专用测试策略, 完整跑通
|
||||
StrategyBacktestService.run() 的 minute_filter 分支:
|
||||
逐日回放 (与实盘选股同一条 StrategyEngine.run 路径) → 信号分钟收盘入场
|
||||
→ 涨停拒买 → 日K矩阵离场 → 交易记录携带分钟时间戳。
|
||||
|
||||
核心断言:
|
||||
- 日线窗口因果性: T 日的日线条件窗口只含 T-1 及更早 (测试策略内置守卫,
|
||||
窗口含 T 则拒绝命中 — 若回放器传错窗口, 全部用例的信号归零);
|
||||
- 入场价 = 触发分钟收盘价 (entry_price_override 机制);
|
||||
- 涨停拒买: 触发分钟收盘 >= 当日涨停价 (T-1 收盘 + 板块规则) 不成交;
|
||||
- 缺分钟分区的交易日显式跳过 (不回退最近分区);
|
||||
- 离场复用日K口径 (max_hold → 次日开盘)。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import date, datetime, timedelta
|
||||
from pathlib import Path
|
||||
|
||||
import polars as pl
|
||||
import pytest
|
||||
|
||||
from app.backtest.engine import BacktestEngine
|
||||
from app.backtest.strategy import StrategyBacktestConfig, StrategyBacktestService
|
||||
from app.strategy.engine import StrategyEngine
|
||||
|
||||
# ── 测试策略: 内置因果性守卫 ──────────────────────────────────────
|
||||
# 命中条件: 当日某分钟 close > T-1 close * 1.05, 触发分钟 = 首根满足条件的K。
|
||||
# daily 窗口的最后一个日期必须 < 触发日, 否则返回空 (回放器传错窗口时信号归零)。
|
||||
TEST_STRATEGY_SOURCE = '''
|
||||
import polars as pl
|
||||
|
||||
META = {
|
||||
"id": "test_minute_ping",
|
||||
"name": "test_minute_ping",
|
||||
"asset_types": ["stock"],
|
||||
"timeframes": ["1m"],
|
||||
"daily_history_bars": 5,
|
||||
"order_by": "close",
|
||||
"descending": True,
|
||||
"limit": 100,
|
||||
}
|
||||
EXECUTION_BACKEND = "minute_filter"
|
||||
|
||||
|
||||
def filter_minute_history(df, params, *, daily=None):
|
||||
if daily is None or daily.is_empty():
|
||||
return pl.DataFrame()
|
||||
trigger_day = df.select(pl.col("datetime").max()).item().date()
|
||||
# 因果性守卫: 日线窗口不得包含触发日。
|
||||
if daily.get_column("date").max() >= trigger_day:
|
||||
return pl.DataFrame()
|
||||
prev = (
|
||||
daily.sort("date").group_by("symbol").last()
|
||||
.select(pl.col("symbol"), pl.col("close").alias("prev_close"))
|
||||
)
|
||||
joined = df.join(prev, on="symbol", how="inner")
|
||||
hits = joined.filter(pl.col("close") > pl.col("prev_close") * 1.05)
|
||||
if hits.is_empty():
|
||||
return pl.DataFrame()
|
||||
return (
|
||||
hits.sort("datetime").group_by("symbol").first()
|
||||
.select(
|
||||
pl.col("symbol"),
|
||||
pl.col("datetime").alias("last_datetime"),
|
||||
pl.col("close"),
|
||||
)
|
||||
)
|
||||
'''
|
||||
|
||||
|
||||
# ── 合成数据 ─────────────────────────────────────────────────────
|
||||
def _trading_days(n: int, start: date = date(2026, 7, 1)) -> list[date]:
|
||||
days: list[date] = []
|
||||
cur = start
|
||||
while len(days) < n:
|
||||
if cur.weekday() < 5:
|
||||
days.append(cur)
|
||||
cur += timedelta(days=1)
|
||||
return days
|
||||
|
||||
|
||||
def _daily_panel(days: list[date], symbols: list[str]) -> pl.DataFrame:
|
||||
"""合成日线面板: 三个符号的慢涨走势, raw_close == close (复权因子 1)。"""
|
||||
rows = []
|
||||
for sym_idx, sym in enumerate(symbols):
|
||||
base = 10.0 + sym_idx * 4.0
|
||||
for t, day in enumerate(days):
|
||||
close = round(base * (1 + t * 0.002), 3)
|
||||
open_p = round(close - 0.05, 3)
|
||||
rows.append({
|
||||
"symbol": sym,
|
||||
"date": day,
|
||||
"open": open_p,
|
||||
"high": round(close + 0.08, 3),
|
||||
"low": round(open_p - 0.06, 3),
|
||||
"close": close,
|
||||
"raw_close": close,
|
||||
# 成交额需过 DEFAULT_BASIC_FILTER.amount_min (2e8) — 命中行的
|
||||
# amount 由 T-1 enriched 快照联表注入 (与实盘同路径)。
|
||||
"volume": 2e7,
|
||||
"amount": round(close * 2e7, 3),
|
||||
"name": f"股票{sym_idx}",
|
||||
"total_shares": 5e8,
|
||||
"float_shares": 4e8,
|
||||
"signal_limit_up": False,
|
||||
"signal_limit_down": False,
|
||||
})
|
||||
return pl.DataFrame(rows).sort(["symbol", "date"]).with_columns(
|
||||
pl.col("date").cast(pl.Date),
|
||||
)
|
||||
|
||||
|
||||
def _minute_frame(day: date, bars: list[tuple[str, str, float]]) -> pl.DataFrame:
|
||||
"""bars: (symbol, "HH:MM"(北京), close)。分区 datetime 为 naive-UTC 存储 (北京 - 8h)。"""
|
||||
rows = []
|
||||
for sym, hm, close in bars:
|
||||
local = datetime(day.year, day.month, day.day, int(hm[:2]), int(hm[3:]))
|
||||
rows.append({
|
||||
"symbol": sym,
|
||||
"datetime": local - timedelta(hours=8),
|
||||
"open": close - 0.01,
|
||||
"high": close + 0.01,
|
||||
"low": close - 0.02,
|
||||
"close": close,
|
||||
"volume": 1000.0,
|
||||
"amount": close * 1000.0,
|
||||
})
|
||||
return pl.DataFrame(rows).sort(["symbol", "datetime"]).with_columns(
|
||||
pl.col("datetime").cast(pl.Datetime("us")),
|
||||
)
|
||||
|
||||
|
||||
class _FakeMinuteRepo:
|
||||
"""仅实现分钟回测所需的最小 repo 接口。"""
|
||||
|
||||
def __init__(self, minute_frames: dict[date, pl.DataFrame]) -> None:
|
||||
self.minute_frames = minute_frames
|
||||
self.store = None
|
||||
|
||||
def list_minute_dates(self, start, end, asset_type="stock"):
|
||||
return sorted(d for d in self.minute_frames if start <= d <= end)
|
||||
|
||||
def get_minute_by_dates(self, symbols, dates, asset_type="stock"):
|
||||
frames = [self.minute_frames[d] for d in dates if d in self.minute_frames]
|
||||
if not frames:
|
||||
return pl.DataFrame(
|
||||
schema={"symbol": pl.Utf8, "datetime": pl.Datetime("us"),
|
||||
"open": pl.Float64, "high": pl.Float64, "low": pl.Float64,
|
||||
"close": pl.Float64, "volume": pl.Float64, "amount": pl.Float64},
|
||||
)
|
||||
df = pl.concat(frames)
|
||||
if symbols:
|
||||
df = df.filter(pl.col("symbol").is_in(list(symbols)))
|
||||
return df.sort(["symbol", "datetime"])
|
||||
|
||||
def earliest_minute_date(self):
|
||||
return min(self.minute_frames) if self.minute_frames else None
|
||||
|
||||
def get_index_daily(self, *args, **kwargs) -> pl.DataFrame:
|
||||
return pl.DataFrame()
|
||||
|
||||
|
||||
def _make_service(
|
||||
tmp_path: Path, panel: pl.DataFrame, minute_frames: dict[date, pl.DataFrame],
|
||||
) -> StrategyBacktestService:
|
||||
strat_dir = tmp_path / "strategies"
|
||||
strat_dir.mkdir(exist_ok=True)
|
||||
(strat_dir / "test_minute_ping.py").write_text(TEST_STRATEGY_SOURCE, encoding="utf-8")
|
||||
strategy_engine = StrategyEngine(strategy_dirs=[strat_dir])
|
||||
|
||||
repo = _FakeMinuteRepo(minute_frames)
|
||||
bt_engine = BacktestEngine(repo)
|
||||
|
||||
def _load_panel(self, symbols, start, end, feature_plan, asset_type="stock", **kw):
|
||||
df = panel.filter((pl.col("date") >= start) & (pl.col("date") <= end))
|
||||
if symbols:
|
||||
df = df.filter(pl.col("symbol").is_in(list(symbols)))
|
||||
keep = set(feature_plan.base_columns) | set(feature_plan.instrument_columns) | {"symbol", "date"}
|
||||
return df.select(sorted(c for c in df.columns if c in keep))
|
||||
|
||||
bt_engine.load_panel_for_backtest = _load_panel.__get__(bt_engine)
|
||||
return StrategyBacktestService(bt_engine, strategy_engine)
|
||||
|
||||
|
||||
def _config(start: date, end: date, **kw) -> StrategyBacktestConfig:
|
||||
defaults = dict(
|
||||
strategy_id="test_minute_ping",
|
||||
symbols=None,
|
||||
start=start,
|
||||
end=end,
|
||||
exit_fill="open_t+1",
|
||||
max_positions=10,
|
||||
mode="position",
|
||||
holding_days=1,
|
||||
overrides={"max_hold_days": 1},
|
||||
)
|
||||
defaults.update(kw)
|
||||
return StrategyBacktestConfig(**defaults)
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def scenario(tmp_path: Path):
|
||||
"""三个符号 x 三个回测日。面板共 30 个交易日 (指数慢涨, 涨停价按 T-1 收盘 +10%)。
|
||||
|
||||
- 000001.SZ: T1 触发 (close 10.72 > prev 10.19*1.05), T2/T3 不再触发;
|
||||
- 000002.SZ: 三天都不触发 (涨幅不足 5%);
|
||||
- 600000.SH: T2 触发但触发分钟收盘已达涨停价 → 拒买; T3 正常触发。
|
||||
"""
|
||||
days = _trading_days(30)
|
||||
t1, t2, t3 = days[-4], days[-3], days[-2] # 留一天做 T+1 离场
|
||||
symbols = ["000001.SZ", "000002.SZ", "600000.SH"]
|
||||
panel = _daily_panel(days, symbols)
|
||||
|
||||
def _prev_close(sym: str, before: date) -> float:
|
||||
return panel.filter(
|
||||
(pl.col("symbol") == sym) & (pl.col("date") < before)
|
||||
).sort("date").get_column("close")[-1]
|
||||
|
||||
minute_frames = {
|
||||
t1: _minute_frame(t1, [
|
||||
("000001.SZ", "09:31", round(_prev_close("000001.SZ", t1) * 1.005, 3)),
|
||||
("000001.SZ", "09:35", round(_prev_close("000001.SZ", t1) * 1.07, 3)), # 触发
|
||||
("000002.SZ", "09:31", round(_prev_close("000002.SZ", t1) * 1.01, 3)),
|
||||
("600000.SH", "09:31", round(_prev_close("600000.SH", t1) * 1.01, 3)),
|
||||
]),
|
||||
t2: _minute_frame(t2, [
|
||||
("000001.SZ", "09:31", round(_prev_close("000001.SZ", t2) * 1.004, 3)),
|
||||
("000002.SZ", "09:31", round(_prev_close("000002.SZ", t2) * 1.01, 3)),
|
||||
# 涨停拒买: 触发分钟收盘 = T-1收盘 * 1.10 (主板涨停价, 半进位后相等)
|
||||
("600000.SH", "09:40", round(_prev_close("600000.SH", t2) * 1.10, 3)),
|
||||
]),
|
||||
t3: _minute_frame(t3, [
|
||||
("000001.SZ", "09:31", round(_prev_close("000001.SZ", t3) * 1.004, 3)),
|
||||
("000002.SZ", "09:31", round(_prev_close("000002.SZ", t3) * 1.01, 3)),
|
||||
("600000.SH", "09:50", round(_prev_close("600000.SH", t3) * 1.06, 3)), # 触发
|
||||
]),
|
||||
}
|
||||
service = _make_service(tmp_path, panel, minute_frames)
|
||||
return service, panel, {"t1": t1, "t2": t2, "t3": t3, "t4": days[-1]}, minute_frames
|
||||
|
||||
|
||||
def test_entry_at_trigger_minute_price(scenario):
|
||||
service, panel, days, _ = scenario
|
||||
result = service.run(_config(days["t1"], days["t3"]))
|
||||
assert not result.error, result.error
|
||||
entries = [t for t in result.trades if t["symbol"] == "000001.SZ"]
|
||||
assert len(entries) == 1
|
||||
trade = entries[0]
|
||||
# 入场价 = 触发分钟 (09:35) 收盘价, 入场时间戳精确到分钟
|
||||
prev_close = panel.filter(
|
||||
(pl.col("symbol") == "000001.SZ") & (pl.col("date") < days["t1"])
|
||||
).sort("date").get_column("close")[-1]
|
||||
expected_price = round(prev_close * 1.07, 3)
|
||||
assert trade["entry_price"] == pytest.approx(expected_price, abs=1e-6)
|
||||
assert trade["entry_date"].startswith(f"{days['t1']} 09:35")
|
||||
|
||||
|
||||
def test_daily_window_strictly_before_trigger_day(scenario):
|
||||
"""因果性: 测试策略拒绝含触发日的日线窗口 — 有信号即证明窗口止于 T-1。"""
|
||||
service, _, days, _ = scenario
|
||||
result = service.run(_config(days["t1"], days["t3"]))
|
||||
assert not result.error, result.error
|
||||
assert result.trades, "日线窗口若含触发日, 测试策略会拒绝命中 — 信号归零"
|
||||
|
||||
|
||||
def test_limit_up_entry_rejected(scenario):
|
||||
service, panel, days, _ = scenario
|
||||
result = service.run(_config(days["t1"], days["t3"]))
|
||||
assert not result.error, result.error
|
||||
# T2 的 600000.SH 触发分钟收盘 = 涨停价 → 拒买; T3 才有它的成交
|
||||
entries_600000 = [t for t in result.trades if t["symbol"] == "600000.SH"]
|
||||
assert all(t["entry_date"][:10] == str(days["t3"]) for t in entries_600000)
|
||||
execution = result.stats.get("execution", {})
|
||||
assert execution.get("buy_limit_up", 0) >= 1
|
||||
replay_stats = result.stats.get("minute_replay", {})
|
||||
assert replay_stats.get("replayed_days") == 3
|
||||
|
||||
|
||||
def test_missing_partition_day_skipped(tmp_path):
|
||||
days = _trading_days(30)
|
||||
t1, t2, t3 = days[-4], days[-3], days[-2]
|
||||
symbols = ["000001.SZ"]
|
||||
panel = _daily_panel(days, symbols)
|
||||
|
||||
def _prev(before: date) -> float:
|
||||
return panel.filter(
|
||||
(pl.col("symbol") == "000001.SZ") & (pl.col("date") < before)
|
||||
).sort("date").get_column("close")[-1]
|
||||
|
||||
frames = {
|
||||
t1: _minute_frame(t1, [("000001.SZ", "09:35", round(_prev(t1) * 1.07, 3))]),
|
||||
t3: _minute_frame(t3, [("000001.SZ", "09:35", round(_prev(t3) * 1.06, 3))]),
|
||||
# t2 无分区 → 应被跳过, 而不是回退到 t1/t3 的数据
|
||||
}
|
||||
service = _make_service(tmp_path, panel, frames)
|
||||
result = service.run(_config(t1, t3))
|
||||
assert not result.error, result.error
|
||||
replay_stats = result.stats.get("minute_replay", {})
|
||||
assert replay_stats.get("replayed_days") == 2
|
||||
assert str(t2) in replay_stats.get("skipped_days", [])
|
||||
entry_days = {t["entry_date"][:10] for t in result.trades}
|
||||
assert str(t2) not in entry_days
|
||||
|
||||
|
||||
def test_exit_reuses_daily_next_open(scenario):
|
||||
"""离场复用日K口径: max_hold=1 → 次日开盘卖出。"""
|
||||
service, panel, days, _ = scenario
|
||||
result = service.run(_config(days["t1"], days["t3"]))
|
||||
assert not result.error, result.error
|
||||
trade = next(t for t in result.trades if t["symbol"] == "000001.SZ")
|
||||
entry_day = date.fromisoformat(trade["entry_date"][:10])
|
||||
exit_day = date.fromisoformat(str(trade["exit_date"])[:10])
|
||||
assert exit_day > entry_day
|
||||
next_open = panel.filter(
|
||||
(pl.col("symbol") == "000001.SZ") & (pl.col("date") == exit_day)
|
||||
).get_column("open")[0]
|
||||
assert trade["exit_price"] == pytest.approx(next_open, abs=1e-6)
|
||||
|
||||
|
||||
def test_guards(scenario, tmp_path):
|
||||
service, panel, days, _ = scenario
|
||||
# 信号触发卖出离场口径不支持 (通用校验或分钟分支守卫, 任一拒绝即可)
|
||||
result = service.run(_config(days["t1"], days["t3"], exit_fill="signal_next_minute"))
|
||||
assert result.error and "分钟" in result.error
|
||||
# 无分钟分区 → 明确报错
|
||||
empty_service = _make_service(tmp_path, panel, {})
|
||||
result = empty_service.run(_config(days["t1"], days["t3"]))
|
||||
assert "分钟K" in (result.error or "")
|
||||
@@ -0,0 +1,169 @@
|
||||
"""分钟红7 — 开盘 N 根 (当日最早) 分钟K多数收红, 且最高的 top_red 根全红。
|
||||
|
||||
数据契约: filter_minute_history 接收当日全市场分钟K窗口
|
||||
(symbol, datetime, open, high, low, close, volume, amount),
|
||||
由 ScreenerService.build_strategy_context 的 1m 分支从本地 kline_minute
|
||||
分区注入; 策略本身不感知数据来源 (本地同步 / 盘中增量刷新对它透明)。
|
||||
|
||||
META["daily_history_bars"] 声明叠加日线维度的条件 (N 日内涨停过):
|
||||
引擎会以 daily= 关键字注入日线 enriched 窗口, 涨停判定直接复用
|
||||
enriched 预计算信号 — signal_limit_up (收盘封板) 或 signal_broken_limit_up
|
||||
(炸板: 盘中触及涨停未封住), 任一命中即算"盘中涨停过"。
|
||||
|
||||
本文件是分钟红7的参考实现 (测试夹具): 该策略按用户要求以自定义策略形态
|
||||
交付, 正式位置为运行时 data/strategies/custom/minute_red_streak.py
|
||||
(gitignore, 用户可自行修改); 引擎按 id 加载, 参数覆盖不受位置影响。
|
||||
"""
|
||||
|
||||
import polars as pl
|
||||
|
||||
META = {
|
||||
"id": "minute_red_streak",
|
||||
"name": "分钟红7",
|
||||
"description": "开盘前7根1分钟K至少5根收红, 最高的2根(按最高价)都是红K, 且近20日盘中触及过涨停",
|
||||
"tags": ["分钟", "形态", "短线"],
|
||||
"asset_types": ["stock"],
|
||||
"timeframes": ["1m"],
|
||||
# 日线 enriched 窗口 (交易日语义, 含 as_of): 覆盖 limit_up_days 参数上限
|
||||
"daily_history_bars": 60,
|
||||
"params": [
|
||||
{
|
||||
"id": "bars",
|
||||
"label": "开盘K线数",
|
||||
"type": "int",
|
||||
"default": 7,
|
||||
"min": 5,
|
||||
"max": 15,
|
||||
"step": 1,
|
||||
},
|
||||
{
|
||||
"id": "min_red",
|
||||
"label": "最少红K数",
|
||||
"type": "int",
|
||||
"default": 5,
|
||||
"min": 1,
|
||||
"max": 15,
|
||||
"step": 1,
|
||||
},
|
||||
{
|
||||
"id": "top_red",
|
||||
"label": "最高K需红数",
|
||||
"type": "int",
|
||||
"default": 2,
|
||||
"min": 1,
|
||||
"max": 3,
|
||||
"step": 1,
|
||||
},
|
||||
{
|
||||
"id": "rank_by_close",
|
||||
"label": "最高K按收盘价排序",
|
||||
"type": "bool",
|
||||
"default": False,
|
||||
},
|
||||
{
|
||||
"id": "require_limit_up",
|
||||
"label": "要求N日内涨停过",
|
||||
"type": "bool",
|
||||
"default": True,
|
||||
},
|
||||
{
|
||||
"id": "limit_up_days",
|
||||
"label": "涨停回看天数",
|
||||
"type": "int",
|
||||
"default": 20,
|
||||
"min": 5,
|
||||
"max": 60,
|
||||
"step": 1,
|
||||
},
|
||||
],
|
||||
"order_by": "red_count",
|
||||
"descending": True,
|
||||
"limit": 100,
|
||||
}
|
||||
|
||||
EXECUTION_BACKEND = "minute_filter"
|
||||
ENTRY_SIGNALS: list[str] = []
|
||||
EXIT_SIGNALS: list[str] = []
|
||||
|
||||
|
||||
def _recent_limit_ups(daily: pl.DataFrame | None, lookback: int) -> pl.DataFrame:
|
||||
"""日线窗口 → (symbol, recent_limit_ups) 近 lookback 个交易日的涨停次数。
|
||||
|
||||
涨停过 = signal_limit_up (收盘封板) 或 signal_broken_limit_up (炸板触及)。
|
||||
日线窗口缺失 / 无涨停信号列 → 返回空表 (调用方 inner join 即失败闭合,
|
||||
宁可漏过不可错报)。
|
||||
"""
|
||||
empty = pl.DataFrame(schema={"symbol": pl.Utf8, "recent_limit_ups": pl.UInt32})
|
||||
if daily is None or daily.is_empty():
|
||||
return empty
|
||||
if not {"signal_limit_up", "signal_broken_limit_up"}.issubset(daily.columns):
|
||||
return empty
|
||||
return (
|
||||
daily.select("symbol", "date", "signal_limit_up", "signal_broken_limit_up")
|
||||
.sort(["symbol", "date"])
|
||||
.filter(pl.int_range(pl.len()).over("symbol") >= pl.len().over("symbol") - lookback)
|
||||
.group_by("symbol")
|
||||
.agg(
|
||||
recent_limit_ups=(
|
||||
pl.col("signal_limit_up").fill_null(False)
|
||||
| pl.col("signal_broken_limit_up").fill_null(False)
|
||||
).sum()
|
||||
)
|
||||
.filter(pl.col("recent_limit_ups") > 0)
|
||||
)
|
||||
|
||||
|
||||
def filter_minute_history(df: pl.DataFrame, params: dict, *, daily: pl.DataFrame | None = None) -> pl.DataFrame:
|
||||
"""红K形态过滤: 全向量化, 无逐行 Python 循环。
|
||||
|
||||
- 每标的按时间取当日最早 bars 根 (开盘窗口); 不足 bars 根不触发
|
||||
- 红 = close > open; 窗口内红K数 >= min_red
|
||||
- 按 rank_by (high / close) 降序取前 top_red 根, 同值取时间更晚者, 需全红
|
||||
- require_limit_up: 近 limit_up_days 个交易日盘中触及过涨停 (日线维度,
|
||||
由 daily 窗口的预计算涨停信号判定; 窗口缺失时失败闭合不触发)
|
||||
"""
|
||||
bars = int(params.get("bars") or 7)
|
||||
min_red = min(int(params.get("min_red") or 5), bars)
|
||||
top_red = min(int(params.get("top_red") or 2), bars)
|
||||
rank_by = "close" if params.get("rank_by_close") else "high"
|
||||
if rank_by not in df.columns:
|
||||
rank_by = "high"
|
||||
|
||||
windowed = (
|
||||
df.sort(["symbol", "datetime"])
|
||||
.filter(pl.int_range(pl.len()).over("symbol") < bars)
|
||||
.with_columns(_red=(pl.col("close") > pl.col("open")).cast(pl.Int32))
|
||||
)
|
||||
|
||||
window = windowed.group_by("symbol").agg(
|
||||
bars_checked=pl.len(),
|
||||
red_count=pl.col("_red").sum(),
|
||||
last_datetime=pl.col("datetime").max(),
|
||||
# 输出列名用 close: 基础过滤的股价区间作用于开盘窗口末根收盘价
|
||||
close=pl.col("close").sort_by("datetime").last(),
|
||||
window_high=pl.col("high").max(),
|
||||
window_low=pl.col("low").min(),
|
||||
window_volume=pl.col("volume").sum(),
|
||||
window_amount=pl.col("amount").sum(),
|
||||
)
|
||||
|
||||
top = (
|
||||
windowed.sort([rank_by, "datetime"], descending=[True, True])
|
||||
.filter(pl.int_range(pl.len()).over("symbol") < top_red)
|
||||
.group_by("symbol")
|
||||
.agg(top_red_count=pl.col("_red").sum())
|
||||
)
|
||||
|
||||
result = (
|
||||
window.join(top, on="symbol", how="inner")
|
||||
.filter(
|
||||
(pl.col("bars_checked") >= bars)
|
||||
& (pl.col("red_count") >= min_red)
|
||||
& (pl.col("top_red_count") >= top_red)
|
||||
)
|
||||
.drop("bars_checked")
|
||||
)
|
||||
if params.get("require_limit_up", True):
|
||||
lookback = max(5, min(int(params.get("limit_up_days") or 20), 60))
|
||||
result = result.join(_recent_limit_ups(daily, lookback), on="symbol", how="inner")
|
||||
return result
|
||||
@@ -0,0 +1,75 @@
|
||||
"""盘中异动聚合测试 (build_intraday, 不依赖真实网络/enriched)。
|
||||
|
||||
覆盖: 信号命中过滤、counts 计数、优先级排序 (涨停 > 炸板 > …)、
|
||||
多信号行、limit 截断、空快照与缺信号列的降级。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import date
|
||||
|
||||
import polars as pl
|
||||
|
||||
from app.services.abnormal_moves import build_intraday
|
||||
|
||||
|
||||
class _FakeRepo:
|
||||
def __init__(self, df: pl.DataFrame):
|
||||
self._df = df
|
||||
|
||||
def get_enriched_latest(self):
|
||||
return self._df, date(2026, 8, 28)
|
||||
|
||||
|
||||
def _df(rows: list[dict]) -> pl.DataFrame:
|
||||
cols = ["symbol", "name", "close", "change_pct", "amplitude", "vol_ratio_5d",
|
||||
"turnover_rate", "consecutive_limit_ups",
|
||||
"signal_limit_up", "signal_broken_limit_up", "signal_limit_down_recovery",
|
||||
"signal_limit_down", "signal_n_day_high", "signal_n_day_low",
|
||||
"signal_volume_surge"]
|
||||
base = {c: [] for c in cols}
|
||||
for r in rows:
|
||||
for c in cols:
|
||||
base[c].append(r.get(c))
|
||||
return pl.DataFrame(base)
|
||||
|
||||
|
||||
def test_counts_filter_and_priority():
|
||||
repo = _FakeRepo(_df([
|
||||
{"symbol": "A1", "name": "甲", "close": 10.0, "change_pct": 0.1,
|
||||
"signal_limit_up": True, "signal_n_day_high": True},
|
||||
{"symbol": "B1", "name": "乙", "close": 5.0, "change_pct": -0.05,
|
||||
"signal_limit_down": True},
|
||||
{"symbol": "C1", "name": "丙", "close": 8.0, "change_pct": 0.02,
|
||||
"signal_volume_surge": True},
|
||||
{"symbol": "D1", "name": "丁", "close": 7.0, "change_pct": None}, # 无信号 → 不出现
|
||||
]))
|
||||
out = build_intraday(repo)
|
||||
assert out["cache_date"] == "2026-08-28"
|
||||
assert out["counts"] == {"limit_up": 1, "broken": 0, "recovery": 0,
|
||||
"limit_down": 1, "new_high": 1, "new_low": 0,
|
||||
"volume_surge": 1}
|
||||
syms = [r["symbol"] for r in out["rows"]]
|
||||
assert syms == ["A1", "B1", "C1"] # 优先级: 涨停 > 跌停 > 放量; 无信号被过滤
|
||||
assert out["rows"][0]["signals"] == ["limit_up", "new_high"] # 多信号按优先级序
|
||||
|
||||
|
||||
def test_limit_truncates():
|
||||
repo = _FakeRepo(_df([
|
||||
{"symbol": f"S{i}", "signal_volume_surge": True, "change_pct": 0.01} for i in range(10)
|
||||
]))
|
||||
out = build_intraday(repo, limit=3)
|
||||
assert len(out["rows"]) == 3
|
||||
assert out["counts"]["volume_surge"] == 10 # counts 不受 limit 影响
|
||||
|
||||
|
||||
def test_empty_snapshot():
|
||||
repo = _FakeRepo(pl.DataFrame({"symbol": [], "name": []}))
|
||||
out = build_intraday(repo)
|
||||
assert out["rows"] == [] and out["counts"] == {}
|
||||
|
||||
|
||||
def test_missing_signal_columns_degrades():
|
||||
repo = _FakeRepo(pl.DataFrame({"symbol": ["A1"], "name": ["甲"]}))
|
||||
out = build_intraday(repo)
|
||||
assert out["rows"] == [] and out["counts"] == {}
|
||||
@@ -0,0 +1,161 @@
|
||||
"""Windows 读锁竞态下 parquet 原子替换的重试测试。
|
||||
|
||||
根因: polars scan_parquet / DuckDB read_parquet 扫描进行中持有分区句柄,
|
||||
Windows os.replace 替换"仍被打开"的目标文件抛 PermissionError (WinError 5);
|
||||
Linux 的 inode 交换语义无此限制。表现为个股分时"补齐数据"500。
|
||||
|
||||
修复: replace_with_retry 短退避重试穿过读窗口; 永久占用则原样抛出。
|
||||
两处 _atomic_write_parquet (repository / kline_sync) 均接入。
|
||||
|
||||
另含 DuckDB 句柄泄漏回归: latest_minute_date 等曾用 self.db.execute(...)
|
||||
.fetchone() 直连共享连接, 未消费结果集把首个分区句柄钉死在连接上,
|
||||
导致同步 os.replace 永久被拒 (修为 execute_one cursor+close)。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import sys
|
||||
from datetime import date, datetime
|
||||
|
||||
import polars as pl
|
||||
import pytest
|
||||
|
||||
from app.services import kline_sync
|
||||
from app.tickflow import repository
|
||||
|
||||
try:
|
||||
import psutil
|
||||
|
||||
_PSUTIL = True
|
||||
except ImportError: # pragma: no cover
|
||||
_PSUTIL = False
|
||||
|
||||
|
||||
def _minute_frame() -> pl.DataFrame:
|
||||
return pl.DataFrame({
|
||||
"symbol": ["600519.SH"],
|
||||
"datetime": [datetime(2026, 1, 15, 9, 30)],
|
||||
"open": [10.0], "high": [10.5], "low": [9.5], "close": [10.2],
|
||||
"volume": [100.0], "amount": [1020.0],
|
||||
})
|
||||
|
||||
|
||||
def _flaky_replace(monkeypatch, fail_times: int) -> dict:
|
||||
"""os.replace 前 fail_times 次 raise PermissionError, 之后正常执行。"""
|
||||
real_replace = os.replace
|
||||
state = {"calls": 0}
|
||||
|
||||
def _flaky(src, dst):
|
||||
state["calls"] += 1
|
||||
if state["calls"] <= fail_times:
|
||||
raise PermissionError(5, "拒绝访问。")
|
||||
return real_replace(src, dst)
|
||||
|
||||
monkeypatch.setattr(os, "replace", _flaky)
|
||||
return state
|
||||
|
||||
|
||||
# ---------- replace_with_retry 本体 ----------
|
||||
|
||||
def test_retry_succeeds_after_transient_blocks(tmp_path, monkeypatch):
|
||||
out = tmp_path / "part.parquet"
|
||||
out.write_bytes(b"old")
|
||||
src = tmp_path / "part.parquet.tmp"
|
||||
src.write_bytes(b"new")
|
||||
state = _flaky_replace(monkeypatch, fail_times=2)
|
||||
|
||||
repository.replace_with_retry(src, out, attempts=5, delay_s=0)
|
||||
|
||||
assert out.read_bytes() == b"new"
|
||||
assert state["calls"] == 3
|
||||
assert not src.exists()
|
||||
|
||||
|
||||
def test_retry_exhausted_raises_last_error(tmp_path, monkeypatch):
|
||||
monkeypatch.setattr(os, "replace", lambda s, d: (_ for _ in ()).throw(PermissionError(5, "拒绝访问。")))
|
||||
src = tmp_path / "a.tmp"
|
||||
src.write_bytes(b"x")
|
||||
|
||||
with pytest.raises(PermissionError, match="拒绝访问"):
|
||||
repository.replace_with_retry(src, tmp_path / "a.parquet", attempts=3, delay_s=0)
|
||||
assert src.exists() # 未被消费, 目标未生成
|
||||
|
||||
|
||||
def test_retry_no_block_single_attempt(tmp_path, monkeypatch):
|
||||
out = tmp_path / "part.parquet"
|
||||
out.write_bytes(b"old")
|
||||
src = tmp_path / "part.parquet.tmp"
|
||||
src.write_bytes(b"new")
|
||||
state = _flaky_replace(monkeypatch, fail_times=0)
|
||||
|
||||
repository.replace_with_retry(src, out, attempts=5, delay_s=0)
|
||||
|
||||
assert state["calls"] == 1
|
||||
assert out.read_bytes() == b"new"
|
||||
|
||||
|
||||
# ---------- 两处 _atomic_write_parquet 接入 ----------
|
||||
|
||||
def test_kline_sync_atomic_write_survives_transient_lock(tmp_path, monkeypatch):
|
||||
state = _flaky_replace(monkeypatch, fail_times=1)
|
||||
out = tmp_path / "date=2026-01-15" / "part.parquet"
|
||||
out.parent.mkdir(parents=True)
|
||||
|
||||
kline_sync._atomic_write_parquet(_minute_frame(), out)
|
||||
|
||||
assert out.exists()
|
||||
assert state["calls"] == 2
|
||||
assert pl.read_parquet(out).height == 1
|
||||
|
||||
|
||||
def test_repository_atomic_write_survives_transient_lock(tmp_path, monkeypatch):
|
||||
state = _flaky_replace(monkeypatch, fail_times=1)
|
||||
out = tmp_path / "kline_minute" / "date=2026-01-15" / "part.parquet"
|
||||
out.parent.mkdir(parents=True)
|
||||
|
||||
repository.KlineRepository._atomic_write_parquet(_minute_frame(), out)
|
||||
|
||||
assert out.exists()
|
||||
assert state["calls"] == 2
|
||||
|
||||
|
||||
def test_write_minute_partition_survives_reader_race(tmp_path, monkeypatch):
|
||||
"""集成: _write_minute_partition 读旧→concat→写新全程有读锁竞态仍完成。"""
|
||||
state = _flaky_replace(monkeypatch, fail_times=2)
|
||||
# 预置旧分区 (读改写路径)
|
||||
old_dir = tmp_path / "date=2026-01-15"
|
||||
old_dir.mkdir(parents=True)
|
||||
_minute_frame().write_parquet(old_dir / "part.parquet")
|
||||
|
||||
written = kline_sync._write_minute_partition(_minute_frame(), tmp_path)
|
||||
|
||||
assert written == 1
|
||||
assert state["calls"] >= 3 # 至少经历了重试
|
||||
|
||||
|
||||
# ---------- DuckDB 句柄泄漏回归 (Windows 实测语义) ----------
|
||||
|
||||
@pytest.mark.skipif(sys.platform != "win32" or not _PSUTIL, reason="Windows 句柄语义 + psutil")
|
||||
def test_minute_date_queries_do_not_pin_partition_handles(tmp_path):
|
||||
"""latest_minute_date 等查询后不得残留分区句柄。
|
||||
|
||||
旧实现 self.db.execute(...).fetchone() 的未消费结果集经 DuckDB buffer
|
||||
manager 钉住首个分区句柄, 后续同步 os.replace 永久 PermissionError。
|
||||
"""
|
||||
from app.tickflow.repository import DataStore, KlineRepository
|
||||
|
||||
minute_dir = tmp_path / "kline_minute"
|
||||
kline_sync._write_minute_partition(
|
||||
_minute_frame(), minute_dir) # date=2026-01-15
|
||||
repo = KlineRepository(DataStore(data_dir=tmp_path))
|
||||
|
||||
assert repo.latest_minute_date("600519.SH") == date(2026, 1, 15)
|
||||
assert repo.latest_minute_date_global() == date(2026, 1, 15)
|
||||
assert repo.earliest_minute_date() == date(2026, 1, 15)
|
||||
|
||||
me = psutil.Process()
|
||||
held = [f.path for f in me.open_files() if "kline_minute" in f.path]
|
||||
assert held == []
|
||||
|
||||
# 钉住场景的端到端后果: 查询后重写同一分区必须成功 (旧实现在此 PermissionError)
|
||||
assert kline_sync._write_minute_partition(_minute_frame(), minute_dir) == 1
|
||||
@@ -0,0 +1,178 @@
|
||||
"""盘前风向标服务测试 (不依赖真实网络)。
|
||||
|
||||
覆盖: 交易日回退、历史日 JSON 缓存命中与落盘、收益 enrich 数学 (当日oc/全天/次日)、
|
||||
fuyao 未配置降级、目标日失败 fallback_prev、彻底失败 no_data、AI 复盘摘要段。
|
||||
日期用 2026-08-26/27/28 (写作时为过去交易日), 与仓库既有绝对日期测试风格一致。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from datetime import date
|
||||
from pathlib import Path
|
||||
|
||||
import polars as pl
|
||||
import pytest
|
||||
|
||||
from app.plugins.fuyao.client import FuyaoError
|
||||
from app.services import auction_benchmark as ab
|
||||
|
||||
|
||||
def _write_kline(data_dir: Path, day: str, rows: list[tuple[str, float, float]]) -> None:
|
||||
part = data_dir / "kline_daily" / f"date={day}"
|
||||
part.mkdir(parents=True, exist_ok=True)
|
||||
df = pl.DataFrame(
|
||||
{
|
||||
"symbol": [r[0] for r in rows],
|
||||
"open": [r[1] for r in rows],
|
||||
"close": [r[2] for r in rows],
|
||||
}
|
||||
)
|
||||
df.write_parquet(part / "part-0.parquet")
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def data_dir(tmp_path: Path) -> Path:
|
||||
for d in ("2026-08-26", "2026-08-27", "2026-08-28"):
|
||||
(tmp_path / "kline_daily" / f"date={d}").mkdir(parents=True, exist_ok=True)
|
||||
_write_kline(tmp_path, "2026-08-26", [("600519.SH", 1690.0, 1700.0), ("000858.SZ", 130.0, 131.0)])
|
||||
_write_kline(tmp_path, "2026-08-27", [("600519.SH", 1717.0, 1734.0), ("000858.SZ", 132.0, 130.0)])
|
||||
_write_kline(tmp_path, "2026-08-28", [("600519.SH", 1734.0, 1768.68), ("000858.SZ", 129.0, 133.0)])
|
||||
return tmp_path
|
||||
|
||||
|
||||
class _FakeProvider:
|
||||
"""记录调用; fail_dates 中的日期抛 FuyaoError。"""
|
||||
|
||||
def __init__(self, fail_dates: set[str] | None = None):
|
||||
self.calls: list[str | None] = []
|
||||
self.fail_dates = fail_dates or set()
|
||||
|
||||
def short_term_benchmark(self, date_iso: str | None) -> dict:
|
||||
self.calls.append(date_iso)
|
||||
if date_iso in self.fail_dates:
|
||||
raise FuyaoError(f"code=3002: {date_iso} 未就绪")
|
||||
return {
|
||||
"date": date_iso or "2026-08-28",
|
||||
"date_ms": 0,
|
||||
"item": [
|
||||
{"thscode": "600519.SH", "ticker": "600519", "name": "贵州茅台",
|
||||
"auction_pct": 1.0, "tags": ["白酒", "超级品牌"]},
|
||||
{"thscode": "000858.SZ", "ticker": "000858", "name": "五粮液",
|
||||
"auction_pct": -2.5, "tags": ["白酒"]},
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def _use_provider(monkeypatch, provider) -> _FakeProvider:
|
||||
monkeypatch.setattr(ab, "_provider", lambda: provider)
|
||||
return provider
|
||||
|
||||
|
||||
# ---- 交易日解析 ----
|
||||
|
||||
def test_resolve_rolls_back_non_trading_day(data_dir):
|
||||
assert ab.resolve_trade_date(data_dir, date(2026, 8, 30)) == date(2026, 8, 28)
|
||||
assert ab.resolve_trade_date(data_dir, date(2026, 8, 27)) == date(2026, 8, 27)
|
||||
|
||||
|
||||
# ---- 状态与缓存 ----
|
||||
|
||||
def test_source_unavailable_without_fuyao(data_dir, monkeypatch):
|
||||
monkeypatch.setattr(ab, "_provider", lambda: None)
|
||||
out = ab.get_auction_benchmark(data_dir, None)
|
||||
assert out["state"] == "source_unavailable"
|
||||
|
||||
|
||||
def test_fetch_stores_cache_then_hits_cache(data_dir, monkeypatch):
|
||||
provider = _use_provider(monkeypatch, _FakeProvider())
|
||||
out = ab.get_auction_benchmark(data_dir, None) # 默认 → 最近分区 08-28
|
||||
assert out["state"] == "ok" and out["trade_date"] == "2026-08-28"
|
||||
assert out["count"] == 2 and len(out["items"]) == 2
|
||||
assert provider.calls == ["2026-08-28"]
|
||||
assert (data_dir / "auction_benchmark" / "date=2026-08-28.json").exists()
|
||||
|
||||
provider.calls.clear()
|
||||
out2 = ab.get_auction_benchmark(data_dir, date(2026, 8, 30)) # 周日 → 08-28 → 命中缓存
|
||||
assert out2["state"] == "ok" and out2["trade_date"] == "2026-08-28"
|
||||
assert provider.calls == []
|
||||
|
||||
|
||||
def test_explicit_history_date_uses_cache(data_dir, monkeypatch):
|
||||
provider = _use_provider(monkeypatch, _FakeProvider())
|
||||
ab.get_auction_benchmark(data_dir, date(2026, 8, 27))
|
||||
assert provider.calls == ["2026-08-27"]
|
||||
provider.calls.clear()
|
||||
ab.get_auction_benchmark(data_dir, date(2026, 8, 27))
|
||||
assert provider.calls == []
|
||||
|
||||
|
||||
def test_failure_falls_back_to_prev(data_dir, monkeypatch):
|
||||
provider = _use_provider(monkeypatch, _FakeProvider(fail_dates={"2026-08-28"}))
|
||||
out = ab.get_auction_benchmark(data_dir, date(2026, 8, 28))
|
||||
assert out["state"] == "fallback_prev"
|
||||
assert out["trade_date"] == "2026-08-27"
|
||||
assert out["requested_date"] == "2026-08-28"
|
||||
# 回退日缓存以 ok 落盘 (不污染直查)
|
||||
cached = json.loads((data_dir / "auction_benchmark" / "date=2026-08-27.json").read_text(encoding="utf-8"))
|
||||
assert cached["state"] == "ok"
|
||||
|
||||
|
||||
def test_total_failure_returns_no_data(data_dir, monkeypatch):
|
||||
_use_provider(monkeypatch, _FakeProvider(fail_dates={"2026-08-28", "2026-08-27"}))
|
||||
out = ab.get_auction_benchmark(data_dir, date(2026, 8, 28))
|
||||
assert out["state"] == "no_data"
|
||||
assert "2026-08-28" in out.get("message", "")
|
||||
|
||||
|
||||
def test_corrupt_cache_refetches(data_dir, monkeypatch):
|
||||
provider = _use_provider(monkeypatch, _FakeProvider())
|
||||
cache = data_dir / "auction_benchmark" / "date=2026-08-28.json"
|
||||
cache.parent.mkdir(parents=True, exist_ok=True)
|
||||
cache.write_text("{broken json", encoding="utf-8")
|
||||
out = ab.get_auction_benchmark(data_dir, date(2026, 8, 28))
|
||||
assert out["state"] == "ok"
|
||||
assert provider.calls # 缓存损坏 → 重新拉取
|
||||
|
||||
|
||||
# ---- 收益 enrich ----
|
||||
|
||||
def test_enrich_math_with_local_kline(data_dir, monkeypatch):
|
||||
# 显式查 08-27: prev=08-26, next=08-28
|
||||
_use_provider(monkeypatch, _FakeProvider())
|
||||
out = ab.get_auction_benchmark(data_dir, date(2026, 8, 27))
|
||||
by = {i["thscode"]: i for i in out["items"]}
|
||||
mt = by["600519.SH"]
|
||||
# day0_oc = 1734/1717-1; day0_pct = 1734/1700-1; d1 = 1768.68/1734-1
|
||||
assert mt["day0_oc"] == pytest.approx(1734.0 / 1717.0 - 1)
|
||||
assert mt["day0_pct"] == pytest.approx(1734.0 / 1700.0 - 1)
|
||||
assert mt["d1_pct"] == pytest.approx(1768.68 / 1734.0 - 1)
|
||||
wly = by["000858.SZ"]
|
||||
assert wly["day0_oc"] == pytest.approx(130.0 / 132.0 - 1)
|
||||
assert wly["d1_pct"] == pytest.approx(133.0 / 130.0 - 1)
|
||||
# 原始字段透传
|
||||
assert mt["auction_pct"] == 1.0 and mt["tags"] == ["白酒", "超级品牌"]
|
||||
|
||||
|
||||
def test_enrich_missing_kline_gives_none(data_dir, monkeypatch):
|
||||
# 最新分区 08-28 无次日 → d1_pct=None; kline 行存在则 oc/pct 正常
|
||||
_use_provider(monkeypatch, _FakeProvider())
|
||||
out = ab.get_auction_benchmark(data_dir, None)
|
||||
for i in out["items"]:
|
||||
assert i["d1_pct"] is None
|
||||
assert i["day0_oc"] is not None
|
||||
|
||||
|
||||
# ---- AI 复盘摘要 ----
|
||||
|
||||
def test_build_recap_context_contains_summary(data_dir, monkeypatch):
|
||||
_use_provider(monkeypatch, _FakeProvider())
|
||||
ctx = ab.build_recap_context(data_dir)
|
||||
assert "盘前风向标名单" in ctx and "贵州茅台" in ctx
|
||||
assert "白酒" in ctx # 概念标签
|
||||
assert "当日" in ctx # 收益对照
|
||||
|
||||
|
||||
def test_build_recap_context_empty_without_source(data_dir, monkeypatch):
|
||||
monkeypatch.setattr(ab, "_provider", lambda: None)
|
||||
assert ab.build_recap_context(data_dir) == ""
|
||||
@@ -12,7 +12,7 @@ from app.tickflow.capabilities import Cap, CapabilityLimits, CapabilitySet
|
||||
from app.tickflow.policy import _augment_custom_sources
|
||||
|
||||
|
||||
def _set_providers(monkeypatch, *, daily="tickflow", adj="same_as_daily",
|
||||
def _set_providers(monkeypatch, *, daily="tickflow", adj="tickflow",
|
||||
minute="tickflow", financial="tickflow") -> None:
|
||||
"""mock preferences 各数据集 provider getter。"""
|
||||
from app.services import preferences
|
||||
@@ -42,9 +42,9 @@ def test_daily_custom_source_grants_daily_batch(monkeypatch):
|
||||
assert not capset.has(Cap.FINANCIAL)
|
||||
|
||||
|
||||
def test_adj_same_as_daily_resolves_to_daily_provider(monkeypatch):
|
||||
"""adj_factor_provider=same_as_daily → 跟随 daily provider 判定。"""
|
||||
_set_providers(monkeypatch, daily="mock_src", adj="same_as_daily")
|
||||
def test_adj_custom_source_grants_adj_factor(monkeypatch):
|
||||
"""adj 显式路由到声明除权的自定义源 → 补授能力 (跟随日K已下线, 独立判定)。"""
|
||||
_set_providers(monkeypatch, adj="mock_src")
|
||||
_set_datasets(monkeypatch, {"adj_factor"})
|
||||
capset = CapabilitySet()
|
||||
_augment_custom_sources(capset)
|
||||
|
||||
@@ -0,0 +1,256 @@
|
||||
"""能力路由矩阵契约测试。
|
||||
|
||||
覆盖: 注册表与路由偏好字段一一对应、候选按各源 datasets 声明过滤、
|
||||
候选只含当前可用源 (未就绪插件进 pending 并携带原因)、TickFlow 候选
|
||||
按当前订阅档位过滤、偏好指向 tickflow 但档位不足时的 tf_available
|
||||
标记、usable 跟随生效源 (各页能力门控的统一判定)、每能力独立路由
|
||||
(「跟随日K」特殊值已下线, 存量旧值由 preferences getter 自愈回退)、
|
||||
偏好指向未知源时的回退形态。全部用假插件/自定义源,
|
||||
不依赖真实网络与本地 data/ 目录。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from app.data_providers import custom as custom_sources
|
||||
from app.data_providers.capabilities import CAPABILITY_REGISTRY, build_capability_matrix
|
||||
|
||||
DEFAULT_CURRENT = {
|
||||
"daily_data_provider": "tickflow",
|
||||
"adj_factor_provider": "tickflow",
|
||||
"minute_data_provider": "tickflow",
|
||||
"depth5_data_provider": "tickflow",
|
||||
"realtime_data_provider": "tickflow",
|
||||
"financial_data_provider": "tickflow",
|
||||
}
|
||||
|
||||
|
||||
def _fake_sources(monkeypatch, plugins: list[dict], customs: list[dict] | None = None) -> None:
|
||||
monkeypatch.setattr(custom_sources, "list_plugins", lambda: plugins)
|
||||
monkeypatch.setattr(custom_sources, "list_sources", lambda: customs or [])
|
||||
|
||||
|
||||
def _by_id(matrix: dict) -> dict[str, dict]:
|
||||
return {c["id"]: c for c in matrix["capabilities"]}
|
||||
|
||||
|
||||
def test_registry_covers_all_routing_fields():
|
||||
"""注册表是能力的单一权威: 可路由能力与偏好键一一对应、无重复;
|
||||
full_minute 为不可路由能力 (field=None, 仅 TickFlow Expert 提供)。"""
|
||||
routable = [c["field"] for c in CAPABILITY_REGISTRY if c["field"] is not None]
|
||||
assert sorted(routable) == sorted(DEFAULT_CURRENT)
|
||||
assert len(set(routable)) == len(routable)
|
||||
assert {c["id"] for c in CAPABILITY_REGISTRY} == {
|
||||
"realtime", "daily", "minute", "full_minute", "depth5", "adj_factor", "financial",
|
||||
}
|
||||
full_minute = next(c for c in CAPABILITY_REGISTRY if c["id"] == "full_minute")
|
||||
assert full_minute["field"] is None
|
||||
assert full_minute["tf_tier"] == "expert"
|
||||
for cap in CAPABILITY_REGISTRY:
|
||||
assert cap["default"] == "tickflow"
|
||||
assert cap["tf_tier"] in ("none", "starter", "pro", "expert")
|
||||
assert "follow" not in cap
|
||||
|
||||
|
||||
def test_matrix_without_third_party_sources(monkeypatch):
|
||||
"""无插件无自定义源: 每个能力只剩 TickFlow 候选, 默认路由全部生效。"""
|
||||
_fake_sources(monkeypatch, [])
|
||||
matrix = build_capability_matrix(dict(DEFAULT_CURRENT), tickflow_tier="expert")
|
||||
assert matrix["tickflow_tier"] == "expert"
|
||||
assert len(matrix["capabilities"]) == 7
|
||||
for cap in matrix["capabilities"]:
|
||||
names = [c["name"] for c in cap["candidates"]]
|
||||
assert names == ["tickflow"]
|
||||
assert cap["candidates"][0]["kind"] == "builtin"
|
||||
assert cap["tf_available"] is True
|
||||
assert cap["usable"] is True
|
||||
assert cap["pending"] == []
|
||||
assert cap["current"] == cap["effective"] == "tickflow"
|
||||
|
||||
|
||||
def test_candidates_only_available_unready_goes_pending(monkeypatch):
|
||||
"""候选只包含声明了该能力且可用的源; 未就绪插件进 pending 并携带原因。"""
|
||||
_fake_sources(
|
||||
monkeypatch,
|
||||
[
|
||||
{"name": "fuyao", "display_name": "fuyao", "datasets": ["realtime"],
|
||||
"available": True, "status": "ok"},
|
||||
{"name": "sdk", "display_name": "SDK", "datasets": ["daily", "minute"],
|
||||
"available": False, "status": "依赖未安装"},
|
||||
],
|
||||
[{"name": "myhttp", "display_name": "MyHTTP", "datasets": ["financial", "realtime"]}],
|
||||
)
|
||||
caps = _by_id(build_capability_matrix(dict(DEFAULT_CURRENT), tickflow_tier="expert"))
|
||||
assert [c["name"] for c in caps["realtime"]["candidates"]] == ["tickflow", "fuyao", "myhttp"]
|
||||
assert [c["name"] for c in caps["daily"]["candidates"]] == ["tickflow"]
|
||||
assert [c["name"] for c in caps["daily"]["pending"]] == ["sdk"]
|
||||
assert caps["daily"]["pending"][0]["available"] is False
|
||||
assert caps["daily"]["pending"][0]["note"] == "依赖未安装"
|
||||
assert [c["name"] for c in caps["financial"]["candidates"]] == ["tickflow", "myhttp"]
|
||||
assert caps["financial"]["candidates"][1]["kind"] == "custom"
|
||||
assert [c["name"] for c in caps["adj_factor"]["candidates"]] == ["tickflow"]
|
||||
|
||||
|
||||
def test_tickflow_candidates_filtered_by_tier(monkeypatch):
|
||||
"""TickFlow 只出现在当前档位确实提供的能力候选里 (free: 仅日K)。"""
|
||||
_fake_sources(monkeypatch, [])
|
||||
caps = _by_id(build_capability_matrix(dict(DEFAULT_CURRENT), tickflow_tier="free"))
|
||||
assert caps["daily"]["tf_available"] is True
|
||||
assert [c["name"] for c in caps["daily"]["candidates"]] == ["tickflow"]
|
||||
for cap_id in ("realtime", "minute", "depth5", "adj_factor", "financial"):
|
||||
assert caps[cap_id]["tf_available"] is False
|
||||
assert [c["name"] for c in caps[cap_id]["candidates"]] == []
|
||||
# starter 解锁实时与除权, 分钟/五档/财务仍锁
|
||||
caps = _by_id(build_capability_matrix(dict(DEFAULT_CURRENT), tickflow_tier="starter"))
|
||||
assert caps["realtime"]["tf_available"] is True
|
||||
assert caps["adj_factor"]["tf_available"] is True
|
||||
assert caps["minute"]["tf_available"] is False
|
||||
assert caps["depth5"]["tf_available"] is False
|
||||
assert caps["financial"]["tf_available"] is False
|
||||
|
||||
|
||||
def test_current_tickflow_unmet_tier_flagged(monkeypatch):
|
||||
"""偏好仍指向 tickflow 但档位不足: current 不动, tf_available=False 供前端警示。"""
|
||||
_fake_sources(monkeypatch, [])
|
||||
cap = _by_id(
|
||||
build_capability_matrix(dict(DEFAULT_CURRENT), tickflow_tier="free"),
|
||||
)["realtime"]
|
||||
assert cap["current"] == cap["effective"] == "tickflow"
|
||||
assert cap["tf_available"] is False
|
||||
assert cap["usable"] is False
|
||||
assert [c["name"] for c in cap["candidates"]] == []
|
||||
|
||||
|
||||
def test_usable_follows_effective_provider(monkeypatch):
|
||||
"""usable 跟随生效源而非 TickFlow 套餐: 各页能力门控的统一判定。
|
||||
|
||||
- 生效源是可用插件 → usable (即使 TickFlow 档位不足);
|
||||
- 生效源是未就绪插件 → 不可用 (即使有其他可用候选);
|
||||
- 除权独立路由, 档位门槛/插件可用性同样生效。
|
||||
"""
|
||||
_fake_sources(
|
||||
monkeypatch,
|
||||
[
|
||||
{"name": "fuyao", "display_name": "fuyao", "datasets": ["realtime", "minute"],
|
||||
"available": True, "status": "ok"},
|
||||
{"name": "sdk", "display_name": "SDK", "datasets": ["daily", "adj_factor", "minute"],
|
||||
"available": False, "status": "依赖未安装"},
|
||||
],
|
||||
)
|
||||
caps = _by_id(
|
||||
build_capability_matrix(
|
||||
dict(DEFAULT_CURRENT, realtime_data_provider="fuyao", minute_data_provider="sdk"),
|
||||
tickflow_tier="free",
|
||||
),
|
||||
)
|
||||
# 路由到可用插件: TickFlow 档位不足不影响 usable
|
||||
assert caps["realtime"]["tf_available"] is False
|
||||
assert caps["realtime"]["usable"] is True
|
||||
# 路由到未就绪插件: 有可用候选 (fuyao) 也不算 usable
|
||||
minute = caps["minute"]
|
||||
assert [c["name"] for c in minute["candidates"]] == ["fuyao"]
|
||||
assert minute["usable"] is False
|
||||
# 除权默认路由 tickflow: 自身档位门槛 (starter+) 生效, free 档不可用
|
||||
# (独立路由, 不再随日K联动)
|
||||
assert caps["adj_factor"]["effective"] == "tickflow"
|
||||
assert caps["adj_factor"]["tf_available"] is False
|
||||
assert caps["adj_factor"]["usable"] is False
|
||||
# 除权显式路由到未就绪 sdk → 不可用 (有 tickflow 候选也不算, 日K不受影响)
|
||||
caps = _by_id(
|
||||
build_capability_matrix(
|
||||
dict(DEFAULT_CURRENT, adj_factor_provider="sdk"), tickflow_tier="expert",
|
||||
),
|
||||
)
|
||||
assert [c["name"] for c in caps["adj_factor"]["candidates"]] == ["tickflow"]
|
||||
assert caps["adj_factor"]["effective"] == "sdk"
|
||||
assert caps["adj_factor"]["usable"] is False
|
||||
assert caps["daily"]["usable"] is True
|
||||
|
||||
|
||||
def test_unknown_or_empty_tier_fails_closed(monkeypatch):
|
||||
"""未知档 (探测缺失) 与空档按 none 处理: 仅全档位能力 (日K) 保留 TickFlow。"""
|
||||
_fake_sources(monkeypatch, [])
|
||||
for tier in ("", "unknown", None):
|
||||
matrix = build_capability_matrix(dict(DEFAULT_CURRENT), tickflow_tier=tier)
|
||||
caps = _by_id(matrix)
|
||||
assert [c["name"] for c in caps["daily"]["candidates"]] == ["tickflow"]
|
||||
assert caps["realtime"]["tf_available"] is False
|
||||
assert caps["realtime"]["candidates"] == []
|
||||
assert caps["minute"]["tf_available"] is False
|
||||
|
||||
|
||||
def test_adj_factor_routes_independently(monkeypatch):
|
||||
"""除权独立路由 (跟随日K已下线): 显式切到声明除权的插件即生效, 与日K当前源无关。
|
||||
|
||||
历史遗留值 same_as_daily 不再被矩阵特判 — 存量配置经 preferences getter
|
||||
按非法源回退 tickflow (getter 层自愈), 矩阵只信任注入值。
|
||||
"""
|
||||
_fake_sources(
|
||||
monkeypatch,
|
||||
[{"name": "sdk", "display_name": "SDK", "datasets": ["adj_factor"],
|
||||
"available": True, "status": "ok"}],
|
||||
)
|
||||
# 日K走 tickflow, 除权显式走 sdk → 互不影响 (TickFlow none 档下插件照常可用)
|
||||
caps = _by_id(
|
||||
build_capability_matrix(
|
||||
dict(DEFAULT_CURRENT, adj_factor_provider="sdk"), tickflow_tier="none",
|
||||
),
|
||||
)
|
||||
adj = caps["adj_factor"]
|
||||
assert adj["current"] == adj["effective"] == "sdk"
|
||||
assert adj["usable"] is True
|
||||
assert caps["daily"]["effective"] == "tickflow"
|
||||
assert caps["daily"]["usable"] is True
|
||||
|
||||
|
||||
def test_depth5_capability_semantics(monkeypatch):
|
||||
"""五档: pro+ 档 TickFlow 可供 (usable); 档位不足时不可用且无候选。
|
||||
|
||||
插件数据集白名单未开放 depth5, 假插件即使声明其他数据集也不进五档候选;
|
||||
未来契约开放后声明 depth5 的源会自然成为候选 (candidates 按 datasets 过滤)。
|
||||
"""
|
||||
_fake_sources(
|
||||
monkeypatch,
|
||||
[{"name": "fuyao", "display_name": "fuyao", "datasets": ["realtime"],
|
||||
"available": True, "status": "ok"}],
|
||||
)
|
||||
# pro 档: TickFlow 进候选, 默认路由 tickflow → usable
|
||||
cap = _by_id(build_capability_matrix(dict(DEFAULT_CURRENT), tickflow_tier="pro"))["depth5"]
|
||||
assert cap["tf_available"] is True
|
||||
assert [c["name"] for c in cap["candidates"]] == ["tickflow"]
|
||||
assert cap["usable"] is True
|
||||
# starter 档: 档位不足 → 无候选, usable False (连板梯队封单缺数据)
|
||||
cap = _by_id(build_capability_matrix(dict(DEFAULT_CURRENT), tickflow_tier="starter"))["depth5"]
|
||||
assert cap["tf_available"] is False
|
||||
assert cap["candidates"] == []
|
||||
assert cap["usable"] is False
|
||||
|
||||
|
||||
def test_unknown_current_display_falls_back_to_name(monkeypatch):
|
||||
"""偏好指向未注册源 (正常经 getters 校验不会发生): 展示回退为原始名, 不抛异常。"""
|
||||
_fake_sources(monkeypatch, [])
|
||||
caps = _by_id(
|
||||
build_capability_matrix(
|
||||
dict(DEFAULT_CURRENT, realtime_data_provider="ghost"), tickflow_tier="expert",
|
||||
),
|
||||
)
|
||||
assert caps["realtime"]["current"] == "ghost"
|
||||
assert caps["realtime"]["current_display"] == "ghost"
|
||||
assert caps["realtime"]["effective_display"] == "ghost"
|
||||
|
||||
|
||||
def test_full_minute_row_is_non_routable_expert_only(monkeypatch):
|
||||
"""全量分钟行: field=None 不可路由, 生效源恒为 TickFlow, 按 expert 档判定可用。"""
|
||||
_fake_sources(monkeypatch, [])
|
||||
# 即使有插件声明别的数据集也不会成为全量分钟候选 (契约不开放该数据集)
|
||||
caps = _by_id(build_capability_matrix(dict(DEFAULT_CURRENT), tickflow_tier="expert"))
|
||||
fm = caps["full_minute"]
|
||||
assert fm["field"] is None
|
||||
assert [c["name"] for c in fm["candidates"]] == ["tickflow"]
|
||||
assert fm["usable"] is True
|
||||
assert fm["tf_available"] is True
|
||||
assert fm["effective"] == "tickflow"
|
||||
|
||||
caps_pro = _by_id(build_capability_matrix(dict(DEFAULT_CURRENT), tickflow_tier="pro"))
|
||||
fm_pro = caps_pro["full_minute"]
|
||||
assert fm_pro["candidates"] == []
|
||||
assert fm_pro["usable"] is False
|
||||
assert fm_pro["tf_available"] is False
|
||||
@@ -1,15 +1,19 @@
|
||||
"""自定义源实时行情涨跌幅单位自适应归一测试。
|
||||
"""自定义源实时行情比例字段单位归一测试 (CONTRIBUTING §3.1)。
|
||||
|
||||
契约要求 change_pct/amplitude/turnover_rate 用小数制 (0.0366 = 3.66%),
|
||||
但不少第三方接口(如 a-stock-data)直接返回 3.66 表示 3.66%。未归一会把
|
||||
行业/概念统计与前端 x100 展示整体放大 100 倍(用户反馈)。
|
||||
契约: change_pct/amplitude/turnover_rate 为小数制 (0.0366 = 3.66%)。
|
||||
单位只认显式声明 pct_unit: percent|decimal, 不靠数值猜:
|
||||
- 声明 percent → 无条件 /100; 声明 decimal → 无条件透传;
|
||||
- 未声明 → change_pct 保留截面中位数判定(涨跌停 30% 上限物理可判),
|
||||
amplitude/turnover_rate 置 None 交下游重算(fail-closed),
|
||||
已被 transforms 显式处理过的列视为用户接管单位, 透传。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import polars as pl
|
||||
import pytest
|
||||
|
||||
from app.data_providers.custom.config import CustomSourceConfig, DatasetConfig
|
||||
from app.data_providers.custom.config import CustomSourceConfig, DatasetConfig, config_from_dict
|
||||
from app.data_providers.custom.provider import GenericHTTPProvider, _normalize_pct_units
|
||||
|
||||
|
||||
@@ -22,24 +26,64 @@ def _df(pcts, amps=None, turnovers=None):
|
||||
return pl.DataFrame(data)
|
||||
|
||||
|
||||
def test_percent_unit_batch_is_divided_by_100():
|
||||
out = _normalize_pct_units(_df(
|
||||
# ---- 显式声明: percent ----
|
||||
|
||||
|
||||
def test_declared_percent_divides_all_columns():
|
||||
out = _normalize_pct_units(
|
||||
_df(
|
||||
[1.5, -2.2, 0.9, 2.8, -1.1, 0.6, 3.3, -0.8],
|
||||
amps=[2.0, 3.5, 1.8, 4.0, 2.5, 1.2, 5.0, 1.6],
|
||||
turnovers=[0.5, 1.2, 0.8, 2.0, 0.9, 0.4, 1.5, 0.7],
|
||||
))
|
||||
),
|
||||
pct_unit="percent",
|
||||
)
|
||||
assert out["change_pct"][0] == pytest.approx(0.015)
|
||||
assert out["amplitude"][0] == pytest.approx(0.02)
|
||||
assert out["turnover_rate"][0] == pytest.approx(0.005)
|
||||
|
||||
|
||||
def test_fraction_unit_batch_untouched():
|
||||
def test_declared_percent_wins_even_when_values_look_decimal():
|
||||
# 百分制低波动日: 0.25 表示 0.25%, 数值落在小数制区间内——声明优先, 不靠猜
|
||||
out = _normalize_pct_units(
|
||||
_df(
|
||||
[0.25, 0.30, 0.28, 0.27, 0.26, 0.22],
|
||||
amps=[0.4, 0.5, 0.45, 0.6, 0.5, 0.4],
|
||||
turnovers=[0.05, 0.08, 0.06, 0.1, 0.07, 0.05],
|
||||
),
|
||||
pct_unit="percent",
|
||||
)
|
||||
assert out["change_pct"][0] == pytest.approx(0.0025)
|
||||
assert out["amplitude"][0] == pytest.approx(0.004)
|
||||
assert out["turnover_rate"][0] == pytest.approx(0.0005)
|
||||
|
||||
|
||||
# ---- 显式声明: decimal ----
|
||||
|
||||
|
||||
def test_declared_decimal_passes_through():
|
||||
pcts = [0.015, -0.022, 0.009, 0.028, -0.011, 0.006, 0.033, -0.008]
|
||||
out = _normalize_pct_units(_df(pcts, amps=[0.02, 0.035, 0.018, 0.04, 0.025, 0.012, 0.05, 0.016]))
|
||||
out = _normalize_pct_units(
|
||||
_df(pcts, amps=[0.02, 0.035, 0.018, 0.04, 0.025, 0.012, 0.05, 0.016]), pct_unit="decimal"
|
||||
)
|
||||
assert out["change_pct"].to_list() == pcts
|
||||
assert out["amplitude"][0] == pytest.approx(0.02)
|
||||
|
||||
|
||||
def test_declared_decimal_wins_even_when_values_look_percent():
|
||||
# 用户声明了小数制就按小数制契约透传, 不替用户"修正"数据
|
||||
out = _normalize_pct_units(_df([3.66, -2.15, 0.9, 2.8, 1.1]), pct_unit="decimal")
|
||||
assert out["change_pct"][0] == pytest.approx(3.66)
|
||||
|
||||
|
||||
# ---- 未声明: change_pct 保留截面判定(物理可判) ----
|
||||
|
||||
|
||||
def test_undeclared_change_pct_percent_batch_normalized():
|
||||
out = _normalize_pct_units(_df([1.5, -2.2, 0.9, 2.8, 3.3, 0.6]))
|
||||
assert out["change_pct"][0] == pytest.approx(0.015)
|
||||
|
||||
|
||||
def test_limit_up_fraction_30cm_not_divided():
|
||||
# 北交所 30% 涨跌停的小数制极值不应被误判为百分制
|
||||
out = _normalize_pct_units(_df([0.30, 0.29, 0.28, 0.27, 0.26]))
|
||||
@@ -60,38 +104,85 @@ def test_string_values_are_cast():
|
||||
assert out["change_pct"][0] == pytest.approx(0.015)
|
||||
|
||||
|
||||
# ---- 未声明: amplitude/turnover_rate fail-closed (核心修复) ----
|
||||
|
||||
|
||||
def test_undeclared_amplitude_and_turnover_are_nulled():
|
||||
# 百分制 0.05 = 0.05% 与小数制 0.05 = 5% 数值相同, 不可判定 → 置 None
|
||||
out = _normalize_pct_units(
|
||||
_df(
|
||||
[1.5, -2.2, 0.9, 2.8, 3.3, 0.6],
|
||||
amps=[2.0, 3.5, 1.8, 4.0, 5.0, 1.6],
|
||||
turnovers=[0.05, 1.2, 0.8, 2.0, 1.5, 0.7],
|
||||
)
|
||||
)
|
||||
assert out["amplitude"].null_count() == 6
|
||||
assert out["turnover_rate"].null_count() == 6
|
||||
# change_pct 仍正常归一
|
||||
assert out["change_pct"][0] == pytest.approx(0.015)
|
||||
|
||||
|
||||
def test_undeclared_transformed_column_passes_through():
|
||||
# 用户已用 transforms 显式处理过单位(如 value / 100)的列: 视为接管, 不置 None
|
||||
out = _normalize_pct_units(
|
||||
_df([1.5, -2.2, 0.9, 2.8, 3.3, 0.6], turnovers=[0.005, 0.012, 0.008, 0.02, 0.015, 0.007]),
|
||||
transformed_cols=frozenset({"turnover_rate"}),
|
||||
)
|
||||
assert out["turnover_rate"][0] == pytest.approx(0.005)
|
||||
# 未 transform 的 amplitude 仍 fail-closed
|
||||
assert "amplitude" not in out.columns
|
||||
|
||||
|
||||
def test_missing_or_null_columns_noop():
|
||||
out = _normalize_pct_units(pl.DataFrame({"close": [1.0, 2.0]}))
|
||||
assert out.columns == ["close"]
|
||||
out2 = _normalize_pct_units(_df([None, None, None, None, None, None]))
|
||||
assert out2["change_pct"].null_count() == 6
|
||||
# 全 null 的不可判定列保持 null
|
||||
out3 = _normalize_pct_units(_df([1.5, -2.2, 0.9, 2.8, 3.3, 0.6], turnovers=[None] * 6))
|
||||
assert out3["turnover_rate"].null_count() == 6
|
||||
|
||||
|
||||
def _realtime_provider(rows):
|
||||
provider = GenericHTTPProvider(CustomSourceConfig(
|
||||
# ---- provider 集成 ----
|
||||
|
||||
|
||||
def _realtime_provider(rows, **ds_kwargs):
|
||||
provider = GenericHTTPProvider(
|
||||
CustomSourceConfig(
|
||||
name="pct_source",
|
||||
display_name="Pct Source",
|
||||
datasets={"realtime": DatasetConfig(
|
||||
datasets={
|
||||
"realtime": DatasetConfig(
|
||||
url="https://example.test/realtime",
|
||||
field_map={
|
||||
"code": "symbol", "price": "last_price", "pre_close": "prev_close",
|
||||
"pct": "change_pct", "amp": "amplitude", "turnover": "turnover_rate",
|
||||
"code": "symbol",
|
||||
"price": "last_price",
|
||||
"pre_close": "prev_close",
|
||||
"pct": "change_pct",
|
||||
"amp": "amplitude",
|
||||
"turnover": "turnover_rate",
|
||||
},
|
||||
)},
|
||||
))
|
||||
**ds_kwargs,
|
||||
)
|
||||
},
|
||||
)
|
||||
)
|
||||
provider._request_rows = lambda cfg, **kwargs: rows
|
||||
return provider
|
||||
|
||||
|
||||
def test_get_realtime_normalizes_percent_source():
|
||||
provider = _realtime_provider([
|
||||
_ROWS = [
|
||||
{"code": "S1", "price": 10.0, "pre_close": 9.85, "pct": 1.52, "amp": 2.4, "turnover": 1.1},
|
||||
{"code": "S2", "price": 20.0, "pre_close": 20.44, "pct": -2.15, "amp": 3.1, "turnover": 0.8},
|
||||
{"code": "S3", "price": 30.0, "pre_close": 29.8, "pct": 0.67, "amp": 1.9, "turnover": 0.5},
|
||||
{"code": "S4", "price": 40.0, "pre_close": 38.9, "pct": 2.83, "amp": 4.2, "turnover": 2.0},
|
||||
{"code": "S5", "price": 50.0, "pre_close": 50.55, "pct": -1.09, "amp": 2.0, "turnover": 0.9},
|
||||
{"code": "S6", "price": 60.0, "pre_close": 59.64, "pct": 0.60, "amp": 1.6, "turnover": 0.7},
|
||||
])
|
||||
]
|
||||
|
||||
|
||||
def test_get_realtime_declared_percent_source():
|
||||
provider = _realtime_provider(_ROWS, pct_unit="percent")
|
||||
try:
|
||||
rows = provider.get_realtime()
|
||||
finally:
|
||||
@@ -101,3 +192,120 @@ def test_get_realtime_normalizes_percent_source():
|
||||
assert by_sym["S1"]["amplitude"] == pytest.approx(0.024)
|
||||
assert by_sym["S1"]["turnover_rate"] == pytest.approx(0.011)
|
||||
assert by_sym["S2"]["change_pct"] == pytest.approx(-0.0215)
|
||||
|
||||
|
||||
def test_get_realtime_undeclared_nulls_ambiguous_columns():
|
||||
provider = _realtime_provider(_ROWS)
|
||||
try:
|
||||
rows = provider.get_realtime()
|
||||
finally:
|
||||
provider.close()
|
||||
by_sym = {r["symbol"]: r for r in rows}
|
||||
# change_pct 截面判定仍归一
|
||||
assert by_sym["S1"]["change_pct"] == pytest.approx(0.0152)
|
||||
# 不可判定列 fail-closed
|
||||
assert by_sym["S1"]["amplitude"] is None
|
||||
assert by_sym["S1"]["turnover_rate"] is None
|
||||
|
||||
|
||||
def test_get_realtime_transformed_turnover_kept():
|
||||
provider = _realtime_provider(_ROWS, transforms={"turnover_rate": "value / 100"})
|
||||
try:
|
||||
rows = provider.get_realtime()
|
||||
finally:
|
||||
provider.close()
|
||||
by_sym = {r["symbol"]: r for r in rows}
|
||||
assert by_sym["S1"]["turnover_rate"] == pytest.approx(0.011)
|
||||
assert by_sym["S1"]["amplitude"] is None
|
||||
|
||||
|
||||
# ---- 配置解析与校验 ----
|
||||
|
||||
|
||||
def test_config_parses_pct_unit():
|
||||
cfg = config_from_dict(
|
||||
{
|
||||
"name": "s",
|
||||
"datasets": {
|
||||
"realtime": {
|
||||
"url": "https://example.test",
|
||||
"pct_unit": "Percent",
|
||||
}
|
||||
},
|
||||
}
|
||||
)
|
||||
assert cfg.datasets["realtime"].pct_unit == "percent"
|
||||
|
||||
|
||||
def test_config_rejects_invalid_pct_unit():
|
||||
with pytest.raises(ValueError, match="pct_unit"):
|
||||
config_from_dict(
|
||||
{
|
||||
"name": "s",
|
||||
"datasets": {
|
||||
"realtime": {
|
||||
"url": "https://example.test",
|
||||
"pct_unit": "basis_point",
|
||||
}
|
||||
},
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def test_validate_flags_pct_unit_on_non_realtime():
|
||||
provider = GenericHTTPProvider(
|
||||
CustomSourceConfig(
|
||||
name="s",
|
||||
display_name="S",
|
||||
datasets={
|
||||
"daily": DatasetConfig(
|
||||
url="https://example.test",
|
||||
field_map={
|
||||
"c": "symbol",
|
||||
"d": "date",
|
||||
"o": "open",
|
||||
"h": "high",
|
||||
"l": "low",
|
||||
"cl": "close",
|
||||
"v": "volume",
|
||||
"a": "amount",
|
||||
},
|
||||
pct_unit="percent",
|
||||
)
|
||||
},
|
||||
)
|
||||
)
|
||||
try:
|
||||
errors = provider.validate()
|
||||
finally:
|
||||
provider.close()
|
||||
assert any("pct_unit" in e and "realtime" in e for e in errors)
|
||||
|
||||
|
||||
def test_validate_flags_invalid_pct_unit_value():
|
||||
provider = GenericHTTPProvider(
|
||||
CustomSourceConfig(
|
||||
name="s",
|
||||
display_name="S",
|
||||
datasets={
|
||||
"realtime": DatasetConfig(
|
||||
url="https://example.test",
|
||||
field_map={
|
||||
"c": "symbol",
|
||||
"p": "last_price",
|
||||
"pc": "prev_close",
|
||||
"o": "open",
|
||||
"h": "high",
|
||||
"l": "low",
|
||||
"v": "volume",
|
||||
},
|
||||
pct_unit="bp",
|
||||
)
|
||||
},
|
||||
)
|
||||
)
|
||||
try:
|
||||
errors = provider.validate()
|
||||
finally:
|
||||
provider.close()
|
||||
assert any("pct_unit" in e for e in errors)
|
||||
|
||||
@@ -0,0 +1,158 @@
|
||||
"""板块分时 (dimension-intraday) 纯函数测试。
|
||||
|
||||
夹具: snapshot 扩展配置 (所属概念) + kline_minute/kline_daily 分区,
|
||||
验证等权口径、停牌 ffill、prev_close/首根基准与各降级状态。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
|
||||
import polars as pl
|
||||
|
||||
from app.api.ext_data import _dimension_intraday_compute
|
||||
from app.services.ext_data import ExtConfig
|
||||
|
||||
|
||||
def _mk_config() -> ExtConfig:
|
||||
return ExtConfig(id="ext_gn", label="测试概念", mode="snapshot", fields=[])
|
||||
|
||||
|
||||
def _write_ext(data_dir: Path, values: dict[str, str]) -> None:
|
||||
"""snapshot 扩展数据: symbol → 所属概念 标签串。"""
|
||||
cfg_dir = data_dir / "ext_data" / "ext_gn"
|
||||
cfg_dir.mkdir(parents=True, exist_ok=True)
|
||||
df = pl.DataFrame({
|
||||
"symbol": list(values.keys()),
|
||||
"所属概念": list(values.values()),
|
||||
})
|
||||
df.write_parquet(cfg_dir / "part.parquet")
|
||||
|
||||
|
||||
def _write_minute(data_dir: Path, day: str, rows: list[tuple[str, str, float]]) -> None:
|
||||
part = data_dir / "kline_minute" / f"date={day}" / "part.parquet"
|
||||
part.parent.mkdir(parents=True, exist_ok=True)
|
||||
df = pl.DataFrame(
|
||||
{
|
||||
"symbol": [r[0] for r in rows],
|
||||
"datetime": [datetime.fromisoformat(r[1]) for r in rows],
|
||||
"close": [r[2] for r in rows],
|
||||
},
|
||||
schema_overrides={"datetime": pl.Datetime("us")},
|
||||
)
|
||||
df.write_parquet(part)
|
||||
|
||||
|
||||
def _write_daily(data_dir: Path, day: str, closes: dict[str, float]) -> None:
|
||||
part = data_dir / "kline_daily" / f"date={day}" / "part.parquet"
|
||||
part.parent.mkdir(parents=True, exist_ok=True)
|
||||
pl.DataFrame({
|
||||
"symbol": list(closes.keys()),
|
||||
"close": list(closes.values()),
|
||||
}).write_parquet(part)
|
||||
|
||||
|
||||
def test_dimension_intraday_equal_weight_and_ffill(tmp_path: Path) -> None:
|
||||
data_dir = tmp_path / "data"
|
||||
_write_ext(data_dir, {"000001.SZ": "人工智能、芯片", "000002.SZ": "人工智能"})
|
||||
# 前收: 000001=10.0 (+5%/+6%/+4%), 000002=20.0, 600000 非成分股
|
||||
_write_daily(data_dir, "2026-08-27", {"000001.SZ": 10.0, "000002.SZ": 20.0, "600000.SH": 5.0})
|
||||
# 000002 在 09:32 无成交 (停牌分钟) → ffill 沿用 20.4
|
||||
_write_minute(data_dir, "2026-08-28", [
|
||||
("000001.SZ", "2026-08-28T09:31:00", 10.5),
|
||||
("000002.SZ", "2026-08-28T09:31:00", 20.4),
|
||||
("600000.SH", "2026-08-28T09:31:00", 5.05),
|
||||
("000001.SZ", "2026-08-28T09:32:00", 10.6),
|
||||
("600000.SH", "2026-08-28T09:32:00", 5.10),
|
||||
("000001.SZ", "2026-08-28T09:33:00", 10.4),
|
||||
("000002.SZ", "2026-08-28T09:33:00", 20.8),
|
||||
("600000.SH", "2026-08-28T09:33:00", 5.20),
|
||||
])
|
||||
|
||||
payload = _dimension_intraday_compute(_mk_config(), data_dir, "所属概念", "人工智能", None)
|
||||
|
||||
assert payload["status"] == "ok"
|
||||
assert payload["date"] == "2026-08-28"
|
||||
assert payload["basis"] == "prev_close"
|
||||
assert payload["member_count"] == 2
|
||||
assert payload["members_with_minute"] == 2
|
||||
points = payload["points"]
|
||||
assert [p["time"] for p in points] == ["09:31", "09:32", "09:33"]
|
||||
# 09:31: 成分等权 (5% + 2%)/2 = 3.5%; 全市场 (5+2+1)/3 ≈ 2.667% (小数制)
|
||||
assert points[0]["sector"] == 0.035
|
||||
assert points[0]["market"] == 0.0267
|
||||
# 09:32: 000002 ffill 20.4 → (6% + 2%)/2 = 4.0% (无 ffill 会是 6.0%)
|
||||
assert points[1]["sector"] == 0.04
|
||||
assert points[1]["market"] == 0.04
|
||||
# 09:33: (4% + 4%)/2 = 4.0%
|
||||
assert points[2]["sector"] == 0.04
|
||||
|
||||
|
||||
def test_dimension_intraday_tag_no_partial_match(tmp_path: Path) -> None:
|
||||
data_dir = tmp_path / "data"
|
||||
_write_ext(data_dir, {"000001.SZ": "人工智能体"})
|
||||
_write_minute(data_dir, "2026-08-28", [("000001.SZ", "2026-08-28T09:31:00", 10.5)])
|
||||
payload = _dimension_intraday_compute(_mk_config(), data_dir, "所属概念", "人工智能", None)
|
||||
assert payload["status"] == "empty"
|
||||
assert payload["reason"] == "no_members"
|
||||
|
||||
|
||||
def test_dimension_intraday_no_minute_store(tmp_path: Path) -> None:
|
||||
data_dir = tmp_path / "data"
|
||||
_write_ext(data_dir, {"000001.SZ": "人工智能"})
|
||||
payload = _dimension_intraday_compute(_mk_config(), data_dir, "所属概念", "人工智能", None)
|
||||
assert payload["status"] == "no_data"
|
||||
assert payload["reason"] == "minute_missing"
|
||||
|
||||
|
||||
def test_dimension_intraday_requested_date_absent(tmp_path: Path) -> None:
|
||||
data_dir = tmp_path / "data"
|
||||
_write_ext(data_dir, {"000001.SZ": "人工智能"})
|
||||
_write_minute(data_dir, "2026-08-28", [("000001.SZ", "2026-08-28T09:31:00", 10.5)])
|
||||
payload = _dimension_intraday_compute(_mk_config(), data_dir, "所属概念", "人工智能", "2026-08-27")
|
||||
assert payload["status"] == "no_data"
|
||||
assert payload["reason"] == "minute_missing"
|
||||
|
||||
|
||||
def test_dimension_intraday_members_without_bars(tmp_path: Path) -> None:
|
||||
"""成分股全是 ETF 等无分钟数据的标的 → empty/no_member_bars。"""
|
||||
data_dir = tmp_path / "data"
|
||||
_write_ext(data_dir, {"510050.SH": "人工智能"})
|
||||
_write_daily(data_dir, "2026-08-27", {"510050.SH": 3.0, "000001.SZ": 10.0})
|
||||
_write_minute(data_dir, "2026-08-28", [("000001.SZ", "2026-08-28T09:31:00", 10.5)])
|
||||
payload = _dimension_intraday_compute(_mk_config(), data_dir, "所属概念", "人工智能", None)
|
||||
assert payload["status"] == "empty"
|
||||
assert payload["reason"] == "no_member_bars"
|
||||
assert payload["member_count"] == 1
|
||||
|
||||
|
||||
def test_dimension_intraday_first_close_basis(tmp_path: Path) -> None:
|
||||
"""无前一交易日日K → 基准退化为当日首根 close, 曲线起点 ≈ 0。"""
|
||||
data_dir = tmp_path / "data"
|
||||
_write_ext(data_dir, {"000001.SZ": "人工智能"})
|
||||
_write_minute(data_dir, "2026-08-28", [
|
||||
("000001.SZ", "2026-08-28T09:31:00", 10.0),
|
||||
("000001.SZ", "2026-08-28T09:32:00", 10.3),
|
||||
])
|
||||
payload = _dimension_intraday_compute(_mk_config(), data_dir, "所属概念", "人工智能", None)
|
||||
assert payload["status"] == "ok"
|
||||
assert payload["basis"] == "first_close"
|
||||
assert payload["points"][0]["sector"] == 0.0
|
||||
assert payload["points"][1]["sector"] == 0.03
|
||||
|
||||
|
||||
def test_dimension_intraday_explicit_date_uses_that_partition(tmp_path: Path) -> None:
|
||||
data_dir = tmp_path / "data"
|
||||
_write_ext(data_dir, {"000001.SZ": "人工智能"})
|
||||
_write_daily(data_dir, "2026-08-26", {"000001.SZ": 10.0})
|
||||
_write_daily(data_dir, "2026-08-27", {"000001.SZ": 11.0})
|
||||
_write_minute(data_dir, "2026-08-27", [("000001.SZ", "2026-08-27T09:31:00", 10.5)])
|
||||
_write_minute(data_dir, "2026-08-28", [("000001.SZ", "2026-08-28T09:31:00", 12.1)])
|
||||
# 默认取最新分区 2026-08-28 → prev 为 08-27 的 11.0 → +10%
|
||||
latest = _dimension_intraday_compute(_mk_config(), data_dir, "所属概念", "人工智能", None)
|
||||
assert latest["date"] == "2026-08-28"
|
||||
assert latest["points"][0]["sector"] == 0.1
|
||||
# 显式指定 08-27 → prev 为 08-26 的 10.0 → +5%
|
||||
explicit = _dimension_intraday_compute(_mk_config(), data_dir, "所属概念", "人工智能", "2026-08-27")
|
||||
assert explicit["date"] == "2026-08-27"
|
||||
assert explicit["points"][0]["sector"] == 0.05
|
||||
@@ -0,0 +1,157 @@
|
||||
"""龙虎榜服务测试 (不依赖真实网络)。
|
||||
|
||||
覆盖: 交易日回退 (非交易日/目标日解析)、历史日 JSON 缓存命中与落盘、
|
||||
fuyao 未配置降级、当日未发布 fallback_prev、彻底失败 no_data、
|
||||
AI 复盘摘要段构建。
|
||||
日期用 2026-08-27/28 (写作时为过去交易日), 与仓库既有绝对日期测试风格一致。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
from datetime import date
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
from app.plugins.fuyao.client import FuyaoError
|
||||
from app.services import dragon_tiger as dt
|
||||
|
||||
|
||||
def _mk_days(data_dir: Path, *days: str) -> None:
|
||||
for d in days:
|
||||
(data_dir / "kline_daily" / f"date={d}").mkdir(parents=True, exist_ok=True)
|
||||
|
||||
|
||||
@pytest.fixture()
|
||||
def data_dir(tmp_path: Path) -> Path:
|
||||
_mk_days(tmp_path, "2026-08-26", "2026-08-27", "2026-08-28")
|
||||
return tmp_path
|
||||
|
||||
|
||||
class _FakeProvider:
|
||||
"""按 (board, date) 记录调用; fail_dates 中的日期抛 FuyaoError。"""
|
||||
|
||||
def __init__(self, fail_dates: set[str] | None = None):
|
||||
self.calls: list[tuple] = []
|
||||
self.fail_dates = fail_dates or set()
|
||||
|
||||
def dragon_tiger(self, board_type: str, date: str | None) -> dict:
|
||||
self.calls.append((board_type, date))
|
||||
if date in self.fail_dates:
|
||||
raise FuyaoError(f"code=3002: {date} 未就绪")
|
||||
iso = date or "2026-08-28"
|
||||
return {
|
||||
"trade_date": iso,
|
||||
"count": 2,
|
||||
"stock_count": 2,
|
||||
"stock_items": [
|
||||
{"thscode": "600519.SH", "ticker": "600519", "name": "贵州茅台",
|
||||
"change": 0.05, "net_value": 1.5e8, "net_rate": 0.01,
|
||||
"buy_value": 2e8, "sell_value": 0.5e8, "hot_rank": 3, "range_days": 1,
|
||||
"org_net_value": 0.8e8},
|
||||
{"thscode": "000858.SZ", "ticker": "000858", "name": "五粮液",
|
||||
"change": -0.03, "net_value": -0.6e8, "net_rate": -0.004,
|
||||
"buy_value": 0.4e8, "sell_value": 1e8, "range_days": 3},
|
||||
],
|
||||
"hot_money_items": [
|
||||
{"name": "宁波桑田路", "buying": 1.9e8,
|
||||
"rows": [{"thscode": "600519.SH", "name": "贵州茅台",
|
||||
"hot_money_item_net_value": 1.2e8}]},
|
||||
],
|
||||
}
|
||||
|
||||
|
||||
def _use_provider(monkeypatch, provider) -> _FakeProvider:
|
||||
monkeypatch.setattr(dt, "_provider", lambda: provider)
|
||||
return provider
|
||||
|
||||
|
||||
# ---- 交易日解析 ----
|
||||
|
||||
def test_resolve_rolls_back_non_trading_day(data_dir):
|
||||
# 周日 08-30 → 最近分区 08-28
|
||||
assert dt.resolve_trade_date(data_dir, date(2026, 8, 30)) == date(2026, 8, 28)
|
||||
# 08-27 是交易日 → 原样
|
||||
assert dt.resolve_trade_date(data_dir, date(2026, 8, 27)) == date(2026, 8, 27)
|
||||
|
||||
|
||||
def test_resolve_older_than_all_partitions_returns_target(data_dir):
|
||||
# 早于全部本地分区 → 原样返回, 由调用方显式尝试 (失败如实 no_data)
|
||||
assert dt.resolve_trade_date(data_dir, date(2020, 1, 1)) == date(2020, 1, 1)
|
||||
|
||||
|
||||
# ---- 状态与缓存 ----
|
||||
|
||||
def test_source_unavailable_without_fuyao(data_dir, monkeypatch):
|
||||
monkeypatch.setattr(dt, "_provider", lambda: None)
|
||||
out = dt.get_dragon_tiger(data_dir, None)
|
||||
assert out["state"] == "source_unavailable"
|
||||
|
||||
|
||||
def test_fetch_stores_cache_then_hits_cache(data_dir, monkeypatch):
|
||||
provider = _use_provider(monkeypatch, _FakeProvider())
|
||||
out = dt.get_dragon_tiger(data_dir, None) # 默认 → 最近分区 08-28
|
||||
assert out["state"] == "ok"
|
||||
assert out["trade_date"] == "2026-08-28"
|
||||
assert len(provider.calls) == 3 # 三榜一次取齐
|
||||
cache = data_dir / "dragon_tiger" / "date=2026-08-28.json"
|
||||
assert cache.exists()
|
||||
|
||||
provider.calls.clear()
|
||||
out2 = dt.get_dragon_tiger(data_dir, date(2026, 8, 30)) # 周日 → 08-28 → 命中缓存
|
||||
assert out2["state"] == "ok" and out2["trade_date"] == "2026-08-28"
|
||||
assert provider.calls == [] # 纯本地, 不打网络
|
||||
|
||||
|
||||
def test_explicit_history_date_uses_cache(data_dir, monkeypatch):
|
||||
provider = _use_provider(monkeypatch, _FakeProvider())
|
||||
dt.get_dragon_tiger(data_dir, date(2026, 8, 27))
|
||||
assert provider.calls[0] == ("all", "2026-08-27")
|
||||
provider.calls.clear()
|
||||
dt.get_dragon_tiger(data_dir, date(2026, 8, 27))
|
||||
assert provider.calls == []
|
||||
|
||||
|
||||
def test_unpublished_today_falls_back_to_prev(data_dir, monkeypatch):
|
||||
# 08-28 拉取失败 (未发布/未就绪) → 回退 08-27, state=fallback_prev
|
||||
provider = _use_provider(monkeypatch, _FakeProvider(fail_dates={"2026-08-28"}))
|
||||
out = dt.get_dragon_tiger(data_dir, date(2026, 8, 28))
|
||||
assert out["state"] == "fallback_prev"
|
||||
assert out["trade_date"] == "2026-08-27"
|
||||
assert out["requested_date"] == "2026-08-28"
|
||||
# 回退日缓存以 ok 状态落盘 (不污染直查)
|
||||
cached = json.loads((data_dir / "dragon_tiger" / "date=2026-08-27.json").read_text(encoding="utf-8"))
|
||||
assert cached["state"] == "ok"
|
||||
|
||||
|
||||
def test_total_failure_returns_no_data(data_dir, monkeypatch):
|
||||
provider = _use_provider(monkeypatch, _FakeProvider(fail_dates={"2026-08-28", "2026-08-27"}))
|
||||
out = dt.get_dragon_tiger(data_dir, date(2026, 8, 28))
|
||||
assert out["state"] == "no_data"
|
||||
assert "2026-08-28" in out.get("message", "")
|
||||
|
||||
|
||||
def test_corrupt_cache_refetches(data_dir, monkeypatch):
|
||||
provider = _use_provider(monkeypatch, _FakeProvider())
|
||||
cache = data_dir / "dragon_tiger" / "date=2026-08-28.json"
|
||||
cache.parent.mkdir(parents=True, exist_ok=True)
|
||||
cache.write_text("{broken json", encoding="utf-8")
|
||||
out = dt.get_dragon_tiger(data_dir, date(2026, 8, 28))
|
||||
assert out["state"] == "ok"
|
||||
assert provider.calls # 缓存损坏 → 重新拉取
|
||||
|
||||
|
||||
# ---- AI 复盘摘要 ----
|
||||
|
||||
def test_build_recap_context_contains_summary(data_dir, monkeypatch):
|
||||
_use_provider(monkeypatch, _FakeProvider())
|
||||
ctx = dt.build_recap_context(data_dir)
|
||||
assert "净买入居前" in ctx and "贵州茅台" in ctx
|
||||
assert "机构净买居前" in ctx
|
||||
assert "宁波桑田路" in ctx
|
||||
|
||||
|
||||
def test_build_recap_context_empty_without_source(data_dir, monkeypatch):
|
||||
monkeypatch.setattr(dt, "_provider", lambda: None)
|
||||
assert dt.build_recap_context(data_dir) == ""
|
||||
@@ -0,0 +1,52 @@
|
||||
"""全量分钟能力 (Cap.INTRADAY_UNIVERSE) 契约。
|
||||
|
||||
- 能力位存在且值为 "intraday.universe"
|
||||
- tiers.yaml 仅 expert 档声明该能力 (Pro/自定义源天然没有)
|
||||
- 探测层注册了该能力的探测调用, 显示标签为「全量分钟」
|
||||
- 缓存 schema 已 bump (旧 capabilities.json 触发重探测)
|
||||
- 盘中分钟服务门控挂在该能力位上
|
||||
"""
|
||||
from pathlib import Path
|
||||
from types import SimpleNamespace
|
||||
|
||||
import polars as pl
|
||||
|
||||
from app.services.minute_refresh import MinuteRefreshService
|
||||
from app.tickflow.capabilities import Cap, CapabilityLimits, CapabilitySet
|
||||
from app.tickflow.policy import _CACHE_SCHEMA_VERSION, _CAP_ALIASES, _load_tiers_yaml
|
||||
|
||||
|
||||
def test_capability_enum_value():
|
||||
assert Cap("intraday.universe") is Cap.INTRADAY_UNIVERSE
|
||||
|
||||
|
||||
def test_tiers_yaml_grants_universe_to_expert_only():
|
||||
tiers = _load_tiers_yaml()
|
||||
assert "intraday.universe" in tiers["expert"]
|
||||
for tier in ("free", "starter", "pro"):
|
||||
assert "intraday.universe" not in tiers[tier]
|
||||
|
||||
|
||||
def test_policy_labels_and_cache_schema():
|
||||
assert _CAP_ALIASES[Cap.INTRADAY_UNIVERSE] == "全量分钟"
|
||||
assert _CACHE_SCHEMA_VERSION >= 6
|
||||
|
||||
|
||||
def test_service_gate_requires_universe_not_batch_alone():
|
||||
class _Repo:
|
||||
store = SimpleNamespace(data_dir=Path("."))
|
||||
|
||||
def get_instruments(self) -> pl.DataFrame:
|
||||
return pl.DataFrame({"symbol": []})
|
||||
|
||||
def _svc_with(caps: dict) -> MinuteRefreshService:
|
||||
svc = MinuteRefreshService(_Repo())
|
||||
svc.set_app_state(SimpleNamespace(capabilities=CapabilitySet(caps)))
|
||||
return svc
|
||||
|
||||
universe = {Cap.INTRADAY_UNIVERSE: CapabilityLimits(rpm=20)}
|
||||
batch_only = {Cap.INTRADAY_BATCH: CapabilityLimits(rpm=60, batch=200)}
|
||||
|
||||
assert _svc_with(universe).capability_ok() is True
|
||||
assert _svc_with(batch_only).capability_ok() is False # 仅有 intraday.batch 不放行
|
||||
assert _svc_with({}).capability_ok() is False
|
||||
@@ -0,0 +1,242 @@
|
||||
"""fuyao 财务适配测试 (不依赖真实网络)。
|
||||
|
||||
覆盖: 三大报表字段映射 (canonical 列名 + 扩展列透传 + ISO 日期口径)、
|
||||
latest_only 分档 (limit 1 vs 8)、metrics 组装 (eps_basic 顺带 / bps 估值反推 /
|
||||
指标 index_id 映射与未知 id 透传 / 单股指标失败不弃行)、shares 恒空、
|
||||
报告期合并写入的逐列填空语义 (并集共存, 新行缺列不覆盖旧值)。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import polars as pl
|
||||
import pytest
|
||||
|
||||
from app.plugins.fuyao import client as fc
|
||||
from app.plugins.fuyao import provider as fp
|
||||
from app.plugins.fuyao.provider import FuyaoProvider
|
||||
from app.services.financial_sync import _merge_report_history
|
||||
|
||||
|
||||
class _FakeFinClient:
|
||||
"""财务端点假客户端: 记录调用入参, 按表返回预置行。"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
statements: dict[str, list[dict]] | None = None,
|
||||
indicators: dict[str, list[dict]] | None = None,
|
||||
indicator_error: Exception | None = None,
|
||||
valuations: list[dict] | None = None,
|
||||
prices: list[dict] | None = None,
|
||||
):
|
||||
self.statements = statements or {}
|
||||
self.indicators = indicators or {}
|
||||
self.indicator_error = indicator_error
|
||||
self.valuations = valuations or []
|
||||
self.prices = prices or []
|
||||
self.stmt_calls: list[tuple] = []
|
||||
self.ind_calls: list[str] = []
|
||||
|
||||
def financial_statements(self, stmt, thscode, limit=1):
|
||||
self.stmt_calls.append((stmt, thscode, limit))
|
||||
return [dict(r, thscode=thscode) for r in self.statements.get(stmt, [])]
|
||||
|
||||
def financial_indicators(self, thscode, report):
|
||||
self.ind_calls.append(f"{thscode}@{report}")
|
||||
if self.indicator_error:
|
||||
raise self.indicator_error
|
||||
return self.indicators.get(report, [])
|
||||
|
||||
def valuations_snapshot(self, thscodes):
|
||||
return [r for r in self.valuations if r.get("thscode") in thscodes]
|
||||
|
||||
def price_snapshot_batch(self, thscodes):
|
||||
return [r for r in self.prices if r.get("thscode") in thscodes]
|
||||
|
||||
|
||||
def _provider_with(monkeypatch, fake):
|
||||
monkeypatch.setattr(
|
||||
fp, "fuyao_client", type("M", (), {"FuyaoClient": lambda **kw: fake})
|
||||
)
|
||||
monkeypatch.setattr(fp, "get_api_key", lambda: "test-key")
|
||||
monkeypatch.setattr(fp, "_HIST_INTERVAL_S", 0)
|
||||
return FuyaoProvider()
|
||||
|
||||
|
||||
# period_end_ms: 2026-06-30 上海零点; report_date_ms: 2026-08-15 上海零点
|
||||
_INCOME_ROW = {
|
||||
"period": "quarterly",
|
||||
"fiscal_year": 2026,
|
||||
"fiscal_period": "Q2",
|
||||
"report_date_ms": 1786723200000,
|
||||
"period_end_ms": 1782748800000,
|
||||
"currency": "CNY",
|
||||
"operating_income": 90703260964.48,
|
||||
"net_profit": 46033330566.78,
|
||||
"parent_holder_net_profit": 44516880421.86,
|
||||
"basic_eps": 35.57,
|
||||
"operating_expenses": 50000000000.0, # 扶摇独有 → 扩展列
|
||||
}
|
||||
|
||||
|
||||
def test_income_mapping_canonical_columns(monkeypatch):
|
||||
fake = _FakeFinClient(statements={"income": [_INCOME_ROW]})
|
||||
provider = _provider_with(monkeypatch, fake)
|
||||
df = provider.get_financials("income", ["600519.SH"], latest_only=True)
|
||||
row = df.to_dicts()[0]
|
||||
assert row["symbol"] == "600519.SH"
|
||||
assert row["period_end"] == "2026-06-30"
|
||||
assert row["announce_date"] == "2026-08-15"
|
||||
assert row["revenue"] == pytest.approx(_INCOME_ROW["operating_income"])
|
||||
assert row["net_income"] == pytest.approx(_INCOME_ROW["net_profit"])
|
||||
assert row["net_income_attributable"] == pytest.approx(
|
||||
_INCOME_ROW["parent_holder_net_profit"]
|
||||
)
|
||||
assert row["basic_eps"] == 35.57
|
||||
# 扩展列: 扶摇独有数字字段原名透传; 字符串元数据不透传
|
||||
assert row["operating_expenses"] == 50000000000.0
|
||||
assert "thscode" not in df.columns and "ticker" not in df.columns
|
||||
# 原始名不残留 (已映射字段)
|
||||
assert "operating_income" not in df.columns and "net_profit" not in df.columns
|
||||
|
||||
|
||||
def test_statements_limit_latest_vs_history(monkeypatch):
|
||||
fake = _FakeFinClient(statements={"income": [_INCOME_ROW]})
|
||||
provider = _provider_with(monkeypatch, fake)
|
||||
provider.get_financials("income", ["600519.SH"], latest_only=True)
|
||||
assert fake.stmt_calls == [("income", "600519.SH", 1)]
|
||||
provider.get_financials("income", ["600519.SH"], latest_only=False)
|
||||
assert fake.stmt_calls[-1] == ("income", "600519.SH", fp._FINANCIAL_HISTORY_PERIODS)
|
||||
|
||||
|
||||
def test_balance_and_cashflow_mapping(monkeypatch):
|
||||
balance = {
|
||||
"period_end_ms": 1782748800000,
|
||||
"report_date_ms": 1786723200000,
|
||||
"assets_total": 309050784569.31,
|
||||
"total_debt": 46954432394.95,
|
||||
"holder_equity_total": 262096352174.36,
|
||||
}
|
||||
cashflow = {
|
||||
"period_end_ms": 1782748800000,
|
||||
"report_date_ms": 1786723200000,
|
||||
"act_cash_flow_net": 92000000000.0,
|
||||
"invest_cash_flow_net": -3000000000.0,
|
||||
"pay_dividends_profits_interest_cash": 64000000000.0, # 扶摇独有 → 扩展列
|
||||
}
|
||||
fake = _FakeFinClient(
|
||||
statements={"balance_sheet": [balance], "cash_flow": [cashflow]}
|
||||
)
|
||||
provider = _provider_with(monkeypatch, fake)
|
||||
bal = provider.get_financials("balance_sheet", ["600519.SH"]).to_dicts()[0]
|
||||
assert bal["total_assets"] == pytest.approx(balance["assets_total"])
|
||||
assert bal["total_liabilities"] == pytest.approx(balance["total_debt"])
|
||||
assert bal["total_equity"] == pytest.approx(balance["holder_equity_total"])
|
||||
cf = provider.get_financials("cash_flow", ["600519.SH"]).to_dicts()[0]
|
||||
assert cf["net_operating_cash_flow"] == pytest.approx(92000000000.0)
|
||||
assert cf["net_investing_cash_flow"] == pytest.approx(-3000000000.0)
|
||||
assert cf["pay_dividends_profits_interest_cash"] == pytest.approx(64000000000.0)
|
||||
|
||||
|
||||
_METRICS_ABILITIES = [
|
||||
{
|
||||
"ability": "profitability",
|
||||
"indicators": [
|
||||
{"index_id": "index_weighted_avg_roe", "value": "16.7500"},
|
||||
{"index_id": "sale_gross_margin", "value": "89.5552"},
|
||||
],
|
||||
},
|
||||
{
|
||||
"ability": "growth",
|
||||
# 实测 index_id 与文档有出入; 未映射 id 原名透传
|
||||
"indicators": [{"index_id": "fixed_asset_invest_expansion_ratio", "value": "2.12587300"}],
|
||||
},
|
||||
{
|
||||
"ability": "solvency",
|
||||
"indicators": [{"index_id": "earned_interest_multiple", "value": None}], # null → 不写列
|
||||
},
|
||||
]
|
||||
|
||||
|
||||
def test_metrics_assembly(monkeypatch):
|
||||
fake = _FakeFinClient(
|
||||
statements={"income": [_INCOME_ROW]},
|
||||
indicators={"2026-2": _METRICS_ABILITIES},
|
||||
valuations=[{"thscode": "600519.SH", "pb_mrq": 6.455055}],
|
||||
prices=[{"thscode": "600519.SH", "last_price": 1297.4}],
|
||||
)
|
||||
provider = _provider_with(monkeypatch, fake)
|
||||
df = provider.get_financials("metrics", ["600519.SH"], latest_only=True)
|
||||
row = df.to_dicts()[0]
|
||||
assert row["period_end"] == "2026-06-30"
|
||||
assert row["announce_date"] == "2026-08-15"
|
||||
assert row["eps_basic"] == 35.57 # 顺带取自利润表
|
||||
assert row["bps"] == pytest.approx(1297.4 / 6.455055) # 估值反推
|
||||
assert row["roe"] == pytest.approx(16.75) # 字符串 → float
|
||||
assert row["gross_margin"] == pytest.approx(89.5552)
|
||||
assert row["fixed_asset_invest_expansion_ratio"] == pytest.approx(2.125873)
|
||||
assert "earned_interest_multiple" not in df.columns # 全空指标不成列
|
||||
assert fake.ind_calls == ["600519.SH@2026-2"] # report 由利润表最新期反推
|
||||
|
||||
|
||||
def test_metrics_indicator_failure_keeps_row(monkeypatch):
|
||||
"""指标端点单股失败 (如未披露期 code=5003) → 行仍写入 (eps/bps 保留)。"""
|
||||
fake = _FakeFinClient(
|
||||
statements={"income": [_INCOME_ROW]},
|
||||
indicator_error=fc.FuyaoError("code=5003"),
|
||||
valuations=[{"thscode": "600519.SH", "pb_mrq": 6.455055}],
|
||||
prices=[{"thscode": "600519.SH", "last_price": 1297.4}],
|
||||
)
|
||||
provider = _provider_with(monkeypatch, fake)
|
||||
df = provider.get_financials("metrics", ["600519.SH"], latest_only=True)
|
||||
row = df.to_dicts()[0]
|
||||
assert row["symbol"] == "600519.SH"
|
||||
assert row["eps_basic"] == 35.57
|
||||
assert "roe" not in df.columns
|
||||
|
||||
|
||||
def test_metrics_skips_symbol_without_income(monkeypatch):
|
||||
fake = _FakeFinClient(statements={"income": []})
|
||||
provider = _provider_with(monkeypatch, fake)
|
||||
assert provider.get_financials("metrics", ["600519.SH"]).is_empty()
|
||||
|
||||
|
||||
def test_shares_returns_empty(monkeypatch):
|
||||
provider = _provider_with(monkeypatch, _FakeFinClient())
|
||||
assert provider.get_financials("shares", ["600519.SH"]).is_empty()
|
||||
|
||||
|
||||
def test_merge_fills_missing_cells_from_old_rows():
|
||||
"""逐列填空: 同报告期新行缺的列由旧行补齐, 有值则覆盖 (并集共存语义)。"""
|
||||
old = pl.DataFrame({
|
||||
"symbol": ["600519.SH", "600519.SH"],
|
||||
"period_end": ["2026-03-31", "2026-06-30"],
|
||||
"announce_date": ["2026-04-20", "2026-08-10"],
|
||||
"diluted_eps": [68.1, 70.2], # tickflow 提供, fuyao 没有
|
||||
"net_income": [280.0, 460.0],
|
||||
})
|
||||
new = pl.DataFrame({
|
||||
"symbol": ["600519.SH"],
|
||||
"period_end": ["2026-06-30"],
|
||||
"announce_date": ["2026-08-15"], # 更晚公告 → 该期以新行为基准
|
||||
"net_income": [461.5], # 修正值覆盖
|
||||
# diluted_eps 缺失 → 由旧行 70.2 补齐
|
||||
"bps": [200.99], # fuyao 扩展列, 旧行没有
|
||||
})
|
||||
merged = _merge_report_history(old, new).to_dicts()
|
||||
assert len(merged) == 2
|
||||
q2 = next(r for r in merged if r["period_end"] == "2026-06-30")
|
||||
assert q2["net_income"] == pytest.approx(461.5) # 新值覆盖
|
||||
assert q2["diluted_eps"] == pytest.approx(70.2) # 旧行补齐
|
||||
assert q2["bps"] == pytest.approx(200.99) # 扩展列并入
|
||||
q1 = next(r for r in merged if r["period_end"] == "2026-03-31")
|
||||
assert q1["diluted_eps"] == pytest.approx(68.1) # 未触碰期原样保留
|
||||
# 旧公告覆盖新公告的倒序场景: announce 早的行不覆盖晚的
|
||||
reversed_new = pl.DataFrame({
|
||||
"symbol": ["600519.SH"],
|
||||
"period_end": ["2026-06-30"],
|
||||
"announce_date": ["2026-08-01"],
|
||||
"net_income": [999.0],
|
||||
})
|
||||
q2b = _merge_report_history(old, reversed_new).to_dicts()[1]
|
||||
# 公告更晚的 old 行 (08-10) 胜出, 早公告的新行不覆盖 → 业绩修正以最新公告为准
|
||||
assert q2b["net_income"] == pytest.approx(460.0)
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,90 @@
|
||||
"""fetch_intraday_full_market_burst 单块容错契约。
|
||||
|
||||
一个块失败不得拖垮整轮: 成功块必须照常返回供落盘; 失败块单独重试一次;
|
||||
失败块过多 (系统性故障) 时跳过重试。全部用假 client, 不发真实网络请求。
|
||||
"""
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import patch
|
||||
|
||||
import polars as pl
|
||||
|
||||
from app.services import kline_sync
|
||||
from app.tickflow.capabilities import Cap, CapabilityLimits, CapabilitySet
|
||||
|
||||
|
||||
def _capset(batch: int = 2) -> CapabilitySet:
|
||||
return CapabilitySet({Cap.INTRADAY_BATCH: CapabilityLimits(rpm=60, batch=batch)})
|
||||
|
||||
|
||||
def _frame() -> pl.DataFrame:
|
||||
# _normalize_minute 的最小输入: 毫秒 timestamp → 北京墙钟 datetime。
|
||||
# 时间必须落在交易时段 (时区契约守卫会拒绝非交易小时的脏数据)
|
||||
from datetime import datetime
|
||||
from zoneinfo import ZoneInfo
|
||||
ts = int(datetime(2026, 8, 28, 9, 31, tzinfo=ZoneInfo("Asia/Shanghai")).timestamp() * 1000)
|
||||
return pl.DataFrame({
|
||||
"timestamp": [ts],
|
||||
"open": [1.0], "high": [1.0], "low": [1.0], "close": [1.0],
|
||||
"volume": [100.0], "amount": [100.0],
|
||||
})
|
||||
|
||||
|
||||
class _FakeKlines:
|
||||
"""intraday_batch 假实现: fail_once 首次失败重试成功, fail_always 恒失败。"""
|
||||
|
||||
def __init__(self, fail_once: set[str], fail_always: set[str]) -> None:
|
||||
self.fail_once = set(fail_once)
|
||||
self.fail_always = set(fail_always)
|
||||
self.calls: list[list[str]] = []
|
||||
|
||||
def intraday_batch(self, chunk, **kwargs):
|
||||
self.calls.append(list(chunk))
|
||||
syms = set(chunk)
|
||||
if syms & self.fail_always:
|
||||
raise RuntimeError("permanent failure")
|
||||
if syms & self.fail_once:
|
||||
self.fail_once -= syms
|
||||
raise RuntimeError("transient failure")
|
||||
return {s: _frame() for s in chunk}
|
||||
|
||||
|
||||
def _run(symbols: list[str], fake: _FakeKlines, batch: int = 2):
|
||||
client = SimpleNamespace(klines=fake)
|
||||
with patch.object(kline_sync, "get_client", return_value=client):
|
||||
return kline_sync.fetch_intraday_full_market_burst(symbols, _capset(batch))
|
||||
|
||||
|
||||
def test_transient_chunk_failure_retried_and_all_symbols_returned():
|
||||
symbols = [f"S{i}" for i in range(6)] # 3 chunks (batch=2)
|
||||
fake = _FakeKlines(fail_once={"S0", "S1"}, fail_always=set())
|
||||
df, requests = _run(symbols, fake)
|
||||
# 失败块 (S0,S1) 重试后成功 → 6 只全在, 请求数 = 3 块 + 1 次重试
|
||||
assert set(df["symbol"].to_list()) == set(symbols)
|
||||
assert requests == 4
|
||||
|
||||
|
||||
def test_permanent_chunk_failure_skipped_without_losing_other_chunks():
|
||||
symbols = [f"S{i}" for i in range(6)]
|
||||
fake = _FakeKlines(fail_once=set(), fail_always={"S4", "S5"})
|
||||
df, requests = _run(symbols, fake)
|
||||
# 失败块重试仍失败 → 只跳过该块, 其余 4 只必须返回 (旧实现会整轮丢弃)
|
||||
assert set(df["symbol"].to_list()) == {"S0", "S1", "S2", "S3"}
|
||||
assert requests == 4
|
||||
|
||||
|
||||
def test_all_chunks_succeed_requests_equals_chunk_count():
|
||||
symbols = [f"S{i}" for i in range(6)]
|
||||
fake = _FakeKlines(fail_once=set(), fail_always=set())
|
||||
df, requests = _run(symbols, fake)
|
||||
assert set(df["symbol"].to_list()) == set(symbols)
|
||||
assert requests == 3
|
||||
|
||||
|
||||
def test_systemic_failure_skips_retry_to_avoid_pressuring_overloaded_server():
|
||||
symbols = [f"S{i}" for i in range(12)] # 6 chunks (batch=2)
|
||||
# 5 个块恒失败 (>4) → 系统性故障, 不再重试
|
||||
fake = _FakeKlines(fail_once=set(), fail_always={s for s in symbols if s not in ("S0", "S1")})
|
||||
df, requests = _run(symbols, fake)
|
||||
assert set(df["symbol"].to_list()) == {"S0", "S1"}
|
||||
assert requests == 6 # 无重试
|
||||
assert len(fake.calls) == 6
|
||||
@@ -0,0 +1,125 @@
|
||||
"""回归测试: job 记录跨进程死亡持久化(「数据在、记录丢」补丁)。
|
||||
|
||||
背景(用户反馈): 全市场同步 12:11~12:42 成功结束后 0.7s, uvicorn --reload
|
||||
检测到代码变更杀死 worker, 恰好落在管道完成与 job_store.succeed() 落盘之间
|
||||
—— 数据已写盘但同步历史无任何记录。旧实现 pending/running 仅存内存、终态才
|
||||
落盘, 存在整段丢失窗口。
|
||||
|
||||
修复后契约:
|
||||
- create()/start() 即落盘 pending/running 快照;
|
||||
- 下次进程启动(= 新 JobStore 实例, 同目录)把遗留的 pending/running
|
||||
孤儿记录补标为 failed(中断), finished_at 取文件 mtime;
|
||||
- 终态记录不受补录影响; 终态写入覆盖 running 快照(同一文件)。
|
||||
均为纯逻辑, 不触网。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
|
||||
from app.services.pipeline_jobs import JobStore
|
||||
|
||||
|
||||
def _read_disk(d, jid: str) -> dict:
|
||||
return json.loads((d / f"{jid}.json").read_text("utf-8"))
|
||||
|
||||
|
||||
# ── 创建/启动即落盘 ──────────────────────────────────────────────────────
|
||||
|
||||
def test_create_writes_pending_snapshot_to_disk(tmp_path):
|
||||
d = tmp_path / "jobs"
|
||||
store = JobStore(store_dir=d)
|
||||
jid, _ = store.create(timeout_s=60)
|
||||
|
||||
disk = _read_disk(d, jid)
|
||||
assert disk["status"] == "pending"
|
||||
assert disk["stage"] == "init"
|
||||
|
||||
|
||||
def test_start_updates_disk_snapshot_to_running(tmp_path):
|
||||
d = tmp_path / "jobs"
|
||||
store = JobStore(store_dir=d)
|
||||
jid, _ = store.create(timeout_s=60)
|
||||
store.start(jid)
|
||||
|
||||
disk = _read_disk(d, jid)
|
||||
assert disk["status"] == "running"
|
||||
assert disk["started_at"] is not None
|
||||
|
||||
|
||||
# ── 进程死亡 → 下次启动补录 ──────────────────────────────────────────────
|
||||
|
||||
def test_orphan_running_record_is_reaped_on_next_boot(tmp_path):
|
||||
"""核心场景: 进程死在 running(甚至工作已做完但未终态), 记录必须可见。"""
|
||||
d = tmp_path / "jobs"
|
||||
dead = JobStore(store_dir=d)
|
||||
jid, _ = dead.create(timeout_s=60)
|
||||
dead.start(jid)
|
||||
dead.progress(jid, "sync", 50, "halfway") # 进度只更新内存
|
||||
|
||||
# 新进程 = 同目录新实例(内存为空, 只有磁盘)
|
||||
revived = JobStore(store_dir=d)
|
||||
j = revived.get(jid)
|
||||
assert j is not None
|
||||
assert j["status"] == "failed"
|
||||
assert "中断" in j["error"]
|
||||
assert j["finished_at"] is not None
|
||||
# finished_at 基于文件 mtime(≈ start 时刻), 时长不得虚增为负或巨大
|
||||
assert j["duration_s"] is not None
|
||||
assert 0 <= j["duration_s"] <= 60
|
||||
# 同步历史列表可见
|
||||
assert any(x["id"] == jid for x in revived.list_recent())
|
||||
|
||||
|
||||
def test_orphan_pending_record_is_reaped(tmp_path):
|
||||
"""进程死在 create() 与 start() 之间: 记录同样可见, 时长为 None。"""
|
||||
d = tmp_path / "jobs"
|
||||
dead = JobStore(store_dir=d)
|
||||
jid, _ = dead.create(timeout_s=60)
|
||||
# 未 start 即死亡
|
||||
|
||||
revived = JobStore(store_dir=d)
|
||||
j = revived.get(jid)
|
||||
assert j["status"] == "failed"
|
||||
assert j["duration_s"] is None
|
||||
|
||||
|
||||
def test_reap_does_not_touch_terminal_records(tmp_path):
|
||||
d = tmp_path / "jobs"
|
||||
store = JobStore(store_dir=d)
|
||||
jid, _ = store.create(timeout_s=60)
|
||||
store.start(jid)
|
||||
store.succeed(jid, {"daily_rows": 100})
|
||||
|
||||
revived = JobStore(store_dir=d)
|
||||
j = revived.get(jid)
|
||||
assert j["status"] == "succeeded"
|
||||
assert j["result"] == {"daily_rows": 100}
|
||||
|
||||
|
||||
def test_reap_allows_new_job_after_dead_orphan(tmp_path):
|
||||
"""补录后旧 job 已 failed: 新进程 create() 不被死孤儿阻塞(单飞只看内存)。"""
|
||||
d = tmp_path / "jobs"
|
||||
dead = JobStore(store_dir=d)
|
||||
old_jid, _ = dead.create(timeout_s=60)
|
||||
dead.start(old_jid)
|
||||
|
||||
revived = JobStore(store_dir=d)
|
||||
new_jid, is_new = revived.create(timeout_s=60)
|
||||
assert is_new is True
|
||||
assert new_jid != old_jid
|
||||
|
||||
|
||||
# ── 终态覆盖快照 ─────────────────────────────────────────────────────────
|
||||
|
||||
def test_terminal_write_replaces_running_snapshot(tmp_path):
|
||||
d = tmp_path / "jobs"
|
||||
store = JobStore(store_dir=d)
|
||||
jid, _ = store.create(timeout_s=60)
|
||||
store.start(jid)
|
||||
store.fail(jid, "boom")
|
||||
|
||||
files = list(d.glob("*.json"))
|
||||
assert len(files) == 1
|
||||
disk = _read_disk(d, jid)
|
||||
assert disk["status"] == "failed"
|
||||
assert disk["error"] == "boom"
|
||||
@@ -0,0 +1,137 @@
|
||||
"""个股详情分时轮询的 live 直拉路径测试。
|
||||
|
||||
背景: 盘中分钟增量落盘后, 当日本地分区很快达到 90% 完整度,
|
||||
/api/kline/minute 的本地优先启发式会拦截实时补拉, 详情分时图停在
|
||||
上一增量轮 (≥60s 滞后)。live=1 让详情轮询在连续竞价时段绕过本地优先。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import date, datetime
|
||||
|
||||
import polars as pl
|
||||
import pytest
|
||||
from fastapi import FastAPI
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from app.api.kline import router
|
||||
from app.market_time import CN_TZ, in_continuous_session
|
||||
|
||||
# 2026-08-26 是周三; 10:00 处于上午连续竞价, expected(已交易分钟) = 30
|
||||
_NOW = datetime(2026, 8, 26, 10, 0, tzinfo=CN_TZ)
|
||||
_TODAY = date(2026, 8, 26)
|
||||
_LOCAL_ROWS = 30
|
||||
|
||||
|
||||
class _FakeRepo:
|
||||
def resolve_asset_type(self, symbol: str) -> str:
|
||||
return "stock"
|
||||
|
||||
def get_instruments(self) -> pl.DataFrame:
|
||||
return pl.DataFrame(
|
||||
{"symbol": [], "name": [], "total_shares": [], "float_shares": []}
|
||||
)
|
||||
|
||||
def get_daily_asset(self, asset_type, symbol, start, end, columns=None):
|
||||
return pl.DataFrame({"date": [], "close": []})
|
||||
|
||||
def get_minute(self, symbol, trade_date, asset_type="stock") -> pl.DataFrame:
|
||||
return pl.DataFrame({
|
||||
"datetime": [
|
||||
datetime(2026, 8, 26, 9, 30 + offset // 60, offset % 60)
|
||||
for offset in range(_LOCAL_ROWS)
|
||||
],
|
||||
"close": [10.0] * _LOCAL_ROWS,
|
||||
})
|
||||
|
||||
|
||||
def _client() -> TestClient:
|
||||
app = FastAPI()
|
||||
app.include_router(router)
|
||||
app.state.repo = _FakeRepo()
|
||||
return TestClient(app)
|
||||
|
||||
|
||||
def _patch_market(monkeypatch, *, in_session: bool) -> None:
|
||||
import app.api.kline as kline_api
|
||||
|
||||
monkeypatch.setattr(kline_api, "cn_now", lambda: _NOW)
|
||||
monkeypatch.setattr(kline_api, "cn_today", lambda: _TODAY)
|
||||
monkeypatch.setattr(kline_api, "in_continuous_session", lambda: in_session)
|
||||
|
||||
|
||||
def _patch_live_fetch(monkeypatch) -> None:
|
||||
import app.api.kline as kline_api
|
||||
|
||||
def _fake_fetch(symbol, trade_date, asset_type="stock"):
|
||||
return pl.DataFrame({
|
||||
"datetime": [datetime(2026, 8, 26, 9, 59)],
|
||||
"close": [11.11],
|
||||
})
|
||||
|
||||
monkeypatch.setattr(
|
||||
kline_api.kline_sync, "fetch_minute_single", _fake_fetch
|
||||
)
|
||||
|
||||
|
||||
def test_minute_live_param_bypasses_local_first_during_session(monkeypatch):
|
||||
_patch_market(monkeypatch, in_session=True)
|
||||
_patch_live_fetch(monkeypatch)
|
||||
|
||||
resp = _client().get(
|
||||
"/api/kline/minute", params={"symbol": "600000.SH", "live": 1}
|
||||
)
|
||||
|
||||
assert resp.status_code == 200
|
||||
body = resp.json()
|
||||
assert body["source"] == "live"
|
||||
assert body["rows"][0]["close"] == 11.11
|
||||
|
||||
|
||||
def test_minute_without_live_keeps_local_first(monkeypatch):
|
||||
_patch_market(monkeypatch, in_session=True)
|
||||
_patch_live_fetch(monkeypatch)
|
||||
|
||||
resp = _client().get(
|
||||
"/api/kline/minute", params={"symbol": "600000.SH"}
|
||||
)
|
||||
|
||||
assert resp.status_code == 200
|
||||
body = resp.json()
|
||||
# 本地 30 根 >= expected(30)*0.9 → 完整, 走本地
|
||||
assert body["source"] == "local"
|
||||
assert body["rows"][0]["close"] == 10.0
|
||||
|
||||
|
||||
def test_minute_live_param_falls_back_to_local_off_session(monkeypatch):
|
||||
_patch_market(monkeypatch, in_session=False)
|
||||
_patch_live_fetch(monkeypatch)
|
||||
|
||||
resp = _client().get(
|
||||
"/api/kline/minute", params={"symbol": "600000.SH", "live": 1}
|
||||
)
|
||||
|
||||
assert resp.status_code == 200
|
||||
body = resp.json()
|
||||
assert body["source"] == "local"
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
("hour", "minute", "expected"),
|
||||
[
|
||||
(9, 29, False),
|
||||
(9, 30, True),
|
||||
(11, 30, True),
|
||||
(11, 31, False),
|
||||
(12, 30, False),
|
||||
(13, 0, True),
|
||||
(15, 0, True),
|
||||
(15, 1, False),
|
||||
],
|
||||
)
|
||||
def test_in_continuous_session_boundaries(hour: int, minute: int, expected: bool):
|
||||
now = datetime(2026, 8, 26, hour, minute, tzinfo=CN_TZ) # 周三
|
||||
assert in_continuous_session(now) is expected
|
||||
|
||||
|
||||
def test_in_continuous_session_rejects_weekend():
|
||||
assert in_continuous_session(datetime(2026, 8, 29, 10, 0, tzinfo=CN_TZ)) is False
|
||||
@@ -0,0 +1,62 @@
|
||||
"""分钟源历史深度能力 (minute_history_days) 契约测试。
|
||||
|
||||
provider 可选类属性 minute_history_days 声明 1 分钟历史深度(交易日):
|
||||
- stock-sdk = 5 (免费分时接口仅保留最近 5 个交易日)
|
||||
- 未声明 / 走 tickflow → None (深历史)
|
||||
preferences GET 带出该字段, 前端分时档位据此收窄 (浅源默认 5日, 深源默认 20日)。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from types import SimpleNamespace
|
||||
|
||||
from app.api import settings
|
||||
from app.services import preferences
|
||||
|
||||
|
||||
def _mock_resolver(monkeypatch, provider, fallback, err=None):
|
||||
monkeypatch.setattr(
|
||||
"app.services.kline_sync._resolve_minute_provider",
|
||||
lambda name: (provider, fallback, err),
|
||||
)
|
||||
|
||||
|
||||
def test_stocksdk_declares_five_day_history():
|
||||
from app.plugins.stocksdk.provider import StockSDKProvider
|
||||
|
||||
assert StockSDKProvider.minute_history_days == 5
|
||||
|
||||
|
||||
def test_history_days_from_custom_provider(monkeypatch):
|
||||
"""自定义浅源 → 声明值; 前端据此只显示 1/5 日档。"""
|
||||
_mock_resolver(monkeypatch, SimpleNamespace(minute_history_days=5), False)
|
||||
monkeypatch.setattr(preferences, "get_minute_data_provider", lambda: "stocksdk")
|
||||
assert settings._minute_history_days() == 5
|
||||
|
||||
|
||||
def test_history_days_none_for_undeclared_provider(monkeypatch):
|
||||
"""未声明的自定义源 → None (深历史基准)。"""
|
||||
_mock_resolver(monkeypatch, SimpleNamespace(), False)
|
||||
monkeypatch.setattr(preferences, "get_minute_data_provider", lambda: "my_source")
|
||||
assert settings._minute_history_days() is None
|
||||
|
||||
|
||||
def test_history_days_none_for_tickflow(monkeypatch):
|
||||
"""tickflow (回退路径) → None (深历史)。"""
|
||||
_mock_resolver(monkeypatch, None, True)
|
||||
monkeypatch.setattr(preferences, "get_minute_data_provider", lambda: "tickflow")
|
||||
assert settings._minute_history_days() is None
|
||||
|
||||
|
||||
def test_history_days_none_when_resolver_fails(monkeypatch):
|
||||
"""resolver 异常 (registry 损坏) → 降级 None, 不抛 500。"""
|
||||
_mock_resolver(monkeypatch, None, True, err="registry broken")
|
||||
monkeypatch.setattr(preferences, "get_minute_data_provider", lambda: "stocksdk")
|
||||
assert settings._minute_history_days() is None
|
||||
|
||||
|
||||
def test_preferences_get_includes_history_days(monkeypatch):
|
||||
"""GET /preferences 响应包含 minute_history_days 字段。"""
|
||||
_mock_resolver(monkeypatch, SimpleNamespace(minute_history_days=5), False)
|
||||
payload = settings.get_preferences()
|
||||
assert payload["minute_history_days"] == 5
|
||||
assert "minute_data_provider" in payload
|
||||
@@ -0,0 +1,305 @@
|
||||
"""盘中分钟增量刷新服务 (minute_refresh) 测试。
|
||||
|
||||
覆盖:
|
||||
- 连续竞价时段判定 (含边界)
|
||||
- 门控链: 开关关闭 / 自定义分钟源让位 / 能力缺失 / 时段外 / 放行
|
||||
- 单轮: mock 边界层脉冲 + 落盘, 校验状态字段与 universe 来源
|
||||
- 偏好读写: 默认关闭、间隔 clamp [60, 300]
|
||||
- API: /minute-refresh/status 无服务时 available=false
|
||||
|
||||
不发起真实网络请求: fetch_intraday_full_market_burst 与 _write_minute_partition
|
||||
均 monkeypatch 替换。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
|
||||
import polars as pl
|
||||
|
||||
from app.services import minute_refresh, preferences
|
||||
from app.services.minute_refresh import MinuteRefreshService, _in_continuous_session
|
||||
|
||||
|
||||
def _isolated_prefs(tmp_path, monkeypatch):
|
||||
path = tmp_path / "preferences.json"
|
||||
monkeypatch.setattr(preferences, "_path", lambda: path)
|
||||
preferences._invalidate_cache()
|
||||
return path
|
||||
|
||||
|
||||
class _FakeCapSet:
|
||||
def __init__(self, has_intraday_batch: bool):
|
||||
self._has = has_intraday_batch
|
||||
|
||||
def has(self, cap) -> bool:
|
||||
from app.tickflow.capabilities import Cap
|
||||
|
||||
# 服务门控挂 INTRADAY_UNIVERSE, 修复轮用 INTRADAY_BATCH — 两者同档, 一起授/不授
|
||||
return self._has and cap in (Cap.INTRADAY_BATCH, Cap.INTRADAY_UNIVERSE)
|
||||
|
||||
|
||||
class _FakeAppState:
|
||||
def __init__(self, has_intraday_batch: bool):
|
||||
self.capabilities = _FakeCapSet(has_intraday_batch)
|
||||
|
||||
|
||||
class _FakeRepo:
|
||||
def __init__(self, symbols: list[str]):
|
||||
from pathlib import Path
|
||||
self._inst = pl.DataFrame({"symbol": symbols})
|
||||
self.store = type("S", (), {"data_dir": Path(".")})()
|
||||
|
||||
def get_instruments(self) -> pl.DataFrame:
|
||||
return self._inst
|
||||
|
||||
|
||||
# ── 时段判定 ────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_continuous_session_boundaries():
|
||||
wk = datetime(2026, 8, 25, 10, 0) # 周二
|
||||
assert _in_continuous_session(wk)
|
||||
assert not _in_continuous_session(datetime(2026, 8, 25, 9, 29))
|
||||
assert not _in_continuous_session(datetime(2026, 8, 25, 11, 31)) # 午休
|
||||
assert _in_continuous_session(datetime(2026, 8, 25, 13, 0)) # 午后恢复
|
||||
assert _in_continuous_session(datetime(2026, 8, 25, 15, 0)) # 收盘瞬时
|
||||
assert not _in_continuous_session(datetime(2026, 8, 25, 15, 1))
|
||||
assert not _in_continuous_session(datetime(2026, 8, 22, 10, 0)) # 周六
|
||||
|
||||
|
||||
# ── 门控链 ──────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def _svc(tmp_path, monkeypatch, *, enabled=True, custom_provider=False, capability=True, in_hours=True):
|
||||
_isolated_prefs(tmp_path, monkeypatch)
|
||||
preferences.save({"minute_refresh_enabled": enabled})
|
||||
if custom_provider:
|
||||
# 模拟已注册的自定义分钟源 (真实注册表在测试环境未加载)
|
||||
monkeypatch.setattr(preferences, "get_minute_data_provider", lambda: "a-stock-data")
|
||||
svc = MinuteRefreshService(_FakeRepo(["600000.SH"]))
|
||||
svc.set_app_state(_FakeAppState(capability))
|
||||
monkeypatch.setattr(minute_refresh, "_in_continuous_session", lambda now=None: in_hours)
|
||||
# 交易日探针默认未知 (None → 放行): 隔离真实网络探测, holiday 分支在
|
||||
# test_trading_day.py 单独覆盖
|
||||
from app.services import trading_day
|
||||
monkeypatch.setattr(trading_day, "is_trading_day", lambda now=None: None)
|
||||
return svc
|
||||
|
||||
|
||||
def test_gate_disabled(tmp_path, monkeypatch):
|
||||
assert _svc(tmp_path, monkeypatch, enabled=False)._gate_reason() == "disabled"
|
||||
|
||||
|
||||
def test_gate_custom_provider_yields(tmp_path, monkeypatch):
|
||||
svc = _svc(tmp_path, monkeypatch, custom_provider=True)
|
||||
assert svc._gate_reason() == "custom_minute_provider"
|
||||
|
||||
|
||||
def test_gate_capability_missing(tmp_path, monkeypatch):
|
||||
svc = _svc(tmp_path, monkeypatch, capability=False)
|
||||
assert svc._gate_reason() == "capability"
|
||||
|
||||
|
||||
def test_gate_outside_trading_hours(tmp_path, monkeypatch):
|
||||
svc = _svc(tmp_path, monkeypatch, in_hours=False)
|
||||
assert svc._gate_reason() == "outside_trading_hours"
|
||||
|
||||
|
||||
def test_gate_pass(tmp_path, monkeypatch):
|
||||
svc = _svc(tmp_path, monkeypatch)
|
||||
assert svc._gate_reason() is None
|
||||
assert svc.capability_ok() and not svc.custom_provider_active()
|
||||
|
||||
|
||||
# ── 单轮 ────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_run_round_writes_partition_and_updates_status(tmp_path, monkeypatch):
|
||||
svc = _svc(tmp_path, monkeypatch)
|
||||
minute_df = pl.DataFrame({
|
||||
"symbol": ["600000.SH"],
|
||||
"datetime": [datetime(2026, 8, 25, 1, 30)],
|
||||
"open": [10.0], "high": [10.5], "low": [9.9], "close": [10.2],
|
||||
"volume": [1000.0], "amount": [10200.0],
|
||||
})
|
||||
calls: dict = {}
|
||||
|
||||
def fake_burst(symbols, capset, *, count=300):
|
||||
calls["symbols"] = list(symbols)
|
||||
return (minute_df, 1)
|
||||
|
||||
def fake_write(df, minute_dir):
|
||||
calls["dir"] = minute_dir
|
||||
calls["rows"] = df.height
|
||||
return df.height
|
||||
|
||||
monkeypatch.setattr(
|
||||
"app.services.kline_sync.fetch_intraday_full_market_burst", fake_burst
|
||||
)
|
||||
monkeypatch.setattr("app.services.kline_sync._write_minute_partition", fake_write)
|
||||
|
||||
svc._run_round()
|
||||
|
||||
assert calls["symbols"] == ["600000.SH"]
|
||||
assert calls["rows"] == 1
|
||||
st = svc.status()
|
||||
assert st["rounds"] == 1
|
||||
assert st["last_rows"] == 1
|
||||
assert st["last_symbols"] == 1
|
||||
assert st["last_requests"] == 1
|
||||
assert st["last_round_at"] is not None
|
||||
assert st["last_error"] is None
|
||||
assert st["capability_ok"] is True
|
||||
|
||||
|
||||
def test_run_round_records_error_when_burst_empty(tmp_path, monkeypatch):
|
||||
svc = _svc(tmp_path, monkeypatch)
|
||||
monkeypatch.setattr(
|
||||
"app.services.kline_sync.fetch_intraday_full_market_burst",
|
||||
lambda symbols, capset, *, count=300: (pl.DataFrame(), 3),
|
||||
)
|
||||
svc._run_round()
|
||||
st = svc.status()
|
||||
assert st["rounds"] == 0
|
||||
assert "no data" in st["last_error"]
|
||||
assert st["last_requests"] == 3
|
||||
|
||||
|
||||
# ── 两段式模式选择: 冷启动全天 → 稳态增量 ───────────────────────────
|
||||
|
||||
|
||||
def _patch_round(monkeypatch, *, lag, inc_df, burst_df):
|
||||
calls: dict = {"modes": []}
|
||||
|
||||
monkeypatch.setattr(
|
||||
minute_refresh.MinuteRefreshService, "_today_coverage_lag_minutes",
|
||||
lambda self: lag, raising=True,
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
"app.services.kline_sync.fetch_intraday_universe_increment",
|
||||
lambda *a, **k: (calls["modes"].append("increment"), (inc_df, 1))[1],
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
"app.services.kline_sync.fetch_intraday_full_market_burst",
|
||||
lambda symbols, capset, *, count=300: (calls["modes"].append("full"), (burst_df, 28))[1],
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
"app.services.kline_sync._write_minute_partition",
|
||||
lambda df, minute_dir: df.height,
|
||||
)
|
||||
return calls
|
||||
|
||||
|
||||
def _inc_df():
|
||||
return pl.DataFrame({
|
||||
"symbol": ["600000.SH", "000001.SZ"],
|
||||
"datetime": [datetime(2026, 8, 25, 10, 0)] * 2,
|
||||
"open": [10.0] * 2, "high": [10.5] * 2, "low": [9.9] * 2, "close": [10.2] * 2,
|
||||
"volume": [1000.0] * 2, "amount": [10200.0] * 2,
|
||||
})
|
||||
|
||||
|
||||
def _full_df():
|
||||
return _inc_df()
|
||||
|
||||
|
||||
def test_cold_start_no_local_data_uses_full_mode(tmp_path, monkeypatch):
|
||||
"""当日无数据 (lag=None, 如 10 点冷启动) → 全天修复轮。"""
|
||||
svc = _svc(tmp_path, monkeypatch)
|
||||
calls = _patch_round(monkeypatch, lag=None, inc_df=_inc_df(), burst_df=_full_df())
|
||||
svc._run_round()
|
||||
assert calls["modes"] == ["full"]
|
||||
st = svc.status()
|
||||
assert st["last_mode"] == "full"
|
||||
assert st["last_rows"] == 2
|
||||
|
||||
|
||||
def test_healthy_coverage_uses_increment_mode(tmp_path, monkeypatch):
|
||||
"""当日覆盖新鲜 (lag ≤ 3 分钟) → universe 单请求增量, 不打 burst。"""
|
||||
svc = _svc(tmp_path, monkeypatch)
|
||||
calls = _patch_round(monkeypatch, lag=0.2, inc_df=_inc_df(), burst_df=_full_df())
|
||||
svc._run_round()
|
||||
assert calls["modes"] == ["increment"]
|
||||
st = svc.status()
|
||||
assert st["last_mode"] == "increment"
|
||||
assert st["last_requests"] == 1
|
||||
assert st["last_symbols"] == 2
|
||||
assert st["last_rows"] == 2
|
||||
|
||||
|
||||
def test_stale_coverage_beyond_bar_headroom_falls_back_to_full(tmp_path, monkeypatch):
|
||||
"""覆盖滞后超过 3 分钟 (超过 universe 3 根余量) → 全天修复轮。"""
|
||||
svc = _svc(tmp_path, monkeypatch)
|
||||
calls = _patch_round(monkeypatch, lag=5.0, inc_df=_inc_df(), burst_df=_full_df())
|
||||
svc._run_round()
|
||||
assert calls["modes"] == ["full"]
|
||||
|
||||
|
||||
def test_consecutive_empty_rounds_escalate_to_full(tmp_path, monkeypatch):
|
||||
"""universe 连续 2 轮空返回 → 第 3 轮自动升级全天修复 (自愈)。"""
|
||||
svc = _svc(tmp_path, monkeypatch)
|
||||
calls = _patch_round(
|
||||
monkeypatch,
|
||||
lag=0.2,
|
||||
inc_df=pl.DataFrame(), # 增量恒空 (模拟 universe 端点持续异常)
|
||||
burst_df=_full_df(),
|
||||
)
|
||||
svc._run_round()
|
||||
svc._run_round()
|
||||
assert calls["modes"] == ["increment", "increment"]
|
||||
assert svc.status()["rounds"] == 0
|
||||
svc._run_round()
|
||||
assert calls["modes"] == ["increment", "increment", "full"]
|
||||
assert svc.status()["last_mode"] == "full"
|
||||
|
||||
|
||||
def test_status_reports_gate_reason_when_stopped(tmp_path, monkeypatch):
|
||||
svc = _svc(tmp_path, monkeypatch, enabled=False)
|
||||
st = svc.status()
|
||||
assert st["enabled"] is False
|
||||
assert st["running"] is False
|
||||
assert st["gate_reason"] == "disabled"
|
||||
assert st["interval_seconds"] == 6
|
||||
|
||||
|
||||
# ── 偏好 ────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_refresh_preferences_defaults_and_clamp(tmp_path, monkeypatch):
|
||||
_isolated_prefs(tmp_path, monkeypatch)
|
||||
assert preferences.get_minute_refresh_enabled() is False
|
||||
assert preferences.get_minute_refresh_interval() == 6
|
||||
preferences.save({"minute_refresh_interval": 1})
|
||||
assert preferences.get_minute_refresh_interval() == 3 # 下限
|
||||
preferences.save({"minute_refresh_interval": 999})
|
||||
assert preferences.get_minute_refresh_interval() == 300 # 上限
|
||||
preferences.save({"minute_refresh_interval": 15})
|
||||
assert preferences.get_minute_refresh_interval() == 15
|
||||
|
||||
def test_realtime_monitor_config_owns_refresh_keys(tmp_path, monkeypatch):
|
||||
"""盘中增量配置归属实时监控端点 (set_realtime_monitor_config), 并 clamp 到 [3,300]。"""
|
||||
_isolated_prefs(tmp_path, monkeypatch)
|
||||
saved = preferences.set_realtime_monitor_config({
|
||||
"minute_refresh_enabled": True,
|
||||
"minute_refresh_interval": 1, # 越界 → clamp 到下限
|
||||
})
|
||||
assert saved["minute_refresh_enabled"] is True
|
||||
assert saved["minute_refresh_interval"] == 3
|
||||
saved = preferences.set_realtime_monitor_config({"minute_refresh_interval": 400})
|
||||
assert saved["minute_refresh_interval"] == 300
|
||||
saved = preferences.set_realtime_monitor_config({"minute_refresh_interval": 6})
|
||||
assert saved["minute_refresh_interval"] == 6
|
||||
|
||||
|
||||
def test_status_endpoint_without_service():
|
||||
from fastapi import FastAPI
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from app.api.settings import router
|
||||
|
||||
app = FastAPI()
|
||||
app.include_router(router)
|
||||
client = TestClient(app)
|
||||
resp = client.get("/api/settings/minute-refresh/status")
|
||||
assert resp.status_code == 200
|
||||
assert resp.json() == {"available": False}
|
||||
@@ -0,0 +1,526 @@
|
||||
"""分钟策略 (minute_filter 后端) 测试。
|
||||
|
||||
覆盖:
|
||||
- minute_red_streak 形态: 命中 / 不足根数不触发 / 最高K不红 / rank_by 两口径 /
|
||||
乱序输入 / 最高价并列取更晚K线 / 开盘窗口(当日最早N根, 与最近N根区分)
|
||||
- 引擎加载校验: 只能声明 filter_minute_history、timeframes 必须且只能是 ["1m"]
|
||||
- 引擎 1m 运行: enriched 联表基础过滤 (剔除ST / 股价区间)、entry hits、
|
||||
日线 context 拒绝
|
||||
- ScreenerService 1m context: 当日分区优先、缺失回退最近分区、空库报错、
|
||||
非股票资产拒绝
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import datetime as _dt
|
||||
import importlib.util
|
||||
from datetime import date, datetime
|
||||
from pathlib import Path
|
||||
|
||||
import polars as pl
|
||||
|
||||
from app.services.screener import ScreenerService
|
||||
from app.strategy.engine import StrategyDataContext, StrategyEngine
|
||||
|
||||
# 分钟红7 已从内置策略改为自定义策略 (运行时 data/strategies/custom/, 不入库);
|
||||
# 测试通过仓库内的参考实现夹具加载, 覆盖同一份策略逻辑。
|
||||
STRATEGY_FIXTURE_DIR = Path(__file__).resolve().parent / "fixtures" / "strategies"
|
||||
_spec = importlib.util.spec_from_file_location(
|
||||
"minute_red_streak_fixture", STRATEGY_FIXTURE_DIR / "minute_red_streak.py"
|
||||
)
|
||||
minute_red_streak = importlib.util.module_from_spec(_spec)
|
||||
_spec.loader.exec_module(minute_red_streak)
|
||||
|
||||
|
||||
def _bars(symbol: str, candles: list[tuple[float, float, float]], start_hour: int = 9) -> pl.DataFrame:
|
||||
"""candles: (open, close, high) 序列, 时间从 start_hour:30 起每分钟一根。"""
|
||||
n = len(candles)
|
||||
base = datetime(2026, 8, 25, start_hour, 30)
|
||||
return pl.DataFrame({
|
||||
"symbol": [symbol] * n,
|
||||
"datetime": [base + _dt.timedelta(minutes=i) for i in range(n)],
|
||||
"open": [float(c[0]) for c in candles],
|
||||
"high": [float(c[2]) for c in candles],
|
||||
"low": [float(min(c[0], c[1])) for c in candles],
|
||||
"close": [float(c[1]) for c in candles],
|
||||
"volume": [100.0] * n,
|
||||
"amount": [10000.0] * n,
|
||||
})
|
||||
|
||||
|
||||
# ── 形态 ────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_pattern_hits_five_red_of_seven_with_red_top_two():
|
||||
# 7根: 5红2绿, 绿K的最高价都压得比红K低 → 最高的两根(10.9/10.7)都是红
|
||||
candles = [
|
||||
(10.0, 10.2, 10.30), # 红
|
||||
(10.2, 10.1, 10.25), # 绿 (低高点)
|
||||
(10.1, 10.4, 10.50), # 红
|
||||
(10.4, 10.6, 10.70), # 红 (次高)
|
||||
(10.6, 10.5, 10.65), # 绿 (低高点)
|
||||
(10.5, 10.7, 10.80), # 红
|
||||
(10.7, 10.8, 10.90), # 红 (最高)
|
||||
]
|
||||
out = minute_red_streak.filter_minute_history(_bars("600000.SH", candles), {"require_limit_up": False})
|
||||
assert out["symbol"].to_list() == ["600000.SH"]
|
||||
row = out.row(0, named=True)
|
||||
assert row["red_count"] == 5
|
||||
assert row["top_red_count"] == 2
|
||||
assert row["close"] == 10.8
|
||||
|
||||
|
||||
def test_pattern_insufficient_bars_never_triggers():
|
||||
out = minute_red_streak.filter_minute_history(_bars("600000.SH", [(10.0, 10.2, 10.3)] * 6), {"require_limit_up": False})
|
||||
assert out.is_empty()
|
||||
|
||||
|
||||
def test_pattern_green_at_top_blocks_hit():
|
||||
# 5红, 但最高的一根是绿 (高开回落) → 最高两根不全红, 不触发
|
||||
candles = [
|
||||
(10.0, 10.2, 10.30), # 红
|
||||
(10.1, 10.4, 10.50), # 红
|
||||
(10.3, 10.6, 10.70), # 红
|
||||
(10.6, 10.5, 10.65), # 绿 (低高点)
|
||||
(10.4, 10.5, 10.55), # 红 (低高点)
|
||||
(11.5, 11.0, 12.00), # 绿 (最高)
|
||||
(11.0, 11.4, 11.90), # 红 (次高)
|
||||
]
|
||||
out = minute_red_streak.filter_minute_history(_bars("600000.SH", candles), {"require_limit_up": False})
|
||||
assert out.is_empty()
|
||||
|
||||
|
||||
def test_pattern_rank_by_close_uses_close_not_high():
|
||||
# high 口径最高两根是绿K冲高; close 口径最高两根是红K → 仅 close 口径命中
|
||||
candles = [
|
||||
(10.0, 10.5, 10.60), # 红
|
||||
(10.5, 10.9, 11.50), # 绿 (high 最高, 并列)
|
||||
(10.9, 11.2, 11.40), # 红
|
||||
(11.2, 11.3, 11.35), # 红
|
||||
(11.3, 11.4, 11.45), # 红 (close 次高)
|
||||
(11.4, 11.1, 11.50), # 绿 (high 最高, 并列)
|
||||
(11.1, 11.5, 11.55), # 红 (close 最高)
|
||||
]
|
||||
by_high = minute_red_streak.filter_minute_history(_bars("600000.SH", candles), {"require_limit_up": False})
|
||||
by_close = minute_red_streak.filter_minute_history(
|
||||
_bars("600000.SH", candles), {"rank_by_close": True, "require_limit_up": False}
|
||||
)
|
||||
assert by_high.is_empty()
|
||||
assert by_close["symbol"].to_list() == ["600000.SH"]
|
||||
|
||||
|
||||
def test_pattern_sorts_unordered_input_by_datetime():
|
||||
bars = pl.concat([
|
||||
_bars("600000.SH", [(10.0, 10.2, 10.30)]),
|
||||
_bars("600000.SH", [
|
||||
(10.2, 10.1, 10.25), (10.1, 10.4, 10.50), (10.4, 10.6, 10.70),
|
||||
(10.6, 10.5, 10.65), (10.5, 10.7, 10.80), (10.7, 10.8, 10.90),
|
||||
]),
|
||||
]).sample(fraction=1.0, shuffle=True, seed=7)
|
||||
out = minute_red_streak.filter_minute_history(bars, {"require_limit_up": False})
|
||||
assert out["symbol"].to_list() == ["600000.SH"]
|
||||
assert out.row(0, named=True)["close"] == 10.8 # 最后一根(时间最大)的收盘
|
||||
|
||||
|
||||
def test_pattern_three_way_high_tie_prefers_later_bars():
|
||||
# 三根 high 并列最高: 更早的绿K应被更晚的两根红K挤出 top2 → 命中
|
||||
# (若并列取更早, top2 = {红, 绿} → 不命中; 该测试固定 "同值取更晚" 契约)
|
||||
candles = [
|
||||
(10.0, 10.2, 10.30), # 红
|
||||
(10.1, 10.4, 10.50), # 红
|
||||
(10.2, 10.1, 10.25), # 绿 (低高点)
|
||||
(10.3, 10.6, 10.70), # 红
|
||||
(10.8, 10.5, 10.90), # 绿 (并列最高, 最早 → 被 top2 排除)
|
||||
(10.5, 10.6, 10.90), # 红 (并列最高, 中间)
|
||||
(10.6, 10.8, 10.90), # 红 (并列最高, 最晚)
|
||||
]
|
||||
out = minute_red_streak.filter_minute_history(_bars("600000.SH", candles), {"require_limit_up": False})
|
||||
assert out["symbol"].to_list() == ["600000.SH"]
|
||||
assert out.row(0, named=True)["top_red_count"] == 2
|
||||
|
||||
|
||||
def test_pattern_min_red_threshold_respected():
|
||||
# 4红3绿, 最高的两根红 → min_red=5 不命中, min_red=4 命中
|
||||
candles = [
|
||||
(10.0, 10.2, 10.30), # 红
|
||||
(10.2, 10.1, 10.25), # 绿
|
||||
(10.1, 10.4, 10.50), # 红
|
||||
(10.4, 10.3, 10.45), # 绿
|
||||
(10.3, 10.6, 10.70), # 红
|
||||
(10.6, 10.5, 10.65), # 绿
|
||||
(10.5, 10.8, 10.90), # 红
|
||||
]
|
||||
bars = _bars("600000.SH", candles)
|
||||
assert minute_red_streak.filter_minute_history(bars, {"min_red": 5, "require_limit_up": False}).is_empty()
|
||||
assert not minute_red_streak.filter_minute_history(bars, {"min_red": 4, "require_limit_up": False}).is_empty()
|
||||
|
||||
|
||||
def test_pattern_uses_opening_bars_even_if_day_turns_green():
|
||||
# 开盘7根 = 5红2绿命中; 第8/9根大绿回落 → 开盘窗口语义下仍命中,
|
||||
# 且 close 取窗口末根 (10.8) 而非全天最新价
|
||||
candles = [
|
||||
(10.0, 10.2, 10.30), # 红
|
||||
(10.2, 10.1, 10.25), # 绿 (低高点)
|
||||
(10.1, 10.4, 10.50), # 红
|
||||
(10.4, 10.6, 10.70), # 红 (次高)
|
||||
(10.6, 10.5, 10.65), # 绿 (低高点)
|
||||
(10.5, 10.7, 10.80), # 红
|
||||
(10.7, 10.8, 10.90), # 红 (最高) ← 窗口末根
|
||||
(10.8, 10.0, 10.85), # 开盘窗口外的绿
|
||||
(10.0, 9.5, 10.05), # 开盘窗口外的绿
|
||||
]
|
||||
out = minute_red_streak.filter_minute_history(_bars("600000.SH", candles), {"require_limit_up": False})
|
||||
assert out["symbol"].to_list() == ["600000.SH"]
|
||||
row = out.row(0, named=True)
|
||||
assert row["red_count"] == 5
|
||||
assert row["close"] == 10.8 # 窗口末根收盘, 不是第9根的 9.5
|
||||
assert row["last_datetime"] == datetime(2026, 8, 25, 9, 36)
|
||||
|
||||
|
||||
def test_pattern_opening_window_miss_not_rescued_by_late_reds():
|
||||
# 开盘7根仅4红不命中; 第8/9根转红 (最近7根口径会命中) → 开盘窗口仍不触发
|
||||
candles = [
|
||||
(10.0, 10.2, 10.30), # 红
|
||||
(10.2, 10.1, 10.25), # 绿
|
||||
(10.1, 10.4, 10.50), # 红
|
||||
(10.4, 10.3, 10.45), # 绿
|
||||
(10.3, 10.6, 10.70), # 红
|
||||
(10.6, 10.5, 10.65), # 绿
|
||||
(10.5, 10.8, 10.90), # 红
|
||||
(10.8, 10.9, 11.00), # 红 (窗口外)
|
||||
(10.9, 11.0, 11.10), # 红 (窗口外)
|
||||
]
|
||||
out = minute_red_streak.filter_minute_history(_bars("600000.SH", candles), {"require_limit_up": False})
|
||||
assert out.is_empty()
|
||||
|
||||
|
||||
# ── 涨停条件 (日线维度) ─────────────────────────────────────────────
|
||||
|
||||
|
||||
_HIT_CANDLES = [
|
||||
(10.0, 10.2, 10.30), # 红
|
||||
(10.2, 10.1, 10.25), # 绿 (低高点)
|
||||
(10.1, 10.4, 10.50), # 红
|
||||
(10.4, 10.6, 10.70), # 红 (次高)
|
||||
(10.6, 10.5, 10.65), # 绿 (低高点)
|
||||
(10.5, 10.7, 10.80), # 红
|
||||
(10.7, 10.8, 10.90), # 红 (最高)
|
||||
]
|
||||
|
||||
|
||||
def _daily(
|
||||
symbol: str,
|
||||
days: int,
|
||||
flag_on: set[int] | None = None,
|
||||
*,
|
||||
broken: bool = False,
|
||||
) -> pl.DataFrame:
|
||||
"""days 个交易日的日线帧; flag_on 指定第几天 (0=最早) 触发涨停信号。"""
|
||||
flag_on = flag_on or set()
|
||||
base = date(2026, 8, 25)
|
||||
return pl.DataFrame({
|
||||
"symbol": [symbol] * days,
|
||||
"date": [base - _dt.timedelta(days=days - i) for i in range(days)],
|
||||
"signal_limit_up": [i in flag_on and not broken for i in range(days)],
|
||||
"signal_broken_limit_up": [i in flag_on and broken for i in range(days)],
|
||||
})
|
||||
|
||||
|
||||
def test_pattern_limit_up_condition_filters_by_daily_signals():
|
||||
bars = pl.concat([
|
||||
_bars("600001.SH", _HIT_CANDLES),
|
||||
_bars("600002.SH", _HIT_CANDLES),
|
||||
_bars("600003.SH", _HIT_CANDLES),
|
||||
])
|
||||
daily = pl.concat([
|
||||
_daily("600001.SH", 20, {3}), # 收盘涨停 → 过
|
||||
_daily("600002.SH", 20, {15}, broken=True), # 炸板触及 → 过
|
||||
_daily("600003.SH", 20), # 无涨停 → 剔除
|
||||
])
|
||||
out = minute_red_streak.filter_minute_history(bars, {}, daily=daily)
|
||||
assert sorted(out["symbol"].to_list()) == ["600001.SH", "600002.SH"]
|
||||
assert sorted(out["recent_limit_ups"].to_list()) == [1, 1]
|
||||
|
||||
|
||||
def test_pattern_limit_up_lookback_window_boundary():
|
||||
# 25 个交易日, 涨停仅发生在第 5 天 (0=最早): 回看 20 日窗口 = 最后 20 根
|
||||
# (索引 5..24), 第 5 天在窗外 → 不命中; 回看放宽到 25 → 命中
|
||||
bars = _bars("600000.SH", _HIT_CANDLES)
|
||||
daily = _daily("600000.SH", 25, {4})
|
||||
assert minute_red_streak.filter_minute_history(bars, {}, daily=daily).is_empty()
|
||||
out = minute_red_streak.filter_minute_history(
|
||||
bars, {"limit_up_days": 25}, daily=daily
|
||||
)
|
||||
assert out["symbol"].to_list() == ["600000.SH"]
|
||||
|
||||
|
||||
def test_pattern_limit_up_fails_closed_without_daily():
|
||||
# 日线窗口缺失时失败闭合 (宁可漏过不可错报)
|
||||
out = minute_red_streak.filter_minute_history(_bars("600000.SH", _HIT_CANDLES), {})
|
||||
assert out.is_empty()
|
||||
|
||||
|
||||
def test_pattern_limit_up_disabled_ignores_daily():
|
||||
out = minute_red_streak.filter_minute_history(
|
||||
_bars("600000.SH", _HIT_CANDLES), {"require_limit_up": False}
|
||||
)
|
||||
assert out["symbol"].to_list() == ["600000.SH"]
|
||||
assert "recent_limit_ups" not in out.columns
|
||||
|
||||
|
||||
# ── 引擎加载与运行 ──────────────────────────────────────────────────
|
||||
|
||||
|
||||
def test_custom_minute_strategy_loads_with_minute_filter_backend():
|
||||
# 自定义策略与内置策略共用同一加载器: 夹具目录即一个 custom 目录
|
||||
engine = StrategyEngine(strategy_dirs=[STRATEGY_FIXTURE_DIR])
|
||||
assert not [e for e in engine.load_errors() if "minute" in e["file"]]
|
||||
s = engine.get("minute_red_streak")
|
||||
assert s.execution_backend == "minute_filter"
|
||||
assert s.filter_minute_history_fn is not None
|
||||
assert s.meta["timeframes"] == ["1m"]
|
||||
assert s.source == "custom"
|
||||
|
||||
|
||||
def _minute_code(sid: str, timeframes: str = '["1m"]', extra: str = "") -> str:
|
||||
return f'''import polars as pl
|
||||
META = {{"id": "{sid}", "name": "{sid}", "asset_types": ["stock"], "timeframes": {timeframes}}}
|
||||
EXECUTION_BACKEND = "minute_filter"
|
||||
{extra}
|
||||
def filter_minute_history(df, params):
|
||||
return df.group_by("symbol").agg(
|
||||
close=pl.col("close").max(), last_datetime=pl.col("datetime").max()
|
||||
)
|
||||
'''
|
||||
|
||||
|
||||
def test_minute_filter_backend_validation(tmp_path):
|
||||
(tmp_path / "ok.py").write_text(_minute_code("m_ok"))
|
||||
(tmp_path / "bad_filter.py").write_text(
|
||||
_minute_code("m_bad1", extra="def filter(df, params):\n return pl.lit(True)")
|
||||
)
|
||||
(tmp_path / "bad_tf.py").write_text(_minute_code("m_bad2", timeframes='["1d", "1m"]'))
|
||||
engine = StrategyEngine(strategy_dirs=[tmp_path])
|
||||
ids = {m["id"] for m in engine.list_strategies(include_research=True)}
|
||||
assert "m_ok" in ids
|
||||
assert "m_bad1" not in ids
|
||||
assert "m_bad2" not in ids
|
||||
assert any("only filter_minute_history" in e["error"] for e in engine.load_errors())
|
||||
assert any("timeframes" in e["error"] for e in engine.load_errors())
|
||||
|
||||
|
||||
def test_minute_filter_daily_history_validation(tmp_path):
|
||||
# 声明 daily_history_bars: fn 必须接受 daily 关键字, 且范围 [0, 250]
|
||||
(tmp_path / "m_daily_ok.py").write_text(
|
||||
'import polars as pl\n'
|
||||
'META = {"id": "m_daily_ok", "name": "x", "asset_types": ["stock"], '
|
||||
'"timeframes": ["1m"], "daily_history_bars": 20}\n'
|
||||
'EXECUTION_BACKEND = "minute_filter"\n'
|
||||
'def filter_minute_history(df, params, *, daily=None):\n'
|
||||
' return df.group_by("symbol").agg(close=pl.col("close").max())\n'
|
||||
)
|
||||
(tmp_path / "m_daily_kw.py").write_text(
|
||||
'import polars as pl\n'
|
||||
'META = {"id": "m_daily_kw", "name": "x", "asset_types": ["stock"], '
|
||||
'"timeframes": ["1m"], "daily_history_bars": 20}\n'
|
||||
'EXECUTION_BACKEND = "minute_filter"\n'
|
||||
'def filter_minute_history(df, params):\n'
|
||||
' return df.group_by("symbol").agg(close=pl.col("close").max())\n'
|
||||
)
|
||||
(tmp_path / "m_daily_range.py").write_text(
|
||||
'import polars as pl\n'
|
||||
'META = {"id": "m_daily_range", "name": "x", "asset_types": ["stock"], '
|
||||
'"timeframes": ["1m"], "daily_history_bars": 300}\n'
|
||||
'EXECUTION_BACKEND = "minute_filter"\n'
|
||||
'def filter_minute_history(df, params, *, daily=None):\n'
|
||||
' return df.group_by("symbol").agg(close=pl.col("close").max())\n'
|
||||
)
|
||||
engine = StrategyEngine(strategy_dirs=[tmp_path])
|
||||
assert engine.has("m_daily_ok")
|
||||
assert engine.get("m_daily_ok").minute_daily_bars == 20
|
||||
assert not engine.has("m_daily_kw")
|
||||
assert not engine.has("m_daily_range")
|
||||
assert any("'daily' keyword" in e["error"] for e in engine.load_errors())
|
||||
assert any("[0, 250]" in e["error"] for e in engine.load_errors())
|
||||
|
||||
|
||||
def test_minute_run_injects_daily_history(tmp_path):
|
||||
# fn 直接消费 daily (对涨停信号求和), 验证引擎把 context.daily_history 注入
|
||||
(tmp_path / "m_use_daily.py").write_text(
|
||||
'import polars as pl\n'
|
||||
'META = {"id": "m_use_daily", "name": "x", "asset_types": ["stock"], '
|
||||
'"timeframes": ["1m"], "daily_history_bars": 10}\n'
|
||||
'EXECUTION_BACKEND = "minute_filter"\n'
|
||||
'def filter_minute_history(df, params, *, daily=None):\n'
|
||||
' if daily is None:\n'
|
||||
' return pl.DataFrame(schema={"symbol": pl.Utf8})\n'
|
||||
' return daily.group_by("symbol").agg(\n'
|
||||
' close=pl.col("signal_limit_up").sum() + 10.0)\n'
|
||||
)
|
||||
engine = StrategyEngine(strategy_dirs=[tmp_path])
|
||||
context = StrategyDataContext(
|
||||
asset_type="stock",
|
||||
timeframe="1m",
|
||||
as_of=date(2026, 8, 25),
|
||||
current=pl.DataFrame({
|
||||
"symbol": ["600001.SH"],
|
||||
"name": ["正常股"],
|
||||
"total_shares": [1e8],
|
||||
"float_shares": [5e7],
|
||||
"amount": [3e8],
|
||||
"change_pct": [0.01],
|
||||
}),
|
||||
history=_bars("600001.SH", [(10.0, 10.2, 10.3)] * 7),
|
||||
daily_history=_daily("600001.SH", 10, {2}),
|
||||
)
|
||||
result = engine.run("m_use_daily", context)
|
||||
assert result.total == 1
|
||||
assert result.rows[0]["close"] == 11 # 10 + 窗口内 1 次收盘涨停
|
||||
|
||||
|
||||
def test_minute_context_run_applies_enriched_basic_filter(tmp_path):
|
||||
(tmp_path / "m_basic.py").write_text(_minute_code("m_basic"))
|
||||
engine = StrategyEngine(strategy_dirs=[tmp_path])
|
||||
|
||||
hist = pl.concat([
|
||||
_bars("600001.SH", [(10.0, 20.0, 25.0)] * 7), # 命中, 收盘 20
|
||||
_bars("600002.SH", [(10.0, 20.0, 25.0)] * 7), # 命中但 ST → 剔除
|
||||
_bars("600003.SH", [(10.0, 20.0, 25.0)] * 7), # 命中
|
||||
_bars("600004.SH", [(100.0, 200.0, 250.0)] * 7), # 命中但收盘 200 → 超上限剔除
|
||||
])
|
||||
current = pl.DataFrame({
|
||||
"symbol": ["600001.SH", "600002.SH", "600003.SH", "600004.SH"],
|
||||
"name": ["正常股", "ST垃圾", "正常股2", "高价股"],
|
||||
"total_shares": [1e8, 1e8, 1e8, 1e8],
|
||||
"float_shares": [5e7, 5e7, 5e7, 5e7],
|
||||
"amount": [3e8, 3e8, 3e8, 3e8],
|
||||
"change_pct": [0.01, 0.01, 0.01, 0.01],
|
||||
})
|
||||
context = StrategyDataContext(
|
||||
asset_type="stock",
|
||||
timeframe="1m",
|
||||
as_of=date(2026, 8, 25),
|
||||
current=current,
|
||||
history=hist,
|
||||
)
|
||||
result = engine.run(
|
||||
"m_basic", context, overrides={"basic_filter": {"price_max": 150.0}}
|
||||
)
|
||||
symbols = {r["symbol"] for r in result.rows}
|
||||
assert symbols == {"600001.SH", "600003.SH"}
|
||||
assert all("name" in r for r in result.rows) # enriched 列已联表
|
||||
assert {h["symbol"] for h in result.entry_signal_hits} == symbols
|
||||
|
||||
|
||||
def test_minute_strategy_rejects_daily_context(tmp_path):
|
||||
(tmp_path / "m_daily.py").write_text(_minute_code("m_daily"))
|
||||
engine = StrategyEngine(strategy_dirs=[tmp_path])
|
||||
context = StrategyDataContext(
|
||||
asset_type="stock",
|
||||
timeframe="1d",
|
||||
as_of=date(2026, 8, 25),
|
||||
current=pl.DataFrame({"symbol": ["600001.SH"]}),
|
||||
)
|
||||
try:
|
||||
engine.run("m_daily", context)
|
||||
raise AssertionError("expected ValueError")
|
||||
except ValueError as e:
|
||||
assert "timeframe" in str(e)
|
||||
|
||||
|
||||
# ── ScreenerService 1m context ──────────────────────────────────────
|
||||
|
||||
|
||||
class _FakeMinuteRepo:
|
||||
def __init__(self, partitions: dict[date, pl.DataFrame]):
|
||||
self.partitions = partitions
|
||||
|
||||
def get_minute_by_dates(self, symbols, dates, asset_type="stock"):
|
||||
frames = [self.partitions[d] for d in dates if d in self.partitions]
|
||||
if not frames:
|
||||
return pl.DataFrame()
|
||||
return pl.concat(frames).filter(pl.col("symbol").is_in(symbols))
|
||||
|
||||
def latest_minute_date_global(self):
|
||||
return max(self.partitions) if self.partitions else None
|
||||
|
||||
|
||||
def _svc(partitions: dict[date, pl.DataFrame], asset_type: str = "stock") -> ScreenerService:
|
||||
return ScreenerService(_FakeMinuteRepo(partitions), asset_type=asset_type) # type: ignore[arg-type]
|
||||
|
||||
|
||||
def test_minute_context_prefers_as_of_partition():
|
||||
d1, d2 = date(2026, 8, 24), date(2026, 8, 25)
|
||||
svc = _svc({
|
||||
d1: _bars("600001.SH", [(10.0, 10.2, 10.3)] * 3),
|
||||
d2: _bars("600001.SH", [(10.0, 10.2, 10.3)] * 4),
|
||||
})
|
||||
ctx = svc.build_strategy_context(
|
||||
None, d1, [], timeframe="1m",
|
||||
current=pl.DataFrame({"symbol": ["600001.SH"], "name": ["x"]}),
|
||||
)
|
||||
assert ctx.history.height == 3 # as_of 当日分区, 不取更新的 d2
|
||||
assert ctx.timeframe == "1m"
|
||||
|
||||
|
||||
def test_minute_context_falls_back_to_latest_partition():
|
||||
d1, d2 = date(2026, 8, 24), date(2026, 8, 25)
|
||||
svc = _svc({
|
||||
d1: _bars("600001.SH", [(10.0, 10.2, 10.3)] * 3),
|
||||
d2: _bars("600001.SH", [(10.0, 10.2, 10.3)] * 4),
|
||||
})
|
||||
ctx = svc.build_strategy_context(
|
||||
None, date(2026, 8, 20), [], timeframe="1m",
|
||||
current=pl.DataFrame({"symbol": ["600001.SH"]}),
|
||||
)
|
||||
assert ctx.history.height == 4 # 回退到最近分区 d2
|
||||
|
||||
|
||||
def test_minute_context_empty_store_raises_with_guidance():
|
||||
svc = _svc({})
|
||||
try:
|
||||
svc.build_strategy_context(
|
||||
None, date(2026, 8, 25), [], timeframe="1m",
|
||||
current=pl.DataFrame({"symbol": ["600001.SH"]}),
|
||||
)
|
||||
raise AssertionError("expected ValueError")
|
||||
except ValueError as e:
|
||||
assert "分钟K" in str(e)
|
||||
|
||||
|
||||
def test_minute_context_rejects_non_stock_asset():
|
||||
svc = _svc({date(2026, 8, 25): _bars("510300.SH", [(10.0, 10.2, 10.3)] * 3)}, asset_type="etf")
|
||||
try:
|
||||
svc.build_strategy_context(
|
||||
None, date(2026, 8, 25), [], timeframe="1m",
|
||||
current=pl.DataFrame({"symbol": ["510300.SH"]}),
|
||||
)
|
||||
raise AssertionError("expected ValueError")
|
||||
except ValueError as e:
|
||||
assert "A 股" in str(e)
|
||||
|
||||
|
||||
def test_minute_context_loads_daily_history_for_declared_strategies():
|
||||
class _FakeEngine:
|
||||
def minute_daily_history_bars(self, strategy_ids):
|
||||
return 5
|
||||
|
||||
daily = _daily("600001.SH", 6, {1})
|
||||
repo = _FakeMinuteRepo({date(2026, 8, 25): _bars("600001.SH", [(10.0, 10.2, 10.3)] * 3)})
|
||||
repo.get_enriched_history = lambda target_date, lookback_days: daily # type: ignore[method-assign]
|
||||
repo.get_instruments_asset = lambda asset_type: None # type: ignore[method-assign]
|
||||
svc = ScreenerService(repo, asset_type="stock") # type: ignore[arg-type]
|
||||
ctx = svc.build_strategy_context(
|
||||
_FakeEngine(), date(2026, 8, 25), ["m_x"], timeframe="1m",
|
||||
current=pl.DataFrame({"symbol": ["600001.SH"], "name": ["x"]}),
|
||||
)
|
||||
assert ctx.daily_history is not None
|
||||
assert ctx.daily_history.height == 6 # 引擎声明 5 → 装配日线窗口
|
||||
|
||||
|
||||
def test_minute_context_without_engine_skips_daily_history():
|
||||
svc = _svc({date(2026, 8, 25): _bars("600001.SH", [(10.0, 10.2, 10.3)] * 3)})
|
||||
ctx = svc.build_strategy_context(
|
||||
None, date(2026, 8, 25), [], timeframe="1m",
|
||||
current=pl.DataFrame({"symbol": ["600001.SH"]}),
|
||||
)
|
||||
assert ctx.daily_history is None # 无引擎声明 → 不装配日线
|
||||
@@ -0,0 +1,185 @@
|
||||
"""分钟 K datetime 北京墙钟契约测试。
|
||||
|
||||
契约 (CONTRIBUTING §3.3): kline_minute.datetime 必须是北京墙钟 naive。
|
||||
守卫 _enforce_minute_beijing_wallclock 在两个源头入口强制:
|
||||
- _normalize_minute (TickFlow 帧, timestamp 毫秒为 UTC 基准)
|
||||
- _try_custom_minute (插件/自定义源帧)
|
||||
|
||||
覆盖: 显式转换 / 北京墙钟直通 / UTC 特征自愈 +8 / tz-aware 换算 /
|
||||
fail-closed 拒收 / 路由级契约违规回退 TickFlow。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import UTC, datetime
|
||||
from unittest.mock import MagicMock
|
||||
|
||||
import polars as pl
|
||||
import pytest
|
||||
|
||||
from app.services import kline_sync
|
||||
|
||||
|
||||
def _minute_frame(datetimes: list, symbol: str = "600519.SH") -> pl.DataFrame:
|
||||
n = len(datetimes)
|
||||
return pl.DataFrame({
|
||||
"symbol": [symbol] * n,
|
||||
"datetime": datetimes,
|
||||
"open": [10.0] * n,
|
||||
"high": [10.5] * n,
|
||||
"low": [9.5] * n,
|
||||
"close": [10.2] * n,
|
||||
"volume": [100.0] * n,
|
||||
"amount": [1020.0] * n,
|
||||
})
|
||||
|
||||
|
||||
def _beijing_day() -> list[datetime]:
|
||||
"""一个正常交易日墙钟样本: 开盘/午盘首/收盘。"""
|
||||
return [
|
||||
datetime(2026, 1, 15, 9, 30),
|
||||
datetime(2026, 1, 15, 13, 0),
|
||||
datetime(2026, 1, 15, 15, 0),
|
||||
]
|
||||
|
||||
|
||||
# ---------- TickFlow 路径: timestamp 毫秒 (UTC 基准) → 北京墙钟 ----------
|
||||
|
||||
def test_tickflow_timestamp_normalizes_to_beijing_wallclock():
|
||||
"""09:30 北京 = 01:30 UTC; SDK 帧 timestamp 毫秒归一后必须回到 09:30。"""
|
||||
ts_ms = [
|
||||
int(datetime(2026, 1, 15, 1, 30, tzinfo=UTC).timestamp() * 1000), # 09:30 北京
|
||||
int(datetime(2026, 1, 15, 5, 0, tzinfo=UTC).timestamp() * 1000), # 13:00 北京
|
||||
int(datetime(2026, 1, 15, 7, 0, tzinfo=UTC).timestamp() * 1000), # 15:00 北京
|
||||
]
|
||||
df = pl.DataFrame({
|
||||
"symbol": ["600519.SH"] * 3,
|
||||
"timestamp": ts_ms,
|
||||
"open": [10.0] * 3, "high": [10.5] * 3,
|
||||
"low": [9.5] * 3, "close": [10.2] * 3,
|
||||
"volume": [100.0] * 3, "amount": [1020.0] * 3,
|
||||
})
|
||||
out = kline_sync._normalize_minute(df)
|
||||
assert out["datetime"].to_list() == _beijing_day()
|
||||
|
||||
|
||||
def test_tickflow_timestamp_partial_day_lunch_unaffected():
|
||||
"""午间 11:30 (03:30 UTC) 同样正确归一, 不被误判为越界。"""
|
||||
df = pl.DataFrame({
|
||||
"symbol": ["000001.SZ"],
|
||||
"timestamp": [int(datetime(2026, 1, 15, 3, 30, tzinfo=UTC).timestamp() * 1000)],
|
||||
"open": [1.0], "high": [1.0], "low": [1.0], "close": [1.0],
|
||||
"volume": [1.0], "amount": [1.0],
|
||||
})
|
||||
out = kline_sync._normalize_minute(df)
|
||||
assert out["datetime"].to_list() == [datetime(2026, 1, 15, 11, 30)]
|
||||
|
||||
|
||||
# ---------- 守卫: 各口径分类 ----------
|
||||
|
||||
def test_guard_beijing_naive_passthrough():
|
||||
df = _minute_frame(_beijing_day())
|
||||
out = kline_sync._enforce_minute_beijing_wallclock(df, source="t")
|
||||
assert out["datetime"].to_list() == _beijing_day()
|
||||
|
||||
|
||||
def test_guard_utc_naive_selfhealed_plus8():
|
||||
"""01:30/05:00/07:00 (UTC 墙钟特征) → 自动 +8 → 09:30/13:00/15:00。"""
|
||||
df = _minute_frame([
|
||||
datetime(2026, 1, 15, 1, 30),
|
||||
datetime(2026, 1, 15, 5, 0),
|
||||
datetime(2026, 1, 15, 7, 0),
|
||||
])
|
||||
out = kline_sync._enforce_minute_beijing_wallclock(df, source="t")
|
||||
assert out["datetime"].to_list() == _beijing_day()
|
||||
|
||||
|
||||
def test_guard_selfheal_is_idempotent():
|
||||
df = _minute_frame([datetime(2026, 1, 15, 1, 30), datetime(2026, 1, 15, 3, 0)])
|
||||
once = kline_sync._enforce_minute_beijing_wallclock(df, source="t")
|
||||
twice = kline_sync._enforce_minute_beijing_wallclock(once, source="t")
|
||||
assert once["datetime"].to_list() == twice["datetime"].to_list()
|
||||
|
||||
|
||||
def test_guard_tzaware_utc_converted():
|
||||
"""tz-aware UTC 01:30 → 北京墙钟 09:30 (naive)。"""
|
||||
df = _minute_frame([datetime(2026, 1, 15, 1, 30, tzinfo=UTC)])
|
||||
out = kline_sync._enforce_minute_beijing_wallclock(df, source="t")
|
||||
assert out["datetime"].dtype == pl.Datetime("us")
|
||||
assert out["datetime"].to_list() == [datetime(2026, 1, 15, 9, 30)]
|
||||
|
||||
|
||||
def test_guard_tzaware_shanghai_converted():
|
||||
"""tz-aware +08:00 09:30 → 北京墙钟 09:30 (naive), 数值不变。"""
|
||||
df = _minute_frame([datetime(2026, 1, 15, 9, 30, tzinfo=_shanghai_tz())])
|
||||
out = kline_sync._enforce_minute_beijing_wallclock(df, source="t")
|
||||
assert out["datetime"].to_list() == [datetime(2026, 1, 15, 9, 30)]
|
||||
|
||||
|
||||
def _shanghai_tz():
|
||||
from zoneinfo import ZoneInfo
|
||||
return ZoneInfo("Asia/Shanghai")
|
||||
|
||||
|
||||
def test_guard_unrecognized_convention_fails_closed():
|
||||
"""21:30/22:15 (境外墙钟特征) 既非北京时段也非 UTC 平移 → 拒收。"""
|
||||
df = _minute_frame([datetime(2026, 1, 15, 21, 30), datetime(2026, 1, 15, 22, 15)])
|
||||
with pytest.raises(ValueError, match="口径无法识别"):
|
||||
kline_sync._enforce_minute_beijing_wallclock(df, source="t")
|
||||
|
||||
|
||||
def test_guard_all_null_datetimes_passthrough():
|
||||
"""全 null datetime 维持原行为 (下游落盘过滤), 不误伤。"""
|
||||
df = _minute_frame([None, None])
|
||||
out = kline_sync._enforce_minute_beijing_wallclock(df, source="t")
|
||||
assert out.height == 2
|
||||
assert out["datetime"].null_count() == 2
|
||||
|
||||
|
||||
def test_guard_string_datetimes_classified_after_parse():
|
||||
"""trade_time 字符串路径: 先解析再分类 (UTC 特征串同样自愈)。"""
|
||||
df = pl.DataFrame({
|
||||
"symbol": ["600519.SH"],
|
||||
"trade_time": ["2026-01-15 01:30:00"],
|
||||
"open": [1.0], "high": [1.0], "low": [1.0], "close": [1.0],
|
||||
"volume": [1.0], "amount": [1.0],
|
||||
}).rename({"trade_time": "datetime"})
|
||||
out = kline_sync._enforce_minute_beijing_wallclock(df, source="t")
|
||||
assert out["datetime"].to_list() == [datetime(2026, 1, 15, 9, 30)]
|
||||
|
||||
|
||||
# ---------- 路由级: 自定义源契约违规 → 回退 TickFlow ----------
|
||||
|
||||
def _setup_custom_provider(monkeypatch, provider: object) -> None:
|
||||
monkeypatch.setattr(kline_sync.preferences, "get_minute_data_provider", lambda: "mock_src")
|
||||
monkeypatch.setattr("app.data_providers.custom.provider_has_dataset", lambda name, ds: True)
|
||||
monkeypatch.setattr("app.data_providers.custom.get_provider", lambda name: provider)
|
||||
|
||||
|
||||
def test_custom_provider_utc_frame_selfhealed(monkeypatch):
|
||||
"""插件返回 UTC 墙钟帧 → 路由层守卫 +8 后下发, 不回退。"""
|
||||
mock_provider = MagicMock()
|
||||
mock_provider.get_minute = MagicMock(return_value=_minute_frame(
|
||||
[datetime(2026, 1, 15, 1, 30), datetime(2026, 1, 15, 5, 0)]))
|
||||
_setup_custom_provider(monkeypatch, mock_provider)
|
||||
|
||||
df, fallback = kline_sync._try_custom_minute(
|
||||
["600519.SH"], datetime(2026, 1, 15, 9, 25), datetime(2026, 1, 15, 15, 5),
|
||||
asset_type="stock",
|
||||
)
|
||||
assert fallback is False
|
||||
assert df["datetime"].to_list() == [datetime(2026, 1, 15, 9, 30), datetime(2026, 1, 15, 13, 0)]
|
||||
|
||||
|
||||
def test_custom_provider_garbage_datetime_falls_back(monkeypatch):
|
||||
"""插件返回无法识别口径 → fail-closed 回退 TickFlow。"""
|
||||
mock_provider = MagicMock()
|
||||
mock_provider.get_minute = MagicMock(return_value=_minute_frame(
|
||||
[datetime(2026, 1, 15, 21, 30)]))
|
||||
_setup_custom_provider(monkeypatch, mock_provider)
|
||||
|
||||
df, fallback = kline_sync._try_custom_minute(
|
||||
["600519.SH"], datetime(2026, 1, 15, 9, 25), datetime(2026, 1, 15, 15, 5),
|
||||
asset_type="stock",
|
||||
)
|
||||
assert fallback is True
|
||||
assert df is None
|
||||
@@ -0,0 +1,48 @@
|
||||
"""实时行情轮询间隔下限契约测试。
|
||||
|
||||
下限语义 (中立能力原则):
|
||||
- 实时源路由到插件/自定义源时, TickFlow 档位限速保护不适用 → 通用下限 1s;
|
||||
- 实时源为 tickflow 时, 仍按当前订阅档位查表 (none/free=6s, starter=6s, pro=3s, expert=1s);
|
||||
- 默认间隔 6s 不因路由变化而改变 (只放宽下限, 不动存量偏好)。
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from app.services.quote_service import QuoteService
|
||||
|
||||
|
||||
def _route(monkeypatch, provider: str, tier: str) -> None:
|
||||
from app.services import preferences
|
||||
monkeypatch.setattr(preferences, "get_realtime_data_provider", lambda: provider)
|
||||
monkeypatch.setattr(QuoteService, "_current_tier", classmethod(lambda cls: tier))
|
||||
|
||||
|
||||
def _bare() -> QuoteService:
|
||||
"""绕过 __init__ (单例/线程副作用), 只用无状态方法。"""
|
||||
return QuoteService.__new__(QuoteService)
|
||||
|
||||
|
||||
def test_custom_provider_min_interval_1s(monkeypatch):
|
||||
"""插件/自定义源: 不受 TickFlow 档位保护, 下限放宽到 1s。"""
|
||||
_route(monkeypatch, "fuyao", "none")
|
||||
assert _bare().get_min_interval() == 1.0
|
||||
|
||||
|
||||
def test_custom_provider_clamp_allows_1s(monkeypatch):
|
||||
"""fuyao 路由下设置 1s 不再被抬到 6s; 超过上限仍被压回。"""
|
||||
_route(monkeypatch, "fuyao", "none")
|
||||
qs = _bare()
|
||||
assert qs._clamp_interval(1.0) == 1.0
|
||||
assert qs._clamp_interval(0.5) == 1.0
|
||||
assert qs._clamp_interval(120.0) == QuoteService.MAX_INTERVAL
|
||||
|
||||
|
||||
def test_tickflow_tier_floor_unchanged(monkeypatch):
|
||||
"""TickFlow 路由: 档位查表行为不变 (none/free/starter=6s, pro=3s, expert=1s)。"""
|
||||
for tier, expect in (("none", 6.0), ("free", 6.0), ("starter", 6.0), ("pro", 3.0), ("expert", 1.0)):
|
||||
_route(monkeypatch, "tickflow", tier)
|
||||
assert _bare().get_min_interval() == expect, tier
|
||||
|
||||
|
||||
def test_default_interval_unchanged():
|
||||
"""默认间隔仍是 6s — 放宽的只是下限, 不是默认值。"""
|
||||
assert QuoteService.DEFAULT_INTERVAL == 6.0
|
||||
@@ -0,0 +1,31 @@
|
||||
"""回归测试: 实时行情模式判定 — 免费档不再提供自选实时降级。
|
||||
|
||||
watchlist(自选前 5 只)模式已于 2026-08 移除: 自定义实时源(如 fuyao)的
|
||||
全市场快照免费且更优, TickFlow 免费档不再保留降级通路。锁定判定结果,
|
||||
防止该通路被无意恢复。
|
||||
"""
|
||||
from app.services.quote_service import QuoteService
|
||||
|
||||
|
||||
def test_custom_realtime_source_is_full_market(monkeypatch):
|
||||
"""自定义实时源(如 fuyao)无视 TickFlow 档位, 恒为全市场。"""
|
||||
from app.services import preferences
|
||||
monkeypatch.setattr(preferences, "get_realtime_data_provider", lambda: "fuyao")
|
||||
monkeypatch.setattr(QuoteService, "_current_tier", lambda: "free")
|
||||
assert QuoteService.realtime_mode() == "full_market"
|
||||
|
||||
|
||||
def test_tickflow_free_has_no_realtime(monkeypatch):
|
||||
"""TickFlow 免费档 = 无实时(不再降级为自选模式)。"""
|
||||
from app.services import preferences
|
||||
monkeypatch.setattr(preferences, "get_realtime_data_provider", lambda: "tickflow")
|
||||
monkeypatch.setattr(QuoteService, "_current_tier", lambda: "free")
|
||||
assert QuoteService.realtime_mode() == "none"
|
||||
assert QuoteService.is_realtime_allowed() is False
|
||||
|
||||
|
||||
def test_tickflow_paid_is_full_market(monkeypatch):
|
||||
from app.services import preferences
|
||||
monkeypatch.setattr(preferences, "get_realtime_data_provider", lambda: "tickflow")
|
||||
monkeypatch.setattr(QuoteService, "_current_tier", lambda: "pro")
|
||||
assert QuoteService.realtime_mode() == "full_market"
|
||||
@@ -41,6 +41,7 @@ def test_all_builtin_strategies_declare_asset_types_and_timeframes():
|
||||
assert engine.load_errors() == []
|
||||
for meta in engine.list_strategies():
|
||||
assert meta["asset_types"]
|
||||
# 分钟红7已迁至自定义策略目录, 内置策略均为日线
|
||||
assert meta["timeframes"] == ["1d"]
|
||||
|
||||
|
||||
@@ -48,11 +49,13 @@ def test_all_builtin_strategies_use_matrix_backend_only():
|
||||
engine = _engine()
|
||||
assert engine.load_errors() == []
|
||||
strategies = [engine.get(meta["id"]) for meta in engine.list_strategies()]
|
||||
assert len(strategies) == 18
|
||||
assert all(strategy.execution_backend == "matrix_native" for strategy in strategies)
|
||||
assert all(strategy.matrix_strategy is not None for strategy in strategies)
|
||||
assert all(strategy.filter_fn is None for strategy in strategies)
|
||||
assert all(strategy.filter_history_fn is None for strategy in strategies)
|
||||
matrix_strategies = [s for s in strategies if s.execution_backend == "matrix_native"]
|
||||
assert len(matrix_strategies) == 18
|
||||
assert all(s.matrix_strategy is not None for s in matrix_strategies)
|
||||
assert all(s.filter_fn is None for s in matrix_strategies)
|
||||
assert all(s.filter_history_fn is None for s in matrix_strategies)
|
||||
# 分钟形态策略 (minute_filter) 已迁至自定义策略目录, 不在 builtin 加载范围
|
||||
assert [s.meta["id"] for s in strategies if s.execution_backend == "minute_filter"] == []
|
||||
|
||||
|
||||
def test_all_builtin_matrix_formulas_accept_base_market_matrix():
|
||||
@@ -79,10 +82,14 @@ def test_all_builtin_matrix_formulas_accept_base_market_matrix():
|
||||
from app.backtest.matrix import build_market_data_matrix
|
||||
|
||||
fields = set()
|
||||
for strategy in (engine.get(meta["id"]) for meta in engine.list_strategies()):
|
||||
matrix_metas = [
|
||||
m for m in engine.list_strategies()
|
||||
if engine.get(m["id"]).execution_backend == "matrix_native"
|
||||
]
|
||||
for strategy in (engine.get(meta["id"]) for meta in matrix_metas):
|
||||
fields.update(engine._matrix_field_columns(strategy))
|
||||
market = build_market_data_matrix(panel, field_columns=fields)
|
||||
for meta in engine.list_strategies():
|
||||
for meta in matrix_metas:
|
||||
strategy = engine.get(meta["id"])
|
||||
signals = strategy.matrix_strategy.compute_signals(market, {})
|
||||
assert signals.shape == market.shape, meta["id"]
|
||||
|
||||
@@ -226,3 +226,82 @@ def test_builtin_not_editable():
|
||||
raise AssertionError("expected ValueError for builtin")
|
||||
except ValueError:
|
||||
pass
|
||||
|
||||
|
||||
# ---------- 分钟 open 数据卫生 (stock-sdk 上游区间模式给日级常量伪 open) ----------
|
||||
|
||||
def _sdk_rows(day: str, opens, closes):
|
||||
"""构造 bridge 返回形状的分钟行 (timestamp 为北京墙钟对应 UTC 毫秒)。"""
|
||||
from datetime import UTC, datetime, timedelta
|
||||
rows = []
|
||||
for i, (o, c) in enumerate(zip(opens, closes, strict=True)):
|
||||
dt = datetime.fromisoformat(f"{day} 09:30:00") + timedelta(minutes=i)
|
||||
ts = int(dt.replace(tzinfo=UTC).timestamp() * 1000) - 8 * 3600_000
|
||||
rows.append({"timestamp": ts, "open": o, "high": max(o, c), "low": min(o, c),
|
||||
"close": c, "volume": 100 + i, "amount": 1000.0 + i})
|
||||
return rows
|
||||
|
||||
|
||||
def test_minute_degenerate_open_nulled():
|
||||
"""历史日 open 为全天常量(uniq=1)而 close 多值 → open 置 null。"""
|
||||
n = 30
|
||||
rows = _sdk_rows("2026-08-27", [8.0] * n, [10 + i * 0.01 for i in range(n)])
|
||||
df = StockSDKProvider._minute_df(rows, "600664.SH")
|
||||
assert df.height == n
|
||||
assert df["open"].null_count() == n # 伪 open 全部置 null
|
||||
assert df["close"].null_count() == 0 # close/high/low 保留
|
||||
|
||||
|
||||
def test_minute_real_open_kept():
|
||||
"""真实分钟 open (多唯一值) 原样保留。"""
|
||||
n = 30
|
||||
rows = _sdk_rows("2026-08-28", [10 + i * 0.01 for i in range(n)], [10.05 + i * 0.01 for i in range(n)])
|
||||
df = StockSDKProvider._minute_df(rows, "600664.SH")
|
||||
assert df["open"].null_count() == 0
|
||||
assert df["open"].n_unique() == n
|
||||
|
||||
|
||||
def test_minute_short_day_open_kept():
|
||||
"""短交易日 (rows<=10, 如半日/首日少量bar) 不误杀。"""
|
||||
rows = _sdk_rows("2026-08-26", [8.0] * 6, [8.0 + i * 0.1 for i in range(6)])
|
||||
df = StockSDKProvider._minute_df(rows, "600664.SH")
|
||||
assert df["open"].null_count() == 0
|
||||
|
||||
|
||||
def test_get_minute_splits_tail_into_single_day_jobs(monkeypatch):
|
||||
"""多日区间 → 末尾 3 自然日(跳过周末)逐日单拉 (保住最新交易日真实 open)。"""
|
||||
from datetime import datetime
|
||||
|
||||
jobs = []
|
||||
|
||||
def fake_run_job(job, timeout=None):
|
||||
jobs.append(job)
|
||||
return {"ok": True, "op": "minute", "rows": {}}
|
||||
|
||||
monkeypatch.setattr(sp.bridge, "run_job", fake_run_job)
|
||||
p = StockSDKProvider()
|
||||
p.get_minute(["600519.SH"], datetime(2026, 8, 20), datetime(2026, 8, 29, 23, 0))
|
||||
|
||||
# 末尾 4 个自然日 (08-26..08-29) 逐日单拉, 周六 08-29 跳过;
|
||||
# 前段 = [08-20, 08-25] 一个区间任务
|
||||
spans = [(j["start"], j["end"]) for j in jobs]
|
||||
for day in ("20260826", "20260827", "20260828"):
|
||||
assert (day, day) in spans
|
||||
assert ("20260829", "20260829") not in spans # 周六不单拉
|
||||
assert ("20260820", "20260825") in spans
|
||||
|
||||
|
||||
def test_get_minute_single_day_no_split(monkeypatch):
|
||||
"""单日区间不拆分, 保持一个任务。"""
|
||||
from datetime import datetime
|
||||
|
||||
jobs = []
|
||||
|
||||
def fake_run_job(job, timeout=None):
|
||||
jobs.append(job)
|
||||
return {"ok": True, "op": "minute", "rows": {}}
|
||||
|
||||
monkeypatch.setattr(sp.bridge, "run_job", fake_run_job)
|
||||
StockSDKProvider().get_minute(["600519.SH"], datetime(2026, 8, 28), datetime(2026, 8, 28, 15, 0))
|
||||
assert len(jobs) == 1
|
||||
assert jobs[0]["start"] == "20260828"
|
||||
|
||||
@@ -0,0 +1,257 @@
|
||||
"""交易日探针 (trading_day oracle) 与两个消费方接入的测试。
|
||||
|
||||
不依赖真实网络: 探测函数 (_probe_fuyao / _probe_tickflow) 全部 monkeypatch。
|
||||
覆盖: 周末零成本直判、探测链优先级 (fuyao 日历权威, 无开盘缓冲问题)、
|
||||
tickflow 戳的 OR 语义与开盘缓冲窗、失败/无权限 → None、TTL 缓存、
|
||||
实时行情门控与分钟增量 gate_reason 的 holiday 分支。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import date, datetime, time as dt_time, timezone, timedelta
|
||||
|
||||
import pytest
|
||||
|
||||
from app.services import trading_day
|
||||
from app.services.trading_day import is_trading_day, reset_cache
|
||||
|
||||
CN = timezone(timedelta(hours=8))
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def _clean_cache():
|
||||
reset_cache()
|
||||
yield
|
||||
reset_cache()
|
||||
|
||||
|
||||
def _no_probes(monkeypatch):
|
||||
"""探测函数替换为爆炸 — 用于验证未被打到。"""
|
||||
monkeypatch.setattr(trading_day, "_probe_fuyao", lambda now: (_ for _ in ()).throw(AssertionError("不应探测")))
|
||||
monkeypatch.setattr(trading_day, "_probe_tickflow", lambda now: (_ for _ in ()).throw(AssertionError("不应探测")))
|
||||
|
||||
|
||||
# ---- 周末零成本直判 ----
|
||||
|
||||
def test_weekend_returns_false_without_probing(monkeypatch):
|
||||
sat = datetime(2026, 8, 29, 10, 0, tzinfo=CN) # 周六
|
||||
sun = datetime(2026, 8, 30, 10, 0, tzinfo=CN) # 周日
|
||||
_no_probes(monkeypatch)
|
||||
assert is_trading_day(sat) is False
|
||||
assert is_trading_day(sun) is False
|
||||
|
||||
|
||||
# ---- 探测链优先级 ----
|
||||
|
||||
def test_fuyao_calendar_is_authoritative_even_before_open(monkeypatch):
|
||||
"""fuyao 日历结论无时段依赖: 开盘缓冲窗内也直接生效。"""
|
||||
holiday_mon = datetime(2026, 9, 7, 9, 31, tzinfo=CN) # 周一 (缓冲窗内)
|
||||
monkeypatch.setattr(trading_day, "_probe_fuyao", lambda now: False)
|
||||
monkeypatch.setattr(
|
||||
trading_day, "_probe_tickflow",
|
||||
lambda now: (_ for _ in ()).throw(AssertionError("fuyao 已有结论不应继续探测")),
|
||||
)
|
||||
assert is_trading_day(holiday_mon) is False
|
||||
|
||||
|
||||
def test_chain_falls_through_to_tickflow_when_fuyao_unknown(monkeypatch):
|
||||
monday = datetime(2026, 9, 7, 10, 0, tzinfo=CN)
|
||||
monkeypatch.setattr(trading_day, "_probe_fuyao", lambda now: None)
|
||||
monkeypatch.setattr(trading_day, "_probe_tickflow", lambda now: True)
|
||||
assert is_trading_day(monday) is True
|
||||
|
||||
|
||||
def test_all_probes_unknown_returns_none(monkeypatch):
|
||||
monday = datetime(2026, 9, 7, 10, 0, tzinfo=CN)
|
||||
monkeypatch.setattr(trading_day, "_probe_fuyao", lambda now: None)
|
||||
monkeypatch.setattr(trading_day, "_probe_tickflow", lambda now: None)
|
||||
assert is_trading_day(monday) is None
|
||||
|
||||
|
||||
# ---- tickflow 戳语义 ----
|
||||
|
||||
def test_tickflow_stale_stamp_before_buffer_is_unknown(monkeypatch):
|
||||
"""开盘缓冲窗内戳停在昨日: 可能是集合竞价未翻新, 保守判未知。"""
|
||||
monday_935 = datetime(2026, 9, 7, 9, 35, tzinfo=CN)
|
||||
import app.tickflow.client as tf_client_mod
|
||||
|
||||
class _FakeQuotes:
|
||||
def get(self, symbols):
|
||||
# 周一 9:35 拉到上周五 15:30 的戳
|
||||
return [{"symbol": s, "timestamp": 1787902251001} for s in symbols]
|
||||
|
||||
class _FakeClient:
|
||||
quotes = _FakeQuotes()
|
||||
|
||||
monkeypatch.setattr(tf_client_mod, "get_client", lambda: _FakeClient())
|
||||
monkeypatch.setattr(trading_day, "_probe_fuyao", lambda now: None)
|
||||
assert trading_day._probe_tickflow(monday_935) is None
|
||||
|
||||
|
||||
def test_tickflow_stale_stamp_after_buffer_is_holiday(monkeypatch):
|
||||
monday_1041 = datetime(2026, 9, 7, 10, 41, tzinfo=CN)
|
||||
import app.tickflow.client as tf_client_mod
|
||||
|
||||
class _FakeQuotes:
|
||||
def get(self, symbols):
|
||||
return [{"symbol": s, "timestamp": 1787902251001} for s in symbols] # 上周五
|
||||
|
||||
class _FakeClient:
|
||||
quotes = _FakeQuotes()
|
||||
|
||||
monkeypatch.setattr(tf_client_mod, "get_client", lambda: _FakeClient())
|
||||
assert trading_day._probe_tickflow(monday_1041) is False
|
||||
|
||||
|
||||
def test_tickflow_fresh_stamp_is_trading_anytime(monkeypatch):
|
||||
"""缓冲窗内只要戳是今天就判交易日 (OR 语义, 任一翻新即可)。"""
|
||||
monday_920 = datetime(2026, 9, 7, 9, 20, tzinfo=CN)
|
||||
import app.tickflow.client as tf_client_mod
|
||||
|
||||
class _FakeQuotes:
|
||||
def get(self, symbols):
|
||||
# 一只翻新 + 其余停在周五 → max 为今日
|
||||
fresh_ms = int(monday_920.timestamp() * 1000)
|
||||
return [
|
||||
{"symbol": "000001.SZ", "timestamp": fresh_ms},
|
||||
{"symbol": "600519.SH", "timestamp": 1787902251001},
|
||||
]
|
||||
|
||||
class _FakeClient:
|
||||
quotes = _FakeQuotes()
|
||||
|
||||
monkeypatch.setattr(tf_client_mod, "get_client", lambda: _FakeClient())
|
||||
assert trading_day._probe_tickflow(monday_920) is True
|
||||
|
||||
|
||||
def test_tickflow_error_returns_none(monkeypatch):
|
||||
monday = datetime(2026, 9, 7, 10, 41, tzinfo=CN)
|
||||
import app.tickflow.client as tf_client_mod
|
||||
|
||||
def _boom():
|
||||
raise RuntimeError("无实时权限 (free 档)")
|
||||
|
||||
monkeypatch.setattr(tf_client_mod, "get_client", _boom)
|
||||
assert trading_day._probe_tickflow(monday) is None
|
||||
|
||||
|
||||
# ---- TTL 缓存 ----
|
||||
|
||||
def test_verdict_cached_within_ttl(monkeypatch):
|
||||
monday = datetime(2026, 9, 7, 10, 0, tzinfo=CN)
|
||||
calls = {"n": 0}
|
||||
|
||||
def _counting_probe(now):
|
||||
calls["n"] += 1
|
||||
return True
|
||||
|
||||
monkeypatch.setattr(trading_day, "_probe_fuyao", _counting_probe)
|
||||
assert is_trading_day(monday) is True
|
||||
assert is_trading_day(monday) is True
|
||||
assert is_trading_day(monday) is True
|
||||
assert calls["n"] == 1 # 命中缓存, 只探一次
|
||||
|
||||
|
||||
def test_unknown_verdict_retries_after_short_ttl(monkeypatch):
|
||||
monday = datetime(2026, 9, 7, 10, 0, tzinfo=CN)
|
||||
calls = {"n": 0}
|
||||
|
||||
def _counting_probe(now):
|
||||
calls["n"] += 1
|
||||
return None
|
||||
|
||||
monkeypatch.setattr(trading_day, "_probe_fuyao", _counting_probe)
|
||||
monkeypatch.setattr(trading_day, "_probe_tickflow", lambda now: None) # 隔离真实网络
|
||||
assert is_trading_day(monday) is None
|
||||
# 手动把缓存时间拨回 10 分钟前 (超过 unknown TTL 300s) → 重探
|
||||
with trading_day._CACHE_LOCK:
|
||||
trading_day._CACHE.probed_at -= 600
|
||||
assert is_trading_day(monday) is None
|
||||
assert calls["n"] == 2
|
||||
|
||||
|
||||
# ---- 消费方 1: 实时行情门控 ----
|
||||
|
||||
def test_quote_service_holiday_gate_blocks_polling(monkeypatch):
|
||||
from app.services.quote_service import QuoteService
|
||||
|
||||
qs = QuoteService()
|
||||
monkeypatch.setattr(trading_day, "is_trading_day", lambda now=None: False)
|
||||
assert qs._should_poll_for_phase("morning") is False
|
||||
assert qs._should_poll_for_phase("preopen") is False
|
||||
# final 定版同样被剔除: 休市日没有需要定版的当日行情
|
||||
assert qs._should_poll_for_phase("close_final") is False
|
||||
|
||||
|
||||
def test_quote_service_polls_when_trading_or_unknown(monkeypatch):
|
||||
from app.services.quote_service import QuoteService
|
||||
|
||||
qs = QuoteService()
|
||||
monkeypatch.setattr(trading_day, "is_trading_day", lambda now=None: True)
|
||||
assert qs._should_poll_for_phase("morning") is True
|
||||
# 未知 → 维持现状 (周几近似)
|
||||
monkeypatch.setattr(trading_day, "is_trading_day", lambda now=None: None)
|
||||
assert qs._should_poll_for_phase("morning") is True
|
||||
|
||||
|
||||
# ---- 消费方 2: 分钟增量 gate_reason ----
|
||||
|
||||
def _minute_service(monkeypatch):
|
||||
from app.services.minute_refresh import MinuteRefreshService
|
||||
|
||||
svc = MinuteRefreshService.__new__(MinuteRefreshService)
|
||||
svc._app_state = None
|
||||
monkeypatch.setattr(
|
||||
"app.services.minute_refresh.preferences.get_minute_refresh_enabled",
|
||||
lambda: True,
|
||||
)
|
||||
monkeypatch.setattr(svc, "custom_provider_active", lambda: False)
|
||||
monkeypatch.setattr(svc, "capability_ok", lambda: True)
|
||||
return svc
|
||||
|
||||
|
||||
def test_minute_refresh_gate_returns_holiday(monkeypatch):
|
||||
"""周几+时段门控放行 (周一盘中) 但探针判休市 → holiday。"""
|
||||
svc = _minute_service(monkeypatch)
|
||||
monday_1030 = datetime(2026, 9, 7, 10, 30, tzinfo=CN)
|
||||
monkeypatch.setattr(
|
||||
"app.services.minute_refresh._in_continuous_session", lambda now=None: True
|
||||
)
|
||||
monkeypatch.setattr(trading_day, "is_trading_day", lambda now=None: False)
|
||||
assert svc._gate_reason() == "holiday"
|
||||
|
||||
|
||||
def test_minute_refresh_gate_passes_when_trading(monkeypatch):
|
||||
svc = _minute_service(monkeypatch)
|
||||
monkeypatch.setattr(
|
||||
"app.services.minute_refresh._in_continuous_session", lambda now=None: True
|
||||
)
|
||||
monkeypatch.setattr(trading_day, "is_trading_day", lambda now=None: True)
|
||||
assert svc._gate_reason() is None
|
||||
|
||||
|
||||
# ---- fuyao 日历解析 ----
|
||||
|
||||
def test_fuyao_provider_trading_days_conversion(monkeypatch):
|
||||
from app.plugins.fuyao.provider import FuyaoProvider
|
||||
from app.plugins.fuyao import provider as fp
|
||||
|
||||
class _CalClient:
|
||||
def trading_days(self):
|
||||
# 上海零点戳按 provider 的 _ms_of_date 口径构造
|
||||
def _midnight_ms(d):
|
||||
import calendar
|
||||
return (calendar.timegm(d.timetuple()) - 28800) * 1000
|
||||
|
||||
d4, d7 = date(2026, 9, 4), date(2026, 9, 7)
|
||||
return [
|
||||
{"date": "20260904", "date_ms": _midnight_ms(d4)},
|
||||
{"date": "20260907", "date_ms": _midnight_ms(d7)},
|
||||
]
|
||||
|
||||
monkeypatch.setattr(
|
||||
fp, "fuyao_client", type("M", (), {"FuyaoClient": lambda **kw: _CalClient()})
|
||||
)
|
||||
monkeypatch.setattr(fp, "get_api_key", lambda: "test-key")
|
||||
days = FuyaoProvider().trading_days()
|
||||
assert days == {date(2026, 9, 4), date(2026, 9, 7)}
|
||||
@@ -0,0 +1,274 @@
|
||||
"""轮询放量监控 (volume_delta) 测试: 引擎命中/冷却/批量合并 + 基础过滤 + 快照差值边界。"""
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import date
|
||||
|
||||
import polars as pl
|
||||
import pytest
|
||||
|
||||
from app.strategy import monitor_rules
|
||||
from app.strategy.monitor import MonitorRuleEngine
|
||||
|
||||
|
||||
def _df(rows: list[dict]) -> pl.DataFrame:
|
||||
"""rows 每项: symbol/_volume_delta 必填, 其余可选 (close/amount/total_shares/float_shares)。"""
|
||||
base = {
|
||||
"symbol": [], "close": [], "change_pct": [],
|
||||
"_volume_delta": [], "_volume_delta_amount": [], "_volume_delta_span": [],
|
||||
}
|
||||
optional = ["amount", "total_shares", "float_shares"]
|
||||
for r in rows:
|
||||
base["symbol"].append(r["symbol"])
|
||||
base["close"].append(r.get("close", 10.0))
|
||||
base["change_pct"].append(0.01)
|
||||
base["_volume_delta"].append(r["_volume_delta"])
|
||||
base["_volume_delta_amount"].append(r.get("_volume_delta_amount", r["_volume_delta"] * 1000.0))
|
||||
base["_volume_delta_span"].append(6.0)
|
||||
data = {k: v for k, v in base.items()}
|
||||
for col in optional:
|
||||
vals = [r.get(col) for r in rows]
|
||||
if any(v is not None for v in vals):
|
||||
data[col] = [v if v is not None else 0.0 for v in vals]
|
||||
return pl.DataFrame(data)
|
||||
|
||||
|
||||
def _rule(**kw):
|
||||
r = {
|
||||
"id": "vd1", "name": "轮询放量", "type": "volume_delta",
|
||||
"asset_type": "stock", "scope": "all", "enabled": True,
|
||||
"threshold_volume": 9000, "cooldown_seconds": 300,
|
||||
"severity": "warn",
|
||||
}
|
||||
r.update(kw)
|
||||
return r
|
||||
|
||||
|
||||
def test_volume_delta_hits_above_threshold():
|
||||
eng = MonitorRuleEngine()
|
||||
eng.set_rules([_rule()])
|
||||
events = eng.evaluate(_df([
|
||||
{"symbol": "S1.SH", "_volume_delta": 9500.0, "close": 10.0},
|
||||
{"symbol": "S2.SH", "_volume_delta": 8999.0, "close": 20.0},
|
||||
]))
|
||||
assert [e["symbol"] for e in events] == ["S1.SH"]
|
||||
ev = events[0]
|
||||
assert ev["source"] == "volume_delta"
|
||||
assert "9,500" in ev["message"] and "9,000" in ev["message"] and "间隔 6s" in ev["message"]
|
||||
assert ev["volume_delta"] == 9500.0
|
||||
|
||||
|
||||
def test_volume_delta_no_column_degrades_silently():
|
||||
eng = MonitorRuleEngine()
|
||||
eng.set_rules([_rule()])
|
||||
plain = pl.DataFrame({"symbol": ["S1.SH"], "close": [10.0]})
|
||||
assert eng.evaluate(plain) == []
|
||||
|
||||
|
||||
def test_volume_delta_cooldown_suppresses_repeat():
|
||||
eng = MonitorRuleEngine()
|
||||
eng.set_rules([_rule(cooldown=300)])
|
||||
df = _df([{"symbol": "S1.SH", "_volume_delta": 12000.0}])
|
||||
assert len(eng.evaluate(df)) == 1
|
||||
assert eng.evaluate(df) == []
|
||||
|
||||
|
||||
def test_volume_delta_batch_merge_over_five():
|
||||
eng = MonitorRuleEngine()
|
||||
eng.set_rules([_rule()])
|
||||
rows = [{"symbol": f"S{i}.SH", "_volume_delta": 20000.0 + i} for i in range(8)]
|
||||
events = eng.evaluate(_df(rows))
|
||||
assert len(events) == 1
|
||||
assert events[0]["symbol"] == ""
|
||||
assert "共 8 只" in events[0]["message"]
|
||||
|
||||
|
||||
def test_volume_delta_scope_filters():
|
||||
eng = MonitorRuleEngine()
|
||||
eng.set_rules([_rule(scope="symbols", symbols=["S2.SH"])])
|
||||
events = eng.evaluate(_df([
|
||||
{"symbol": "S1.SH", "_volume_delta": 9500.0},
|
||||
{"symbol": "S2.SH", "_volume_delta": 9500.0},
|
||||
]))
|
||||
assert [e["symbol"] for e in events] == ["S2.SH"]
|
||||
|
||||
|
||||
def test_volume_delta_metric_amount():
|
||||
eng = MonitorRuleEngine()
|
||||
eng.set_rules([_rule(metric="amount", threshold_amount=5e6)])
|
||||
events = eng.evaluate(_df([
|
||||
{"symbol": "S1.SH", "_volume_delta": 100.0, "_volume_delta_amount": 6e6},
|
||||
{"symbol": "S2.SH", "_volume_delta": 20000.0, "_volume_delta_amount": 4.9e6},
|
||||
]))
|
||||
assert [e["symbol"] for e in events] == ["S1.SH"]
|
||||
assert "万元" in events[0]["message"]
|
||||
|
||||
|
||||
def test_volume_delta_basic_filter_price_and_amount():
|
||||
eng = MonitorRuleEngine()
|
||||
eng.set_rules([_rule(basic_filter={
|
||||
"price_min": 5, "price_max": 100, "amount_min": 1e8, "exclude_st": False,
|
||||
})])
|
||||
events = eng.evaluate(_df([
|
||||
# 价低被滤
|
||||
{"symbol": "LOW.SH", "_volume_delta": 20000.0, "close": 3.0, "amount": 5e8},
|
||||
# 价过高被滤
|
||||
{"symbol": "HIGH.SH", "_volume_delta": 20000.0, "close": 200.0, "amount": 5e8},
|
||||
# 成交额不足被滤
|
||||
{"symbol": "THIN.SH", "_volume_delta": 20000.0, "close": 10.0, "amount": 5e7},
|
||||
# 通过
|
||||
{"symbol": "OK.SH", "_volume_delta": 20000.0, "close": 10.0, "amount": 5e8},
|
||||
]))
|
||||
assert [e["symbol"] for e in events] == ["OK.SH"]
|
||||
|
||||
|
||||
def test_volume_delta_basic_filter_market_cap():
|
||||
eng = MonitorRuleEngine()
|
||||
eng.set_rules([_rule(basic_filter={
|
||||
"market_cap_min": 20e8, "price_min": None, "price_max": None,
|
||||
"amount_min": None, "exclude_st": False,
|
||||
})])
|
||||
# close × total_shares: BIG 10×3e8=30亿 通过; SMALL 10×1e8=10亿 被滤
|
||||
events = eng.evaluate(_df([
|
||||
{"symbol": "BIG.SH", "_volume_delta": 20000.0, "total_shares": 3e8},
|
||||
{"symbol": "SMALL.SH", "_volume_delta": 20000.0, "total_shares": 1e8},
|
||||
]))
|
||||
assert [e["symbol"] for e in events] == ["BIG.SH"]
|
||||
|
||||
|
||||
def test_volume_delta_basic_filter_exclude_st():
|
||||
eng = MonitorRuleEngine()
|
||||
eng.set_name_map({"STOCK.SH": "平安银行", "STK.SH": "ST 某某"})
|
||||
eng.set_rules([_rule(basic_filter={
|
||||
"price_min": None, "price_max": None, "amount_min": None, "exclude_st": True,
|
||||
})])
|
||||
events = eng.evaluate(_df([
|
||||
{"symbol": "STOCK.SH", "_volume_delta": 20000.0},
|
||||
{"symbol": "STK.SH", "_volume_delta": 20000.0},
|
||||
]))
|
||||
assert [e["symbol"] for e in events] == ["STOCK.SH"]
|
||||
|
||||
|
||||
def test_validate_and_normalize_defaults():
|
||||
r = monitor_rules.normalize({"id": "vd2", "type": "volume_delta"})
|
||||
assert r["threshold_volume"] == 9000
|
||||
assert r["scope"] == "all"
|
||||
assert r["cooldown_seconds"] == 300
|
||||
assert r["metric"] == "volume"
|
||||
assert r["basic_filter"]["price_min"] == 3
|
||||
assert r["basic_filter"]["exclude_st"] is True
|
||||
# 用户字段覆盖默认
|
||||
r2 = monitor_rules.normalize({"id": "vd5", "type": "volume_delta", "basic_filter": {"price_min": 1, "exclude_st": False}})
|
||||
assert r2["basic_filter"]["price_min"] == 1
|
||||
assert r2["basic_filter"]["exclude_st"] is False
|
||||
assert r2["basic_filter"]["price_max"] == 300 # 未覆盖项保留默认
|
||||
monitor_rules.validate({"id": "vd2", "name": "n", "type": "volume_delta", "threshold_volume": 1})
|
||||
with pytest.raises(ValueError):
|
||||
monitor_rules.validate({"id": "vd3", "name": "n", "type": "volume_delta", "threshold_volume": 0})
|
||||
with pytest.raises(ValueError):
|
||||
monitor_rules.validate({"id": "vd4", "name": "n", "type": "volume_delta", "asset_type": "etf"})
|
||||
with pytest.raises(ValueError):
|
||||
monitor_rules.validate({"id": "vd6", "name": "n", "type": "volume_delta",
|
||||
"metric": "amount", "threshold_amount": 0})
|
||||
with pytest.raises(ValueError):
|
||||
monitor_rules.validate({"id": "vd7", "name": "n", "type": "volume_delta",
|
||||
"basic_filter": {"price_min": -1}})
|
||||
with pytest.raises(ValueError):
|
||||
monitor_rules.validate({"id": "vd8", "name": "n", "type": "volume_delta",
|
||||
"basic_filter": {"unknown_field": 1}})
|
||||
|
||||
|
||||
# ── 快照差值状态 (QuoteService) ──────────────────────────
|
||||
|
||||
def _qs(monkeypatch, *, continuous=True):
|
||||
from app.services.quote_service import QuoteService
|
||||
qs = QuoteService.__new__(QuoteService)
|
||||
qs._prev_stock_volume = None
|
||||
qs._prev_volume_fetched_at = None
|
||||
qs._prev_volume_date = None
|
||||
qs._volume_delta = {}
|
||||
qs._volume_delta_span_s = 0.0
|
||||
monkeypatch.setattr(QuoteService, "_is_continuous_trading", lambda self: continuous)
|
||||
monkeypatch.setattr(
|
||||
QuoteService, "_continuous_session_start_ms",
|
||||
staticmethod(lambda: 0.0),
|
||||
)
|
||||
monkeypatch.setattr("app.services.quote_service.cn_today", lambda: date(2026, 8, 25))
|
||||
return qs
|
||||
|
||||
|
||||
def test_delta_computed_and_prev_updated(monkeypatch):
|
||||
qs = _qs(monkeypatch)
|
||||
t0 = 1_000_000.0
|
||||
qs._update_volume_delta(
|
||||
[{"symbol": "S1.SH", "volume": 10000, "amount": 5e6},
|
||||
{"symbol": "S2.SH", "volume": 500, "amount": 1e6}], t0,
|
||||
)
|
||||
assert qs._volume_delta == {} # 首轮无 prev
|
||||
qs._update_volume_delta(
|
||||
[{"symbol": "S1.SH", "volume": 19500, "amount": 9.5e6},
|
||||
{"symbol": "S2.SH", "volume": 400, "amount": 2e6}], t0 + 6000,
|
||||
)
|
||||
# S2 volume cur < prev (重置) → 丢弃; S1 差值 (9500 手, 450 万元)
|
||||
assert qs._volume_delta == {"S1.SH": (9500.0, 4.5e6)}
|
||||
assert qs._volume_delta_span_s == 6.0
|
||||
|
||||
|
||||
def test_delta_cross_day_reset(monkeypatch):
|
||||
import app.services.quote_service as qsm
|
||||
qs = _qs(monkeypatch)
|
||||
qs._update_volume_delta([{"symbol": "S1.SH", "volume": 10000}], 1000.0)
|
||||
assert qs._prev_volume_date == date(2026, 8, 25)
|
||||
monkeypatch.setattr(qsm, "cn_today", lambda: date(2026, 8, 26))
|
||||
qs._update_volume_delta([{"symbol": "S1.SH", "volume": 20000}], 2000.0)
|
||||
assert qs._volume_delta == {}
|
||||
assert qs._prev_volume_date == date(2026, 8, 26)
|
||||
|
||||
|
||||
def test_delta_open_protection(monkeypatch):
|
||||
qs = _qs(monkeypatch)
|
||||
# 9:29 的 prev (早于 9:30 时段起点) → 9:31 本轮不触发
|
||||
session_start = 1_000_000.0
|
||||
monkeypatch.setattr(
|
||||
type(qs), "_continuous_session_start_ms",
|
||||
staticmethod(lambda: session_start),
|
||||
)
|
||||
qs._update_volume_delta([{"symbol": "S1.SH", "volume": 10000}], session_start - 60_000)
|
||||
qs._update_volume_delta([{"symbol": "S1.SH", "volume": 99999}], session_start + 60_000)
|
||||
assert qs._volume_delta == {}
|
||||
# 之后一轮 prev 已在时段内 → 恢复计算
|
||||
qs._update_volume_delta([{"symbol": "S1.SH", "volume": 109999}], session_start + 66_000)
|
||||
assert qs._volume_delta == {"S1.SH": (10000.0, 0.0)}
|
||||
|
||||
|
||||
def test_delta_not_continuous_trading(monkeypatch):
|
||||
qs = _qs(monkeypatch, continuous=False)
|
||||
qs._update_volume_delta([{"symbol": "S1.SH", "volume": 10000}], 1000.0)
|
||||
qs._update_volume_delta([{"symbol": "S1.SH", "volume": 99999}], 7000.0)
|
||||
# 非连续竞价 (如午休) 不产差值, 但 prev 持续更新
|
||||
assert qs._volume_delta == {}
|
||||
assert qs._prev_stock_volume == {"S1.SH": (99999.0, 0.0)}
|
||||
|
||||
|
||||
def test_inject_volume_delta_join():
|
||||
from app.services.quote_service import QuoteService
|
||||
qs = QuoteService.__new__(QuoteService)
|
||||
qs._volume_delta = {"S1.SH": (900.0, 9e5), "S9.SH": (500.0, 5e5)}
|
||||
qs._volume_delta_span_s = 6.0
|
||||
base = pl.DataFrame({"symbol": ["S1.SH", "S2.SH"], "close": [10.0, 20.0]})
|
||||
out = qs._inject_volume_delta(base)
|
||||
assert out.filter(pl.col("symbol") == "S1.SH")["_volume_delta"][0] == 900.0
|
||||
assert out.filter(pl.col("symbol") == "S1.SH")["_volume_delta_amount"][0] == 9e5
|
||||
# 未命中股票为 null (不触发)
|
||||
assert out.filter(pl.col("symbol") == "S2.SH")["_volume_delta"][0] is None
|
||||
# 空差值原样返回
|
||||
qs._volume_delta = {}
|
||||
assert qs._inject_volume_delta(base).columns == ["symbol", "close"]
|
||||
|
||||
|
||||
def test_session_start_ms_matches_clock():
|
||||
from app.services.quote_service import QuoteService
|
||||
from datetime import datetime, time as dt_time, timedelta, timezone
|
||||
|
||||
now = QuoteService._continuous_session_start_ms() / 1000.0
|
||||
start_dt = datetime.fromtimestamp(now, tz=timezone(timedelta(hours=8)))
|
||||
assert start_dt.time() in (dt_time(9, 30), dt_time(13, 0))
|
||||
@@ -1,123 +0,0 @@
|
||||
"""回归测试: Free 档自选实时 symbols 超过 capability batch 上限时分批请求 (PR #46 问题 4)。"""
|
||||
from contextlib import ExitStack
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import polars as pl
|
||||
|
||||
from app.services.quote_service import QuoteService
|
||||
from app.tickflow.capabilities import Cap, CapabilityLimits, CapabilitySet
|
||||
|
||||
|
||||
def _make_svc(engine_rules: dict) -> QuoteService:
|
||||
"""创建最小可用的 QuoteService 实例 (跳过 __init__)。"""
|
||||
svc = QuoteService.__new__(QuoteService)
|
||||
svc._app_state = MagicMock()
|
||||
svc._repo = MagicMock()
|
||||
svc._lock = MagicMock()
|
||||
|
||||
engine = MagicMock()
|
||||
engine.rules = engine_rules
|
||||
svc._app_state.monitor_engine = engine
|
||||
svc._app_state.repo = svc._repo
|
||||
|
||||
svc._repo.get_index_symbol_set.return_value = {"000001.SH"}
|
||||
svc._repo.get_etf_symbol_set.return_value = set()
|
||||
return svc
|
||||
|
||||
|
||||
def _run_fetch(svc, tf, watchlist: list[str], capset: CapabilitySet):
|
||||
"""在完整 patch 环境下执行 _fetch_watchlist_quotes。"""
|
||||
with ExitStack() as stack:
|
||||
stack.enter_context(patch(
|
||||
"app.services.preferences.get_realtime_watchlist_symbols",
|
||||
return_value=watchlist,
|
||||
))
|
||||
stack.enter_context(patch(
|
||||
"app.tickflow.client.get_paid_realtime_client", return_value=tf,
|
||||
))
|
||||
stack.enter_context(patch(
|
||||
"app.tickflow.policy.detect_capabilities", return_value=capset,
|
||||
))
|
||||
stack.enter_context(patch("app.tickflow.rate_limits.sleep_between_batches"))
|
||||
# patch 分批之后的下游处理
|
||||
stack.enter_context(patch.object(
|
||||
QuoteService, "_build_daily", return_value=pl.DataFrame(),
|
||||
))
|
||||
stack.enter_context(patch.object(
|
||||
QuoteService, "_build_quote_extra", return_value=pl.DataFrame(),
|
||||
))
|
||||
stack.enter_context(patch.object(
|
||||
QuoteService, "_build_index_quotes", return_value=pl.DataFrame(),
|
||||
))
|
||||
stack.enter_context(patch.object(QuoteService, "_broadcast_quote_updated"))
|
||||
stack.enter_context(patch.object(QuoteService, "_evaluate_monitors"))
|
||||
stack.enter_context(patch("app.services.quote_service._persist_last_fetch"))
|
||||
svc._fetch_watchlist_quotes()
|
||||
|
||||
|
||||
def test_watchlist_batch_respects_capability_limit():
|
||||
"""6 symbols / batch 5 → 分 2 批请求, 不整轮失败。"""
|
||||
engine_rules = {
|
||||
"r_idx": {"enabled": True, "asset_type": "index", "scope": "symbols",
|
||||
"symbols": ["000001.SH"]},
|
||||
}
|
||||
svc = _make_svc(engine_rules)
|
||||
|
||||
tf = MagicMock()
|
||||
tf.quotes.get.return_value = [
|
||||
{"symbol": "600000.SH", "last_price": 10.0, "prev_close": 9.9, "ext": {}},
|
||||
]
|
||||
capset = CapabilitySet({Cap.QUOTE_BY_SYMBOL: CapabilityLimits(batch=5, rpm=60)})
|
||||
|
||||
_run_fetch(svc, tf,
|
||||
["600000.SH", "600001.SH", "600002.SH", "600003.SH", "600004.SH"],
|
||||
capset)
|
||||
|
||||
# 5 股票 + 1 指数 = 6 symbols, batch 5 → 2 批
|
||||
assert tf.quotes.get.call_count == 2
|
||||
first_batch = tf.quotes.get.call_args_list[0][1]["symbols"]
|
||||
second_batch = tf.quotes.get.call_args_list[1][1]["symbols"]
|
||||
assert len(first_batch) == 5
|
||||
assert len(second_batch) == 1
|
||||
assert "000001.SH" in second_batch
|
||||
|
||||
|
||||
def test_watchlist_batch_partial_failure_keeps_other_batches():
|
||||
"""某一批拉取失败不影响其他批次 (已有股票实时刷新不丢失)。"""
|
||||
engine_rules = {
|
||||
"r_idx": {"enabled": True, "asset_type": "index", "scope": "symbols",
|
||||
"symbols": ["000001.SH"]},
|
||||
}
|
||||
svc = _make_svc(engine_rules)
|
||||
|
||||
tf = MagicMock()
|
||||
# 第一批 (股票) 成功, 第二批 (指数) 失败
|
||||
tf.quotes.get.side_effect = [
|
||||
[{"symbol": "600000.SH", "last_price": 10.0, "prev_close": 9.9, "ext": {}}],
|
||||
ConnectionError("timeout"),
|
||||
]
|
||||
capset = CapabilitySet({Cap.QUOTE_BY_SYMBOL: CapabilityLimits(batch=5, rpm=60)})
|
||||
|
||||
_run_fetch(svc, tf,
|
||||
["600000.SH", "600001.SH", "600002.SH", "600003.SH", "600004.SH"],
|
||||
capset)
|
||||
|
||||
# 两批都被尝试 (第二批失败不阻断)
|
||||
assert tf.quotes.get.call_count == 2
|
||||
|
||||
|
||||
def test_watchlist_no_index_rules_no_extra_symbols():
|
||||
"""无指数监控规则时, symbols 不追加指数标的。"""
|
||||
svc = _make_svc({}) # 无规则
|
||||
|
||||
tf = MagicMock()
|
||||
tf.quotes.get.return_value = [
|
||||
{"symbol": "600000.SH", "last_price": 10.0, "prev_close": 9.9, "ext": {}},
|
||||
]
|
||||
capset = CapabilitySet({Cap.QUOTE_BY_SYMBOL: CapabilityLimits(batch=5, rpm=60)})
|
||||
|
||||
_run_fetch(svc, tf, ["600000.SH", "600001.SH"], capset)
|
||||
|
||||
# 2 symbols / batch 5 → 1 批
|
||||
assert tf.quotes.get.call_count == 1
|
||||
assert tf.quotes.get.call_args_list[0][1]["symbols"] == ["600000.SH", "600001.SH"]
|
||||
@@ -1,19 +0,0 @@
|
||||
"""Free 档自选实时资产分流测试。"""
|
||||
from app.services.quote_service import QuoteService
|
||||
|
||||
|
||||
def test_split_records_by_asset():
|
||||
records = [
|
||||
{"symbol": "600000.SH"}, {"symbol": "510300.SH"}, {"symbol": "000001.SH"},
|
||||
]
|
||||
index, etf, stock = QuoteService._split_records_by_asset(
|
||||
records, {"000001.SH"}, {"510300.SH"},
|
||||
)
|
||||
assert [r["symbol"] for r in index] == ["000001.SH"]
|
||||
assert [r["symbol"] for r in etf] == ["510300.SH"]
|
||||
assert [r["symbol"] for r in stock] == ["600000.SH"]
|
||||
# etf 优先于 index (与 resolve_asset_type 判定顺序一致)
|
||||
index2, etf2, stock2 = QuoteService._split_records_by_asset(
|
||||
[{"symbol": "X"}], {"X"}, {"X"},
|
||||
)
|
||||
assert etf2 and not index2 and not stock2
|
||||
Generated
+4
-4
@@ -2493,15 +2493,15 @@ wheels = [
|
||||
|
||||
[[package]]
|
||||
name = "tickflow"
|
||||
version = "0.1.24"
|
||||
version = "0.1.25"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "httpx" },
|
||||
{ name = "typing-extensions" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/aa/56/d911f7d03363a06f69838878df4f92dd01235c899431197f94fb4c0e36ad/tickflow-0.1.24.tar.gz", hash = "sha256:13f6464a9dd1bdf98a312bc8c313fcc24ad0ad83c16d4f83b76bee83df6e11df", size = 36867, upload-time = "2026-06-20T02:39:35.891Z" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/d1/c9/facd0cd7568ea3c7dcfb0d52ded7d0b52ad871ca45fa97d7df0925b26091/tickflow-0.1.25.tar.gz", hash = "sha256:a86929bc99167014567c3d8b99af1c2f498922ad476bb4507edb57164c576d04", size = 37357, upload-time = "2026-08-29T04:43:27.255Z" }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/4c/9d/6c03706054f3bcca8a7113a60258e1a52a762077a555d52a5a84e4f6895f/tickflow-0.1.24-py3-none-any.whl", hash = "sha256:e898867b0e3e668618135c78e3a367542f81b7a289567335d298c707452e5f42", size = 43031, upload-time = "2026-06-20T02:39:34.307Z" },
|
||||
{ url = "https://files.pythonhosted.org/packages/ea/37/f80b8f6e435f1825ea8384b1605a01c323998f00b1c112c59e72a9d7dd9a/tickflow-0.1.25-py3-none-any.whl", hash = "sha256:69687235e85b44eae077262325ddef15e82d5adafda075945612f8cf0815a44c", size = 43510, upload-time = "2026-08-29T04:43:25.785Z" },
|
||||
]
|
||||
|
||||
[package.optional-dependencies]
|
||||
@@ -2513,7 +2513,7 @@ all = [
|
||||
|
||||
[[package]]
|
||||
name = "tickflow-stock-panel-backend"
|
||||
version = "0.2.1"
|
||||
version = "0.2.2"
|
||||
source = { editable = "." }
|
||||
dependencies = [
|
||||
{ name = "apscheduler" },
|
||||
|
||||
@@ -125,7 +125,23 @@ datasets:
|
||||
|
||||
建议实时接口额外提供 `amount`、`change_pct`、`change_amount`、`amplitude`、`turnover_rate`、`name`。缺失时部分字段会由 pipeline 回算,但精度取决于可用输入。
|
||||
|
||||
`change_pct` 和 `amplitude` 使用小数制,例如 `0.0366` 表示 `3.66%`(`turnover_rate` 同)。若接口直接返回百分数值 `3.66`,实时行情会按截面中位数自动归一为小数制,但仍建议接口直接提供小数制以避免小样本歧义。
|
||||
`change_pct`、`amplitude`、`turnover_rate` 统一使用小数制,例如 `0.0366` 表示 `3.66%`。百分制单位必须在 realtime 数据集上**显式声明**,不做数值猜测(数值无法区分两种单位:`0.05` 既可能是 0.05% 也可能是 5%):
|
||||
|
||||
```yaml
|
||||
datasets:
|
||||
realtime:
|
||||
url: https://api.example.com/snapshot
|
||||
pct_unit: percent # 接口返回 3.66 表示 3.66%;小数制源声明 decimal 或省略
|
||||
```
|
||||
|
||||
处理规则:
|
||||
|
||||
| 声明 | 行为 |
|
||||
| --- | --- |
|
||||
| `pct_unit: percent` | `change_pct` / `amplitude` / `turnover_rate` 无条件 `/100` |
|
||||
| `pct_unit: decimal` | 三列原样透传 |
|
||||
| 未声明 | `change_pct` 按截面中位数归一(A 股涨跌停 30% 上限使两种单位物理可分);`amplitude` / `turnover_rate` **置 `None`** 交由 pipeline 按价格与股本口径重算,并记录 WARNING |
|
||||
| 列已配置 `transforms` | 视为用户已接管该列单位,原样透传 |
|
||||
|
||||
## 请求约定
|
||||
|
||||
@@ -210,7 +226,7 @@ cp docs/examples/custom-data-source/mock_source.yaml data/data_sources/mock_sour
|
||||
5. 保存数据源选择:
|
||||
|
||||
- 日K: `mock_source`
|
||||
- 除权因子: `same_as_daily` 或 `mock_source`
|
||||
- 除权因子: `mock_source` (或保持默认 `tickflow`)
|
||||
- 实时行情: `mock_source`
|
||||
|
||||
6. 触发同步或开启实时行情。
|
||||
@@ -273,12 +289,16 @@ cp docs/examples/custom-data-source/mock_source.yaml data/data_sources/mock_sour
|
||||
amount = 成交额
|
||||
change_pct = 涨跌幅 (小数, 0.0366 = 3.66%)
|
||||
change_amount = 涨跌额
|
||||
amplitude = 振幅
|
||||
turnover_rate = 换手率 (小数, 0.05 = 5%; 若上游返回 5 表示 5%, 配置 transforms: turnover_rate: "value / 100")
|
||||
amplitude = 振幅 (小数, 0.024 = 2.4%)
|
||||
turnover_rate = 换手率 (小数, 0.05 = 5%)
|
||||
# 上游若返回百分数值 (3.66 表示 3.66%), 在 realtime 数据集声明 pct_unit: percent,
|
||||
# 不要依赖数值自动识别; 逐列转换也可用 transforms: turnover_rate: "value / 100"
|
||||
|
||||
分钟K (minute):
|
||||
symbol = 股票代码
|
||||
datetime = 时间戳 (YYYY-MM-DD HH:MM:SS)
|
||||
# datetime 必须是北京时间墙钟 (如 2026-08-28 09:35:00), 不要返回 UTC;
|
||||
# 入口守卫会自动纠偏 UTC 特征帧, 但契约仍要求源头写对
|
||||
datetime = 北京时间墙钟 (YYYY-MM-DD HH:MM:SS)
|
||||
open / high / low / close = OHLC
|
||||
volume = 成交量
|
||||
amount = 成交额
|
||||
|
||||
+22
-4
@@ -16,7 +16,7 @@
|
||||
| 量价 / 涨停 | 量价齐升 · 高换手强势 · 连板股 · 断板反包 · 涨停动量 |
|
||||
| 反转 / 波动 | 超跌反弹 · 超卖反转 · 新低反转 · 低波动龙头 · 回踩 MA20 |
|
||||
|
||||
全 A 股一次扫表,Polars 毫秒级返回。选股页点策略卡片即可扫描,结果支持导出。
|
||||
全 A 股一次扫表,Polars 毫秒级返回。选股页点策略卡片即可扫描,结果支持导出。**分钟策略**与日线策略共用同一个标的池,按策略声明的周期自动路由执行(分钟策略依赖数据页开启分钟数据落盘)。
|
||||
|
||||
**ETF 支持**:选股页顶部可切换 `股票 / ETF`。ETF 复用已算好的 `kline_etf_enriched` 技术指标,仅开放**技术类内置策略**(趋势/量价/反转/波动);依赖涨停信号的策略(连板股、断板反包)为股票专有,ETF 模式下不显示。需先在数据页开启 ETF 拉取(`pipeline_pull_etf`)并跑一次盘后管道。
|
||||
|
||||
@@ -26,7 +26,7 @@
|
||||
|
||||
## 📊 指标流水线(Indicators)
|
||||
|
||||
原生 Polars 向量化,全 A 股一次扫表落盘 enriched Parquet:
|
||||
原生 Polars 向量化,全 A 股一次扫表落盘 enriched Parquet(基础行情列),指标在读取时现算并带进程内快照缓存:
|
||||
|
||||
- **均线 / 趋势**:MA(5-60) · EMA · MACD · 动量 · 布林带
|
||||
- **震荡 / 波动**:RSI · KDJ · ATR · 年化波动率 · 振幅
|
||||
@@ -54,6 +54,10 @@
|
||||
|
||||
输出净值曲线 · 夏普 · 最大回撤 · 胜率 · 交易明细。SSE 流式进度支持切页重连,不会丢失回测任务。
|
||||
|
||||
**分钟策略回测**:分钟级策略逐交易日回放信号、以分钟收盘价入场,输出分钟级成交明细,与日线回测同一套 T+1 / 费用约束。
|
||||
|
||||
**结果沉淀**:回测结果可**导出 CSV**(概要 / 净值曲线 / 交易明细 / 分标的统计四段),并可**保存候选**(完整参数与指标快照);候选在回测页「候选方案」中查看,支持**一键载入复测**,回填全部参数继续研究。
|
||||
|
||||
**因子与策略挖掘**:复用已有日频因子和 matrix-native 策略，通过 T-1 市场环境、相关去重和嵌套样本外验证生成研究候选。任务在 spawn worker 中运行，支持持久 run ID、取消、刷新重连和显式发布；自动周度任务默认关闭且永不自动发布。完整口径见 [因子与策略挖掘](./mining.md)。
|
||||
|
||||
**市场阶段与主线**:市场环境页在原 5 档状态之外新增情绪周期阶段(冰点/启动/主升/高潮/退潮/修复,由连板梯队的高度、宽度、晋级率、梯队完整度判定,平均段长约 10 天)与主线识别(概念/行业维度的涨停梯队聚合排名,可配置宽基标签过滤)。完整口径见 [市场阶段与主线识别](./market-phase.md)。
|
||||
@@ -88,6 +92,20 @@
|
||||
|
||||
---
|
||||
|
||||
## 🚨 异动监控(Abnormal Moves)
|
||||
|
||||
独立的异动页,按交易时间线组织为三个 tab,覆盖盘前到多日:
|
||||
|
||||
| Tab | 口径 |
|
||||
| :--- | :--- |
|
||||
| 竞价异动(盘前) | fuyao 同花顺盘前风向标:当日竞价强势榜(5-6 只/日,含竞价涨幅与概念标签),附带**当日 / 次日真实收益对照**与**追高风险标记**(高开 ≥5% 追高当日均值显著为负,来自 60 日回测) |
|
||||
| 盘中异动 | 涨停 / 炸板 / 翘板 / 跌停 / 新高 / 新低 / 放量等当日信号聚合,按信号优先级 + 涨跌幅排序,复用 enriched 信号列,零新增采集 |
|
||||
| 偏移异动(多日累计) | 交易所异动偏离值口径(按板块分档:主板 3 日 ±20% · 创业板/科创板 ±30% · 北交所 ±40%;10 日 +100%/−50%,30 日 +200%/−70%),实时计算接近度(≥100% 已触发 · ≥70% 边缘 · ≥50% 观察) |
|
||||
|
||||
竞价数据来自 fuyao 数据源(历史按日缓存,当日不缓存);未配置 fuyao key 时竞价 tab 显示引导提示。
|
||||
|
||||
---
|
||||
|
||||
## 📈 个股分析(Beta)
|
||||
|
||||
以「行情 + 关键价位」为主体的单标的决策页:
|
||||
@@ -102,7 +120,7 @@
|
||||
|
||||
- **连板梯队**:实时统计各连板层级(首板 / 2 连板 / 3 连板...)的标的与封单,捕捉市场情绪与题材热度
|
||||
- **概念涨幅轮动**:基于 ths 概念 / 行业,统计概念板块涨幅与 RPS 轮动,AI 分析资金主线
|
||||
- **盘后 AI 复盘**:盘后自动生成市场复盘,可推送至飞书群
|
||||
- **盘后 AI 复盘**:盘后自动生成市场复盘,注入**龙虎榜资金动向**(席位净买/机构与游资 Top5)与**盘前风向标**对照,可定时执行、推送飞书群、下载 Markdown;复盘页内亦有龙虎榜卡片(可排序席位明细)
|
||||
|
||||
---
|
||||
|
||||
@@ -110,7 +128,7 @@
|
||||
|
||||
### 数据源插件化
|
||||
|
||||
内置 [TickFlow](https://tickflow.org) 官方 SDK(日 K / 分钟 K / 指数 / 财务 / 实时行情),同时支持接入第三方数据源:YAML 声明自有 HTTP 接口,或开发插件源(参考实现 stock-sdk,Docker 镜像已内置)。五个数据集(daily / adj_factor / realtime / minute / financial)按源声明自由组合,在「设置 → 数据源」切换。
|
||||
内置 [TickFlow](https://tickflow.org) 官方 SDK(日 K / 分钟 K / 指数 / 财务 / 实时行情),同时支持接入第三方数据源:YAML 声明自有 HTTP 接口,或开发插件源(参考实现 stock-sdk,Docker 镜像已内置)。多个数据集(日K / 除权因子 / 实时行情 / 分钟K / 五档盘口 / 财务,注册表持续扩展)按源声明自由组合,在「设置 → 数据源」切换。
|
||||
|
||||
详见 [custom-data-source.md](./custom-data-source.md) 与 [plugin-development.md](./plugin-development.md)。
|
||||
|
||||
|
||||
+156
-34
@@ -1,9 +1,13 @@
|
||||
# 数据源插件开发指南
|
||||
|
||||
数据源插件是可选的行情数据来源(stock-sdk、akshare 等),作为独立模块放在
|
||||
`backend/app/plugins/` 下。用户**手动安装依赖**后才可用(开发模式);不安装完全不影响主功能。
|
||||
数据源插件是可选的行情数据来源(fuyao、stock-sdk、akshare 等),作为独立模块放在
|
||||
`backend/app/plugins/` 下。services 层(kline_sync / quote_service / financial_sync)
|
||||
全部通过统一路由点分流:插件声明了某数据集就走插件,未声明自动回退 TickFlow。
|
||||
因此**一个合格的插件只需要正确实现契约,不需要改动任何 service / API 代码**;
|
||||
反过来,插件也必须遵守内部数据契约(单位、代码格式、复权口径),框架不会替你转换。
|
||||
|
||||
> ⚠️ **Docker 默认不打包 stock-sdk**(合规考虑:它抓取第三方财经网站接口,存在版权与反爬风险)。如需在 Docker 中启用,构建时传 `--build-arg INCLUDE_STOCKSDK=1`,使用风险自负。下方"手动安装依赖"适用于开发模式及自定义 Docker 构建。
|
||||
> 无代码接入(纯 HTTP YAML 配置)请看 [custom-data-source.md](./custom-data-source.md),
|
||||
> 两种方式遵循同一套内部数据契约。
|
||||
|
||||
## 快速上手
|
||||
|
||||
@@ -13,7 +17,7 @@
|
||||
backend/app/plugins/<your_plugin>/
|
||||
├── plugin.yaml # 清单(必需)
|
||||
├── provider.py # Provider 实现(必需)
|
||||
├── ... # 桥接/依赖文件(按需)
|
||||
├── ... # client/桥接/依赖文件(按需)
|
||||
```
|
||||
|
||||
### plugin.yaml 字段
|
||||
@@ -21,16 +25,20 @@ backend/app/plugins/<your_plugin>/
|
||||
```yaml
|
||||
name: my_source # 唯一标识, 只允许 [a-z0-9_], 也是 provider name
|
||||
display_name: "我的数据源" # 设置页显示名
|
||||
runtime: python # 运行时类型: node | python | none
|
||||
runtime: none # 运行时类型: node | python | none
|
||||
entry: app.plugins.my_source.provider:MyProvider # provider 类的导入路径
|
||||
check: app.plugins.my_source.bridge:availability # 可用性检测函数(可选)
|
||||
datasets: [daily, adj_factor, minute, realtime] # 支持的数据集
|
||||
datasets: [realtime] # 支持的数据集: daily/adj_factor/minute/realtime/financial
|
||||
api_key_env: MY_SOURCE_API_KEY # (可选)声明后设置页提供 Key 输入框
|
||||
hidden: false # (可选)true = 已加载但对设置页隐藏,不注册不展示
|
||||
description: "数据源描述"
|
||||
install_hint: "pip install xxx" # 未装依赖时显示的安装提示
|
||||
homepage: "https://example.com" # (可选)官网/申请地址, 显示在设置页 Key 配置说明中
|
||||
```
|
||||
|
||||
只声明真实提供的数据集;未声明的数据集 `provider_has_dataset` 返回 False,自动回退
|
||||
TickFlow。不要声明做不了的数据集(粒度含义见下文"能力声明的粒度")。
|
||||
|
||||
#### api_key_env(界面配置 API Key)
|
||||
|
||||
声明 `api_key_env` 的插件可以在设置页的数据源卡片中直接填写 Key, 对齐
|
||||
@@ -49,36 +57,74 @@ TickFlow 的「先探后存」语义:
|
||||
| runtime | 含义 | 典型场景 |
|
||||
|---|---|---|
|
||||
| `python` | 纯 Python 依赖, `pip install` | akshare、tushare |
|
||||
| `node` | 需要 Node.js 运行时, `npm install` | stock-sdk(Docker 默认不打包,见 [deployment.md](./deployment.md)) |
|
||||
|
||||
> stock-sdk 在 Docker 中默认不打包(合规考虑);如需启用,构建时传 `--build-arg INCLUDE_STOCKSDK=1`,开发模式下需手动 `npm install`。
|
||||
| `node` | 需要 Node.js 运行时, `npm install` | stock-sdk |
|
||||
| `none` | 无额外依赖 | 纯 HTTP API 源 |
|
||||
|
||||
> ⚠️ stock-sdk 在 Docker 中默认不打包(合规考虑:它抓取第三方财经网站接口,存在版权与
|
||||
> 反爬风险)。如需启用,构建时传 `--build-arg INCLUDE_STOCKSDK=1`,使用风险自负。
|
||||
> 详见 [deployment.md](./deployment.md)。
|
||||
|
||||
`runtime` 字段当前仅用于 UI 展示, 实际依赖检测由 `check` 函数负责。
|
||||
|
||||
### check 函数
|
||||
|
||||
插件自己负责检测依赖是否已安装。后端启动时会调用此函数:
|
||||
插件自己负责检测依赖/Key 是否就绪。后端启动时会调用此函数:
|
||||
|
||||
```python
|
||||
# app/plugins/my_source/bridge.py
|
||||
# app/plugins/my_source/provider.py (或 bridge.py)
|
||||
def availability() -> tuple[bool, str]:
|
||||
"""返回 (是否可用, 原因)。不抛异常。"""
|
||||
try:
|
||||
import akshare # noqa: F401
|
||||
if not get_api_key():
|
||||
return False, "未配置 MY_SOURCE_API_KEY(可在设置页数据源卡片中直接填写)"
|
||||
return True, "ok"
|
||||
except ImportError:
|
||||
return False, "未安装 akshare, 运行: pip install akshare"
|
||||
```
|
||||
|
||||
- **可用** → 插件注册进路由表, 设置页可切换
|
||||
- **不可用** → 设置页显示插件卡片但灰显, 展示 `install_hint`
|
||||
- **不可用** → 设置页显示插件卡片但灰显, 展示原因/`install_hint`
|
||||
|
||||
## 内部数据契约(所有数据集必须遵守)
|
||||
|
||||
以下口径是全项目红线(详见 CONTRIBUTING §3)。金融数据错误往往不抛异常,而是生成
|
||||
**看似合理的错误结果**——单位、代码格式、复权口径错了,页面照样能渲染,只是数字全错。
|
||||
插件必须在 provider 内完成适配。
|
||||
|
||||
### 代码格式
|
||||
|
||||
- symbol 统一带交易所后缀: `600519.SH` / `000001.SZ` / `300750.SZ`; ETF、指数同格式。
|
||||
- 接口返回裸代码(如 `600519`)或异构格式时,在 client 层实测一页并归一,不要直接透传。
|
||||
|
||||
### 单位制
|
||||
|
||||
| 字段 | 契约 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| `change_pct` | **小数制**, `0.0366` = 3.66% | 接口给百分数(3.66)时必须在 provider 内显式 /100 |
|
||||
| `turnover_rate`(realtime 入口) | **小数制**, `0.05` = 5% | 下游 enriched 管道统一转百分数值存储 |
|
||||
| `volume` | 股 | |
|
||||
| `amount` / `turnover` | 元 | |
|
||||
| 日K OHLC | **不复权原始价** | 复权由 adj_factor + enriched 管道处理, provider 不得自行复权 |
|
||||
|
||||
### 缺字段与空数据
|
||||
|
||||
- 接口不提供的字段返回 `None`,禁止"数值小于 1 就乘 100"之类启发式补全——那会掩盖
|
||||
真实的数据错误。
|
||||
- 可推导字段按固定口径推导: `change_pct = change_amount / prev_close`(小数制,不乘 100)。
|
||||
- 接口结构整体变化(如所有行都识别不出 symbol)要打明确告警日志,不要静默返回空数据。
|
||||
|
||||
## 能力声明的粒度(重要)
|
||||
|
||||
`datasets` 声明是**数据集级**的,不是资产类型级的:声明了 `realtime`,整个全市场实时
|
||||
轮询周期(含指数与 ETF 部分)就全部路由给插件。若你的快照只覆盖 A 股股票:
|
||||
|
||||
- 指数行情自动降级为日线推导值(非实时),不报错;
|
||||
- ETF 实时计数为 0。
|
||||
|
||||
这是当前框架的设计行为。要么在数据里尽量覆盖指数/ETF,要么接受降级并在
|
||||
`description` 里向用户说明覆盖范围。
|
||||
|
||||
## Provider 接口契约
|
||||
|
||||
Provider 是一个普通 Python 类(无需继承基类), 实现以下方法签名。方法签名对齐
|
||||
`GenericHTTPProvider`, 这样 services 层(kline_sync / quote_service 等)的路由逻辑
|
||||
零改动即可路由到插件。
|
||||
Provider 是普通 Python 类(无需继承基类),方法签名对齐 `GenericHTTPProvider`,
|
||||
services 层零改动即可路由。只实现已声明数据集对应的方法,其余可缺省。
|
||||
|
||||
```python
|
||||
class MyProvider:
|
||||
@@ -91,22 +137,72 @@ class MyProvider:
|
||||
def close(self) -> None:
|
||||
"""清理资源(load_all 重建注册表时会调)。"""
|
||||
|
||||
def get_daily(self, symbols, start_time, end_time, asset_type="stock", on_chunk_done=None) -> pl.DataFrame:
|
||||
"""日K: 返回 schema [symbol, date, open, high, low, close, volume, amount]"""
|
||||
def get_daily(self, symbols, start_time, end_time, asset_type="stock",
|
||||
on_chunk_done=None) -> pl.DataFrame:
|
||||
"""日K: [symbol, date, open, high, low, close, volume, amount]; 不复权"""
|
||||
|
||||
def get_adj_factors(self, symbols, start_time, end_time, asset_type="stock", on_chunk_done=None) -> pl.DataFrame:
|
||||
"""除权因子: 返回 schema [symbol, trade_date, ex_factor]"""
|
||||
def get_adj_factors(self, symbols, start_time, end_time, asset_type="stock",
|
||||
on_chunk_done=None) -> pl.DataFrame:
|
||||
"""除权因子: [symbol, trade_date, ex_factor]"""
|
||||
|
||||
def get_minute(self, symbols, start_time, end_time, asset_type="stock", on_chunk_done=None, freq="1m") -> pl.DataFrame:
|
||||
"""分钟K: 返回 schema [symbol, datetime, open, high, low, close, volume, amount]"""
|
||||
def get_minute(self, symbols, start_time, end_time, asset_type="stock",
|
||||
on_chunk_done=None, freq="1m") -> pl.DataFrame:
|
||||
"""分钟K: [symbol, datetime(北京墙钟), open, high, low, close, volume, amount]"""
|
||||
|
||||
def get_realtime(self) -> list[dict]:
|
||||
"""全市场实时快照: 返回 list[dict], 每行含 symbol/last_price/prev_close/open/high/low/volume"""
|
||||
"""全市场实时快照 → list[dict]。失败软返回 [], 不抛异常(不阻断轮询线程)。"""
|
||||
|
||||
def get_financials(self, table, symbols, latest_only=False) -> pl.DataFrame:
|
||||
"""财务数据(声明 financial 数据集时实现, table 见 financial_sync 调用)。"""
|
||||
|
||||
def get_instruments(self, asset_type="stock") -> list[dict]:
|
||||
"""标的维表(可选): 返回 tickflow Instrument 形状的行, 供 instrument_sync 复用 flatten"""
|
||||
"""(可选)标的维表: 返回 tickflow Instrument 形状的行, 供 instrument_sync 复用 flatten"""
|
||||
|
||||
def test_dataset(self, dataset: str, symbols=None) -> dict:
|
||||
"""(强烈建议)设置页"试拉"按钮。
|
||||
返回 {provider, dataset, rows, columns, preview, error?}; 未支持的数据集
|
||||
返回 error 字段说明会回退 TickFlow。"""
|
||||
```
|
||||
|
||||
### get_minute 的 datetime 时区契约
|
||||
|
||||
`datetime` 必须是**北京时间墙钟**（naive，如 `2026-08-28 09:35:00`），与日K的
|
||||
`date` 语义对齐；不要返回 UTC 或带时区的时间。前端分时图按交易时段时轴
|
||||
（09:30–11:30 / 13:00–15:00）映射每根K线，UTC 口径的帧会导致全部点位落在时轴外、
|
||||
分时图空白。
|
||||
|
||||
入口守卫（`kline_sync._enforce_minute_beijing_wallclock`）对所有分钟源强制归一：
|
||||
带时区 → 自动换算成北京墙钟；naive 但整体呈 UTC 特征（如 01:30）→ 自动 +8 纠偏并
|
||||
记日志；完全无法识别的口径 → 拒收并回退 TickFlow。契约仍要求源头写对，守卫只是兜底。
|
||||
|
||||
可选类属性 `minute_history_days = 5` 声明 1 分钟历史深度（交易日）；未声明视为
|
||||
深历史（TickFlow 基准）。浅源（如 stock-sdk 免费分时仅保留最近 5 个交易日）声明后，
|
||||
个股分时档位自动收窄为可行选项并默认 5 日，深源默认 20 日。
|
||||
|
||||
### 异常语义
|
||||
|
||||
| 方法 | 失败行为 |
|
||||
| --- | --- |
|
||||
| `get_realtime` | **软失败**: 返回 `[]` + warning 日志, 保证轮询线程不中断 |
|
||||
| `get_minute` | 抛异常时调用方自动回退 TickFlow 重试 |
|
||||
| `get_daily` / `get_adj_factors` / `get_financials` | 异常由上层同步流程捕获记录; 无数据返回空 DataFrame |
|
||||
|
||||
### get_realtime 行字段
|
||||
|
||||
| 字段 | 必需 | 契约 |
|
||||
| --- | --- | --- |
|
||||
| `symbol` | ✅ | 标准代码带后缀 |
|
||||
| `last_price` | ✅ | 最新价 |
|
||||
| `prev_close` | ✅ | 昨收, 涨跌幅推导基准 |
|
||||
| `open` / `high` / `low` | ✅ | 当日 OHLC |
|
||||
| `volume` | ✅ | 股 |
|
||||
| `amount` | 建议 | 成交额(元) |
|
||||
| `change_pct` | 建议 | **小数制**; 缺失时下游按 change_amount/prev_close 推导 |
|
||||
| `change_amount` | 建议 | 涨跌额(元) |
|
||||
| `timestamp` | 建议 | 毫秒; 优先用服务端时间(行情归属), 缺失退本地时间 |
|
||||
| `name` | 可选 | 快照无名称时置 None, 下游用标的维表关联 |
|
||||
| `amplitude` / `turnover_rate` / `session` | 可选 | 缺失置 None, 不启发式伪造; turnover_rate 入口为小数制 |
|
||||
|
||||
### config.datasets 的作用
|
||||
|
||||
`provider_has_dataset(name, dataset)` 通过 `dataset in provider.config.datasets` 判断。
|
||||
@@ -118,14 +214,40 @@ class MyConfig:
|
||||
datasets = {"daily": ..., "realtime": ...} # key 是数据集名, value 任意
|
||||
```
|
||||
|
||||
## 限频与性能
|
||||
|
||||
- realtime 默认 6s 轮询一轮。优先确认服务端单次 limit 上限: fuyao 实测单页
|
||||
limit=6000 可一次拉完全市场(~5600 只), 1 请求/轮; 若服务端强制小页, 必须做
|
||||
页间隔/自限速(参考 fuyao 的 0.15s 页间隔兜底), 并建议用户把轮询间隔调大(15-30s)。
|
||||
- 分页必须有页数上限(防 count 异常导致死循环)和空页终止条件。
|
||||
- 拉取由 fetch 锁串行化, 慢不会并发重叠; 实际刷新周期 = 轮询间隔 + 拉取耗时,
|
||||
串行分页的全量快照本身就需要数秒, 不要按"6s 内必须完成"设计。
|
||||
|
||||
## 测试要求
|
||||
|
||||
插件 PR 必须带契约测试(CONTRIBUTING §9), **不依赖真实网络与 API Key**——用假
|
||||
Client/桥接注入。以 `backend/tests/test_fuyao_provider.py` 为范本, 至少覆盖:
|
||||
|
||||
1. 字段映射与单位转换: 百分数→小数制、volume 股→手、*ms 零点戳时区换算、缺失字段按口径推导、缺失字段置 None 不伪造
|
||||
2. 接口响应结构变体: 实测结构 vs 官方文档示例双兼容(供应商文档与实际不一致是常态)
|
||||
3. 分页: 多页合并、空页终止、页数上限
|
||||
4. 软失败: 接口报错返回 []; 整页 schema 变化有告警而非静默空数据
|
||||
5. 能力声明: 未声明数据集 `provider_has_dataset` 为 False
|
||||
6. Key 语义: 先探后存(无效不落盘)、secrets.json > .env 优先级、availability 两态
|
||||
7. loader 集成: 清单解析后正确注册(或 hidden 时正确跳过)
|
||||
|
||||
```bash
|
||||
cd backend && uv run --extra dev python -m pytest tests/test_<your_plugin>_provider.py -q
|
||||
uv run --extra dev python -m ruff check app/plugins/<your_plugin>/ tests/test_<your_plugin>_provider.py
|
||||
```
|
||||
|
||||
## 现有插件参考
|
||||
|
||||
- **`backend/app/plugins/fuyao/`** — 同花顺官方 REST 数据源(runtime: none, 纯 HTTP 零依赖)
|
||||
- 当前提供 `realtime`(A 股全市场快照, 分页拉取); Key 在设置页卡片直接配置(先探后存), 或 `.env` 配 `FUYAO_API_KEY`
|
||||
- `client.py` — httpx 客户端(X-api-key 认证 + 统一信封解包 + 分页)
|
||||
- `provider.py` — Provider 实现(字段映射、百分数→小数制单位转换、软失败、Key 探测)
|
||||
- 单位口径注意: 扶摇 `price_change_ratio_pct` 为百分数数值(1.74 = +1.74%),
|
||||
内部 `change_pct` 契约为小数制, provider 内显式 / 100(见 CONTRIBUTING §3.1)
|
||||
- 提供 `realtime`(A 股全市场快照, 分页拉取)、`daily`(原始价日K三档: 近端窗口走 daily-k-10d dump, 深窗口走 daily-k 10 年全量 dump(172MB 一次下载、缓存复用、10d 补尾), 兜底单标的接口按 10 年自动分片)、`adj_factor`(事件 dump + 前收盘价从本地日K dump 一次取齐、缺价标的回退单标的接口, 按交易所公式推导单事件比值, 涨跌停自检; 全市场配价从逐标的 ~13 分钟降为秒级); Key 在设置页卡片直接配置(先探后存), 或 `.env` 配 `FUYAO_API_KEY`
|
||||
- `client.py` — httpx 客户端(X-api-key 认证 + 统一信封解包 + 分页 + 页间隔限频 + 单标的日K + dump 预签名下载, S3 下载不带 Key 头)
|
||||
- `provider.py` — Provider 实现(实测/文档双字段名映射、百分数→小数制、volume 股→手、上海零点戳 +8h 时区、dump 按 release 版本缓存、软失败、Key 探测)
|
||||
- `tests/test_fuyao_provider.py` — 73 个契约测试, 是新插件的测试范本
|
||||
- **`backend/app/plugins/stocksdk/`** — Node 型插件, 通过 subprocess 桥接调用 stock-sdk
|
||||
- `bridge.py` — Python↔Node 桥接 + availability 检测
|
||||
- `bridge.mjs` — Node 端(并发池、重试、SDK 解析)
|
||||
@@ -133,9 +255,9 @@ class MyConfig:
|
||||
|
||||
## 路由机制(无需关心, 仅参考)
|
||||
|
||||
后端启动时, `loader.py` 的 `_load_builtin_plugins()` 扫描 `plugins/` 目录:
|
||||
后端启动时, `loader.py` 扫描 `plugins/` 目录:
|
||||
1. 读每个子目录的 `plugin.yaml`
|
||||
2. 调 `check` 函数检测可用性
|
||||
2. `hidden: true` → 跳过(不注册不展示); 否则调 `check` 函数检测可用性
|
||||
3. 可用 → 动态 import `entry` 指向的 Provider 类 → 注册进 `_PROVIDERS`
|
||||
4. 不可用 → 记录状态, 设置页显示但不可切换
|
||||
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "tickflow-stock-panel-frontend",
|
||||
"private": true,
|
||||
"version": "0.2.1",
|
||||
"version": "0.2.2",
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"dev": "vite",
|
||||
|
||||
@@ -66,6 +66,8 @@ export function CandlestickChart({ data, height = 480 }: Props) {
|
||||
width: el.clientWidth,
|
||||
height,
|
||||
layout: {
|
||||
// 关闭 TV 角标 (licence 归属改由 README 技术栈外链承担)
|
||||
attributionLogo: false,
|
||||
background: { color: THEME.background },
|
||||
textColor: ctRef.current.text,
|
||||
fontFamily: 'JetBrains Mono, monospace',
|
||||
|
||||
@@ -1,12 +1,22 @@
|
||||
import { useEffect, useMemo, useRef, useState } from 'react'
|
||||
import { useQuery } from '@tanstack/react-query'
|
||||
import { useVirtualizer } from '@tanstack/react-virtual'
|
||||
import { Building2, ChevronRight, RefreshCw, Search, Tags, Users, X } from 'lucide-react'
|
||||
import { Link } from 'react-router-dom'
|
||||
import {
|
||||
createChart,
|
||||
LineStyle,
|
||||
type IChartApi,
|
||||
type ISeriesApi,
|
||||
type LineData,
|
||||
type Time,
|
||||
} from 'lightweight-charts'
|
||||
import { Activity, Building2, ChevronRight, Database, RefreshCw, Search, Tags, Users, X } from 'lucide-react'
|
||||
import { Modal } from '@/components/Modal'
|
||||
import { boardTag } from '@/components/stock-table/primitives'
|
||||
import { api, type MarketSnapshotRow } from '@/lib/api'
|
||||
import { api, type DimensionIntradayPoint, type MarketSnapshotRow } from '@/lib/api'
|
||||
import { QK } from '@/lib/queryKeys'
|
||||
import { fmtBigNum, fmtPct, fmtPrice, priceColorClass } from '@/lib/format'
|
||||
import { useChartTheme } from '@/lib/theme'
|
||||
|
||||
export type DimensionKind = 'concept' | 'industry'
|
||||
|
||||
@@ -218,6 +228,16 @@ function DimensionMembersDialogContent({ target, onClose, onStockClick }: Omit<P
|
||||
<Summary label="平均涨跌" value={fmtPct(stats.average)} className={priceColorClass(stats.average)} />
|
||||
</div>
|
||||
|
||||
{source && (
|
||||
<DimensionIntradaySection
|
||||
configId={source.configId}
|
||||
field={source.field}
|
||||
value={target.value}
|
||||
date={target.date}
|
||||
kind={target.kind}
|
||||
/>
|
||||
)}
|
||||
|
||||
<div className="flex shrink-0 items-center gap-2 border-b border-border px-4 py-2.5">
|
||||
<div className="relative min-w-0 flex-1">
|
||||
<Search className="pointer-events-none absolute left-2.5 top-1/2 h-3.5 w-3.5 -translate-y-1/2 text-muted" />
|
||||
@@ -300,3 +320,219 @@ function Summary({ label, value, className }: { label: string; value: string | n
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// 板块分时 (等权): 点击触发 + 60s 轮询续期, 不预计算
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const INTRADAY_SECTOR_COLOR: Record<DimensionKind, string> = {
|
||||
concept: '#F97316',
|
||||
industry: '#0EA5E9',
|
||||
}
|
||||
const INTRADAY_MARKET_COLOR = '#94A3B8'
|
||||
// 横轴伪时间戳基点 (2020-01-01 UTC), 每点 +60s 保持均匀间距
|
||||
const INTRADAY_BASE_TS = 1577836800
|
||||
|
||||
function lastNonNull(points: DimensionIntradayPoint[], key: 'sector' | 'market'): number | null {
|
||||
for (let i = points.length - 1; i >= 0; i--) {
|
||||
const value = points[i]?.[key]
|
||||
if (value != null) return value
|
||||
}
|
||||
return null
|
||||
}
|
||||
|
||||
function DimensionIntradaySection({ configId, field, value, date, kind }: {
|
||||
configId: string
|
||||
field: string
|
||||
value: string
|
||||
date?: string
|
||||
kind: DimensionKind
|
||||
}) {
|
||||
const query = useQuery({
|
||||
queryKey: QK.dimensionIntraday(configId, field, value, date),
|
||||
queryFn: () => api.dimensionIntraday(configId, { field, value, date }),
|
||||
staleTime: 15_000,
|
||||
refetchInterval: 60_000,
|
||||
})
|
||||
const data = query.data
|
||||
|
||||
return (
|
||||
<section className="shrink-0 border-b border-border bg-surface/30">
|
||||
<div className="flex items-center gap-2 px-4 pt-2">
|
||||
<Activity className="h-3 w-3 text-muted" />
|
||||
<span className="text-[10px] font-medium text-muted">分时走势 · 等权</span>
|
||||
{data?.member_count != null && data.members_with_minute != null && (
|
||||
<span className="rounded bg-elevated px-1 py-px font-mono text-[9px] text-muted" title="有当日分钟数据的成分股数">
|
||||
{data.members_with_minute}/{data.member_count}只
|
||||
</span>
|
||||
)}
|
||||
{data?.basis && data.basis !== 'prev_close' && (
|
||||
<span
|
||||
className="rounded bg-amber-500/10 px-1 py-px text-[9px] text-amber-600 dark:text-amber-400"
|
||||
title="前一交易日收盘缺失, 部分标的以当日首根分钟价为基准, 曲线起点约为 0"
|
||||
>
|
||||
基准:当日首价
|
||||
</span>
|
||||
)}
|
||||
{data?.status === 'ok' && (
|
||||
<div className="ml-auto flex items-center gap-2.5 font-mono text-[10px]">
|
||||
<span className="inline-flex items-center gap-1">
|
||||
<span className="h-[3px] w-3 rounded-full" style={{ background: INTRADAY_SECTOR_COLOR[kind] }} />
|
||||
<span className="text-muted">板块</span>
|
||||
<span className={priceColorClass(lastNonNull(data.points, 'sector'))}>
|
||||
{fmtPct(lastNonNull(data.points, 'sector'))}
|
||||
</span>
|
||||
</span>
|
||||
<span className="inline-flex items-center gap-1">
|
||||
<span className="h-[3px] w-3 rounded-full" style={{ background: INTRADAY_MARKET_COLOR }} />
|
||||
<span className="text-muted">全市场</span>
|
||||
<span className={priceColorClass(lastNonNull(data.points, 'market'))}>
|
||||
{fmtPct(lastNonNull(data.points, 'market'))}
|
||||
</span>
|
||||
</span>
|
||||
{data.date && <span className="text-muted">{data.date}</span>}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{query.isLoading ? (
|
||||
<div className="mx-4 mb-2 mt-1.5 h-[132px] animate-pulse rounded-md bg-elevated/50" />
|
||||
) : query.isError ? (
|
||||
<div className="mx-4 mb-2 mt-1 grid h-[72px] place-items-center rounded-md border border-dashed border-border px-4 text-center text-[11px] text-muted">
|
||||
分时加载失败:{String((query.error as Error).message)}
|
||||
</div>
|
||||
) : data?.status === 'no_data' ? (
|
||||
<div className="mx-4 mb-2 mt-1 flex h-[96px] flex-col items-center justify-center gap-1 rounded-md border border-dashed border-border">
|
||||
<Database className="h-4 w-4 text-muted" />
|
||||
<p className="text-[11px] text-muted">分钟数据未落盘, 暂无分时走势</p>
|
||||
<p className="text-[10px] text-muted/70">需 TickFlow Pro+ 盘后分钟同步 / Expert 盘中增量, 或自定义分钟源</p>
|
||||
<Link to="/data" className="text-[10px] text-accent hover:text-accent/80">前往数据页 →</Link>
|
||||
</div>
|
||||
) : !data || data.status === 'empty' || data.points.length < 2 ? (
|
||||
<div className="grid h-[44px] place-items-center text-[11px] text-muted">
|
||||
{data?.reason === 'no_member_bars' ? '成分股当日无分钟数据 (ETF 等标的无分钟落盘)' : '暂无成分股分时数据'}
|
||||
</div>
|
||||
) : (
|
||||
<IntradayChart points={data.points} kind={kind} />
|
||||
)}
|
||||
</section>
|
||||
)
|
||||
}
|
||||
|
||||
function IntradayChart({ points, kind }: { points: DimensionIntradayPoint[]; kind: DimensionKind }) {
|
||||
const containerRef = useRef<HTMLDivElement>(null)
|
||||
const chartRef = useRef<IChartApi | null>(null)
|
||||
const sectorRef = useRef<ISeriesApi<'Line'> | null>(null)
|
||||
const marketRef = useRef<ISeriesApi<'Line'> | null>(null)
|
||||
const ct = useChartTheme()
|
||||
const ctRef = useRef(ct)
|
||||
ctRef.current = ct
|
||||
// v4 不支持字符串时间: 用均匀伪时间戳作横轴, 标签经 formatter 映射回 HH:MM
|
||||
const labelsRef = useRef<string[]>([])
|
||||
const labelAt = (time: number) => labelsRef.current[time - INTRADAY_BASE_TS] ?? ''
|
||||
|
||||
useEffect(() => {
|
||||
const el = containerRef.current
|
||||
if (!el) return
|
||||
|
||||
const chart = createChart(el, {
|
||||
width: el.clientWidth,
|
||||
height: 132,
|
||||
layout: {
|
||||
// 关闭 TV 角标 (licence 归属改由 README 技术栈外链承担)
|
||||
attributionLogo: false,
|
||||
background: { color: 'transparent' },
|
||||
textColor: ctRef.current.text,
|
||||
fontFamily: 'JetBrains Mono, monospace',
|
||||
fontSize: 10,
|
||||
},
|
||||
grid: {
|
||||
vertLines: { color: ctRef.current.grid },
|
||||
horzLines: { color: ctRef.current.grid },
|
||||
},
|
||||
rightPriceScale: { borderColor: ctRef.current.border, scaleMargins: { top: 0.12, bottom: 0.04 } },
|
||||
timeScale: {
|
||||
borderColor: ctRef.current.border,
|
||||
rightOffset: 2,
|
||||
barSpacing: 4,
|
||||
tickMarkFormatter: (time: number) => labelAt(time),
|
||||
},
|
||||
localization: {
|
||||
timeFormatter: (time: number) => labelAt(time),
|
||||
},
|
||||
crosshair: {
|
||||
vertLine: { labelVisible: false },
|
||||
horzLine: { labelVisible: true },
|
||||
},
|
||||
handleScroll: false,
|
||||
handleScale: false,
|
||||
})
|
||||
const sector = chart.addLineSeries({
|
||||
color: INTRADAY_SECTOR_COLOR[kind],
|
||||
lineWidth: 2,
|
||||
priceLineVisible: false,
|
||||
lastValueVisible: true,
|
||||
priceFormat: { type: 'custom', formatter: (v: number) => `${(v * 100).toFixed(2)}%`, minMove: 0.0001 },
|
||||
crosshairMarkerRadius: 3,
|
||||
})
|
||||
const market = chart.addLineSeries({
|
||||
color: INTRADAY_MARKET_COLOR,
|
||||
lineWidth: 1,
|
||||
lineStyle: LineStyle.Dashed,
|
||||
priceLineVisible: false,
|
||||
lastValueVisible: false,
|
||||
crosshairMarkerRadius: 2,
|
||||
})
|
||||
sector.createPriceLine({
|
||||
price: 0,
|
||||
color: ctRef.current.border,
|
||||
lineWidth: 1,
|
||||
lineStyle: LineStyle.Dashed,
|
||||
axisLabelVisible: false,
|
||||
})
|
||||
chartRef.current = chart
|
||||
sectorRef.current = sector
|
||||
marketRef.current = market
|
||||
|
||||
const observer = new ResizeObserver(() => {
|
||||
chart.applyOptions({ width: el.clientWidth })
|
||||
})
|
||||
observer.observe(el)
|
||||
return () => {
|
||||
observer.disconnect()
|
||||
chart.remove()
|
||||
chartRef.current = null
|
||||
sectorRef.current = null
|
||||
marketRef.current = null
|
||||
}
|
||||
}, [kind])
|
||||
|
||||
useEffect(() => {
|
||||
chartRef.current?.applyOptions({
|
||||
layout: { textColor: ct.text },
|
||||
grid: { vertLines: { color: ct.grid }, horzLines: { color: ct.grid } },
|
||||
rightPriceScale: { borderColor: ct.border },
|
||||
timeScale: { borderColor: ct.border },
|
||||
})
|
||||
}, [ct])
|
||||
|
||||
useEffect(() => {
|
||||
const sectorSeries = sectorRef.current
|
||||
const marketSeries = marketRef.current
|
||||
if (!sectorSeries || !marketSeries) return
|
||||
labelsRef.current = points.map(p => p.time)
|
||||
const toData = (key: 'sector' | 'market'): LineData[] =>
|
||||
points
|
||||
.map((p, i) => ({ time: (INTRADAY_BASE_TS + i * 60) as Time, value: p[key] }))
|
||||
.filter((d): d is LineData => d.value != null)
|
||||
sectorSeries.setData(toData('sector'))
|
||||
marketSeries.setData(toData('market'))
|
||||
chartRef.current?.timeScale().fitContent()
|
||||
}, [points])
|
||||
|
||||
return (
|
||||
<div className="px-2 pb-2 pt-1">
|
||||
<div ref={containerRef} className="w-full" />
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
@@ -72,6 +72,9 @@ function buildOption(data: MinuteKlineRow[], prevClose: number | undefined, avgP
|
||||
const volumes = new Array(FULL_DAY_TIMES.length).fill(null) as (any | null)[]
|
||||
|
||||
const volNeutral = 'rgba(161,161,170,0.5)'
|
||||
// 量柱着色基准: 前一分钟 close; 第一根用昨收。
|
||||
// 不用 row.open — stock-sdk 历史日无真实分钟 open(为 null), close-vs-open 会全偏。
|
||||
let prevRef: number | null = prevClose ?? null
|
||||
for (let i = 0; i < data.length; i++) {
|
||||
const timeKey = formatMinuteTime(data[i].datetime)
|
||||
const idx = timeIndexMap.get(timeKey)
|
||||
@@ -83,9 +86,16 @@ function buildOption(data: MinuteKlineRow[], prevClose: number | undefined, avgP
|
||||
volumes[idx] = {
|
||||
value: data[i].volume,
|
||||
itemStyle: {
|
||||
color: data[i].close > data[i].open ? THEME.volUp : data[i].close < data[i].open ? THEME.volDown : volNeutral,
|
||||
color: prevRef == null
|
||||
? volNeutral
|
||||
: data[i].close > prevRef
|
||||
? THEME.volUp
|
||||
: data[i].close < prevRef
|
||||
? THEME.volDown
|
||||
: volNeutral,
|
||||
},
|
||||
}
|
||||
prevRef = data[i].close
|
||||
}
|
||||
}
|
||||
|
||||
@@ -570,7 +580,7 @@ export function EChartsIntraday({
|
||||
<>
|
||||
{date && <span className="text-muted">{date}</span>}
|
||||
<span className="text-muted">开</span>
|
||||
<span style={{ color: priceClr }}>{d.open.toFixed(2)}</span>
|
||||
<span style={{ color: priceClr }}>{d.open != null ? d.open.toFixed(2) : '—'}</span>
|
||||
<span className="text-muted">高</span>
|
||||
<span style={{ color: priceClr }}>{d.high.toFixed(2)}</span>
|
||||
<span className="text-muted">低</span>
|
||||
|
||||
@@ -74,6 +74,9 @@ function buildModel(sessions: MinuteKlineSession[]) {
|
||||
|
||||
const dayValues: (number | null)[] = []
|
||||
const dayAverages: (number | null)[] = []
|
||||
// 量柱着色基准: 前一分钟 close; 当日第一根用 session 昨收。
|
||||
// 不用 row.open — stock-sdk 历史日无真实分钟 open(为 null), close-vs-open 会全偏。
|
||||
let prevRef: number | null = session.prev_close
|
||||
for (const time of FULL_DAY_TIMES) {
|
||||
const point = rowsByTime.get(time)
|
||||
const index = categories.length
|
||||
@@ -91,13 +94,16 @@ function buildModel(sessions: MinuteKlineSession[]) {
|
||||
volumeData.push({
|
||||
value: row.volume,
|
||||
itemStyle: {
|
||||
color: row.close > row.open
|
||||
color: prevRef == null
|
||||
? COLORS.volumeFlat
|
||||
: row.close > prevRef
|
||||
? COLORS.volumeUp
|
||||
: row.close < row.open
|
||||
: row.close < prevRef
|
||||
? COLORS.volumeDown
|
||||
: COLORS.volumeFlat,
|
||||
},
|
||||
})
|
||||
prevRef = row.close
|
||||
priceValues.push(row.low, row.high, average)
|
||||
pointByIndex.set(index, {
|
||||
date: session.date,
|
||||
@@ -408,7 +414,7 @@ export function EChartsMultiDayIntraday({
|
||||
{info ? (
|
||||
<>
|
||||
<span className="text-muted">{info.date} {formatMinuteTime(info.row.datetime)}</span>
|
||||
<span className="text-muted">开</span><span style={{ color: infoColor }}>{info.row.open.toFixed(2)}</span>
|
||||
<span className="text-muted">开</span><span style={{ color: infoColor }}>{info.row.open != null ? info.row.open.toFixed(2) : '—'}</span>
|
||||
<span className="text-muted">高</span><span style={{ color: infoColor }}>{info.row.high.toFixed(2)}</span>
|
||||
<span className="text-muted">低</span><span style={{ color: infoColor }}>{info.row.low.toFixed(2)}</span>
|
||||
<span className="text-muted">收</span><span className="font-semibold" style={{ color: infoColor }}>{info.row.close.toFixed(2)}</span>
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { useEffect, useMemo, useRef, useState, Suspense } from 'react'
|
||||
import { useEffect, useLayoutEffect, useMemo, useRef, useState, Suspense } from 'react'
|
||||
import { NavLink, Outlet, useNavigate, useLocation } from 'react-router-dom'
|
||||
import { useQuery, useQueryClient } from '@tanstack/react-query'
|
||||
import { motion } from 'framer-motion'
|
||||
@@ -10,7 +10,7 @@ import { AiReportBubble } from '@/components/financials/AiReportBubble'
|
||||
import { StockAnalysisHost } from '@/components/stock-analysis/StockAnalysisHost'
|
||||
import { StockAnalysisBubble } from '@/components/stock-analysis/StockAnalysisBubble'
|
||||
import {
|
||||
useCapabilities,
|
||||
useCapabilityMatrix,
|
||||
useSettings,
|
||||
usePreferences,
|
||||
useQuoteStatus,
|
||||
@@ -53,7 +53,7 @@ import {
|
||||
PanelLeftOpen,
|
||||
} from 'lucide-react'
|
||||
import { Logo } from './Logo'
|
||||
import { api, type IndexQuote } from '@/lib/api'
|
||||
import { api, type CapabilityMatrix, type IndexQuote } from '@/lib/api'
|
||||
import { cn } from '@/lib/cn'
|
||||
import { resolveWatchlistGroupColor } from '@/lib/watchlist-group-colors'
|
||||
import { computeGroupPcts, groupPctColor, groupPctTitle } from '@/lib/watchlistGroupStats'
|
||||
@@ -171,80 +171,140 @@ function SidebarIndexQuotes({ rows, items }: { rows: IndexQuote[] | undefined; i
|
||||
)
|
||||
}
|
||||
|
||||
// ===== 档位卡片 =====
|
||||
function TierBadge({ label, hasKey, providerName, isTickflow }: { label: string; hasKey?: boolean; providerName: string; isTickflow: boolean }) {
|
||||
const base = label.split(' ')[0].split('+')[0].toLowerCase()
|
||||
const isNone = base === 'none'
|
||||
// ===== 数据源能力健康卡 =====
|
||||
// 能力路由架构下的侧栏状态: 不再展示「主数据源 + TickFlow 档位」(单源时代遗留 —
|
||||
// 五个能力各自路由, 拿日K的源代表全局是随意的), 改为回答「各能力当前是否都有源在供」。
|
||||
// 档位/订阅信息归设置页 TickFlow 介绍卡 (档位词仅出现在 TickFlow 专属界面的设计规则)。
|
||||
// 单能力方格: 可用=绿 / 日K缺失=红 / 其他缺失=琥珀 (与悬浮卡中同色, 一眼对应)
|
||||
function capSquareCls(c: { id: string; usable: boolean }) {
|
||||
return c.usable ? 'bg-accent' : c.id === 'daily' ? 'bg-danger' : 'bg-warning/80'
|
||||
}
|
||||
|
||||
const tierConfig: Record<string, {
|
||||
desc: string
|
||||
dotStyle: React.CSSProperties
|
||||
tagBg: React.CSSProperties
|
||||
labelTextStyle: React.CSSProperties
|
||||
}> = {
|
||||
none: {
|
||||
desc: '未配置 Key · 仅历史日K',
|
||||
dotStyle: { background: '#52525b' },
|
||||
tagBg: { background: 'rgba(113,113,122,0.15)' },
|
||||
labelTextStyle: { color: '#71717a' },
|
||||
},
|
||||
free: {
|
||||
desc: '基础日K · 自选实时',
|
||||
dotStyle: { background: '#71717a' },
|
||||
tagBg: { background: 'rgba(113,113,122,0.3)' },
|
||||
labelTextStyle: { color: '#a1a1aa' },
|
||||
},
|
||||
starter: {
|
||||
desc: '批量同步 · 行情池',
|
||||
dotStyle: { background: '#3b82f6' },
|
||||
tagBg: { background: 'rgba(59,130,246,0.2)' },
|
||||
labelTextStyle: { color: '#60a5fa' },
|
||||
},
|
||||
pro: {
|
||||
desc: '分钟K · 实时行情 · 盘口',
|
||||
dotStyle: { background: 'linear-gradient(135deg, #a855f7, #7c3aed)' },
|
||||
tagBg: { background: 'linear-gradient(135deg, rgba(168,85,247,0.2), rgba(124,58,237,0.15))' },
|
||||
labelTextStyle: { background: 'linear-gradient(135deg, #c084fc, #a855f7)', WebkitBackgroundClip: 'text', backgroundClip: 'text', color: 'transparent' },
|
||||
},
|
||||
expert: {
|
||||
desc: 'WebSocket · 财务数据',
|
||||
dotStyle: { background: 'linear-gradient(135deg, #3b82f6, #a855f7, #f59e0b)' },
|
||||
tagBg: { background: 'linear-gradient(135deg, rgba(59,130,246,0.2), rgba(168,85,247,0.2), rgba(245,158,11,0.2))' },
|
||||
labelTextStyle: { background: 'linear-gradient(135deg, #60a5fa, #c084fc, #fbbf24)', WebkitBackgroundClip: 'text', backgroundClip: 'text', color: 'transparent' },
|
||||
},
|
||||
function DataSourceHealthBadge({ matrix }: { matrix: CapabilityMatrix | undefined }) {
|
||||
const caps = matrix?.capabilities ?? []
|
||||
const loading = caps.length === 0
|
||||
const usableCount = caps.filter(c => c.usable).length
|
||||
const down = caps.filter(c => !c.usable)
|
||||
// 日K是核心能力 (其他一切派生于它): 挂了用危险色; 一般缺项琥珀; 全可用绿
|
||||
const level = loading
|
||||
? 'loading'
|
||||
: down.length === 0 ? 'ok' : down.some(c => c.id === 'daily') ? 'danger' : 'warn'
|
||||
const countCls = level === 'ok' ? 'text-accent/80'
|
||||
: level === 'danger' ? 'text-danger'
|
||||
: level === 'warn' ? 'text-warning'
|
||||
: 'text-muted'
|
||||
|
||||
// 悬浮卡: 侧栏 aside 是 overflow-hidden, 用 fixed 定位逃逸裁剪 (坐标取自徽标实时位置)。
|
||||
// 徽标靠近屏幕顶部时居中定位会把卡片上半截推出视口 → 渲染后按实际高度钳制进视口。
|
||||
const linkRef = useRef<HTMLAnchorElement>(null)
|
||||
const popRef = useRef<HTMLDivElement>(null)
|
||||
const closeTimer = useRef<number | undefined>(undefined)
|
||||
const [popPos, setPopPos] = useState<{ left: number; top: number } | null>(null)
|
||||
const openPop = () => {
|
||||
window.clearTimeout(closeTimer.current)
|
||||
const rect = linkRef.current?.getBoundingClientRect()
|
||||
if (rect) setPopPos({ left: rect.right, top: rect.top + rect.height / 2 })
|
||||
}
|
||||
|
||||
const t = tierConfig[base] || tierConfig.none
|
||||
const displayLabel = isNone ? 'None' : (label || 'None')
|
||||
const descText = isNone && !hasKey ? '配置 Key 解锁更多能力' : t.desc
|
||||
const closePop = () => {
|
||||
closeTimer.current = window.setTimeout(() => setPopPos(null), 80)
|
||||
}
|
||||
useEffect(() => () => window.clearTimeout(closeTimer.current), [])
|
||||
useLayoutEffect(() => {
|
||||
if (!popPos || !popRef.current) return
|
||||
const h = popRef.current.offsetHeight
|
||||
const margin = 8
|
||||
const minCenter = margin + h / 2
|
||||
const maxCenter = window.innerHeight - margin - h / 2
|
||||
const clamped = Math.min(maxCenter, Math.max(minCenter, popPos.top))
|
||||
if (clamped !== popPos.top) setPopPos({ ...popPos, top: clamped })
|
||||
}, [popPos])
|
||||
|
||||
return (
|
||||
<>
|
||||
<NavLink
|
||||
ref={linkRef}
|
||||
to="/settings?tab=data-sources"
|
||||
aria-label={`数据源能力 ${usableCount}/${caps.length || 5} 可用, 点击前往数据源配置`}
|
||||
onMouseEnter={openPop}
|
||||
onMouseLeave={closePop}
|
||||
onFocus={openPop}
|
||||
onBlur={closePop}
|
||||
onKeyDown={e => { if (e.key === 'Escape') setPopPos(null) }}
|
||||
className="group relative flex items-center gap-2 overflow-hidden rounded-md py-1.5 pl-2.5 pr-2 transition-colors duration-150 hover:bg-elevated/70"
|
||||
title={`数据源 · ${providerName} — ${descText}`}
|
||||
>
|
||||
<span
|
||||
className="pointer-events-none absolute inset-y-1.5 left-0 w-[2px] rounded-full bg-accent/50 transition-colors group-hover:bg-accent"
|
||||
style={base === 'expert' ? { background: 'linear-gradient(180deg, #60a5fa, #c084fc, #fbbf24)' } : undefined}
|
||||
/>
|
||||
<span className="pointer-events-none absolute inset-y-1.5 left-0 w-[2px] rounded-full bg-accent/50 transition-colors group-hover:bg-accent" />
|
||||
<DatabaseZap className="h-3.5 w-3.5 shrink-0 text-muted group-hover:text-accent transition-colors" />
|
||||
<span className="min-w-0 truncate text-[11px] font-medium text-secondary group-hover:text-foreground transition-colors">
|
||||
{providerName || '数据源'}
|
||||
{/* 能力方格 (按注册顺序: 实时/日K/分钟/除权/财务), 与悬浮卡逐格同色对应 */}
|
||||
<span className="flex items-center gap-1 shrink-0">
|
||||
{loading
|
||||
? Array.from({ length: 5 }, (_, i) => (
|
||||
<span key={i} className="h-2 w-2 rounded-[2px] bg-muted animate-pulse" />
|
||||
))
|
||||
: caps.map(c => (
|
||||
<span key={c.id} className={`h-2 w-2 rounded-[2px] ${capSquareCls(c)}`} />
|
||||
))}
|
||||
</span>
|
||||
<span
|
||||
className="h-1.5 w-1.5 rounded-full shrink-0"
|
||||
style={{ ...t.dotStyle, ...(base === 'expert' ? { animation: 'pulse 2s infinite' } : {}) }}
|
||||
/>
|
||||
{isTickflow && (
|
||||
<span
|
||||
className="ml-auto inline-flex items-center rounded px-1.5 py-0.5 text-[10px] font-bold font-mono leading-none shrink-0"
|
||||
style={t.tagBg}
|
||||
>
|
||||
<span className="truncate" style={t.labelTextStyle}>{displayLabel}</span>
|
||||
{!loading && (
|
||||
<span className={`ml-auto text-[10px] font-mono font-bold leading-none shrink-0 ${countCls}`}>
|
||||
{usableCount}/{caps.length}
|
||||
</span>
|
||||
)}
|
||||
</NavLink>
|
||||
{popPos && (
|
||||
<div
|
||||
ref={popRef}
|
||||
className="fixed z-50 -translate-y-1/2 pl-3"
|
||||
style={{ left: popPos.left, top: popPos.top }}
|
||||
onMouseEnter={() => window.clearTimeout(closeTimer.current)}
|
||||
onMouseLeave={closePop}
|
||||
>
|
||||
<motion.div
|
||||
initial={{ opacity: 0, x: -6 }}
|
||||
animate={{ opacity: 1, x: 0 }}
|
||||
transition={{ duration: 0.15, ease: [0.16, 1, 0.3, 1] }}
|
||||
className="w-64 rounded-md border border-border bg-surface py-2.5 pl-3 pr-3.5 shadow-2xl shadow-black/40"
|
||||
>
|
||||
<div className="mb-2 flex items-center justify-between">
|
||||
<span className="flex items-center gap-1.5 text-xs font-medium text-foreground">
|
||||
<DatabaseZap className="h-3.5 w-3.5 text-accent" />
|
||||
数据源能力
|
||||
</span>
|
||||
<span className={`text-[10px] font-mono font-bold ${countCls}`}>
|
||||
{loading ? '获取中…' : `${usableCount}/${caps.length} 可用`}
|
||||
</span>
|
||||
</div>
|
||||
<div className="space-y-1.5 border-t border-border/60 pt-2">
|
||||
{loading ? (
|
||||
<div className="py-0.5 text-[11px] text-muted">正在获取能力路由状态…</div>
|
||||
) : caps.map(c => (
|
||||
<div key={c.id} className="flex min-w-0 items-center gap-2">
|
||||
<span className={`h-2 w-2 shrink-0 rounded-[2px] ${capSquareCls(c)}`} />
|
||||
<span className="shrink-0 text-xs font-medium text-secondary">{c.label}</span>
|
||||
<span className="ml-auto flex min-w-0 shrink items-center gap-1.5">
|
||||
{c.usable ? (
|
||||
<>
|
||||
<span className="truncate text-[11px] text-muted">{c.effective_display}</span>
|
||||
<CheckCircle2 className="h-3 w-3 shrink-0 text-accent" />
|
||||
</>
|
||||
) : (
|
||||
<span className="text-[11px] text-muted/70">未接入</span>
|
||||
)}
|
||||
</span>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
{/* 分时有分钟K功能替身 (intraday_monitor_support 三路可达), 不单独占能力格, 在此备注 */}
|
||||
<div className="mt-1.5 text-[10px] leading-relaxed text-muted/70">
|
||||
分时信号监控可由分钟 K 数据驱动，不单独设能力格
|
||||
</div>
|
||||
<div className="mt-2 flex items-center gap-1 border-t border-border/60 pt-1.5 text-[10px] text-muted">
|
||||
点击前往数据源配置
|
||||
<ChevronRight className="h-3 w-3" />
|
||||
</div>
|
||||
</motion.div>
|
||||
</div>
|
||||
)}
|
||||
</>
|
||||
)
|
||||
}
|
||||
|
||||
@@ -275,8 +335,8 @@ function AIConfigBadge({ configured, model }: { configured?: boolean; model?: st
|
||||
|
||||
export function Layout() {
|
||||
// ===== 共享 hooks (替代内联 useQuery) =====
|
||||
const { data: caps } = useCapabilities()
|
||||
const { data: settingsState } = useSettings()
|
||||
const { data: matrix } = useCapabilityMatrix()
|
||||
const { data: versionData } = useVersion()
|
||||
const { data: prefs } = usePreferences()
|
||||
// 数据源列表 (用于实时行情状态显示当前数据源名称)
|
||||
@@ -446,13 +506,6 @@ export function Layout() {
|
||||
? '关闭实时行情'
|
||||
: '开启实时行情'
|
||||
|
||||
// 当前主数据源 (用于侧边栏数据源状态卡)
|
||||
const activeProvider = prefs?.daily_data_provider || 'tickflow'
|
||||
const activeProviderName = activeProvider === 'tickflow'
|
||||
? 'TickFlow'
|
||||
: (dataSources?.custom?.find(s => s.name === activeProvider)?.display_name || activeProvider)
|
||||
const isCustomActive = activeProvider !== 'tickflow'
|
||||
|
||||
// 轮询触发记录总数 → 更新监控中心徽标 (每 15 秒; 后台标签页由 SSE 事件驱动, 不轮询)
|
||||
const alertsTotalQuery = useQuery({
|
||||
queryKey: ['alerts-total'],
|
||||
@@ -522,10 +575,6 @@ export function Layout() {
|
||||
toast('当前数据源无实时行情能力, 请先配置数据源', 'error')
|
||||
return
|
||||
}
|
||||
if (fresh.mode === 'watchlist' && (prefs?.realtime_watchlist_symbols?.length ?? 0) === 0) {
|
||||
navigate('/watchlist')
|
||||
return
|
||||
}
|
||||
}
|
||||
await toggleQuote.mutateAsync(enabled)
|
||||
// 仅在交易时段立即获取一次行情
|
||||
@@ -575,12 +624,7 @@ export function Layout() {
|
||||
{/* 状态卡 — 收起时隐藏 */}
|
||||
{!navCollapsed && (
|
||||
<div className="mt-2.5 border-t border-border/60 pt-1">
|
||||
<TierBadge
|
||||
label={caps?.label ?? ''}
|
||||
hasKey={settingsState?.mode !== 'none'}
|
||||
providerName={activeProviderName}
|
||||
isTickflow={!isCustomActive}
|
||||
/>
|
||||
<DataSourceHealthBadge matrix={matrix} />
|
||||
<div className="mx-2 border-t border-border/45" aria-hidden="true" />
|
||||
<AIConfigBadge
|
||||
configured={settingsState?.ai_configured ?? settingsState?.has_ai_key}
|
||||
@@ -754,7 +798,7 @@ export function Layout() {
|
||||
当前数据源无实时行情权限,
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => navigate('/settings?tab=data-sources')}
|
||||
onClick={() => navigate('/settings?tab=data-sources&highlight=data-sources')}
|
||||
className="mx-0.5 text-accent/80 hover:text-accent hover:underline"
|
||||
>
|
||||
去配置数据源
|
||||
@@ -777,7 +821,7 @@ export function Layout() {
|
||||
</div>
|
||||
<div className="flex shrink-0 items-center gap-1">
|
||||
<button
|
||||
onClick={() => navigate('/settings?tab=monitoring')}
|
||||
onClick={() => navigate('/settings?tab=monitoring&highlight=quotes')}
|
||||
aria-label="打开实时监控设置"
|
||||
className="flex h-7 w-7 items-center justify-center rounded-btn text-muted transition-colors hover:bg-elevated hover:text-foreground"
|
||||
title="实时监控设置"
|
||||
|
||||
@@ -54,6 +54,8 @@ interface Props {
|
||||
onDataChange?: (result: StockDailyKChartResult) => void
|
||||
/** 扩展数据列参数（逗号分隔 config_id.field_name），透传给 klineDaily 接口 */
|
||||
extColumns?: string
|
||||
/** 日K自动刷新间隔(ms)。undefined = 不轮询(默认)。个股对话框实时刷新时传入, 盘中今日蜡烛随之更新 */
|
||||
refetchIntervalMs?: number
|
||||
}
|
||||
|
||||
function isValidRow(r: any): boolean {
|
||||
@@ -136,6 +138,7 @@ export function StockDailyKChart({
|
||||
onPriceDoubleClick,
|
||||
onDataChange,
|
||||
extColumns,
|
||||
refetchIntervalMs,
|
||||
}: Props) {
|
||||
const [activeIndicators, setActiveIndicators] = useState<string[]>(['vol'])
|
||||
const [showMarkers, setShowMarkers] = useState(true)
|
||||
@@ -150,6 +153,7 @@ export function StockDailyKChart({
|
||||
queryKey: QK.kline(symbol, dateRange.start, dateRange.end, extColumns),
|
||||
queryFn: () => api.klineDaily(symbol, days, dateRange, extColumns),
|
||||
enabled: !!symbol,
|
||||
refetchInterval: refetchIntervalMs,
|
||||
placeholderData: (prev) => prev,
|
||||
})
|
||||
|
||||
|
||||
@@ -36,7 +36,9 @@ export function StockIntradayChart({
|
||||
|
||||
const minute = useQuery({
|
||||
queryKey: QK.klineMinute(symbol, date ?? ''),
|
||||
queryFn: () => api.klineMinute(symbol, date ?? undefined),
|
||||
// 轮询上下文 (个股详情) 传 live: 当日盘中后端直接实时拉取最新K,
|
||||
// 避免读到分钟增量落盘的上一轮本地分区; 历史日期后端自行忽略 live。
|
||||
queryFn: () => api.klineMinute(symbol, date ?? undefined, refetchIntervalMs != null),
|
||||
enabled: !!symbol && !!date,
|
||||
refetchInterval: refetchIntervalMs,
|
||||
})
|
||||
|
||||
@@ -37,7 +37,8 @@ export function StockMultiDayIntradayChart({
|
||||
})
|
||||
const latest = useQuery({
|
||||
queryKey: QK.klineMinute(symbol, ''),
|
||||
queryFn: () => api.klineMinute(symbol),
|
||||
// live: 当日盘中直接实时拉取, 不被分钟增量落盘的本地分区(≥60s一轮)拖慢
|
||||
queryFn: () => api.klineMinute(symbol, undefined, true),
|
||||
enabled: !!symbol,
|
||||
refetchInterval: refetchIntervalMs,
|
||||
})
|
||||
|
||||
@@ -167,6 +167,7 @@ export function StockPanel({
|
||||
onDataChange={setDailyResult}
|
||||
visibleBars={showIntraday ? 40 : 60}
|
||||
extColumns={extColumns}
|
||||
refetchIntervalMs={refetchIntervalMs}
|
||||
/>
|
||||
|
||||
{showIntraday && selectedDate && !intradayDismissed && (
|
||||
|
||||
@@ -14,7 +14,7 @@ import { DatePicker } from '@/components/DatePicker'
|
||||
import { RuleEditor } from '@/components/monitor/RuleEditor'
|
||||
import { PriceAlertDialog } from '@/components/stock-analysis/PriceAlertDialog'
|
||||
import { buildMonitorPriceLines } from '@/lib/price-alerts'
|
||||
import { usePreferences, useQuoteStatus } from '@/lib/useSharedQueries'
|
||||
import { usePreferences } from '@/lib/useSharedQueries'
|
||||
import { setFocusSymbol, clearFocusSymbol } from '@/lib/useQuoteStream'
|
||||
import { useDialogBackdrop } from '@/lib/useDialogBackdrop'
|
||||
import { storage } from '@/lib/storage'
|
||||
@@ -50,11 +50,11 @@ interface PriceAlertDraft {
|
||||
}
|
||||
const INTRADAY_DAY_OPTIONS = [1, 5, 10, 20] as const
|
||||
|
||||
function loadIntradayDays(): number {
|
||||
function loadIntradayDays(): number | null {
|
||||
const saved = storage.stockPreviewIntradayDays.get(10)
|
||||
return INTRADAY_DAY_OPTIONS.includes(saved as typeof INTRADAY_DAY_OPTIONS[number])
|
||||
? saved
|
||||
: 10
|
||||
: null
|
||||
}
|
||||
|
||||
function boardTag(symbol: string): { label: string; color: string } | null {
|
||||
@@ -81,7 +81,7 @@ function fmtAbnormalCalcTime(asofSec: number): string {
|
||||
|
||||
export function StockPreviewDialog({ symbol, name, onClose, triggerInfo }: Props) {
|
||||
const [view, setView] = useState<PreviewView>('daily')
|
||||
const [intradayDays, setIntradayDays] = useState(loadIntradayDays)
|
||||
const [intradayDays, setIntradayDays] = useState<number | null>(loadIntradayDays)
|
||||
const [dateRange, setDateRange] = useState(getDefaultRange)
|
||||
const [showMonitorEditor, setShowMonitorEditor] = useState(false)
|
||||
const [priceAlertDraft, setPriceAlertDraft] = useState<PriceAlertDraft | null>(null)
|
||||
@@ -161,15 +161,26 @@ export function StockPreviewDialog({ symbol, name, onClose, triggerInfo }: Props
|
||||
return () => clearFocusSymbol()
|
||||
}, [symbol])
|
||||
|
||||
// 分时图实时轮询: 复用自选列表的「分时刷新开关 + 间隔」偏好。
|
||||
// 仅实时行情运行 且 用户开启分时刷新时才轮询; 否则 undefined (定格)。
|
||||
// 分时图实时轮询: 详情打开即独立轮询, 不再依赖自选列表的「分时刷新」开关
|
||||
// 与实时行情运行状态 (打开详情就是要看实时分时); 间隔沿用偏好, 默认 6s。
|
||||
// 最新一根K由后端 live 参数直接实时拉取, 与行情列表节奏一致。
|
||||
const { data: prefs } = usePreferences()
|
||||
const { data: quoteStatus } = useQuoteStatus()
|
||||
const realtimeRunning = quoteStatus?.running ?? false
|
||||
const intradayRefreshOn = prefs?.minute_intraday_refresh ?? false
|
||||
const intradayRefetchMs = (intradayRefreshOn && realtimeRunning)
|
||||
? (prefs?.minute_intraday_refresh_interval ?? 6) * 1000
|
||||
: undefined
|
||||
const intradayRefetchMs = (prefs?.minute_intraday_refresh_interval ?? 6) * 1000
|
||||
|
||||
// 分时档位按分钟源历史深度收窄: 浅源(如 stock-sdk=5日)只显示可行档位、默认 5日;
|
||||
// 深源(tickflow/未声明)全档位、默认 20日。用户已保存的可行选择优先保留。
|
||||
const minuteHistoryDays = prefs?.minute_history_days ?? null
|
||||
const dayOptions = useMemo<number[]>(
|
||||
() => INTRADAY_DAY_OPTIONS.filter(d => minuteHistoryDays == null || d <= minuteHistoryDays),
|
||||
[minuteHistoryDays],
|
||||
)
|
||||
const defaultIntradayDays = minuteHistoryDays != null && minuteHistoryDays < 20 ? 5 : 20
|
||||
const effectiveIntradayDays = intradayDays ?? defaultIntradayDays
|
||||
useEffect(() => {
|
||||
if (!dayOptions.includes(effectiveIntradayDays)) {
|
||||
setIntradayDays(defaultIntradayDays)
|
||||
}
|
||||
}, [dayOptions, effectiveIntradayDays, defaultIntradayDays])
|
||||
|
||||
const handleRefresh = () => {
|
||||
if (!symbol) return
|
||||
@@ -274,14 +285,14 @@ export function StockPreviewDialog({ symbol, name, onClose, triggerInfo }: Props
|
||||
) : (
|
||||
<div className="flex items-center gap-1">
|
||||
<div className="inline-flex shrink-0 items-center rounded border border-border bg-elevated p-0.5" aria-label="分时周期">
|
||||
{INTRADAY_DAY_OPTIONS.map(days => (
|
||||
{dayOptions.map(days => (
|
||||
<button
|
||||
key={days}
|
||||
type="button"
|
||||
aria-pressed={intradayDays === days}
|
||||
aria-pressed={effectiveIntradayDays === days}
|
||||
onClick={() => selectIntradayDays(days)}
|
||||
className={`h-5 rounded px-1.5 font-mono text-[10px] transition-colors ${
|
||||
intradayDays === days
|
||||
effectiveIntradayDays === days
|
||||
? 'bg-accent/20 text-accent'
|
||||
: 'text-muted hover:text-secondary'
|
||||
}`}
|
||||
@@ -479,6 +490,7 @@ export function StockPreviewDialog({ symbol, name, onClose, triggerInfo }: Props
|
||||
dateRange={dateRange}
|
||||
priceLines={monitorPriceLines}
|
||||
onPriceDoubleClick={openPriceAlert}
|
||||
refetchIntervalMs={intradayRefetchMs}
|
||||
/>
|
||||
) : (
|
||||
<>
|
||||
@@ -489,7 +501,7 @@ export function StockPreviewDialog({ symbol, name, onClose, triggerInfo }: Props
|
||||
/>
|
||||
<StockMultiDayIntradayChart
|
||||
symbol={symbol}
|
||||
days={intradayDays}
|
||||
days={effectiveIntradayDays}
|
||||
height={480}
|
||||
refetchIntervalMs={intradayRefetchMs}
|
||||
priceLines={monitorPriceLines}
|
||||
|
||||
@@ -5,8 +5,9 @@ import { api } from '@/lib/api'
|
||||
import { QK } from '@/lib/queryKeys'
|
||||
import { MissingCapChip } from '@/lib/capability-labels'
|
||||
|
||||
export function ExtendHistoryPanel({ caps, isRunning, earliestDate, onStart }: {
|
||||
caps: { label: string; capabilities: Record<string, { rpm: number | null; batch: number | null; subscribe: number | null }> } | undefined
|
||||
// hasCap: 日K批量能力当前是否可用 (路由矩阵判定, 生效源含插件/自定义源)
|
||||
export function ExtendHistoryPanel({ hasCap, isRunning, earliestDate, onStart }: {
|
||||
hasCap: boolean
|
||||
isRunning: boolean
|
||||
earliestDate: string | null
|
||||
onStart: () => void
|
||||
@@ -14,7 +15,7 @@ export function ExtendHistoryPanel({ caps, isRunning, earliestDate, onStart }: {
|
||||
const qc = useQueryClient()
|
||||
const [value, setValue] = useState(6)
|
||||
const [unit, setUnit] = useState<'month' | 'year'>('month')
|
||||
const hasBatchCap = !!caps?.capabilities?.['kline.daily.batch']
|
||||
const hasBatchCap = hasCap
|
||||
|
||||
const extend = useMutation({
|
||||
mutationFn: () => api.extendHistory(value, unit),
|
||||
|
||||
@@ -5,7 +5,8 @@ import { api } from '@/lib/api'
|
||||
import { QK } from '@/lib/queryKeys'
|
||||
import { MissingCapChip } from '@/lib/capability-labels'
|
||||
|
||||
export function MinuteSyncConfig({ caps, onJobStart }: { caps: { label: string; capabilities: Record<string, { rpm: number | null; batch: number | null; subscribe: number | null }> } | undefined; onJobStart?: (jobId: string) => void }) {
|
||||
// hasCap: 分钟K能力当前是否可用 (路由矩阵判定, 生效源含插件/自定义源)
|
||||
export function MinuteSyncConfig({ hasCap, onJobStart }: { hasCap: boolean; onJobStart?: (jobId: string) => void }) {
|
||||
const qc = useQueryClient()
|
||||
const prefs = useQuery({
|
||||
queryKey: QK.preferences,
|
||||
@@ -17,7 +18,7 @@ export function MinuteSyncConfig({ caps, onJobStart }: { caps: { label: string;
|
||||
onSuccess: () => qc.invalidateQueries({ queryKey: QK.preferences }),
|
||||
})
|
||||
|
||||
const hasMinuteCap = !!caps?.capabilities?.['kline.minute.batch']
|
||||
const hasMinuteCap = hasCap
|
||||
const enabled = prefs.data?.minute_sync_enabled ?? false
|
||||
const days = prefs.data?.minute_sync_days ?? 5
|
||||
const segmentDays = prefs.data?.minute_sync_segment_days ?? 20
|
||||
@@ -27,6 +28,9 @@ export function MinuteSyncConfig({ caps, onJobStart }: { caps: { label: string;
|
||||
useEffect(() => { setLocalDays(days) }, [days])
|
||||
useEffect(() => { setLocalSegment(segmentDays) }, [segmentDays])
|
||||
|
||||
useEffect(() => { setLocalDays(days) }, [days])
|
||||
useEffect(() => { setLocalSegment(segmentDays) }, [segmentDays])
|
||||
|
||||
const handleToggle = () => {
|
||||
if (!hasMinuteCap) return
|
||||
update.mutate({ enabled: !enabled, days: localDays })
|
||||
|
||||
@@ -18,14 +18,15 @@ function daysAgo(n: number): string {
|
||||
return `${d.getFullYear()}-${pad(d.getMonth() + 1)}-${pad(d.getDate())}`
|
||||
}
|
||||
|
||||
export function RepairDailyPanel({ caps, isRunning, latestDate, onStart }: {
|
||||
caps: { label: string; capabilities: Record<string, { rpm: number | null; batch: number | null; subscribe: number | null }> } | undefined
|
||||
// hasCap: 日K批量能力当前是否可用 (路由矩阵判定, 生效源含插件/自定义源)
|
||||
export function RepairDailyPanel({ hasCap, isRunning, latestDate, onStart }: {
|
||||
hasCap: boolean
|
||||
isRunning: boolean
|
||||
latestDate: string | null
|
||||
onStart: () => void
|
||||
}) {
|
||||
const qc = useQueryClient()
|
||||
const hasBatchCap = !!caps?.capabilities?.['kline.daily.batch']
|
||||
const hasBatchCap = hasCap
|
||||
|
||||
// 默认起始日期: 最新数据往前推 30 天 (兼顾补缺口 + 复核近期数据, 成本不高)
|
||||
const [startDate, setStartDate] = useState(daysAgo(30))
|
||||
|
||||
@@ -32,6 +32,10 @@ export function Pill({ label, value }: { label: string; value: number | string }
|
||||
)
|
||||
}
|
||||
|
||||
// 卡片能力徽章的输入: TickFlow 套餐限额对象, 或路由门控合并后的布尔可用性
|
||||
// (矩阵 usable 合并进 caps 时, 非对象真值表示「经其他数据源可用, 无套餐限额可显」)。
|
||||
export type CapLimitValue = { rpm: number | null; batch: number | null; subscribe: number | null } | boolean
|
||||
|
||||
function CapBadge({ hasCap, isLocal, missingCapName, capInfo, localSuffix, customProvider }: {
|
||||
hasCap: boolean
|
||||
isLocal: boolean
|
||||
@@ -101,7 +105,7 @@ export function StatCard({
|
||||
skipped?: boolean
|
||||
stagePct?: number
|
||||
tierKey?: string
|
||||
capLimits?: Record<string, { rpm: number | null; batch: number | null; subscribe: number | null }>
|
||||
capLimits?: Record<string, CapLimitValue>
|
||||
customProvider?: string | null
|
||||
onSettings?: () => void
|
||||
onShowFields?: (table?: string) => void
|
||||
@@ -123,8 +127,10 @@ export function StatCard({
|
||||
|
||||
const meta = tierKey ? CARD_META[tierKey] : undefined
|
||||
const isLocal = meta?.capKey === ''
|
||||
const capInfo = meta?.capKey ? capLimits?.[meta.capKey] : undefined
|
||||
const hasCap = isLocal || !!capInfo
|
||||
// 布尔值 (路由门控合并) 只表达可用性; 限额信息仅当值为套餐对象时展示
|
||||
const rawCap = meta?.capKey ? capLimits?.[meta.capKey] : undefined
|
||||
const capInfo = rawCap && typeof rawCap === 'object' ? rawCap : undefined
|
||||
const hasCap = isLocal || !!rawCap
|
||||
|
||||
// 渲染字段说明入口图标
|
||||
// - fieldTabs 提供时: 返回 null (图标由 renderSubLabelInline 内联到文字后)
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import { useEffect, useMemo, useRef, useState } from 'react'
|
||||
import { Link } from 'react-router-dom'
|
||||
import { useMutation, useQuery, useQueryClient } from '@tanstack/react-query'
|
||||
import { Activity, Building2, ChartNoAxesCombined, Check, ChevronDown, ChevronUp, Eraser, Layers3, ListPlus, Plus, RadioTower, Save, Search, Siren, Tags, TrendingUp, Waypoints, X } from 'lucide-react'
|
||||
import { Activity, BarChart3, Building2, ChartNoAxesCombined, Check, ChevronDown, ChevronUp, Eraser, Layers3, ListPlus, Plus, RadioTower, Save, Search, Siren, Tags, TrendingUp, Waypoints, X } from 'lucide-react'
|
||||
import { api, genRuleId, type MonitorRule, type MonitorCondition, type SectorKind, type SectorMonitorTarget, type StrategyNotifyEvent } from '@/lib/api'
|
||||
import { DEFAULT_STRATEGY_NOTIFY_EVENTS, LEGACY_STRATEGY_NOTIFY_EVENTS, STRATEGY_NOTIFY_EVENT_OPTIONS } from '@/lib/strategyMonitorEvents'
|
||||
import { QK } from '@/lib/queryKeys'
|
||||
@@ -9,7 +9,7 @@ import { boardTag } from '@/components/stock-table/primitives'
|
||||
import { resolveWatchlistGroupColor } from '@/lib/watchlist-group-colors'
|
||||
import { SignalPicker } from '@/components/screener/SignalPicker'
|
||||
import { MONITOR_INTRADAY_SIGNAL_OPTIONS, SIGNAL_OPTIONS, cnSignal } from '@/lib/signals'
|
||||
import { usePreferences } from '@/lib/useSharedQueries'
|
||||
import { usePreferences, useQuoteStatus } from '@/lib/useSharedQueries'
|
||||
|
||||
interface Props {
|
||||
/** 编辑现有规则;null=新建 */
|
||||
@@ -23,7 +23,7 @@ interface Props {
|
||||
}
|
||||
|
||||
const TYPE_DEFAULT_NAME: Record<string, string> = {
|
||||
signal: '信号监控', price: '价格监控', market: '市场异动监控', strategy: '策略监控', sector: '板块监控', abnormal: '异动监控',
|
||||
signal: '信号监控', price: '价格监控', market: '市场异动监控', strategy: '策略监控', sector: '板块监控', abnormal: '异动监控', volume_delta: '轮询放量监控',
|
||||
}
|
||||
|
||||
const TYPE_ICONS = {
|
||||
@@ -33,6 +33,7 @@ const TYPE_ICONS = {
|
||||
strategy: Waypoints,
|
||||
sector: Layers3,
|
||||
abnormal: Siren,
|
||||
volume_delta: BarChart3,
|
||||
}
|
||||
|
||||
const SECTOR_KIND_OPTIONS: Array<{ key: SectorKind; label: string; icon: typeof ChartNoAxesCombined }> = [
|
||||
@@ -73,6 +74,7 @@ const emptyRule = (preset?: Partial<MonitorRule>): MonitorRule => ({
|
||||
cooldown_seconds: 3600,
|
||||
severity: 'info',
|
||||
message: '',
|
||||
threshold_volume: 9000,
|
||||
...preset,
|
||||
})
|
||||
|
||||
@@ -80,6 +82,8 @@ export function RuleEditor({ rule, preset, simple, onClose, onSaved }: Props) {
|
||||
const qc = useQueryClient()
|
||||
const options = useQuery({ queryKey: QK.monitorRuleOptions, queryFn: api.monitorRuleOptions })
|
||||
const { data: prefs } = usePreferences()
|
||||
const { data: quoteStatus } = useQuoteStatus()
|
||||
const quoteInterval = quoteStatus?.interval_s
|
||||
const feishuConfigured = !!(prefs?.feishu_webhook_url)
|
||||
const wecomConfigured = !!(prefs?.wecom_webhook_url)
|
||||
const [editing] = useState(!!rule)
|
||||
@@ -211,6 +215,18 @@ export function RuleEditor({ rule, preset, simple, onClose, onSaved }: Props) {
|
||||
if ((d.threshold_pct ?? 0) < 1 || (d.threshold_pct ?? 0) > 150) {
|
||||
throw new Error('接近度阈值必须在 1 到 150 之间 (70=边缘, 100=已触发)')
|
||||
}
|
||||
} else if (d.type === 'volume_delta') {
|
||||
delete d.score_min
|
||||
delete d.score_max
|
||||
d.conditions = []
|
||||
delete d.notify_events
|
||||
if (d.metric === 'amount') {
|
||||
if (!Number.isFinite(d.threshold_amount) || (d.threshold_amount ?? 0) < 1) {
|
||||
throw new Error('金额阈值必须是 ≥1 的数字 (万元)')
|
||||
}
|
||||
} else if (!Number.isFinite(d.threshold_volume) || (d.threshold_volume ?? 0) < 1) {
|
||||
throw new Error('单轮放量阈值必须是 ≥1 的手数')
|
||||
}
|
||||
} else {
|
||||
delete d.score_min
|
||||
delete d.score_max
|
||||
@@ -591,12 +607,24 @@ export function RuleEditor({ rule, preset, simple, onClose, onSaved }: Props) {
|
||||
return {
|
||||
...d,
|
||||
type,
|
||||
// 轮询放量依赖全市场股票快照, 仅支持个股
|
||||
asset_type: type === 'volume_delta' ? 'stock' : d.asset_type,
|
||||
notify_events: type === 'strategy'
|
||||
? [...(d.notify_events ?? DEFAULT_STRATEGY_NOTIFY_EVENTS)]
|
||||
: undefined,
|
||||
scope: type === 'sector' || type === 'abnormal'
|
||||
scope: type === 'sector' || type === 'abnormal' || type === 'volume_delta'
|
||||
? 'all'
|
||||
: type === 'strategy' && d.scope === 'symbols' && d.symbols.length === 0 ? 'all' : d.scope,
|
||||
// 轮询放量: 冷却期默认 300s (持续放量会连续多轮达标); 切走时还原 3600
|
||||
cooldown_seconds: type === 'volume_delta' && d.type !== 'volume_delta' ? 300
|
||||
: type !== 'volume_delta' && d.type === 'volume_delta' ? 3600
|
||||
: d.cooldown_seconds,
|
||||
// 轮询放量: metric / 金额阈值 / 基础过滤默认 (与策略 basic_filter 对齐)
|
||||
metric: type === 'volume_delta' && d.type !== 'volume_delta' ? 'volume' : d.metric,
|
||||
threshold_amount: type === 'volume_delta' && d.type !== 'volume_delta' ? 1e6 : d.threshold_amount,
|
||||
basic_filter: type === 'volume_delta' && d.type !== 'volume_delta'
|
||||
? { price_min: 3, price_max: 300, market_cap_min: 10e8, float_cap_min: null, float_cap_max: null, amount_min: 0.2e8, exclude_st: true }
|
||||
: d.basic_filter,
|
||||
direction: type === 'sector' ? 'up'
|
||||
: type === 'abnormal' ? 'both'
|
||||
: d.type === 'sector' || d.type === 'abnormal' ? 'entry' : d.direction,
|
||||
@@ -895,6 +923,119 @@ export function RuleEditor({ rule, preset, simple, onClose, onSaved }: Props) {
|
||||
</div>
|
||||
)}
|
||||
|
||||
{draft.type === 'volume_delta' && (
|
||||
<div className="space-y-4 border-t border-border/60 pt-4">
|
||||
<div className="grid gap-3 sm:grid-cols-2">
|
||||
<div className="space-y-1.5">
|
||||
<span className="text-[11px] text-muted">阈值口径</span>
|
||||
<div className="grid h-9 grid-cols-2 overflow-hidden rounded-btn border border-border bg-base">
|
||||
{([['volume', '按手数'], ['amount', '按金额']] as const).map(([key, label]) => (
|
||||
<button
|
||||
key={key}
|
||||
type="button"
|
||||
aria-pressed={(draft.metric ?? 'volume') === key}
|
||||
onClick={() => setDraft(d => ({ ...d, metric: key }))}
|
||||
className={`text-[11px] font-medium transition-colors cursor-pointer ${
|
||||
(draft.metric ?? 'volume') === key ? 'bg-accent/10 text-accent' : 'text-muted hover:text-foreground'
|
||||
}`}
|
||||
>
|
||||
{label}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
<label className="space-y-1.5">
|
||||
<span className="text-[11px] text-muted">
|
||||
单轮放量阈值{draft.metric === 'amount' ? ' (万元)' : ' (手)'}
|
||||
</span>
|
||||
<span className="relative block">
|
||||
<input
|
||||
type="number"
|
||||
min="1"
|
||||
step={draft.metric === 'amount' ? 10 : 1000}
|
||||
value={draft.metric === 'amount'
|
||||
? Math.round((draft.threshold_amount ?? 1e6) / 1e4)
|
||||
: (draft.threshold_volume ?? 9000)}
|
||||
onChange={event => setDraft(d => {
|
||||
const v = Number(event.target.value)
|
||||
return d.metric === 'amount'
|
||||
? { ...d, threshold_amount: v * 1e4 }
|
||||
: { ...d, threshold_volume: v }
|
||||
})}
|
||||
className="h-9 w-full rounded-btn border border-border bg-base pl-3 pr-12 text-xs font-mono text-foreground"
|
||||
/>
|
||||
<span className="absolute right-3 top-2.5 text-xs text-muted">
|
||||
{draft.metric === 'amount' ? '万元' : '手'}
|
||||
</span>
|
||||
</span>
|
||||
<span className="block text-[10px] text-muted/70">
|
||||
相邻两次全市场轮询 (当前间隔 {Math.round(quoteInterval ?? 6)} 秒) 间的增量 ≥ 此值即提醒;
|
||||
金额口径对不同股价更公平
|
||||
</span>
|
||||
</label>
|
||||
</div>
|
||||
|
||||
<div className="space-y-2">
|
||||
<span className="text-[11px] text-muted">基础过滤 (与策略选股口径对齐, 留空不过滤)</span>
|
||||
<div className="grid gap-2 sm:grid-cols-3">
|
||||
<label className="space-y-1">
|
||||
<span className="text-[10px] text-muted/70">股价区间 (元)</span>
|
||||
<div className="flex items-center gap-1">
|
||||
<input type="number" min="0" step="0.5" placeholder="下限"
|
||||
value={draft.basic_filter?.price_min ?? ''}
|
||||
onChange={e => setDraft(d => ({ ...d, basic_filter: { ...d.basic_filter, price_min: e.target.value === '' ? null : Number(e.target.value) } }))}
|
||||
className="h-8 w-full rounded border border-border bg-base px-2 text-xs font-mono text-foreground" />
|
||||
<span className="text-[10px] text-muted">—</span>
|
||||
<input type="number" min="0" step="0.5" placeholder="上限"
|
||||
value={draft.basic_filter?.price_max ?? ''}
|
||||
onChange={e => setDraft(d => ({ ...d, basic_filter: { ...d.basic_filter, price_max: e.target.value === '' ? null : Number(e.target.value) } }))}
|
||||
className="h-8 w-full rounded border border-border bg-base px-2 text-xs font-mono text-foreground" />
|
||||
</div>
|
||||
</label>
|
||||
<label className="space-y-1">
|
||||
<span className="text-[10px] text-muted/70">总市值下限 (亿元)</span>
|
||||
<input type="number" min="0" step="1" placeholder="不限"
|
||||
value={draft.basic_filter?.market_cap_min != null ? draft.basic_filter.market_cap_min / 1e8 : ''}
|
||||
onChange={e => setDraft(d => ({ ...d, basic_filter: { ...d.basic_filter, market_cap_min: e.target.value === '' ? null : Number(e.target.value) * 1e8 } }))}
|
||||
className="h-8 w-full rounded border border-border bg-base px-2 text-xs font-mono text-foreground" />
|
||||
</label>
|
||||
<label className="space-y-1">
|
||||
<span className="text-[10px] text-muted/70">当日成交额下限 (万元)</span>
|
||||
<input type="number" min="0" step="100" placeholder="不限"
|
||||
value={draft.basic_filter?.amount_min != null ? draft.basic_filter.amount_min / 1e4 : ''}
|
||||
onChange={e => setDraft(d => ({ ...d, basic_filter: { ...d.basic_filter, amount_min: e.target.value === '' ? null : Number(e.target.value) * 1e4 } }))}
|
||||
className="h-8 w-full rounded border border-border bg-base px-2 text-xs font-mono text-foreground" />
|
||||
</label>
|
||||
<label className="space-y-1">
|
||||
<span className="text-[10px] text-muted/70">流通市值下限 (亿元)</span>
|
||||
<input type="number" min="0" step="1" placeholder="不限"
|
||||
value={draft.basic_filter?.float_cap_min != null ? draft.basic_filter.float_cap_min / 1e8 : ''}
|
||||
onChange={e => setDraft(d => ({ ...d, basic_filter: { ...d.basic_filter, float_cap_min: e.target.value === '' ? null : Number(e.target.value) * 1e8 } }))}
|
||||
className="h-8 w-full rounded border border-border bg-base px-2 text-xs font-mono text-foreground" />
|
||||
</label>
|
||||
<label className="space-y-1">
|
||||
<span className="text-[10px] text-muted/70">流通市值上限 (亿元)</span>
|
||||
<input type="number" min="0" step="1" placeholder="不限"
|
||||
value={draft.basic_filter?.float_cap_max != null ? draft.basic_filter.float_cap_max / 1e8 : ''}
|
||||
onChange={e => setDraft(d => ({ ...d, basic_filter: { ...d.basic_filter, float_cap_max: e.target.value === '' ? null : Number(e.target.value) * 1e8 } }))}
|
||||
className="h-8 w-full rounded border border-border bg-base px-2 text-xs font-mono text-foreground" />
|
||||
</label>
|
||||
<label className="flex items-center gap-2 pt-4">
|
||||
<input type="checkbox" checked={draft.basic_filter?.exclude_st ?? true}
|
||||
onChange={e => setDraft(d => ({ ...d, basic_filter: { ...d.basic_filter, exclude_st: e.target.checked } }))}
|
||||
className="h-3.5 w-3.5 accent-[hsl(var(--accent))]" />
|
||||
<span className="text-[11px] text-secondary">剔除 ST / 风险警示</span>
|
||||
</label>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="rounded-btn bg-base px-3 py-2 text-[10px] leading-relaxed text-muted">
|
||||
捕捉单次轮询间隔内的突发放量 (大单连续扫货)。开盘首轮与暂停恢复后的第一轮不触发,
|
||||
防止集合竞价撮合量误报; 冷却期内同一标的不重复提醒, 命中超过 5 只时合并为一条批量通知。
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* 作用范围 */}
|
||||
{draft.type !== 'sector' && <div className="space-y-2">
|
||||
<span className="text-[11px] text-muted">作用范围</span>
|
||||
@@ -1106,7 +1247,7 @@ export function RuleEditor({ rule, preset, simple, onClose, onSaved }: Props) {
|
||||
</div>}
|
||||
|
||||
{/* 触发条件 (非 strategy) */}
|
||||
{draft.type !== 'strategy' && draft.type !== 'sector' && draft.type !== 'abnormal' && (
|
||||
{draft.type !== 'strategy' && draft.type !== 'sector' && draft.type !== 'abnormal' && draft.type !== 'volume_delta' && (
|
||||
<div className="space-y-3">
|
||||
<div className="flex items-center justify-between">
|
||||
<span className="text-[11px] text-muted">触发条件</span>
|
||||
@@ -1402,7 +1543,7 @@ export function RuleEditor({ rule, preset, simple, onClose, onSaved }: Props) {
|
||||
return (
|
||||
<p className="text-[10px] leading-relaxed text-warning/80">
|
||||
{unconfigured.join('、')}尚未配置,
|
||||
<Link to="/settings?tab=monitoring" className="text-accent hover:text-accent/80">前往设置页配置 →</Link>
|
||||
<Link to="/settings?tab=monitoring&highlight=webhooks" className="text-accent hover:text-accent/80">前往设置页配置 →</Link>
|
||||
</p>
|
||||
)
|
||||
})()}
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user