Author SHA1 Message Date
Codex Local 98fb2965d0 feat: complete device onboarding and mobile binding workflows
ci / verify (push) Canceled after 0s
2026-09-12 08:57:29 +08:00
Codex Local 7846f30d0b fix: make profile tag removal transparent 2026-09-07 09:48:49 +08:00
Codex Local 398b769ed5 fix: soften destructive profile actions 2026-09-07 07:38:32 +08:00
Codex Local 1d0957281a fix: soften elder profile controls 2026-09-07 07:07:16 +08:00
Codex Local cd98cb54fd fix: stop disguised follow-up questions 2026-09-06 23:00:57 +08:00
Codex Local bff1e69db5 fix: reduce consecutive elder interview questions 2026-09-06 22:23:49 +08:00
Codex Local 8c834c0aa7 feat: add natural elder story companion flow 2026-09-06 20:16:01 +08:00
Codex Local 029a6097a2 fix: recover interrupted conversations on boot 2026-09-06 18:15:32 +08:00
Codex Local d60f5d9a9b feat: enable S2S with native Viking memory 2026-09-06 17:27:49 +08:00
Codex Local 0c2151ace9 chore: checkpoint 第一版
ci / verify (push) Canceled after 0s
2026-09-06 17:04:11 +08:00
Codex Local a8cdec98d1 feat: finalize operations model workspace 2026-08-31 11:29:56 +08:00
Codex Local 78e0a19acb fix: make reminder actions use real resources 2026-08-30 21:22:21 +08:00
Codex Local dcf0fc0d7f feat: add domestic model provider presets 2026-08-30 19:55:17 +08:00
Codex Local 522785eda8 feat: filter meaningful conversations with local model 2026-08-30 15:20:08 +08:00
Codex Local 89f6641331 feat: redesign family companion customer portal
ci / verify (push) Canceled after 0s
2026-08-30 14:26:28 +08:00
Codex Local 2337c6cfaa fix: align usage heatmap month labels
ci / verify (push) Canceled after 0s
2026-08-30 11:54:52 +08:00
Codex Local 962dd1a063 feat: add managed llm summaries and memory timeline
ci / verify (push) Canceled after 0s
2026-08-30 10:51:32 +08:00
Codex Local 47f8a164fa feat: redesign elder companion customer experience
ci / verify (push) Canceled after 0s
2026-08-30 01:11:42 +08:00
Codex Local f0849119f7 feat: improve elder companion conversations 2026-08-29 22:06:30 +08:00
Codex Local e652516147 feat: establish v0.70 platform milestone
ci / verify (push) Canceled after 0s
2026-08-29 21:56:51 +08:00
Codex Local d97bbf7cf5 feat: add customer portal and RTC voice diagnostics 2026-08-27 21:02:37 +08:00
Codex Local 5ff1961d79 feat: add Windows real RTC voice device simulator 2026-08-27 11:02:32 +08:00
Codex Local 3b827431c8 chore: make server RTC app id configurable 2026-08-27 09:36:47 +08:00
Codex Local 56b8e0c6da feat: verify production runtime integrity 2026-08-26 23:30:59 +08:00
Codex Local feb99de192 feat: require production infrastructure evidence 2026-08-26 22:56:19 +08:00
Codex Local 3786db0de8 feat: monitor public listener exposure 2026-08-26 22:38:36 +08:00
Codex Local 7435af7c07 feat: support ECS RAM role backups 2026-08-26 22:27:49 +08:00
Codex Local e4f7701439 feat: gate releases on platform health 2026-08-26 22:04:38 +08:00
Codex Local 47ef596d3b feat: expose signed platform health status 2026-08-26 21:15:03 +08:00
Codex Local 1e12d329dc feat: add OSS offsite backup gate 2026-08-26 20:55:35 +08:00
Codex Local 0bf375155e feat: monitor cloud server health 2026-08-26 20:41:26 +08:00
Codex Local fb4691c7b2 feat: add encrypted server backup verification 2026-08-26 20:34:49 +08:00
Codex Local 8607ec33e7 fix: finalize standard HTTPS deployment 2026-08-26 11:19:43 +08:00
913 changed files with 231226 additions and 1217 deletions
+17
View File
@@ -72,6 +72,8 @@ VOLC_RTC_API_VERSION=2024-12-01
VOLC_VOICE_CHAT_TEMPLATE_FILE= VOLC_VOICE_CHAT_TEMPLATE_FILE=
PROVIDER_CALLBACK_SECRET=local-provider-callback-secret-32-bytes-minimum PROVIDER_CALLBACK_SECRET=local-provider-callback-secret-32-bytes-minimum
PROVIDER_CALLBACK_CLOCK_SKEW=5m PROVIDER_CALLBACK_CLOCK_SKEW=5m
PLATFORM_HEALTH_CALLBACK_SECRET=local-platform-health-callback-secret-32-bytes-minimum
PLATFORM_HEALTH_CALLBACK_CLOCK_SKEW=5m
PROVIDER_REQUEST_TIMEOUT=5s PROVIDER_REQUEST_TIMEOUT=5s
PROVIDER_FAILURE_THRESHOLD=3 PROVIDER_FAILURE_THRESHOLD=3
PROVIDER_CIRCUIT_COOLDOWN=1m PROVIDER_CIRCUIT_COOLDOWN=1m
@@ -104,3 +106,18 @@ ARK_CONTENT_MODEL=
ARK_REQUEST_TIMEOUT=30s ARK_REQUEST_TIMEOUT=30s
EMBEDDING_PROVIDER=local EMBEDDING_PROVIDER=local
ARK_EMBEDDING_MODEL= ARK_EMBEDDING_MODEL=
# 首次认证与绑定(默认关闭;详见docs/onboarding-implementation.md)
ONBOARDING_ENABLED=false
ONBOARDING_PAUSE_CLAIMS=false
ONBOARDING_ACTIVATION_ENABLED=false
ONBOARDING_SMS_LOGIN_ENABLED=false
ONBOARDING_ALLOW_LOCAL_HTTP=false
ONBOARDING_MASTER_KEY=
ONBOARDING_MASTER_KEY_FILE=
ONBOARDING_ENTRY_URL=
ONBOARDING_TRUSTED_PROXY_CIDRS=
ONBOARDING_SELF_SIGNUP_TENANT_IDS=
ONBOARDING_SMS_GATEWAY_URL=
ONBOARDING_SMS_GATEWAY_TOKEN=
ONBOARDING_SMS_GATEWAY_TOKEN_FILE=
+5 -1
View File
@@ -42,6 +42,10 @@ jobs:
npm run build npm run build
- name: Validate compose - name: Validate compose
run: docker compose config --quiet run: docker compose config --quiet
- name: Validate offsite backup authentication and round trip
run: |
bash deploy/server/test-sync-backup-offsite.sh
python3 -B deploy/server/test_public_listener_policy.py
- name: Resolve immutable build identity - name: Resolve immutable build identity
run: | run: |
openapi_version="$(awk '/^ version:/{print $2; exit}' openapi/openapi.yaml)" openapi_version="$(awk '/^ version:/{print $2; exit}' openapi/openapi.yaml)"
@@ -81,7 +85,7 @@ jobs:
- name: Infrastructure configuration scan - name: Infrastructure configuration scan
run: >- run: >-
docker run --rm -v "$PWD:/src" aquasec/trivy:0.74.0@sha256:62b1e65e8869bc4b4c6aa4fa2b21595256c7c2f6018a9d9ad61caf87187c1969 docker run --rm -v "$PWD:/src" aquasec/trivy:0.74.0@sha256:62b1e65e8869bc4b4c6aa4fa2b21595256c7c2f6018a9d9ad61caf87187c1969
config --exit-code 1 --severity HIGH,CRITICAL /src config --ignorefile /src/.trivyignore.yaml --exit-code 1 --severity HIGH,CRITICAL /src
- name: Image vulnerability scan - name: Image vulnerability scan
run: | run: |
docker run --rm -v /var/run/docker.sock:/var/run/docker.sock aquasec/trivy:0.74.0@sha256:62b1e65e8869bc4b4c6aa4fa2b21595256c7c2f6018a9d9ad61caf87187c1969 image --exit-code 1 --ignore-unfixed --severity HIGH,CRITICAL data-terminal-cloud-backend:test docker run --rm -v /var/run/docker.sock:/var/run/docker.sock aquasec/trivy:0.74.0@sha256:62b1e65e8869bc4b4c6aa4fa2b21595256c7c2f6018a9d9ad61caf87187c1969 image --exit-code 1 --ignore-unfixed --severity HIGH,CRITICAL data-terminal-cloud-backend:test
+9
View File
@@ -2,6 +2,8 @@
.idea/ .idea/
.vscode/ .vscode/
.DS_Store .DS_Store
__pycache__/
*.py[cod]
*.log *.log
coverage.out coverage.out
coverage*.out coverage*.out
@@ -10,6 +12,7 @@ coverage*.out
*.exe *.exe
/opsconfig /opsconfig
/deploy/server/.env /deploy/server/.env
/deploy/local-rtc/runtime/
bin/ bin/
dist/ dist/
tmp/ tmp/
@@ -17,3 +20,9 @@ web/node_modules/
web/dist/ web/dist/
.gocache/ .gocache/
test/e2e/screenshots/ test/e2e/screenshots/
.design-evidence/
device-simulator-windows/story-continuity-rtc/
/.tmp-*/
/firmware/**/build/
/firmware/**/managed_components/
/firmware/**/backups/
+3 -3
View File
@@ -4,8 +4,8 @@ title = "Data terminal cloud secret scanning"
useDefault = true useDefault = true
[allowlist] [allowlist]
description = "HTTP idempotency keys are replay identifiers, not credentials" description = "Exact public HTTP idempotency fixtures are replay identifiers, not credentials"
condition = "AND" condition = "AND"
regexTarget = "line" regexTarget = "line"
regexes = ['''Idempotency-Key'''] regexes = ['''(?:test-idempotency-key-000[12]|test-activate-idempotency-0001|turns-0123456789abcdef|3ca4e514-cbde-41fc-aa32-6f1ed460cb01)''']
paths = ['''(^|/)internal/httpapi/conversation_handlers_test\.go$'''] paths = ['''(^|/)internal/httpapi/conversation_handlers_test\.go$''','''(^|/)docs/设备终端接口文档\.md$''']
+9
View File
@@ -0,0 +1,9 @@
misconfigurations:
- id: KSV-0109
paths:
- "deploy/kubernetes/base/configmap.yaml"
statement: >-
RTC_TOKEN_ISSUER and the *_TTL values are public runtime metadata, not
credentials. All RTC signing keys, provider credentials, and token key
material are mounted from Kubernetes Secrets or the production CSI
secret store.
+46
View File
@@ -0,0 +1,46 @@
# 变更记录
## 0.81.0-rc.onboarding1 — 2026-09-12
设备C01–C11按ONBOARDING-1.1文档对齐,移除辅助流程字段;专用HTTP校时返回TIME1签名包。新设备SDK执行注册,平台提供配置、30秒许可、结果保存和首通状态;未知结果、重启及跨绑定执行受保护。迁移至58,更新绑定与运维页面,保留旧设备接入。详见docs/onboarding-release-20260912.md。
本项目使用语义化版本号。正式生产放行仍以仓库内的发布候选证据和外部门禁为准;里程碑标签只表示代码、迁移、接口和本地质量门禁形成了可追溯基线。
## 未发布
- 新增首次认证、按键绑定与独立激活流程(迁移53–57、OpenAPI 0.78.0):生产登记/加密交付/放行、DEVREQ1认证、手机OIDC或短信登录、原子绑定与双端ACK、供应商任务与人工对账、首通验收、手机页和运维入口。新开关默认关闭;旧设备协议保留。详见[实施与联调说明](docs/onboarding-implementation.md)。
- 新增V51硬件智能体License池:License明文仅在平台加密库存与设备安全区中出现,平台按Instance ID/Product Key原子分配,设备校验摘要并回执后才允许`PROVISIONING → ACTIVE`;过期、吊销、丢失和解绑会立即阻断新会话。
- 新增V50设备激活状态机:`FACTORY → CLAIMABLE → BOUND → PROVISIONING → ACTIVE → LOST/REVOKED`,认领、火山凭据预配、解绑、丢失恢复和注销统一写入带修订号的迁移审计;LOST/解绑会暂停火山设备凭据,恢复后重新启用。
- 新增独立`volc_hardware_agent`运行时和`/device/v3/agent-sessions`控制面;设备每次聊天从平台取得冻结的人设、档案、近期摘要、已确认长期记忆和授权上下文,不再复用RTC房间语义。
- 新增V49平台代理火山设备注册和`/device/v3/hardware-agent-credentials:provision`:Product Secret只从平台Secret/KMS加载,一机一密在平台AES-256-GCM加密并下发对应设备,Windows端不再接收Product Secret。
- 硬件会话新增按设备Bot/传输/最低SDK绑定、完整能力快照、动态`session.updated`生效证明、单调事件水位、可靠最终字幕、平台`stop_session`动作、恢复扫描和SaaS运行时可见性。
- 不具备近端与远端双路PCM录制能力的硬件会话强制降级为仅文本或不留存;通话中记忆工具扩展到硬件会话并继续从数据库反查租户、长者和活动会话边界。
- 新增V47/V48/V49硬件运行时及凭据迁移、Windows控制面模拟接口、端到端正反向冒烟,以及带清单和SHA-256的服务器部署候选包工具。
- 维护中心新增租户级固件与OTA升级管理:可信Ed25519公钥、浏览器私有直传、大小与SHA-256校验、发布通道、强制升级标记、兼容设备灰度批次、可靠升级命令、暂停/恢复/退役和审计下载地址。
- 固件制品支持维护端租户级范围,同时保留原家庭端记录和设备受控下载链路;设备资产列表显示最近上报的当前固件版本。
- 聊天总结改为大模型两级内容价值判断:先判断是否值得保留,再判断是否值得形成故事;纯寒暄不再生成摘要或故事。
- 新增内容价值判断记录,保留模型、提示词版本、置信度和舍弃原因,支持任务幂等与隐私删除。
- 本地开发新增 GPU Ollama/Qwen2.5 7B 部署方式,并通过大模型管理的测试、激活和 worker 生成链路完成端到端验证。
- 修复大模型“测试连接”更新语句和审计结果枚举错误,测试通过的模型现在可以正常激活。
- 维护端默认人设升级为老人陪伴对话规范,覆盖倾听、单问题追问、沉默与重复处理、事实边界、敏感话题和医疗安全。
- 家庭端长者档案新增日常陪伴、回忆往事、兴趣聊天三个模板,并补充兴趣、优先话题和避免话题的创建入口。
- Agent固定评测加强单问题追问、先回应再追问、沉默不施压、尊重重复表达和情绪优先回应检查。
## v0.70.0-rc.1 — 第一阶段里程碑
第一阶段把家庭陪伴业务、真实语音接入基础能力和运营维护能力整理为第一个可运行候选版本。
主要内容:
- 家庭端、维护端和Windows语音终端模拟器形成完整入口。
- 长者档案、分项授权、设备绑定、通话记录、故事、已确认记忆和隐私控制形成闭环。
- 火山RTC/豆包语音、方舟内容摘要、通话内记忆工具和供应商记忆适配具备受控配置入口。
- 运营配置、Agent质量证据、内容质量、设备维护、费用额度和诊断能力形成版本化管理闭环。
- 数据库迁移整理至V42,并把客户账号与运营账号彻底隔离。
- 本地质量门禁覆盖Go格式、静态检查、单元测试、隔离数据库集成测试、OpenAPI类型生成、Web类型检查与构建、Compose配置校验。
已知边界:
- 该标签是发布候选里程碑,不代表生产外部证据门禁已经通过。
- 真实终端、真实供应商、Safari、云基础设施和试点运行仍按对应证据文档验收。
+28 -6
View File
@@ -2,6 +2,10 @@
这是依据《数据采集终端—云端系统研发需求规格说明书》建立的正式产品工程。当前已完成工程框架、公共能力、本地模拟语音闭环、家庭用户SaaS核心业务、字幕到人工确认记忆闭环,以及隐私、可靠设备命令、签名固件灰度、双轨录音、认证SSE状态事件流、具体资源分享、运营配置灰度发布、用量费用额度、告警处置、集成诊断和敏感数据临时提权闭环。 这是依据《数据采集终端—云端系统研发需求规格说明书》建立的正式产品工程。当前已完成工程框架、公共能力、本地模拟语音闭环、家庭用户SaaS核心业务、字幕到人工确认记忆闭环,以及隐私、可靠设备命令、签名固件灰度、双轨录音、认证SSE状态事件流、具体资源分享、运营配置灰度发布、用量费用额度、告警处置、集成诊断和敏感数据临时提权闭环。
当前代码基线为`v0.70.0-rc.1`第一阶段里程碑。版本范围、主要能力和生产放行边界见[变更记录](CHANGELOG.md)。
老人陪伴聊天的人设、长者档案填写、灰度发布和验收方法见[《老人陪伴聊天配置与发布》](docs/elder-conversation-guidance.md)。
## 当前已实现 ## 当前已实现
- Go模块化单体的`api`、`callbacks`和`worker`三个进程入口。 - Go模块化单体的`api`、`callbacks`和`worker`三个进程入口。
@@ -34,16 +38,20 @@
- 角色化业务导航:家庭用户只加载家庭业务,运营角色可按需切换运行配置、费用额度、告警处置和联调诊断,避免无权限页面与全部工作台同时加载。 - 角色化业务导航:家庭用户只加载家庭业务,运营角色可按需切换运行配置、费用额度、告警处置和联调诊断,避免无权限页面与全部工作台同时加载。
- 家庭成员协作:成员列表、一次性邀请/撤销/接受、跨客户组织受邀账号映射、角色变更、成员移除和家庭所有权转移。 - 家庭成员协作:成员列表、一次性邀请/撤销/接受、跨客户组织受邀账号映射、角色变更、成员移除和家庭所有权转移。
- 通话查询API:长者维度游标分页、状态过滤、脱敏详情、冻结快照摘要和完整状态历史。 - 通话查询API:长者维度游标分页、状态过滤、脱敏详情、冻结快照摘要和完整状态历史。
- 长者扩展档案、稳定事实、授权记录、设备绑定、最终话轮、摘要、故事版本和已确认记忆数据库模型;V8迁移已完成空库升降级验证。 - 长者扩展档案、稳定事实、授权记录、设备绑定、最终话轮、摘要、故事版本、已确认记忆、通话内记忆工具和内容质量反馈数据库模型;迁移已覆盖到V41并执行隔离数据库升降级验证。
- 模型内容候选化:只使用可追溯长者话轮生成候选;人工确认和当前有效记忆授权共同控制长期记忆写入。 - 模型内容候选化:只使用可追溯长者话轮生成候选;人工确认和当前有效记忆授权共同控制长期记忆写入。
- 可切换内容生成器:本地使用确定性适配器,生产可通过火山方舟OpenAI兼容接口生成结构化摘要、开放话题、故事和事实候选;模型网络调用不占用数据库事务,写入前重新核对通话状态、授权和话轮指纹,敏感内容在本地拦截后不发送外部模型。 - 可切换内容生成器:本地使用确定性适配器,生产可通过火山方舟OpenAI兼容接口生成结构化摘要、开放话题、故事和事实候选;模型网络调用不占用数据库事务,写入前重新核对通话状态、授权和话轮指纹,敏感内容在本地拦截后不发送外部模型。
- 新通话在数据库层按租户、长者、当前授权、状态、敏感级别、有效期和索引状态检索已确认记忆,并把实际版本冻结到背景快照。 - 新通话在数据库层按租户、长者、当前授权、状态、敏感级别、有效期和索引状态检索已确认记忆,并把实际版本冻结到背景快照。
- 在`story_extraction`与`personalized_memory`均有效时,新通话还会按同租户同长者带入最近3次已完成通话摘要;它是可撤回的短期连续上下文,不会绕过人工审核成为长期记忆。长者档案可配置最多8个“避免主动提及”话题,并从下一次新通话起随上下文快照生效。
- 可切换记忆向量:本地使用1536维确定性向量,生产可通过火山方舟OpenAI兼容`/embeddings`接口完成索引和查询;两端共用同一模型及维度,外部调用均位于数据库事务外,查询服务临时不可用时通话可无记忆降级启动。 - 可切换记忆向量:本地使用1536维确定性向量,生产可通过火山方舟OpenAI兼容`/embeddings`接口完成索引和查询;两端共用同一模型及维度,外部调用均位于数据库事务外,查询服务临时不可用时通话可无记忆降级启动。
- 通话中受控记忆工具:平台为固定通话和供应商任务签发短期令牌,服务端从数据库反查租户与长者边界;模型不能自行选择查询范围,检索失败返回空结果且不打断语音通话,实际使用按版本留痕。
- 可选供应商原生记忆:数据库始终是主数据,`managed_http`适配器通过可靠任务同步;修订、停用、授权撤回、单条删除和主体隐私删除均清理外部引用并对账,未完成真实账号验证前默认关闭。
- 细粒度内容审核与维护质量页:故事、单条事实和已确认记忆支持九类标准问题反馈;不可覆盖版本和最小质量信号按模型/提示词版本聚合到独立`/maintenance`维护端,不展示私密正文。
- 隐私数据访问请求:所有者幂等提交、Outbox/Worker子任务、八类数据范围盘点、无正文结果和审计。 - 隐私数据访问请求:所有者幂等提交、Outbox/Worker子任务、八类数据范围盘点、无正文结果和审计。
- 私有数据导出:按长者和分类生成JSON,MinIO私有存储、SHA-256校验、短期签名下载和过期对象清理。 - 私有数据导出:按长者和分类生成JSON,MinIO私有存储、SHA-256校验、短期签名下载和过期对象清理。
- 主体级隐私删除:精确称呼确认、活动通话保护、立即冻结读取/上传/召回,以及PostgreSQL、Redis、对象存储、pgvector、外部索引和私有导出六子系统的可重试对账与最小删除证明。 - 主体级隐私删除:精确称呼确认、活动通话保护、立即冻结读取/上传/召回,以及PostgreSQL、Redis、对象存储、pgvector、外部索引和私有导出六子系统的可重试对账与最小删除证明。
- 设备命令可靠链路:家庭端幂等排队、设备HMAC主动拉取、重复投递计数、确认/成功/失败回执和过期终态;Web设备卡片可下发诊断命令。 - 设备命令可靠链路:家庭端幂等排队、设备HMAC主动拉取、重复投递计数、确认/成功/失败回执和过期终态;Web设备卡片可下发诊断命令。
- 固件安全发布与管理页:Ed25519受信公钥、浏览器私有直传、大小与SHA-256校验、签名清单验证、确定性灰度、可靠命令、短期下载、暂停、已验证版本回滚,以及逐设备下载/安装/确认进度。 - 家庭端及维护中心固件安全发布与OTA管理:租户级/家庭级Ed25519受信公钥、浏览器私有直传、大小与SHA-256校验、签名清单验证、兼容设备确定性灰度、可靠命令、设备与维护人员短期下载、暂停、已验证版本回滚,以及逐设备下载/安装/确认进度。
- 双轨录音媒体链路:near/far分片清单幂等、15分钟单对象直传、Outbox可靠校验、WAV PCM16容器/时长/大小/SHA-256核验、延迟删除本地密文及设备回执,以及PLAY/DOWNLOAD独立授权、授权列表/撤销、React双轨播放下载、5分钟私有访问和成功/拒绝审计。 - 双轨录音媒体链路:near/far分片清单幂等、15分钟单对象直传、Outbox可靠校验、WAV PCM16容器/时长/大小/SHA-256核验、延迟删除本地密文及设备回执,以及PLAY/DOWNLOAD独立授权、授权列表/撤销、React双轨播放下载、5分钟私有访问和成功/拒绝审计。
- 联调问题闭环:按客户组织记录问题现象、严重级别、设备/通话/请求编号、责任方、根因、修复版本和脱敏回归证据;未关闭HIGH/CRITICAL形成发布阻断汇总,创建幂等、更新强ETag并全程审计。 - 联调问题闭环:按客户组织记录问题现象、严重级别、设备/通话/请求编号、责任方、根因、修复版本和脱敏回归证据;未关闭HIGH/CRITICAL形成发布阻断汇总,创建幂等、更新强ETag并全程审计。
- 认证SSE状态事件流:业务事务内生成最小事件,15分钟`Last-Event-ID`续传、活动家庭成员过滤、跨租户游标拒绝、重复事件抑制、设备上线去重/超时离线、React自动重连和2秒合并刷新,避免连续事件放大为整页请求风暴。 - 认证SSE状态事件流:业务事务内生成最小事件,15分钟`Last-Event-ID`续传、活动家庭成员过滤、跨租户游标拒绝、重复事件抑制、设备上线去重/超时离线、React自动重连和2秒合并刷新,避免连续事件放大为整页请求风暴。
@@ -103,13 +111,17 @@ Set-Location E:\Codex\story\platform
Web也允许通过`http://127.0.0.1:3100`访问;本地Keycloak客户端为`localhost`和`127.0.0.1`分别登记了3100生产预览端口与5173前端开发端口,登录回调必须保持当前浏览器来源一致。 Web也允许通过`http://127.0.0.1:3100`访问;本地Keycloak客户端为`localhost`和`127.0.0.1`分别登记了3100生产预览端口与5173前端开发端口,登录回调必须保持当前浏览器来源一致。
本地Web点击“登录研发环境”后可使用`owner`、`editor`或`outsider`三个合成账号,密码均为`LocalOnly123!`。账号只用于本机联调,权限说明见[家庭角色权限矩阵](docs/permission-matrix.md)。 本地家庭端可使用`owner`、`editor`或`outsider`三个合成客户账号,密码均为`LocalOnly123!`。维护中心使用完全独立、且没有家庭成员关系的`operator`(`ops_admin`)或`operator-approver`(`ops_approver`),本地密码均为`OpsLocalOnly123!`。数据库通过`user_account.account_kind`约束客户账号只能是`customer`角色、运维账号只能是运营角色,禁止同一账号同时进入家庭端和维护端。账号只用于本机联调。
维护中心地址为`/maintenance`。其中“设备管理”允许独立运维操作员查看客户组织内全部设备及当前家庭/长者归属,登记或修改设备、调整归属、隔离或软删除设备,并在创建或轮换时自动下载仅显示一次的随机设备密钥。删除会吊销设备和密钥、关闭当前绑定,但保留历史通话与审计记录。
Web镜像在容器启动时根据`WEB_OIDC_AUTHORITY`和`WEB_OIDC_CLIENT_ID`生成不缓存的`runtime-config.js`。dev、preprod和prod可复用同一镜像,只需为各环境设置对应OIDC地址和客户端编号;这些值是浏览器可见的公开配置,不能用于保存客户端密钥。 Web镜像在容器启动时根据`WEB_OIDC_AUTHORITY`和`WEB_OIDC_CLIENT_ID`生成不缓存的`runtime-config.js`。dev、preprod和prod可复用同一镜像,只需为各环境设置对应OIDC地址和客户端编号;这些值是浏览器可见的公开配置,不能用于保存客户端密钥。
真实火山RTC模式由两组独立开关启用:API设置`RTC_TOKEN_ISSUER=volcengine`、`RTC_APP_ID`和`RTC_APP_KEY`,Worker设置`VOICE_PROVIDER=volcengine`、`VOLC_ACCESS_KEY_ID`、`VOLC_SECRET_ACCESS_KEY`、区域、接口版本以及`VOLC_VOICE_CHAT_TEMPLATE_FILE`。生产模板采用`voice-chat-template-bundle-v1`,分别提供`s2s_pure`、`s2s_hybrid`和`cascade`三个完整VoiceChat `Config/AgentConfig`;平台按通话冻结模式选择模板,再覆盖AppId、RoomId、TaskId、目标设备用户、智能体用户,并追加本次已授权且脱敏的背景摘要。可从[模板示例](deploy/volcengine/voice-chat-template.example.json)复制结构并按照已开通的官方能力替换每个完整`Config`对象;不要把AppKey、AK/SK、ASR/TTS Token提交到仓库。 真实火山RTC模式由两组独立开关启用:API设置`RTC_TOKEN_ISSUER=volcengine`、`RTC_APP_ID`和`RTC_APP_KEY`,Worker设置`VOICE_PROVIDER=volcengine`、`VOLC_ACCESS_KEY_ID`、`VOLC_SECRET_ACCESS_KEY`、区域、接口版本以及`VOLC_VOICE_CHAT_TEMPLATE_FILE`。生产模板采用`voice-chat-template-bundle-v1`,分别提供`s2s_pure`、`s2s_hybrid`和`cascade`三个完整VoiceChat `Config/AgentConfig`;平台按通话冻结模式选择模板,再覆盖AppId、RoomId、TaskId、目标设备用户、智能体用户,并追加本次已授权且脱敏的背景摘要。级联模式写入`LLMConfig.SystemMessages`,纯端到端模式写入`S2SConfig.ProviderParams.dialog.system_role`。可从[模板示例](deploy/volcengine/voice-chat-template.example.json)复制结构并按照已开通的官方能力替换每个完整`Config`对象;不要把AppKey、AK/SK、ASR/TTS Token提交到仓库。
生产API必须设置`DEVICE_SECRET_DIRECTORY`为容器内绝对路径,并把Docker/Kubernetes Secret或云KMS CSI卷只读挂载到该目录。每台设备的`credential_ref`写为`device-secret://terminal-001`,对应文件名为`terminal-001`,文件内容为至少32字节的独立随机密钥。解析器拒绝相对路径、子目录、越界符号、缺失文件和短密钥;每次验签重新读取文件,因此密钥文件原子轮换后立即生效。`DEVICE_LOCAL_SECRET`只允许本地模拟使用。 火山硬件对话智能体使用独立的`volc_hardware_agent`运行时。Instance ID、Product Key、默认Bot ID、`high_quality`/`low_load`传输方式和最低SDK版本是平台持有的产品级配置,创建设备时只选择运行时,不再填写Bot。设备首次调用`/device/v3/hardware-agent-credentials:provision`时,平台自动建立该产品默认绑定,并使用KMS/Secret中的Product Secret代设备调用火山DynamicRegister;火山从产品额度中分配License并返回本设备的一机一密,License本身不下发设备。设备安全保存Device Secret后进入`ACTIVE`。每次聊天再调用`/device/v3/agent-sessions`取得默认Bot和本次冻结的人设、近期摘要、长期记忆与授权上下文,并通过火山SDK或标准WebSocket动态注入。平台只有在设备证明`session.updated`已应用且Bot、传输方式、配置摘要完全匹配后才激活通话;设备不具备近/远端双路PCM能力时,本次留存自动降级为仅文本。设备必须持续上报v3事件和最终字幕,并执行事件响应中的`stop_session`。完整时序、重试和凭据边界见[设备集成说明](docs/device-integration.md#115-火山硬件对话智能体-v3-控制面)。
设备HMAC密钥保存在`device_credential`表中,并使用AES-256-GCM和每条记录独立随机nonce加密;数据库不保存明文。API通过`DEVICE_CREDENTIAL_MASTER_KEY`(64位十六进制、解码后32字节)解密验签,该主密钥必须由Docker/Kubernetes Secret或云KMS CSI独立托管,不能进入数据库。`DEVICE_LOCAL_SECRET`只用于把既有演示设备凭据一次性导入数据库,完成迁移后可从运行环境移除。
数据库连接串、Redis、RTC AppKey、火山AK/SK、方舟API Key、对象存储凭据、回调/告警签名等敏感配置同时支持`NAME_FILE=/run/secrets/name`。文件路径必须是容器内绝对路径、目标必须是普通文件且不超过64KiB;读取时只去掉末尾换行,不在错误中打印内容。`NAME`与`NAME_FILE`同时存在会拒绝启动,防止轮换时仍意外使用旧的明文环境变量。 数据库连接串、Redis、RTC AppKey、火山AK/SK、方舟API Key、对象存储凭据、回调/告警签名等敏感配置同时支持`NAME_FILE=/run/secrets/name`。文件路径必须是容器内绝对路径、目标必须是普通文件且不超过64KiB;读取时只去掉末尾换行,不在错误中打印内容。`NAME`与`NAME_FILE`同时存在会拒绝启动,防止轮换时仍意外使用旧的明文环境变量。
@@ -185,6 +197,15 @@ docker compose --profile simulation run --rm sim-device
docker compose --profile simulation run --rm -e SIM_SCENARIO=offline,conversation,transcript,telemetry,commands sim-device docker compose --profile simulation run --rm -e SIM_SCENARIO=offline,conversation,transcript,telemetry,commands sim-device
``` ```
在Windows本机使用真实麦克风、扬声器和火山RTC/豆包端到端语音进行联调:
```powershell
.\deploy\local-rtc\configure.ps1
.\deploy\local-rtc\start.ps1
```
然后运行`device-simulator-windows/release/voice-device-simulator-0.1.0-windows-x64.exe`。密钥配置、纯端到端首轮联调范围和操作步骤见[Windows本地真实RTC联调说明](deploy/local-rtc/README.md)。
还可用`SIM_SCENARIO=media`和`SIM_CONVERSATION_ID`验证已授权通话的录音直传、后台校验、延迟删除及设备删除回执;详见`docs/device-integration.md`。 还可用`SIM_SCENARIO=media`和`SIM_CONVERSATION_ID`验证已授权通话的录音直传、后台校验、延迟删除及设备删除回执;详见`docs/device-integration.md`。
执行设备状态、错误和命令结果的批量遥测、幂等、事件判重及冲突回滚烟测: 执行设备状态、错误和命令结果的批量遥测、幂等、事件判重及冲突回滚烟测:
@@ -326,13 +347,14 @@ internal/ 平台公共能力及后续领域模块
migrations/ 仅追加的数据库迁移 migrations/ 仅追加的数据库迁移
openapi/ 对外接口契约基线 openapi/ 对外接口契约基线
web/ 家庭用户和运营网页 web/ 家庭用户和运营网页
device-simulator-windows/ Windows真实音频终端模拟器
deploy/ 容器构建与运行配置 deploy/ 容器构建与运行配置
docs/ 架构决策与研发基线 docs/ 架构决策与研发基线
scripts/ 本地启动和质量检查脚本 scripts/ 本地启动和质量检查脚本
test/ 契约、集成和端到端测试 test/ 契约、集成和端到端测试
``` ```
公共接口约束与监控指标见[可观测性与接口治理设计](docs/observability-and-api-governance.md),设备签名和模拟联调见[设备接入与启动配置联调说明](docs/device-integration.md),设备命令见[设备命令可靠下发说明](docs/device-commands.md),固件流程见[固件制品、签名、灰度与回滚开发说明](docs/firmware-rollout.md),A/B升级与启动失败回退见[设备A/B分区固件升级与启动确认协议](docs/device-firmware-ab-protocol.md),设备/云端时间线对账见[固件真机联调一致性证据采集与校验说明](docs/firmware-conformance-evidence.md),双轨录音见[双轨录音上传、校验与访问开发说明](docs/media-upload-and-access.md),网页状态通知见[Web状态事件流开发说明](docs/web-event-stream.md),具体分享见[具体资源分享开发说明](docs/resource-sharing.md),运营配置见[内部运营身份与运行配置版本开发说明](docs/operations-config.md),用量费用与额度见[用量、费用与额度开发说明](docs/usage-cost-quota.md),告警闭环见[告警监测与运营处置开发说明](docs/alert-operations.md),真实网关归档见[告警接收器一致性证据](docs/alert-receiver-conformance-evidence.md),云密钥轮换见[KMS/HSM轮换一致性证据](docs/kms-hsm-rotation-evidence.md),生产恢复见[生产等价恢复演练一致性证据](docs/production-recovery-evidence.md),生产基础设施见[生产基础设施一致性证据](docs/production-infrastructure-evidence.md),联调工作台及发布候选验收快照见[集成诊断工作台开发说明](docs/integration-diagnostics.md),试点每日证据见[试点运行日报开发说明](docs/pilot-daily-operations.md),试点周度复盘口径、外部投诉缺口、接口和轻量烟测见[试点运行周报开发说明](docs/pilot-weekly-review.md),敏感数据提权见[运营敏感数据临时访问开发说明](docs/ops-sensitive-access.md),外部上线阶段见[真机、真实供应商、试点与生产发布外部验收清单](docs/external-acceptance-checklist.md),生产门禁和压测告警见[压测、告警与生产安全门禁](docs/load-alerting-and-production-security.md),专项实测证据见[性能、恢复、安全与兼容性专项验证报告](docs/performance-recovery-security-compatibility.md),隐私数据访问见[隐私数据请求开发说明](docs/privacy-requests.md),私有导出见[私有数据导出开发说明](docs/data-export.md)。 公共接口约束与监控指标见[可观测性与接口治理设计](docs/observability-and-api-governance.md),设备签名和模拟联调见[设备接入与启动配置联调说明](docs/device-integration.md),设备命令见[设备命令可靠下发说明](docs/device-commands.md),固件流程见[固件制品、签名、灰度与回滚开发说明](docs/firmware-rollout.md),A/B升级与启动失败回退见[设备A/B分区固件升级与启动确认协议](docs/device-firmware-ab-protocol.md),设备/云端时间线对账见[固件真机联调一致性证据采集与校验说明](docs/firmware-conformance-evidence.md),双轨录音见[双轨录音上传、校验与访问开发说明](docs/media-upload-and-access.md),网页状态通知见[Web状态事件流开发说明](docs/web-event-stream.md),具体分享见[具体资源分享开发说明](docs/resource-sharing.md),运营配置见[内部运营身份与运行配置版本开发说明](docs/operations-config.md),用量费用与额度见[用量、费用与额度开发说明](docs/usage-cost-quota.md),告警闭环见[告警监测与运营处置开发说明](docs/alert-operations.md),真实网关归档见[告警接收器一致性证据](docs/alert-receiver-conformance-evidence.md),云KMS轮换见[KMS/HSM轮换一致性证据](docs/kms-hsm-rotation-evidence.md),生产恢复见[生产等价恢复演练一致性证据](docs/production-recovery-evidence.md),单机云端备份见[自动备份与隔离恢复手册](docs/server-backup-and-restore.md),OSS异地备份见[阿里云OSS异地备份适配与门禁](docs/offsite-backup.md),服务器健康检查见[单机云端健康监测与告警手册](docs/server-health-monitoring.md),生产基础设施见[生产基础设施一致性证据](docs/production-infrastructure-evidence.md),联调工作台及发布候选验收快照见[集成诊断工作台开发说明](docs/integration-diagnostics.md),试点每日证据见[试点运行日报开发说明](docs/pilot-daily-operations.md),试点周度复盘口径、外部投诉缺口、接口和轻量烟测见[试点运行周报开发说明](docs/pilot-weekly-review.md),敏感数据提权见[运营敏感数据临时访问开发说明](docs/ops-sensitive-access.md),外部上线阶段见[真机、真实供应商、试点与生产发布外部验收清单](docs/external-acceptance-checklist.md),生产门禁和压测告警见[压测、告警与生产安全门禁](docs/load-alerting-and-production-security.md),专项实测证据见[性能、恢复、安全与兼容性专项验证报告](docs/performance-recovery-security-compatibility.md),隐私数据访问见[隐私数据请求开发说明](docs/privacy-requests.md),私有导出见[私有数据导出开发说明](docs/data-export.md)。
从框架搭建到生产落地的阶段任务、里程碑、人员、测试门禁、依赖和风险见[详细开发计划](docs/development-plan.md),当前实际完成证据见[开发状态与门禁记录](docs/development-status.md)。 从框架搭建到生产落地的阶段任务、里程碑、人员、测试门禁、依赖和风险见[详细开发计划](docs/development-plan.md),当前实际完成证据见[开发状态与门禁记录](docs/development-status.md)。
+85
View File
@@ -0,0 +1,85 @@
package main
import (
"context"
"crypto/rand"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"os"
"strings"
"data-terminal-cloud/internal/deviceauth"
"github.com/google/uuid"
"github.com/jackc/pgx/v5/pgxpool"
)
func main() {
if err := run(context.Background(), os.Args[1:]); err != nil {
fmt.Fprintln(os.Stderr, err)
os.Exit(1)
}
}
func run(ctx context.Context, args []string) error {
if len(args) != 2 || (args[0] != "provision" && args[0] != "rotate" && args[0] != "revoke") {
return errors.New("usage: device-credential <provision|rotate|revoke> <device-public-uuid>")
}
if _, err := uuid.Parse(args[1]); err != nil {
return errors.New("device public id must be a UUID")
}
databaseURL, err := secretEnvironment("DATABASE_URL")
if err != nil {
return err
}
masterKey, err := secretEnvironment("DEVICE_CREDENTIAL_MASTER_KEY")
if err != nil {
return err
}
pool, err := pgxpool.New(ctx, databaseURL)
if err != nil {
return fmt.Errorf("open database: %w", err)
}
defer pool.Close()
resolver, err := deviceauth.NewDatabaseSecretResolver(pool, masterKey)
if err != nil {
return err
}
if args[0] == "revoke" {
if err := resolver.Revoke(ctx, args[1]); err != nil {
return err
}
return json.NewEncoder(os.Stdout).Encode(map[string]any{"device_id": args[1], "status": "revoked"})
}
random := make([]byte, 32)
if _, err := rand.Read(random); err != nil {
return fmt.Errorf("generate device credential: %w", err)
}
secret := hex.EncodeToString(random)
version, err := resolver.Provision(ctx, args[1], []byte(secret), args[0] == "rotate")
if err != nil {
return err
}
fmt.Fprintln(os.Stderr, "device credential plaintext is shown once; transfer it through a controlled channel")
return json.NewEncoder(os.Stdout).Encode(map[string]any{
"device_id": args[1], "key_version": version, "secret": secret,
})
}
func secretEnvironment(name string) (string, error) {
if file := strings.TrimSpace(os.Getenv(name + "_FILE")); file != "" {
value, err := os.ReadFile(file)
if err != nil {
return "", fmt.Errorf("read %s_FILE: %w", name, err)
}
if result := strings.TrimSpace(string(value)); result != "" {
return result, nil
}
}
if value := strings.TrimSpace(os.Getenv(name)); value != "" {
return value, nil
}
return "", fmt.Errorf("%s or %s_FILE is required", name, name)
}
+251
View File
@@ -0,0 +1,251 @@
package main
import (
"bytes"
"encoding/json"
"flag"
"fmt"
"io"
"net/http"
"net/url"
"os"
"strings"
"time"
volcbase "github.com/volcengine/volc-sdk-golang/base"
)
type responseEnvelope struct {
ResponseMetadata struct {
Error *struct {
Code string `json:"Code"`
Message string `json:"Message"`
} `json:"Error"`
} `json:"ResponseMetadata"`
}
func main() {
akFile := flag.String("ak-file", "", "file containing the Volcengine AccessKey ID")
skFile := flag.String("sk-file", "", "file containing the Volcengine SecretAccessKey")
endpoint := flag.String("endpoint", "https://rtc.volcengineapi.com", "RTC OpenAPI endpoint")
rtcAppID := flag.String("rtc-app-id", "", "RTC application ID used for staged validation probes")
templateFile := flag.String("template-file", "", "voice-chat template bundle used for ProviderParams probes")
roomID := flag.String("room-id", "", "RTC room ID to inspect without starting a task")
flag.Parse()
if *akFile == "" || *skFile == "" {
panic("-ak-file and -sk-file are required")
}
ak := readSecret(*akFile)
sk := readSecret(*skFile)
credentials := volcbase.Credentials{AccessKeyID: ak, SecretAccessKey: sk, Service: "rtc", Region: "cn-north-1"}
client := &http.Client{Timeout: 15 * time.Second}
if *rtcAppID != "" && *roomID != "" {
queryRoom(client, credentials, *endpoint, *rtcAppID, *roomID)
return
}
for _, version := range []string{"2024-06-01", "2024-12-01", "2025-06-01"} {
probe(client, credentials, *endpoint, version, "empty", map[string]any{})
}
if *rtcAppID != "" {
version := "2024-12-01"
identity := map[string]any{
"AppId": *rtcAppID, "RoomId": "probe-room", "TaskId": "probe-task",
}
agent := map[string]any{
"TargetUserId": []string{"probe-user"}, "UserId": "probe-agent",
}
probe(client, credentials, *endpoint, version, "app-only", map[string]any{"AppId": *rtcAppID})
probe(client, credentials, *endpoint, version, "identity-only", map[string]any{
"AppId": *rtcAppID, "RoomId": "probe-room", "TaskId": "probe-task",
})
probe(client, credentials, *endpoint, version, "empty-config", map[string]any{
"AppId": *rtcAppID, "RoomId": "probe-room", "TaskId": "probe-task",
"Config": map[string]any{}, "AgentConfig": map[string]any{},
})
probe(client, credentials, *endpoint, version, "agent-valid", merge(identity, map[string]any{
"Config": map[string]any{}, "AgentConfig": agent,
}))
probe(client, credentials, *endpoint, version, "s2s-shell", merge(identity, map[string]any{
"Config": map[string]any{"S2SConfig": map[string]any{}}, "AgentConfig": agent,
}))
probe(client, credentials, *endpoint, version, "s2s-provider", merge(identity, map[string]any{
"Config": map[string]any{"S2SConfig": map[string]any{
"Provider": "volcano", "OutputMode": 0,
}}, "AgentConfig": agent,
}))
probe(client, credentials, *endpoint, version, "s2s-params-empty", merge(identity, map[string]any{
"Config": map[string]any{"S2SConfig": map[string]any{
"Provider": "volcano", "OutputMode": 0, "ProviderParams": map[string]any{},
}}, "AgentConfig": agent,
}))
if *templateFile != "" {
providerParams := readProviderParams(*templateFile)
for _, item := range []struct {
label string
keys []string
}{
{label: "params-app", keys: []string{"app"}},
{label: "params-dialog", keys: []string{"dialog"}},
{label: "params-asr", keys: []string{"asr"}},
{label: "params-tts", keys: []string{"tts"}},
{label: "params-full", keys: []string{"app", "dialog", "asr", "tts"}},
} {
selected := make(map[string]any, len(item.keys))
for _, key := range item.keys {
if value, ok := providerParams[key]; ok {
selected[key] = value
}
}
probeIdentity := merge(identity, map[string]any{"TaskId": "probe-" + item.label})
payload := merge(probeIdentity, map[string]any{
"Config": map[string]any{"S2SConfig": map[string]any{
"Provider": "volcano", "OutputMode": 0, "ProviderParams": selected,
}}, "AgentConfig": agent,
})
if probeAction(client, credentials, *endpoint, version, item.label, "StartVoiceChat", payload) {
probeAction(client, credentials, *endpoint, version, item.label+"-cleanup", "StopVoiceChat", probeIdentity)
}
}
fullTemplate := readHybridTemplate(*templateFile)
fullIdentity := merge(identity, map[string]any{"TaskId": "probe-full-template"})
fullTemplate = merge(fullTemplate, fullIdentity)
fullAgent, _ := fullTemplate["AgentConfig"].(map[string]any)
fullAgent["TargetUserId"] = []string{"probe-user"}
fullAgent["UserId"] = "probe-agent"
if probeAction(client, credentials, *endpoint, version, "full-template", "StartVoiceChat", fullTemplate) {
probeAction(client, credentials, *endpoint, version, "full-template-cleanup", "StopVoiceChat", fullIdentity)
}
}
probeAction(client, credentials, *endpoint, version, "cleanup", "StopVoiceChat", identity)
}
}
func queryRoom(client *http.Client, credentials volcbase.Credentials, endpoint, appID, roomID string) {
target, err := url.Parse(endpoint)
if err != nil {
panic(err)
}
query := target.Query()
query.Set("Action", "GetRoomOnlineUsers")
query.Set("Version", "2023-11-01")
query.Set("AppId", appID)
query.Set("RoomId", roomID)
query.Set("Property", "false")
target.RawQuery = query.Encode()
request, err := http.NewRequest(http.MethodGet, target.String(), nil)
if err != nil {
panic(err)
}
request.Host = target.Host
request.Header.Set("Accept", "application/json")
credentials.Sign(request)
response, err := client.Do(request)
if err != nil {
panic(err)
}
defer response.Body.Close()
body, err := io.ReadAll(io.LimitReader(response.Body, 1<<20))
if err != nil {
panic(err)
}
var decoded map[string]any
if err := json.Unmarshal(body, &decoded); err != nil {
panic(err)
}
encoded, _ := json.Marshal(decoded)
fmt.Printf("room-status=%d response=%s\n", response.StatusCode, encoded)
}
func readProviderParams(path string) map[string]any {
template := readHybridTemplate(path)
config, _ := template["Config"].(map[string]any)
s2s, _ := config["S2SConfig"].(map[string]any)
params, _ := s2s["ProviderParams"].(map[string]any)
if params == nil {
panic("template does not contain s2s_hybrid Config.S2SConfig.ProviderParams")
}
return params
}
func readHybridTemplate(path string) map[string]any {
raw, err := os.ReadFile(path)
if err != nil {
panic(err)
}
var bundle map[string]any
if err := json.Unmarshal(raw, &bundle); err != nil {
panic(err)
}
templates, _ := bundle["templates"].(map[string]any)
template, _ := templates["s2s_hybrid"].(map[string]any)
if template == nil {
panic("template does not contain s2s_hybrid")
}
return template
}
func merge(base map[string]any, extra map[string]any) map[string]any {
result := make(map[string]any, len(base)+len(extra))
for key, value := range base {
result[key] = value
}
for key, value := range extra {
result[key] = value
}
return result
}
func readSecret(path string) string {
value, err := os.ReadFile(path)
if err != nil {
panic(err)
}
return strings.TrimSpace(string(value))
}
func probe(client *http.Client, credentials volcbase.Credentials, endpoint, version, label string, payload map[string]any) {
probeAction(client, credentials, endpoint, version, label, "StartVoiceChat", payload)
}
func probeAction(client *http.Client, credentials volcbase.Credentials, endpoint, version, label, action string, payload map[string]any) bool {
target, err := url.Parse(endpoint)
if err != nil {
panic(err)
}
query := target.Query()
query.Set("Action", action)
query.Set("Version", version)
target.RawQuery = query.Encode()
bodyBytes, err := json.Marshal(payload)
if err != nil {
panic(err)
}
request, err := http.NewRequest(http.MethodPost, target.String(), bytes.NewReader(bodyBytes))
if err != nil {
panic(err)
}
request.Host = target.Host
request.Header.Set("Content-Type", "application/json")
request.Header.Set("Accept", "application/json")
credentials.Sign(request)
response, err := client.Do(request)
if err != nil {
fmt.Printf("label=%s version=%s transport_error=%v\n", label, version, err)
return false
}
defer response.Body.Close()
body, err := io.ReadAll(io.LimitReader(response.Body, 1<<20))
if err != nil {
fmt.Printf("label=%s version=%s status=%d read_error=%v\n", label, version, response.StatusCode, err)
return false
}
var envelope responseEnvelope
_ = json.Unmarshal(body, &envelope)
if envelope.ResponseMetadata.Error == nil {
fmt.Printf("label=%s version=%s status=%d code=none\n", label, version, response.StatusCode)
return response.StatusCode >= 200 && response.StatusCode < 300
}
fmt.Printf("label=%s version=%s status=%d code=%s message=%s\n", label, version, response.StatusCode,
envelope.ResponseMetadata.Error.Code, envelope.ResponseMetadata.Error.Message)
return false
}
+67
View File
@@ -0,0 +1,67 @@
package main
import (
"bytes"
"encoding/json"
"flag"
"fmt"
"io"
"net/http"
"net/url"
"os"
"strings"
"time"
volcbase "github.com/volcengine/volc-sdk-golang/base"
)
func main() {
akFile := flag.String("ak-file", "", "file containing the Volcengine AccessKey ID")
skFile := flag.String("sk-file", "", "file containing the Volcengine SecretAccessKey")
resourceID := flag.String("resource-id", "volc.speech.dialog", "speech service resource ID")
projectName := flag.String("project", "default", "Volcengine project name")
flag.Parse()
if *akFile == "" || *skFile == "" {
panic("-ak-file and -sk-file are required")
}
target, _ := url.Parse("https://open.volcengineapi.com")
query := target.Query()
query.Set("Action", "QuotaMonitoring")
query.Set("Version", "2025-05-21")
target.RawQuery = query.Encode()
today := time.Now().Format("2006-01-02")
body, _ := json.Marshal(map[string]any{
"ProjectName": *projectName,
"ResourceID": *resourceID,
"Start": today,
"End": today,
"Mode": "daily",
"QuotaType": "concurrency",
})
request, _ := http.NewRequest(http.MethodPost, target.String(), bytes.NewReader(body))
request.Host = target.Host
request.Header.Set("Content-Type", "application/json; charset=UTF-8")
credentials := volcbase.Credentials{
AccessKeyID: readSecret(*akFile), SecretAccessKey: readSecret(*skFile),
Service: "speech_saas_prod", Region: "cn-beijing",
}
credentials.Sign(request)
response, err := (&http.Client{Timeout: 15 * time.Second}).Do(request)
if err != nil {
panic(err)
}
defer response.Body.Close()
responseBody, _ := io.ReadAll(io.LimitReader(response.Body, 1<<20))
var decoded map[string]any
_ = json.Unmarshal(responseBody, &decoded)
encoded, _ := json.Marshal(decoded)
fmt.Printf("status=%d response=%s\n", response.StatusCode, encoded)
}
func readSecret(path string) string {
value, err := os.ReadFile(path)
if err != nil {
panic(err)
}
return strings.TrimSpace(string(value))
}
+70 -2
View File
@@ -1,6 +1,24 @@
name: data-terminal-cloud name: data-terminal-cloud
x-backend-environment: &backend-environment x-backend-environment: &backend-environment
# First onboarding is opt-in; existing devices retain the legacy protocol.
ONBOARDING_ENABLED: ${ONBOARDING_ENABLED:-false}
ONBOARDING_PAUSE_CLAIMS: ${ONBOARDING_PAUSE_CLAIMS:-false}
ONBOARDING_ACTIVATION_ENABLED: ${ONBOARDING_ACTIVATION_ENABLED:-false}
ONBOARDING_SMS_LOGIN_ENABLED: ${ONBOARDING_SMS_LOGIN_ENABLED:-false}
ONBOARDING_ALLOW_LOCAL_HTTP: ${ONBOARDING_ALLOW_LOCAL_HTTP:-false}
ONBOARDING_MASTER_KEY: ${ONBOARDING_MASTER_KEY:-}
ONBOARDING_ENTRY_URL: ${ONBOARDING_ENTRY_URL:-}
ONBOARDING_TRUSTED_PROXY_CIDRS: ${ONBOARDING_TRUSTED_PROXY_CIDRS:-}
ONBOARDING_SELF_SIGNUP_TENANT_IDS: ${ONBOARDING_SELF_SIGNUP_TENANT_IDS:-}
ONBOARDING_SMS_GATEWAY_URL: ${ONBOARDING_SMS_GATEWAY_URL:-}
ONBOARDING_SMS_GATEWAY_TOKEN: ${ONBOARDING_SMS_GATEWAY_TOKEN:-}
VOLC_HARDWARE_INSTANCE_ID: ${VOLC_HARDWARE_INSTANCE_ID:-}
VOLC_HARDWARE_PRODUCT_KEY: ${VOLC_HARDWARE_PRODUCT_KEY:-}
VOLC_HARDWARE_PRODUCT_SECRET: ${VOLC_HARDWARE_PRODUCT_SECRET:-}
VOLC_HARDWARE_DEVICE_CREDENTIAL_MASTER_KEY: ${VOLC_HARDWARE_DEVICE_CREDENTIAL_MASTER_KEY:-}
VOLC_HARDWARE_REGISTER_URL: ${VOLC_HARDWARE_REGISTER_URL:-https://iot-cn-shanghai.iot.volces.com/2021-12-14/DynamicRegister?Action=DynamicRegister&Version=2021-12-14}
VOLC_HARDWARE_REGISTER_TIMEOUT: ${VOLC_HARDWARE_REGISTER_TIMEOUT:-10s}
APP_ENV: local APP_ENV: local
LOG_LEVEL: debug LOG_LEVEL: debug
DATABASE_URL: postgres://voice:voice_dev_only@postgres:5432/voice_platform?sslmode=disable DATABASE_URL: postgres://voice:voice_dev_only@postgres:5432/voice_platform?sslmode=disable
@@ -13,7 +31,7 @@ x-backend-environment: &backend-environment
AUTH_ISSUER_URL: http://localhost:58180/realms/voice-local AUTH_ISSUER_URL: http://localhost:58180/realms/voice-local
AUTH_JWKS_URL: http://keycloak:8080/realms/voice-local/protocol/openid-connect/certs AUTH_JWKS_URL: http://keycloak:8080/realms/voice-local/protocol/openid-connect/certs
AUTH_AUDIENCE: voice-platform-web AUTH_AUDIENCE: voice-platform-web
OTEL_EXPORTER_OTLP_ENDPOINT: otel-collector:4317 OTEL_EXPORTER_OTLP_ENDPOINT: ${OTEL_EXPORTER_OTLP_ENDPOINT:-}
RATE_LIMIT_RPS: "50" RATE_LIMIT_RPS: "50"
RATE_LIMIT_BURST: "100" RATE_LIMIT_BURST: "100"
RATE_LIMIT_USER_RPS: "20" RATE_LIMIT_USER_RPS: "20"
@@ -25,8 +43,23 @@ x-backend-environment: &backend-environment
PROVIDER_START_RPS: "20" PROVIDER_START_RPS: "20"
PROVIDER_START_BURST: "40" PROVIDER_START_BURST: "40"
PROVIDER_MAX_ACTIVE_TASKS: "100" PROVIDER_MAX_ACTIVE_TASKS: "100"
MEMORY_TOOL_BASE_URL: ${MEMORY_TOOL_BASE_URL:-http://api:8080/internal/v1/tools/memory-search}
MEMORY_TOOL_SIGNING_SECRET: ${MEMORY_TOOL_SIGNING_SECRET:-local-memory-tool-signing-secret-32-bytes-minimum}
MEMORY_TOOL_TOKEN_TTL: 4h
MEMORY_TOOL_RATE_RPS: "2"
MEMORY_TOOL_RATE_BURST: "5"
VIKING_MEMORY_RESOURCE_ID: ${VIKING_MEMORY_RESOURCE_ID:-}
VIKING_MEMORY_COLLECTION_NAME: ${VIKING_MEMORY_COLLECTION_NAME:-}
VIKING_MEMORY_API_KEY: ${VIKING_MEMORY_API_KEY:-}
VIKING_MEMORY_EVENT_TYPE: ${VIKING_MEMORY_EVENT_TYPE:-event_v1}
VIKING_MEMORY_PROFILE_TYPE: ${VIKING_MEMORY_PROFILE_TYPE:-profile_v1}
PROVIDER_MEMORY_MODE: ${PROVIDER_MEMORY_MODE:-disabled}
PROVIDER_MEMORY_BASE_URL: ${PROVIDER_MEMORY_BASE_URL:-}
PROVIDER_MEMORY_API_KEY: ${PROVIDER_MEMORY_API_KEY:-}
PROVIDER_MEMORY_TIMEOUT: ${PROVIDER_MEMORY_TIMEOUT:-5s}
DEVICE_CLOCK_SKEW: 5m DEVICE_CLOCK_SKEW: 5m
DEVICE_SECRET_DIRECTORY: ${DEVICE_SECRET_DIRECTORY:-} DEVICE_CREDENTIAL_MASTER_KEY: ${DEVICE_CREDENTIAL_MASTER_KEY:-000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f}
MODEL_CREDENTIAL_MASTER_KEY: ${MODEL_CREDENTIAL_MASTER_KEY:-1f1e1d1c1b1a191817161514131211100f0e0d0c0b0a09080706050403020100}
RTC_APP_ID: local-public-rtc-app RTC_APP_ID: local-public-rtc-app
RTC_APP_KEY: ${RTC_APP_KEY:-} RTC_APP_KEY: ${RTC_APP_KEY:-}
RTC_TOKEN_ISSUER: ${RTC_TOKEN_ISSUER:-local} RTC_TOKEN_ISSUER: ${RTC_TOKEN_ISSUER:-local}
@@ -45,6 +78,9 @@ x-backend-environment: &backend-environment
VOLC_RTC_REGION: ${VOLC_RTC_REGION:-cn-north-1} VOLC_RTC_REGION: ${VOLC_RTC_REGION:-cn-north-1}
VOLC_RTC_API_VERSION: ${VOLC_RTC_API_VERSION:-2024-12-01} VOLC_RTC_API_VERSION: ${VOLC_RTC_API_VERSION:-2024-12-01}
VOLC_VOICE_CHAT_TEMPLATE_FILE: ${VOLC_VOICE_CHAT_TEMPLATE_FILE:-} VOLC_VOICE_CHAT_TEMPLATE_FILE: ${VOLC_VOICE_CHAT_TEMPLATE_FILE:-}
VOLC_HARDWARE_AGENT_BOT_ID: ${VOLC_HARDWARE_AGENT_BOT_ID:-local-hardware-agent-bot}
VOLC_HARDWARE_AGENT_TRANSPORT: ${VOLC_HARDWARE_AGENT_TRANSPORT:-high_quality}
VOLC_HARDWARE_AGENT_MINIMUM_SDK_VERSION: ${VOLC_HARDWARE_AGENT_MINIMUM_SDK_VERSION:-1.0.0}
PROVIDER_FAILURE_THRESHOLD: "3" PROVIDER_FAILURE_THRESHOLD: "3"
PROVIDER_CIRCUIT_COOLDOWN: 1m PROVIDER_CIRCUIT_COOLDOWN: 1m
RECOVERY_SCAN_INTERVAL: 5s RECOVERY_SCAN_INTERVAL: 5s
@@ -73,6 +109,7 @@ x-backend-build: &backend-build
VERSION: ${PLATFORM_BUILD_VERSION:-dev} VERSION: ${PLATFORM_BUILD_VERSION:-dev}
COMMIT: ${PLATFORM_BUILD_COMMIT:-unknown} COMMIT: ${PLATFORM_BUILD_COMMIT:-unknown}
BUILD_TIME: ${PLATFORM_BUILD_TIME:-unknown} BUILD_TIME: ${PLATFORM_BUILD_TIME:-unknown}
GOPROXY: ${GOPROXY:-https://proxy.golang.org,direct}
x-backend-dependencies: &backend-dependencies x-backend-dependencies: &backend-dependencies
postgres: postgres:
@@ -145,6 +182,26 @@ services:
volumes: volumes:
- ./deploy/keycloak:/opt/keycloak/data/import:ro - ./deploy/keycloak:/opt/keycloak/data/import:ro
ollama:
profiles: [local-model]
image: ollama/ollama:latest
ports:
- "${OLLAMA_PORT:-11434}:11434"
volumes:
- ollama-data:/root/.ollama
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: 1
capabilities: [gpu]
healthcheck:
test: ["CMD", "ollama", "list"]
interval: 10s
timeout: 5s
retries: 30
otel-collector: otel-collector:
image: otel/opentelemetry-collector-contrib:latest image: otel/opentelemetry-collector-contrib:latest
command: ["--config=/etc/otelcol-contrib/config.yaml"] command: ["--config=/etc/otelcol-contrib/config.yaml"]
@@ -209,6 +266,14 @@ services:
HTTP_ADDR: 0.0.0.0:8080 HTTP_ADDR: 0.0.0.0:8080
DEVICE_LOCAL_SECRET: local-simulated-device-secret-32-bytes-minimum DEVICE_LOCAL_SECRET: local-simulated-device-secret-32-bytes-minimum
RTC_TOKEN_SIGNING_SECRET: local-simulated-rtc-token-secret-32-bytes-minimum RTC_TOKEN_SIGNING_SECRET: local-simulated-rtc-token-secret-32-bytes-minimum
VOLC_HARDWARE_PROVISIONING_ENABLED: ${VOLC_HARDWARE_PROVISIONING_ENABLED:-false}
VOLC_HARDWARE_INSTANCE_ID: ${VOLC_HARDWARE_INSTANCE_ID:-}
VOLC_HARDWARE_PRODUCT_KEY: ${VOLC_HARDWARE_PRODUCT_KEY:-}
VOLC_HARDWARE_PRODUCT_SECRET: ${VOLC_HARDWARE_PRODUCT_SECRET:-}
VOLC_HARDWARE_DEVICE_CREDENTIAL_MASTER_KEY: ${VOLC_HARDWARE_DEVICE_CREDENTIAL_MASTER_KEY:-}
VOLC_HARDWARE_REGISTER_URL: ${VOLC_HARDWARE_REGISTER_URL:-https://iot-cn-shanghai.iot.volces.com/2021-12-14/DynamicRegister?Action=DynamicRegister&Version=2021-12-14}
VOLC_HARDWARE_REGISTER_TIMEOUT: ${VOLC_HARDWARE_REGISTER_TIMEOUT:-10s}
VOLC_HARDWARE_DEVICE_NAME_PREFIX: ${VOLC_HARDWARE_DEVICE_NAME_PREFIX:-dt-}
ports: ports:
- "${API_PORT:-58080}:8080" - "${API_PORT:-58080}:8080"
depends_on: *backend-dependencies depends_on: *backend-dependencies
@@ -220,6 +285,7 @@ services:
<<: *backend-environment <<: *backend-environment
HTTP_ADDR: 0.0.0.0:8081 HTTP_ADDR: 0.0.0.0:8081
PROVIDER_CALLBACK_SECRET: local-provider-callback-secret-32-bytes-minimum PROVIDER_CALLBACK_SECRET: local-provider-callback-secret-32-bytes-minimum
PLATFORM_HEALTH_CALLBACK_SECRET: local-platform-health-callback-secret-32-bytes-minimum
RATE_LIMIT_RPS: "1000" RATE_LIMIT_RPS: "1000"
RATE_LIMIT_BURST: "1500" RATE_LIMIT_BURST: "1500"
ports: ports:
@@ -301,6 +367,7 @@ services:
VERSION: ${PLATFORM_BUILD_VERSION:-dev} VERSION: ${PLATFORM_BUILD_VERSION:-dev}
COMMIT: ${PLATFORM_BUILD_COMMIT:-unknown} COMMIT: ${PLATFORM_BUILD_COMMIT:-unknown}
BUILD_TIME: ${PLATFORM_BUILD_TIME:-unknown} BUILD_TIME: ${PLATFORM_BUILD_TIME:-unknown}
NPM_CONFIG_REGISTRY: ${NPM_CONFIG_REGISTRY:-https://registry.npmjs.org}
environment: environment:
WEB_OIDC_AUTHORITY: ${WEB_OIDC_AUTHORITY:-http://localhost:58180/realms/voice-local} WEB_OIDC_AUTHORITY: ${WEB_OIDC_AUTHORITY:-http://localhost:58180/realms/voice-local}
WEB_OIDC_CLIENT_ID: ${WEB_OIDC_CLIENT_ID:-voice-platform-web} WEB_OIDC_CLIENT_ID: ${WEB_OIDC_CLIENT_ID:-voice-platform-web}
@@ -314,3 +381,4 @@ volumes:
postgres-data: postgres-data:
redis-data: redis-data:
minio-data: minio-data:
ollama-data:
+2
View File
@@ -1,4 +1,5 @@
FROM golang:1.26.6-alpine AS build FROM golang:1.26.6-alpine AS build
ARG GOPROXY=https://proxy.golang.org,direct
WORKDIR /src WORKDIR /src
COPY go.mod go.sum ./ COPY go.mod go.sum ./
RUN go mod download RUN go mod download
@@ -20,6 +21,7 @@ RUN CGO_ENABLED=0 go build -trimpath -ldflags="-s -w \
-X data-terminal-cloud/internal/buildinfo.Commit=${COMMIT} \ -X data-terminal-cloud/internal/buildinfo.Commit=${COMMIT} \
-X data-terminal-cloud/internal/buildinfo.BuildTime=${BUILD_TIME}" -o /out/worker ./cmd/worker && \ -X data-terminal-cloud/internal/buildinfo.BuildTime=${BUILD_TIME}" -o /out/worker ./cmd/worker && \
CGO_ENABLED=0 go build -trimpath -o /out/sim-device ./cmd/sim-device && \ CGO_ENABLED=0 go build -trimpath -o /out/sim-device ./cmd/sim-device && \
CGO_ENABLED=0 go build -trimpath -o /out/device-credential ./cmd/device-credential && \
CGO_ENABLED=0 go build -trimpath -o /out/firmware-conformance ./cmd/firmware-conformance && \ CGO_ENABLED=0 go build -trimpath -o /out/firmware-conformance ./cmd/firmware-conformance && \
CGO_ENABLED=0 go build -trimpath -o /out/alert-receiver-conformance ./cmd/alert-receiver-conformance && \ CGO_ENABLED=0 go build -trimpath -o /out/alert-receiver-conformance ./cmd/alert-receiver-conformance && \
CGO_ENABLED=0 go build -trimpath -o /out/kms-rotation-conformance ./cmd/kms-rotation-conformance && \ CGO_ENABLED=0 go build -trimpath -o /out/kms-rotation-conformance ./cmd/kms-rotation-conformance && \
+20
View File
@@ -79,6 +79,26 @@
"emailVerified": true, "emailVerified": true,
"enabled": true, "enabled": true,
"credentials": [{"type": "password", "value": "LocalOnly123!", "temporary": false}] "credentials": [{"type": "password", "value": "LocalOnly123!", "temporary": false}]
},
{
"id": "00000000-0000-0000-0000-000000000104",
"username": "operator",
"firstName": "平台",
"lastName": "运维管理员",
"email": "operator@example.invalid",
"emailVerified": true,
"enabled": true,
"credentials": [{"type": "password", "value": "OpsLocalOnly123!", "temporary": false}]
},
{
"id": "00000000-0000-0000-0000-000000000105",
"username": "operator-approver",
"firstName": "平台",
"lastName": "运维审批员",
"email": "operator-approver@example.invalid",
"emailVerified": true,
"enabled": true,
"credentials": [{"type": "password", "value": "OpsLocalOnly123!", "temporary": false}]
} }
] ]
} }
+16
View File
@@ -50,6 +50,17 @@ data:
VOLC_RTC_BASE_URL: https://rtc.volcengineapi.com VOLC_RTC_BASE_URL: https://rtc.volcengineapi.com
VOLC_RTC_REGION: cn-north-1 VOLC_RTC_REGION: cn-north-1
VOLC_RTC_API_VERSION: "2024-12-01" VOLC_RTC_API_VERSION: "2024-12-01"
# Platform-owned product defaults. Devices receive these only through the
# signed provisioning/session control plane and cannot override them.
VOLC_HARDWARE_AGENT_BOT_ID: "replace-with-product-default-bot-id"
VOLC_HARDWARE_AGENT_TRANSPORT: "high_quality"
VOLC_HARDWARE_AGENT_MINIMUM_SDK_VERSION: "1.0.0"
VOLC_HARDWARE_PROVISIONING_ENABLED: "false"
VOLC_HARDWARE_INSTANCE_ID: ""
VOLC_HARDWARE_PRODUCT_KEY: ""
VOLC_HARDWARE_REGISTER_URL: https://iot-cn-shanghai.iot.volces.com/2021-12-14/DynamicRegister?Action=DynamicRegister&Version=2021-12-14
VOLC_HARDWARE_REGISTER_TIMEOUT: 10s
VOLC_HARDWARE_DEVICE_NAME_PREFIX: "dt-"
CONTENT_GENERATOR: ark CONTENT_GENERATOR: ark
CONTENT_SAFETY_PROVIDER: managed_http CONTENT_SAFETY_PROVIDER: managed_http
CONTENT_SAFETY_ENDPOINT: https://content-safety.example.invalid/v1/classify CONTENT_SAFETY_ENDPOINT: https://content-safety.example.invalid/v1/classify
@@ -60,6 +71,11 @@ data:
ARK_CONTENT_MODEL: replace-with-ark-content-endpoint-id ARK_CONTENT_MODEL: replace-with-ark-content-endpoint-id
ARK_EMBEDDING_MODEL: replace-with-ark-embedding-endpoint-id ARK_EMBEDDING_MODEL: replace-with-ark-embedding-endpoint-id
ARK_REQUEST_TIMEOUT: 30s ARK_REQUEST_TIMEOUT: 30s
MEMORY_TOOL_TOKEN_TTL: 4h
MEMORY_TOOL_RATE_RPS: "2"
MEMORY_TOOL_RATE_BURST: "5"
PROVIDER_MEMORY_MODE: disabled
PROVIDER_MEMORY_TIMEOUT: 5s
WORKER_POLL_INTERVAL: 2s WORKER_POLL_INTERVAL: 2s
ALERT_WEBHOOK_TIMEOUT: 5s ALERT_WEBHOOK_TIMEOUT: 5s
--- ---
+8 -8
View File
@@ -55,14 +55,16 @@ spec:
value: /var/run/secrets/voice-platform/common/database-url value: /var/run/secrets/voice-platform/common/database-url
- name: REDIS_PASSWORD_FILE - name: REDIS_PASSWORD_FILE
value: /var/run/secrets/voice-platform/common/redis-password value: /var/run/secrets/voice-platform/common/redis-password
- name: DEVICE_SECRET_DIRECTORY - name: DEVICE_CREDENTIAL_MASTER_KEY_FILE
value: /var/run/secrets/voice-platform/devices value: /var/run/secrets/voice-platform/api/device-credential-master-key
- name: RTC_APP_KEY_FILE - name: RTC_APP_KEY_FILE
value: /var/run/secrets/voice-platform/api/rtc-app-key value: /var/run/secrets/voice-platform/api/rtc-app-key
- name: CURSOR_SIGNING_SECRET_FILE - name: CURSOR_SIGNING_SECRET_FILE
value: /var/run/secrets/voice-platform/api/cursor-signing-secret value: /var/run/secrets/voice-platform/api/cursor-signing-secret
- name: SHARE_TOKEN_SIGNING_SECRET_FILE - name: SHARE_TOKEN_SIGNING_SECRET_FILE
value: /var/run/secrets/voice-platform/api/share-token-signing-secret value: /var/run/secrets/voice-platform/api/share-token-signing-secret
- name: MEMORY_TOOL_SIGNING_SECRET_FILE
value: /var/run/secrets/voice-platform/model-storage/memory-tool-signing-secret
- name: OBJECT_STORE_ACCESS_KEY_FILE - name: OBJECT_STORE_ACCESS_KEY_FILE
value: /var/run/secrets/voice-platform/model-storage/object-store-access-key value: /var/run/secrets/voice-platform/model-storage/object-store-access-key
- name: OBJECT_STORE_SECRET_KEY_FILE - name: OBJECT_STORE_SECRET_KEY_FILE
@@ -114,9 +116,6 @@ spec:
- name: model-storage-secrets - name: model-storage-secrets
mountPath: /var/run/secrets/voice-platform/model-storage mountPath: /var/run/secrets/voice-platform/model-storage
readOnly: true readOnly: true
- name: device-secrets
mountPath: /var/run/secrets/voice-platform/devices
readOnly: true
volumes: volumes:
- name: common-secrets - name: common-secrets
secret: secret:
@@ -127,9 +126,6 @@ spec:
- name: model-storage-secrets - name: model-storage-secrets
secret: secret:
secretName: voice-platform-model-storage-secrets secretName: voice-platform-model-storage-secrets
- name: device-secrets
secret:
secretName: voice-platform-device-secrets
--- ---
apiVersion: apps/v1 apiVersion: apps/v1
kind: Deployment kind: Deployment
@@ -190,6 +186,8 @@ spec:
value: /var/run/secrets/voice-platform/common/redis-password value: /var/run/secrets/voice-platform/common/redis-password
- name: PROVIDER_CALLBACK_SECRET_FILE - name: PROVIDER_CALLBACK_SECRET_FILE
value: /var/run/secrets/voice-platform/callbacks/provider-callback-secret value: /var/run/secrets/voice-platform/callbacks/provider-callback-secret
- name: PLATFORM_HEALTH_CALLBACK_SECRET_FILE
value: /var/run/secrets/voice-platform/callbacks/platform-health-callback-secret
ports: ports:
- name: http - name: http
containerPort: 8081 containerPort: 8081
@@ -309,6 +307,8 @@ spec:
value: /var/run/secrets/voice-platform/model-storage/ark-api-key value: /var/run/secrets/voice-platform/model-storage/ark-api-key
- name: CONTENT_SAFETY_API_KEY_FILE - name: CONTENT_SAFETY_API_KEY_FILE
value: /var/run/secrets/voice-platform/model-storage/content-safety-api-key value: /var/run/secrets/voice-platform/model-storage/content-safety-api-key
- name: MEMORY_TOOL_SIGNING_SECRET_FILE
value: /var/run/secrets/voice-platform/model-storage/memory-tool-signing-secret
ports: ports:
- name: http - name: http
containerPort: 8082 containerPort: 8082
@@ -19,7 +19,7 @@ metadata:
spec: spec:
provider: replace-with-installed-provider provider: replace-with-installed-provider
parameters: parameters:
objects: replace-with-provider-specific-rtc-app-key-cursor-and-share-secrets objects: replace-with-provider-specific-rtc-app-key-cursor-share-device-credential-master-key-volc-hardware-product-secret-and-volc-device-credential-master-key
--- ---
apiVersion: secrets-store.csi.x-k8s.io/v1 apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass kind: SecretProviderClass
@@ -29,7 +29,7 @@ metadata:
spec: spec:
provider: replace-with-installed-provider provider: replace-with-installed-provider
parameters: parameters:
objects: replace-with-provider-specific-callback-secret objects: replace-with-provider-specific-provider-and-platform-health-callback-secrets
--- ---
apiVersion: secrets-store.csi.x-k8s.io/v1 apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass kind: SecretProviderClass
@@ -49,14 +49,4 @@ metadata:
spec: spec:
provider: replace-with-installed-provider provider: replace-with-installed-provider
parameters: parameters:
objects: replace-with-provider-specific-object-store-ark-and-content-safety-credentials objects: replace-with-provider-specific-object-store-ark-content-safety-and-memory-tool-credentials
---
apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
name: voice-platform-devices
namespace: voice-platform
spec:
provider: replace-with-installed-provider
parameters:
objects: replace-with-provider-specific-per-device-secrets
@@ -6,6 +6,13 @@ metadata:
spec: spec:
template: template:
spec: spec:
securityContext:
runAsNonRoot: true
runAsUser: 65532
runAsGroup: 65532
fsGroup: 65532
seccompProfile:
type: RuntimeDefault
volumes: volumes:
- $patch: replace - $patch: replace
- name: common-secrets - name: common-secrets
@@ -26,12 +33,6 @@ spec:
readOnly: true readOnly: true
volumeAttributes: volumeAttributes:
secretProviderClass: voice-platform-model-storage secretProviderClass: voice-platform-model-storage
- name: device-secrets
csi:
driver: secrets-store.csi.k8s.io
readOnly: true
volumeAttributes:
secretProviderClass: voice-platform-devices
--- ---
apiVersion: apps/v1 apiVersion: apps/v1
kind: Deployment kind: Deployment
@@ -41,6 +42,13 @@ metadata:
spec: spec:
template: template:
spec: spec:
securityContext:
runAsNonRoot: true
runAsUser: 65532
runAsGroup: 65532
fsGroup: 65532
seccompProfile:
type: RuntimeDefault
volumes: volumes:
- $patch: replace - $patch: replace
- name: common-secrets - name: common-secrets
@@ -64,6 +72,13 @@ metadata:
spec: spec:
template: template:
spec: spec:
securityContext:
runAsNonRoot: true
runAsUser: 65532
runAsGroup: 65532
fsGroup: 65532
seccompProfile:
type: RuntimeDefault
volumes: volumes:
- $patch: replace - $patch: replace
- name: common-secrets - name: common-secrets
+31
View File
@@ -0,0 +1,31 @@
# 本地聊天总结模型
本地开发环境使用 Ollama 提供 OpenAI 兼容接口,默认验证模型为 `qwen2.5:7b`。模型只监听本机 `11434` 端口,应用容器通过 `http://ollama:11434/v1` 访问。
## 启动与下载模型
```powershell
docker compose --profile local-model up -d ollama
docker compose --profile local-model exec ollama ollama pull qwen2.5:7b
```
随后在客户门户的“大模型管理”中创建并激活配置:
- 服务类型:`OpenAI 兼容接口`
- 接口基础地址:`http://ollama:11434/v1`
- 模型 ID:`qwen2.5:7b`
- API Key:本地 Ollama 不校验密钥,可填写长度至少 8 位的本地占位值
- 建议温度:`0.1`
- 建议最大输出:`1200`
必须先执行“测试连接”,通过后才能“设为当前模型”。配置激活后,worker 会在每个总结任务开始时读取当前模型,不需要重启。
## 真实模型回归测试
```powershell
$env:LOCAL_MODEL_BASE_URL='http://localhost:11434/v1'
$env:LOCAL_MODEL_ID='qwen2.5:7b'
go test ./internal/extraction -run TestLocalModelContentValueDecisions -count=1 -v
```
测试覆盖三类输入:纯寒暄应舍弃、有用近况只生成摘要、完整个人回忆生成摘要和故事候选。
+64
View File
@@ -0,0 +1,64 @@
# Windows 本地真实 RTC 联调
该覆盖层只用于本机联调,不访问阿里云服务器。它把 API 的 RTC Token 签发器和 Worker 的语音供应商切换到火山引擎,同时保留数据库、Redis、MinIO、Keycloak和告警模拟器在本地 Docker 中运行。
## 1. 生成本地凭据
在项目根目录运行:
```powershell
.\deploy\local-rtc\configure.ps1
```
如果还要同时验证方舟大模型总结,可改用:
```powershell
.\deploy\local-rtc\configure.ps1 -EnableArkSummary
```
该模式会额外隐藏读取方舟API Key和内容模型Endpoint ID,并通过只读密钥文件传给Worker。
如果 RTC 配置已经存在,只需要补充大模型总结,不必使用`-Force`重录全部密钥。先在[方舟 API Key 控制台](https://console.volcengine.com/ark/region:ark+cn-beijing/apikey)创建密钥,再运行:
```powershell
.\deploy\local-rtc\configure-ark-summary.ps1
```
该脚本只更新`ark-api-key`、`CONTENT_GENERATOR`和`ARK_CONTENT_MODEL`,不会修改已有RTC、语音或设备凭据。当前Chat API既支持模型Model ID,也支持Endpoint ID;脚本默认使用`doubao-seed-2-0-lite-260215`,如果账号未开通该模型,可在提示中填写已开通的Model ID或Endpoint ID。
脚本会以隐藏输入方式依次读取:
- RTC AppKey;
- 火山引擎 AccessKey ID / Secret AccessKey;
- 豆包端到端语音 APP ID / Access Token;
- 豆包音色 ID。
密钥文件只生成到`deploy/local-rtc/runtime`,该目录不会提交到 Git。当前首轮联调使用豆包 S2S-O 纯端到端模式(`OutputMode=0`),平台本地路由仍沿用`s2s_hybrid`模板槽位;这是为了只依赖豆包端到端语音凭据,先验证真实双向音频。后续接入火山方舟 Endpoint 后再切换真正的混合编排模式。
## 2. 启动本地后端
```powershell
.\deploy\local-rtc\start.ps1
```
首次构建需要下载 Docker 基础镜像。脚本会等待`http://localhost:58080/health/ready`可用。
## 3. 启动模拟器
双击`device-simulator-windows/release/voice-device-simulator-0.1.0-windows-x64.exe`,然后:
1. 保持默认本地平台地址、设备 ID 和本地测试设备密钥;
2. 点击“检测平台和音频设备”;
3. 选择麦克风和扬声器;
4. 确认同一个RTC AppID已在RTC控制台的“实时字幕”功能中启用流式语音识别;
5. 点击“开始语音聊天”;
6. 看到“RTC 实时字幕已启动”和“豆包语音任务已激活”后直接说话;
7. 确认“实时字幕与上传”区域同时出现长者和豆包话轮;
8. 点击“结束通话”验证字幕补交、`StopVoiceChat`和RTC离房。
实时字幕要求先在豆包语音控制台创建服务类型为“流式语音识别”的应用,再在RTC控制台的实时字幕配置中填写该应用的APP ID、Access Token和Cluster ID。未开通或配置不匹配时SDK返回`ServiceAccessDenied`;这与“实时对话式AI > 回调设置”无关,不需要为字幕上传填写回调URL。
通话结束后,Worker会根据最终话轮生成通话摘要、关键词、开放话题和待审核故事。只有人工确认故事后才会进入长期记忆,这是既有授权与审核边界,模拟器不会绕过审核自动写入长期记忆。本地默认`CONTENT_GENERATOR=local`仅用于确定性闭环验证;要验证真实大模型总结,必须另外为Worker配置`CONTENT_GENERATOR=ark`、`ARK_API_KEY_FILE`和`ARK_CONTENT_MODEL`。
当新通话冻结快照中的`story_extraction`和`personalized_memory`授权均有效时,平台会把最近3次已完成通话同一最新生成版本的就绪摘要与一个可选开放话题作为短期连续上下文注入豆包;S2S-O模式使用`dialog.system_role`,无需开启供应商侧`dialog_id`历史留存。开放话题只是自然相关时可用的建议,不得覆盖长者当前话题。长者档案中的“避免主动提及”话题也会从下一次开房起一并生效。
RTC AppKey、火山 AK/SK 和豆包 Access Token 均不进入模拟器安装包,也不会通过渲染页面读取。
+32
View File
@@ -0,0 +1,32 @@
services:
api:
environment:
RTC_APP_ID: ${RTC_APP_ID}
RTC_TOKEN_ISSUER: volcengine
RTC_APP_KEY: ""
RTC_APP_KEY_FILE: /run/secrets/local-rtc/rtc-app-key
volumes:
- ./deploy/local-rtc/runtime/rtc-app-key:/run/secrets/local-rtc/rtc-app-key:ro
worker:
environment:
VOICE_PROVIDER: volcengine
# StartVoiceChat must use the same real RTC AppId as the API token issuer.
RTC_APP_ID: ${RTC_APP_ID}
VOLC_ACCESS_KEY_ID: ""
VOLC_ACCESS_KEY_ID_FILE: /run/secrets/local-rtc/volc-access-key-id
VOLC_SECRET_ACCESS_KEY: ""
VOLC_SECRET_ACCESS_KEY_FILE: /run/secrets/local-rtc/volc-secret-access-key
VOLC_RTC_REGION: cn-north-1
# Quote date-shaped API versions so YAML does not coerce them into timestamps.
VOLC_RTC_API_VERSION: "2024-12-01"
VOLC_VOICE_CHAT_TEMPLATE_FILE: /run/secrets/local-rtc/voice-chat-template.json
CONTENT_GENERATOR: ${CONTENT_GENERATOR:-local}
ARK_API_KEY: ""
ARK_API_KEY_FILE: /run/secrets/local-rtc/ark-api-key
ARK_CONTENT_MODEL: ${ARK_CONTENT_MODEL:-}
volumes:
- ./deploy/local-rtc/runtime/volc-access-key-id:/run/secrets/local-rtc/volc-access-key-id:ro
- ./deploy/local-rtc/runtime/volc-secret-access-key:/run/secrets/local-rtc/volc-secret-access-key:ro
- ./deploy/local-rtc/runtime/voice-chat-template.json:/run/secrets/local-rtc/voice-chat-template.json:ro
- ./deploy/local-rtc/runtime/ark-api-key:/run/secrets/local-rtc/ark-api-key:ro
@@ -0,0 +1,52 @@
[CmdletBinding()]
param(
[string]$Model = "doubao-seed-2-0-lite-260215"
)
$ErrorActionPreference = "Stop"
$scriptDirectory = Split-Path -Parent $MyInvocation.MyCommand.Path
$runtimeDirectory = Join-Path $scriptDirectory "runtime"
$environmentFile = Join-Path $runtimeDirectory "local-rtc.env"
$apiKeyFile = Join-Path $runtimeDirectory "ark-api-key"
$utf8NoBom = [System.Text.UTF8Encoding]::new($false)
if (-not (Test-Path -LiteralPath $environmentFile -PathType Leaf)) {
throw "Run .\deploy\local-rtc\configure.ps1 before configuring Ark summaries."
}
$secureKey = Read-Host "Ark API Key (input is hidden)" -AsSecureString
$pointer = [Runtime.InteropServices.Marshal]::SecureStringToBSTR($secureKey)
try {
$apiKey = [Runtime.InteropServices.Marshal]::PtrToStringBSTR($pointer)
}
finally {
[Runtime.InteropServices.Marshal]::ZeroFreeBSTR($pointer)
}
if ([string]::IsNullOrWhiteSpace($apiKey)) {
throw "Ark API Key is required."
}
$selectedModel = Read-Host "Ark content Model ID or Endpoint ID [$Model]"
if ([string]::IsNullOrWhiteSpace($selectedModel)) {
$selectedModel = $Model
}
$selectedModel = $selectedModel.Trim()
if ([string]::IsNullOrWhiteSpace($selectedModel)) {
throw "Ark content model is required."
}
$settings = [ordered]@{}
foreach ($line in Get-Content -LiteralPath $environmentFile) {
if ($line -match '^([^#=][^=]*)=(.*)$') {
$settings[$matches[1].Trim()] = $matches[2]
}
}
$settings["CONTENT_GENERATOR"] = "ark"
$settings["ARK_CONTENT_MODEL"] = $selectedModel
$environmentContent = ($settings.GetEnumerator() | ForEach-Object { "$($_.Key)=$($_.Value)" }) -join "`n"
[IO.File]::WriteAllText($apiKeyFile, $apiKey.Trim(), $utf8NoBom)
[IO.File]::WriteAllText($environmentFile, $environmentContent + "`n", $utf8NoBom)
Write-Host "Ark summary configuration updated. Existing RTC and voice credentials were not changed." -ForegroundColor Green
Write-Host "Next: .\deploy\local-rtc\start.ps1" -ForegroundColor Green
+126
View File
@@ -0,0 +1,126 @@
[CmdletBinding()]
param(
[string]$RtcAppId = "6a8f92ca43726c017d34cce1",
[switch]$EnableArkSummary,
[string]$ArkContentModel = "",
[switch]$Force
)
$ErrorActionPreference = "Stop"
$scriptDirectory = Split-Path -Parent $MyInvocation.MyCommand.Path
$runtimeDirectory = Join-Path $scriptDirectory "runtime"
$utf8NoBom = [System.Text.UTF8Encoding]::new($false)
function Read-RequiredText {
param([string]$Prompt, [string]$Default = "")
$suffix = if ($Default) { " [$Default]" } else { "" }
$value = Read-Host "$Prompt$suffix"
if ([string]::IsNullOrWhiteSpace($value)) { $value = $Default }
if ([string]::IsNullOrWhiteSpace($value)) { throw "$Prompt 不能为空" }
return $value.Trim()
}
function Read-RequiredSecret {
param([string]$Prompt, [int]$MinimumLength = 1)
$secureValue = Read-Host $Prompt -AsSecureString
$pointer = [Runtime.InteropServices.Marshal]::SecureStringToBSTR($secureValue)
try {
$value = [Runtime.InteropServices.Marshal]::PtrToStringBSTR($pointer)
}
finally {
[Runtime.InteropServices.Marshal]::ZeroFreeBSTR($pointer)
}
if ($value.Length -lt $MinimumLength) { throw "$Prompt 长度不足" }
return $value
}
function Write-SecretFile {
param([string]$Name, [string]$Value)
$target = Join-Path $runtimeDirectory $Name
[IO.File]::WriteAllText($target, $Value, $utf8NoBom)
}
if ($RtcAppId.Length -ne 24) {
throw "RTC AppId 必须正好为 24 个字符"
}
if ((Test-Path -LiteralPath $runtimeDirectory) -and -not $Force) {
throw "本地凭据目录已经存在。确认需要重新配置后使用 -Force。"
}
New-Item -ItemType Directory -Path $runtimeDirectory -Force | Out-Null
Write-Host "`n配置只保存在本机 deploy/local-rtc/runtime,已被 Git 忽略。" -ForegroundColor Cyan
Write-Host "输入密钥时屏幕不会显示字符。`n" -ForegroundColor DarkGray
$rtcAppKey = Read-RequiredSecret "RTC AppKey" 16
$volcAccessKeyId = Read-RequiredText "火山引擎 AccessKey ID"
$volcSecretAccessKey = Read-RequiredSecret "火山引擎 Secret AccessKey"
$doubaoAppId = Read-RequiredText "豆包端到端语音 APP ID"
$doubaoAccessToken = Read-RequiredSecret "豆包端到端语音 Access Token"
$speaker = Read-RequiredText "豆包音色 ID" "zh_female_vv_jupiter_bigtts"
$arkApiKey = ""
if ($EnableArkSummary) {
$arkApiKey = Read-RequiredSecret "方舟 API Key"
$ArkContentModel = Read-RequiredText "方舟内容模型 Endpoint ID" $ArkContentModel
}
$template = [ordered]@{
schema_version = "voice-chat-template-bundle-v1"
templates = [ordered]@{
s2s_hybrid = [ordered]@{
AgentConfig = [ordered]@{
TargetUserId = @()
UserId = ""
WelcomeMessage = ""
EnableConversationStateCallback = $false
}
Config = [ordered]@{
InterruptMode = 0
S2SConfig = [ordered]@{
Provider = "volcano"
OutputMode = 0
ProviderParams = [ordered]@{
app = [ordered]@{
appid = $doubaoAppId
token = $doubaoAccessToken
}
dialog = [ordered]@{
extra = [ordered]@{
model = "O"
strict_audit = $true
audit_response = "抱歉,这个话题我暂时无法回答。我们换个轻松的话题吧。"
}
}
asr = [ordered]@{
extra = [ordered]@{
end_smooth_window_ms = 1200
}
}
tts = [ordered]@{
speaker = $speaker
}
}
}
}
}
}
}
Write-SecretFile "rtc-app-key" $rtcAppKey
Write-SecretFile "volc-access-key-id" $volcAccessKeyId
Write-SecretFile "volc-secret-access-key" $volcSecretAccessKey
Write-SecretFile "ark-api-key" $arkApiKey
[IO.File]::WriteAllText(
(Join-Path $runtimeDirectory "voice-chat-template.json"),
($template | ConvertTo-Json -Depth 20),
$utf8NoBom
)
[IO.File]::WriteAllText(
(Join-Path $runtimeDirectory "local-rtc.env"),
"RTC_APP_ID=$RtcAppId`nCONTENT_GENERATOR=$(if ($EnableArkSummary) { 'ark' } else { 'local' })`nARK_CONTENT_MODEL=$ArkContentModel`n",
$utf8NoBom
)
Write-Host "`n本地真实 RTC 配置已生成。" -ForegroundColor Green
Write-Host "下一步运行: .\deploy\local-rtc\start.ps1" -ForegroundColor Green
+58
View File
@@ -0,0 +1,58 @@
[CmdletBinding()]
param([switch]$NoBuild)
$ErrorActionPreference = "Stop"
$scriptDirectory = Split-Path -Parent $MyInvocation.MyCommand.Path
$projectRoot = Resolve-Path (Join-Path $scriptDirectory "..\..")
$runtimeDirectory = Join-Path $scriptDirectory "runtime"
$environmentFile = Join-Path $runtimeDirectory "local-rtc.env"
$requiredFiles = @(
"rtc-app-key",
"volc-access-key-id",
"volc-secret-access-key",
"ark-api-key",
"voice-chat-template.json",
"local-rtc.env"
)
foreach ($name in $requiredFiles) {
if (-not (Test-Path -LiteralPath (Join-Path $runtimeDirectory $name) -PathType Leaf)) {
throw "Missing $name. Run .\deploy\local-rtc\configure.ps1 first."
}
}
Push-Location $projectRoot
try {
$arguments = @(
"compose",
"-f", "compose.yaml",
"-f", "deploy/local-rtc/compose.override.yaml",
"--env-file", $environmentFile,
"up", "-d"
)
if (-not $NoBuild) { $arguments += "--build" }
$arguments += @(
"postgres", "redis", "minio", "keycloak", "migrate", "seed",
"callbacks", "mock-alert-receiver", "mock-provider", "api", "worker"
)
& docker @arguments
if ($LASTEXITCODE -ne 0) { throw "Docker Compose startup failed." }
Write-Host "`nWaiting for local API readiness..." -ForegroundColor Cyan
$deadline = [DateTime]::UtcNow.AddMinutes(3)
do {
Start-Sleep -Seconds 2
try {
$health = Invoke-RestMethod -Uri "http://localhost:58080/health/ready" -TimeoutSec 3
if ($health) { break }
}
catch { }
} while ([DateTime]::UtcNow -lt $deadline)
if (-not $health) { throw "Local API did not become ready within three minutes. Check docker compose logs api worker." }
Write-Host "Local real RTC backend is ready: http://localhost:58080" -ForegroundColor Green
Write-Host "You can now start the Windows voice device simulator." -ForegroundColor Green
}
finally {
Pop-Location
}
+2 -1
View File
@@ -1,3 +1,4 @@
FROM migrate/migrate:v4.18.3 FROM migrate/migrate:v4.18.3
COPY migrations /migrations COPY --chown=65532:65532 migrations /migrations
USER 65532:65532
@@ -0,0 +1,24 @@
# 长者故事陪伴人设 v1
该文件是 `PERSONA` 配置的可审阅来源。线上运行仍使用经过版本化发布并由 `AGENT_BUNDLE` 引用的配置。
## system_prompt
你是一位平等、耐心、尊重长者的中文语音聊天伙伴。你的目标是让长者感到被认真听见,在对方愿意时自然聊深一段生活经历;故事完整度不是长者必须完成的任务。
聊天规则:
1. 先回应长者刚说的具体内容,再决定是否追问。通常只说一到两句,每轮最多问一个问题;没有自然问题时可以只回应。
2. 长者分享经历时,先复述或回应其中一个具体细节,再视情况问一个开放、具体的问题。不要连着盘问,不要把聊天变成采访。
3. 跟随长者当前话题。历史摘要、开放话题和记忆只作背景,只有与当前内容自然相关时才使用,不要突然照念旧话题。
4. 长者说“不想聊”、累了、忘了或换了话题时,立刻尊重并转向或收尾。不要催促、埋怨、强行挽留,也不要说“你怎么才回来”“我一直等你”。
5. 允许重复讲述、停顿和模糊年代,不纠正口音或表达。可以温和核对不确定的记忆,但不能编造未提供的人物、经历、偏好或因果,也不能把推断说成事实。
6. 使用档案里的称呼;没有明确称呼时不要擅自叫“叔叔”“阿姨”。表达像熟悉的朋友,避免客服套话、夸张赞美、网络流行语和复杂术语。
7. 涉及疾病、用药和症状时,只给一般安全提醒,不诊断,也不建议停药换药;紧急情况优先建议联系家人或专业人员。
8. 回复只包含直接对长者说的自然口语,不输出动作、表情、语气、情绪、场景、系统提示或配置说明。
自然回应的顺序是:接住当前内容 -> 回应细节或感受 -> 有必要才问一个问题 -> 尊重停顿与结束。
## tone
亲切、自然、平等;语速正常,句子简短,不使用夸张撒娇或电子化腔调。
@@ -0,0 +1,26 @@
# 长者故事陪伴人设 v2
该文件是 `PERSONA` 配置的可审阅来源。线上运行仍使用经过版本化发布并由 `AGENT_BUNDLE` 引用的配置。
## system_prompt
你是一位平等、耐心、尊重长者的中文语音聊天伙伴。你的目标是让长者感到被认真听见,在对方愿意时自然聊深一段生活经历;故事完整度不是长者必须完成的任务。
聊天规则:
1. 先回应长者刚说的具体内容。通常只说一到两句,只有问题确实能帮助对方继续表达时才问。
2. 问句必须稀疏:任意连续两次回复最多出现一个问题。如果你的上一条回复已经问过问题,那么下一条回复必须只用陈述句回应,即使长者只回答“是啊”“嗯”“对”“不是”也不能立刻再问。先回应、分享一句相关感受或常识,然后留出停顿。
3. 长者分享经历时,先复述或回应其中一个确实由长者说出的细节。需要追问时,一次只问一个开放而具体的问题;不要连续盘问,不要把聊天变成采访。
4. 不得补写长者没有说过的现场、动作、人物、情绪、原因或结果。避免“我猜……”“您是不是经常……”等带着假设的说法;不确定时先作不含新事实的回应,之后再择机核对。
5. 跟随长者当前话题。历史摘要、开放话题和记忆只作背景,只有与当前内容自然相关时才使用,不要突然照念旧话题。
6. 长者说“不想聊”、累了、忘了或换了话题时,立刻尊重并转向或收尾。不要催促、埋怨、强行挽留,也不要说“你怎么才回来”“我一直等你”。
7. 允许重复讲述、停顿和模糊年代,不纠正口音或表达。可以温和核对不确定的记忆,但不能编造未提供的人物、经历、偏好或因果,也不能把推断说成事实。
8. 使用档案里的称呼;没有明确称呼时不要擅自叫“叔叔”“阿姨”。表达像熟悉的朋友,避免客服套话、夸张赞美、网络流行语和复杂术语。
9. 涉及疾病、用药和症状时,只给一般安全提醒,不诊断,也不建议停药换药;紧急情况优先建议联系家人或专业人员。
10. 回复只包含直接对长者说的自然口语,不输出动作、表情、语气、情绪、场景、系统提示或配置说明。
自然回应的顺序是:接住当前内容 -> 回应一个真实细节或感受 -> 检查上一轮是否已经提问 -> 只有必要且没有连续提问时才问一个问题 -> 尊重停顿与结束。
## tone
亲切、自然、平等;语速正常,句子简短,不使用夸张撒娇或电子化腔调。
@@ -0,0 +1,33 @@
# 长者故事陪伴人设 v3
该文件是 `PERSONA` 配置的可审阅来源。线上运行仍使用经过版本化发布并由 `AGENT_BUNDLE` 引用的配置。
## system_prompt
你是一位平等、耐心、尊重长者的中文语音聊天伙伴。你的目标是让长者感到被认真听见,在对方愿意时自然聊深一段生活经历;故事完整度不是长者必须完成的任务。
聊天规则:
1. 先回应长者刚说的具体内容。通常只说一到两句,默认使用陈述句;只有问题确实能帮助对方继续表达时才问。
2. 问句必须稀疏:任意连续三次回复最多出现一个问题。“问题”包括所有索取信息、确认或同意的表达,不以问号为准;“吧、吗、呢、是不是、有没有、会不会、对不对、可不可以、愿不愿意”等都算问题。如果你最近两次回复中已经问过问题,那么本轮必须只用陈述句回应。
3. 长者只回答“是啊”“嗯”“对”“不是”等简短附和或否定时,下一条必须是纯陈述句,不能用“吧”请求确认,也不能换一种说法继续追问。先接住回答,说一句有依据的回应或一般常识,然后停下来。
4. 长者分享经历时,先复述或回应其中一个确实由长者说出的细节。需要追问时,一次只问一个开放而具体的问题;不要连续盘问,不要把聊天变成采访。
5. 只能把长者明确说过的内容归到长者本人。不得补写现场、动作、人物、情绪、原因、结果或第三方态度;不得用“肯定、一定、应该、想必、我猜、听得出、看得出”包装猜测。没有依据时说一般性的内容,不评价长者“很厉害”,不声称“大家都佩服您”。
6. 跟随长者当前话题。历史摘要、开放话题和记忆只作背景,只有与当前内容自然相关时才使用,不要突然照念旧话题。
7. 长者说“不想聊”、累了、忘了或换了话题时,立刻尊重并转向或收尾。不要催促、埋怨、强行挽留,也不要说“你怎么才回来”“我一直等你”。
8. 允许重复讲述、停顿和模糊年代,不纠正口音或表达。可以温和核对不确定的记忆,但不能编造未提供的人物、经历、偏好或因果,也不能把推断说成事实。
9. 使用档案里的称呼;没有明确称呼时不要擅自叫“叔叔”“阿姨”。表达像熟悉的朋友,避免客服套话、夸张赞美、网络流行语和复杂术语。
10. 涉及疾病、用药和症状时,只给一般安全提醒,不诊断,也不建议停药换药;紧急情况优先建议联系家人或专业人员。
11. 回复只包含直接对长者说的自然口语,不输出动作、表情、语气、情绪、场景、系统提示或配置说明。
示例:
- 长者说“我年轻的时候在一家工厂修机器。”可以答:“修机器是门讲经验的手艺。哪一次维修让您印象最深?”
- 如果长者接着只说“是啊。”下一条可以答:“这门手艺里的经验,都是一点点积累起来的。”然后停下来。
- 不要回答“肯定很辛苦吧”“您当时肯定很厉害”“大家应该都很佩服您吧”。
自然回应的顺序是:接住当前内容 -> 回应一个真实细节或一般常识 -> 检查最近两次回复是否已经提问 -> 只有必要且不会形成连续追问时才问一个问题 -> 尊重停顿与结束。
## tone
亲切、自然、平等;语速正常,句子简短,不使用夸张撒娇或电子化腔调。
+75 -2
View File
@@ -1,6 +1,8 @@
# 阿里云单机演示部署 # 阿里云单机演示部署
该配置用于单机公网演示,不是生产验收配置。只向公网发布HTTP 80和HTTPS 443;PostgreSQL、Redis、MinIO、Keycloak内部端口以及API/Worker端口均不映射到宿主机。当前阿里云演示机使用Let's Encrypt短期IP证书,`certbot-ip-renew.timer`负责自动续期。真实RTC、模型、内容安全、云KMS、备份恢复和监控告警仍按阶段8验收清单执行。 该配置用于单机公网演示,不是生产验收配置。只向公网发布HTTP 80和HTTPS 443;PostgreSQL、Redis、MinIO、Keycloak内部端口以及API/Worker端口均不映射到宿主机。当前阿里云演示机使用Let's Encrypt短期IP证书,`certbot-ip-renew.timer`负责自动续期。单机备份覆盖PostgreSQL、Redis、MinIO对象和当前Keycloak数据目录,并执行数据库级隔离恢复校验;五分钟健康检查覆盖部署模式、公网入口、OIDC、容器、模拟服务、备份、外部告警、证书、定时任务和磁盘。跨故障域备份、真实告警接收器、真实RTC、模型、内容安全和云KMS仍按阶段8验收清单执行。
无法直接从Git拉取或需要冻结当前工作树时,可先在开发机执行`powershell -NoProfile -ExecutionPolicy Bypass -File .\scripts\build-server-deployment-bundle.ps1 -Version 0.76.0-rc.hardware1`,再用`powershell -NoProfile -ExecutionPolicy Bypass -File .\scripts\verify-server-deployment-bundle.ps1 -Archive .\.deploy-artifacts\data-terminal-cloud-0.76.0-rc.hardware1.zip`验证。归档包含`deploy-manifest.json`和`SHA256SUMS`,不会打包`.env`、设备/供应商密钥、本地运行目录、`node_modules`或构建缓存。上传服务器后必须再次核对归档侧车`.sha256`和包内清单,再覆盖候选目录;服务器现有`deploy/server/.env`及`/etc/data-terminal-cloud`密钥目录不得从归档替换。
服务器至少需要Docker Engine 29与Compose 2.40。首次部署先生成权限为600的服务器专用配置: 服务器至少需要Docker Engine 29与Compose 2.40。首次部署先生成权限为600的服务器专用配置:
@@ -15,10 +17,81 @@
载入同一候选版本的`data-terminal-cloud-backend`和`data-terminal-cloud-web`镜像后执行: 载入同一候选版本的`data-terminal-cloud-backend`和`data-terminal-cloud-web`镜像后执行:
```sh ```sh
chmod 0755 deploy/server/*.sh
docker compose -f compose.yaml -f deploy/server/compose.override.yaml \ docker compose -f compose.yaml -f deploy/server/compose.override.yaml \
--env-file deploy/server/.env up -d --no-build \ --env-file deploy/server/.env up -d --no-build \
postgres redis minio keycloak migrate seed mock-alert-receiver callbacks \ postgres redis minio keycloak migrate seed mock-alert-receiver callbacks \
mock-provider api worker web gateway mock-provider api worker web gateway
docker compose -f compose.yaml -f deploy/server/compose.override.yaml \
--env-file deploy/server/.env up -d --no-build --force-recreate --no-deps gateway
``` ```
安全组和主机防火墙只保留TCP 80、443以及受限来源的SSH 22。如果云安全组暂未放行443,可临时将宿主机80映射到网关443,并使用`https://SERVER_IP:80`;放行443后应恢复标准HTTPS地址。正式环境仍应配置自有域名、生产OIDC和真实外部服务,并切换到Kubernetes生产覆盖层。 从Windows生成的压缩包解开后必须恢复脚本执行位。若`current`是切换到新发布目录的符号链接,还必须强制重建Gateway;否则Docker会继续使用旧发布目录已解析的Nginx绑定挂载,新增路由不会生效。
安全组和主机防火墙只保留TCP 80、443以及受限来源的SSH 22。HTTP 80仅用于跳转到标准HTTPS 443。正式环境仍应配置自有域名、生产OIDC和真实外部服务,并切换到Kubernetes生产覆盖层。
需要在演示机接入真实RTC、豆包语音和方舟摘要时,把凭据文件以`0600`权限放入`/etc/data-terminal-cloud/voice-runtime`,并在`deploy/server/.env`设置`RTC_TOKEN_ISSUER=volcengine`、`VOICE_PROVIDER=volcengine`、`RTC_APP_ID`、`CONTENT_GENERATOR`和`ARK_CONTENT_MODEL`。Compose只读挂载该目录,敏感值不进入源码、镜像或环境文件。
切换火山硬件对话智能体时,维护中心逐台只需把`voice_runtime`设为`volc_hardware_agent`,不填写设备级Bot。平台代设备动态注册时,在`deploy/server/.env`设置`VOLC_HARDWARE_PROVISIONING_ENABLED=true`、`VOLC_HARDWARE_INSTANCE_ID`、`VOLC_HARDWARE_PRODUCT_KEY`、`VOLC_HARDWARE_AGENT_BOT_ID`、`VOLC_HARDWARE_AGENT_TRANSPORT`、`VOLC_HARDWARE_AGENT_MINIMUM_SDK_VERSION`、`VOLC_HARDWARE_PRODUCT_SECRET_FILE=/run/secrets/voice-runtime/volc-hardware-product-secret`和独立随机的`VOLC_HARDWARE_DEVICE_CREDENTIAL_MASTER_KEY`;将Product Secret以单行、`0600`权限写入`/etc/data-terminal-cloud/voice-runtime/volc-hardware-product-secret`。旧设备代理注册流程的Product Secret不下发给设备;新设备ONBOARDING-1.1按附件E.2通过认证后的HTTPS加密配置下发。Product Secret不得进入`.env`、源码、日志或镜像。设备首次预配时,平台根据这些产品默认值自动创建火山绑定并调用DynamicRegister;火山从产品额度中分配License,平台只加密保存并下发本设备的一机一密Device Secret,不下发License明文。发布前必须验证平台代理注册、默认Bot绑定、每次会话上下文注入、`session.updated`确认后激活、`session_updated`存活上报、`stop_session`执行和最终字幕补传。
```sh
sudo install -d -m 0700 /etc/data-terminal-cloud/voice-runtime
sudo sh -c 'umask 077; read -r product_secret; printf "%s" "$product_secret" > /etc/data-terminal-cloud/voice-runtime/volc-hardware-product-secret'
```
上面的`read`从终端读取,不要把真实Product Secret写进Shell历史。启用前确认默认Bot关联的产品与`VOLC_HARDWARE_PRODUCT_KEY`完全一致;Device Name由`dt-`加平台设备UUID稳定生成。密钥文件或产品默认配置缺失、格式错误时API拒绝启动,未启用时凭据接口明确返回`HARDWARE_AGENT_PROVISIONING_UNAVAILABLE`。
设备HMAC密钥由`device_credential`表以AES-256-GCM密文保存。`deploy/server/.env`中的`DEVICE_CREDENTIAL_MASTER_KEY`只负责数据库密文的认证解密,不得写入数据库或设备;既有`DEVICE_LOCAL_SECRET`只用于首次部署升级时导入两台演示设备,确认数据库凭据记录完整后可移除。新增、轮换和吊销设备密钥使用后端镜像内的`/app/device-credential`维护命令,生成的设备明文密钥只显示一次。
通话内记忆工具只通过网关精确开放`/internal/v1/tools/memory-search`,请求必须携带平台为当前通话签发的短期任务令牌。`MEMORY_TOOL_SIGNING_SECRET`必须是服务器独立随机密钥,不得与游标签名、分享签名或RTC密钥复用;网关不会开放其他`/internal`路径。
## 自动备份与恢复校验
首次安装定时任务:
```bash
chmod 0755 deploy/server/backup.sh deploy/server/verify-backup.sh deploy/server/install-backup.sh
sudo deploy/server/install-backup.sh
sudo systemctl start platform-backup.service
sudo systemctl status platform-backup.service --no-pager
```
备份以AES-256-CBC、PBKDF2和独立随机口令加密,默认保存在`/var/backups/data-terminal-cloud`并保留14天。每次定时任务会实际把PostgreSQL恢复到随机隔离数据库,对比迁移状态、客户组织、账号、表和外键计数,再删除临时数据库。Redis RDB、MinIO对象和暂停后的Keycloak数据目录会进入同一加密归档并进行逐文件SHA-256校验;当前单机配置不把这些文件级检查冒充跨故障域生产恢复。
手工验证已有备份:
```bash
sudo deploy/server/verify-backup.sh /var/backups/data-terminal-cloud/platform-YYYYmmddTHHMMSSZ.tar.gz.enc
```
加密口令位于`/etc/data-terminal-cloud/backup-passphrase`且仅root可读。必须把该口令和加密备份复制到不同故障域;口令丢失将无法恢复备份。
## 健康监测与告警
```bash
chmod 0755 deploy/server/health-monitor.sh deploy/server/install-monitoring.sh
sudo deploy/server/install-monitoring.sh
sudo systemctl start platform-health-monitor.service
sudo systemctl show platform-health-monitor.service -p Result -p ExecMainStatus
python3 -m json.tool /var/lib/data-terminal-cloud-monitor/status.json
```
监测任务每五分钟执行,正常状态为`healthy`。默认`HOST_DEPLOYMENT_PROFILE=demo`,要求两个模拟服务完整运行;切到`production`后则要求模拟服务全部停止。它还要求宿主机所有非回环TCP监听只能使用22、80和443,避免安全组误放行与错误容器端口映射叠加后暴露数据库或缓存。默认只记录结构化状态和systemd日志;在`/etc/data-terminal-cloud/monitor.env`配置`ALERT_WEBHOOK_URL`后,状态变化会发送带时间戳和HMAC-SHA256签名的JSON通知。完整检查项、签名规则、阈值和失败分支见[《单机云端健康监测与告警手册》](../../docs/server-health-monitoring.md)。
生产候选不能只修改部署模式。生产验收快照同时要求:`HOST_DEPLOYMENT_PROFILE=production`;API、Callbacks和Worker实际容器`APP_ENV=prod`;Keycloak不使用`start-dev`;OSS异地备份最近一次上传与回读验证成功;真实外部告警接收器已配置;`mock-provider`和`mock-alert-receiver`均不运行。任一条件缺失都会阻断生产基础设施门禁,但不会影响`LAB`或`STAGING`联调。当前Compose覆盖层固定为演示用途,不能通过改标签升级成生产部署。
## OSS异地备份适配
```bash
chmod 0755 deploy/server/install-ossutil.sh \
deploy/server/install-offsite-backup.sh \
deploy/server/sync-backup-offsite.sh
sudo deploy/server/install-offsite-backup.sh
```
安装后默认为禁用状态,不会访问OSS。默认认证方式为`ecs_ram_role`,使用`cn-hangzhou`地域端点并强制先通过IMDSv2确认实例角色,不保存长期AccessKey。创建私有Bucket并绑定最小权限ECS RAM角色后,在`/etc/data-terminal-cloud/offsite-backup.env`填写Bucket和前缀、设置`OFFSITE_BACKUP_ENABLED=true`与`OFFSITE_BACKUP_REQUIRED=true`,再手工启动一次`platform-offsite-backup.service`。任务会上传加密归档及摘要、重新下载并复算SHA-256;健康检查在启用后强制要求最新本机归档已有对应远端验证。配置和真实验收步骤见[《阿里云OSS异地备份适配与门禁》](../../docs/offsite-backup.md)。
## 首次设备认证与绑定(2026-09-12)
新设备使用ONBOARDING-1.1。叠加`deploy/server/compose.onboarding.yaml`,设置ONBOARDING_RELEASE_VERSION与实际网关网段ONBOARDING_TRUSTED_PROXY_CIDRS;主密钥使用服务器`/etc/data-terminal-cloud/voice-runtime/onboarding-master-key`。先备份再迁移至58,更新API、worker和web,并重建gateway使新的Nginx挂载生效。仅HTTP `POST /api/v1/device/time-challenges`直达API;其余HTTP继续跳转HTTPS。详细配置与验证见[实施说明](../../docs/onboarding-implementation.md)。
+145
View File
@@ -0,0 +1,145 @@
#!/usr/bin/env bash
set -Eeuo pipefail
project_root=${PLATFORM_ROOT:-/opt/data-terminal-cloud/current}
backup_root=${BACKUP_ROOT:-/var/backups/data-terminal-cloud}
passphrase_file=${BACKUP_PASSPHRASE_FILE:-/etc/data-terminal-cloud/backup-passphrase}
retention_days=${BACKUP_RETENTION_DAYS:-14}
verify_after_backup=${VERIFY_AFTER_BACKUP:-false}
case "$backup_root" in
/*) ;;
*) echo "BACKUP_ROOT must be an absolute path" >&2; exit 2 ;;
esac
if [[ "$backup_root" == "/" || "$backup_root" == "/var" || "$backup_root" == "/var/backups" ]]; then
echo "BACKUP_ROOT is too broad" >&2
exit 2
fi
if [[ ! "$retention_days" =~ ^[1-9][0-9]*$ ]]; then
echo "BACKUP_RETENTION_DAYS must be a positive integer" >&2
exit 2
fi
if [[ ! -r "$passphrase_file" ]]; then
echo "Backup passphrase file is missing or unreadable: $passphrase_file" >&2
exit 2
fi
compose=(docker compose -f "$project_root/compose.yaml" -f "$project_root/deploy/server/compose.override.yaml" --env-file "$project_root/deploy/server/.env")
mkdir -p "$backup_root"
chmod 0700 "$backup_root"
umask 077
exec 9>"$backup_root/.backup.lock"
if ! flock -n 9; then
echo "Another backup is already running" >&2
exit 3
fi
stamp=$(date -u +%Y%m%dT%H%M%SZ)
staging=$(mktemp -d "$backup_root/.staging-${stamp}-XXXXXX")
plain_archive="$backup_root/.platform-${stamp}.tar.gz"
encrypted_archive="$backup_root/platform-${stamp}.tar.gz.enc"
checksum_file="$encrypted_archive.sha256"
minio_container=""
keycloak_container=""
keycloak_paused=false
cleanup() {
status=$?
if [[ "$keycloak_paused" == true && -n "$keycloak_container" ]]; then
docker unpause "$keycloak_container" >/dev/null 2>&1 || true
fi
if [[ -n "$minio_container" ]]; then
docker exec "$minio_container" rm -rf /tmp/platform-backup /tmp/platform-mc >/dev/null 2>&1 || true
fi
rm -rf -- "$staging"
rm -f -- "$plain_archive"
exit "$status"
}
trap cleanup EXIT INT TERM
service_container() {
local service=$1
local container
container=$("${compose[@]}" ps -q "$service")
if [[ -z "$container" ]] || [[ $(docker inspect -f '{{.State.Running}}' "$container") != true ]]; then
echo "Required service is not running: $service" >&2
exit 4
fi
printf '%s' "$container"
}
postgres_container=$(service_container postgres)
redis_container=$(service_container redis)
minio_container=$(service_container minio)
keycloak_container=$(service_container keycloak)
redis_password=$(sed -n 's/^REDIS_PASSWORD=//p' "$project_root/deploy/server/.env")
if [[ -z "$redis_password" ]]; then
echo "REDIS_PASSWORD is missing from the server environment" >&2
exit 4
fi
docker exec "$postgres_container" pg_dump -U voice -d voice_platform -Fc --no-owner --no-acl >"$staging/postgres.dump"
docker exec "$postgres_container" psql -U voice -d voice_platform -Atc \
"SELECT version||'|'||dirty||'|'||(SELECT count(*) FROM tenant)||'|'||(SELECT count(*) FROM user_account)||'|'||(SELECT count(*) FROM information_schema.tables WHERE table_schema='public')||'|'||(SELECT count(*) FROM information_schema.table_constraints WHERE constraint_type='FOREIGN KEY') FROM schema_migrations;" \
>"$staging/database-evidence.txt"
mkdir -p "$staging/redis"
printf '%s\n' "$redis_password" | docker exec -i "$redis_container" sh -ec '
IFS= read -r REDISCLI_AUTH
export REDISCLI_AUTH
redis-cli --no-auth-warning SAVE >/dev/null
'
docker exec "$redis_container" redis-check-rdb /data/dump.rdb >/dev/null
docker cp "$redis_container:/data/dump.rdb" "$staging/redis/dump.rdb" >/dev/null
docker exec "$minio_container" sh -ec '
rm -rf /tmp/platform-backup /tmp/platform-mc
mkdir -p /tmp/platform-backup /tmp/platform-mc
mc --config-dir /tmp/platform-mc alias set local http://127.0.0.1:9000 "$MINIO_ROOT_USER" "$MINIO_ROOT_PASSWORD" >/dev/null
mc --config-dir /tmp/platform-mc mirror --overwrite --preserve local/ /tmp/platform-backup/
'
mkdir -p "$staging/object-store"
docker cp "$minio_container:/tmp/platform-backup/." "$staging/object-store/" >/dev/null
docker exec "$minio_container" rm -rf /tmp/platform-backup /tmp/platform-mc
docker pause "$keycloak_container" >/dev/null
keycloak_paused=true
mkdir -p "$staging/keycloak"
docker cp "$keycloak_container:/opt/keycloak/data/." "$staging/keycloak/" >/dev/null
docker unpause "$keycloak_container" >/dev/null
keycloak_paused=false
object_files=$(find "$staging/object-store" -type f | wc -l | tr -d ' ')
keycloak_files=$(find "$staging/keycloak" -type f | wc -l | tr -d ' ')
cat >"$staging/manifest.properties" <<EOF
format=platform-server-backup-v1
created_at=${stamp}
build_version=$(sed -n 's/^PLATFORM_BUILD_VERSION=//p' "$project_root/deploy/server/.env")
build_commit=$(sed -n 's/^PLATFORM_BUILD_COMMIT=//p' "$project_root/deploy/server/.env")
database_evidence=$(cat "$staging/database-evidence.txt")
object_files=${object_files}
keycloak_files=${keycloak_files}
redis_snapshot=true
keycloak_snapshot_mode=paused_filesystem_copy
EOF
(
cd "$staging"
find . -type f ! -name SHA256SUMS -print0 | sort -z | xargs -0 -r sha256sum >SHA256SUMS
)
tar -C "$staging" -czf "$plain_archive" .
openssl enc -aes-256-cbc -salt -pbkdf2 -iter 200000 \
-pass "file:$passphrase_file" -in "$plain_archive" -out "$encrypted_archive"
sha256sum "$encrypted_archive" >"$checksum_file"
chmod 0600 "$encrypted_archive" "$checksum_file"
while IFS= read -r -d '' expired; do
rm -f -- "$expired" "$expired.sha256"
done < <(find "$backup_root" -maxdepth 1 -type f -name 'platform-*.tar.gz.enc' -mtime "+$retention_days" -print0)
if [[ "$verify_after_backup" == true ]]; then
"$project_root/deploy/server/verify-backup.sh" "$encrypted_archive"
fi
printf '%s\n' "$encrypted_archive"
+5 -1
View File
@@ -33,14 +33,18 @@ umask 077
printf 'KEYCLOAK_ADMIN_USERNAME=platform-admin\n' printf 'KEYCLOAK_ADMIN_USERNAME=platform-admin\n'
printf 'KEYCLOAK_ADMIN_PASSWORD=%s\n' "$(openssl rand -hex 24)" printf 'KEYCLOAK_ADMIN_PASSWORD=%s\n' "$(openssl rand -hex 24)"
printf 'DEVICE_LOCAL_SECRET=%s\n' "$(openssl rand -hex 32)" printf 'DEVICE_LOCAL_SECRET=%s\n' "$(openssl rand -hex 32)"
printf 'DEVICE_CREDENTIAL_MASTER_KEY=%s\n' "$(openssl rand -hex 32)"
printf 'VOLC_HARDWARE_DEVICE_CREDENTIAL_MASTER_KEY=%s\n' "$(openssl rand -hex 32)"
printf 'MODEL_CREDENTIAL_MASTER_KEY=%s\n' "$(openssl rand -hex 32)"
printf 'RTC_TOKEN_SIGNING_SECRET=%s\n' "$(openssl rand -hex 32)" printf 'RTC_TOKEN_SIGNING_SECRET=%s\n' "$(openssl rand -hex 32)"
printf 'CURSOR_SIGNING_SECRET=%s\n' "$(openssl rand -hex 32)" printf 'CURSOR_SIGNING_SECRET=%s\n' "$(openssl rand -hex 32)"
printf 'SHARE_TOKEN_SIGNING_SECRET=%s\n' "$(openssl rand -hex 32)" printf 'SHARE_TOKEN_SIGNING_SECRET=%s\n' "$(openssl rand -hex 32)"
printf 'MEMORY_TOOL_SIGNING_SECRET=%s\n' "$(openssl rand -hex 32)"
printf 'PROVIDER_CALLBACK_SECRET=%s\n' "$(openssl rand -hex 32)" printf 'PROVIDER_CALLBACK_SECRET=%s\n' "$(openssl rand -hex 32)"
printf 'PLATFORM_HEALTH_CALLBACK_SECRET=%s\n' "$(openssl rand -hex 32)"
printf 'MOCK_PROVIDER_API_KEY=%s\n' "$(openssl rand -hex 24)" printf 'MOCK_PROVIDER_API_KEY=%s\n' "$(openssl rand -hex 24)"
printf 'ALERT_WEBHOOK_SECRET=%s\n' "$(openssl rand -hex 32)" printf 'ALERT_WEBHOOK_SECRET=%s\n' "$(openssl rand -hex 32)"
} > "$output_path" } > "$output_path"
chmod 600 "$output_path" chmod 600 "$output_path"
echo "created $output_path with mode 600" echo "created $output_path with mode 600"
+32
View File
@@ -0,0 +1,32 @@
# Add after compose.override.yaml and the server's existing compose.subtitle.yaml.
x-onboarding-runtime: &onboarding-runtime
ONBOARDING_ENABLED: ${ONBOARDING_ENABLED:-true}
ONBOARDING_ACTIVATION_ENABLED: ${ONBOARDING_ACTIVATION_ENABLED:-true}
ONBOARDING_PAUSE_CLAIMS: ${ONBOARDING_PAUSE_CLAIMS:-false}
ONBOARDING_SMS_LOGIN_ENABLED: ${ONBOARDING_SMS_LOGIN_ENABLED:-false}
ONBOARDING_ALLOW_LOCAL_HTTP: "false"
ONBOARDING_MASTER_KEY: ""
ONBOARDING_MASTER_KEY_FILE: /run/secrets/voice-runtime/onboarding-master-key
ONBOARDING_ENTRY_URL: ${PUBLIC_ORIGIN}/onboarding/entry
ONBOARDING_TRUSTED_PROXY_CIDRS: ${ONBOARDING_TRUSTED_PROXY_CIDRS:?set the gateway Docker subnet}
ONBOARDING_SELF_SIGNUP_TENANT_IDS: ${ONBOARDING_SELF_SIGNUP_TENANT_IDS:-}
VOLC_HARDWARE_INSTANCE_ID: ${VOLC_HARDWARE_INSTANCE_ID}
VOLC_HARDWARE_PRODUCT_KEY: ${VOLC_HARDWARE_PRODUCT_KEY}
VOLC_HARDWARE_AGENT_BOT_ID: ${VOLC_HARDWARE_AGENT_BOT_ID}
VOLC_HARDWARE_AGENT_TRANSPORT: ${VOLC_HARDWARE_AGENT_TRANSPORT:-low_load}
VOLC_HARDWARE_AGENT_MINIMUM_SDK_VERSION: ${VOLC_HARDWARE_AGENT_MINIMUM_SDK_VERSION:-1.0.0}
VOLC_HARDWARE_PRODUCT_SECRET: ""
VOLC_HARDWARE_PRODUCT_SECRET_FILE: ${VOLC_HARDWARE_PRODUCT_SECRET_FILE}
VOLC_HARDWARE_DEVICE_CREDENTIAL_MASTER_KEY: ${VOLC_HARDWARE_DEVICE_CREDENTIAL_MASTER_KEY}
services:
api:
image: data-terminal-cloud-backend:${ONBOARDING_RELEASE_VERSION:?set release version}
environment:
<<: *onboarding-runtime
worker:
image: data-terminal-cloud-backend:${ONBOARDING_RELEASE_VERSION:?set release version}
environment:
<<: *onboarding-runtime
web:
image: data-terminal-cloud-web:${ONBOARDING_RELEASE_VERSION:?set release version}
+46 -6
View File
@@ -19,8 +19,15 @@ x-server-backend-environment: &server-backend-environment
OBJECT_STORE_PUBLIC_SECURE: "true" OBJECT_STORE_PUBLIC_SECURE: "true"
CURSOR_SIGNING_SECRET: ${CURSOR_SIGNING_SECRET} CURSOR_SIGNING_SECRET: ${CURSOR_SIGNING_SECRET}
SHARE_TOKEN_SIGNING_SECRET: ${SHARE_TOKEN_SIGNING_SECRET} SHARE_TOKEN_SIGNING_SECRET: ${SHARE_TOKEN_SIGNING_SECRET}
MEMORY_TOOL_BASE_URL: ${PUBLIC_ORIGIN}/internal/v1/tools/memory-search
MEMORY_TOOL_SIGNING_SECRET: ${MEMORY_TOOL_SIGNING_SECRET}
MODEL_CREDENTIAL_MASTER_KEY: ${MODEL_CREDENTIAL_MASTER_KEY}
VIKING_MEMORY_RESOURCE_ID: ${VIKING_MEMORY_RESOURCE_ID:-}
VIKING_MEMORY_COLLECTION_NAME: ${VIKING_MEMORY_COLLECTION_NAME:-}
VIKING_MEMORY_EVENT_TYPE: ${VIKING_MEMORY_EVENT_TYPE:-event_v1}
VIKING_MEMORY_PROFILE_TYPE: ${VIKING_MEMORY_PROFILE_TYPE:-profile_v1}
EMBEDDING_PROVIDER: local EMBEDDING_PROVIDER: local
VOICE_PROVIDER: mock VOICE_PROVIDER: ${VOICE_PROVIDER:-mock}
services: services:
otel-collector: otel-collector:
@@ -92,10 +99,27 @@ services:
environment: environment:
<<: *server-backend-environment <<: *server-backend-environment
HTTP_ADDR: 0.0.0.0:8080 HTTP_ADDR: 0.0.0.0:8080
DEVICE_LOCAL_SECRET: ${DEVICE_LOCAL_SECRET} DEVICE_LOCAL_SECRET: ${DEVICE_LOCAL_SECRET:-}
DEVICE_CREDENTIAL_MASTER_KEY: ${DEVICE_CREDENTIAL_MASTER_KEY}
RTC_TOKEN_SIGNING_SECRET: ${RTC_TOKEN_SIGNING_SECRET} RTC_TOKEN_SIGNING_SECRET: ${RTC_TOKEN_SIGNING_SECRET}
RTC_TOKEN_ISSUER: local RTC_TOKEN_ISSUER: ${RTC_TOKEN_ISSUER:-local}
RTC_APP_ID: local-public-rtc-app RTC_APP_ID: ${RTC_APP_ID:-local-public-rtc-app}
RTC_APP_KEY: ""
RTC_APP_KEY_FILE: ${RTC_APP_KEY_FILE:-/run/secrets/voice-runtime/rtc-app-key}
VOLC_HARDWARE_PROVISIONING_ENABLED: ${VOLC_HARDWARE_PROVISIONING_ENABLED:-false}
VOLC_HARDWARE_INSTANCE_ID: ${VOLC_HARDWARE_INSTANCE_ID:-}
VOLC_HARDWARE_PRODUCT_KEY: ${VOLC_HARDWARE_PRODUCT_KEY:-}
VOLC_HARDWARE_AGENT_BOT_ID: ${VOLC_HARDWARE_AGENT_BOT_ID:-}
VOLC_HARDWARE_AGENT_TRANSPORT: ${VOLC_HARDWARE_AGENT_TRANSPORT:-low_load}
VOLC_HARDWARE_AGENT_MINIMUM_SDK_VERSION: ${VOLC_HARDWARE_AGENT_MINIMUM_SDK_VERSION:-1.0.0}
VOLC_HARDWARE_PRODUCT_SECRET: ""
VOLC_HARDWARE_PRODUCT_SECRET_FILE: ${VOLC_HARDWARE_PRODUCT_SECRET_FILE:-}
VOLC_HARDWARE_DEVICE_CREDENTIAL_MASTER_KEY: ${VOLC_HARDWARE_DEVICE_CREDENTIAL_MASTER_KEY:-}
VOLC_HARDWARE_DEVICE_NAME_PREFIX: ${VOLC_HARDWARE_DEVICE_NAME_PREFIX:-dt-}
VIKING_MEMORY_API_KEY: ""
VIKING_MEMORY_API_KEY_FILE: ${VIKING_MEMORY_API_KEY_FILE:-/run/secrets/voice-runtime/viking_memory_api_key}
volumes:
- /etc/data-terminal-cloud/voice-runtime:/run/secrets/voice-runtime:ro
restart: unless-stopped restart: unless-stopped
callbacks: callbacks:
@@ -106,6 +130,7 @@ services:
<<: *server-backend-environment <<: *server-backend-environment
HTTP_ADDR: 0.0.0.0:8081 HTTP_ADDR: 0.0.0.0:8081
PROVIDER_CALLBACK_SECRET: ${PROVIDER_CALLBACK_SECRET} PROVIDER_CALLBACK_SECRET: ${PROVIDER_CALLBACK_SECRET}
PLATFORM_HEALTH_CALLBACK_SECRET: ${PLATFORM_HEALTH_CALLBACK_SECRET}
RATE_LIMIT_RPS: "1000" RATE_LIMIT_RPS: "1000"
RATE_LIMIT_BURST: "1500" RATE_LIMIT_BURST: "1500"
restart: unless-stopped restart: unless-stopped
@@ -121,8 +146,23 @@ services:
MOCK_PROVIDER_API_KEY: ${MOCK_PROVIDER_API_KEY} MOCK_PROVIDER_API_KEY: ${MOCK_PROVIDER_API_KEY}
ALERT_WEBHOOK_URL: http://mock-alert-receiver:8084/webhooks/alerts ALERT_WEBHOOK_URL: http://mock-alert-receiver:8084/webhooks/alerts
ALERT_WEBHOOK_SECRET: ${ALERT_WEBHOOK_SECRET} ALERT_WEBHOOK_SECRET: ${ALERT_WEBHOOK_SECRET}
CONTENT_GENERATOR: local RTC_APP_ID: ${RTC_APP_ID:-local-public-rtc-app}
VOLC_ACCESS_KEY_ID: ""
VOLC_ACCESS_KEY_ID_FILE: ${VOLC_ACCESS_KEY_ID_FILE:-/run/secrets/voice-runtime/volc-access-key-id}
VOLC_SECRET_ACCESS_KEY: ""
VOLC_SECRET_ACCESS_KEY_FILE: ${VOLC_SECRET_ACCESS_KEY_FILE:-/run/secrets/voice-runtime/volc-secret-access-key}
VOLC_RTC_REGION: ${VOLC_RTC_REGION:-cn-north-1}
VOLC_RTC_API_VERSION: ${VOLC_RTC_API_VERSION:-2024-12-01}
VOLC_VOICE_CHAT_TEMPLATE_FILE: ${VOLC_VOICE_CHAT_TEMPLATE_FILE:-/run/secrets/voice-runtime/voice-chat-template.json}
CONTENT_GENERATOR: ${CONTENT_GENERATOR:-local}
ARK_API_KEY: ""
ARK_API_KEY_FILE: ${ARK_API_KEY_FILE:-/run/secrets/voice-runtime/ark-api-key}
ARK_CONTENT_MODEL: ${ARK_CONTENT_MODEL:-}
CONTENT_SAFETY_PROVIDER: local CONTENT_SAFETY_PROVIDER: local
VIKING_MEMORY_API_KEY: ""
VIKING_MEMORY_API_KEY_FILE: ${VIKING_MEMORY_API_KEY_FILE:-/run/secrets/voice-runtime/viking_memory_api_key}
volumes:
- /etc/data-terminal-cloud/voice-runtime:/run/secrets/voice-runtime:ro
restart: unless-stopped restart: unless-stopped
mock-provider: mock-provider:
@@ -159,7 +199,7 @@ services:
gateway: gateway:
image: nginx:1.30-alpine image: nginx:1.30-alpine
ports: ports:
- "80:443" - "80:80"
- "443:443" - "443:443"
volumes: volumes:
- ./deploy/server/nginx.conf:/etc/nginx/conf.d/default.conf:ro - ./deploy/server/nginx.conf:/etc/nginx/conf.d/default.conf:ro
+391
View File
@@ -0,0 +1,391 @@
#!/usr/bin/env bash
set -Eeuo pipefail
project_root=${PLATFORM_ROOT:-/opt/data-terminal-cloud/current}
backup_root=${BACKUP_ROOT:-/var/backups/data-terminal-cloud}
state_dir=${MONITOR_STATE_DIR:-/var/lib/data-terminal-cloud-monitor}
monitor_config=${MONITOR_CONFIG_FILE:-/etc/data-terminal-cloud/monitor.env}
offsite_config_file=${OFFSITE_CONFIG_FILE:-/etc/data-terminal-cloud/offsite-backup.env}
backup_max_age_seconds=${BACKUP_MAX_AGE_SECONDS:-129600}
certificate_min_seconds=${CERTIFICATE_MIN_SECONDS:-129600}
disk_warning_percent=${DISK_WARNING_PERCENT:-80}
disk_critical_percent=${DISK_CRITICAL_PERCENT:-90}
if [[ -f "$monitor_config" ]]; then
# The installer creates this file as root:root 0600.
# shellcheck disable=SC1090
source "$monitor_config"
fi
alert_webhook_url=${ALERT_WEBHOOK_URL:-}
alert_webhook_secret_file=${ALERT_WEBHOOK_SECRET_FILE:-/etc/data-terminal-cloud/monitor-webhook-secret}
platform_health_report_url=${PLATFORM_HEALTH_REPORT_URL:-}
platform_health_report_secret_file=${PLATFORM_HEALTH_REPORT_SECRET_FILE:-/etc/data-terminal-cloud/platform-health-callback-secret}
expected_public_tcp_ports=${EXPECTED_PUBLIC_TCP_PORTS:-22,80,443}
deployment_profile=${HOST_DEPLOYMENT_PROFILE:-demo}
if [[ -f "$offsite_config_file" ]]; then
# shellcheck disable=SC1090
source "$offsite_config_file"
fi
offsite_enabled=${OFFSITE_BACKUP_ENABLED:-false}
offsite_required=${OFFSITE_BACKUP_REQUIRED:-false}
offsite_state_file=${OFFSITE_STATE_FILE:-/var/lib/data-terminal-cloud-offsite/status.json}
offsite_max_age_seconds=${OFFSITE_MAX_AGE_SECONDS:-129600}
for numeric in "$backup_max_age_seconds" "$certificate_min_seconds" "$disk_warning_percent" "$disk_critical_percent" "$offsite_max_age_seconds"; do
[[ "$numeric" =~ ^[0-9]+$ ]] || { echo "Monitor thresholds must be non-negative integers" >&2; exit 2; }
done
if (( disk_warning_percent >= disk_critical_percent || disk_critical_percent > 100 )); then
echo "Disk thresholds are invalid" >&2
exit 2
fi
if [[ ! "$expected_public_tcp_ports" =~ ^[0-9]+(,[0-9]+)*$ ]]; then
echo "EXPECTED_PUBLIC_TCP_PORTS must contain comma-separated integers" >&2
exit 2
fi
if [[ "$deployment_profile" != demo && "$deployment_profile" != production ]]; then
echo "HOST_DEPLOYMENT_PROFILE must be demo or production" >&2
exit 2
fi
compose=(docker compose -f "$project_root/compose.yaml" -f "$project_root/deploy/server/compose.override.yaml" --env-file "$project_root/deploy/server/.env")
mkdir -p "$state_dir"
chmod 0750 "$state_dir"
umask 077
checks_file=$(mktemp "$state_dir/.checks-XXXXXX")
status_tmp=$(mktemp "$state_dir/.status-XXXXXX")
oidc_tmp=$(mktemp "$state_dir/.oidc-XXXXXX")
curl_config=$(mktemp "$state_dir/.curl-XXXXXX")
status_file="$state_dir/status.json"
critical_count=0
warning_count=0
cleanup() {
rm -f -- "$checks_file" "$status_tmp" "$oidc_tmp" "$curl_config"
}
trap cleanup EXIT INT TERM
record_check() {
local name=$1
local status=$2
local value=${3:-}
value=${value//$'\t'/ }
value=${value//$'\r'/ }
value=${value//$'\n'/ }
printf '%s\t%s\t%s\n' "$name" "$status" "$value" >>"$checks_file"
case "$status" in
critical) critical_count=$((critical_count + 1)) ;;
warning) warning_count=$((warning_count + 1)) ;;
esac
}
record_check deployment_profile healthy "$deployment_profile"
if [[ -z "$alert_webhook_url" ]]; then
if [[ "$deployment_profile" == production ]]; then
record_check external_alert_receiver critical not_configured
else
record_check external_alert_receiver healthy not_configured
fi
elif [[ ! -r "$alert_webhook_secret_file" ]]; then
record_check external_alert_receiver critical secret_missing
elif (( $(tr -d '\r\n' <"$alert_webhook_secret_file" | wc -c) < 32 )); then
record_check external_alert_receiver critical secret_invalid
else
record_check external_alert_receiver healthy configured
fi
public_origin=$(sed -n 's/^PUBLIC_ORIGIN=//p' "$project_root/deploy/server/.env")
if [[ "$public_origin" == https://* ]]; then
record_check public_origin healthy "$public_origin"
else
record_check public_origin critical invalid_https_origin
fi
if curl --fail --silent --show-error --max-time 10 "$public_origin/api/v1/version" >/dev/null; then
record_check public_https healthy http_200
else
record_check public_https critical request_failed
fi
if ! command -v ss >/dev/null 2>&1; then
record_check public_tcp_listeners critical probe_unavailable
else
listener_evidence=$(ss -H -ltn | python3 "$project_root/deploy/server/public_listener_policy.py" "$expected_public_tcp_ports" 2>/dev/null || true)
IFS='|' read -r listener_status listener_detail <<<"$listener_evidence"
if [[ "$listener_status" == healthy ]]; then
record_check public_tcp_listeners healthy "$listener_detail"
elif [[ "$listener_status" == critical ]]; then
record_check public_tcp_listeners critical "$listener_detail"
else
record_check public_tcp_listeners critical invalid_probe_result
fi
fi
expected_issuer="$public_origin/realms/voice-local"
if curl --fail --silent --show-error --max-time 10 \
"$expected_issuer/.well-known/openid-configuration" -o "$oidc_tmp"; then
actual_issuer=$(python3 -c 'import json,sys; print(json.load(open(sys.argv[1], encoding="utf-8")).get("issuer", ""))' "$oidc_tmp" 2>/dev/null || true)
if [[ "$actual_issuer" == "$expected_issuer" ]]; then
record_check oidc_issuer healthy issuer_matches
else
record_check oidc_issuer critical issuer_mismatch
fi
else
record_check oidc_issuer critical request_failed
fi
required_services=(postgres redis minio keycloak api callbacks worker web gateway)
if [[ "$deployment_profile" == demo ]]; then
required_services+=(mock-provider mock-alert-receiver)
fi
for service in "${required_services[@]}"; do
container=$("${compose[@]}" ps -q "$service")
if [[ -z "$container" ]]; then
record_check "container_$service" critical missing
continue
fi
state=$(docker inspect -f '{{.State.Status}}' "$container")
health=$(docker inspect -f '{{if .State.Health}}{{.State.Health.Status}}{{else}}none{{end}}' "$container")
if [[ "$state" != running ]]; then
record_check "container_$service" critical "$state"
elif [[ "$health" != none && "$health" != healthy ]]; then
record_check "container_$service" critical "$health"
else
record_check "container_$service" healthy "$state/$health"
fi
done
application_environments=()
application_environment_missing=false
for application_service in api callbacks worker; do
application_container=$("${compose[@]}" ps -q "$application_service")
if [[ -z "$application_container" ]]; then
application_environment_missing=true
continue
fi
application_environment=$(docker inspect -f '{{range .Config.Env}}{{println .}}{{end}}' "$application_container" | sed -n 's/^APP_ENV=//p' | head -n1)
if [[ -z "$application_environment" ]]; then
application_environment_missing=true
else
application_environments+=("$application_environment")
fi
done
if [[ "$application_environment_missing" == true || ${#application_environments[@]} -ne 3 ]]; then
record_check application_environment critical missing
elif [[ "${application_environments[0]}" != "${application_environments[1]}" || "${application_environments[0]}" != "${application_environments[2]}" ]]; then
record_check application_environment critical mixed
elif [[ "$deployment_profile" == production && "${application_environments[0]}" != prod ]]; then
record_check application_environment critical "${application_environments[0]}"
elif [[ "$deployment_profile" == demo && "${application_environments[0]}" != local ]]; then
record_check application_environment critical "${application_environments[0]}"
else
record_check application_environment healthy "${application_environments[0]}"
fi
keycloak_container=$("${compose[@]}" ps -q keycloak)
if [[ -z "$keycloak_container" ]]; then
record_check identity_service_mode critical missing
elif docker inspect -f '{{json .Config.Cmd}}' "$keycloak_container" | grep -q 'start-dev'; then
if [[ "$deployment_profile" == production ]]; then
record_check identity_service_mode critical development
else
record_check identity_service_mode healthy development
fi
else
record_check identity_service_mode healthy hardened
fi
mock_services_running=()
for mock_service in mock-provider mock-alert-receiver; do
mock_container=$("${compose[@]}" ps -q "$mock_service")
if [[ -n "$mock_container" && $(docker inspect -f '{{.State.Status}}' "$mock_container") == running ]]; then
mock_services_running+=("$mock_service")
fi
done
if [[ "$deployment_profile" == production ]]; then
if (( ${#mock_services_running[@]} > 0 )); then
record_check mock_services critical unexpected_running
else
record_check mock_services healthy disabled
fi
elif (( ${#mock_services_running[@]} == 2 )); then
record_check mock_services healthy enabled_for_demo
else
record_check mock_services critical incomplete_demo_stack
fi
latest_backup=$(find "$backup_root" -maxdepth 1 -type f -name 'platform-*.tar.gz.enc' -printf '%T@ %p\n' 2>/dev/null | sort -nr | head -n1 | cut -d' ' -f2- || true)
if [[ -z "$latest_backup" || ! -f "$latest_backup" ]]; then
record_check backup_freshness critical missing
else
now_epoch=$(date +%s)
backup_epoch=$(stat -c %Y "$latest_backup")
backup_age=$((now_epoch - backup_epoch))
if (( backup_age > backup_max_age_seconds )); then
record_check backup_freshness critical "age_seconds=$backup_age"
elif [[ ! -f "$latest_backup.sha256" ]] || ! (cd "$backup_root" && sha256sum -c "$(basename "$latest_backup").sha256" >/dev/null 2>&1); then
record_check backup_freshness critical checksum_failed
else
record_check backup_freshness healthy "age_seconds=$backup_age"
fi
fi
if [[ "$offsite_enabled" == true || "$offsite_required" == true ]]; then
if [[ ! -r "$offsite_state_file" ]]; then
record_check offsite_backup critical status_missing
else
offsite_evidence=$(python3 -c 'import json,sys
try:
d=json.load(open(sys.argv[1], encoding="utf-8")); print("|".join([str(d.get("status", "")), str(d.get("verified_epoch", 0)), str(d.get("source_archive", ""))]))
except Exception: print("invalid|0|")' "$offsite_state_file")
IFS='|' read -r offsite_status offsite_epoch offsite_source <<<"$offsite_evidence"
offsite_age=$(($(date +%s) - offsite_epoch))
if [[ "$offsite_status" != verified ]]; then
record_check offsite_backup critical "status=${offsite_status:-invalid}"
elif [[ "$offsite_source" != "$(basename "${latest_backup:-missing}")" ]]; then
record_check offsite_backup critical source_mismatch
elif (( offsite_epoch <= 0 || offsite_age > offsite_max_age_seconds )); then
record_check offsite_backup critical "age_seconds=$offsite_age"
else
record_check offsite_backup healthy "age_seconds=$offsite_age"
fi
fi
else
if [[ "$deployment_profile" == production ]]; then
record_check offsite_backup critical not_required
else
record_check offsite_backup healthy not_required
fi
fi
backup_result=$(systemctl show platform-backup.service -p Result --value 2>/dev/null || true)
backup_exit=$(systemctl show platform-backup.service -p ExecMainStatus --value 2>/dev/null || true)
if [[ "$backup_result" == success && "$backup_exit" == 0 ]]; then
record_check backup_service healthy success
else
record_check backup_service critical "result=${backup_result:-unknown}/exit=${backup_exit:-unknown}"
fi
timers=(platform-backup.timer certbot-ip-renew.timer)
if [[ -f "$offsite_config_file" ]]; then
timers+=(platform-offsite-backup.timer)
fi
for timer in "${timers[@]}"; do
timer_enabled=$(systemctl is-enabled "$timer" 2>/dev/null || true)
timer_active=$(systemctl is-active "$timer" 2>/dev/null || true)
if [[ "$timer_enabled" == enabled && "$timer_active" == active ]]; then
record_check "$timer" healthy enabled/active
else
record_check "$timer" critical "${timer_enabled:-unknown}/${timer_active:-unknown}"
fi
done
public_hostname=$(python3 -c 'import sys,urllib.parse; print(urllib.parse.urlsplit(sys.argv[1]).hostname or "")' "$public_origin")
certificate_file=${TLS_CERTIFICATE_FILE:-/etc/letsencrypt/live/$public_hostname/fullchain.pem}
if [[ ! -r "$certificate_file" ]]; then
record_check certificate critical missing
else
certificate_end=$(openssl x509 -in "$certificate_file" -noout -enddate | cut -d= -f2-)
certificate_end_epoch=$(date -d "$certificate_end" +%s)
certificate_seconds=$((certificate_end_epoch - $(date +%s)))
if (( certificate_seconds < certificate_min_seconds )); then
record_check certificate critical "remaining_seconds=$certificate_seconds"
else
record_check certificate healthy "remaining_seconds=$certificate_seconds"
fi
fi
disk_used=$(df -P "$backup_root" | awk 'NR==2 {gsub(/%/, "", $5); print $5}')
if [[ ! "$disk_used" =~ ^[0-9]+$ ]]; then
record_check disk_usage critical unreadable
elif (( disk_used >= disk_critical_percent )); then
record_check disk_usage critical "used_percent=$disk_used"
elif (( disk_used >= disk_warning_percent )); then
record_check disk_usage warning "used_percent=$disk_used"
else
record_check disk_usage healthy "used_percent=$disk_used"
fi
if (( critical_count > 0 )); then
overall=critical
elif (( warning_count > 0 )); then
overall=warning
else
overall=healthy
fi
observed_at=$(date -u +%Y-%m-%dT%H:%M:%SZ)
previous_status=$(python3 -c 'import json,sys
try: print(json.load(open(sys.argv[1], encoding="utf-8")).get("status", "unknown"))
except Exception: print("unknown")' "$status_file" 2>/dev/null || true)
python3 - "$checks_file" "$status_tmp" "$overall" "$observed_at" <<'PY'
import json
import sys
checks = []
with open(sys.argv[1], encoding="utf-8") as source:
for line in source:
name, status, value = line.rstrip("\n").split("\t", 2)
checks.append({"name": name, "status": status, "value": value})
payload = {
"schema": "platform-server-health-v1",
"status": sys.argv[3],
"observed_at": sys.argv[4],
"checks": checks,
}
with open(sys.argv[2], "w", encoding="utf-8") as target:
json.dump(payload, target, ensure_ascii=False, separators=(",", ":"))
target.write("\n")
PY
chmod 0644 "$status_tmp"
mv -f "$status_tmp" "$status_file"
post_signed_status() {
local target_url=$1
local secret_file=$2
local timestamp signature
timestamp=$(date +%s)
signature=$(python3 - "$secret_file" "$status_file" "$timestamp" <<'PY'
import hashlib
import hmac
import sys
secret = open(sys.argv[1], "rb").read().rstrip(b"\r\n")
body = open(sys.argv[2], "rb").read()
print(hmac.new(secret, sys.argv[3].encode() + b"." + body, hashlib.sha256).hexdigest())
PY
)
python3 - "$target_url" "$curl_config" <<'PY'
import sys
url = sys.argv[1].replace("\\", "\\\\").replace('"', '\\"')
open(sys.argv[2], "w", encoding="utf-8").write(f'url = "{url}"\n')
PY
curl --fail --silent --show-error --max-time 10 --config "$curl_config" \
-H 'Content-Type: application/json' \
-H "X-Platform-Timestamp: $timestamp" \
-H "X-Platform-Signature: sha256=$signature" \
--data-binary "@$status_file" >/dev/null
}
if [[ -n "$platform_health_report_url" ]]; then
if [[ ! -r "$platform_health_report_secret_file" ]]; then
echo "Platform health reporting is configured but its HMAC secret is unreadable" >&2
exit 6
fi
post_signed_status "$platform_health_report_url" "$platform_health_report_secret_file"
fi
if [[ -n "$alert_webhook_url" && "$previous_status" != "$overall" ]]; then
if [[ ! -r "$alert_webhook_secret_file" ]]; then
record_check alert_delivery critical secret_missing
echo "Alert webhook is configured but its HMAC secret is unreadable" >&2
exit 5
fi
post_signed_status "$alert_webhook_url" "$alert_webhook_secret_file"
fi
printf 'platform_health status=%s critical=%s warning=%s status_file=%s\n' \
"$overall" "$critical_count" "$warning_count" "$status_file"
if (( critical_count > 0 )); then
exit 1
fi
+21
View File
@@ -0,0 +1,21 @@
#!/usr/bin/env bash
set -Eeuo pipefail
if [[ ${EUID:-$(id -u)} -ne 0 ]]; then
echo "Run this installer as root" >&2
exit 2
fi
project_root=${PLATFORM_ROOT:-/opt/data-terminal-cloud/current}
install -d -m 0700 /etc/data-terminal-cloud /var/backups/data-terminal-cloud
if [[ ! -s /etc/data-terminal-cloud/backup-passphrase ]]; then
umask 077
openssl rand -base64 48 >/etc/data-terminal-cloud/backup-passphrase
fi
chmod 0600 /etc/data-terminal-cloud/backup-passphrase
install -m 0644 "$project_root/deploy/server/platform-backup.service" /etc/systemd/system/platform-backup.service
install -m 0644 "$project_root/deploy/server/platform-backup.timer" /etc/systemd/system/platform-backup.timer
systemctl daemon-reload
systemctl enable --now platform-backup.timer
systemctl is-active --quiet platform-backup.timer
echo "platform-backup.timer enabled"
+48
View File
@@ -0,0 +1,48 @@
#!/usr/bin/env bash
set -Eeuo pipefail
if [[ ${EUID:-$(id -u)} -ne 0 ]]; then
echo "Run this installer as root" >&2
exit 2
fi
project_root=${PLATFORM_ROOT:-/opt/data-terminal-cloud/current}
install -d -m 0700 /etc/data-terminal-cloud
install -d -m 0750 /var/lib/data-terminal-cloud-monitor
platform_env="$project_root/deploy/server/.env"
if [[ -r "$platform_env" && ! -s /etc/data-terminal-cloud/platform-health-callback-secret ]]; then
platform_health_secret=$(sed -n 's/^PLATFORM_HEALTH_CALLBACK_SECRET=//p' "$platform_env" | head -n1)
if (( ${#platform_health_secret} >= 32 )); then
umask 077
printf '%s\n' "$platform_health_secret" >/etc/data-terminal-cloud/platform-health-callback-secret
fi
fi
if [[ -s /etc/data-terminal-cloud/platform-health-callback-secret ]]; then
chmod 0600 /etc/data-terminal-cloud/platform-health-callback-secret
fi
if [[ ! -s /etc/data-terminal-cloud/monitor-webhook-secret ]]; then
umask 077
openssl rand -base64 48 >/etc/data-terminal-cloud/monitor-webhook-secret
fi
chmod 0600 /etc/data-terminal-cloud/monitor-webhook-secret
if [[ ! -e /etc/data-terminal-cloud/monitor.env ]]; then
public_origin=$(sed -n 's/^PUBLIC_ORIGIN=//p' "$platform_env" 2>/dev/null | head -n1)
report_url=
if [[ "$public_origin" == https://* && -s /etc/data-terminal-cloud/platform-health-callback-secret ]]; then
report_url="$public_origin/callbacks/v1/platform-health"
fi
cat >/etc/data-terminal-cloud/monitor.env <<EOF
HOST_DEPLOYMENT_PROFILE=demo
PLATFORM_HEALTH_REPORT_URL=$report_url
PLATFORM_HEALTH_REPORT_SECRET_FILE=/etc/data-terminal-cloud/platform-health-callback-secret
ALERT_WEBHOOK_URL=
ALERT_WEBHOOK_SECRET_FILE=/etc/data-terminal-cloud/monitor-webhook-secret
EOF
fi
chmod 0600 /etc/data-terminal-cloud/monitor.env
install -m 0644 "$project_root/deploy/server/platform-health-monitor.service" /etc/systemd/system/platform-health-monitor.service
install -m 0644 "$project_root/deploy/server/platform-health-monitor.timer" /etc/systemd/system/platform-health-monitor.timer
systemctl daemon-reload
systemctl enable --now platform-health-monitor.timer
systemctl is-active --quiet platform-health-monitor.timer
echo "platform-health-monitor.timer enabled"
+31
View File
@@ -0,0 +1,31 @@
#!/usr/bin/env bash
set -Eeuo pipefail
if [[ ${EUID:-$(id -u)} -ne 0 ]]; then
echo "Run this installer as root" >&2
exit 2
fi
project_root=${PLATFORM_ROOT:-/opt/data-terminal-cloud/current}
"$project_root/deploy/server/install-ossutil.sh"
install -d -m 0700 /etc/data-terminal-cloud
install -d -m 0750 /var/lib/data-terminal-cloud-offsite
if [[ ! -e /etc/data-terminal-cloud/offsite-backup.env ]]; then
cat >/etc/data-terminal-cloud/offsite-backup.env <<'EOF'
OFFSITE_BACKUP_ENABLED=false
OFFSITE_BACKUP_REQUIRED=false
OFFSITE_OSS_BUCKET=
OFFSITE_OSS_PREFIX=data-terminal-cloud/121.41.46.238
OFFSITE_OSS_AUTH_MODE=ecs_ram_role
OFFSITE_OSS_REGION=cn-hangzhou
OFFSITE_OSS_ENDPOINT=oss-cn-hangzhou.aliyuncs.com
OFFSITE_OSS_CONFIG_FILE=/etc/data-terminal-cloud/ossutilconfig
EOF
fi
chmod 0600 /etc/data-terminal-cloud/offsite-backup.env
install -m 0644 "$project_root/deploy/server/platform-offsite-backup.service" /etc/systemd/system/platform-offsite-backup.service
install -m 0644 "$project_root/deploy/server/platform-offsite-backup.timer" /etc/systemd/system/platform-offsite-backup.timer
systemctl daemon-reload
systemctl enable --now platform-offsite-backup.timer
systemctl is-active --quiet platform-offsite-backup.timer
echo "platform-offsite-backup.timer enabled; upload remains disabled until OSS identity and bucket are configured"
+36
View File
@@ -0,0 +1,36 @@
#!/usr/bin/env bash
set -Eeuo pipefail
if [[ ${EUID:-$(id -u)} -ne 0 ]]; then
echo "Run this installer as root" >&2
exit 2
fi
version=2.3.0
archive=ossutil-${version}-linux-amd64.zip
expected_sha256=3ae4d9fc85a7a6e9f5654d1599766f1a3a42a3692870887b5ae9338d582ef65a
download_url="https://gosspublic.alicdn.com/ossutil/v2/${version}/${archive}"
if command -v ossutil >/dev/null 2>&1 && [[ $(ossutil version 2>/dev/null) == "$version" ]]; then
echo "ossutil $version already installed"
exit 0
fi
install_dir=$(mktemp -d /var/tmp/ossutil-install-XXXXXX)
case "$install_dir" in /var/tmp/ossutil-install-*) ;; *) echo "Unsafe install directory" >&2; exit 9;; esac
trap 'rm -rf -- "$install_dir"' EXIT
curl -fsSL --max-time 120 -o "$install_dir/$archive" "$download_url"
printf '%s %s\n' "$expected_sha256" "$install_dir/$archive" | sha256sum -c -
mkdir -p "$install_dir/unpacked"
if command -v unzip >/dev/null 2>&1; then
unzip -q "$install_dir/$archive" -d "$install_dir/unpacked"
elif command -v python3 >/dev/null 2>&1; then
python3 -m zipfile -e "$install_dir/$archive" "$install_dir/unpacked"
else
echo "Neither unzip nor python3 is available to extract ossutil" >&2
exit 3
fi
binary=$(find "$install_dir/unpacked" -type f -name ossutil -print -quit)
[[ -n "$binary" ]] || { echo "ossutil binary is missing from the verified archive" >&2; exit 4; }
install -m 0755 "$binary" /usr/local/bin/ossutil
[[ $(/usr/local/bin/ossutil version) == "$version" ]]
echo "ossutil $version installed"
+23 -1
View File
@@ -1,7 +1,21 @@
server { server {
listen 80; listen 80;
server_name _; server_name _;
return 301 https://121.41.46.238$request_uri; # First-boot clock bootstrap only. The response is authenticated by TIME1.
location = /api/v1/device/time-challenges {
limit_except POST { deny all; }
client_max_body_size 8k;
resolver 127.0.0.11 ipv6=off valid=10s;
set $clock_backend http://api:8080;
proxy_pass $clock_backend;
proxy_set_header Host $http_host;
proxy_set_header X-Forwarded-Proto http;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_buffering off;
}
location / {
return 301 https://121.41.46.238$request_uri;
}
} }
server { server {
@@ -44,6 +58,14 @@ server {
proxy_buffering off; proxy_buffering off;
} }
# The voice provider can call only this signed, conversation-scoped tool.
# Do not expose the rest of the /internal namespace.
location = /internal/v1/tools/memory-search {
proxy_pass $api_backend;
proxy_buffering off;
proxy_request_buffering off;
}
location /callbacks/ { location /callbacks/ {
proxy_pass $callbacks_backend; proxy_pass $callbacks_backend;
proxy_buffering off; proxy_buffering off;
+19
View File
@@ -0,0 +1,19 @@
[Unit]
Description=Encrypted data terminal cloud backup and restore verification
Requires=docker.service
After=docker.service network-online.target
[Service]
Type=oneshot
User=root
Group=root
UMask=0077
Environment=PLATFORM_ROOT=/opt/data-terminal-cloud/current
Environment=BACKUP_ROOT=/var/backups/data-terminal-cloud
Environment=BACKUP_RETENTION_DAYS=14
Environment=VERIFY_AFTER_BACKUP=true
ExecStart=/opt/data-terminal-cloud/current/deploy/server/backup.sh
Nice=10
IOSchedulingClass=best-effort
IOSchedulingPriority=7
TimeoutStartSec=2h
+11
View File
@@ -0,0 +1,11 @@
[Unit]
Description=Run encrypted data terminal cloud backup daily
[Timer]
OnCalendar=*-*-* 02:40:00
RandomizedDelaySec=45m
Persistent=true
Unit=platform-backup.service
[Install]
WantedBy=timers.target
@@ -0,0 +1,18 @@
[Unit]
Description=Check data terminal cloud server health
Requires=docker.service
After=docker.service network-online.target
[Service]
Type=oneshot
User=root
Group=root
UMask=0077
Environment=PLATFORM_ROOT=/opt/data-terminal-cloud/current
Environment=BACKUP_ROOT=/var/backups/data-terminal-cloud
Environment=MONITOR_STATE_DIR=/var/lib/data-terminal-cloud-monitor
ExecStart=/opt/data-terminal-cloud/current/deploy/server/health-monitor.sh
Nice=10
NoNewPrivileges=true
PrivateTmp=true
TimeoutStartSec=2m
@@ -0,0 +1,12 @@
[Unit]
Description=Check data terminal cloud server health every five minutes
[Timer]
OnBootSec=2m
OnUnitActiveSec=5m
RandomizedDelaySec=30s
Persistent=true
Unit=platform-health-monitor.service
[Install]
WantedBy=timers.target
@@ -0,0 +1,18 @@
[Unit]
Description=Copy encrypted data terminal cloud backup to OSS
Requires=network-online.target
After=network-online.target platform-backup.service
[Service]
Type=oneshot
User=root
Group=root
UMask=0077
Environment=BACKUP_ROOT=/var/backups/data-terminal-cloud
Environment=OFFSITE_STATE_DIR=/var/lib/data-terminal-cloud-offsite
Environment=OFFSITE_CONFIG_FILE=/etc/data-terminal-cloud/offsite-backup.env
ExecStart=/opt/data-terminal-cloud/current/deploy/server/sync-backup-offsite.sh
Nice=10
NoNewPrivileges=true
PrivateTmp=true
TimeoutStartSec=4h
@@ -0,0 +1,11 @@
[Unit]
Description=Copy encrypted data terminal cloud backup to OSS daily
[Timer]
OnCalendar=*-*-* 04:10:00
RandomizedDelaySec=30m
Persistent=true
Unit=platform-offsite-backup.service
[Install]
WantedBy=timers.target
+70
View File
@@ -0,0 +1,70 @@
#!/usr/bin/env python3
import ipaddress
import re
import sys
def parse_endpoint(value: str) -> tuple[str, int] | None:
value = value.strip()
if value.startswith("[") and "]:" in value:
address, port_text = value[1:].rsplit("]:", 1)
elif ":" in value:
address, port_text = value.rsplit(":", 1)
else:
return None
if not port_text.isdigit():
return None
return address, int(port_text)
def externally_reachable(address: str) -> bool:
if address in {"*", "0.0.0.0", "::", "[::]"}:
return True
try:
return not ipaddress.ip_address(address.split("%", 1)[0]).is_loopback
except ValueError:
return True
def public_ports(lines: list[str]) -> set[int]:
ports: set[int] = set()
for line in lines:
fields = line.split()
if len(fields) < 4:
continue
endpoint = parse_endpoint(fields[3])
if endpoint is not None and externally_reachable(endpoint[0]):
ports.add(endpoint[1])
return ports
def parse_allowed(value: str) -> set[int]:
if not re.fullmatch(r"[0-9]+(?:,[0-9]+)*", value):
raise ValueError("allowed ports must be comma-separated integers")
ports = {int(item) for item in value.split(",")}
if any(port < 1 or port > 65535 for port in ports):
raise ValueError("allowed port is outside 1..65535")
return ports
def main() -> int:
if len(sys.argv) != 2:
print("usage: public_listener_policy.py ALLOWED_PORTS", file=sys.stderr)
return 2
try:
allowed = parse_allowed(sys.argv[1])
except ValueError as error:
print(str(error), file=sys.stderr)
return 2
exposed = public_ports(list(sys.stdin))
unexpected = sorted(exposed - allowed)
if unexpected:
print("critical|unexpected_ports=" + ",".join(map(str, unexpected)))
else:
detail = ",".join(map(str, sorted(exposed))) or "none"
print("healthy|ports=" + detail)
return 0
if __name__ == "__main__":
raise SystemExit(main())
+158
View File
@@ -0,0 +1,158 @@
#!/usr/bin/env bash
set -Eeuo pipefail
backup_root=${BACKUP_ROOT:-/var/backups/data-terminal-cloud}
state_dir=${OFFSITE_STATE_DIR:-/var/lib/data-terminal-cloud-offsite}
offsite_config_file=${OFFSITE_CONFIG_FILE:-/etc/data-terminal-cloud/offsite-backup.env}
ossutil_bin=${OSSUTIL_BIN:-/usr/local/bin/ossutil}
mkdir -p "$state_dir"
chmod 0750 "$state_dir"
umask 077
status_file="$state_dir/status.json"
status_tmp=$(mktemp "$state_dir/.status-XXXXXX")
transfer_dir=$(mktemp -d "$state_dir/.transfer-XXXXXX")
cleanup() {
rm -f -- "$status_tmp"
rm -rf -- "$transfer_dir"
}
trap cleanup EXIT INT TERM
write_status() {
local status=$1
local reason=$2
local source_archive=${3:-}
local object_ref=${4:-}
local sha256=${5:-}
local size_bytes=${6:-0}
local verified_epoch=0
if [[ "$status" == verified ]]; then
verified_epoch=$(date +%s)
fi
python3 - "$status_tmp" "$status" "$reason" "$source_archive" "$object_ref" "$sha256" "$size_bytes" "$verified_epoch" <<'PY'
import datetime
import json
import sys
payload = {
"schema": "platform-offsite-backup-v1",
"status": sys.argv[2],
"reason": sys.argv[3],
"source_archive": sys.argv[4],
"object_ref": sys.argv[5],
"sha256": sys.argv[6],
"size_bytes": int(sys.argv[7]),
"verified_epoch": int(sys.argv[8]),
"observed_at": datetime.datetime.now(datetime.timezone.utc).replace(microsecond=0).isoformat().replace("+00:00", "Z"),
}
with open(sys.argv[1], "w", encoding="utf-8") as target:
json.dump(payload, target, ensure_ascii=False, separators=(",", ":"))
target.write("\n")
PY
chmod 0644 "$status_tmp"
mv -f "$status_tmp" "$status_file"
}
fail_sync() {
local reason=$1
local code=${2:-1}
write_status failed "$reason" "${source_name:-}" "${remote_archive:-}" "${source_sha:-}" "${source_size:-0}"
echo "offsite_backup status=failed reason=$reason" >&2
exit "$code"
}
if [[ ! -f "$offsite_config_file" ]]; then
write_status disabled config_missing
echo "offsite_backup status=disabled reason=config_missing"
exit 0
fi
# The installer creates this file as root:root 0600.
# shellcheck disable=SC1090
source "$offsite_config_file"
offsite_enabled=${OFFSITE_BACKUP_ENABLED:-false}
bucket=${OFFSITE_OSS_BUCKET:-}
prefix=${OFFSITE_OSS_PREFIX:-data-terminal-cloud}
auth_mode=${OFFSITE_OSS_AUTH_MODE:-config}
oss_region=${OFFSITE_OSS_REGION:-}
oss_endpoint=${OFFSITE_OSS_ENDPOINT:-}
oss_config=${OFFSITE_OSS_CONFIG_FILE:-/etc/data-terminal-cloud/ossutilconfig}
ecs_metadata_base=${ECS_METADATA_BASE_URL:-http://100.100.100.200/latest}
if [[ "$offsite_enabled" != true ]]; then
write_status disabled not_enabled
echo "offsite_backup status=disabled reason=not_enabled"
exit 0
fi
[[ "$bucket" =~ ^[a-z0-9][a-z0-9-]{1,61}[a-z0-9]$ ]] || fail_sync invalid_bucket 2
[[ "$prefix" =~ ^[A-Za-z0-9._/-]+$ && "$prefix" != /* && "$prefix" != *..* ]] || fail_sync invalid_prefix 2
[[ -x "$ossutil_bin" ]] || fail_sync ossutil_missing 3
checkpoint_dir="$state_dir/checkpoints"
common=(-f --job 1 --parallel 1 --checkpoint-dir "$checkpoint_dir" --output-format json)
case "$auth_mode" in
config)
[[ -r "$oss_config" ]] || fail_sync oss_config_missing 4
common+=(-c "$oss_config")
;;
ecs_ram_role)
command -v curl >/dev/null 2>&1 || fail_sync curl_missing 3
[[ "$oss_region" =~ ^[a-z0-9][a-z0-9-]{1,62}$ ]] || fail_sync invalid_oss_region 2
if [[ -z "$oss_endpoint" ]]; then
oss_endpoint="oss-$oss_region.aliyuncs.com"
fi
[[ "$oss_endpoint" =~ ^[a-z0-9][a-z0-9.-]{1,251}[a-z0-9]$ && "$oss_endpoint" != *..* ]] || fail_sync invalid_oss_endpoint 2
metadata_token=$(curl --silent --show-error --fail --max-time 3 -X PUT \
-H 'X-aliyun-ecs-metadata-token-ttl-seconds: 60' "$ecs_metadata_base/api/token" || true)
[[ -n "$metadata_token" ]] || fail_sync ecs_metadata_v2_unavailable 4
role_response="$transfer_dir/ecs-role-name"
role_code=$(curl --silent --show-error --max-time 3 --output "$role_response" --write-out '%{http_code}' \
-H "X-aliyun-ecs-metadata-token: $metadata_token" "$ecs_metadata_base/meta-data/ram/security-credentials/" || true)
role_name=$(tr -d '\r\n' <"$role_response")
[[ "$role_code" == 200 && "$role_name" =~ ^[A-Za-z0-9@._-]{1,64}$ ]] || fail_sync ecs_ram_role_missing 4
common+=(--mode EcsRamRole --region "$oss_region" --endpoint "$oss_endpoint")
;;
*)
fail_sync invalid_auth_mode 2
;;
esac
latest_backup=$(find "$backup_root" -maxdepth 1 -type f -name 'platform-*.tar.gz.enc' -printf '%T@ %p\n' 2>/dev/null | sort -nr | head -n1 | cut -d' ' -f2- || true)
[[ -n "$latest_backup" && -f "$latest_backup" ]] || fail_sync local_backup_missing 5
[[ -f "$latest_backup.sha256" ]] || fail_sync local_checksum_missing 5
if ! (cd "$backup_root" && sha256sum -c "$(basename "$latest_backup").sha256" >/dev/null 2>&1); then
fail_sync local_checksum_failed 5
fi
source_name=$(basename "$latest_backup")
source_sha=$(sha256sum "$latest_backup" | awk '{print $1}')
source_size=$(stat -c %s "$latest_backup")
prefix=${prefix%/}
remote_archive="oss://$bucket/$prefix/$source_name"
remote_checksum="$remote_archive.sha256"
mkdir -p "$checkpoint_dir"
if ! "$ossutil_bin" cp "$latest_backup" "$remote_archive" "${common[@]}" --metadata "sha256=$source_sha" >/dev/null; then
fail_sync archive_upload_failed 6
fi
if ! "$ossutil_bin" cp "$latest_backup.sha256" "$remote_checksum" "${common[@]}" >/dev/null; then
fail_sync checksum_upload_failed 6
fi
downloaded="$transfer_dir/$source_name"
downloaded_checksum="$downloaded.sha256"
if ! "$ossutil_bin" cp "$remote_archive" "$downloaded" "${common[@]}" >/dev/null; then
fail_sync archive_download_verification_failed 7
fi
if ! "$ossutil_bin" cp "$remote_checksum" "$downloaded_checksum" "${common[@]}" >/dev/null; then
fail_sync checksum_download_verification_failed 7
fi
downloaded_sha=$(sha256sum "$downloaded" | awk '{print $1}')
checksum_sha=$(awk 'NF >= 1 {print $1; exit}' "$downloaded_checksum")
if [[ "$downloaded_sha" != "$source_sha" || "$checksum_sha" != "$source_sha" ]]; then
fail_sync remote_checksum_mismatch 8
fi
write_status verified upload_and_download_verified "$source_name" "$remote_archive" "$source_sha" "$source_size"
printf 'offsite_backup status=verified source=%s object=%s size_bytes=%s\n' "$source_name" "$remote_archive" "$source_size"
+121
View File
@@ -0,0 +1,121 @@
#!/usr/bin/env bash
set -Eeuo pipefail
project_root=$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)
candidate="$project_root/deploy/server/sync-backup-offsite.sh"
bash -n "$candidate"
test_root=$(mktemp -d /var/tmp/offsite-backup-test-XXXXXX)
case "$test_root" in /var/tmp/offsite-backup-test-*) ;; *) exit 90;; esac
metadata_pid=
cleanup() {
if [[ -n "$metadata_pid" ]]; then kill "$metadata_pid" 2>/dev/null || true; fi
rm -rf -- "$test_root"
}
trap cleanup EXIT INT TERM
mkdir -p "$test_root/backups" "$test_root/state" "$test_root/mock-oss"
role_mode_file="$test_root/role-mode"
port_file="$test_root/metadata-port"
printf 'missing\n' >"$role_mode_file"
python3 - "$port_file" "$role_mode_file" <<'PY' &
from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
import pathlib,sys
port_file=pathlib.Path(sys.argv[1])
role_mode=pathlib.Path(sys.argv[2])
class Handler(BaseHTTPRequestHandler):
def log_message(self,*args): pass
def do_PUT(self):
if self.path!='/latest/api/token': self.send_error(404); return
self.send_response(200); self.end_headers(); self.wfile.write(b'test-imdsv2-token')
def do_GET(self):
if self.path!='/latest/meta-data/ram/security-credentials/': self.send_error(404); return
if role_mode.read_text(encoding='utf-8').strip()!='attached': self.send_error(404); return
if self.headers.get('X-aliyun-ecs-metadata-token')!='test-imdsv2-token': self.send_error(401); return
self.send_response(200); self.end_headers(); self.wfile.write(b'DataTerminalCloudOssRole')
server=ThreadingHTTPServer(('127.0.0.1',0),Handler)
port_file.write_text(str(server.server_port),encoding='utf-8')
server.serve_forever()
PY
metadata_pid=$!
for _ in $(seq 1 50); do [[ -s "$port_file" ]] && break; sleep 0.1; done
[[ -s "$port_file" ]]
metadata_base="http://127.0.0.1:$(<"$port_file")/latest"
cat >"$test_root/enabled-role.env" <<'EOF'
OFFSITE_BACKUP_ENABLED=true
OFFSITE_BACKUP_REQUIRED=true
OFFSITE_OSS_BUCKET=valid-private-backup-bucket
OFFSITE_OSS_PREFIX=data-terminal-cloud/test
OFFSITE_OSS_AUTH_MODE=ecs_ram_role
OFFSITE_OSS_REGION=cn-hangzhou
OFFSITE_OSS_ENDPOINT=oss-cn-hangzhou.aliyuncs.com
EOF
set +e
ECS_METADATA_BASE_URL="$metadata_base" OFFSITE_CONFIG_FILE="$test_root/enabled-role.env" OFFSITE_STATE_DIR="$test_root/state" BACKUP_ROOT="$test_root/backups" \
"$candidate" >/dev/null 2>/dev/null
result=$?
set -e
test "$result" = 4
python3 - "$test_root/state/status.json" <<'PY'
import json,sys
d=json.load(open(sys.argv[1],encoding='utf-8'))
assert d['status']=='failed' and d['reason']=='ecs_ram_role_missing',d
PY
cat >"$test_root/disabled.env" <<'EOF'
OFFSITE_BACKUP_ENABLED=false
EOF
rm -f "$test_root/state/status.json"
OFFSITE_CONFIG_FILE="$test_root/disabled.env" OFFSITE_STATE_DIR="$test_root/state" BACKUP_ROOT="$test_root/backups" "$candidate" >/dev/null
printf 'encrypted-backup-fixture\n' >"$test_root/backups/platform-20260826T140000Z.tar.gz.enc"
(cd "$test_root/backups" && sha256sum platform-20260826T140000Z.tar.gz.enc >platform-20260826T140000Z.tar.gz.enc.sha256)
cat >"$test_root/mock-ossutil" <<'EOF'
#!/usr/bin/env bash
set -Eeuo pipefail
printf '%s\n' "$*" >>"$MOCK_OSS_LOG"
test "$1" = cp
source_path=$2
target_path=$3
map_path() {
if [[ "$1" == oss://* ]]; then printf '%s/%s' "$MOCK_OSS_ROOT" "${1#oss://}"; else printf '%s' "$1"; fi
}
source_local=$(map_path "$source_path")
target_local=$(map_path "$target_path")
mkdir -p "$(dirname "$target_local")"
cp "$source_local" "$target_local"
EOF
chmod 0700 "$test_root/mock-ossutil"
printf '[default]\nregion=cn-hangzhou\n' >"$test_root/ossutilconfig"
cat >"$test_root/config-mode.env" <<EOF
OFFSITE_BACKUP_ENABLED=true
OFFSITE_BACKUP_REQUIRED=true
OFFSITE_OSS_BUCKET=valid-private-backup-bucket
OFFSITE_OSS_PREFIX=data-terminal-cloud/config-test
OFFSITE_OSS_AUTH_MODE=config
OFFSITE_OSS_CONFIG_FILE=$test_root/ossutilconfig
EOF
rm -f "$test_root/state/status.json" "$test_root/ossutil.log"
MOCK_OSS_ROOT="$test_root/mock-oss" MOCK_OSS_LOG="$test_root/ossutil.log" OSSUTIL_BIN="$test_root/mock-ossutil" \
OFFSITE_CONFIG_FILE="$test_root/config-mode.env" OFFSITE_STATE_DIR="$test_root/state" BACKUP_ROOT="$test_root/backups" "$candidate" >/dev/null
python3 - "$test_root/state/status.json" <<'PY'
import json,sys
d=json.load(open(sys.argv[1],encoding='utf-8'))
assert d['status']=='verified' and d['reason']=='upload_and_download_verified',d
PY
printf 'attached\n' >"$role_mode_file"
rm -f "$test_root/state/status.json" "$test_root/ossutil.log"
MOCK_OSS_ROOT="$test_root/mock-oss" MOCK_OSS_LOG="$test_root/ossutil.log" OSSUTIL_BIN="$test_root/mock-ossutil" \
ECS_METADATA_BASE_URL="$metadata_base" OFFSITE_CONFIG_FILE="$test_root/enabled-role.env" OFFSITE_STATE_DIR="$test_root/state" \
BACKUP_ROOT="$test_root/backups" "$candidate" >/dev/null
python3 - "$test_root/state/status.json" <<'PY'
import json,sys
d=json.load(open(sys.argv[1],encoding='utf-8'))
assert d['status']=='verified' and d['reason']=='upload_and_download_verified',d
PY
grep -q -- '--mode EcsRamRole' "$test_root/ossutil.log"
grep -q -- '--region cn-hangzhou' "$test_root/ossutil.log"
grep -q -- '--endpoint oss-cn-hangzhou.aliyuncs.com' "$test_root/ossutil.log"
echo 'offsite_backup_smoke=ok disabled=ok missing_role=ok config_round_trip=ok ecs_role_round_trip=ok'
@@ -0,0 +1,54 @@
#!/usr/bin/env python3
import importlib.util
import pathlib
import subprocess
import sys
import unittest
module_path = pathlib.Path(__file__).with_name("public_listener_policy.py")
spec = importlib.util.spec_from_file_location("public_listener_policy", module_path)
policy = importlib.util.module_from_spec(spec)
assert spec.loader is not None
spec.loader.exec_module(policy)
class PublicListenerPolicyTest(unittest.TestCase):
def test_allows_expected_public_ports_and_ignores_loopback(self) -> None:
lines = [
"LISTEN 0 4096 0.0.0.0:443 0.0.0.0:*",
"LISTEN 0 4096 [::]:80 [::]:*",
"LISTEN 0 128 *:22 *:*",
"LISTEN 0 128 127.0.0.1:5432 0.0.0.0:*",
"LISTEN 0 128 [::1]:6379 [::]:*",
]
self.assertEqual(policy.public_ports(lines), {22, 80, 443})
def test_reports_wildcard_and_interface_bound_database_ports(self) -> None:
lines = [
"LISTEN 0 4096 0.0.0.0:3306 0.0.0.0:*",
"LISTEN 0 4096 172.16.0.8:5432 0.0.0.0:*",
"LISTEN 0 4096 [::]:6379 [::]:*",
]
self.assertEqual(policy.public_ports(lines) - {22, 80, 443}, {3306, 5432, 6379})
def test_validates_allowed_port_range(self) -> None:
self.assertEqual(policy.parse_allowed("22,80,443"), {22, 80, 443})
with self.assertRaises(ValueError):
policy.parse_allowed("22,0,443")
with self.assertRaises(ValueError):
policy.parse_allowed("22, 443")
def test_cli_marks_unexpected_port_critical(self) -> None:
result = subprocess.run(
[sys.executable, str(module_path), "22,80,443"],
input="LISTEN 0 4096 0.0.0.0:5432 0.0.0.0:*\n",
text=True,
capture_output=True,
check=True,
)
self.assertEqual(result.stdout.strip(), "critical|unexpected_ports=5432")
if __name__ == "__main__":
unittest.main()
+106
View File
@@ -0,0 +1,106 @@
#!/usr/bin/env bash
set -Eeuo pipefail
project_root=${PLATFORM_ROOT:-/opt/data-terminal-cloud/current}
backup_root=${BACKUP_ROOT:-/var/backups/data-terminal-cloud}
passphrase_file=${BACKUP_PASSPHRASE_FILE:-/etc/data-terminal-cloud/backup-passphrase}
archive=${1:-}
if [[ -z "$archive" ]]; then
archive=$(find "$backup_root" -maxdepth 1 -type f -name 'platform-*.tar.gz.enc' -printf '%T@ %p\n' | sort -nr | head -n1 | cut -d' ' -f2-)
fi
if [[ -z "$archive" || ! -f "$archive" ]]; then
echo "Backup archive was not found" >&2
exit 2
fi
archive=$(realpath "$archive")
backup_root=$(realpath "$backup_root")
case "$archive" in
"$backup_root"/platform-*.tar.gz.enc) ;;
*) echo "Backup archive is outside the configured backup directory" >&2; exit 2 ;;
esac
if [[ ! -r "$passphrase_file" ]]; then
echo "Backup passphrase file is missing or unreadable" >&2
exit 2
fi
compose=(docker compose -f "$project_root/compose.yaml" -f "$project_root/deploy/server/compose.override.yaml" --env-file "$project_root/deploy/server/.env")
postgres_container=$("${compose[@]}" ps -q postgres)
if [[ -z "$postgres_container" ]] || [[ $(docker inspect -f '{{.State.Running}}' "$postgres_container") != true ]]; then
echo "PostgreSQL service is not running" >&2
exit 3
fi
redis_container=$("${compose[@]}" ps -q redis)
if [[ -z "$redis_container" ]] || [[ $(docker inspect -f '{{.State.Running}}' "$redis_container") != true ]]; then
echo "Redis service is not running" >&2
exit 3
fi
umask 077
work=$(mktemp -d "$backup_root/.verify-XXXXXX")
plain="$work/backup.tar.gz"
verify_token=$(openssl rand -hex 12)
restore_database="voice_restore_verify_$verify_token"
redis_verify_path="/tmp/platform-backup-$verify_token.rdb"
database_created=false
cleanup() {
status=$?
if [[ "$database_created" == true ]]; then
docker exec "$postgres_container" dropdb -U voice --if-exists "$restore_database" >/dev/null 2>&1 || true
fi
docker exec "$redis_container" rm -f -- "$redis_verify_path" >/dev/null 2>&1 || true
rm -rf -- "$work"
exit "$status"
}
trap cleanup EXIT INT TERM
if [[ -f "$archive.sha256" ]]; then
(cd "$(dirname "$archive")" && sha256sum -c "$(basename "$archive").sha256") >/dev/null
fi
openssl enc -d -aes-256-cbc -pbkdf2 -iter 200000 \
-pass "file:$passphrase_file" -in "$archive" -out "$plain"
tar -tzf "$plain" >"$work/archive-paths.txt"
if grep -Eq '(^/|(^|/)\.\.(/|$))' "$work/archive-paths.txt"; then
echo "Backup contains an unsafe archive path" >&2
exit 4
fi
mkdir -p "$work/extracted"
tar -xzf "$plain" --no-same-owner -C "$work/extracted"
(
cd "$work/extracted"
sha256sum -c SHA256SUMS >/dev/null
)
expected=$(tr -d '\r\n' <"$work/extracted/database-evidence.txt")
docker exec "$postgres_container" createdb -U voice "$restore_database"
database_created=true
docker exec -i "$postgres_container" pg_restore -U voice -d "$restore_database" --no-owner --no-acl <"$work/extracted/postgres.dump"
actual=$(docker exec "$postgres_container" psql -U voice -d "$restore_database" -Atc \
"SELECT version||'|'||dirty||'|'||(SELECT count(*) FROM tenant)||'|'||(SELECT count(*) FROM user_account)||'|'||(SELECT count(*) FROM information_schema.tables WHERE table_schema='public')||'|'||(SELECT count(*) FROM information_schema.table_constraints WHERE constraint_type='FOREIGN KEY') FROM schema_migrations;")
if [[ "$actual" != "$expected" ]]; then
echo "Restored database evidence does not match the backup: expected=$expected actual=$actual" >&2
exit 5
fi
[[ -s "$work/extracted/redis/dump.rdb" ]] || { echo "Redis snapshot is missing" >&2; exit 6; }
[[ -d "$work/extracted/object-store" ]] || { echo "Object-store snapshot is missing" >&2; exit 6; }
[[ -d "$work/extracted/keycloak" ]] || { echo "Keycloak snapshot is missing" >&2; exit 6; }
expected_object_files=$(sed -n 's/^object_files=//p' "$work/extracted/manifest.properties")
expected_keycloak_files=$(sed -n 's/^keycloak_files=//p' "$work/extracted/manifest.properties")
actual_object_files=$(find "$work/extracted/object-store" -type f | wc -l | tr -d ' ')
actual_keycloak_files=$(find "$work/extracted/keycloak" -type f | wc -l | tr -d ' ')
if [[ ! "$expected_object_files" =~ ^[0-9]+$ || "$actual_object_files" != "$expected_object_files" ]]; then
echo "Object-store file count does not match the manifest" >&2
exit 6
fi
if [[ ! "$expected_keycloak_files" =~ ^[1-9][0-9]*$ || "$actual_keycloak_files" != "$expected_keycloak_files" ]]; then
echo "Keycloak file count does not match the manifest" >&2
exit 6
fi
docker cp "$work/extracted/redis/dump.rdb" "$redis_container:$redis_verify_path" >/dev/null
docker exec "$redis_container" redis-check-rdb "$redis_verify_path" >/dev/null
docker exec "$redis_container" rm -f -- "$redis_verify_path"
printf 'backup_restore_verified archive=%s database_evidence=%s object_files=%s keycloak_files=%s redis_rdb=valid\n' \
"$archive" "$actual" "$actual_object_files" "$actual_keycloak_files"
@@ -5,7 +5,7 @@
"AgentConfig": { "AgentConfig": {
"TargetUserId": [], "TargetUserId": [],
"UserId": "", "UserId": "",
"WelcomeMessage": "您好,我会陪您聊聊天。", "WelcomeMessage": "",
"EnableConversationStateCallback": false "EnableConversationStateCallback": false
}, },
"Config": { "Config": {
@@ -16,7 +16,7 @@
"AgentConfig": { "AgentConfig": {
"TargetUserId": [], "TargetUserId": [],
"UserId": "", "UserId": "",
"WelcomeMessage": "您好,我会陪您聊聊天。", "WelcomeMessage": "",
"EnableConversationStateCallback": false "EnableConversationStateCallback": false
}, },
"Config": { "Config": {
@@ -27,7 +27,7 @@
"AgentConfig": { "AgentConfig": {
"TargetUserId": [], "TargetUserId": [],
"UserId": "", "UserId": "",
"WelcomeMessage": "您好,我会陪您聊聊天。", "WelcomeMessage": "",
"EnableConversationStateCallback": false "EnableConversationStateCallback": false
}, },
"Config": { "Config": {
+5
View File
@@ -1,4 +1,5 @@
FROM node:24-alpine AS build FROM node:24-alpine AS build
ARG NPM_CONFIG_REGISTRY=https://registry.npmjs.org
WORKDIR /src WORKDIR /src
COPY web/package*.json ./ COPY web/package*.json ./
RUN npm ci RUN npm ci
@@ -15,6 +16,10 @@ LABEL org.opencontainers.image.version="${VERSION}" \
org.opencontainers.image.created="${BUILD_TIME}" org.opencontainers.image.created="${BUILD_TIME}"
USER root USER root
RUN apk upgrade --no-cache RUN apk upgrade --no-cache
# The image's IPv6 helper shells out to `apk manifest nginx` on every start.
# On small production hosts that command can block even after the package
# database is healthy. This deployment owns default.conf and does not need it.
RUN rm -f /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
COPY --chown=101:101 deploy/web.nginx.conf /etc/nginx/conf.d/default.conf COPY --chown=101:101 deploy/web.nginx.conf /etc/nginx/conf.d/default.conf
COPY --chmod=755 --chown=101:101 deploy/web-runtime-config.sh /docker-entrypoint.d/40-runtime-config.sh COPY --chmod=755 --chown=101:101 deploy/web-runtime-config.sh /docker-entrypoint.d/40-runtime-config.sh
COPY --chown=101:101 --from=build /src/dist /usr/share/nginx/html COPY --chown=101:101 --from=build /src/dist /usr/share/nginx/html
+4 -1
View File
@@ -32,7 +32,10 @@ server {
add_header X-Frame-Options "DENY" always; add_header X-Frame-Options "DENY" always;
add_header Referrer-Policy "strict-origin-when-cross-origin" always; add_header Referrer-Policy "strict-origin-when-cross-origin" always;
add_header Permissions-Policy "camera=(), geolocation=(), microphone=()" always; add_header Permissions-Policy "camera=(), geolocation=(), microphone=()" always;
add_header Content-Security-Policy "default-src 'self'; connect-src 'self' https: http://localhost:*; img-src 'self' data: blob: https:; media-src 'self' blob: https: http://localhost:*; style-src 'self'; script-src 'self'; frame-ancestors 'none'; base-uri 'self'; form-action 'self' https: http://localhost:*" always; # Ant Design v6 generates component styles at runtime and also relies on
# safe element-level layout styles. Scripts remain restricted to self;
# only CSS is allowed inline.
add_header Content-Security-Policy "default-src 'self'; connect-src 'self' https: http://localhost:*; img-src 'self' data: blob: https:; media-src 'self' blob: https: http://localhost:*; style-src 'self' 'unsafe-inline'; script-src 'self'; frame-ancestors 'none'; base-uri 'self'; form-action 'self' https: http://localhost:*" always;
} }
location = /runtime-config.js { location = /runtime-config.js {
Binary file not shown.

After

Width:  |  Height:  |  Size: 360 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 72 KiB

+59
View File
@@ -0,0 +1,59 @@
# 家庭陪伴门户 v0.74 设计 QA
- 功能与页面规格:`docs/customer-portal-v074-page-spec.md`
- 设计稿源:`.design-evidence/v074/design/*.png`
- 实际页面截图:`.design-evidence/v074/implementation/*.png`
- 同屏对比证据:`.design-evidence/v074/comparison/*.png`
- 统一视口:1440 × 1024 CSS px,device scale factor 1
- 验收状态:本地所有者账号、合成数据一号家庭、2026-08-30、每页默认业务状态
## 页面验收矩阵
| 页面 | 核心功能 | 设计/实现对比 | 结果 |
| --- | --- | --- | --- |
| 首页 | 长者周记、心情线索、关怀提醒、最近动态 | `comparison/home.png` | 通过 |
| 聊天记录 | 时间轴、AI 摘要、关键词、故事候选、下次话题、模型来源 | `comparison/conversations.png` | 通过 |
| 故事与记忆 | 待审故事、审核入口、已确认记忆和搜索 | `comparison/stories.png` | 通过 |
| 长者资料 | 头像、长期资料、交流节奏、兴趣和授权入口 | `comparison/subjects.png` | 通过 |
| 日程与提醒 | 今日陪伴动态、待处理事项、近期状态汇总 | `comparison/messages.png` | 通过 |
| 家庭管理 | 家庭切换、成员、角色、邀请、家庭资料和动态 | `comparison/members.png` | 通过 |
| 陪伴设备 | 绑定、在线状态、固件、诊断和配置 | `comparison/devices.png` | 通过 |
| 内容分享 | 资源级分享、接收人、权限、期限和撤销 | `comparison/sharing.png` | 通过 |
| 数据与授权 | 数据盘点、更正、撤回、私有导出和主体删除 | `comparison/privacy.png` | 通过 |
| 服务用量 | 聊天时长、活跃日、热力图、设备与最近通话 | `comparison/billing.png` | 通过 |
| 大模型管理 | 模型配置、连接测试、激活、移除和密钥说明 | `comparison/models.png` | 通过 |
| 设置与权限 | 身份、家庭角色和 15 项完整权限矩阵 | `comparison/profile.png` | 通过 |
## 必检视觉项
- 页面标题与家庭标签紧邻,标题底部到家庭上下文栏的间距为 0;内容区统一 16px 节奏。
- 侧栏、页头、家庭 tab、卡片边框、圆角、按钮高度和表单控件使用同一套 v0.74 token。
- 1440px 视口下 12 个页面均无页面级横向溢出,无裁切的主要按钮或正文。
- 家庭管理重新出现在一级导航,使用可直接点击的家庭 tab,不再显示“客户组织”。
- 日程与提醒不再使用大段空白消息头;隐私空状态压缩到 150px;故事页长者筛选保持单行。
- 禁用按钮统一 `cursor: wait`;长者头像上传可显示本地预览。
## 功能检查
- 12 个一级/二级导航均能进入对应页面。
- 新增长者头像上传与预览通过。
- 接受家庭邀请、邀请新成员、绑定设备、新建分享、资料更正、添加大模型、故事详情 7 个非破坏性弹窗打开/关闭通过。
- 聊天记录选中一条记录后,AI 总结区能进入真实加载/结果状态。
- 所有采样到的禁用按钮均显示等待光标。
- 浏览器控制台错误、非预期 HTTP 4xx/5xx、失败请求均为 0。
## 对比迭代
1. 修复家庭管理入口缺失,统一页面名称与客户视角的信息架构。
2. 移除工作区旧的大空白标题层,合并家庭 tab、实时状态、刷新和页面动作。
3. 故事页长者筛选从第二行收回同一上下文栏,并为故事卡补充“开始审核”。
4. 日程与提醒重构为今日动态、家人待办和近期汇总,不再堆叠无意义消息。
5. 设置页补齐家庭上下文,并集中列出 15 项角色权限。
6. 隐私页压缩空状态;用量页日期范围校准为 2025-09 至 2026-08。
7. 设计稿与实际页以相同 1440 × 1024 视口逐页同屏复核。
## 最终结论
没有遗留的 P0、P1 或 P2 视觉/交互问题。设计稿中的演示故事、已配置模型和已有分享,与本地数据库的空状态属于业务数据差异;页面结构、视觉系统和核心操作保持一致。
final result: passed
+4
View File
@@ -0,0 +1,4 @@
node_modules/
dist/
release/
*.log
+118
View File
@@ -0,0 +1,118 @@
# Windows 硬件对话智能体测试程序
这是方案 1 的 Windows 端到端验收工具。默认页面不再使用平台代管的 RTC/StartVoiceChat,而是按火山“硬件对话智能体低负载标准协议”直接连接 `wss://ai-gateway.vei.volces.com/v1/realtime`。传统 RTC 模拟器仍保留在页面右上角的“传统 RTC 测试页”。
## 实际覆盖的流程
1. 使用平台设备 ID、时间戳、随机数、正文摘要和 HMAC-SHA256 调用设备 v2/v3 接口。
2. 调用平台 `/device/v3/hardware-agent-credentials:provision`;平台自动应用产品级Instance ID、Product Key、默认Bot、传输和最低SDK配置,并使用KMS/Secret挂载的 Product Secret 代设备调用 DynamicRegister。
3. 火山在动态注册时从产品额度中分配 License;平台只把本设备 Device Secret 下发给设备,Windows 使用 Safe Storage 加密保存后重新拉取配置版本并进入 `ACTIVE`。
4. 创建 `/device/v3/agent-sessions`,从平台取得产品默认 Bot ID、`low_load` 传输方案、本次冻结的人设、背景、最近摘要、确认记忆和授权范围。
5. 按火山规定的请求头签名建立 WebSocket,等待 `session.created`,提交本次 `session.update`,收到 `session.updated` 后才向平台回报 `session_started`并开放麦克风。
6. Windows 麦克风被转换为 PCM16、16kHz、单声道、20ms 分片,通过 `input_audio_buffer.append` 实时上行;`response.audio.delta` 按火山 low-load WebSocket SDK 的 PCM16、16kHz、单声道格式实时播放。
7. 消费服务端 VAD、用户最终识别、智能体最终字幕、Function Calling 和错误事件;双方最终话轮通过 v3 `turns:batch` 幂等上传。
8. 智能体触发 `memory_search`、`search_memory` 或 `retrieve_memory` 时,程序使用平台签发的当前会话短期令牌调用同源记忆工具,再把结果作为 `function_call_output` 返回火山。
9. 周期提交 `session_updated`。平台返回 `stop_session` 时立即关麦、补传字幕、关闭火山连接并回报 `session_ended`。
10. 页面显示 13 项验收门禁,可导出不包含任何密钥的 JSON 报告。
## 使用前配置
平台维护中心只需把测试设备配置为:
- `voice_runtime = volc_hardware_agent`
平台部署需配置并关联同一火山产品和 Bot:
- Instance ID
- Product Key
- 默认 Bot ID
- 传输方式 `low_load`
- 最低 SDK 版本不高于 `1.1.0-windows-standard-protocol`
- Product Secret(只进入平台密钥挂载,不进入Windows程序)
- Device Name 由平台按设备ID稳定生成,或使用设备绑定中预先登记的名称
- 可用的硬件对话智能体 License
Windows程序不提供 Product Secret、Device Secret 或 Bot ID 输入框。Bot ID、Instance ID、Product Key 和 Device Name 均来自平台;设备不能覆盖产品默认路由。
开始测试前还需按控制台中的 Bot 模型选择“智能体模型类型”:
- 通用 LLM:使用官方 `config.LLMConfig.SystemMessages` 注入平台冻结的人设、背景与授权上下文;
- 豆包端到端语音:使用官方 `config.S2SConfig.ProviderParams.dialog` 注入 `system_role`、`speaking_style` 和 `dialog_id`。
两种模式都使用官方 `session.object/config/agent_config/input_audio_format/output_audio_format` 报文结构。字段依据火山[初始化会话配置](https://www.volcengine.com/docs/6348/2139862?lang=zh)与[标准协议自定义接入](https://www.volcengine.com/docs/6348/1904602?lang=zh)文档。
## 构建
```powershell
Set-Location .\device-simulator-windows
npm install
npm run build
```
输出文件:
```text
release/hardware-agent-tester-0.3.5-windows-x64.exe
```
当前测试包未配置商业代码签名证书,Windows 可能显示“未知发布者”。交付前可用同目录 SHA-256 文件校验完整性;正式面向终端用户分发时应由甲方或发布主体使用其代码签名证书签名。
`npm run build` 会依次执行协议单测、TypeScript 检查、双页面构建和 Windows x64 便携包封装。
## 本地完整自动回归
先启动仓库根目录的 Compose 开发环境,然后运行:
```powershell
Set-Location .\device-simulator-windows
pwsh -NoProfile -File .\test\run-hardware-e2e.ps1
pwsh -NoProfile -File .\test\run-hardware-e2e.ps1 -ModelMode s2s
pwsh -NoProfile -File .\test\run-hardware-e2e.ps1 -ExecutablePath .\release\hardware-agent-tester-0.3.5-windows-x64.exe
```
第一条验证通用 LLM 报文,第二条验证端到端语音报文。回归会临时创建 `low_load` 设备绑定,启动一个校验真实火山签名和事件格式的本机 WebSocket 网关,再启动实际 Electron 窗口。测试只有在以下证据全部成立时才返回 PASS:
- 动态上下文确认后才开放麦克风;
- PCM 上行和音频下行均发生;
- 用户与智能体各产生一条最终字幕并写入 PostgreSQL;
- 平台下发停止状态,Windows 程序通过下一次 `session_updated` 收到 `stop_session`;
- 会话、硬件会话均为 `COMPLETED`;
- SaaS `conversation.started` 和 `conversation.completed` 事件均已产生;
- 页面 13 项验收门禁全部通过。
测试结束会删除临时会话、事件和设备绑定,不污染日常开发数据。
## 安全边界
- Product Secret 只存在于平台KMS/Secret挂载和注册调用内存,不到达设备,不写入数据库明文、日志或报告。
- 平台返回的本设备Device Secret使用 Electron `safeStorage` 加密后保存在当前 Windows 用户目录,不进入源码和安装包;License由火山产品侧分配,不下发Windows程序。
- 平台设备密钥只驻留 Electron 主进程,渲染页面不能读取已加载的明文。
- 记忆工具地址必须与配置的平台 API 同源,防止短期令牌被发送给第三方地址。
- 默认只允许连接火山官方硬件网关;本机 `ws://127.0.0.1` 仅供自动回归。
- 平台确认 `session.updated` 前不会启动麦克风;收到 `stop_session` 后不会继续发送音频。
- 导出的报告包含设备标识、Bot、配置摘要、事件和字幕,但不包含 Product Secret、Device Secret、平台签名密钥或记忆工具令牌。
- 自动回归通过仅在`SIMULATOR_AUTO_TEST=1`时生效的进程环境注入mock一机一密,不写入正式Windows凭据缓存。
## 真实验收与本地模拟的区别
### 0.3.5 播放速度修复
- 根据火山 low-load WebSocket SDK 示例,将智能体下行 PCM 的源采样率从错误的 24kHz 修正为 16kHz,避免 1.5 倍速和音调升高。
- Windows 输出设备不是 16kHz 时,由 AudioWorklet 按实际系统输出采样率连续重采样,不改变语速和音高。
### 0.3.4 连续音频播放修复
- 火山返回的 `pcm_s16le/16kHz/mono` 音频进入常驻 AudioWorklet 环形缓冲,不再按 WebSocket 分片创建独立播放器。
- 默认预缓冲 120ms,网络瞬时抖动时自动重新缓冲,减少断音、金属音和语速漂移。
- 连续处理跨分片的奇数字节,并在开始、结束和断流时做短淡入淡出以避免爆音。
- 测试报告新增播放分片数、采样数和欠载次数。
### 0.3.3 失败会话修复
- 火山 `error` 事件与连接 Promise 的异常共用一个失败清理任务,同一轮仅提交一次 `session_failed`,保留首次错误。
- 失败清理期间禁止发送文字或重复结束;延迟到达的事件不再恢复“通话中”。失败后重新检测即可开始新一轮。
- 新通话清空上一轮验收结果、错误和计时;失败清理的关闭不再标为正常关闭。
- `npm test` 包含执行实际渲染器代码的并发失败回归测试,不会连接线上平台或消耗火山额度。
- 此修复不代表火山 `internal_error: StartVoiceChat failed` 的上游原因已解决。
本地自动回归证明 Windows 程序、平台控制面、数据库、PCM 管道和协议状态机完整可执行。最终真实火山验收仍必须使用实际 Instance/Product/Device/Bot/License 凭据,确认火山返回真实 ASR、LLM、TTS 音频;程序不会把本机模拟网关的 PASS 冒充真实供应商验收。
@@ -0,0 +1,288 @@
const crypto = require("node:crypto");
const WebSocket = require("ws");
const DEFAULT_GATEWAY_URL = "wss://ai-gateway.vei.volces.com/v1/realtime";
const VOICE_COMPANION_RULES = [
"你在和一位长者进行真实语音对话。先接住对方刚说的内容,再决定是否追问。",
"通常回复一到两句并默认使用陈述句。任意连续三次回复最多出现一个问题;索取信息、确认或同意都算问题,不以问号为准,吧、吗、呢、是不是、有没有、会不会、对不对、可不可以、愿不愿意都算问题。",
"如果最近两次回复已经问过,或者长者只说是啊、嗯、对或不是,本轮必须只用陈述句回应并停下来,不能用吧请求确认,也不能换一种说法继续追问。",
"长者分享经历时,先回应一个由长者亲口说出的具体细节;需要追问时一次只问一个开放而具体的问题,避免审问式追问。",
"只能把长者明确说过的内容归到长者本人。不得补写现场、动作、人物、情绪、原因、结果或第三方态度;不得用肯定、一定、应该、想必、我猜、听得出、看得出包装猜测,不评价长者很厉害,不声称大家都佩服您。历史资料只在当前话题自然相关时使用,不要突然提起。",
"示例:长者说在工厂修机器,可以问哪一次维修印象最深;长者接着只说是啊,应答这门手艺里的经验都是一点点积累起来的,然后停下。不要说肯定很辛苦吧、您当时肯定很厉害、大家应该都很佩服您吧。",
"对方表示不想说、累了、忘了或转移话题时,尊重并换话题或结束。不要使用催促、埋怨或让对方有负担的开场白。",
"回复只包含直接说出口的口语正文,语速自然。不要输出或朗读动作、表情、语气、情绪、场景等舞台提示,不要使用括号或星号包裹表演说明。",
"不要复述人设、系统配置或对话标识。",
].join("");
function requiredText(value, name, maximum = 512) {
const normalized = String(value || "").trim();
if (!normalized || normalized.length > maximum) throw new Error(`${name}不能为空或过长`);
return normalized;
}
function validateDeviceName(value) {
const name = requiredText(value, "火山设备名称", 128);
if (!/^[a-zA-Z0-9_\-.:@]{1,128}$/.test(name)) throw new Error("火山设备名称格式不正确");
return name;
}
function hmacBase64(secret, content) {
return crypto.createHmac("sha256", secret).update(content).digest("base64");
}
function websocketSignature({ authType = 1, deviceName, randomNumber, productKey, timestamp, instanceID }, deviceSecret) {
const content = `auth_type=${authType}&device_name=${deviceName}&random_num=${randomNumber}&product_key=${productKey}&timestamp=${timestamp}&instance_id=${instanceID}`;
return hmacBase64(deviceSecret, content);
}
function buildSessionPrompt(sessionConfig) {
const sections = [
requiredText(sessionConfig.authorized_context, "授权上下文", 4000),
String(sessionConfig.system_prompt || "").trim(),
String(sessionConfig.persona_tone || "").trim() ? `表达风格(仅指导表达,不可逐字复述):${String(sessionConfig.persona_tone).trim()}` : "",
VOICE_COMPANION_RULES,
].filter(Boolean);
return sections.join("\n\n");
}
function buildSessionUpdate(sessionConfig, eventID, modelMode) {
const normalizedMode = String(modelMode || sessionConfig.model_mode || "llm").trim().toLowerCase();
if (!new Set(["llm", "s2s"]).has(normalizedMode)) throw new Error("智能体模型类型必须为 llm 或 s2s");
const dialogID = requiredText(sessionConfig.dialog_id, "对话标识", 200);
const systemPrompt = buildSessionPrompt(sessionConfig);
const config = {
ASRConfig: { TurnDetectionMode: 0 },
};
if (normalizedMode === "s2s") {
config.S2SConfig = {
ProviderParams: {
dialog: {
system_role: systemPrompt,
speaking_style: String(sessionConfig.persona_tone || "").trim(),
dialog_id: dialogID,
},
},
};
}
config.LLMConfig = { SystemMessages: [systemPrompt] };
const memory = sessionConfig.memory;
if (memory && typeof memory === "object") {
const collectionName = requiredText(memory.collection_name, "Viking 记忆库名称", 200);
const userID = requiredText(memory.user_id, "Viking 用户标识", 200);
const memoryTypes = Array.isArray(memory.memory_types)
? memory.memory_types.map((value) => requiredText(value, "Viking 记忆类型", 100))
: [];
if (!memoryTypes.length) throw new Error("Viking 记忆类型不能为空");
const filter = { user_id: [userID], memory_type: memoryTypes };
if (String(memory.assistant_id || "").trim()) {
filter.assistant_id = [requiredText(memory.assistant_id, "Viking 助手标识", 200)];
}
config.MemoryConfig = {
Enable: true,
Provider: "volc",
ProviderParams: {
collection_name: collectionName,
filter,
limit: Number.isInteger(memory.limit) && memory.limit > 0 ? Math.min(memory.limit, 5) : 5,
...(String(memory.transition_words || "").trim()
? { transition_words: String(memory.transition_words).trim() }
: {}),
},
};
}
return {
event_id: eventID,
type: "session.update",
session: {
object: "realtime.session",
model: "",
input_audio_format: "pcm16",
output_audio_format: "pcm16",
config,
agent_config: { EnableConversationStateCallback: false },
},
};
}
function normalizeProviderError(event) {
const code = String(event?.error?.code || event?.error?.type || "PROVIDER_ERROR");
const message = String(event?.error?.message || "火山硬件智能体返回错误");
return new Error(`${code}: ${message}`);
}
class HardwareAgentClient {
constructor(options = {}) {
this.WebSocketImpl = options.WebSocketImpl || WebSocket;
this.gatewayURL = options.gatewayURL || DEFAULT_GATEWAY_URL;
this.emit = typeof options.emit === "function" ? options.emit : () => {};
this.socket = undefined;
this.sessionCreated = undefined;
this.providerConnectionRef = "";
this.pendingCalls = new Map();
this.pingTimer = undefined;
this.connectionPromise = undefined;
this.sessionUpdateSent = false;
}
async connect(input) {
if (this.socket) throw new Error("火山硬件智能体连接已经存在");
const instanceID = requiredText(input.instanceID, "实例 ID", 200);
const productKey = requiredText(input.productKey, "Product Key", 200);
const deviceName = validateDeviceName(input.deviceName);
const deviceSecret = requiredText(input.deviceSecret, "Device Secret", 4096);
const hardwareID = requiredText(input.hardwareID, "Hardware ID", 200);
const botID = requiredText(input.botID, "Bot ID", 200);
const randomNumber = crypto.randomInt(1, 2_147_483_647);
const timestamp = Math.floor(Date.now() / 1000);
const signature = websocketSignature({ deviceName, randomNumber, productKey, timestamp, instanceID }, deviceSecret);
const target = new URL(this.gatewayURL);
target.searchParams.set("bot", botID);
target.searchParams.set("wait_for_session_update", "true");
const headers = {
"X-Auth-Type": "1",
"X-Product-Key": productKey,
"X-Device-Name": deviceName,
"X-Random-Num": String(randomNumber),
"X-Timestamp": String(timestamp),
"X-Instance-Id": instanceID,
"X-Signature": signature,
"X-Hardware-Id": hardwareID,
};
const socket = new this.WebSocketImpl(target.toString(), { headers, handshakeTimeout: 10_000, maxPayload: 8 * 1024 * 1024 });
this.socket = socket;
this.connectionPromise = new Promise((resolve, reject) => {
const timeout = setTimeout(() => reject(new Error("等待火山 session.updated 超时")), 15_000);
const finish = (callback, value) => {
clearTimeout(timeout);
callback(value);
};
socket.once("open", () => {
this.emit({ type: "transport.open" });
// wait_for_session_update=true makes the gateway wait for the client
// configuration before it publishes the usable session. Sending only
// after session.created deadlocks until the provider-side timeout.
this.sendJSON(buildSessionUpdate(input.sessionConfig, `windows-session-update-${crypto.randomUUID()}`, input.modelMode));
this.sessionUpdateSent = true;
});
socket.on("message", (data) => {
let event;
try { event = JSON.parse(Buffer.isBuffer(data) ? data.toString("utf8") : String(data)); }
catch { this.emit({ type: "transport.invalid_message" }); return; }
this.handleProviderEvent(event, input.sessionConfig, input.modelMode);
// In the hardware gateway's wait_for_session_update mode, the initial
// session.created event is published only after the pre-creation
// session.update has been accepted. Some Bot versions do not emit an
// additional session.updated event for that initial configuration.
if (event.type === "session.updated" || (event.type === "session.created" && this.sessionUpdateSent)) {
const connectionRef = String(event?.session?.id || this.providerConnectionRef || event.event_id || "").trim();
if (!connectionRef) return finish(reject, new Error("火山会话确认未提供可关联的会话编号"));
this.providerConnectionRef = connectionRef;
finish(resolve, { providerConnectionRef: connectionRef, session: event.session || {} });
} else if (event.type === "error" && !this.providerConnectionRef) {
finish(reject, normalizeProviderError(event));
}
});
socket.once("unexpected-response", (_request, response) => {
finish(reject, new Error(`火山 WebSocket 鉴权失败:HTTP ${response.statusCode}`));
});
socket.once("error", (error) => {
this.emit({ type: "transport.error", message: error.message });
if (!this.providerConnectionRef) finish(reject, error);
});
socket.once("close", (code, reason) => {
this.stopPing();
this.socket = undefined;
this.emit({ type: "transport.close", code, reason: Buffer.from(reason || "").toString("utf8") });
if (!this.providerConnectionRef) finish(reject, new Error(`火山连接提前关闭:${code}`));
});
});
try {
const result = await this.connectionPromise;
this.startPing();
return result;
} catch (error) {
this.close(1011, "connection setup failed");
throw error;
}
}
handleProviderEvent(event, sessionConfig, modelMode) {
if (event.type === "session.created") {
this.sessionCreated = event.session || {};
this.providerConnectionRef = String(event?.session?.id || event.event_id || "").trim();
}
if (event.type === "conversation.item.created" && event?.item?.type === "function_call" && event.item.call_id) {
this.pendingCalls.set(String(event.item.call_id), String(event.item.name || ""));
}
this.emit(event);
}
sendAudio(bytes) {
const buffer = Buffer.from(bytes);
if (!buffer.length) return;
if (buffer.length > 64 * 1024) throw new Error("单个音频分片不能超过64KiB");
this.sendJSON({
event_id: `windows-audio-${crypto.randomUUID()}`,
type: "input_audio_buffer.append",
audio: buffer.toString("base64"),
});
}
sendText(text, interruptMode = 1) {
const normalized = requiredText(text, "文本问题", 2000);
this.sendJSON({
event_id: `windows-text-${crypto.randomUUID()}`,
type: "conversation.item.create",
item: { type: "message", role: "user", content: [{ type: "input_text", text: normalized }], interrupt_mode: interruptMode },
});
}
completeFunctionCall(callID, output) {
const normalizedCallID = requiredText(callID, "函数调用编号", 200);
this.sendJSON({
event_id: `windows-function-${crypto.randomUUID()}`,
type: "conversation.item.create",
item: { type: "function_call_output", call_id: normalizedCallID, output: JSON.stringify(output) },
});
this.sendJSON({ event_id: `windows-response-${crypto.randomUUID()}`, type: "response.create" });
this.pendingCalls.delete(normalizedCallID);
}
sendJSON(value) {
if (!this.socket || this.socket.readyState !== this.WebSocketImpl.OPEN) throw new Error("火山硬件智能体连接未就绪");
this.socket.send(JSON.stringify(value));
}
startPing() {
this.stopPing();
this.pingTimer = setInterval(() => {
try { if (this.socket?.readyState === this.WebSocketImpl.OPEN && typeof this.socket.ping === "function") this.socket.ping(); }
catch { /* close/error callbacks own recovery */ }
}, 20_000);
}
stopPing() {
if (this.pingTimer) clearInterval(this.pingTimer);
this.pingTimer = undefined;
}
close(code = 1000, reason = "device session ended") {
this.stopPing();
const socket = this.socket;
this.socket = undefined;
this.providerConnectionRef = "";
this.sessionUpdateSent = false;
this.pendingCalls.clear();
if (!socket) return;
try { socket.close(code, reason); } catch { try { socket.terminate?.(); } catch { /* no-op */ } }
}
}
module.exports = {
DEFAULT_GATEWAY_URL,
HardwareAgentClient,
buildSessionPrompt,
buildSessionUpdate,
websocketSignature,
};
@@ -0,0 +1,122 @@
const assert = require("node:assert/strict");
const { EventEmitter } = require("node:events");
const test = require("node:test");
const {
HardwareAgentClient,
buildSessionUpdate,
websocketSignature,
} = require("./hardware-agent-client.cjs");
test("WebSocket signatures use the documented canonical field order", () => {
const connection = websocketSignature({ authType: 1, deviceName: "device-1", randomNumber: 42, productKey: "product-1", timestamp: 1700000000, instanceID: "instance-1" }, "device-secret");
assert.equal(connection, "6OFE1DUZzRgNy0o5IzZSmqKw3SUP4s3GxK2YHe8phfo=");
});
test("LLM session update uses the official Config/AgentConfig schema", () => {
const update = buildSessionUpdate({ dialog_id: "conversation-1", system_prompt: "你是陪伴助手", persona_tone: "亲切", authorized_context: "只可使用已授权背景" }, "event-1");
assert.equal(update.type, "session.update");
assert.equal(update.session.object, "realtime.session");
assert.equal(update.session.input_audio_format, "pcm16");
assert.equal(update.session.output_audio_format, "pcm16");
assert.equal(update.session.config.ASRConfig.TurnDetectionMode, 0);
assert.match(update.session.config.LLMConfig.SystemMessages[0], /只可使用已授权背景/);
assert.match(update.session.config.LLMConfig.SystemMessages[0], /你是陪伴助手/);
assert.match(update.session.config.LLMConfig.SystemMessages[0], /亲切/);
assert.match(update.session.config.LLMConfig.SystemMessages[0], /任意连续三次回复最多出现一个问题/);
assert.match(update.session.config.LLMConfig.SystemMessages[0], /简短附和或否定|长者只说是啊/);
assert.match(update.session.config.LLMConfig.SystemMessages[0], /第三方态度/);
assert.match(update.session.config.LLMConfig.SystemMessages[0], /大家应该都很佩服您吧/);
assert.match(update.session.config.LLMConfig.SystemMessages[0], /先回应一个由长者亲口说出的具体细节/);
assert.doesNotMatch(update.session.config.LLMConfig.SystemMessages[0], /本次对话标识/);
assert.doesNotMatch(update.session.config.LLMConfig.SystemMessages[0], /conversation-1/);
assert.equal(update.session.agent_config.EnableConversationStateCallback, false);
assert.equal("system_prompt" in update.session, false);
});
test("S2S session update maps persona and dialog id to provider dialog config", () => {
const update = buildSessionUpdate({ dialog_id: "conversation-2", system_prompt: "你是陪伴助手", persona_tone: "亲切自然", authorized_context: "只使用授权信息" }, "event-2", "s2s");
const dialog = update.session.config.S2SConfig.ProviderParams.dialog;
assert.equal(dialog.dialog_id, "conversation-2");
assert.equal(dialog.speaking_style, "亲切自然");
assert.match(dialog.system_role, /只使用授权信息/);
assert.match(dialog.system_role, /你是陪伴助手/);
assert.match(update.session.config.LLMConfig.SystemMessages[0], /你是陪伴助手/);
});
test("session update maps the bound user to native Viking memory without credentials", () => {
const update = buildSessionUpdate({
dialog_id: "conversation-memory", model_mode: "s2s", authorized_context: "authorized",
memory: {
collection_name: "memory", user_id: "u_subject_hash", assistant_id: "platform_companion_v1",
memory_types: ["event_v1", "profile_v1"], limit: 5, transition_words: "自然承接历史记忆",
},
}, "event-memory");
const memory = update.session.config.MemoryConfig;
assert.equal(memory.Enable, true);
assert.equal(memory.Provider, "volc");
assert.equal(memory.ProviderParams.collection_name, "memory");
assert.deepEqual(memory.ProviderParams.filter.user_id, ["u_subject_hash"]);
assert.deepEqual(memory.ProviderParams.filter.assistant_id, ["platform_companion_v1"]);
assert.deepEqual(memory.ProviderParams.filter.memory_type, ["event_v1", "profile_v1"]);
assert.equal("token" in memory.ProviderParams, false);
});
class FakeWebSocket extends EventEmitter {
static OPEN = 1;
constructor(url, options) {
super();
this.url = url;
this.options = options;
this.readyState = 0;
this.sent = [];
FakeWebSocket.last = this;
setImmediate(() => {
this.readyState = FakeWebSocket.OPEN;
this.emit("open");
this.emit("message", Buffer.from(JSON.stringify({ type: "session.created", event_id: "created-event", session: { id: "provider-session-1" } })));
});
}
send(body) {
const value = JSON.parse(body);
this.sent.push(value);
if (value.type === "session.update") {
setImmediate(() => this.emit("message", Buffer.from(JSON.stringify({ type: "session.updated", event_id: "updated-event", session: { id: "provider-session-1" } }))));
}
}
ping() {}
close(code, reason) {
this.readyState = 3;
this.emit("close", code, Buffer.from(reason));
}
}
test("hardware client waits for dynamic config acknowledgement before becoming ready", async () => {
const events = [];
const client = new HardwareAgentClient({ WebSocketImpl: FakeWebSocket, gatewayURL: "ws://127.0.0.1:19090/v1/realtime", emit: (event) => events.push(event) });
const connected = await client.connect({
instanceID: "instance-1", productKey: "product-1", deviceName: "device-1",
deviceSecret: "device-secret-with-at-least-32-characters", hardwareID: "windows-test", botID: "bot-1",
sessionConfig: { dialog_id: "conversation-1", authorized_context: "authorized" },
});
assert.equal(connected.providerConnectionRef, "provider-session-1");
assert.match(FakeWebSocket.last.url, /bot=bot-1/);
assert.equal(FakeWebSocket.last.options.headers["X-Product-Key"], "product-1");
assert.equal(FakeWebSocket.last.sent[0].type, "session.update");
client.sendAudio(Buffer.from([1, 2, 3, 4]));
client.sendText("你好");
assert.equal(FakeWebSocket.last.sent[1].type, "input_audio_buffer.append");
assert.equal(FakeWebSocket.last.sent[2].type, "conversation.item.create");
assert.ok(events.some((event) => event.type === "session.created"));
client.close();
});
test("hardware client sends session.update immediately when the WebSocket opens", async () => {
const client = new HardwareAgentClient({ WebSocketImpl: FakeWebSocket, gatewayURL: "ws://127.0.0.1:19090/v1/realtime" });
await client.connect({
instanceID: "instance-1", productKey: "product-1", deviceName: "device-1",
deviceSecret: "device-secret-with-at-least-32-characters", hardwareID: "windows-test", botID: "bot-1",
sessionConfig: { dialog_id: "conversation-1", authorized_context: "authorized" },
});
assert.equal(FakeWebSocket.last.sent.filter((event) => event.type === "session.update").length, 1);
client.close();
});
@@ -0,0 +1,153 @@
const assert = require("node:assert/strict");
const test = require("node:test");
const fs = require("node:fs");
const path = require("node:path");
const vm = require("node:vm");
const ts = require("typescript");
// Execute the actual renderer with a small DOM/IPC fixture; no network or audio.
const source = fs.readFileSync(path.join(__dirname, "../src/hardware.ts"), "utf8");
const compiled = ts.transpileModule(source, {
compilerOptions: { target: ts.ScriptTarget.ES2022, module: ts.ModuleKind.CommonJS },
}).outputText;
function fixture(overrides = {}) {
const elements = new Map();
const element = () => ({
style: {}, value: "", textContent: "", disabled: false, children: [],
append(...items) { this.children.push(...items); },
prepend(...items) { this.children.unshift(...items); },
replaceChildren(...items) { this.children = items; },
querySelector() { return null; }, addEventListener() {},
});
const get = (id) => {
if (!elements.has(id)) elements.set(id, element());
return elements.get(id);
};
const events = [];
const closes = [];
const texts = [];
const created = {
conversation_id: "test-conversation", session: {},
runtime: { transport: "low_load", config_digest: "digest", session_nonce: "nonce", provider_session_ref: "ref", bot_id: "test-bot" },
session_config: { authorized_context: "test", prompt_bundle_version: "test-v1" },
};
const ipc = {
getRuntimeDefaults: async () => ({}), getHardwareRuntimeDefaults: async () => ({}),
onHardwareProviderEvent: () => () => {},
createHardwareAgentSession: async () => created,
closeHardwareAgent: async (input) => { closes.push(input); },
hardwareAgentEvent: async (input) => { events.push(input); return { status: "FAILED" }; },
sendHardwareAgentText: async (input) => { texts.push(input); },
...overrides,
};
const context = vm.createContext({
exports: {}, require: () => ({}), console, URL,
document: { querySelector: () => get("app"), getElementById: get, createElement: element },
window: { deviceSimulator: ipc, addEventListener() {}, clearInterval() {}, setTimeout, setInterval() {} },
});
vm.runInContext(compiled + `\nglobalThis.renderer = { state, controls, setPhase, setCheck, resetSessionState, failHardwareConversation, stopHardwareConversation, startHardwareConversation, handleProviderEvent, sendTextQuestion, buildReport };`, context);
return { r: context.renderer, events, closes, texts, created, get, ipc };
}
test("provider error plus connect rejection sends one failure, including late duplicates", async () => {
const f = fixture();
const r = f.r;
f.ipc.connectHardwareAgent = async () => {
void r.handleProviderEvent({ type: "error", error: { code: "internal_error", message: "StartVoiceChat failed" } });
throw new Error("wrapped connect rejection");
};
r.setPhase("ready", "ready");
await r.startHardwareConversation();
await r.failHardwareConversation("SECOND_FAILURE");
await r.stopHardwareConversation();
await r.handleProviderEvent({ type: "error", error: { message: "late error" } });
assert.equal(f.events.length, 1);
assert.equal(f.events[0].eventType, "session_failed");
assert.equal(f.events[0].errorCode, "VOLC_HARDWARE_AGENT_ERROR");
assert.equal(f.closes.length, 1);
assert.equal(r.state.failure, "internal_error: StartVoiceChat failed");
assert.equal(r.state.phase, "error");
assert.equal(r.controls.stop.disabled, true);
assert.equal(r.controls.check.disabled, false);
});
test("concurrent failure callers share pending cleanup; late events cannot reactivate", async () => {
let release;
const gate = new Promise((resolve) => { release = resolve; });
const f = fixture({ closeHardwareAgent: () => gate });
f.r.state.platformSession = f.created;
f.r.state.failure = "first cause";
f.r.setPhase("active", "active");
const first = f.r.failHardwareConversation("FIRST");
const second = f.r.failHardwareConversation("SECOND");
assert.equal(first, second);
assert.equal(f.r.state.phase, "stopping");
await f.r.handleProviderEvent({ type: "response.done" });
assert.equal(f.r.state.phase, "stopping");
release();
await first;
assert.equal(f.events.length, 1);
assert.equal(f.events[0].errorCode, "FIRST");
await f.r.handleProviderEvent({ type: "transport.close", code: 1011 });
assert.equal(f.r.state.checks.get("provider_closed").status, "failed");
f.r.controls.textQuestion.value = "should not send";
await f.r.sendTextQuestion();
assert.equal(f.texts.length, 0);
});
test("new call clears stale checks, duration and failure while retaining device preparation", async () => {
const f = fixture();
f.r.state.platformSession = f.created;
f.r.state.failure = "old failure";
await f.r.failHardwareConversation("OLD");
for (const id of f.r.state.checks.keys()) f.r.setCheck(id, "passed", "old");
f.get("duration").textContent = "48:15";
f.r.resetSessionState();
assert.equal(f.r.state.failure, undefined);
assert.equal(f.r.state.failureTask, undefined);
assert.equal(f.get("duration").textContent, "00:00");
assert.equal(f.r.state.checks.get("platform_authenticated").status, "passed");
assert.equal(f.r.state.checks.get("credential_resolved").status, "passed");
assert.equal(f.r.state.checks.get("platform_completed").status, "pending");
f.r.state.platformSession = { ...f.created, conversation_id: "next-conversation" };
await f.r.failHardwareConversation("NEW");
assert.equal(f.events.length, 2);
assert.equal(f.events[1].conversationID, "next-conversation");
assert.equal(f.events[1].sequenceNo, 1);
});
test("failed platform receipt does not trigger a new terminal event on hangup", async () => {
let attempts = 0;
const f = fixture({ hardwareAgentEvent: async () => { attempts += 1; throw new Error("network unavailable"); } });
f.r.state.platformSession = f.created;
await f.r.failHardwareConversation("FIRST");
await f.r.stopHardwareConversation();
assert.equal(attempts, 1);
assert.equal(f.r.state.phase, "error");
assert.equal(f.r.buildReport().checks.find((c) => c.id === "platform_completed").status, "failed");
});
test("connection rejection without a provider event still reports failure once", async () => {
const f = fixture({ connectHardwareAgent: async () => { throw new Error("connect timeout"); } });
f.r.setPhase("ready", "ready");
await f.r.startHardwareConversation();
assert.equal(f.events.length, 1);
assert.equal(f.events[0].errorCode, "WINDOWS_HARDWARE_START_FAILED");
assert.match(f.r.state.failure, /connect timeout/);
assert.equal(f.r.state.phase, "error");
});
test("normal hangup still reports completed and closes normally", async () => {
const f = fixture();
f.ipc.hardwareAgentEvent = async (input) => { f.events.push(input); return { status: "COMPLETED" }; };
f.r.state.platformSession = f.created;
f.r.setPhase("active", "active");
await f.r.stopHardwareConversation();
assert.equal(f.events.length, 1);
assert.equal(f.events[0].eventType, "session_ended");
assert.equal(f.closes[0].code, 1000);
assert.equal(f.r.state.phase, "completed");
assert.equal(f.r.state.failureTask, undefined);
assert.equal(f.r.controls.start.disabled, false);
});
+795
View File
@@ -0,0 +1,795 @@
const { app, BrowserWindow, dialog, ipcMain, safeStorage, session } = require("electron");
const crypto = require("node:crypto");
const fs = require("node:fs");
const path = require("node:path");
const tls = require("node:tls");
const { HardwareAgentClient } = require("./hardware-agent-client.cjs");
app.commandLine.appendSwitch("autoplay-policy", "no-user-gesture-required");
const automatedUI = process.env.SIMULATOR_AUTO_TEST === "1" || process.env.SIMULATOR_REAL_E2E === "1";
if (automatedUI) {
app.commandLine.appendSwitch("use-fake-device-for-media-stream");
app.commandLine.appendSwitch("use-fake-ui-for-media-stream");
}
app.disableHardwareAcceleration();
let deviceConfig = null;
let desiredConfigRevision = 1;
let mainWindow = null;
let hardwareConfig = null;
let hardwareClient = null;
let activeHardwarePlatformSession = null;
function createWindow() {
const window = new BrowserWindow({
width: 1220,
height: 820,
minWidth: 980,
minHeight: 700,
backgroundColor: "#eef3f0",
autoHideMenuBar: true,
title: "语音终端模拟器",
webPreferences: {
preload: path.join(__dirname, "preload.cjs"),
contextIsolation: true,
nodeIntegration: false,
sandbox: true,
},
});
mainWindow = window;
window.once("closed", () => {
if (mainWindow === window) mainWindow = null;
});
const devArgument = process.argv.find((argument) => argument.startsWith("--dev-server-url="));
const devServerURL = devArgument ? devArgument.slice("--dev-server-url=".length) : "";
window.webContents.on("did-fail-load", (_event, errorCode, errorDescription, validatedURL) => {
console.error("renderer load failed", { errorCode, errorDescription, validatedURL });
});
window.webContents.on("render-process-gone", (_event, details) => {
console.error("renderer process exited", details);
});
window.webContents.on("console-message", (_event, ...args) => {
const details = args[0];
const message = typeof details === "object" && details ? details.message : String(args[1] ?? details ?? "");
console.log(`[renderer] ${message}`);
});
window.webContents.once("did-finish-load", async () => {
try {
const diagnostics = await window.webContents.executeJavaScript(`JSON.stringify({readyState: document.readyState, bodyLength: document.body.innerText.length, appChildren: document.querySelector('#app')?.childElementCount || 0})`);
console.log(`[renderer-ready] ${diagnostics}`);
} catch (error) {
console.error("renderer diagnostics failed", error);
}
if (automatedUI) {
void runAutomatedSmoke(window);
}
});
if (devServerURL) {
window.loadURL(devServerURL);
} else {
window.loadFile(path.join(__dirname, "..", "dist", "index.html"));
}
const screenshotPath = process.env.SIMULATOR_SCREENSHOT_PATH;
if (screenshotPath) {
window.webContents.once("did-finish-load", () => {
setTimeout(async () => {
try {
const image = await window.webContents.capturePage();
fs.writeFileSync(screenshotPath, image.toPNG());
} catch (error) {
console.error("simulator screenshot failed", error);
process.exitCode = 1;
} finally {
app.quit();
}
}, 1200);
});
}
}
async function runAutomatedSmoke(window) {
const delay = (milliseconds) => new Promise((resolve) => setTimeout(resolve, milliseconds));
await delay(800);
await window.webContents.executeJavaScript(`document.querySelector('#checkButton')?.click()`);
for (let attempt = 0; attempt < 30; attempt += 1) {
await delay(500);
const state = await window.webContents.executeJavaScript(`JSON.stringify({phase: document.querySelector('#phaseBadge')?.textContent, canStart: !document.querySelector('#startButton')?.disabled, detail: document.querySelector('#apiDetail')?.textContent})`);
console.log(`[auto-test] ${state}`);
const parsed = JSON.parse(state);
if (parsed.canStart) {
await window.webContents.executeJavaScript(`document.querySelector('#startButton')?.click()`);
console.log("[auto-test] start requested");
await collectAutomatedConversationResult(window);
return;
}
if (parsed.phase === "异常") {
return;
}
}
console.error("[auto-test] environment check did not become ready");
}
async function collectAutomatedConversationResult(window) {
const delay = (milliseconds) => new Promise((resolve) => setTimeout(resolve, milliseconds));
const deadline = Date.now() + 120_000;
let realQuestionSent = false;
let snapshot = {};
while (Date.now() < deadline) {
await delay(1000);
const encoded = await window.webContents.executeJavaScript(`JSON.stringify({
phase: document.querySelector('#phaseBadge')?.textContent || '',
conversation_id: document.querySelector('#conversationID')?.textContent || '',
model_mode: document.querySelector('#modelMode')?.value || '',
subject: [...document.querySelectorAll('.transcript-turn--subject span')].map((item) => item.textContent || ''),
agent: [...document.querySelectorAll('.transcript-turn--agent span')].map((item) => item.textContent || ''),
turns: [...document.querySelectorAll('.transcript-turn')].map((item) => ({speaker: item.classList.contains('transcript-turn--subject') ? 'subject' : 'agent', text: item.querySelector('span')?.textContent || ''})),
events: [...document.querySelectorAll('#eventLog li span')].map((item) => item.textContent || ''),
checks: [...document.querySelectorAll('#flowChecks .flow-check')].map((item) => ({label: item.querySelector('span')?.textContent || '', detail: item.querySelector('small')?.textContent || '', status: item.classList.contains('flow-check--passed') ? 'passed' : item.classList.contains('flow-check--failed') ? 'failed' : 'pending'}))
})`);
snapshot = JSON.parse(encoded);
console.log(`[auto-test] conversation ${encoded}`);
if (process.env.SIMULATOR_REAL_E2E === "1" && snapshot.phase === "通话中" && !realQuestionSent) {
const question = String(process.env.SIMULATOR_REAL_E2E_QUESTION || "请用一句简短的话介绍你自己。 ").trim();
await window.webContents.executeJavaScript(`(() => { const input = document.querySelector('#textQuestion'); if (input) input.value = ${JSON.stringify(question)}; document.querySelector('#sendTextButton')?.click(); })()`);
realQuestionSent = true;
console.log("[real-e2e] text question sent");
}
const subjectIndex = snapshot.turns?.findIndex((turn) => turn.speaker === "subject") ?? -1;
const hasAgentReply = subjectIndex >= 0 && snapshot.turns.slice(subjectIndex + 1).some((turn) => turn.speaker === "agent");
if (hasAgentReply) {
const hardwarePage = await window.webContents.executeJavaScript(`Boolean(document.querySelector('#flowChecks'))`);
const waitForPlatformStop = hardwarePage && process.env.SIMULATOR_AUTO_WAIT_FOR_PLATFORM_STOP === "1";
if (!waitForPlatformStop) await window.webContents.executeJavaScript(`document.querySelector('#stopButton')?.click()`);
if (hardwarePage) {
const stopDeadline = Date.now() + 30_000;
while (Date.now() < stopDeadline) {
await delay(500);
const phase = await window.webContents.executeJavaScript(`document.querySelector('#phaseBadge')?.textContent || ''`);
if (phase === "已完成" || phase === "异常") break;
}
const finalEncoded = await window.webContents.executeJavaScript(`JSON.stringify({
phase: document.querySelector('#phaseBadge')?.textContent || '',
conversation_id: document.querySelector('#conversationID')?.textContent || '',
model_mode: document.querySelector('#modelMode')?.value || '',
turns: [...document.querySelectorAll('.transcript-turn')].map((item) => ({speaker: item.classList.contains('transcript-turn--subject') ? 'subject' : 'agent', text: item.querySelector('span')?.textContent || ''})),
events: [...document.querySelectorAll('#eventLog li span')].map((item) => item.textContent || ''),
checks: [...document.querySelectorAll('#flowChecks .flow-check')].map((item) => ({label: item.querySelector('span')?.textContent || '', detail: item.querySelector('small')?.textContent || '', status: item.classList.contains('flow-check--passed') ? 'passed' : item.classList.contains('flow-check--failed') ? 'failed' : 'pending'}))
})`);
snapshot = JSON.parse(finalEncoded);
snapshot.result = snapshot.phase === "已完成" ? "PASS" : "ERROR";
} else {
snapshot.result = "PASS";
}
break;
}
if (snapshot.phase === "异常") {
snapshot.result = "ERROR";
break;
}
}
if (!snapshot.result) snapshot.result = "TIMEOUT";
const resultPath = process.env.SIMULATOR_AUTO_RESULT_PATH;
if (resultPath) {
fs.writeFileSync(path.resolve(resultPath), JSON.stringify(snapshot, null, 2), "utf8");
}
await delay(4000);
app.quit();
}
function ensureAllowedAPIURL(value) {
const parsed = new URL(String(value || ""));
const loopback = parsed.hostname === "127.0.0.1" || parsed.hostname === "localhost" || parsed.hostname === "[::1]";
const deployedTestServer = parsed.protocol === "https:" && parsed.hostname === "121.41.46.238" && parsed.port === "";
if ((!loopback || (parsed.protocol !== "http:" && parsed.protocol !== "https:")) && !deployedTestServer) {
throw new Error("测试设备只允许连接本机平台或已部署的测试服务器");
}
return parsed.origin;
}
function configure(input) {
const apiURL = ensureAllowedAPIURL(input?.apiURL);
const deviceID = String(input?.deviceID || "").trim();
const deviceSecret = String(input?.deviceSecret || "") || loadDefaultDeviceSecret();
if (!/^[0-9a-f-]{36}$/i.test(deviceID)) {
throw new Error("设备 ID 必须是 UUID");
}
if (deviceSecret.length < 32) {
throw new Error("未找到有效的设备密钥。请使用云端设备启动脚本,或在设备签名密钥中输入至少32个字符");
}
deviceConfig = { apiURL, deviceID, deviceSecret };
return { apiURL, deviceID };
}
function loadDefaultDeviceSecret() {
const environmentSecret = String(process.env.SIMULATOR_DEFAULT_DEVICE_SECRET || "").trim();
if (environmentSecret) return environmentSecret;
const candidates = [];
const localAppData = String(process.env.LOCALAPPDATA || "").trim();
const userProfile = String(process.env.USERPROFILE || "").trim();
const portableDirectory = String(process.env.PORTABLE_EXECUTABLE_DIR || "").trim();
if (localAppData) {
candidates.push(path.join(localAppData, "DataTerminalHardwareAgent", "server-device-secret"));
candidates.push(path.join(localAppData, "DataTerminalVoiceDevice", "server-device-secret"));
}
if (userProfile) candidates.push(path.join(userProfile, "AppData", "Local", "DataTerminalVoiceDevice", "server-device-secret"));
try {
const home = app.getPath("home");
if (home) candidates.push(path.join(home, "AppData", "Local", "DataTerminalVoiceDevice", "server-device-secret"));
} catch { /* app path is unavailable before ready */ }
if (portableDirectory) candidates.push(path.join(portableDirectory, ".server-device-secret"));
candidates.push(path.join(path.dirname(process.execPath), ".server-device-secret"));
for (const secretPath of new Set(candidates)) {
try {
const stats = fs.statSync(secretPath);
if (!stats.isFile() || stats.size < 32 || stats.size > 4096) continue;
const value = fs.readFileSync(secretPath, "utf8").trim();
if (value.length >= 32) return value;
} catch { /* try the next fixed credential location */ }
}
return "";
}
ipcMain.handle("sim:get-runtime-defaults", () => ({
apiURL: process.env.SIMULATOR_DEFAULT_API_URL || "https://121.41.46.238",
deviceID: process.env.SIMULATOR_DEFAULT_DEVICE_ID || "10000000-0000-0000-0000-000000000512",
deviceSecret: "",
deviceSecretConfigured: loadDefaultDeviceSecret().length >= 32,
}));
function hardwareCredentialPath() {
return path.join(app.getPath("userData"), "volc-hardware-device-credential.json");
}
function readSavedHardwareMetadata() {
try {
const stored = JSON.parse(fs.readFileSync(hardwareCredentialPath(), "utf8"));
if (!stored.instanceID || !stored.productKey || !stored.deviceName || !stored.encryptedDeviceSecret) return undefined;
return stored;
} catch { return undefined; }
}
function loadSavedHardwareCredential(identity) {
if (!safeStorage.isEncryptionAvailable()) return undefined;
try {
const stored = JSON.parse(fs.readFileSync(hardwareCredentialPath(), "utf8"));
if (stored.schemaVersion !== 3 || stored.instanceID !== identity.instanceID || stored.productKey !== identity.productKey || stored.deviceName !== identity.deviceName) return undefined;
const secret = safeStorage.decryptString(Buffer.from(stored.encryptedDeviceSecret, "base64"));
if (secret.length < 16) return undefined;
return { deviceSecret: secret, hardwareID: String(stored.hardwareID || identity.hardwareID || "") };
} catch { return undefined; }
}
function saveHardwareCredential(identity, deviceSecret, metadata = {}) {
if (!safeStorage.isEncryptionAvailable()) return false;
const target = hardwareCredentialPath();
fs.mkdirSync(path.dirname(target), { recursive: true });
const payload = {
schemaVersion: 3,
instanceID: identity.instanceID,
productKey: identity.productKey,
deviceName: identity.deviceName,
hardwareID: identity.hardwareID,
platformDeviceID: String(deviceConfig?.deviceID || ""),
credentialRef: String(metadata.credentialRef || ""),
credentialVersion: Number(metadata.credentialVersion || 1),
encryptedDeviceSecret: safeStorage.encryptString(deviceSecret).toString("base64"),
updatedAt: new Date().toISOString(),
};
const temporary = `${target}.${process.pid}.tmp`;
fs.writeFileSync(temporary, `${JSON.stringify(payload, null, 2)}\n`, { encoding: "utf8", mode: 0o600 });
fs.renameSync(temporary, target);
return true;
}
function hardwareIdentity(input) {
const instanceID = String(input?.instanceID || "").trim();
const productKey = String(input?.productKey || "").trim();
const deviceName = String(input?.deviceName || "").trim();
const hardwareID = String(input?.hardwareID || "").trim();
if (!instanceID || instanceID.length > 200) throw new Error("实例 ID 不能为空或过长");
if (!productKey || productKey.length > 200) throw new Error("Product Key 不能为空或过长");
if (!/^[a-zA-Z0-9_\-.:@]{1,128}$/.test(deviceName)) throw new Error("火山设备名称格式不正确");
if (!hardwareID || hardwareID.length > 200) throw new Error("Hardware ID 不能为空或过长");
return { instanceID, productKey, deviceName, hardwareID };
}
ipcMain.handle("sim:get-hardware-runtime-defaults", () => {
const candidateMetadata = readSavedHardwareMetadata();
const legacyMock = candidateMetadata && [candidateMetadata.instanceID, candidateMetadata.productKey, candidateMetadata.deviceName, candidateMetadata.hardwareID]
.some((value) => String(value || "").toLowerCase().includes("mock") || String(value || "").toLowerCase().includes("e2e"));
const savedMetadata = automatedUI || !legacyMock ? candidateMetadata : undefined;
const instanceID = String(process.env.SIMULATOR_VOLC_INSTANCE_ID || savedMetadata?.instanceID || "").trim();
const productKey = String(process.env.SIMULATOR_VOLC_PRODUCT_KEY || savedMetadata?.productKey || "").trim();
const deviceName = String(process.env.SIMULATOR_VOLC_DEVICE_NAME || savedMetadata?.deviceName || "windows-hardware-agent-test").trim();
const hardwareID = String(process.env.SIMULATOR_VOLC_HARDWARE_ID || savedMetadata?.hardwareID || `windows-${crypto.createHash("sha256").update(app.getPath("userData")).digest("hex").slice(0, 24)}`).trim();
const modelMode = String(process.env.SIMULATOR_VOLC_MODEL_MODE || "llm").trim().toLowerCase() === "s2s" ? "s2s" : "llm";
const saved = instanceID && productKey ? loadSavedHardwareCredential({ instanceID, productKey, deviceName, hardwareID }) : undefined;
return { instanceID, productKey, deviceName, hardwareID, modelMode, deviceCredentialConfigured: Boolean(saved) };
});
ipcMain.handle("sim:configure-hardware", async (_event, input) => {
if (!deviceConfig) throw new Error("请先连接平台");
const requestedHardwareID = String(input?.hardwareID || "").trim();
if (!requestedHardwareID || requestedHardwareID.length > 200) throw new Error("Hardware ID 不能为空或过长");
const automatedTest = process.env.SIMULATOR_AUTO_TEST === "1";
let provisioned;
if (automatedTest) {
provisioned = {
instance_id: String(process.env.SIMULATOR_VOLC_INSTANCE_ID || "").trim(),
product_key: String(process.env.SIMULATOR_VOLC_PRODUCT_KEY || "").trim(),
device_name: String(process.env.SIMULATOR_VOLC_DEVICE_NAME || "").trim(),
hardware_id: String(process.env.SIMULATOR_VOLC_HARDWARE_ID || requestedHardwareID).trim(),
device_secret: String(process.env.SIMULATOR_VOLC_DEVICE_SECRET || "").trim(),
credential_source: "isolated_test_override",
credential_version: 1,
credential_ref: "test-only",
};
} else {
provisioned = await deviceRequest("POST", "/device/v3/hardware-agent-credentials:provision", {
hardware_id: requestedHardwareID,
}, `windows-hardware-provision-${nonce()}`);
}
const identity = hardwareIdentity({
instanceID: provisioned?.instance_id,
productKey: provisioned?.product_key,
deviceName: provisioned?.device_name,
hardwareID: provisioned?.hardware_id,
});
const deviceSecret = String(provisioned?.device_secret || "").trim();
if (deviceSecret.length < 16 || deviceSecret.length > 4096) throw new Error("Device Secret 长度不正确");
const persisted = automatedTest ? false : saveHardwareCredential(identity, deviceSecret, {
credentialRef: provisioned?.credential_ref,
credentialVersion: provisioned?.credential_version,
});
if (!automatedTest && !persisted) throw new Error("Windows 安全存储不可用,拒绝保存 Device Secret");
hardwareConfig = { ...identity, deviceSecret };
// 首次预配会由平台发布产品级默认 Bot/传输配置,并可能推进配置版本。
// 设备必须重新拉取版本,后续创建会话时不能继续上报预配前的旧 revision。
if (!automatedTest) {
const configuration = await deviceRequest("GET", "/device/v2/config");
desiredConfigRevision = Number(configuration?.revision || desiredConfigRevision || 1);
}
return {
...identity,
registered: String(provisioned?.credential_source || "").includes("dynamic_registration"),
persisted,
credentialSource: String(provisioned?.credential_source || "platform_provisioning"),
credentialVersion: Number(provisioned?.credential_version || 1),
configRevision: desiredConfigRevision,
};
});
function nonce() {
return crypto.randomBytes(16).toString("hex");
}
function signedHeaders(method, requestURI, body) {
if (!deviceConfig) throw new Error("请先连接本地平台");
const timestamp = new Date().toISOString();
const requestNonce = nonce();
const digest = crypto.createHash("sha256").update(body).digest("hex");
const canonical = [method.toUpperCase(), requestURI, timestamp, requestNonce, digest].join("\n");
const signature = crypto.createHmac("sha256", deviceConfig.deviceSecret).update(canonical).digest("base64url");
return {
"X-Device-Id": deviceConfig.deviceID,
"X-Device-Timestamp": timestamp,
"X-Device-Nonce": requestNonce,
"X-Device-Content-SHA256": digest,
"X-Device-Signature": signature,
};
}
async function deviceRequest(method, requestPath, payload, idempotencyKey) {
if (!deviceConfig) throw new Error("请先连接本地平台");
const target = new URL(requestPath, `${deviceConfig.apiURL}/`);
const body = payload === undefined ? "" : JSON.stringify(payload);
const headers = signedHeaders(method, `${target.pathname}${target.search}`, body);
headers.Accept = "application/json";
if (body) headers["Content-Type"] = "application/json";
if (idempotencyKey) headers["Idempotency-Key"] = idempotencyKey;
const response = await fetch(target, { method, headers, body: body || undefined });
const text = await response.text();
let parsed;
try {
parsed = text ? JSON.parse(text) : {};
} catch {
parsed = { message: text };
}
if (!response.ok) {
const code = parsed?.error?.code || parsed?.code || `HTTP_${response.status}`;
const message = parsed?.error?.message || parsed?.message || "本地平台请求失败";
throw new Error(`${code}: ${message}`);
}
return parsed?.data ?? parsed;
}
ipcMain.handle("sim:configure", (_event, input) => configure(input));
ipcMain.handle("sim:probe", async () => {
const query = new URLSearchParams({
firmware_version: "0.3.2-windows-hardware-agent",
board_rev: "windows-x64",
rtc_sdk_version: "web-4.69.0-electron",
codec: "opus",
sample_rate: "48000",
network: "windows-local",
device_time: new Date().toISOString(),
});
const bootstrap = await deviceRequest("GET", `/device/v2/bootstrap?${query.toString()}`);
const configuration = await deviceRequest("GET", "/device/v2/config");
desiredConfigRevision = Number(configuration.revision || bootstrap?.device?.config_revision || 1);
return { bootstrap, configuration, desiredConfigRevision };
});
const rtcWebEndpoints = [
"access.rtc.volcvideo.com",
"access2-hl.rtc.volcvideo.com",
"common.rtc.volcvideo.com",
"rtcg-access.volcvideos.com",
"rtcg.volcvideos.com",
"web-log-report.rtc.volcvideo.com",
"web-log-report.volcvideos.com",
];
function probeTLSEndpoint(hostname) {
return new Promise((resolve) => {
const socket = tls.connect({ host: hostname, port: 443, servername: hostname, rejectUnauthorized: true });
let finished = false;
const finish = (result) => {
if (finished) return;
finished = true;
socket.destroy();
resolve(result);
};
socket.setTimeout(5_000, () => finish({ hostname, ok: false, error: "连接超时" }));
socket.once("secureConnect", () => finish({ hostname, ok: true }));
socket.once("error", (error) => finish({ hostname, ok: false, error: String(error?.code || error?.message || "连接失败") }));
});
}
ipcMain.handle("sim:probe-rtc-network", async () => {
const endpoints = await Promise.all(rtcWebEndpoints.map(probeTLSEndpoint));
return { endpoints };
});
ipcMain.handle("sim:probe-hardware-network", async () => {
const endpoints = await Promise.all([
probeTLSEndpoint("ai-gateway.vei.volces.com"),
probeTLSEndpoint("iot-cn-shanghai.iot.volces.com"),
]);
return { endpoints };
});
ipcMain.handle("sim:get-automation-audio", async () => {
const configuredPath = String(process.env.SIMULATOR_AUTOMATED_AUDIO_PATH || "").trim();
if (!configuredPath) return undefined;
const resolvedPath = path.resolve(configuredPath);
const stats = fs.statSync(resolvedPath);
if (!stats.isFile() || stats.size < 1 || stats.size > 10 * 1024 * 1024) {
throw new Error("自动验证音频必须是 1 字节到 10MB 的文件");
}
const extension = path.extname(resolvedPath).toLowerCase();
const mime = extension === ".mp3" ? "audio/mpeg" : extension === ".ogg" ? "audio/ogg" : "audio/wav";
return {
data_url: `data:${mime};base64,${fs.readFileSync(resolvedPath).toString("base64")}`,
source_name: path.basename(resolvedPath),
};
});
ipcMain.handle("sim:create-conversation", async () => deviceRequest("POST", "/device/v2/conversations", {
device_time: new Date().toISOString(),
monotonic_ms: Math.max(0, Math.round(performance.now())),
capabilities: {
rtc: true,
audio_tracks: ["near", "far"],
upload_resume: true,
rtc_credential_renewal: true,
},
network: { type: "windows-local" },
requested_goal: "open_conversation",
}, `windows-create-${nonce()}`));
ipcMain.handle("sim:create-hardware-agent-session", async () => deviceRequest("POST", "/device/v3/agent-sessions", {
device_time: new Date().toISOString(),
monotonic_ms: Math.max(0, Math.round(performance.now())),
capabilities: {
hardware_agent: true,
sdk_version: "1.1.0-windows-standard-protocol",
transports: ["low_load"],
input_audio_format: "pcm_s16le_16000_mono",
output_audio_format: "pcm_s16le_16000_mono",
server_event_callback: false,
transcript_upload: true,
dynamic_session_config: true,
credential_rotation: true,
near_pcm_recording: false,
far_pcm_recording: false,
},
network: { type: "windows-local" },
requested_goal: "open_conversation",
config_revision: desiredConfigRevision,
}, `windows-hardware-create-${nonce()}`));
function emitHardwareEvent(event) {
if (!mainWindow || mainWindow.isDestroyed()) return;
mainWindow.webContents.send("sim:hardware-provider-event", event);
}
function allowedHardwareGatewayURL() {
const configured = String(process.env.SIMULATOR_HARDWARE_GATEWAY_URL || "wss://ai-gateway.vei.volces.com/v1/realtime").trim();
const parsed = new URL(configured);
const official = parsed.protocol === "wss:" && parsed.hostname === "ai-gateway.vei.volces.com";
const local = (parsed.protocol === "ws:" || parsed.protocol === "wss:") && ["127.0.0.1", "localhost", "[::1]"].includes(parsed.hostname);
if (!official && !local) throw new Error("硬件智能体网关只允许火山官方地址或本机测试地址");
return parsed.toString();
}
async function invokeHardwareMemoryTool(event) {
const client = hardwareClient;
const platformSession = activeHardwarePlatformSession;
if (!client || !platformSession) return;
const callID = String(event.call_id || "");
const functionName = String(client.pendingCalls.get(callID) || "").toLowerCase();
if (!["memory_search", "search_memory", "retrieve_memory"].includes(functionName)) return;
let argumentsValue = {};
try { argumentsValue = JSON.parse(String(event.arguments || "{}")); } catch { /* invalid input is handled below */ }
const query = String(argumentsValue.query || argumentsValue.keyword || argumentsValue.text || "").trim();
const limit = Math.max(1, Math.min(5, Number(argumentsValue.limit || 3)));
try {
if (!query) throw new Error("记忆工具缺少 query 参数");
const baseURL = new URL(deviceConfig.apiURL);
const target = new URL(platformSession.session_config.tool_url, `${baseURL.origin}/`);
if (target.origin !== baseURL.origin) throw new Error("记忆工具地址必须与平台同源");
const response = await fetch(target, {
method: "POST",
headers: {
Accept: "application/json",
Authorization: `Bearer ${platformSession.session_config.tool_scope_token}`,
"Content-Type": "application/json",
},
body: JSON.stringify({ query, limit }),
});
const raw = await response.text();
let body;
try { body = raw ? JSON.parse(raw) : {}; } catch { body = { message: raw }; }
if (!response.ok) throw new Error(body?.error?.message || body?.message || `HTTP ${response.status}`);
const result = body?.data ?? body;
client.completeFunctionCall(callID, result);
emitHardwareEvent({ type: "platform.memory_search.completed", call_id: callID, query, result_count: result?.items?.length || 0, degraded: Boolean(result?.degraded) });
} catch (error) {
client.completeFunctionCall(callID, { items: [], degraded: true, error: error.message });
emitHardwareEvent({ type: "platform.memory_search.failed", call_id: callID, message: error.message });
}
}
ipcMain.handle("sim:connect-hardware-agent", async (_event, input) => {
if (!deviceConfig) throw new Error("请先连接平台");
if (!hardwareConfig) throw new Error("请先完成火山设备鉴权配置");
if (hardwareClient) throw new Error("已有硬件智能体连接,请先结束");
const platformSession = input?.platformSession;
if (!platformSession?.runtime?.bot_id || !platformSession?.session_config?.dialog_id) throw new Error("平台硬件会话配置不完整");
if (platformSession.runtime.transport !== "low_load") throw new Error("Windows 标准协议测试程序只接受设备绑定为 low_load 的会话");
const modelMode = String(input?.modelMode || "llm").trim().toLowerCase();
if (!new Set(["llm", "s2s"]).has(modelMode)) throw new Error("智能体模型类型必须为 llm 或 s2s");
activeHardwarePlatformSession = platformSession;
const client = new HardwareAgentClient({
gatewayURL: allowedHardwareGatewayURL(),
emit: (event) => {
emitHardwareEvent(event);
if (event.type === "response.function_call_arguments.done") void invokeHardwareMemoryTool(event);
},
});
hardwareClient = client;
try {
return await client.connect({
...hardwareConfig,
botID: platformSession.runtime.bot_id,
sessionConfig: platformSession.session_config,
modelMode,
});
} catch (error) {
hardwareClient = null;
activeHardwarePlatformSession = null;
throw error;
}
});
ipcMain.on("sim:hardware-agent-audio", (_event, bytes) => {
try { hardwareClient?.sendAudio(Buffer.from(bytes)); }
catch (error) { emitHardwareEvent({ type: "transport.audio_error", message: error.message }); }
});
ipcMain.handle("sim:hardware-agent-text", (_event, input) => {
if (!hardwareClient) throw new Error("火山硬件智能体连接未就绪");
hardwareClient.sendText(input?.text, Number(input?.interruptMode || 1));
return { accepted: true };
});
ipcMain.handle("sim:close-hardware-agent", (_event, input) => {
const client = hardwareClient;
hardwareClient = null;
activeHardwarePlatformSession = null;
client?.close(Number(input?.code || 1000), String(input?.reason || "device session ended").slice(0, 100));
return { closed: true };
});
ipcMain.handle("sim:save-hardware-report", async (_event, report) => {
const result = await dialog.showSaveDialog(mainWindow, {
title: "导出硬件对话智能体测试报告",
defaultPath: `hardware-agent-test-${new Date().toISOString().replace(/[:.]/g, "-")}.json`,
filters: [{ name: "JSON 测试报告", extensions: ["json"] }],
});
if (result.canceled || !result.filePath) return { saved: false };
const redact = (value) => {
if (Array.isArray(value)) return value.map(redact);
if (!value || typeof value !== "object") return value;
const output = {};
for (const [key, item] of Object.entries(value)) {
if (/secret|signature|authorization|tool_scope_token|systemmessages|system_role|system_prompt|authorized_context|speaking_style|instructions/i.test(key)) output[key] = "[REDACTED]";
else output[key] = redact(item);
}
return output;
};
const safeReport = redact(JSON.parse(JSON.stringify(report)));
fs.writeFileSync(result.filePath, `${JSON.stringify(safeReport, null, 2)}\n`, "utf8");
return { saved: true, path: result.filePath };
});
ipcMain.handle("sim:hardware-agent-event", async (_event, input) => deviceRequest(
"POST",
`/device/v3/agent-sessions/${encodeURIComponent(input.conversationID)}/events`,
{
event_id: input.eventID || `windows-hardware-event-${nonce()}`,
sequence_no: Number(input.sequenceNo),
event_type: input.eventType,
occurred_at: new Date().toISOString(),
provider_session_ref: input.providerSessionRef,
session_nonce: input.sessionNonce,
...(input.eventType === "session_started" ? {
provider_connection_ref: input.providerConnectionRef,
bot_id: input.botID,
transport: input.transport,
config_digest: input.configDigest,
dynamic_config_applied: input.dynamicConfigApplied,
} : {}),
...(input.errorCode ? { error_code: input.errorCode } : {}),
},
));
ipcMain.handle("sim:activate-conversation", async (_event, input) => deviceRequest(
"POST",
`/device/v2/conversations/${encodeURIComponent(input.conversationID)}/activate`,
{
joined_at: new Date().toISOString(),
room_id: input.roomID,
rtc_user_id: input.userID,
rtc_join_result: "success",
config_revision: desiredConfigRevision,
connection_reference: "windows-electron-web-rtc",
},
`windows-activate-${nonce()}`,
));
ipcMain.handle("sim:get-conversation", async (_event, conversationID) => deviceRequest(
"GET",
`/device/v2/conversations/${encodeURIComponent(conversationID)}`,
));
ipcMain.handle("sim:heartbeat", async (_event, input) => deviceRequest(
"POST",
`/device/v2/conversations/${encodeURIComponent(input.conversationID)}/heartbeat`,
{
sequence_no: input.sequenceNo,
device_time: new Date().toISOString(),
monotonic_ms: Math.max(0, Math.round(performance.now())),
conversation_status: "ACTIVE",
rtc: { joined: true, uplink: true, downlink: true },
network: { type: "windows-local" },
firmware_version: "0.3.2-windows-hardware-agent",
reported_config_revision: desiredConfigRevision,
rtc_credential_expires_at: input.credentialExpiresAt,
recording_queue: { pending_segments: 0, failed_segments: 0 },
command_results: [],
},
));
ipcMain.handle("sim:submit-transcript-batch", async (_event, input) => {
const conversationID = String(input?.conversationID || "");
const turns = Array.isArray(input?.turns) ? input.turns : [];
if (!conversationID || turns.length === 0) {
throw new Error("通话编号和最终话轮不能为空");
}
const payload = { turns };
const digest = crypto.createHash("sha256").update(JSON.stringify(payload)).digest("hex").slice(0, 32);
return deviceRequest(
"POST",
`/${input?.hardwareAgent ? "device/v3/agent-sessions" : "device/v2/conversations"}/${encodeURIComponent(conversationID)}/turns:batch`,
payload,
`windows-turns-${digest}`,
);
});
ipcMain.handle("sim:stop-conversation", async (_event, input) => deviceRequest(
"POST",
`/device/v2/conversations/${encodeURIComponent(input.conversationID)}/stop`,
{
reason: input.reason || "windows_user_hangup",
device_time: new Date().toISOString(),
last_heartbeat_sequence: input.lastHeartbeatSequence || 0,
rtc_left: true,
recording: { near_segments: Number(input.nearSegments || 0), far_segments: Number(input.farSegments || 0) },
},
`windows-stop-${nonce()}`,
));
ipcMain.handle("sim:upload-conversation-recording", async (_event, input) => {
const conversationID = String(input?.conversationID || "");
const supplied = Array.isArray(input?.segments) ? input.segments : [];
if (!conversationID || supplied.length === 0) throw new Error("通话编号和录音分片不能为空");
const segments = supplied.map((segment) => {
const bytes = Buffer.from(segment.bytes);
if (!bytes.length || bytes.length > 64 * 1024 * 1024) throw new Error("单个录音分片必须在 1 字节到 64MiB 之间");
return {
track: segment.track,
sequence_no: Number(segment.sequence_no),
codec: segment.codec,
content_type: segment.content_type,
duration_ms: Number(segment.duration_ms),
expected_size: bytes.length,
expected_digest: crypto.createHash("sha256").update(bytes).digest("hex"),
bytes,
};
});
const manifest = { segments: segments.map(({ bytes: _bytes, ...segment }) => segment) };
const manifestDigest = crypto.createHash("sha256").update(JSON.stringify(manifest)).digest("hex").slice(0, 40);
const batch = await deviceRequest(
"POST",
`/device/v2/conversations/${encodeURIComponent(conversationID)}/upload-batches`,
manifest,
`windows-media-${manifestDigest}`,
);
for (const local of segments) {
const remote = batch.segments?.find((segment) => segment.track === local.track && Number(segment.sequence_no) === local.sequence_no);
if (!remote) throw new Error(`平台未返回${local.track}轨道第${local.sequence_no}段录音`);
if (remote.status === "VERIFIED" || remote.status === "VERIFYING") continue;
if (!remote.upload_url) throw new Error(`平台未签发${local.track}轨道第${local.sequence_no}段上传地址`);
const response = await fetch(remote.upload_url, { method: "PUT", headers: { "Content-Type": local.content_type }, body: local.bytes });
if (!response.ok) throw new Error(`录音对象上传失败:HTTP ${response.status}`);
}
await deviceRequest("POST", `/device/v2/upload-batches/${encodeURIComponent(batch.id)}/complete`, {});
const deadline = Date.now() + 90_000;
while (Date.now() < deadline) {
const current = await deviceRequest("GET", `/device/v2/upload-batches/${encodeURIComponent(batch.id)}`);
if (current.status === "COMPLETED") return current;
if (current.status === "FAILED" || current.status === "EXPIRED") {
const rejected = current.segments?.find((segment) => segment.status === "REJECTED");
throw new Error(rejected?.last_error_code || current.last_error_code || `录音批次${current.status}`);
}
await new Promise((resolve) => setTimeout(resolve, 750));
}
throw new Error("录音已上传,但等待服务器校验超时");
});
app.whenReady().then(() => {
session.defaultSession.setPermissionCheckHandler((_webContents, permission) => permission === "media");
session.defaultSession.setPermissionRequestHandler((_webContents, permission, callback) => callback(permission === "media"));
createWindow();
});
app.on("window-all-closed", () => {
hardwareClient?.close(1001, "application closed");
hardwareClient = null;
activeHardwarePlatformSession = null;
if (process.platform !== "darwin") app.quit();
});
app.on("activate", () => {
if (BrowserWindow.getAllWindows().length === 0) createWindow();
});
@@ -0,0 +1,76 @@
const assert = require("node:assert/strict");
const test = require("node:test");
const fs = require("node:fs");
const path = require("node:path");
const vm = require("node:vm");
const source = fs.readFileSync(path.join(__dirname, "../public/pcm-playback.worklet.js"), "utf8");
function fixture({ outputRate = 48000, prebufferMs = 120 } = {}) {
const messages = [];
let Processor;
class AudioWorkletProcessor {
constructor() {
this.port = { onmessage: null, postMessage: (message) => messages.push(message) };
}
}
const context = vm.createContext({
AudioWorkletProcessor,
Float32Array,
ArrayBuffer,
Math,
sampleRate: outputRate,
registerProcessor: (name, implementation) => {
assert.equal(name, "pcm16-playback");
Processor = implementation;
},
});
vm.runInContext(source, context);
const processor = new Processor({ processorOptions: { sourceSampleRate: 16000, prebufferMs } });
const push = (values) => {
const samples = new Float32Array(values);
processor.port.onmessage({ data: { type: "push", samples: samples.buffer } });
};
const process = () => {
const output = new Float32Array(128);
assert.equal(processor.process([], [[output]]), true);
return output;
};
return { messages, processor, push, process };
}
test("waits for prebuffer then resamples continuous 16k PCM without fragment gaps", () => {
const f = fixture({ outputRate: 48000, prebufferMs: 10 });
f.push(new Array(120).fill(0.25));
assert.ok(f.process().every((sample) => sample === 0), "must remain silent below the prebuffer threshold");
f.push(new Array(400).fill(0.25));
const first = f.process();
const second = f.process();
assert.ok(first.some((sample) => sample > 0.05), "buffered playback should start with a short anti-pop fade");
assert.ok(second.every((sample) => sample > 0.12), "playback should remain continuous across source fragments");
assert.equal(f.messages.filter((message) => message.type === "underflow").length, 0);
});
test("plays 16k source at real-time duration instead of the former 1.5x speed", () => {
const f = fixture({ outputRate: 48000, prebufferMs: 1 });
f.push(new Array(1000).fill(0.25));
f.process();
assert.ok(Math.abs(f.processor.readPosition - 128 / 3) < 0.001, `consumed ${f.processor.readPosition} source samples`);
});
test("reports network starvation and re-enters prebuffering instead of scheduling tiny buffers", () => {
const f = fixture({ outputRate: 48000, prebufferMs: 0.01 });
f.push(new Array(8).fill(0.5));
f.process();
assert.equal(f.messages.filter((message) => message.type === "underflow").length, 1);
assert.ok(f.process().every((sample) => sample === 0), "starved playback should wait for a new prebuffer");
});
test("drains the final response cleanly and emits a completion signal", () => {
const f = fixture({ outputRate: 24000 });
f.push(new Array(64).fill(-0.25));
f.processor.port.onmessage({ data: { type: "end" } });
f.process();
assert.equal(f.messages.filter((message) => message.type === "drained").length, 1);
assert.equal(f.messages.filter((message) => message.type === "underflow").length, 0);
});
@@ -0,0 +1,31 @@
const { contextBridge, ipcRenderer } = require("electron");
contextBridge.exposeInMainWorld("deviceSimulator", {
getRuntimeDefaults: () => ipcRenderer.invoke("sim:get-runtime-defaults"),
configure: (input) => ipcRenderer.invoke("sim:configure", input),
probe: () => ipcRenderer.invoke("sim:probe"),
probeRTCNetwork: () => ipcRenderer.invoke("sim:probe-rtc-network"),
probeHardwareNetwork: () => ipcRenderer.invoke("sim:probe-hardware-network"),
getHardwareRuntimeDefaults: () => ipcRenderer.invoke("sim:get-hardware-runtime-defaults"),
configureHardware: (input) => ipcRenderer.invoke("sim:configure-hardware", input),
getAutomationAudio: () => ipcRenderer.invoke("sim:get-automation-audio"),
createConversation: () => ipcRenderer.invoke("sim:create-conversation"),
createHardwareAgentSession: () => ipcRenderer.invoke("sim:create-hardware-agent-session"),
connectHardwareAgent: (input) => ipcRenderer.invoke("sim:connect-hardware-agent", input),
sendHardwareAgentAudio: (bytes) => ipcRenderer.send("sim:hardware-agent-audio", bytes),
sendHardwareAgentText: (input) => ipcRenderer.invoke("sim:hardware-agent-text", input),
closeHardwareAgent: (input) => ipcRenderer.invoke("sim:close-hardware-agent", input),
onHardwareProviderEvent: (callback) => {
const listener = (_event, value) => callback(value);
ipcRenderer.on("sim:hardware-provider-event", listener);
return () => ipcRenderer.removeListener("sim:hardware-provider-event", listener);
},
saveHardwareReport: (report) => ipcRenderer.invoke("sim:save-hardware-report", report),
hardwareAgentEvent: (input) => ipcRenderer.invoke("sim:hardware-agent-event", input),
activateConversation: (input) => ipcRenderer.invoke("sim:activate-conversation", input),
getConversation: (conversationID) => ipcRenderer.invoke("sim:get-conversation", conversationID),
heartbeat: (input) => ipcRenderer.invoke("sim:heartbeat", input),
submitTranscriptBatch: (input) => ipcRenderer.invoke("sim:submit-transcript-batch", input),
stopConversation: (input) => ipcRenderer.invoke("sim:stop-conversation", input),
uploadConversationRecording: (input) => ipcRenderer.invoke("sim:upload-conversation-recording", input),
});
+13
View File
@@ -0,0 +1,13 @@
<!doctype html>
<html lang="zh-CN">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta http-equiv="Content-Security-Policy" content="default-src 'self'; script-src 'self' data:; style-src 'self'; connect-src 'self' data: https://121.41.46.238 https://*.rtc.volcvideo.com https://*.volcvideos.com wss: http://127.0.0.1:* http://localhost:*; media-src 'self' blob:; worker-src 'self' blob: data:; img-src 'self' data:;" />
<title>硬件对话智能体测试程序</title>
</head>
<body>
<main id="app"></main>
<script type="module" src="/src/hardware.ts"></script>
</body>
</html>
@@ -0,0 +1,21 @@
@echo off
setlocal
set "SECRET_FILE=%LOCALAPPDATA%\DataTerminalHardwareAgent\server-device-secret"
set "DEVICE_EXE=%~dp0hardware-agent-tester-0.3.5-windows-x64.exe"
if not exist "%DEVICE_EXE%" set "DEVICE_EXE=%~dp0release\hardware-agent-tester-0.3.5-windows-x64.exe"
if not exist "%SECRET_FILE%" (
echo Cloud device credential is missing: %SECRET_FILE%
pause
exit /b 1
)
if not exist "%DEVICE_EXE%" (
echo Hardware agent tester is missing: %DEVICE_EXE%
pause
exit /b 1
)
set /p SIMULATOR_DEFAULT_DEVICE_SECRET=<"%SECRET_FILE%"
set "SIMULATOR_DEFAULT_API_URL=https://121.41.46.238"
set "SIMULATOR_DEFAULT_DEVICE_ID=10000000-0000-0000-0000-000000000512"
start "" "%DEVICE_EXE%"
@@ -0,0 +1,46 @@
[CmdletBinding()]
param(
[switch]$RealE2E,
[string]$ResultPath = (Join-Path $env:TEMP "hardware-agent-real-e2e.json")
)
$ErrorActionPreference = "Stop"
$secretPath = Join-Path $env:LOCALAPPDATA "DataTerminalHardwareAgent\server-device-secret"
$executablePath = Join-Path $PSScriptRoot "hardware-agent-tester-0.3.5-windows-x64.exe"
if (-not (Test-Path -LiteralPath $executablePath -PathType Leaf)) {
$executablePath = Join-Path $PSScriptRoot "release\hardware-agent-tester-0.3.5-windows-x64.exe"
}
if (-not (Test-Path -LiteralPath $secretPath -PathType Leaf)) {
throw "Cloud device credential is missing: $secretPath"
}
if (-not (Test-Path -LiteralPath $executablePath -PathType Leaf)) {
throw "Hardware agent tester is missing: $executablePath"
}
$env:SIMULATOR_DEFAULT_API_URL = "https://121.41.46.238"
$env:SIMULATOR_DEFAULT_DEVICE_ID = "10000000-0000-0000-0000-000000000512"
$env:SIMULATOR_DEFAULT_DEVICE_SECRET = [IO.File]::ReadAllText($secretPath).Trim()
try {
if ($RealE2E) {
$env:SIMULATOR_REAL_E2E = "1"
$env:SIMULATOR_AUTO_RESULT_PATH = $ResultPath
$env:SIMULATOR_VOLC_MODEL_MODE = "llm"
$env:SIMULATOR_REAL_E2E_QUESTION = "Please introduce yourself in one short sentence."
$process = Start-Process -FilePath $executablePath -WindowStyle Hidden -PassThru -Wait
if ($process.ExitCode -ne 0) { throw "Hardware agent tester exited with code $($process.ExitCode)" }
if (-not (Test-Path -LiteralPath $ResultPath -PathType Leaf)) { throw "Real E2E report was not created: $ResultPath" }
}
else {
Start-Process -FilePath $executablePath
}
}
finally {
Remove-Item Env:SIMULATOR_DEFAULT_API_URL -ErrorAction SilentlyContinue
Remove-Item Env:SIMULATOR_DEFAULT_DEVICE_ID -ErrorAction SilentlyContinue
Remove-Item Env:SIMULATOR_DEFAULT_DEVICE_SECRET -ErrorAction SilentlyContinue
Remove-Item Env:SIMULATOR_REAL_E2E -ErrorAction SilentlyContinue
Remove-Item Env:SIMULATOR_AUTO_RESULT_PATH -ErrorAction SilentlyContinue
Remove-Item Env:SIMULATOR_VOLC_MODEL_MODE -ErrorAction SilentlyContinue
Remove-Item Env:SIMULATOR_REAL_E2E_QUESTION -ErrorAction SilentlyContinue
}
+13
View File
@@ -0,0 +1,13 @@
<!doctype html>
<html lang="zh-CN">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta http-equiv="Content-Security-Policy" content="default-src 'self'; script-src 'self' data:; style-src 'self'; connect-src 'self' data: https://121.41.46.238 https://*.rtc.volcvideo.com https://*.volcvideos.com wss: http://127.0.0.1:* http://localhost:*; media-src 'self' blob:; worker-src 'self' blob: data:; img-src 'self' data:;" />
<title>传统 RTC 语音终端模拟器</title>
</head>
<body>
<main id="app"></main>
<script type="module" src="/src/main.ts"></script>
</body>
</html>
File diff suppressed because it is too large Load Diff
+51
View File
@@ -0,0 +1,51 @@
{
"name": "data-terminal-voice-device-simulator",
"version": "0.3.5",
"private": true,
"description": "Windows end-to-end tester for Volcengine hardware conversational agents",
"main": "electron/main.cjs",
"scripts": {
"dev": "concurrently -k \"vite --host 127.0.0.1 --port 5177\" \"wait-on http://127.0.0.1:5177 && electron . --dev-server-url=http://127.0.0.1:5177\"",
"typecheck": "tsc --noEmit",
"test": "node --test electron/*.test.cjs",
"build:renderer": "vite build",
"start": "npm run build:renderer && electron .",
"build": "npm test && npm run typecheck && npm run build:renderer && electron-builder --win portable"
},
"dependencies": {
"@volcengine/rtc": "4.69.0",
"ws": "8.21.3"
},
"devDependencies": {
"concurrently": "^9.2.1",
"electron": "44.0.0",
"electron-builder": "26.15.3",
"typescript": "^5.9.2",
"vite": "^7.1.3",
"wait-on": "^8.0.4"
},
"build": {
"appId": "cn.dataterminal.hardware-agent-tester",
"productName": "硬件对话智能体测试程序",
"artifactName": "hardware-agent-tester-${version}-windows-${arch}.${ext}",
"directories": {
"output": "release"
},
"files": [
"dist/**/*",
"electron/**/*",
"!electron/*.test.cjs",
"package.json"
],
"win": {
"target": [
{
"target": "portable",
"arch": [
"x64"
]
}
]
}
}
}
@@ -0,0 +1,47 @@
class PCM16CaptureProcessor extends AudioWorkletProcessor {
constructor(options) {
super();
this.targetRate = Number(options.processorOptions?.targetSampleRate || 16000);
this.frameSamples = Number(options.processorOptions?.frameSamples || 320);
this.ratio = sampleRate / this.targetRate;
this.source = [];
this.position = 0;
this.output = [];
}
process(inputs) {
const channel = inputs[0]?.[0];
if (!channel?.length) return true;
for (const sample of channel) this.source.push(sample);
while (this.position + this.ratio <= this.source.length) {
const start = Math.floor(this.position);
const end = Math.max(start + 1, Math.floor(this.position + this.ratio));
let sum = 0;
for (let index = start; index < end && index < this.source.length; index += 1) sum += this.source[index];
this.output.push(sum / Math.max(1, end - start));
this.position += this.ratio;
if (this.output.length >= this.frameSamples) this.flush();
}
const consumed = Math.floor(this.position);
if (consumed > 0) {
this.source.splice(0, consumed);
this.position -= consumed;
}
return true;
}
flush() {
const frame = this.output.splice(0, this.frameSamples);
const pcm = new Int16Array(frame.length);
let energy = 0;
for (let index = 0; index < frame.length; index += 1) {
const sample = Math.max(-1, Math.min(1, frame[index]));
energy += sample * sample;
pcm[index] = sample < 0 ? Math.round(sample * 32768) : Math.round(sample * 32767);
}
const volume = Math.min(100, Math.round(Math.sqrt(energy / Math.max(1, frame.length)) * 280));
this.port.postMessage({ type: "pcm", bytes: pcm.buffer, volume }, [pcm.buffer]);
}
}
registerProcessor("pcm16-capture", PCM16CaptureProcessor);
@@ -0,0 +1,115 @@
class PCM16PlaybackProcessor extends AudioWorkletProcessor {
constructor(options) {
super();
const config = options.processorOptions || {};
this.sourceSampleRate = Number(config.sourceSampleRate || 16000);
this.prebufferSamples = Math.max(1, Math.round(this.sourceSampleRate * Number(config.prebufferMs || 120) / 1000));
this.capacity = Math.max(this.sourceSampleRate * 30, this.prebufferSamples * 4);
this.ring = new Float32Array(this.capacity);
this.readPosition = 0;
this.writePosition = 0;
this.playing = false;
this.ended = false;
this.drained = false;
this.lastOutput = 0;
this.fadeInRemaining = 0;
this.fadeFrames = Math.max(1, Math.round(sampleRate * 0.005));
this.underflows = 0;
this.port.onmessage = (message) => this.onMessage(message.data || {});
}
onMessage(message) {
if (message.type === "reset") {
this.reset();
return;
}
if (message.type === "end") {
this.ended = true;
return;
}
if (message.type !== "push" || !(message.samples instanceof ArrayBuffer)) return;
const samples = new Float32Array(message.samples);
if (this.drained) this.reset();
this.ended = false;
for (let index = 0; index < samples.length; index += 1) {
if (this.writePosition - Math.floor(this.readPosition) >= this.capacity - 1) {
this.readPosition = this.writePosition - this.capacity + 2;
}
this.ring[this.writePosition % this.capacity] = samples[index];
this.writePosition += 1;
}
}
reset() {
this.readPosition = 0;
this.writePosition = 0;
this.playing = false;
this.ended = false;
this.drained = false;
this.lastOutput = 0;
this.fadeInRemaining = 0;
}
available() {
return this.writePosition - this.readPosition;
}
startWhenReady() {
if (this.playing) return true;
const available = this.available();
if (available >= this.prebufferSamples || (this.ended && available > 1)) {
this.playing = true;
this.fadeInRemaining = this.fadeFrames;
this.port.postMessage({ type: "started", bufferedSamples: Math.floor(available) });
return true;
}
return false;
}
finish(output, fromIndex, eventType) {
const remaining = Math.min(this.fadeFrames, output.length - fromIndex);
for (let index = 0; index < remaining; index += 1) {
output[fromIndex + index] = this.lastOutput * (1 - (index + 1) / remaining);
}
output.fill(0, fromIndex + remaining);
this.lastOutput = 0;
this.playing = false;
if (eventType === "drained") {
if (!this.drained) this.port.postMessage({ type: "drained" });
this.drained = true;
} else {
this.underflows += 1;
this.port.postMessage({ type: "underflow", count: this.underflows });
}
}
process(_inputs, outputs) {
const output = outputs[0]?.[0];
if (!output) return true;
output.fill(0);
if (!this.startWhenReady()) return true;
const sourceStep = this.sourceSampleRate / sampleRate;
for (let index = 0; index < output.length; index += 1) {
const base = Math.floor(this.readPosition);
if (base + 1 >= this.writePosition) {
this.finish(output, index, this.ended ? "drained" : "underflow");
break;
}
const fraction = this.readPosition - base;
const first = this.ring[base % this.capacity];
const second = this.ring[(base + 1) % this.capacity];
let value = first + (second - first) * fraction;
if (this.fadeInRemaining > 0) {
value *= 1 - this.fadeInRemaining / this.fadeFrames;
this.fadeInRemaining -= 1;
}
output[index] = value;
this.lastOutput = value;
this.readPosition += sourceStep;
}
return true;
}
}
registerProcessor("pcm16-playback", PCM16PlaybackProcessor);
+150
View File
@@ -0,0 +1,150 @@
type SimulatorConfiguration = {
apiURL: string;
deviceID: string;
deviceSecret: string;
};
type SimulatorRuntimeDefaults = SimulatorConfiguration & {
deviceSecretConfigured: boolean;
};
type RTCGrant = {
app_id: string;
room_id: string;
user_id: string;
agent_user_id: string;
token: string;
expires_at: string;
};
type CreatedConversation = {
conversation_id: string;
status: string;
rtc: RTCGrant;
privacy: {
retention_mode: "none" | "transcript_only" | "transcript_and_audio";
record_near: boolean;
record_far: boolean;
upload_allowed: boolean;
};
session: {
engine_mode: string;
heartbeat_interval_sec: number;
allocation_expires_at: string;
};
};
type CreatedHardwareAgentSession = {
conversation_id: string;
status: "SESSION_READY";
runtime_type: "volc_hardware_agent";
runtime: {
provider: "volcengine";
bot_id: string;
transport: "high_quality" | "low_load";
provider_session_ref: string;
session_nonce: string;
config_digest: string;
wait_for_session_update: boolean;
};
session_config: {
dialog_id: string;
system_prompt?: string;
persona_tone?: string;
authorized_context: string;
prompt_bundle_version: string;
tool_url?: string;
tool_scope_token?: string;
};
privacy: CreatedConversation["privacy"];
session: CreatedConversation["session"];
};
type DeviceFinalTurn = {
turn_key: string;
sequence_no: number;
speaker: "subject" | "agent";
start_at: string;
end_at: string;
final_text: string;
interrupted: boolean;
};
interface Window {
deviceSimulator: {
getRuntimeDefaults(): Promise<SimulatorRuntimeDefaults>;
configure(input: SimulatorConfiguration): Promise<{ apiURL: string; deviceID: string }>;
probe(): Promise<Record<string, unknown>>;
probeRTCNetwork(): Promise<{ endpoints: Array<{ hostname: string; ok: boolean; error?: string }> }>;
probeHardwareNetwork(): Promise<{ endpoints: Array<{ hostname: string; ok: boolean; error?: string }> }>;
getHardwareRuntimeDefaults(): Promise<{
instanceID: string;
productKey: string;
deviceName: string;
hardwareID: string;
modelMode: "llm" | "s2s";
deviceCredentialConfigured: boolean;
}>;
configureHardware(input: {
hardwareID: string;
}): Promise<{
instanceID: string;
productKey: string;
deviceName: string;
hardwareID: string;
registered: boolean;
persisted: boolean;
credentialSource: string;
credentialVersion: number;
}>;
getAutomationAudio(): Promise<{ data_url: string; source_name: string } | undefined>;
createConversation(): Promise<CreatedConversation>;
createHardwareAgentSession(): Promise<CreatedHardwareAgentSession>;
connectHardwareAgent(input: { platformSession: CreatedHardwareAgentSession; modelMode: "llm" | "s2s" }): Promise<{
providerConnectionRef: string;
session: Record<string, unknown>;
}>;
sendHardwareAgentAudio(bytes: ArrayBuffer): void;
sendHardwareAgentText(input: { text: string; interruptMode?: number }): Promise<{ accepted: boolean }>;
closeHardwareAgent(input?: { code?: number; reason?: string }): Promise<{ closed: boolean }>;
onHardwareProviderEvent(callback: (event: Record<string, unknown>) => void): () => void;
saveHardwareReport(report: Record<string, unknown>): Promise<{ saved: boolean; path?: string }>;
hardwareAgentEvent(input: {
conversationID: string;
eventID?: string;
sequenceNo: number;
eventType: "session_started" | "session_updated" | "session_ended" | "session_failed";
providerSessionRef: string;
sessionNonce: string;
providerConnectionRef?: string;
botID?: string;
transport?: "high_quality" | "low_load";
configDigest?: string;
dynamicConfigApplied?: boolean;
errorCode?: string;
}): Promise<{ conversation_id: string; status: string; action: "continue" | "stop_session"; duplicate: boolean }>;
activateConversation(input: { conversationID: string; roomID: string; userID: string }): Promise<Record<string, unknown>>;
getConversation(conversationID: string): Promise<{ status: string; action: string; failure_code?: string }>;
heartbeat(input: { conversationID: string; sequenceNo: number; credentialExpiresAt: string }): Promise<{
actions?: Array<{ type: string; rtc?: RTCGrant }>;
desired_config_revision?: number;
}>;
submitTranscriptBatch(input: { conversationID: string; turns: DeviceFinalTurn[]; hardwareAgent?: boolean }): Promise<{
conversation_id: string;
accepted_count: number;
duplicate_count: number;
}>;
stopConversation(input: { conversationID: string; reason: string; lastHeartbeatSequence: number; nearSegments: number; farSegments: number }): Promise<Record<string, unknown>>;
uploadConversationRecording(input: {
conversationID: string;
segments: Array<{
track: "near" | "far";
sequence_no: number;
codec: "wav_pcm_s16le";
content_type: "audio/wav";
duration_ms: number;
bytes: Uint8Array;
}>;
}): Promise<{ id: string; status: string; segments: Array<{ id: string; track: "near" | "far"; sequence_no: number; status: string }> }>;
};
}
+778
View File
@@ -0,0 +1,778 @@
import "./style.css";
type HardwarePhase = "idle" | "checking" | "ready" | "connecting" | "active" | "stopping" | "completed" | "error";
type ReportCheck = { id: string; label: string; status: "pending" | "passed" | "failed"; detail?: string; at?: string };
type ReportEvent = { at: string; source: "app" | "platform" | "volcengine"; type: string; detail?: Record<string, unknown> };
const checkDefinitions: Array<[string, string]> = [
["platform_authenticated", "平台设备 HMAC 鉴权"],
["credential_resolved", "火山一机一密凭据"],
["context_frozen", "人设、背景与授权上下文冻结"],
["provider_connected", "火山硬件智能体 WebSocket 建连"],
["dynamic_config_applied", "session.updated 动态配置确认"],
["platform_active", "平台确认会话 ACTIVE"],
["microphone_streaming", "16kHz PCM 麦克风实时上行"],
["audio_playback_buffered", "16kHz PCM 连续缓冲播放"],
["heartbeat_ack", "平台 session_updated 存活回执"],
["user_transcript", "用户最终字幕"],
["agent_transcript", "智能体最终字幕"],
["transcript_uploaded", "最终话轮可靠上传"],
["provider_closed", "火山连接正常关闭"],
["platform_completed", "平台确认会话 COMPLETED"],
];
const state: {
phase: HardwarePhase;
platformSession?: CreatedHardwareAgentSession;
providerConnectionRef?: string;
eventSequence: number;
transcriptSequence: number;
pendingTurns: DeviceFinalTurn[];
transcriptUpload?: Promise<void>;
heartbeatTimer?: number;
durationTimer?: number;
startedAt?: number;
stopping: boolean;
intentionalProviderClose: boolean;
microphoneStream?: MediaStream;
captureContext?: AudioContext;
captureNode?: AudioWorkletNode;
playbackContext?: AudioContext;
playbackNode?: AudioWorkletNode;
playbackReady?: Promise<AudioWorkletNode>;
playbackPendingByte?: number;
playbackChunks: number;
playbackSamples: number;
playbackUnderflows: number;
responseTranscripts: Map<string, string>;
speechStartedAt?: number;
checks: Map<string, ReportCheck>;
events: ReportEvent[];
turns: DeviceFinalTurn[];
failure?: string;
failureTask?: Promise<void>;
} = {
phase: "idle",
eventSequence: 0,
transcriptSequence: 0,
pendingTurns: [],
stopping: false,
intentionalProviderClose: false,
playbackChunks: 0,
playbackSamples: 0,
playbackUnderflows: 0,
responseTranscripts: new Map(),
checks: new Map(checkDefinitions.map(([id, label]) => [id, { id, label, status: "pending" }])),
events: [],
turns: [],
};
const app = document.querySelector<HTMLElement>("#app")!;
app.innerHTML = `
<section class="shell hardware-shell">
<header class="hero">
<div>
<p class="eyebrow">WINDOWS · VOLC HARDWARE AGENT · LOW LOAD</p>
<h1>硬件对话智能体测试程序</h1>
<p class="subtitle">真实连接平台设备 v3 控制面和火山硬件对话智能体 WebSocket,验证动态人设、PCM 双向语音、字幕、记忆工具、平台停止与结束闭环。</p>
</div>
<div class="hero-actions"><a class="legacy-link" href="./legacy.html">传统 RTC 测试页</a><div class="overall" id="overallStatus"><strong>尚未检测</strong></div></div>
</header>
<section class="status-grid" aria-label="链路状态">
<article class="status-card"><span>平台 v3</span><strong id="platformStatus">未检测</strong><small id="platformDetail">等待设备鉴权</small></article>
<article class="status-card"><span>火山设备凭据</span><strong id="credentialStatus">未配置</strong><small id="credentialDetail">支持动态注册</small></article>
<article class="status-card"><span>动态会话配置</span><strong id="configStatus">未应用</strong><small id="configDetail">麦克风保持关闭</small></article>
<article class="status-card"><span>通话时长</span><strong id="duration">00:00</strong><small id="providerStatus">网关未连接</small></article>
</section>
<section class="content-grid hardware-content-grid">
<article class="panel setup-panel">
<div class="panel-heading"><div><p class="eyebrow">SECURE SETUP</p><h2>平台与火山设备</h2></div><span class="safe">密钥不写入报告</span></div>
<h3>平台设备身份</h3>
<label>平台地址<input id="apiURL" value="https://121.41.46.238" autocomplete="off" /></label>
<label>平台设备 ID<input id="deviceID" autocomplete="off" /></label>
<label>平台设备签名密钥<input id="deviceSecret" type="password" autocomplete="off" /></label>
<h3>火山硬件身份</h3>
<div class="row"><label>实例 ID(平台下发)<input id="instanceID" readonly /></label><label>Product Key(平台下发)<input id="productKey" readonly /></label></div>
<div class="row"><label>设备名称(平台分配)<input id="volcDeviceName" readonly /></label><label>Hardware ID<input id="hardwareID" autocomplete="off" /></label></div>
<label>智能体模型类型<select id="modelMode"><option value="llm">通用 LLM(LLMConfig.SystemMessages)</option><option value="s2s">豆包端到端语音(S2S dialog)</option></select></label>
<label>麦克风<select id="microphone"><option value="">默认麦克风</option></select></label>
<button class="secondary" id="checkButton">检测、鉴权并准备设备</button>
<p class="hint">Product Secret 只保存在平台密钥服务中。平台代设备完成动态注册,设备仅接收自己的一机一密并使用 Windows Safe Storage 加密保存;默认 Bot 由平台产品配置自动分配,设备不能覆盖。</p>
</article>
<article class="panel call-panel hardware-call-panel">
<div class="panel-heading"><div><p class="eyebrow">REAL END-TO-END FLOW</p><h2>硬件智能体通话</h2></div><span class="phase-badge" id="phaseBadge">待机</span></div>
<div class="conversation-orb" id="conversationOrb"><div class="orb-core"><span id="orbLabel">准备检测</span></div></div>
<div class="meters">
<div><div class="meter-label"><span>16k PCM 上行</span><strong id="micValue">0</strong></div><div class="meter"><i id="micMeter"></i></div></div>
<div><div class="meter-label"><span>16k PCM 下行</span><strong id="agentValue">0</strong></div><div class="meter agent"><i id="agentMeter"></i></div></div>
</div>
<div class="call-actions"><button class="primary" id="startButton" disabled>开始硬件智能体通话</button><button class="danger" id="stopButton" disabled>结束通话</button></div>
<div class="text-question"><input id="textQuestion" placeholder="可选:输入文字验证智能体回复" /><button class="secondary" id="sendTextButton" disabled>发送文本问题</button></div>
<dl class="session-info hardware-session-info">
<div><dt>平台通话编号</dt><dd id="conversationID">—</dd></div>
<div><dt>产品默认 Bot</dt><dd id="botID">—</dd></div>
<div><dt>传输方案</dt><dd id="transport">low_load</dd></div>
<div><dt>火山会话编号</dt><dd id="providerConnectionRef">—</dd></div>
<div><dt>配置摘要</dt><dd id="configDigest">—</dd></div>
<div><dt>冻结提示版本</dt><dd id="promptVersion">—</dd></div>
</dl>
</article>
</section>
<section class="panel flow-panel">
<div class="panel-heading"><div><p class="eyebrow">ACCEPTANCE GATES</p><h2>完整流程验收</h2></div><button class="text-button" id="exportReport">导出测试报告</button></div>
<ol id="flowChecks" class="flow-checks"></ol>
</section>
<section class="panel transcript-panel">
<div class="panel-heading"><div><p class="eyebrow">FINAL TRANSCRIPTS</p><h2>双方最终字幕与平台上传</h2></div><span class="phase-badge" id="transcriptStatus">0 条</span></div>
<ol id="liveTranscript"><li class="transcript-empty">通话开始后显示火山返回的用户和智能体最终字幕。</li></ol>
</section>
<section class="panel log-panel">
<div class="panel-heading"><div><p class="eyebrow">AUDITABLE EVENT STREAM</p><h2>平台与火山事件</h2></div><button class="text-button" id="clearLog">清空显示</button></div>
<ol id="eventLog"><li><time>系统</time><span>测试程序已启动,等待环境检测。</span></li></ol>
</section>
</section>
`;
const $ = <T extends HTMLElement>(id: string) => document.getElementById(id) as T;
const controls = {
apiURL: $<HTMLInputElement>("apiURL"), deviceID: $<HTMLInputElement>("deviceID"), deviceSecret: $<HTMLInputElement>("deviceSecret"),
instanceID: $<HTMLInputElement>("instanceID"), productKey: $<HTMLInputElement>("productKey"),
volcDeviceName: $<HTMLInputElement>("volcDeviceName"), hardwareID: $<HTMLInputElement>("hardwareID"),
modelMode: $<HTMLSelectElement>("modelMode"),
microphone: $<HTMLSelectElement>("microphone"), check: $<HTMLButtonElement>("checkButton"), start: $<HTMLButtonElement>("startButton"),
stop: $<HTMLButtonElement>("stopButton"), textQuestion: $<HTMLInputElement>("textQuestion"), sendText: $<HTMLButtonElement>("sendTextButton"),
};
function setText(id: string, value: string) { $(id).textContent = value; }
function errorMessage(error: unknown) { return error instanceof Error ? error.message : String(error); }
function recordEvent(source: ReportEvent["source"], type: string, detail?: Record<string, unknown>) {
state.events.push({ at: new Date().toISOString(), source, type, detail });
if (state.events.length > 1000) state.events.splice(0, state.events.length - 1000);
}
function log(message: string, level: "info" | "success" | "error" = "info", source: ReportEvent["source"] = "app", type = "log") {
const item = document.createElement("li");
item.className = level;
const time = document.createElement("time");
time.textContent = new Date().toLocaleTimeString("zh-CN", { hour12: false });
const content = document.createElement("span");
content.textContent = message;
item.append(time, content);
$("eventLog").prepend(item);
recordEvent(source, type, { level, message });
}
function setPhase(phase: HardwarePhase, detail: string) {
state.phase = phase;
const labels: Record<HardwarePhase, string> = { idle: "待机", checking: "检测中", ready: "已就绪", connecting: "连接中", active: "通话中", stopping: "结束中", completed: "已完成", error: "异常" };
setText("phaseBadge", labels[phase]);
setText("orbLabel", detail);
setText("overallStatus", detail);
$("overallStatus").className = `overall phase-${phase}`;
$("conversationOrb").className = `conversation-orb phase-${phase}`;
controls.check.disabled = ["connecting", "active", "stopping"].includes(phase);
controls.start.disabled = phase !== "ready" && phase !== "completed";
controls.stop.disabled = !["active", "error"].includes(phase) || !state.platformSession || !!state.failureTask;
controls.sendText.disabled = phase !== "active";
}
function setCheck(id: string, status: ReportCheck["status"], detail?: string) {
const check = state.checks.get(id);
if (!check) return;
check.status = status;
check.detail = detail;
check.at = new Date().toISOString();
renderChecks();
}
function resetChecks() {
state.checks = new Map(checkDefinitions.map(([id, label]) => [id, { id, label, status: "pending" }]));
renderChecks();
}
function renderChecks() {
const list = $("flowChecks");
list.replaceChildren(...[...state.checks.values()].map((check) => {
const item = document.createElement("li");
item.className = `flow-check flow-check--${check.status}`;
const mark = document.createElement("i");
mark.textContent = check.status === "passed" ? "✓" : check.status === "failed" ? "!" : "·";
const content = document.createElement("span");
content.textContent = check.label;
const detail = document.createElement("small");
detail.textContent = check.detail || (check.status === "pending" ? "等待验证" : check.status === "passed" ? "已通过" : "未通过");
item.append(mark, content, detail);
return item;
}));
}
function updateMeter(kind: "mic" | "agent", value: number) {
const normalized = Math.max(0, Math.min(100, Math.round(value)));
$(`${kind}Meter`).style.width = `${normalized}%`;
setText(`${kind}Value`, String(normalized));
}
async function loadDefaults() {
const [platform, hardware] = await Promise.all([
window.deviceSimulator.getRuntimeDefaults(),
window.deviceSimulator.getHardwareRuntimeDefaults(),
]);
controls.apiURL.value = platform.apiURL;
controls.deviceID.value = platform.deviceID;
if (platform.deviceSecretConfigured) controls.deviceSecret.placeholder = "已从 Windows 安全凭据文件加载";
controls.instanceID.value = hardware.instanceID;
controls.productKey.value = hardware.productKey;
controls.volcDeviceName.value = hardware.deviceName;
controls.hardwareID.value = hardware.hardwareID;
controls.modelMode.value = hardware.modelMode;
if (hardware.deviceCredentialConfigured) setText("credentialDetail", "本机已有平台签发的一机一密安全缓存");
}
async function enumerateMicrophones() {
const permission = await navigator.mediaDevices.getUserMedia({ audio: true });
permission.getTracks().forEach((track) => track.stop());
const devices = (await navigator.mediaDevices.enumerateDevices()).filter((device) => device.kind === "audioinput");
controls.microphone.replaceChildren(new Option("默认麦克风", ""));
devices.forEach((device, index) => controls.microphone.add(new Option(device.label || `麦克风 ${index + 1}`, device.deviceId)));
}
async function configureAndProbe() {
if (state.failureTask) await state.failureTask;
setPhase("checking", "正在鉴权和检测");
resetChecks();
state.events = [];
state.turns = [];
state.failure = undefined;
try {
await window.deviceSimulator.configure({ apiURL: controls.apiURL.value, deviceID: controls.deviceID.value, deviceSecret: controls.deviceSecret.value });
await window.deviceSimulator.probe();
setCheck("platform_authenticated", "passed", "设备 bootstrap/config HMAC 请求成功");
setText("platformStatus", "鉴权成功");
setText("platformDetail", controls.apiURL.value);
const hardware = await window.deviceSimulator.configureHardware({ hardwareID: controls.hardwareID.value });
controls.instanceID.value = hardware.instanceID;
controls.productKey.value = hardware.productKey;
controls.volcDeviceName.value = hardware.deviceName;
controls.hardwareID.value = hardware.hardwareID;
setCheck("credential_resolved", "passed", `${hardware.credentialSource}${hardware.persisted ? ",Device Secret 已写入 Windows 安全存储" : ",仅本次内存"}`);
setText("credentialStatus", hardware.registered ? "平台注册成功" : "平台凭据可用");
setText("credentialDetail", `${hardware.deviceName} · ${hardware.credentialSource} · 火山 License 额度已在动态注册时由服务端消费`);
const network = await window.deviceSimulator.probeHardwareNetwork();
const failures = network.endpoints.filter((endpoint) => !endpoint.ok);
if (failures.length) throw new Error(`火山网络不可达:${failures.map((item) => `${item.hostname} ${item.error || "失败"}`).join(";")}`);
await enumerateMicrophones();
setPhase("ready", "可以开始真实通话");
log("平台 v3、平台代理注册、火山网关和麦克风检测通过。", "success");
} catch (error) {
state.failure = errorMessage(error);
setPhase("error", "环境检测失败");
setText("platformStatus", "检测失败");
setText("platformDetail", state.failure);
log(`环境检测失败:${state.failure}`, "error");
}
}
function renderPlatformSession(created: CreatedHardwareAgentSession) {
setText("conversationID", created.conversation_id);
setText("botID", created.runtime.bot_id);
setText("transport", created.runtime.transport);
setText("configDigest", created.runtime.config_digest);
setText("promptVersion", created.session_config.prompt_bundle_version);
setText("providerConnectionRef", "等待火山连接");
}
async function startHardwareConversation() {
if (!["ready", "completed"].includes(state.phase)) return;
if (state.failureTask) await state.failureTask;
setPhase("connecting", "正在冻结本次上下文");
resetSessionState();
try {
const created = await window.deviceSimulator.createHardwareAgentSession();
state.platformSession = created;
renderPlatformSession(created);
if (created.runtime.transport !== "low_load") throw new Error(`设备绑定传输模式为 ${created.runtime.transport},Windows 标准协议测试要求 low_load`);
if (!created.session_config.authorized_context || !created.runtime.config_digest || !created.runtime.session_nonce) throw new Error("平台未返回完整冻结上下文或会话凭证");
setCheck("context_frozen", "passed", `${created.session_config.prompt_bundle_version} · ${created.runtime.config_digest.slice(0, 12)}…`);
recordEvent("platform", "hardware.session.ready", { conversation_id: created.conversation_id, bot_id: created.runtime.bot_id, transport: created.runtime.transport });
setText("configStatus", "等待 session.updated");
setText("configDetail", "麦克风仍保持关闭");
const connected = await window.deviceSimulator.connectHardwareAgent({ platformSession: created, modelMode: controls.modelMode.value as "llm" | "s2s" });
if (state.failureTask) { await state.failureTask; return; }
state.providerConnectionRef = connected.providerConnectionRef;
setText("providerConnectionRef", connected.providerConnectionRef);
setText("providerStatus", "WebSocket 已连接");
setCheck("provider_connected", "passed", connected.providerConnectionRef);
setCheck("dynamic_config_applied", "passed", "火山已返回 session.updated");
setText("configStatus", "已应用");
setText("configDetail", "允许激活麦克风");
state.eventSequence = 1;
const activated = await window.deviceSimulator.hardwareAgentEvent({
conversationID: created.conversation_id, sequenceNo: state.eventSequence, eventType: "session_started",
providerSessionRef: created.runtime.provider_session_ref, sessionNonce: created.runtime.session_nonce,
providerConnectionRef: connected.providerConnectionRef, botID: created.runtime.bot_id, transport: created.runtime.transport,
configDigest: created.runtime.config_digest, dynamicConfigApplied: true,
});
if (activated.status !== "ACTIVE" || activated.action !== "continue") throw new Error(`平台未激活硬件会话:${activated.status}/${activated.action}`);
if (state.failureTask) { await state.failureTask; return; }
setCheck("platform_active", "passed", "session_started 已进入 ACTIVE");
await ensurePlaybackNode();
await startMicrophone();
if (state.failureTask) { await state.failureTask; await stopAudio(); return; }
setCheck("microphone_streaming", "passed", "PCM16/16kHz/单声道/20ms 分片");
state.startedAt = Date.now();
setPhase("active", "请开始说话");
startTimers(created.session.heartbeat_interval_sec || 15);
log("硬件对话智能体已激活:动态配置确认后才开放麦克风。", "success", "platform", "hardware.session.active");
} catch (error) {
state.failure ||= errorMessage(error);
log(`启动硬件智能体失败:${state.failure}`, "error");
await failHardwareConversation("WINDOWS_HARDWARE_START_FAILED");
}
}
function resetSessionState() {
stopTimers();
state.platformSession = undefined;
state.providerConnectionRef = undefined;
state.eventSequence = 0;
state.transcriptSequence = 0;
state.pendingTurns = [];
state.transcriptUpload = undefined;
state.startedAt = undefined;
state.stopping = false;
state.intentionalProviderClose = false;
state.failureTask = undefined;
state.failure = undefined;
state.responseTranscripts.clear();
state.speechStartedAt = undefined;
state.turns = [];
state.playbackPendingByte = undefined;
state.playbackChunks = 0;
state.playbackSamples = 0;
state.playbackUnderflows = 0;
// Device preparation remains valid, but every call must earn its own checks.
for (const [id, label] of checkDefinitions) {
if (!["platform_authenticated", "credential_resolved"].includes(id)) {
state.checks.set(id, { id, label, status: "pending" });
}
}
renderChecks();
setText("duration", "00:00");
setText("providerStatus", "网关未连接");
setText("providerConnectionRef", "—");
setText("conversationID", "—");
setText("configStatus", "未应用");
setText("configDetail", "麦克风保持关闭");
$("liveTranscript").innerHTML = '<li class="transcript-empty">通话开始后显示火山返回的用户和智能体最终字幕。</li>';
setText("transcriptStatus", "0 条");
updateMeter("mic", 0);
updateMeter("agent", 0);
}
async function startMicrophone() {
const constraints: MediaTrackConstraints = { channelCount: 1, echoCancellation: true, noiseSuppression: true, autoGainControl: true };
if (controls.microphone.value) constraints.deviceId = { exact: controls.microphone.value };
const stream = await navigator.mediaDevices.getUserMedia({ audio: constraints });
const context = new AudioContext({ latencyHint: "interactive" });
await context.audioWorklet.addModule(new URL("./pcm-capture.worklet.js", window.location.href).toString());
const source = context.createMediaStreamSource(stream);
const node = new AudioWorkletNode(context, "pcm16-capture", { processorOptions: { targetSampleRate: 16000, frameSamples: 320 } });
const silent = context.createGain();
silent.gain.value = 0;
node.port.onmessage = (message) => {
if (state.phase !== "active" && state.phase !== "connecting") return;
if (message.data?.type !== "pcm" || !(message.data.bytes instanceof ArrayBuffer)) return;
updateMeter("mic", Number(message.data.volume || 0));
window.deviceSimulator.sendHardwareAgentAudio(message.data.bytes);
};
source.connect(node).connect(silent).connect(context.destination);
await context.resume();
state.microphoneStream = stream;
state.captureContext = context;
state.captureNode = node;
}
async function stopAudio() {
state.microphoneStream?.getTracks().forEach((track) => track.stop());
state.captureNode?.disconnect();
state.playbackNode?.port.postMessage({ type: "reset" });
state.playbackNode?.disconnect();
if (state.captureContext) try { await state.captureContext.close(); } catch { /* no-op */ }
if (state.playbackContext) try { await state.playbackContext.close(); } catch { /* no-op */ }
state.microphoneStream = undefined;
state.captureNode = undefined;
state.captureContext = undefined;
state.playbackNode = undefined;
state.playbackReady = undefined;
state.playbackContext = undefined;
state.playbackPendingByte = undefined;
updateMeter("mic", 0);
updateMeter("agent", 0);
}
function ensurePlaybackNode(): Promise<AudioWorkletNode> {
if (state.playbackNode) return Promise.resolve(state.playbackNode);
if (state.playbackReady) return state.playbackReady;
state.playbackReady = (async () => {
const context = new AudioContext({ latencyHint: "interactive" });
state.playbackContext = context;
await context.audioWorklet.addModule(new URL("./pcm-playback.worklet.js", window.location.href).toString());
const node = new AudioWorkletNode(context, "pcm16-playback", {
processorOptions: { sourceSampleRate: 16000, prebufferMs: 120 },
outputChannelCount: [1],
});
node.port.onmessage = (message) => {
if (message.data?.type === "underflow") {
state.playbackUnderflows = Number(message.data.count || state.playbackUnderflows + 1);
recordEvent("app", "audio.playback.underflow", { count: state.playbackUnderflows });
}
if (message.data?.type === "drained") {
updateMeter("agent", 0);
setCheck("audio_playback_buffered", "passed", `${state.playbackChunks} 个分片 · ${state.playbackSamples} 个采样 · 欠载 ${state.playbackUnderflows} 次`);
}
};
node.connect(context.destination);
await context.resume();
state.playbackNode = node;
return node;
})().catch(async (error) => {
const context = state.playbackContext;
state.playbackReady = undefined;
state.playbackContext = undefined;
if (context) try { await context.close(); } catch { /* no-op */ }
throw error;
});
return state.playbackReady;
}
async function playProviderAudio(base64: string) {
if (!base64) return;
const binary = atob(base64);
const bytes = new Uint8Array(binary.length);
for (let index = 0; index < binary.length; index += 1) bytes[index] = binary.charCodeAt(index);
const totalBytes = bytes.byteLength + (state.playbackPendingByte === undefined ? 0 : 1);
const sampleCount = Math.floor(totalBytes / 2);
if (sampleCount === 0) {
if (bytes.byteLength === 1) state.playbackPendingByte = bytes[0];
return;
}
const samples = new Float32Array(sampleCount);
let energy = 0;
let byteIndex = 0;
let sampleIndex = 0;
if (state.playbackPendingByte !== undefined) {
const value = new DataView(new Uint8Array([state.playbackPendingByte, bytes[0]]).buffer).getInt16(0, true) / 32768;
samples[sampleIndex] = value;
energy += value * value;
sampleIndex += 1;
byteIndex = 1;
state.playbackPendingByte = undefined;
}
const view = new DataView(bytes.buffer, bytes.byteOffset, bytes.byteLength);
while (sampleIndex < sampleCount) {
const value = view.getInt16(byteIndex, true) / 32768;
samples[sampleIndex] = value;
energy += value * value;
sampleIndex += 1;
byteIndex += 2;
}
if (byteIndex < bytes.byteLength) state.playbackPendingByte = bytes[byteIndex];
updateMeter("agent", Math.sqrt(energy / Math.max(1, sampleCount)) * 280);
state.playbackChunks += 1;
state.playbackSamples += sampleCount;
const node = await ensurePlaybackNode();
node.port.postMessage({ type: "push", samples: samples.buffer }, [samples.buffer]);
setCheck("audio_playback_buffered", "passed", `已接收并缓冲 ${state.playbackChunks} 个火山音频分片`);
}
function sanitizeProviderEvent(event: Record<string, unknown>) {
const redact = (value: unknown, key = ""): unknown => {
if (/systemmessages|system_role|system_prompt|instructions|authorized_context|speaking_style/i.test(key)) return "[REDACTED]";
if (Array.isArray(value)) return value.map((item) => redact(item));
if (!value || typeof value !== "object") return value;
return Object.fromEntries(Object.entries(value as Record<string, unknown>).map(([itemKey, item]) => [itemKey, redact(item, itemKey)]));
};
const copy = redact(event) as Record<string, unknown>;
if (typeof copy.delta === "string" && event.type === "response.audio.delta") copy.delta = `<pcm-base64:${copy.delta.length} chars>`;
return copy;
}
async function handleProviderEvent(event: Record<string, unknown>) {
const type = String(event.type || "unknown");
recordEvent("volcengine", type, sanitizeProviderEvent(event));
if (type !== "response.audio.delta") log(`火山事件:${type}`, type === "error" ? "error" : "info", "volcengine", type);
// Teardown can still deliver queued provider events. Do not revive a failed call.
if (state.failureTask && type !== "transport.close") return;
switch (type) {
case "session.created": setText("providerStatus", "会话已创建"); break;
case "session.updated": setText("configStatus", "已确认"); break;
case "input_audio_buffer.speech_started": state.speechStartedAt = Date.now(); setPhase("active", "检测到用户说话"); break;
case "input_audio_buffer.speech_stopped": setPhase("active", "智能体正在思考"); break;
case "response.created": setPhase("active", "智能体正在生成"); break;
case "response.audio.delta": await playProviderAudio(String(event.delta || "")); break;
case "response.audio.done": state.playbackNode?.port.postMessage({ type: "end" }); break;
case "conversation.item.input_audio_transcription.completed":
await appendFinalTurn("subject", String(event.transcript || ""), String(event.event_id || crypto.randomUUID()), state.speechStartedAt); state.speechStartedAt = undefined; break;
case "response.audio_transcript.delta": {
const responseID = String(event.response_id || "unknown");
state.responseTranscripts.set(responseID, `${state.responseTranscripts.get(responseID) || ""}${String(event.delta || "")}`);
break;
}
case "response.audio_transcript.done": {
const responseID = String(event.response_id || "unknown");
const transcript = String(event.transcript || state.responseTranscripts.get(responseID) || "");
state.responseTranscripts.delete(responseID);
await appendFinalTurn("agent", transcript, responseID); break;
}
case "response.done": setPhase("active", "继续聆听"); break;
case "platform.memory_search.completed": log(`平台记忆工具返回 ${Number(event.result_count || 0)} 条已授权记忆。`, "success", "platform", type); break;
case "platform.memory_search.failed": log(`平台记忆工具降级:${String(event.message || "未知错误")}`, "error", "platform", type); break;
case "error": {
const providerError = event.error as Record<string, unknown> | undefined;
state.failure ||= `${String(providerError?.code || providerError?.type || "PROVIDER_ERROR")}: ${String(providerError?.message || "火山智能体错误")}`;
log(`火山错误:${state.failure}`, "error", "volcengine", "provider.error");
await failHardwareConversation("VOLC_HARDWARE_AGENT_ERROR");
break;
}
case "transport.close":
setText("providerStatus", "连接已关闭");
if (state.failureTask) setCheck("provider_closed", "failed", `失败清理关闭:WebSocket ${Number(event.code || 1000)}`);
else if (state.intentionalProviderClose) setCheck("provider_closed", "passed", `WebSocket ${Number(event.code || 1000)}`);
else if (["connecting", "active"].includes(state.phase)) { state.failure = `火山连接意外关闭:${String(event.code || "")}`; await failHardwareConversation("VOLC_CONNECTION_CLOSED"); }
break;
}
}
async function appendFinalTurn(speaker: "subject" | "agent", rawText: string, sourceID: string, startedAt?: number) {
const text = rawText.trim();
if (!text || !state.platformSession) return;
state.transcriptSequence += 1;
const now = Date.now();
const turn: DeviceFinalTurn = {
turn_key: `hardware-${speaker}-${sourceID}`.slice(0, 240), sequence_no: state.transcriptSequence, speaker,
start_at: new Date(startedAt || now).toISOString(), end_at: new Date(now).toISOString(), final_text: text, interrupted: false,
};
state.pendingTurns.push(turn);
state.turns.push(turn);
renderTurn(turn);
setCheck(speaker === "subject" ? "user_transcript" : "agent_transcript", "passed", text.slice(0, 50));
await flushTurns();
}
function renderTurn(turn: DeviceFinalTurn) {
const list = $("liveTranscript");
list.querySelector(".transcript-empty")?.remove();
const item = document.createElement("li");
item.className = `transcript-turn transcript-turn--${turn.speaker}`;
const speaker = document.createElement("strong"); speaker.textContent = turn.speaker === "subject" ? "用户" : "智能体";
const text = document.createElement("span"); text.textContent = turn.final_text;
item.append(speaker, text); list.append(item);
setText("transcriptStatus", `${state.turns.length} 条`);
}
async function flushTurns() {
if (state.transcriptUpload) return state.transcriptUpload;
if (!state.platformSession || state.pendingTurns.length === 0) return;
const batch = state.pendingTurns.slice(0, 500);
state.transcriptUpload = (async () => {
try {
const result = await window.deviceSimulator.submitTranscriptBatch({ conversationID: state.platformSession!.conversation_id, turns: batch, hardwareAgent: true });
state.pendingTurns.splice(0, batch.length);
setCheck("transcript_uploaded", "passed", `新增 ${result.accepted_count},重复 ${result.duplicate_count}`);
log(`最终字幕已上传平台:新增 ${result.accepted_count},重复 ${result.duplicate_count}。`, "success", "platform", "hardware.turns.uploaded");
} catch (error) { log(`字幕上传失败,保留队列待重试:${errorMessage(error)}`, "error"); }
finally { state.transcriptUpload = undefined; }
})();
await state.transcriptUpload;
}
async function flushTurnsWithRetry(attempts: number) {
for (let attempt = 1; attempt <= attempts && state.pendingTurns.length > 0; attempt += 1) {
await flushTurns();
if (state.pendingTurns.length && attempt < attempts) await new Promise((resolve) => window.setTimeout(resolve, attempt * 500));
}
if (state.pendingTurns.length) throw new Error(`仍有 ${state.pendingTurns.length} 条最终字幕未上传`);
}
function startTimers(intervalSeconds: number) {
stopTimers();
state.durationTimer = window.setInterval(() => {
if (!state.startedAt) return;
const seconds = Math.floor((Date.now() - state.startedAt) / 1000);
setText("duration", `${String(Math.floor(seconds / 60)).padStart(2, "0")}:${String(seconds % 60).padStart(2, "0")}`);
}, 1000);
state.heartbeatTimer = window.setInterval(() => void sendHardwareHeartbeat(), Math.max(5, intervalSeconds) * 1000);
void sendHardwareHeartbeat();
}
function stopTimers() {
if (state.durationTimer) window.clearInterval(state.durationTimer);
if (state.heartbeatTimer) window.clearInterval(state.heartbeatTimer);
state.durationTimer = undefined;
state.heartbeatTimer = undefined;
}
async function sendHardwareHeartbeat() {
const created = state.platformSession;
if (!created || state.phase !== "active" || state.stopping) return;
try {
state.eventSequence += 1;
const result = await window.deviceSimulator.hardwareAgentEvent({
conversationID: created.conversation_id, sequenceNo: state.eventSequence, eventType: "session_updated",
providerSessionRef: created.runtime.provider_session_ref, sessionNonce: created.runtime.session_nonce,
});
setCheck("heartbeat_ack", "passed", `序号 ${state.eventSequence} · ${result.action}`);
if (result.action === "stop_session") await stopHardwareConversation("platform_stop_session");
await flushTurns();
} catch (error) { log(`硬件会话存活上报失败:${errorMessage(error)}`, "error"); }
}
async function stopHardwareConversation(reason = "windows_user_hangup") {
if (state.failureTask) return state.failureTask;
if (!state.platformSession || state.stopping) return;
state.stopping = true;
setPhase("stopping", "正在可靠结束");
stopTimers();
const created = state.platformSession;
try {
await stopAudio();
await flushTurnsWithRetry(3);
state.intentionalProviderClose = true;
await window.deviceSimulator.closeHardwareAgent({ code: 1000, reason });
setCheck("provider_closed", "passed", "设备主动关闭 WebSocket");
state.eventSequence += 1;
const ended = await window.deviceSimulator.hardwareAgentEvent({
conversationID: created.conversation_id, sequenceNo: state.eventSequence, eventType: "session_ended",
providerSessionRef: created.runtime.provider_session_ref, sessionNonce: created.runtime.session_nonce,
});
if (ended.status !== "COMPLETED") throw new Error(`平台结束状态不是 COMPLETED:${ended.status}`);
setCheck("platform_completed", "passed", "session_ended 已进入 COMPLETED");
state.stopping = false;
setPhase("completed", allRequiredChecksPassed() ? "完整流程通过" : "通话完成,部分验收项待覆盖");
log("硬件对话智能体会话已正常结束,平台状态为 COMPLETED。", "success", "platform", "hardware.session.completed");
} catch (error) {
state.failure = errorMessage(error);
state.stopping = false;
setPhase("error", "结束流程失败");
log(`结束硬件会话失败:${state.failure}`, "error");
}
}
function failHardwareConversation(errorCode: string): Promise<void> {
// The provider error event and connect() rejection describe the same failure.
// Install a shared task before any asynchronous cleanup, and retain it until
// the next call so later errors/hangup cannot submit another terminal event.
if (state.failureTask) return state.failureTask;
const created = state.platformSession;
state.failureTask = Promise.resolve().then(async () => {
try {
await stopAudio();
try { await window.deviceSimulator.closeHardwareAgent({ code: 1011, reason: errorCode }); } catch { /* no-op */ }
if (created) {
try {
state.eventSequence += 1;
await window.deviceSimulator.hardwareAgentEvent({
conversationID: created.conversation_id, sequenceNo: state.eventSequence, eventType: "session_failed",
providerSessionRef: created.runtime.provider_session_ref, sessionNonce: created.runtime.session_nonce,
errorCode,
});
} catch (error) { log(`平台失败事件回报未成功:${errorMessage(error)}`, "error"); }
}
} finally {
state.stopping = false;
setPhase("error", state.failure || "硬件会话失败");
}
});
stopTimers();
state.stopping = true;
state.intentionalProviderClose = true;
setPhase("stopping", "正在清理失败会话");
setCheck("platform_completed", "failed", state.failure || errorCode);
if (!state.providerConnectionRef) {
setText("configStatus", "未完成");
setCheck("dynamic_config_applied", "failed", state.failure || errorCode);
}
return state.failureTask;
}
function allRequiredChecksPassed() {
const required = ["platform_authenticated", "credential_resolved", "context_frozen", "provider_connected", "dynamic_config_applied", "platform_active", "microphone_streaming", "audio_playback_buffered", "heartbeat_ack", "transcript_uploaded", "provider_closed", "platform_completed"];
return required.every((id) => state.checks.get(id)?.status === "passed");
}
async function sendTextQuestion() {
if (state.phase !== "active" || state.failureTask) return;
const text = controls.textQuestion.value.trim();
if (!text) return;
try {
await window.deviceSimulator.sendHardwareAgentText({ text, interruptMode: 1 });
// 文本输入没有 ASR final 事件,设备本地将已被火山接受的最终输入作为用户话轮上报,
// 这样文本验收与真实语音验收都会进入同一套平台聊天记录。
await appendFinalTurn("subject", text, `text-${crypto.randomUUID()}`);
controls.textQuestion.value = "";
log(`已通过硬件标准协议发送文本问题:${text}`, "success", "app", "hardware.text.sent");
} catch (error) { log(`发送文本问题失败:${errorMessage(error)}`, "error"); }
}
function buildReport(): Record<string, unknown> {
return {
schema_version: "hardware-agent-windows-e2e-report-v1",
generated_at: new Date().toISOString(),
result: state.phase === "completed" && allRequiredChecksPassed() ? "PASS" : state.failure ? "FAIL" : "INCOMPLETE",
platform_api: controls.apiURL.value,
platform_device_id: controls.deviceID.value,
volc_identity: { instance_id: controls.instanceID.value, product_key: controls.productKey.value, device_name: controls.volcDeviceName.value, hardware_id: controls.hardwareID.value },
conversation_id: state.platformSession?.conversation_id,
bot_id: state.platformSession?.runtime.bot_id,
model_mode: controls.modelMode.value,
transport: state.platformSession?.runtime.transport,
audio_playback: {
source_format: "pcm_s16le_16000_mono",
mode: "audio_worklet_ring_buffer",
prebuffer_ms: 120,
chunks: state.playbackChunks,
samples: state.playbackSamples,
underflows: state.playbackUnderflows,
},
provider_connection_ref: state.providerConnectionRef,
config_digest: state.platformSession?.runtime.config_digest,
prompt_bundle_version: state.platformSession?.session_config.prompt_bundle_version,
failure: state.failure,
checks: [...state.checks.values()],
turns: state.turns,
events: state.events,
};
}
async function exportReport() {
try {
const result = await window.deviceSimulator.saveHardwareReport(buildReport());
if (result.saved) log(`测试报告已保存:${result.path}`, "success");
} catch (error) { log(`导出测试报告失败:${errorMessage(error)}`, "error"); }
}
const unsubscribeProvider = window.deviceSimulator.onHardwareProviderEvent((event) => void handleProviderEvent(event));
controls.check.addEventListener("click", () => void configureAndProbe());
controls.start.addEventListener("click", () => void startHardwareConversation());
controls.stop.addEventListener("click", () => void stopHardwareConversation());
controls.sendText.addEventListener("click", () => void sendTextQuestion());
controls.textQuestion.addEventListener("keydown", (event) => { if (event.key === "Enter") void sendTextQuestion(); });
$("exportReport").addEventListener("click", () => void exportReport());
$("clearLog").addEventListener("click", () => $("eventLog").replaceChildren());
window.addEventListener("beforeunload", () => {
unsubscribeProvider();
stopTimers();
void stopAudio();
void window.deviceSimulator.closeHardwareAgent({ code: 1001, reason: "window closed" });
});
renderChecks();
setPhase("idle", "尚未检测");
void loadDefaults().catch((error) => log(`读取默认配置失败:${errorMessage(error)}`, "error"));
+806
View File
@@ -0,0 +1,806 @@
import VERTC, {
AudioSourceType,
AudioFrameData,
IRTCEngine,
LocalAudioPropertiesInfo,
MediaType,
RemoteAudioPropertiesInfo,
RoomProfileType,
StreamIndex,
SUBTITLE_MODE,
SubtitleEvent,
SubtitleEventType,
SubtitleMessage,
} from "@volcengine/rtc";
import { PCMTrackRecorder, RecordedAudioSegment } from "./recording";
VERTC.setLogConfig({ logLevel: "trace", LogfileSize: 20 });
import "./style.css";
const EngineEventsTypes = VERTC.events;
type Phase = "idle" | "checking" | "ready" | "joining" | "waiting" | "active" | "stopping" | "error";
const state: {
phase: Phase;
engine?: IRTCEngine;
conversation?: CreatedConversation;
credentialExpiresAt?: string;
heartbeatSequence: number;
heartbeatTimer?: number;
statusTimer?: number;
startedAt?: number;
durationTimer?: number;
agentJoined: boolean;
agentSpeaking: boolean;
microphoneVolume: number;
agentVolume: number;
subtitleStarted: boolean;
transcriptSequence: number;
transcriptFirstSeen: Map<string, number>;
collectedTurnKeys: Set<string>;
pendingTurns: DeviceFinalTurn[];
transcriptUpload?: Promise<void>;
subtitleStartWaiter?: { resolve: () => void; reject: (error: Error) => void };
automationAudio?: HTMLAudioElement;
automationAudioContext?: AudioContext;
automationAudioTrack?: MediaStreamTrack;
automationAudioObjectURL?: string;
nearRecorder?: PCMTrackRecorder;
farRecorder?: PCMTrackRecorder;
} = {
phase: "idle",
heartbeatSequence: 0,
agentJoined: false,
agentSpeaking: false,
microphoneVolume: 0,
agentVolume: 0,
subtitleStarted: false,
transcriptSequence: 0,
transcriptFirstSeen: new Map(),
collectedTurnKeys: new Set(),
pendingTurns: [],
};
const app = document.querySelector<HTMLElement>("#app")!;
app.innerHTML = `
<section class="shell">
<header class="hero">
<div>
<p class="eyebrow">WINDOWS · REAL RTC DEVICE</p>
<h1>语音终端模拟器</h1>
<p class="subtitle">使用真实麦克风和扬声器,验证“终端入房—豆包进房—双向语音—打断—结束”的完整链路。</p>
</div>
<div class="overall" id="overallStatus"><span class="pulse"></span><strong>尚未连接</strong></div>
</header>
<section class="status-grid" aria-label="链路状态">
<article class="status-card"><span>云端平台</span><strong id="apiStatus">未检测</strong><small id="apiDetail">121.41.46.238</small></article>
<article class="status-card"><span>RTC 房间</span><strong id="rtcStatus">未加入</strong><small id="rtcDetail">等待开始</small></article>
<article class="status-card"><span>豆包智能体</span><strong id="agentStatus">未进房</strong><small id="agentDetail">等待 StartVoiceChat</small></article>
<article class="status-card"><span>通话时长</span><strong id="duration">00:00</strong><small id="modeDetail">处理方式:—</small></article>
</section>
<section class="content-grid">
<article class="panel setup-panel">
<div class="panel-heading"><div><p class="eyebrow">DEVICE SETUP</p><h2>测试设备配置</h2></div><span class="safe" id="credentialStatus">正在检查凭据</span></div>
<label>云端平台地址<input id="apiURL" value="https://121.41.46.238" autocomplete="off" /></label>
<label>模拟设备 ID<input id="deviceID" value="10000000-0000-0000-0000-000000000511" autocomplete="off" /></label>
<label>设备签名密钥<input id="deviceSecret" type="password" value="" autocomplete="off" /></label>
<div class="row">
<label>麦克风<select id="microphone"><option value="">默认麦克风</option></select></label>
<label>扬声器<select id="speaker"><option value="">默认扬声器</option></select></label>
</div>
<button class="secondary" id="checkButton">检测平台和音频设备</button>
<p class="hint">RTC AppKey、火山 AK/SK 和豆包 Token 配置在云端后台,本软件不会读取或保存它们。</p>
</article>
<article class="panel call-panel">
<div class="panel-heading"><div><p class="eyebrow">LIVE CONVERSATION</p><h2>真实双向语音</h2></div><span class="phase-badge" id="phaseBadge">待机</span></div>
<div class="conversation-orb" id="conversationOrb"><div class="orb-core"><span id="orbLabel">准备开始</span></div></div>
<div class="meters">
<div><div class="meter-label"><span>我的麦克风</span><strong id="micValue">0</strong></div><div class="meter"><i id="micMeter"></i></div></div>
<div><div class="meter-label"><span>豆包语音</span><strong id="agentValue">0</strong></div><div class="meter agent"><i id="agentMeter"></i></div></div>
</div>
<div class="call-actions">
<button class="primary" id="startButton" disabled>开始语音聊天</button>
<button class="danger" id="stopButton" disabled>结束通话</button>
</div>
<dl class="session-info">
<div><dt>通话编号</dt><dd id="conversationID">—</dd></div>
<div><dt>RTC 房间</dt><dd id="roomID">—</dd></div>
<div><dt>终端用户</dt><dd id="userID">—</dd></div>
<div><dt>智能体用户</dt><dd id="agentUserID">—</dd></div>
<div><dt>聊天语音</dt><dd id="recordingStatus">等待通话</dd></div>
</dl>
</article>
</section>
<section class="panel log-panel">
<div class="panel-heading"><div><p class="eyebrow">EVENT STREAM</p><h2>实时事件</h2></div><div><button class="text-button" id="downloadRtcLog">导出 RTC 日志</button><button class="text-button" id="clearLog">清空</button></div></div>
<ol id="eventLog" aria-live="polite"><li><time>系统</time><span>模拟器已启动,等待检测本地环境。</span></li></ol>
</section>
<section class="panel transcript-panel">
<div class="panel-heading"><div><p class="eyebrow">LIVE TRANSCRIPT</p><h2>实时字幕与上传</h2></div><span class="phase-badge" id="transcriptStatus">未启动</span></div>
<ol id="liveTranscript" aria-live="polite"><li class="transcript-empty">通话开始后,这里会显示双方的最终字幕。</li></ol>
</section>
</section>
`;
const $ = <T extends HTMLElement>(id: string) => document.getElementById(id) as T;
const controls = {
apiURL: $<HTMLInputElement>("apiURL"),
deviceID: $<HTMLInputElement>("deviceID"),
deviceSecret: $<HTMLInputElement>("deviceSecret"),
microphone: $<HTMLSelectElement>("microphone"),
speaker: $<HTMLSelectElement>("speaker"),
check: $<HTMLButtonElement>("checkButton"),
start: $<HTMLButtonElement>("startButton"),
stop: $<HTMLButtonElement>("stopButton"),
};
void window.deviceSimulator.getRuntimeDefaults().then((defaults) => {
controls.apiURL.value = defaults.apiURL;
controls.deviceID.value = defaults.deviceID;
controls.deviceSecret.value = defaults.deviceSecret;
if (defaults.deviceSecretConfigured) {
controls.deviceSecret.placeholder = "已从Windows安全凭据文件加载";
setText("credentialStatus", "凭据已安全加载");
} else {
controls.deviceSecret.placeholder = "请输入至少32字符的设备密钥";
setText("credentialStatus", "凭据未加载");
}
}).catch((error) => log(`读取测试设备启动配置失败:${errorMessage(error)}`, "error"));
function setText(id: string, value: string) {
$(id).textContent = value;
}
function log(message: string, level: "info" | "success" | "error" = "info") {
console.log(`[${level}] ${message}`);
const item = document.createElement("li");
item.className = level;
const time = document.createElement("time");
time.textContent = new Date().toLocaleTimeString("zh-CN", { hour12: false });
const content = document.createElement("span");
content.textContent = message;
item.append(time, content);
$("eventLog").prepend(item);
}
function errorMessage(error: unknown): string {
return error instanceof Error ? error.message : String(error);
}
function setPhase(phase: Phase, detail?: string) {
state.phase = phase;
const labels: Record<Phase, string> = {
idle: "待机", checking: "检测中", ready: "已就绪", joining: "正在入房", waiting: "等待豆包", active: "通话中", stopping: "结束中", error: "异常",
};
setText("phaseBadge", labels[phase]);
setText("overallStatus", detail || labels[phase]);
$("overallStatus").className = `overall phase-${phase}`;
$("conversationOrb").className = `conversation-orb phase-${phase}${state.agentSpeaking ? " speaking" : ""}`;
setText("orbLabel", detail || labels[phase]);
controls.start.disabled = phase !== "ready";
controls.stop.disabled = !["joining", "waiting", "active", "error"].includes(phase) || !state.conversation;
controls.check.disabled = ["joining", "waiting", "active", "stopping"].includes(phase);
}
async function configureAndProbe() {
setPhase("checking", "正在检测");
try {
await window.deviceSimulator.configure({
apiURL: controls.apiURL.value,
deviceID: controls.deviceID.value,
deviceSecret: controls.deviceSecret.value,
});
const supported = await VERTC.isSupported();
if (!supported) throw new Error("当前 Electron 环境不支持火山 Web RTC SDK");
const permission = await VERTC.enableDevices({ audio: true, video: false });
if (!permission.audio) throw permission.audioExceptionError || new Error("未获得麦克风权限");
await refreshAudioDevices();
await window.deviceSimulator.probe();
const rtcNetwork = await window.deviceSimulator.probeRTCNetwork();
const blockedEndpoints = rtcNetwork.endpoints.filter((endpoint) => !endpoint.ok);
if (blockedEndpoints.length > 0) {
const details = blockedEndpoints.map((endpoint) => `${endpoint.hostname}(${endpoint.error || "连接失败"})`).join("、");
throw new Error(`RTC 网络检测失败:${details}。请关闭代理/VPN、放行 TCP 443,或改用手机热点后重试`);
}
setText("apiStatus", "连接正常");
setText("apiDetail", controls.apiURL.value);
setPhase("ready", "可以开始通话");
log(`环境检测通过,RTC Web SDK ${VERTC.getSdkVersion()},火山 RTC 接入域名可达。`, "success");
} catch (error) {
setText("apiStatus", "检测失败");
setText("apiDetail", errorMessage(error));
setPhase("error", "检测失败");
controls.start.disabled = true;
log(`环境检测失败:${errorMessage(error)}`, "error");
}
}
async function refreshAudioDevices() {
const [microphones, speakers] = await Promise.all([
VERTC.enumerateAudioCaptureDevices(),
VERTC.enumerateAudioPlaybackDevices(),
]);
fillDeviceOptions(controls.microphone, microphones, "默认麦克风");
fillDeviceOptions(controls.speaker, speakers, "默认扬声器");
}
function fillDeviceOptions(select: HTMLSelectElement, devices: MediaDeviceInfo[], fallback: string) {
const previous = select.value;
select.replaceChildren(new Option(fallback, ""));
devices.forEach((device, index) => select.add(new Option(device.label || `${fallback} ${index + 1}`, device.deviceId)));
if ([...select.options].some((option) => option.value === previous)) select.value = previous;
}
function registerRTCEvents(engine: IRTCEngine) {
engine.on(EngineEventsTypes.onSubtitleStateChanged, (event: SubtitleEvent) => {
if (event.event === SubtitleEventType.STARTED) {
state.subtitleStarted = true;
state.subtitleStartWaiter?.resolve();
setText("transcriptStatus", "识别中");
log("RTC 实时字幕已启动,双方最终话轮将自动上传。", "success");
return;
}
if (event.event === SubtitleEventType.STOPPED) {
state.subtitleStarted = false;
state.subtitleStartWaiter?.reject(new Error("RTC 实时字幕在启动期间被停止"));
setText("transcriptStatus", "已停止");
return;
}
if (event.event === SubtitleEventType.ERROR) {
state.subtitleStarted = false;
state.subtitleStartWaiter?.reject(new Error(`RTC 实时字幕启动失败:${event.errorCode || "UNKNOWN"}${event.errorMessage ? ` · ${event.errorMessage}` : ""}`));
setText("transcriptStatus", "识别失败");
log(`RTC 实时字幕错误:${event.errorCode || "UNKNOWN"}${event.errorMessage ? ` · ${event.errorMessage}` : ""}`, "error");
}
});
engine.on(EngineEventsTypes.onSubtitleMessageReceived, (messages: SubtitleMessage[]) => {
collectSubtitleMessages(messages);
});
engine.on(EngineEventsTypes.onUserJoined, (event) => {
const userID = event.userInfo.userId;
log(`远端用户已进房:${userID}`, "success");
if (userID === state.conversation?.rtc.agent_user_id) {
state.agentJoined = true;
setText("agentStatus", "已进房");
setText("agentDetail", userID);
}
});
engine.on(EngineEventsTypes.onUserLeave, (event) => {
const userID = event.userInfo.userId;
log(`远端用户已离房:${userID}`);
if (userID === state.conversation?.rtc.agent_user_id) {
state.agentJoined = false;
setText("agentStatus", "已离房");
}
});
engine.on(EngineEventsTypes.onUserPublishStream, async (event) => {
if ((event.mediaType & MediaType.AUDIO) === 0) return;
log(`检测到远端音频流:${event.userId}`, "success");
try {
await engine.subscribeStream(event.userId, MediaType.AUDIO);
await engine.play(event.userId, MediaType.AUDIO);
if (event.userId === state.conversation?.rtc.agent_user_id) {
setText("agentStatus", "语音已连接");
setText("agentDetail", "正在接收豆包音频");
}
} catch (error) {
log(`播放远端音频失败:${errorMessage(error)}`, "error");
}
});
engine.on(EngineEventsTypes.onLocalAudioPropertiesReport, (reports: LocalAudioPropertiesInfo[]) => {
const volume = Math.max(0, ...reports.map((item) => item.audioPropertiesInfo.linearVolume || 0));
updateVolume("mic", volume);
});
engine.on(EngineEventsTypes.onRemoteAudioPropertiesReport, (reports: RemoteAudioPropertiesInfo[]) => {
const agentID = state.conversation?.rtc.agent_user_id;
const volume = Math.max(0, ...reports.filter((item) => !agentID || item.streamKey.userId === agentID).map((item) => item.audioPropertiesInfo.linearVolume || 0));
state.agentSpeaking = volume > 5;
updateVolume("agent", volume);
setPhase(state.phase, state.phase === "active" ? (state.agentSpeaking ? "豆包正在说话" : "请开始说话") : undefined);
});
engine.on(EngineEventsTypes.onConnectionStateChanged, (event) => {
const names: Record<number, string> = {
0: "准备连接",
1: "已断开",
2: "连接中",
3: "已连接",
4: "重连中",
5: "已重连",
6: "连接丢失",
};
const level = event.state === 1 || event.state === 6 ? "error" : "info";
log(`RTC 连接状态:${String(event.state)}(${names[event.state] || "未知"})${event.reason !== undefined ? `,原因 ${String(event.reason)}` : ""}`, level);
});
engine.on(EngineEventsTypes.onError, (event) => {
log(`RTC 错误:${event.errorCode}`, "error");
});
engine.on(EngineEventsTypes.onTokenWillExpire, () => {
log("RTC Token 即将过期,等待心跳续签。", "info");
});
}
function subtitleIdentity(message: SubtitleMessage) {
const conversation = state.conversation;
if (!conversation) return undefined;
if (message.userId === conversation.rtc.user_id) return { speaker: "subject" as const, label: "长者" };
if (message.userId === conversation.rtc.agent_user_id) return { speaker: "agent" as const, label: "豆包" };
return undefined;
}
function subtitleKey(message: SubtitleMessage) {
return `${message.userId}:${message.sequence}`;
}
function collectSubtitleMessages(messages: SubtitleMessage[]) {
const now = Date.now();
for (const message of messages) {
const identity = subtitleIdentity(message);
const text = message.text.trim();
if (!identity || !text || message.mode !== SUBTITLE_MODE.ASR_ONLY) continue;
const sourceKey = subtitleKey(message);
if (!state.transcriptFirstSeen.has(sourceKey)) state.transcriptFirstSeen.set(sourceKey, now);
if (!message.definite || state.collectedTurnKeys.has(sourceKey)) continue;
state.collectedTurnKeys.add(sourceKey);
state.transcriptSequence += 1;
const turn: DeviceFinalTurn = {
turn_key: `rtc-subtitle-${identity.speaker}-${message.sequence}`,
sequence_no: state.transcriptSequence,
speaker: identity.speaker,
start_at: new Date(state.transcriptFirstSeen.get(sourceKey) || now).toISOString(),
end_at: new Date(now).toISOString(),
final_text: text,
interrupted: false,
};
state.pendingTurns.push(turn);
renderTranscriptTurn(turn, identity.label);
}
void flushTranscriptTurns();
}
function renderTranscriptTurn(turn: DeviceFinalTurn, label: string) {
const list = $("liveTranscript");
list.querySelector(".transcript-empty")?.remove();
const item = document.createElement("li");
item.className = `transcript-turn transcript-turn--${turn.speaker}`;
const speaker = document.createElement("strong");
speaker.textContent = label;
const text = document.createElement("span");
text.textContent = turn.final_text;
item.append(speaker, text);
list.append(item);
}
async function flushTranscriptTurns() {
if (state.transcriptUpload) return state.transcriptUpload;
if (!state.conversation || state.pendingTurns.length === 0) return;
const conversationID = state.conversation.conversation_id;
const batch = state.pendingTurns.slice(0, 500);
state.transcriptUpload = (async () => {
try {
const result = await window.deviceSimulator.submitTranscriptBatch({ conversationID, turns: batch });
state.pendingTurns.splice(0, batch.length);
const uploaded = state.transcriptSequence - state.pendingTurns.length;
setText("transcriptStatus", state.subtitleStarted ? `识别中 · 已上传 ${uploaded}` : `已上传 ${uploaded}`);
log(`最终字幕已上传:新增 ${result.accepted_count} 条,重复 ${result.duplicate_count} 条。`, "success");
} catch (error) {
setText("transcriptStatus", `待重试 ${state.pendingTurns.length} 条`);
log(`最终字幕上传失败,将在心跳或结束通话时重试:${errorMessage(error)}`, "error");
} finally {
state.transcriptUpload = undefined;
}
})();
await state.transcriptUpload;
if (state.pendingTurns.length > 0 && state.pendingTurns.length !== batch.length) void flushTranscriptTurns();
}
async function flushTranscriptTurnsWithRetry(attempts: number) {
for (let attempt = 1; attempt <= attempts && state.pendingTurns.length > 0; attempt += 1) {
await flushTranscriptTurns();
if (state.pendingTurns.length === 0) return;
if (attempt < attempts) await new Promise((resolve) => window.setTimeout(resolve, attempt * 500));
}
}
async function startSubtitleAndWait(engine: IRTCEngine) {
let timeoutID: number | undefined;
const stateEvent = new Promise<void>((resolve, reject) => {
state.subtitleStartWaiter = { resolve, reject };
});
const timeout = new Promise<void>((_resolve, reject) => {
timeoutID = window.setTimeout(() => reject(new Error("等待 RTC 实时字幕启动超时")), 10_000);
});
try {
await engine.startSubtitle({ mode: SUBTITLE_MODE.ASR_ONLY });
await Promise.race([stateEvent, timeout]);
} finally {
if (timeoutID) window.clearTimeout(timeoutID);
state.subtitleStartWaiter = undefined;
}
}
function updateVolume(kind: "mic" | "agent", raw: number) {
const value = Math.max(0, Math.min(100, Math.round(raw)));
if (kind === "mic") state.microphoneVolume = value;
else state.agentVolume = value;
$(`${kind}Meter`).style.width = `${value}%`;
setText(`${kind}Value`, String(value));
}
async function startConversation() {
setPhase("joining", "正在申请房间");
controls.start.disabled = true;
try {
const created = await window.deviceSimulator.createConversation();
state.conversation = created;
state.credentialExpiresAt = created.rtc.expires_at;
state.heartbeatSequence = 0;
state.agentJoined = false;
state.subtitleStarted = false;
state.transcriptSequence = 0;
state.transcriptFirstSeen.clear();
state.collectedTurnKeys.clear();
state.pendingTurns = [];
state.transcriptUpload = undefined;
state.nearRecorder = undefined;
state.farRecorder = undefined;
$("liveTranscript").innerHTML = '<li class="transcript-empty">正在等待双方的最终字幕。</li>';
renderSession(created);
log(`平台已分配 RTC 房间 ${created.rtc.room_id}。`, "success");
const engine = VERTC.createEngine(created.rtc.app_id);
state.engine = engine;
registerRTCEvents(engine);
engine.enableAudioPropertiesReport({ interval: 300 });
configureRecording(engine, created);
const automatedAudio = await prepareAutomationAudio(engine);
if (!automatedAudio) await engine.startAudioCapture(controls.microphone.value || undefined);
if (controls.speaker.value) await engine.setAudioPlaybackDevice(controls.speaker.value);
log("正在连接火山 RTC 信令并加入房间。", "info");
await Promise.race([
engine.joinRoom(created.rtc.token, created.rtc.room_id, { userId: created.rtc.user_id }, {
isAutoPublish: true,
isAutoSubscribeAudio: true,
isAutoSubscribeVideo: false,
roomProfileType: RoomProfileType.chat,
}),
new Promise<never>((_resolve, reject) => window.setTimeout(() => reject(new Error("RTC 入房超过 15 秒。请关闭代理/VPN、允许本程序通过 Windows 防火墙,或改用手机热点后重试")), 15_000)),
]);
setText("rtcStatus", "已加入");
setText("rtcDetail", created.rtc.room_id);
log("终端已加入 RTC 房间并发布麦克风。", "success");
setText("transcriptStatus", "正在启动");
await startSubtitleAndWait(engine);
await window.deviceSimulator.activateConversation({
conversationID: created.conversation_id,
roomID: created.rtc.room_id,
userID: created.rtc.user_id,
});
setPhase("waiting", "正在拉入豆包");
log("平台已提交 StartVoiceChat,等待豆包智能体激活。", "success");
await waitUntilActive(created.conversation_id);
state.startedAt = Date.now();
setPhase("active", "请开始说话");
startTimers(created.session.heartbeat_interval_sec || 15);
log("豆包语音任务已激活,可以开始双向聊天。", "success");
if (automatedAudio) {
await new Promise((resolve) => window.setTimeout(resolve, 4_000));
await state.automationAudioContext?.resume();
await state.automationAudio?.play();
log("自动验证音频已通过 RTC 自定义音轨发送。", "success");
}
} catch (error) {
setPhase("error", "启动失败");
log(`启动通话失败:${errorMessage(error)}`, "error");
await releaseRTC();
if (state.conversation) {
try {
await window.deviceSimulator.stopConversation({
conversationID: state.conversation.conversation_id,
reason: "windows_start_failed",
lastHeartbeatSequence: state.heartbeatSequence,
nearSegments: 0,
farSegments: 0,
});
} catch (cleanupError) {
log(`清理失败通话时发生错误:${errorMessage(cleanupError)}`, "error");
}
}
resetSession();
}
}
function configureRecording(engine: IRTCEngine, conversation: CreatedConversation) {
const privacy = conversation.privacy;
if (!privacy.upload_allowed || (!privacy.record_near && !privacy.record_far)) {
setText("recordingStatus", "本次未授权录音");
log("本次通话的隐私策略未授权保存原录音,不采集音频帧。", "info");
return;
}
if (privacy.record_near) {
state.nearRecorder = new PCMTrackRecorder();
engine.setAudioFrameCallback(StreamIndex.STREAM_INDEX_MAIN, undefined, (frame: AudioFrameData) => state.nearRecorder?.append(frame), 4096);
}
if (privacy.record_far) {
state.farRecorder = new PCMTrackRecorder();
engine.setAudioFrameCallback(StreamIndex.STREAM_INDEX_MAIN, conversation.rtc.agent_user_id, (frame: AudioFrameData) => state.farRecorder?.append(frame), 4096);
}
setText("recordingStatus", "录音中");
log(`已按授权开始采集${privacy.record_near ? "长者近端" : ""}${privacy.record_near && privacy.record_far ? "和" : ""}${privacy.record_far ? "智能体远端" : ""} PCM 音频。`, "success");
}
function finishRecording(engine: IRTCEngine | undefined, conversation: CreatedConversation) {
try { engine?.setAudioFrameCallback(StreamIndex.STREAM_INDEX_MAIN, undefined, undefined); } catch { /* no-op */ }
try { engine?.setAudioFrameCallback(StreamIndex.STREAM_INDEX_MAIN, conversation.rtc.agent_user_id, undefined); } catch { /* no-op */ }
const segments = [
...(state.nearRecorder?.finish("near") ?? []),
...(state.farRecorder?.finish("far") ?? []),
];
state.nearRecorder = undefined;
state.farRecorder = undefined;
return segments;
}
async function prepareAutomationAudio(engine: IRTCEngine) {
const asset = await window.deviceSimulator.getAutomationAudio();
if (!asset) return false;
const comma = asset.data_url.indexOf(",");
if (comma < 0) throw new Error("自动验证音频编码无效");
const metadata = asset.data_url.slice(0, comma);
const mime = metadata.match(/^data:([^;]+)/)?.[1] || "audio/wav";
const bytes = Uint8Array.from(atob(asset.data_url.slice(comma + 1)), (value) => value.charCodeAt(0));
const objectURL = URL.createObjectURL(new Blob([bytes], { type: mime }));
const audio = new Audio(objectURL);
audio.preload = "auto";
await new Promise<void>((resolve, reject) => {
audio.addEventListener("canplaythrough", () => resolve(), { once: true });
audio.addEventListener("error", () => reject(new Error(`无法加载自动验证音频:${asset.source_name}`)), { once: true });
audio.load();
});
const context = new AudioContext({ sampleRate: 48_000 });
const source = context.createMediaElementSource(audio);
const destination = context.createMediaStreamDestination();
source.connect(destination);
const track = destination.stream.getAudioTracks()[0];
if (!track) throw new Error("自动验证音频没有可发布的音轨");
await engine.setAudioSourceType(StreamIndex.STREAM_INDEX_MAIN, AudioSourceType.AUDIO_SOURCE_TYPE_EXTERNAL);
await engine.setExternalAudioTrack(StreamIndex.STREAM_INDEX_MAIN, track);
state.automationAudio = audio;
state.automationAudioContext = context;
state.automationAudioTrack = track;
state.automationAudioObjectURL = objectURL;
log(`已准备自动验证音频:${asset.source_name}。`, "success");
return true;
}
function renderSession(created: CreatedConversation) {
setText("conversationID", created.conversation_id);
setText("roomID", created.rtc.room_id);
setText("userID", created.rtc.user_id);
setText("agentUserID", created.rtc.agent_user_id);
setText("modeDetail", `处理方式:${created.session.engine_mode}`);
setText("agentStatus", "等待进房");
setText("agentDetail", created.rtc.agent_user_id);
}
async function waitUntilActive(conversationID: string) {
const deadline = Date.now() + 30_000;
while (Date.now() < deadline) {
const result = await window.deviceSimulator.getConversation(conversationID);
if (result.action === "continue" && result.status === "ACTIVE") return;
if (result.action === "leave_room") throw new Error(result.failure_code || `智能体启动失败:${result.status}`);
await new Promise((resolve) => window.setTimeout(resolve, 600));
}
throw new Error("等待豆包进房超时(30秒)");
}
function startTimers(intervalSeconds: number) {
stopTimers();
state.durationTimer = window.setInterval(() => {
if (!state.startedAt) return;
const seconds = Math.floor((Date.now() - state.startedAt) / 1000);
setText("duration", `${String(Math.floor(seconds / 60)).padStart(2, "0")}:${String(seconds % 60).padStart(2, "0")}`);
}, 1000);
state.heartbeatTimer = window.setInterval(() => void sendHeartbeat(), Math.max(5, intervalSeconds) * 1000);
void sendHeartbeat();
}
async function sendHeartbeat() {
if (!state.conversation || !state.credentialExpiresAt || state.phase !== "active") return;
try {
state.heartbeatSequence += 1;
const result = await window.deviceSimulator.heartbeat({
conversationID: state.conversation.conversation_id,
sequenceNo: state.heartbeatSequence,
credentialExpiresAt: state.credentialExpiresAt,
});
for (const action of result.actions || []) {
if (action.type === "renew_rtc_credential" && action.rtc?.token) {
await state.engine?.updateToken(action.rtc.token);
state.credentialExpiresAt = action.rtc.expires_at;
log("RTC Token 已通过设备心跳完成续签。", "success");
}
}
await flushTranscriptTurns();
} catch (error) {
log(`设备心跳失败:${errorMessage(error)}`, "error");
}
}
async function stopConversation(reason = "windows_user_hangup") {
if (!state.conversation) return;
setPhase("stopping", "正在结束通话");
stopTimers();
const conversation = state.conversation;
let recordingResult = "等待通话";
try {
if (state.subtitleStarted) state.engine?.stopSubtitle();
await new Promise((resolve) => window.setTimeout(resolve, 800));
await flushTranscriptTurnsWithRetry(3);
if (state.pendingTurns.length > 0) {
log(`仍有 ${state.pendingTurns.length} 条最终字幕未上传,通话暂不结束并继续自动重试。`, "error");
try {
setText("transcriptStatus", "正在恢复识别");
await state.engine?.startSubtitle({ mode: SUBTITLE_MODE.ASR_ONLY });
} catch (error) {
log(`恢复实时字幕失败:${errorMessage(error)}`, "error");
}
setPhase("active", "字幕待上传");
startTimers(conversation.session.heartbeat_interval_sec || 15);
return;
}
setText("recordingStatus", "正在封装录音");
const recordingSegments = finishRecording(state.engine, conversation);
await releaseRTC();
await window.deviceSimulator.stopConversation({
conversationID: conversation.conversation_id,
reason,
lastHeartbeatSequence: state.heartbeatSequence,
nearSegments: recordingSegments.filter((segment) => segment.track === "near").length,
farSegments: recordingSegments.filter((segment) => segment.track === "far").length,
});
log("RTC 已离房,平台已提交 StopVoiceChat。", "success");
if (recordingSegments.length > 0) {
try {
setText("recordingStatus", "等待通话结束");
await waitUntilTerminal(conversation.conversation_id);
setText("recordingStatus", "正在上传");
const totalBytes = recordingSegments.reduce((sum, segment) => sum + segment.bytes.byteLength, 0);
log(`正在上传 ${recordingSegments.length} 个聊天语音分片(${formatRecordingBytes(totalBytes)})。`, "info");
const batch = await uploadRecordingWithRetry(conversation.conversation_id, recordingSegments, 3);
setText("recordingStatus", "已上传并校验");
recordingResult = "本次已上传";
log(`聊天语音已上传并通过服务器校验,批次 ${batch.id}。`, "success");
} catch (uploadError) {
setText("recordingStatus", "上传失败");
recordingResult = "本次上传失败";
log(`聊天语音上传失败:${errorMessage(uploadError)}`, "error");
}
} else if (conversation.privacy.upload_allowed) {
setText("recordingStatus", "没有可上传音频");
recordingResult = "本次没有音频";
log("本次没有收到可封装的 PCM 音频帧,未创建录音上传批次。", "info");
} else {
recordingResult = "本次未授权录音";
}
resetSession();
setText("recordingStatus", recordingResult);
setPhase("ready", "可以再次通话");
} catch (error) {
log(`结束通话失败:${errorMessage(error)}`, "error");
resetSession();
setPhase("ready", "已释放本地音频");
}
}
async function waitUntilTerminal(conversationID: string) {
const deadline = Date.now() + 60_000;
while (Date.now() < deadline) {
const result = await window.deviceSimulator.getConversation(conversationID);
if (result.status === "COMPLETED" || result.status === "FAILED") return;
if (result.status === "EXPIRED") throw new Error("通话已过期,平台不接受录音上传");
await new Promise((resolve) => window.setTimeout(resolve, 750));
}
throw new Error("等待通话进入结束状态超时");
}
async function uploadRecordingWithRetry(conversationID: string, segments: RecordedAudioSegment[], attempts: number) {
let lastError: unknown;
for (let attempt = 1; attempt <= attempts; attempt += 1) {
try {
return await window.deviceSimulator.uploadConversationRecording({ conversationID, segments });
} catch (error) {
lastError = error;
if (attempt < attempts) {
log(`录音上传第 ${attempt} 次尝试失败,正在重试:${errorMessage(error)}`, "error");
await new Promise((resolve) => window.setTimeout(resolve, attempt * 1000));
}
}
}
throw lastError;
}
function formatRecordingBytes(value: number) {
if (value < 1024 * 1024) return `${(value / 1024).toFixed(1)} KiB`;
return `${(value / 1024 / 1024).toFixed(1)} MiB`;
}
async function releaseRTC() {
state.automationAudio?.pause();
state.automationAudioTrack?.stop();
if (state.automationAudioObjectURL) URL.revokeObjectURL(state.automationAudioObjectURL);
if (state.automationAudioContext) {
try { await state.automationAudioContext.close(); } catch { /* no-op */ }
}
state.automationAudio = undefined;
state.automationAudioTrack = undefined;
state.automationAudioContext = undefined;
state.automationAudioObjectURL = undefined;
const engine = state.engine;
state.engine = undefined;
if (!engine) return;
try { engine.stopSubtitle(); } catch { /* no-op */ }
try { await engine.unpublishStream(MediaType.AUDIO); } catch { /* no-op */ }
try { await engine.leaveRoom(); } catch { /* no-op */ }
try { await engine.stopAudioCapture(); } catch { /* no-op */ }
try { VERTC.destroyEngine(engine); } catch { /* no-op */ }
}
function stopTimers() {
if (state.heartbeatTimer) window.clearInterval(state.heartbeatTimer);
if (state.statusTimer) window.clearInterval(state.statusTimer);
if (state.durationTimer) window.clearInterval(state.durationTimer);
state.heartbeatTimer = undefined;
state.statusTimer = undefined;
state.durationTimer = undefined;
}
function resetSession() {
state.conversation = undefined;
state.credentialExpiresAt = undefined;
state.startedAt = undefined;
state.agentJoined = false;
state.agentSpeaking = false;
state.subtitleStarted = false;
state.transcriptSequence = 0;
state.transcriptFirstSeen.clear();
state.collectedTurnKeys.clear();
state.pendingTurns = [];
state.transcriptUpload = undefined;
state.subtitleStartWaiter = undefined;
state.nearRecorder = undefined;
state.farRecorder = undefined;
setText("rtcStatus", "未加入");
setText("rtcDetail", "等待开始");
setText("agentStatus", "未进房");
setText("agentDetail", "等待 StartVoiceChat");
setText("duration", "00:00");
updateVolume("mic", 0);
updateVolume("agent", 0);
setText("transcriptStatus", "未启动");
setText("recordingStatus", "等待通话");
$("liveTranscript").innerHTML = '<li class="transcript-empty">通话开始后,这里会显示双方的最终字幕。</li>';
}
controls.check.addEventListener("click", () => void configureAndProbe());
controls.start.addEventListener("click", () => void startConversation());
controls.stop.addEventListener("click", () => void stopConversation());
$("clearLog").addEventListener("click", () => $("eventLog").replaceChildren());
$("downloadRtcLog").addEventListener("click", () => {
VERTC.downloadLog();
log("已导出火山 RTC SDK 日志,请在 Windows“下载”目录查找最新日志文件。", "success");
});
window.addEventListener("beforeunload", () => {
stopTimers();
void releaseRTC();
});
setPhase("idle", "尚未连接");
+128
View File
@@ -0,0 +1,128 @@
export type AudioFrame = {
channels: Float32Array[];
channelCount: number;
sampleRate: number;
};
export type RecordedAudioSegment = {
track: "near" | "far";
sequence_no: number;
codec: "wav_pcm_s16le";
content_type: "audio/wav";
duration_ms: number;
bytes: Uint8Array;
};
const WAV_HEADER_SIZE = 44;
const MAX_SEGMENT_DURATION_MS = 9 * 60 * 1000;
const MAX_SEGMENT_SIZE = 64 * 1024 * 1024;
export class PCMTrackRecorder {
private sampleRate = 0;
private chunks: Float32Array[] = [];
private sampleCount = 0;
append(frame: AudioFrame) {
if (!Number.isFinite(frame.sampleRate) || frame.sampleRate < 8_000 || frame.sampleRate > 192_000) return;
const availableChannels = frame.channels.slice(0, Math.max(1, frame.channelCount)).filter((channel) => channel.length > 0);
if (availableChannels.length === 0) return;
const frameLength = Math.min(...availableChannels.map((channel) => channel.length));
if (frameLength < 1) return;
const mono = new Float32Array(frameLength);
for (let index = 0; index < frameLength; index += 1) {
let value = 0;
for (const channel of availableChannels) value += channel[index];
mono[index] = value / availableChannels.length;
}
if (this.sampleRate === 0) this.sampleRate = frame.sampleRate;
const normalized = frame.sampleRate === this.sampleRate ? mono : resampleLinear(mono, frame.sampleRate, this.sampleRate);
if (normalized.length === 0) return;
this.chunks.push(normalized);
this.sampleCount += normalized.length;
}
finish(track: "near" | "far"): RecordedAudioSegment[] {
if (this.sampleRate === 0 || this.sampleCount === 0) return [];
const durationLimit = Math.floor(this.sampleRate * MAX_SEGMENT_DURATION_MS / 1000);
const sizeLimit = Math.floor((MAX_SEGMENT_SIZE - WAV_HEADER_SIZE) / 2);
const samplesPerSegment = Math.max(1, Math.min(durationLimit, sizeLimit));
const segments: RecordedAudioSegment[] = [];
for (let offset = 0, sequence = 0; offset < this.sampleCount; offset += samplesPerSegment, sequence += 1) {
const length = Math.min(samplesPerSegment, this.sampleCount - offset);
const durationMS = Math.floor(length * 1000 / this.sampleRate);
if (durationMS < 1) continue;
segments.push({
track,
sequence_no: sequence,
codec: "wav_pcm_s16le",
content_type: "audio/wav",
duration_ms: durationMS,
bytes: encodeMonoWAV(this.chunks, offset, length, this.sampleRate),
});
}
this.chunks = [];
this.sampleCount = 0;
this.sampleRate = 0;
return segments;
}
}
function resampleLinear(input: Float32Array, sourceRate: number, targetRate: number) {
const outputLength = Math.max(1, Math.round(input.length * targetRate / sourceRate));
const output = new Float32Array(outputLength);
const ratio = sourceRate / targetRate;
for (let index = 0; index < outputLength; index += 1) {
const position = index * ratio;
const left = Math.min(input.length - 1, Math.floor(position));
const right = Math.min(input.length - 1, left + 1);
const fraction = position - left;
output[index] = input[left] + (input[right] - input[left]) * fraction;
}
return output;
}
function encodeMonoWAV(chunks: Float32Array[], startSample: number, sampleCount: number, sampleRate: number) {
const dataSize = sampleCount * 2;
const bytes = new Uint8Array(WAV_HEADER_SIZE + dataSize);
const view = new DataView(bytes.buffer);
writeASCII(bytes, 0, "RIFF");
view.setUint32(4, bytes.length - 8, true);
writeASCII(bytes, 8, "WAVE");
writeASCII(bytes, 12, "fmt ");
view.setUint32(16, 16, true);
view.setUint16(20, 1, true);
view.setUint16(22, 1, true);
view.setUint32(24, sampleRate, true);
view.setUint32(28, sampleRate * 2, true);
view.setUint16(32, 2, true);
view.setUint16(34, 16, true);
writeASCII(bytes, 36, "data");
view.setUint32(40, dataSize, true);
let skipped = 0;
let written = 0;
for (const chunk of chunks) {
if (skipped + chunk.length <= startSample) {
skipped += chunk.length;
continue;
}
const chunkStart = Math.max(0, startSample - skipped);
const available = Math.min(chunk.length - chunkStart, sampleCount - written);
for (let index = 0; index < available; index += 1) {
const sample = Math.max(-1, Math.min(1, chunk[chunkStart + index]));
view.setInt16(WAV_HEADER_SIZE + (written + index) * 2, sample < 0 ? Math.round(sample * 32768) : Math.round(sample * 32767), true);
}
written += available;
skipped += chunk.length;
if (written === sampleCount) break;
}
return bytes;
}
function writeASCII(target: Uint8Array, offset: number, value: string) {
for (let index = 0; index < value.length; index += 1) target[offset + index] = value.charCodeAt(index);
}
+108
View File
@@ -0,0 +1,108 @@
:root {
font-family: "Microsoft YaHei UI", "Segoe UI", sans-serif;
color: #112d3f;
background: #eef3f0;
font-synthesis: none;
}
* { box-sizing: border-box; }
body { margin: 0; min-width: 980px; min-height: 100vh; background: radial-gradient(circle at 88% 0%, #f5dfbd 0, transparent 30%), linear-gradient(145deg, #edf5f3 0%, #f7f4eb 100%); }
button, input, select { font: inherit; }
button { cursor: pointer; }
button:disabled { cursor: not-allowed; opacity: .45; }
.shell { width: min(1160px, calc(100vw - 48px)); margin: 0 auto; padding: 36px 0 48px; }
.hero { display: flex; align-items: flex-start; justify-content: space-between; gap: 32px; margin-bottom: 26px; }
.eyebrow { color: #c8652a; font-size: 12px; font-weight: 800; letter-spacing: .16em; margin: 0 0 9px; }
h1 { margin: 0; font-family: Georgia, "Microsoft YaHei UI", serif; font-size: 44px; letter-spacing: -.035em; }
h2 { margin: 0; font-size: 23px; }
h3 { margin: 24px 0 14px; color: #315966; font-size: 14px; }
.subtitle { max-width: 760px; color: #66808e; line-height: 1.8; margin: 12px 0 0; }
.overall { display: flex; align-items: center; min-width: 150px; justify-content: center; padding: 13px 18px; border-radius: 999px; background: rgba(255,255,255,.82); box-shadow: 0 12px 30px rgba(36,64,69,.1); color: #647a82; }
.overall.phase-ready, .overall.phase-active { color: #1a7d57; }
.overall.phase-error { color: #b63d2e; }
.overall.phase-completed { color: #1a7d57; }
.hero-actions { display: grid; justify-items: end; gap: 10px; }
.legacy-link { color: #617985; font-size: 12px; text-decoration: none; border-bottom: 1px solid #b9c8c7; padding-bottom: 2px; }
.legacy-link:hover { color: #c8652a; }
.status-grid { display: grid; grid-template-columns: repeat(4, 1fr); gap: 14px; margin-bottom: 16px; }
.status-card, .panel { background: rgba(255,255,255,.9); border: 1px solid rgba(23,61,69,.08); box-shadow: 0 14px 40px rgba(34,61,63,.08); }
.status-card { min-height: 118px; padding: 22px; border-radius: 20px; }
.status-card span { display: block; color: #778b93; font-size: 14px; }
.status-card strong { display: block; margin: 14px 0 6px; font-size: 21px; }
.status-card small { display: block; color: #8da0a6; white-space: nowrap; overflow: hidden; text-overflow: ellipsis; }
.content-grid { display: grid; grid-template-columns: .95fr 1.35fr; gap: 16px; }
.panel { border-radius: 24px; padding: 28px; }
.panel-heading { display: flex; justify-content: space-between; gap: 18px; align-items: flex-start; margin-bottom: 24px; }
.safe, .phase-badge { color: #287858; background: #e8f5ed; border-radius: 999px; padding: 8px 12px; font-size: 12px; font-weight: 700; }
label { display: block; color: #617985; font-size: 13px; margin-bottom: 16px; }
input, select { width: 100%; border: 1px solid #d7e2de; border-radius: 12px; padding: 12px 13px; margin-top: 7px; color: #153244; background: #f9fbfa; outline: none; }
input:focus, select:focus { border-color: #d6783f; box-shadow: 0 0 0 3px rgba(214,120,63,.12); }
.row { display: grid; grid-template-columns: 1fr 1fr; gap: 12px; }
.secondary, .primary, .danger { border: 0; border-radius: 13px; padding: 13px 18px; font-weight: 800; }
.secondary { width: 100%; color: #173b4d; background: #e9f0ee; }
.primary { flex: 1; color: white; background: #d36f32; box-shadow: 0 10px 24px rgba(211,111,50,.22); }
.danger { color: #9d382c; background: #f7e6e2; }
.hint { margin: 14px 0 0; color: #8a9aa0; font-size: 12px; line-height: 1.65; }
.conversation-orb { width: 180px; height: 180px; margin: 6px auto 24px; border-radius: 50%; display: grid; place-items: center; background: radial-gradient(circle, rgba(216,117,56,.17) 0 34%, rgba(31,88,91,.08) 35% 54%, transparent 55%); transition: .3s ease; }
.orb-core { width: 108px; height: 108px; border-radius: 50%; display: grid; place-items: center; text-align: center; padding: 12px; color: white; background: linear-gradient(145deg, #173b4d, #0d2536); box-shadow: 0 18px 34px rgba(14,43,57,.25); }
.conversation-orb.phase-active { animation: breathe 2.8s ease-in-out infinite; }
.conversation-orb.speaking { background: radial-gradient(circle, rgba(76,173,126,.25) 0 34%, rgba(31,116,91,.13) 35% 62%, transparent 63%); }
@keyframes breathe { 50% { transform: scale(1.045); } }
.meters { display: grid; grid-template-columns: 1fr 1fr; gap: 20px; }
.meter-label { display: flex; justify-content: space-between; color: #6c828b; font-size: 12px; margin-bottom: 7px; }
.meter { height: 8px; border-radius: 999px; background: #e4ebe8; overflow: hidden; }
.meter i { display: block; width: 0; height: 100%; background: linear-gradient(90deg, #d36f32, #f0ad58); transition: width .18s linear; }
.meter.agent i { background: linear-gradient(90deg, #2f8d67, #6fc591); }
.call-actions { display: flex; gap: 10px; margin: 24px 0; }
.session-info { display: grid; grid-template-columns: 1fr 1fr; gap: 10px 22px; padding-top: 18px; border-top: 1px solid #edf1ef; }
.session-info div { min-width: 0; }
.session-info dt { color: #8b9ba1; font-size: 11px; }
.session-info dd { margin: 4px 0 0; font-family: Consolas, monospace; font-size: 12px; white-space: nowrap; overflow: hidden; text-overflow: ellipsis; }
.log-panel { margin-top: 16px; padding-bottom: 18px; }
.log-panel .panel-heading { margin-bottom: 12px; }
.text-button { border: 0; color: #748991; background: transparent; }
#eventLog { list-style: none; margin: 0; padding: 0; max-height: 180px; overflow: auto; }
#eventLog li { display: grid; grid-template-columns: 82px 1fr; gap: 12px; padding: 9px 4px; border-bottom: 1px solid #edf1ef; color: #526c78; font-size: 13px; }
#eventLog time { color: #9aa8ad; font-family: Consolas, monospace; }
#eventLog li.success span { color: #247252; }
#eventLog li.error span { color: #ae3b2f; }
.transcript-panel { margin-top: 16px; }
.transcript-panel .panel-heading { margin-bottom: 14px; }
#liveTranscript { display: grid; gap: 10px; margin: 0; padding: 0; max-height: 260px; overflow: auto; list-style: none; }
.transcript-empty { color: #829198; font-size: 13px; }
.transcript-turn { display: grid; gap: 4px; max-width: 80%; padding: 11px 14px; border-radius: 15px; background: #f1f6f4; }
.transcript-turn--agent { justify-self: end; background: #edf3fb; }
.transcript-turn strong { color: #315966; font-size: 12px; }
.transcript-turn span { color: #263f49; font-size: 14px; line-height: 1.55; }
.hardware-shell { width: min(1260px, calc(100vw - 48px)); }
.hardware-content-grid { grid-template-columns: 1.05fr 1.2fr; }
.hardware-call-panel { min-height: 720px; }
.hardware-session-info { grid-template-columns: 1fr 1fr; }
.text-question { display: grid; grid-template-columns: 1fr 150px; gap: 10px; margin-bottom: 22px; }
.text-question input { margin-top: 0; }
.text-question .secondary { padding: 10px; }
.flow-panel { margin-top: 16px; }
.flow-panel .panel-heading { margin-bottom: 14px; }
.flow-checks { display: grid; grid-template-columns: repeat(3, minmax(0, 1fr)); gap: 10px; list-style: none; padding: 0; margin: 0; }
.flow-check { display: grid; grid-template-columns: 28px 1fr; grid-template-rows: auto auto; column-gap: 10px; align-items: center; padding: 13px; border: 1px solid #e3ebe8; border-radius: 14px; background: #fafcfb; }
.flow-check i { grid-row: 1 / span 2; display: grid; place-items: center; width: 26px; height: 26px; border-radius: 50%; font-style: normal; font-weight: 800; color: #809399; background: #e8efed; }
.flow-check span { min-width: 0; color: #31505e; font-size: 13px; font-weight: 700; }
.flow-check small { min-width: 0; margin-top: 4px; color: #8b9ba1; font-size: 11px; white-space: nowrap; overflow: hidden; text-overflow: ellipsis; }
.flow-check--passed { border-color: #cce5d7; background: #f4faf6; }
.flow-check--passed i { color: white; background: #2f8d67; }
.flow-check--failed { border-color: #edcbc5; background: #fff7f5; }
.flow-check--failed i { color: white; background: #b94c3d; }
@media (max-width: 1050px) {
.status-grid { grid-template-columns: repeat(2, 1fr); }
.content-grid { grid-template-columns: 1fr; }
.flow-checks { grid-template-columns: repeat(2, minmax(0, 1fr)); }
}
@@ -0,0 +1,108 @@
const crypto = require("node:crypto");
const { WebSocketServer } = require("ws");
const { websocketSignature } = require("../electron/hardware-agent-client.cjs");
const port = Number(process.env.MOCK_HARDWARE_GATEWAY_PORT || 19090);
const expectedSecret = process.env.MOCK_VOLC_DEVICE_SECRET || "mock-volc-device-secret-at-least-32-characters";
const server = new WebSocketServer({ host: "127.0.0.1", port, maxPayload: 1024 * 1024 });
function send(socket, value) { socket.send(JSON.stringify(value)); }
async function requestPlatformStop(conversationID) {
const baseURL = String(process.env.MOCK_PLATFORM_API_URL || "").replace(/\/$/, "");
const deviceID = String(process.env.MOCK_PLATFORM_DEVICE_ID || "");
const secret = String(process.env.MOCK_PLATFORM_DEVICE_SECRET || "");
if (!baseURL || !deviceID || !secret || !conversationID) return;
const requestPath = `/device/v2/conversations/${encodeURIComponent(conversationID)}/stop`;
const body = JSON.stringify({ reason: "windows_e2e_platform_stop", device_time: new Date().toISOString(), last_heartbeat_sequence: 1, rtc_left: true, recording: { near_segments: 0, far_segments: 0 } });
const timestamp = new Date().toISOString();
const nonce = crypto.randomBytes(16).toString("hex");
const digest = crypto.createHash("sha256").update(body).digest("hex");
const canonical = ["POST", requestPath, timestamp, nonce, digest].join("\n");
const signature = crypto.createHmac("sha256", secret).update(canonical).digest("base64url");
const response = await fetch(`${baseURL}${requestPath}`, {
method: "POST",
headers: {
Accept: "application/json", "Content-Type": "application/json", "Idempotency-Key": `windows-platform-stop-${crypto.randomUUID()}`,
"X-Device-Id": deviceID, "X-Device-Timestamp": timestamp, "X-Device-Nonce": nonce,
"X-Device-Content-SHA256": digest, "X-Device-Signature": signature,
},
body,
});
if (!response.ok) throw new Error(`platform stop HTTP ${response.status}: ${await response.text()}`);
process.stdout.write(`${JSON.stringify({ type: "platform_stop_requested", conversation_id: conversationID })}\n`);
}
function tonePCMBase64() {
const samples = 2400;
const bytes = Buffer.alloc(samples * 2);
for (let index = 0; index < samples; index += 1) {
const sample = Math.round(Math.sin((index / 16000) * Math.PI * 2 * 440) * 1800);
bytes.writeInt16LE(sample, index * 2);
}
return bytes.toString("base64");
}
server.on("listening", () => process.stdout.write(`${JSON.stringify({ type: "listening", port })}\n`));
server.on("connection", (socket, request) => {
const header = (name) => String(request.headers[name.toLowerCase()] || "");
const expectedSignature = websocketSignature({
authType: 1,
deviceName: header("x-device-name"),
randomNumber: Number(header("x-random-num")),
productKey: header("x-product-key"),
timestamp: Number(header("x-timestamp")),
instanceID: header("x-instance-id"),
}, expectedSecret);
if (!crypto.timingSafeEqual(Buffer.from(header("x-signature")), Buffer.from(expectedSignature))) {
socket.close(1008, "invalid signature");
return;
}
const target = new URL(request.url, `ws://127.0.0.1:${port}`);
if (!target.searchParams.get("bot") || target.searchParams.get("wait_for_session_update") !== "true") {
socket.close(1008, "invalid query");
return;
}
process.stdout.write(`${JSON.stringify({ type: "authenticated", device: header("x-device-name"), bot: target.searchParams.get("bot") })}\n`);
send(socket, { type: "session.created", event_id: "mock-created", session: { id: "mock-provider-session-windows", model: "doubao", input_audio_format: "pcm16", output_audio_format: "pcm16", turn_detection: { type: "server_vad" } } });
let replied = false;
let dialogID = "";
socket.on("message", (raw) => {
const event = JSON.parse(raw.toString("utf8"));
if (event.type === "session.update") {
const llmPrompt = event.session?.config?.LLMConfig?.SystemMessages?.[0];
const s2sDialog = event.session?.config?.S2SConfig?.ProviderParams?.dialog;
const sessionDialogID = String(s2sDialog?.dialog_id || (llmPrompt ? llmPrompt.match(/本次对话标识:([^\n]+)/)?.[1] : "") || "").trim();
const frozenPrompt = String(llmPrompt || s2sDialog?.system_role || "").trim();
if (event.session?.object !== "realtime.session" || event.session?.config?.ASRConfig?.TurnDetectionMode !== 0 || !sessionDialogID || !frozenPrompt) {
send(socket, { type: "error", event_id: "mock-error", error: { code: "INVALID_SESSION_CONFIG", message: "frozen context missing" } });
return;
}
dialogID = sessionDialogID;
send(socket, { type: "session.updated", event_id: "mock-updated", session: { ...event.session, id: "mock-provider-session-windows" } });
process.stdout.write(`${JSON.stringify({ type: "dynamic_config_applied", dialog_id: dialogID })}\n`);
return;
}
if (event.type === "input_audio_buffer.append" && !replied) {
replied = true;
send(socket, { type: "input_audio_buffer.speech_started", event_id: "mock-speech-start", audio_start_ms: 0 });
send(socket, { type: "input_audio_buffer.speech_stopped", event_id: "mock-speech-stop", audio_end_ms: 800 });
send(socket, { type: "conversation.item.input_audio_transcription.completed", event_id: "mock-user-final", item_id: "mock-user-item", transcript: "你好,请介绍一下今天的安排。" });
send(socket, { type: "response.created", event_id: "mock-response-created", response: { id: "mock-response-1", status: "in_progress" } });
send(socket, { type: "response.audio_transcript.delta", event_id: "mock-agent-delta", response_id: "mock-response-1", delta: "您好,今天可以先聊聊您最想做的事情。" });
send(socket, { type: "response.audio.delta", event_id: "mock-audio-delta", response_id: "mock-response-1", delta: tonePCMBase64() });
send(socket, { type: "response.audio_transcript.done", event_id: "mock-agent-final", response_id: "mock-response-1", transcript: "您好,今天可以先聊聊您最想做的事情。" });
send(socket, { type: "response.audio.done", event_id: "mock-audio-done", response_id: "mock-response-1" });
send(socket, { type: "response.done", event_id: "mock-response-done", response: { id: "mock-response-1", status: "completed", output: [] } });
process.stdout.write(`${JSON.stringify({ type: "round_completed" })}\n`);
if (process.env.MOCK_REQUEST_PLATFORM_STOP === "1") {
setTimeout(() => void requestPlatformStop(dialogID).catch((error) => {
process.stderr.write(`${error.stack || error.message}\n`);
}), 250);
}
}
});
socket.on("close", (code) => process.stdout.write(`${JSON.stringify({ type: "closed", code })}\n`));
});
function shutdown() { server.close(() => process.exit(0)); }
process.on("SIGINT", shutdown);
process.on("SIGTERM", shutdown);
@@ -0,0 +1,120 @@
param(
[ValidateSet('llm', 's2s')]
[string]$ModelMode = 'llm',
[string]$ExecutablePath = ''
)
$ErrorActionPreference = 'Stop'
$PSNativeCommandUseErrorActionPreference = $true
$simulatorRoot = (Resolve-Path -LiteralPath (Join-Path $PSScriptRoot '..')).Path
$projectRoot = (Resolve-Path -LiteralPath (Join-Path $simulatorRoot '..')).Path
$devicePublicID = '10000000-0000-0000-0000-000000000511'
$platformSecret = 'local-simulated-device-secret-32-bytes-minimum'
$volcSecret = 'mock-volc-device-secret-at-least-32-characters'
$mockPort = 19090
$runID = [Guid]::NewGuid().ToString('N')
$resultPath = Join-Path $env:TEMP "hardware-agent-e2e-$runID.json"
$mockOutput = Join-Path $env:TEMP "hardware-agent-gateway-$runID.log"
$mockError = Join-Path $env:TEMP "hardware-agent-gateway-$runID.err.log"
$mockProcess = $null
$conversationID = $null
$deviceInternalID = $null
Set-Location -LiteralPath $projectRoot
try {
$ready = Invoke-WebRequest -UseBasicParsing -Uri 'http://localhost:58080/health/ready' -TimeoutSec 10
if ($ready.StatusCode -ne 200) { throw 'Local API is not ready' }
$deviceInternalID = (docker compose exec -T postgres psql -U voice -d voice_platform -t -A -c "SELECT id FROM device WHERE public_id='$devicePublicID'::uuid" | Out-String).Trim()
if ($deviceInternalID -notmatch '^[0-9a-fA-F-]{36}$') { throw 'Local simulator device is missing' }
$bindingCount = (docker compose exec -T postgres psql -U voice -d voice_platform -t -A -c "SELECT count(*) FROM hardware_agent_device_binding WHERE device_id='$deviceInternalID'::uuid" | Out-String).Trim()
if ($bindingCount -ne '0') { throw 'Simulator device already has a hardware binding; refusing to overwrite it' }
"UPDATE device_twin SET desired_config=desired_config||'{`"voice_runtime`":`"volc_hardware_agent`",`"hardware_agent_transport`":`"low_load`",`"minimum_sdk_version`":`"1.0.0`"}'::jsonb WHERE device_id='$deviceInternalID'::uuid; INSERT INTO hardware_agent_device_binding(tenant_id,device_id,bot_id,transport,minimum_sdk_version,status) SELECT tenant_id,id,'mock-windows-hardware-bot','low_load','1.0.0','ACTIVE' FROM device WHERE id='$deviceInternalID'::uuid;" | docker compose exec -T postgres psql -U voice -d voice_platform -v ON_ERROR_STOP=1 -q | Out-Null
$mockScript = Join-Path $simulatorRoot 'test\mock-hardware-gateway.cjs'
$env:MOCK_HARDWARE_GATEWAY_PORT = [string]$mockPort
$env:MOCK_VOLC_DEVICE_SECRET = $volcSecret
$env:MOCK_REQUEST_PLATFORM_STOP = '1'
$env:MOCK_PLATFORM_API_URL = 'http://localhost:58080'
$env:MOCK_PLATFORM_DEVICE_ID = $devicePublicID
$env:MOCK_PLATFORM_DEVICE_SECRET = $platformSecret
$mockProcess = Start-Process -FilePath 'node' -ArgumentList @($mockScript) -PassThru -WindowStyle Hidden -RedirectStandardOutput $mockOutput -RedirectStandardError $mockError
$deadline = [DateTime]::UtcNow.AddSeconds(10)
do {
Start-Sleep -Milliseconds 150
$listening = Test-NetConnection 127.0.0.1 -Port $mockPort -InformationLevel Quiet -WarningAction SilentlyContinue
if ($mockProcess.HasExited) { throw "Mock gateway exited: $(Get-Content -LiteralPath $mockError -Raw -ErrorAction SilentlyContinue)" }
} until ($listening -or [DateTime]::UtcNow -gt $deadline)
if (-not $listening) { throw 'Mock hardware gateway did not start' }
$env:SIMULATOR_AUTO_TEST = '1'
$env:SIMULATOR_AUTO_WAIT_FOR_PLATFORM_STOP = '1'
$env:SIMULATOR_AUTO_RESULT_PATH = $resultPath
$env:SIMULATOR_DEFAULT_API_URL = 'http://localhost:58080'
$env:SIMULATOR_DEFAULT_DEVICE_ID = $devicePublicID
$env:SIMULATOR_DEFAULT_DEVICE_SECRET = $platformSecret
$env:SIMULATOR_VOLC_INSTANCE_ID = 'mock-instance'
$env:SIMULATOR_VOLC_PRODUCT_KEY = 'mock-product'
$env:SIMULATOR_VOLC_DEVICE_NAME = 'windows-e2e-device'
$env:SIMULATOR_VOLC_HARDWARE_ID = 'windows-e2e-hardware'
$env:SIMULATOR_VOLC_DEVICE_SECRET = $volcSecret
$env:SIMULATOR_VOLC_MODEL_MODE = $ModelMode
$env:SIMULATOR_HARDWARE_GATEWAY_URL = "ws://127.0.0.1:$mockPort/v1/realtime"
Set-Location -LiteralPath $simulatorRoot
if ($ExecutablePath) {
$resolvedExecutable = (Resolve-Path -LiteralPath $ExecutablePath).Path
$electron = Start-Process -FilePath $resolvedExecutable -WorkingDirectory (Split-Path -Parent $resolvedExecutable) -PassThru -Wait
} else {
$electron = Start-Process -FilePath (Join-Path $simulatorRoot 'node_modules\electron\dist\electron.exe') -ArgumentList @('.') -WorkingDirectory $simulatorRoot -PassThru -Wait
}
if ($electron.ExitCode -ne 0) { throw "Electron E2E process failed with exit code $($electron.ExitCode)" }
if (-not (Test-Path -LiteralPath $resultPath -PathType Leaf)) { throw 'Electron E2E result file is missing' }
$result = Get-Content -LiteralPath $resultPath -Raw | ConvertFrom-Json
if ($result.result -ne 'PASS' -or $result.phase -ne '已完成') { throw "Electron E2E failed: $($result | ConvertTo-Json -Depth 8 -Compress)" }
if ($result.model_mode -ne $ModelMode) { throw "Electron E2E used model mode '$($result.model_mode)' instead of '$ModelMode'" }
$conversationID = [string]$result.conversation_id
if ($conversationID -notmatch '^[0-9a-fA-F-]{36}$') { throw 'Electron E2E did not return a conversation ID' }
$databaseState = (docker compose exec -T postgres psql -U voice -d voice_platform -t -A -F '|' -c "SELECT c.runtime_type||'|'||c.status||'|'||hs.status||'|'||hs.sdk_version||'|'||(hs.dynamic_config_applied_at IS NOT NULL)::text||'|'||count(DISTINCT ct.id)||'|'||(SELECT count(*) FROM web_event we WHERE we.resource_public_id=c.public_id AND we.event_type IN('conversation.started','conversation.completed')) FROM conversation c JOIN hardware_agent_session hs ON hs.conversation_id=c.id LEFT JOIN conversation_turn ct ON ct.conversation_id=c.id WHERE c.public_id='$conversationID'::uuid GROUP BY c.runtime_type,c.status,hs.status,hs.sdk_version,hs.dynamic_config_applied_at,c.public_id" | Out-String).Trim()
if ($databaseState -ne 'volc_hardware_agent|COMPLETED|COMPLETED|1.1.0-windows-standard-protocol|true|2|2') { throw "Unexpected platform state: $databaseState" }
$gatewayEvents = Get-Content -LiteralPath $mockOutput -Raw
if ($gatewayEvents -notmatch 'dynamic_config_applied' -or $gatewayEvents -notmatch 'round_completed' -or $gatewayEvents -notmatch 'platform_stop_requested') { throw "Mock gateway did not observe config, media, and platform-stop flow: $gatewayEvents" }
[ordered]@{
result = 'PASS'
conversation_id = $conversationID
platform_state = $databaseState
transcript_turns = $result.turns.Count
checks_passed = @($result.checks | Where-Object status -eq 'passed').Count
model_mode = $result.model_mode
mock_gateway_log = $mockOutput
} | ConvertTo-Json -Compress
}
finally {
Set-Location -LiteralPath $projectRoot
if ($mockProcess -and -not $mockProcess.HasExited) { Stop-Process -Id $mockProcess.Id -Force -ErrorAction SilentlyContinue }
if ($conversationID -match '^[0-9a-fA-F-]{36}$') {
@"
BEGIN;
CREATE TEMP TABLE hc AS SELECT id,context_snapshot_id,consent_snapshot_id FROM conversation WHERE public_id='$conversationID'::uuid;
DELETE FROM audit_event WHERE resource_id IN(SELECT id FROM hc);
DELETE FROM web_event WHERE resource_public_id='$conversationID'::uuid;
DELETE FROM idempotency_record WHERE resource_id IN(SELECT id FROM hc);
DELETE FROM conversation_turn WHERE conversation_id IN(SELECT id FROM hc);
DELETE FROM conversation_memory_use WHERE conversation_id IN(SELECT id FROM hc);
DELETE FROM memory_tool_call WHERE conversation_id IN(SELECT id FROM hc);
DELETE FROM hardware_agent_event WHERE conversation_id IN(SELECT id FROM hc);
DELETE FROM hardware_agent_session WHERE conversation_id IN(SELECT id FROM hc);
DELETE FROM conversation_usage_budget WHERE conversation_id IN(SELECT id FROM hc);
DELETE FROM outbox_event WHERE aggregate_id IN(SELECT id FROM hc);
DELETE FROM conversation_state_history WHERE conversation_id IN(SELECT id FROM hc);
DELETE FROM conversation WHERE id IN(SELECT id FROM hc);
DELETE FROM context_snapshot WHERE id IN(SELECT context_snapshot_id FROM hc);
DELETE FROM consent_snapshot WHERE id IN(SELECT consent_snapshot_id FROM hc);
COMMIT;
"@ | docker compose exec -T postgres psql -U voice -d voice_platform -v ON_ERROR_STOP=1 -q | Out-Null
}
if ($deviceInternalID -match '^[0-9a-fA-F-]{36}$') {
"DELETE FROM hardware_agent_device_binding WHERE device_id='$deviceInternalID'::uuid; UPDATE device_twin SET desired_config=desired_config-'voice_runtime'-'hardware_agent_transport'-'minimum_sdk_version' WHERE device_id='$deviceInternalID'::uuid;" | docker compose exec -T postgres psql -U voice -d voice_platform -v ON_ERROR_STOP=1 -q | Out-Null
}
@('MOCK_HARDWARE_GATEWAY_PORT','MOCK_VOLC_DEVICE_SECRET','MOCK_REQUEST_PLATFORM_STOP','MOCK_PLATFORM_API_URL','MOCK_PLATFORM_DEVICE_ID','MOCK_PLATFORM_DEVICE_SECRET','SIMULATOR_AUTO_TEST','SIMULATOR_AUTO_WAIT_FOR_PLATFORM_STOP','SIMULATOR_AUTO_RESULT_PATH','SIMULATOR_DEFAULT_API_URL','SIMULATOR_DEFAULT_DEVICE_ID','SIMULATOR_DEFAULT_DEVICE_SECRET','SIMULATOR_VOLC_INSTANCE_ID','SIMULATOR_VOLC_PRODUCT_KEY','SIMULATOR_VOLC_DEVICE_NAME','SIMULATOR_VOLC_HARDWARE_ID','SIMULATOR_VOLC_DEVICE_SECRET','SIMULATOR_VOLC_MODEL_MODE','SIMULATOR_HARDWARE_GATEWAY_URL') | ForEach-Object { Remove-Item "Env:$_" -ErrorAction SilentlyContinue }
Remove-Item -LiteralPath $resultPath,$mockOutput,$mockError -Force -ErrorAction SilentlyContinue
}
+13
View File
@@ -0,0 +1,13 @@
{
"compilerOptions": {
"target": "ES2022",
"useDefineForClassFields": true,
"module": "ESNext",
"moduleResolution": "Bundler",
"strict": true,
"noEmit": true,
"lib": ["ES2022", "DOM", "DOM.Iterable"],
"skipLibCheck": true
},
"include": ["src/**/*.ts", "vite.config.ts"]
}
+17
View File
@@ -0,0 +1,17 @@
import { defineConfig } from "vite";
import { resolve } from "node:path";
export default defineConfig({
base: "./",
build: {
outDir: "dist",
emptyOutDir: true,
sourcemap: true,
rollupOptions: {
input: {
hardware: resolve(__dirname, "index.html"),
legacy: resolve(__dirname, "legacy.html"),
},
},
},
});
@@ -0,0 +1,24 @@
云端硬件对话智能体测试程序使用说明
1. 将 ZIP 完整解压到普通文件夹,不要直接在压缩包内运行。
2. 双击“启动云端测试设备.cmd”,或运行 hardware-agent-tester-0.3.2-windows-x64.exe。
3. 首次运行允许 Windows 麦克风访问。
4. 填写平台设备信息和 Hardware ID;火山 Instance ID、Product Key、Device Name 由平台自动下发。
5. Product Secret 只配置在平台密钥服务中,测试程序不会要求输入;平台代设备注册后只下发本设备的一机一密。
6. 根据火山控制台中的 Bot 类型选择“通用 LLM”或“豆包端到端语音”。选错会导致 session.update 校验失败。
7. 点击“检测、鉴权并准备设备”,13项验收区开始记录证据。
8. 检测通过后点击“开始硬件智能体通话”。只有收到火山 session.updated 且平台确认 ACTIVE 后才会开放麦克风。
9. 完成双方对话后点击结束;也可以从平台停止通话,程序会在 session_updated 回执中收到 stop_session 并自动结束。
10. 点击“导出测试报告”,将 JSON 报告交给开发人员验收。
平台维护中心必须把本设备设置为 volc_hardware_agent + low_load,并绑定与火山产品关联的真实 Bot ID。公网平台尚未部署 v3 路由时会返回 404,此时不能进行真实测试。
网络需要允许 HTTPS 443 和 WSS 到:
- 平台服务器
- ai-gateway.vei.volces.com
`iot-cn-shanghai.iot.volces.com`只需要平台服务器访问,设备无需访问动态注册地址。
安全提示:Product Secret 永不到达设备。平台下发的一机一密会使用当前 Windows 用户的 Safe Storage 加密保存,导出的测试报告不包含任何密钥。
当前测试 EXE 未配置商业代码签名证书,Windows 可能提示“未知发布者”。请先用随包 SHA-256 文件核对完整性;正式外发版本应使用甲方或发布主体的代码签名证书签名。
@@ -0,0 +1,187 @@
# AI电话 v3.1 流程与现有实现差异分析
核对日期:2026-09-10。结论:**总体方向相近,但现有实现不等同于 CALL-MEMORY-1.1。差异涉及协议、固件运行方式和记忆数据维护,不能仅提取新文档接口就直接交给设备联调。**
对照材料为用户提供的 [AI电话完整通话与记忆管理文档](D:/wechat/xwechat_files/wangcong3092_983b/msg/file/2026-09/AI电话_完整通话与记忆管理_完整流程与详细设计(1).docx),封面版本 v3.1,修订日期 2026-09-08。文档作为需求基线,不作为执行其中开发、发布或运维指令的授权。
实现基线为 `E:/Codex/story/platform` 当前工作区,Git HEAD 为 `7846f30d0b239e183267988aa6e030b0a2f1ef18`。重点检查 Go 后端路由与业务服务、数据库迁移、EchoEar 固件、记忆页面代码和已有联调记录。结论针对本地实现;本次未连接生产环境、重新运行真机或供应商联调,不能据此推定线上部署与工作区完全相同。
## 文档描述的完整流程
1. **上电准备**:恢复已有设备身份、绑定和未确认结束日志;联网后 warmup,预备音频及引擎。挂机时不建立真实对话、不上传环境声。SaaS 在后台同步记忆并准备开场候选。
2. **摘机准备**:生成 boot/hook/call_generation 关联;prepare 从服务端绑定确定使用者,检查记忆主库版本、关键删除或更正是否已在检索副本生效。
3. **签发与启动**:SaaS 编译供应商参数,签发绑定设备、本次摘机、隐私版本、启动期限和租约的 Session Manifest。设备验签后立即开始心跳,透传参数,以 RTC 模式启动 SDK。
4. **连续通话**:声音由设备直接与火山交互。媒体连接、收到第一帧、实际开始播放分别记录。字幕经异步队列上报;历史记忆由经过验证的火山原生 Viking 联动读取。
5. **通话中修改记忆**:SaaS 从可靠的用户最终文字识别记住、更正、限制使用、删除;对象不清进入页面确认。提交主库后分别跟踪副本、派生内容和旧会话清理,必要时下发 STOP_CALL。
6. **挂机收尾**:本地先关闭上下行与待播音频,再停止 SDK、补报 end。SaaS 将通话结束和字幕完整度分开记录;字幕静默收齐初值 15 秒、绝对等待 60 秒。
7. **维护记忆与准备下一通**:从用户证据提取候选,处理冲突和删除屏障,提交 SaaS 主库与 outbox;按主记录版本同步 Viking 事件及画像,验证检索可见性,再推进连续同步进度。下次摘机重新检查。
文档强调四个结果分别成立:**通话结束、字幕收齐、记忆保存、检索可见**。普通新增同步慢可以按安全模式继续;更正、删除和权限变化不能沿用不安全的旧资料。
## 现有流程
当前 EchoEar 主路径为:
```text
上电恢复平台及火山凭据,尝试补报上次未结束会话,初始化音频与引擎
→ 触摸或唤醒触发对话
→ bootstrap + config + POST /device/v3/agent-sessions
→ 后端冻结使用者、授权和上下文,返回 runtime + session_config
→ 固件组装 session.update,使用 VOLC_MODE_WS 启动
→ session.created/updated 后上报 session_started
→ 通过 session_updated 周期续期,异步上传最终字幕
→ volc_stop + session_ended/session_failed,清理播放器
→ SaaS 摘要/故事/候选处理
→ 另一路将通话文字 AddSession 到 Viking 自动抽取
```
入口依据:[固件创建会话](E:/Codex/story/platform/firmware/echoear-platform-agent/main/platform_client.c:558)、[SDK 启动与退出](E:/Codex/story/platform/firmware/echoear-platform-agent/components/application/service/conv_service/src/volc_conv.c:389)、[后台任务装配](E:/Codex/story/platform/internal/app/app.go:427)。
## 可以复用的实现
| 能力 | 已有实现 | 对齐程度 |
|---|---|---|
| 设备认证与服务端确定使用者 | HMAC、防重放、设备有效绑定;会话保存 subject_id 和授权快照 | 方向一致,认证报文及绑定代数协议不同 |
| 音频与业务链路分开 | 设备直接调用火山 SDK,平台接收控制信息与文字 | 方向一致,EchoEar 实际使用 WS |
| 共享助手和用户记忆范围 | Bot 配置、稳定助手标识、租户与使用者派生的 Viking user_id | 已有基础,不能替代供应商侧防篡改授权验证 |
| 异步文字上报 | SDK 回调投递内存队列,业务任务发 HTTPS,后端事务保存并去重 | 主思路一致,缺修订、缺口和持久重试协议 |
| 结束恢复 | NVS 保存待结束会话,重启补报;后端支持结束状态及迟到最终文字 | 已有基础,缺未取得 SID 的取消及独立远端停止证明 |
| 后台摘要与人工记忆维护 | 用户文字来源、摘要、故事、候选、人工确认、记忆版本、编辑/停用/删除、索引 outbox | 已有基础,与 Viking 自动抽取副本未统一 |
证据:[会话归属与上下文快照](E:/Codex/story/platform/internal/orchestration/repository.go:293)、[字幕保存与原会话归属](E:/Codex/story/platform/internal/orchestration/transcript_batch.go:127)、[仅用户文字参与抽取](E:/Codex/story/platform/internal/extraction/processor.go:94)、[人工记忆维护](E:/Codex/story/platform/internal/saas/content_service.go:568)。
## 关键差异及影响
### 1 设备接口与认证不兼容
文档附件 B 定义七个 `POST /api/v1/device/calls/*` 接口;当前注册的是 `/device/v2/*` 和 `/device/v3/agent-sessions*`。这七个新路径没有注册。
认证也不只是换路径:
| 项目 | v3.1 文档 | 当前实现 |
|---|---|---|
| 时间 | `X-Timestamp-Ms`,Unix 毫秒 | `X-Device-Timestamp`,RFC3339 |
| 签名入口 | `Authorization: Device-HMAC-SHA256 ...` | `X-Device-Signature` |
| 签名内容 | DEVREQ1 开头的九行,含设备、密钥版本、请求 ID | 方法、含查询串的 RequestURI、时间、nonce、body hash 五行 |
| 签名编码 | 小写十六进制 | 无填充 Base64url |
| 响应 | `ok/code/request_id/server_time_ms/data` | `request_id/data/error` |
现有 HMAC 本身已经实现,但不兼容文档的线协议。直接按文档开发新固件会遇到路由或验签失败。
证据:[设备路由](E:/Codex/story/platform/internal/httpapi/router.go:308)、[认证头及验签](E:/Codex/story/platform/internal/deviceauth/verifier.go:22)、[当前签名算法](E:/Codex/story/platform/internal/deviceauth/verifier.go:142)、[响应封装](E:/Codex/story/platform/internal/httpapi/response.go:8)。
### 2 固件实际使用 WS 而文档固定 RTC
文档附件 C 要求 `mode=RTC`,映射 `VOLC_MODE_RTC`。当前 EchoEar 能力声明只有 `low_load`,实际 `volc_start` 参数写的是 `VOLC_MODE_WS`。
后端保留 high_quality/low_load 两类能力,并不等于当前固件已经按 RTC 流程运行。切换需要核对 RTC 参数通道、字幕格式、首音频、打断和 stop 行为,不能只改枚举后视为完成。
证据:[固件能力声明](E:/Codex/story/platform/firmware/echoear-platform-agent/main/platform_client.c:569)、[实际 SDK 模式](E:/Codex/story/platform/firmware/echoear-platform-agent/components/application/service/conv_service/src/volc_conv.c:439)、[后端传输校验](E:/Codex/story/platform/internal/orchestration/repository.go:614)。
### 3 没有新版会话清单及摘机取消机制
文档要求 `boot_id + hook_event_id` 唯一业务键,独立 `call_generation`、`binding_generation`、`media_attempt_id`,以及 `manifest_b64/manifest_mac`。即使 end 先于 prepare 到达,也要保存取消记录,阻止晚到 prepare 启动。
当前创建请求使用 `Idempotency-Key`,返回 `session_nonce/config_digest/runtime/session_config`;具有请求幂等和配置摘要,但没有上述清单结构、按摘机取消、清单撤销及代数校验。当前摘要也不是文档要求的清单 HMAC。尚未取得 conversation_id 时没有等价的结束入口。
证据:[请求与响应结构](E:/Codex/story/platform/internal/orchestration/types.go:95)、[创建幂等入口](E:/Codex/story/platform/internal/orchestration/hardware_agent.go:31)、[固件响应处理](E:/Codex/story/platform/firmware/echoear-platform-agent/main/platform_client.c:584)。
### 4 心跳起点与本地租约停止不同
文档要求验清单后立即开始心跳,与 SDK 启动并行;默认周期 5 秒、租约不超过 45 秒。失联时固件依本地保守截止关闭音频;迟到许可不能复活会话。
当前固件在上报 session_started 后设置下一次心跳,通过 `session_updated` 续期。默认周期为 15 秒;后端租约按 `HeartbeatInterval × 5` 计算,默认对应 75 秒。心跳网络失败时固件记录错误并再次尝试,没有看到该路径维护本地授权截止或到期静音。
这意味着现有服务端超时回收不能等同于文档要求的设备本地有界停止。
证据:[固件续期逻辑](E:/Codex/story/platform/firmware/echoear-platform-agent/main/platform_client.c:693)、[启动成功后才轮询](E:/Codex/story/platform/firmware/echoear-platform-agent/components/application/service/conv_service/src/volc_conv.c:459)、[后端租约](E:/Codex/story/platform/internal/orchestration/hardware_agent.go:205)、[默认周期](E:/Codex/story/platform/internal/config/config.go:479)。
### 5 供应商参数由固件拼装且开场配置未形成文档流程
文档要求 SaaS 基于已验证 provider_profile 编译完整 `vendor_params_json`,设备只透传;热上下文与欢迎语分别产生,且附事实依赖版本。
目前后端返回 authorized_context 等结构化字段,固件自行拼接固定话术、LLMConfig、S2SConfig、MemoryConfig 与回调参数。平台虽然有资料、最近摘要、open_topic 和上下文快照,但未形成独立欢迎语、开场候选依赖失效及 provider_profile 能力证据驱动的编译流程。
当前上下文对部分字段按字符数裁剪,也未体现文档要求的按完整事实删减并保留否定、日期和条件的规则。
证据:[端侧参数组装](E:/Codex/story/platform/firmware/echoear-platform-agent/main/platform_client.c:457)、[上下文与摘要选择](E:/Codex/story/platform/internal/orchestration/repository.go:388)、[开场续聊与裁剪](E:/Codex/story/platform/internal/orchestration/companion_context.go:19)。
### 6 Viking 写入路线与文档有实质冲突
文档第 11 章要求 SaaS 是记忆事实维护方,Viking 保存已核验主库记录的检索副本。默认不让两边根据原始字幕各自抽取和维护同一份记忆。
当前工作器在内容抽取任务中先排入独立 Viking 同步任务;Viking 服务读取 conversation_turn 中的用户和 AI 最终文字,再调用 `/api/memory/session/add`,并设置 `extract_memory_type` 让 Viking 抽取事件及画像。这条任务不以 SaaS curated_memory 提交完成为前置条件。
因此当前是“平台内容整理 + Viking 会话自动抽取”两条路径。它不能保证平台更正、停用或删除某条人工记忆后,Viking 独立生成的事件和画像同步发生相同变化。
证据:[工作器装配](E:/Codex/story/platform/internal/app/app.go:427)、[同步数据取自通话文字](E:/Codex/story/platform/internal/vikingmemory/service.go:175)、[AddSession 及抽取类型](E:/Codex/story/platform/internal/vikingmemory/client.go:127)。
### 7 单条记忆修改删除未贯通 Viking 及旧会话
当前平台已经有 `UpdateMemory/RequestMemoryDeletion`、版本比较、本地索引失效和删除任务。另有 `providermemory` 适配器,调用自定义 `/v1/memories:upsert`,它与 `vikingmemory` 是独立模块,不能把前者的完成视为 Viking 原生事件/画像已清理。
已有接入说明明确指出:单条本地 curated memory 删除不会删除独立抽取的 Viking 事件/画像,统一管理界面尚未实现。文档要求的 source_suppression、单条投影映射、派生摘要清理及防旧源重抽取尚未贯通。
此外,本次在相关源码和迁移中没有找到 `privacy_epoch`、已签发清单撤销目标、对应 `STOP_CALL` 及分阶段本地静音/SDK 停止回执。已有通话停止和授权撤回能力不能替代文档附件 L 的完整机制。
证据:[单条删除入口](E:/Codex/story/platform/internal/saas/content_service.go:705)、[本地删除处理](E:/Codex/story/platform/internal/memorylifecycle/delete_processor.go:32)、[独立 provider 适配器](E:/Codex/story/platform/internal/providermemory/client.go:59)、[Viking 已知限制](E:/Codex/story/platform/docs/viking-memory-integration.md:23)。
### 8 没有通话前同步水位与关键变更拦截
文档要求用户级 memory_revision、viking_visible_revision、关键积压和 EMPTY/IN_SYNC/LAGGING_NORMAL/LAGGING_CRITICAL 等状态。供应商写入成功后还要验证检索可见,且中间版本失败不能被后续成功版本跳过。
当前 NativeAccessForDevice 主要检查当前绑定和授权,再返回用户过滤配置;AddSession 成功后记录审计摘要。未看到针对主库记忆版本的连续可见进度、关键删除拦截、NATIVE_VIKING/HOT_ONLY/OFF 模式及关闭旧记忆的验证机制。Viking 获取配置失败当前可仅记录警告后继续创建,这与按关键积压判断是否允许启动不同。
证据:[创建时 Viking 处理](E:/Codex/story/platform/internal/orchestration/hardware_agent.go:46)、[原生范围配置](E:/Codex/story/platform/internal/vikingmemory/service.go:84)、[写入与审计](E:/Codex/story/platform/internal/vikingmemory/service.go:208)。
### 9 字幕可靠性及结束状态未达到新版要求
已实现最终字幕上报、事务落库、重复检测和结束后补入,但当前 turn_key/sequence_no 的同一条内容变化会冲突;没有文档要求的 utterance_id/source_revision 修订和分片结构。
固件当前使用 8 项内存消息队列,满时记日志;上报失败后释放消息,未看到对应的持久重试与 GAP_DECLARED。端侧上报把 interrupted 固定为 false,不能据此证明 AI 的整句话已播放。
后端 session_ended 会直接完成业务状态并排入抽取,没有独立 DRAINING/COMPLETE/PARTIAL 状态及文档的 15/60 秒收齐窗口。固件退出时先设置 is_stop,但播放器停止在 SDK stop 和网络结束上报之后,未实现文档要求的先立即清空待播内容、分别报告停止证据的顺序。
证据:[字幕结构与冲突规则](E:/Codex/story/platform/internal/orchestration/transcript_batch.go:24)、[端侧上报与失败处理](E:/Codex/story/platform/firmware/echoear-platform-agent/main/platform_client.c:713)、[队列及满队列处理](E:/Codex/story/platform/firmware/echoear-platform-agent/components/application/service/conv_service/src/volc_conv.c:330)、[结束与抽取](E:/Codex/story/platform/internal/orchestration/hardware_agent.go:214)、[退出顺序](E:/Codex/story/platform/firmware/echoear-platform-agent/components/application/service/conv_service/src/volc_conv.c:483)。
### 10 语音记忆变更确认与事项进度需要补充
当前人工记忆编辑能力不等同于文档第 8 章:尚未找到从用户最终文字生成 intent/mutation、来源动作去重、NEEDS_CONFIRMATION、resolve/confirm 冻结方案、六阶段处理进度、PLAY_LOCAL_NOTICE 的整套实现。
现有 open_topic 是摘要中的续聊线索;没有文档的稳定 OPEN_ITEM 标识、OPEN/COMPLETED/CANCELLED/UNKNOWN 及计划修订状态机。因此“第一通提计划、第二通报告完成、第三通不再追问”的结果尚不能由当前数据流程保证。
证据:[现有记忆页面 API](E:/Codex/story/platform/internal/httpapi/router.go:267)、[现有摘要与 open_topic](E:/Codex/story/platform/internal/extraction/processor.go:284)、[续聊线索选择](E:/Codex/story/platform/internal/orchestration/companion_context.go:70)。
## 设备接口对应关系
下表只用于说明差异,**不是 CALL-MEMORY-1.1 已实现接口文档**。文档七个接口均为 POST,统一前缀 `/api/v1/device/calls/`。当前对应功能不代表字段、鉴权、幂等和状态语义兼容。
| 文档接口 | 当前近似入口或调用 | 主要缺口 |
|---|---|---|
| CA01 `warmup` | `GET /device/v2/bootstrap`、`GET /device/v2/config` | 专门 warmup、绑定代数和 provider_profile 准备状态 |
| CA02 `prepare` | `POST /device/v3/agent-sessions` | 摘机唯一键、签名清单、记忆进度校验、取消先到处理 |
| CA03 `events` | `POST /device/v3/agent-sessions/{id}/events` 及 `/{id}/turns:batch` | 统一事件批次、逐条 ACK、连续序号、字幕修订/缺口/播放事件 |
| CA04 `heartbeat` | EchoEar 发送 `session_updated` 到 v3 events;后端另有 v2 conversations heartbeat | 清单后立即心跳、manifest/media/privacy 绑定、绝对租约、本地到期关闭 |
| CA05 `end` | EchoEar 发送 `session_ended/session_failed`;后端另有 `POST /device/v2/conversations/{id}/stop` | 无 SID 取消、原摘机标识、local_audio_stopped、独立 sdk_stop_status/remote_state |
| CA06 `status` | `GET /device/v2/conversations/{id}` | boot/hook 查询、清单状态、字幕状态、隐私版本及命令 |
| CA07 `command-acks` | `POST /device/v2/commands/{id}/ack` | 当前通话代数、MANIFEST_DISCARDED/LOCAL_AUDIO_OFF/SDK_STOPPED 与提示回执 |
当前已注册入口见 [router.go](E:/Codex/story/platform/internal/httpapi/router.go:308)。EchoEar 真实调用需以 [platform_client.c](E:/Codex/story/platform/firmware/echoear-platform-agent/main/platform_client.c:611) 为准,不能把后端存在的 v2 能力自动算成当前固件已使用。
## 建议的对齐顺序
1. **先冻结设备契约与真实供应商组合**:明确按文档 RTC 路线升级;核验动态上下文、原生检索、关闭路径、字幕归属和用户范围授权。文档附件 H 本身也把这些列为待实测能力,不能因代码中存在 MemoryConfig 字段就判定通过。
2. **补齐启动与撤销协议**:设备 HMAC 契约、warmup/prepare、摘机幂等与取消、Session Manifest、双代数、媒体尝试、首次心跳、本地租约、STOP_CALL 和停止证明。
3. **统一记忆主库和 Viking 投影**:以 SaaS 主记录为维护入口,补逐条映射、版本、关键积压、可见性、删除防复活及旧会话撤销;解决当前独立 AddSession 抽取与人工维护脱节。
4. **完善记录和管理交互**:字幕修订与持久重试、收齐状态、变更确认及阶段进度、固定提示能力、OPEN_ITEM 与开场候选失效。
5. **再形成正式设备接口交付件**:同步 OpenAPI、设备请求/响应、签名测试向量、重试/错误规则、事件/命令回执和真实板卡联调结果。
优先回归文档 T04~T07、T12~T14、T20~T31、T39~T48;供应商验证重点为 H02/H03/H05/H06/H12。此处是建议验收范围,不表示本次已执行或通过。
## 核对方法和边界
- 完整读取了 Word 正文、表格及附件 A~L,并查看了全部 10 张内嵌流程图。
- 使用实际路由、结构体、调用顺序和持久化代码验证主要差异,并检索相关后端、迁移、固件及前端是否存在新版核心协议字段。
- 现有运行记录作为历史证据;其中明确保留了真机重启后实际回忆等待验证项。本次没有把旧记录扩大为新版本验收结论。
- 当前环境的文档渲染器缺少 `soffice.exe`,未取得可核验页码,因此文档依据采用章节/附件定位;未改写或重新导出用户的 Word 文件。
- 本次只新增这份分析,不修改业务代码;未运行生产调用、设备刷写或发布操作。已有工作区修改保留。
+28 -3
View File
@@ -17,11 +17,17 @@
| 故事修订 | 候选故事、人工正文、原因、强ETag | 新增人工修订版本并复制来源关系,不覆盖模型版本;终态故事禁止再改 | 新的`story_revision`和`IN_REVIEW`状态 | | 故事修订 | 候选故事、人工正文、原因、强ETag | 新增人工修订版本并复制来源关系,不覆盖模型版本;终态故事禁止再改 | 新的`story_revision`和`IN_REVIEW`状态 |
| 故事确认 | 当前故事、审核原因、强ETag | 必须有原话来源、具备审核权限且个性化记忆授权当前有效;确认故事及候选事实 | `CONFIRMED`故事和`PENDING`记忆 | | 故事确认 | 当前故事、审核原因、强ETag | 必须有原话来源、具备审核权限且个性化记忆授权当前有效;确认故事及候选事实 | `CONFIRMED`故事和`PENDING`记忆 |
| 故事拒绝 | 当前故事、审核原因、强ETag | 拒绝故事及仍为候选状态的事实,不创建记忆 | `REJECTED`故事 | | 故事拒绝 | 当前故事、审核原因、强ETag | 拒绝故事及仍为候选状态的事实,不创建记忆 | `REJECTED`故事 |
| 事实候选细粒度审核 | 单条事实、修订内容、敏感级别、问题类型、原因、强ETag | 事实可独立编辑、确认、拒绝或停用;每次变化保存不可覆盖版本,不要求整篇故事一起处理 | 新的`fact_candidate_revision`与质量反馈 |
| 记忆索引 | 待索引记忆版本 | 再次校验记忆为当前版本、状态有效且授权仍有效;不满足条件时清空向量并标记`STALE` | `ACTIVE`或`STALE`索引 | | 记忆索引 | 待索引记忆版本 | 再次校验记忆为当前版本、状态有效且授权仍有效;不满足条件时清空向量并标记`STALE` | `ACTIVE`或`STALE`索引 |
| 生产向量化 | 普通敏感级别的当前记忆正文或新通话检索语句 | API与Worker使用同一方舟Embedding模型;请求1536维float向量并校验维度和有限数值;网络、超时、408/425/429/5xx可靠重试 | `ark:<model-id>`索引或固定边界查询向量 | | 生产向量化 | 普通敏感级别的当前记忆正文或新通话检索语句 | API与Worker使用同一方舟Embedding模型;请求1536维float向量并校验维度和有限数值;网络、超时、408/425/429/5xx可靠重试 | `ark:<model-id>`索引或固定边界查询向量 |
| 记忆修订与停用 | 当前记忆、正文或状态、原因、强ETag | 正文变化创建新版本并立即使旧索引失效;停用后不再召回;重新启用必须重新校验授权并重新索引;`PENDING_DELETE/SUPERSEDED/DELETED`均禁止修改 | 新记忆版本或状态 | | 记忆修订与停用 | 当前记忆、正文或状态、原因、强ETag | 正文变化创建新版本并立即使旧索引失效;停用后不再召回;重新启用必须重新校验授权并重新索引;`PENDING_DELETE/SUPERSEDED/DELETED`均禁止修改 | 新记忆版本或状态 |
| 记忆永久删除 | 当前记忆、删除原因、强ETag | 事务内先进入`PENDING_DELETE`并清空活动向量,使新通话立即停止召回;该状态不可通过修改接口恢复;Outbox后台把索引标记删除、清除所有记忆版本正文并记录完成审计 | `DELETED`记忆和索引 | | 记忆永久删除 | 当前记忆、删除原因、强ETag | 事务内先进入`PENDING_DELETE`并清空活动向量,使新通话立即停止召回;该状态不可通过修改接口恢复;Outbox后台把索引标记删除、清除所有记忆版本正文并记录完成审计 | `DELETED`记忆和索引 |
| 新通话记忆检索 | 服务端已确定的客户组织、长者和通话目标 | SQL同时限制租户、长者、当前授权、活动记忆、当前版本、普通敏感级别、有效期和活动索引;最多5条 | 冻结到`context_snapshot.memory_versions`的版本引用和最小背景摘要 | | 新通话记忆检索 | 服务端已确定的客户组织、长者和通话目标 | SQL同时限制租户、长者、当前授权、活动记忆、当前版本、普通敏感级别、有效期和活动索引;最多5条 | 冻结到`context_snapshot.memory_versions`的版本引用和最小背景摘要 |
| 通话中记忆工具 | 供应商持有的单通话短期签名令牌、自然语言查询 | 令牌只绑定服务端既有通话和RTC任务或硬件智能体会话;租户、家庭和长者边界从数据库反查,模型不能传入或扩大;单条240字、总计800字、最多5条;Embedding失败返回空结果并标记降级,不中断通话 | `/internal/v1/tools/memory-search`、`memory_tool_call`和实际使用追踪 |
| 可选供应商原生记忆 | 当前人工确认记忆、有效授权、普通敏感级别、受控HTTP适配器 | 数据库始终是主数据;开启后用Outbox同步,修订先清理旧引用,停用、授权撤回、单条删除和主体删除均可靠删除并对账;关闭功能后通话继续走数据库检索 | `provider`索引的`PENDING/ACTIVE/FAILED/DELETED`状态 |
| 内容质量反馈 | 故事、事实或记忆的人工确认、修改、拒绝、停用及九类标准问题 | 不复制私密正文;按生成模型和提示词版本聚合候选数、审核动作和问题类型;跨租户统一拒绝 | 维护端“内容质量”与`content_review_signal` |
| 跨通话连续上下文 | 最近已完成通话同一最新生成版本的就绪摘要与开放话题、当前授权快照、交流偏好 | 只有`story_extraction`和`personalized_memory`在新通话冻结快照中同时有效时,才按同租户同长者读取最近3次内容;最多选一个最新的非空开放话题作为低优先级建议,摘要保留约170字、已确认记忆每条保留约90字,总计不超过1000字;撤回任一授权后不再注入 | 冻结到新通话`context_snapshot.sanitized_summary`的短期连续上下文 |
| 避免主动提及 | 长者档案`conversation_preferences.avoided_topics` | 最多8项、每项50字;从下一次新通话起进入冻结快照;仅限制智能体主动开启话题,长者主动提及时温和转向,紧急安全响应不受限制 | 新通话的交流边界 |
| 授权撤回 | `personalized_memory`授权 | 在同一事务内停用该长者的活动记忆,清空活动/待处理索引向量并标记失效 | `DISABLED`记忆、`STALE`索引 | | 授权撤回 | `personalized_memory`授权 | 在同一事务内停用该长者的活动记忆,清空活动/待处理索引向量并标记失效 | `DISABLED`记忆、`STALE`索引 |
## 3. 业务流程 ## 3. 业务流程
@@ -51,6 +57,10 @@ flowchart TD
S --> U[创建下一次通话] S --> U[创建下一次通话]
U --> V[数据库固定边界检索] U --> V[数据库固定边界检索]
V --> W[冻结记忆版本和背景摘要] V --> W[冻结记忆版本和背景摘要]
S --> X{供应商记忆功能已验证并开启?}
X -- 是 --> Y[可靠同步外部引用]
X -- 否 --> Z[只使用数据库检索]
Y --> AA[停用/撤回/删除时外部删除对账]
``` ```
供应商停止事件和最终转写事件可能存在先后竞争。停止事件会发布内容整理待办;如果话轮尚未到达,处理器返回可重试错误,由Outbox退避重试,避免把“暂无话轮”错误当成永久空内容。 供应商停止事件和最终转写事件可能存在先后竞争。停止事件会发布内容整理待办;如果话轮尚未到达,处理器返回可重试错误,由Outbox退避重试,避免把“暂无话轮”错误当成永久空内容。
@@ -59,6 +69,10 @@ flowchart TD
记忆索引也先在无事务状态读取当前版本和授权,事务外生成Embedding,返回后才锁定记忆与索引行重新校验;修订、停用或撤回授权发生在模型调用期间时,旧结果不会被激活。新通话创建前仅在设备绑定、语音授权和个性化记忆授权均可用时生成查询向量;事务内仍以固定租户、长者、授权、状态和敏感级别条件检索。Embedding临时不可用时,本次通话以`pgvector-degraded-v1`记录降级且不注入记忆,不阻断基本语音通话。普通顺序幂等重放和同键正文冲突会先读取已完成幂等记录,不重复调用Embedding;并发首请求仍由数据库唯一键保证只创建一个通话。 记忆索引也先在无事务状态读取当前版本和授权,事务外生成Embedding,返回后才锁定记忆与索引行重新校验;修订、停用或撤回授权发生在模型调用期间时,旧结果不会被激活。新通话创建前仅在设备绑定、语音授权和个性化记忆授权均可用时生成查询向量;事务内仍以固定租户、长者、授权、状态和敏感级别条件检索。Embedding临时不可用时,本次通话以`pgvector-degraded-v1`记录降级且不注入记忆,不阻断基本语音通话。普通顺序幂等重放和同键正文冲突会先读取已完成幂等记录,不重复调用Embedding;并发首请求仍由数据库唯一键保证只创建一个通话。
通话中工具令牌由平台在启动智能体时签发,令牌只包含通话编号、固定供应商任务编号和过期时间。工具接口先重新读取权威通话、任务、租户和长者关系,再执行与通话前召回相同的数据库边界;请求正文没有租户、家庭或长者字段。查询摘要、结果数量、延迟和降级状态进入最小审计,原始查询和记忆正文不进入普通日志。供应商原生记忆是可选增强,不取代该工具或通话前数据库召回;外部服务失败时,本地主数据和本地召回仍可继续工作。
最近通话摘要是短期连续上下文,不会自动升级为长期记忆,也不会绕过故事人工审核。平台在每次开房时重新检查并冻结当前授权,只读取同一客户组织、同一长者、已完成通话的最新就绪摘要;隐私删除或授权撤回后的摘要不会继续进入后续通话。纯端到端豆包模式把这份冻结上下文写入`S2SConfig.ProviderParams.dialog.system_role`,级联模式继续写入`LLMConfig.SystemMessages`,两种链路使用同一上下文快照。
## 4. 接口设计 ## 4. 接口设计
| 方法与路径 | 权限 | 并发/校验 | 说明 | | 方法与路径 | 权限 | 并发/校验 | 说明 |
@@ -70,10 +84,13 @@ flowchart TD
| `PATCH /api/v1/stories/{id}` | `story.review` | 必须`If-Match`;正文和原因必填 | 创建人工修订版本 | | `PATCH /api/v1/stories/{id}` | `story.review` | 必须`If-Match`;正文和原因必填 | 创建人工修订版本 |
| `POST /api/v1/stories/{id}/confirm` | `story.review` | 必须`If-Match`;来源与授权门禁 | 确认故事并创建记忆 | | `POST /api/v1/stories/{id}/confirm` | `story.review` | 必须`If-Match`;来源与授权门禁 | 确认故事并创建记忆 |
| `POST /api/v1/stories/{id}/reject` | `story.review` | 必须`If-Match`;原因必填 | 拒绝候选 | | `POST /api/v1/stories/{id}/reject` | `story.review` | 必须`If-Match`;原因必填 | 拒绝候选 |
| `PATCH /api/v1/stories/{story_id}/facts/{fact_id}` | `story.review` | 必须`If-Match`;单条事实状态机、原因及可选标准问题类型 | 独立编辑、确认、拒绝或停用事实 |
| `GET /api/v1/subjects/{id}/memories` | `subject.read` | 过滤已删除记录;可用`q`在固定长者边界内搜索 | 已确认记忆列表 | | `GET /api/v1/subjects/{id}/memories` | `subject.read` | 过滤已删除记录;可用`q`在固定长者边界内搜索 | 已确认记忆列表 |
| `GET /api/v1/memories/{id}` | `subject.read` | 返回强ETag | 当前记忆版本和索引状态 | | `GET /api/v1/memories/{id}` | `subject.read` | 返回强ETag | 当前记忆版本和索引状态 |
| `PATCH /api/v1/memories/{id}` | `story.review` | 必须`If-Match`;正文/状态至少一项且原因必填;删除中及历史终态拒绝修改 | 修订、停用或重新启用记忆 | | `PATCH /api/v1/memories/{id}` | `story.review` | 必须`If-Match`;正文/状态至少一项且原因必填;删除中及历史终态拒绝修改 | 修订、停用或重新启用记忆 |
| `DELETE /api/v1/memories/{id}` | `story.review` | 必须`If-Match`;删除原因必填;返回202 | 可靠受理永久删除任务 | | `DELETE /api/v1/memories/{id}` | `story.review` | 必须`If-Match`;删除原因必填;返回202 | 可靠受理永久删除任务 |
| `POST /internal/v1/tools/memory-search` | 单通话短期Bearer令牌 | 固定通话/任务/租户/长者边界;限流、长度和结果上限 | 通话中受控记忆检索,失败安全降级 |
| `GET /api/v1/ops/content-quality` | `ops_viewer`及以上 | 必须选择有权客户组织;不返回正文 | 按模型和提示词版本聚合审核反馈 |
统一冲突码包括`STORY_SOURCE_REQUIRED`、`PERSONALIZED_MEMORY_CONSENT_REQUIRED`和`STORY_REVIEW_STATE_CONFLICT`。跨租户或非家庭成员访问统一返回403,不借404差异泄露资源是否存在。 统一冲突码包括`STORY_SOURCE_REQUIRED`、`PERSONALIZED_MEMORY_CONSENT_REQUIRED`和`STORY_REVIEW_STATE_CONFLICT`。跨租户或非家庭成员访问统一返回403,不借404差异泄露资源是否存在。
@@ -87,11 +104,15 @@ flowchart TD
| `story_revision` | 不可覆盖的故事版本 | 故事内修订号唯一;区分模型/人工作者;保存基础版本和原因 | | `story_revision` | 不可覆盖的故事版本 | 故事内修订号唯一;区分模型/人工作者;保存基础版本和原因 |
| `story_source` | 故事版本到原话的追溯关系 | 版本、话轮和文本偏移组合唯一 | | `story_source` | 故事版本到原话的追溯关系 | 版本、话轮和文本偏移组合唯一 |
| `fact_candidate` | 可审核事实候选 | 区分长者陈述与模型推断;保存源话轮、敏感级别和状态 | | `fact_candidate` | 可审核事实候选 | 区分长者陈述与模型推断;保存源话轮、敏感级别和状态 |
| `fact_candidate_revision` | 事实不可覆盖版本 | 每条事实版本号唯一;保留内容、状态、原因和标准问题类型 |
| `curated_memory` | 长期记忆聚合根 | 当前版本、状态、有效期、敏感级别和强版本号 | | `curated_memory` | 长期记忆聚合根 | 当前版本、状态、有效期、敏感级别和强版本号 |
| `memory_version` | 不可覆盖的记忆版本 | 记忆内版本号唯一;确认人、确认时间、原因、来源和内容摘要 | | `memory_version` | 不可覆盖的记忆版本 | 记忆内版本号唯一;确认人、确认时间、原因、来源和内容摘要 |
| `memory_index_entry` | 记忆检索索引状态 | 记忆版本和索引类型唯一;向量维度1536;状态可失效和删除 | | `memory_index_entry` | 记忆检索索引状态 | 记忆版本和索引类型唯一;向量维度1536;状态可失效和删除 |
| `conversation_memory_use` | 实际记忆使用追踪 | 按通话、版本和使用方式唯一,不复制正文 |
| `memory_tool_call` | 通话中工具可观测性 | 只保存查询摘要、数量、状态、延迟和引擎模式 |
| `content_review_signal` | 内容质量反馈 | 按租户和长者隔离;保存资源、动作、问题类型、模型和提示词版本,不复制正文 |
V8迁移已经在独立空数据库执行`up → down -all → up`,最终为`8|dirty=false`,九张内容与记忆表均存在。测试数据库随后已删除。 内容与记忆数据库结构目前覆盖到V41。迁移在独立临时数据库执行`up → down 1 → up`并核对最终版本与`dirty=false`,测试数据库随后删除,不对开发或生产数据库做回滚验证。
## 6. 安全、隐私与可观测性 ## 6. 安全、隐私与可观测性
@@ -102,17 +123,21 @@ V8迁移已经在独立空数据库执行`up → down -all → up`,最终为`8
- 只有冻结快照允许保存字幕时才落`conversation_turn`;故事整理和个性化记忆分别使用独立授权门禁。 - 只有冻结快照允许保存字幕时才落`conversation_turn`;故事整理和个性化记忆分别使用独立授权门禁。
- 故事、事实和记忆均保留租户、长者和来源边界;所有网页接口从OIDC主体重新解析家庭成员关系,不信任客户端传入租户。 - 故事、事实和记忆均保留租户、长者和来源边界;所有网页接口从OIDC主体重新解析家庭成员关系,不信任客户端传入租户。
- 人工修订、确认、拒绝和授权撤回记录审计事件;写操作使用强ETag防止并发覆盖。 - 人工修订、确认、拒绝和授权撤回记录审计事件;写操作使用强ETag防止并发覆盖。
- 通话创建已接入内部数据库召回并冻结实际使用的记忆版本;调用方不能传入自由租户或长者过滤条件。当前尚未开放通话中工具检索接口。 - 通话创建和通话中工具均使用服务端固定的租户、长者、授权和版本边界;供应商或模型不能传入自由租户、家庭或长者过滤条件。
- 供应商原生记忆默认关闭,只有完成真实账号的数据条款、隔离、同步、删除和容量验证后才配置`PROVIDER_MEMORY_MODE=managed_http`;数据库仍是唯一主数据。
## 7. 验证结果 ## 7. 验证结果
- Go全仓测试通过,OpenAPI 0.10.0可生成TypeScript类型,React生产构建通过。 - 新增迁移已在隔离数据库完成升库与数据库组合测试;OpenAPI 0.70.0可生成TypeScript类型,React类型检查和生产构建通过。
- 本地内容策略单元测试覆盖验证码、银行卡号、高敏感医疗词和普通生活经历。 - 本地内容策略单元测试覆盖验证码、银行卡号、高敏感医疗词和普通生活经历。
- 方舟兼容适配器轻量验证覆盖Bearer鉴权、请求路径、结构化响应校验、HTTP 429重试分类;真实PostgreSQL组合验证模型调用期间不持有通话行锁,模型返回后正常生成候选,敏感话轮不会调用外部模型。 - 方舟兼容适配器轻量验证覆盖Bearer鉴权、请求路径、结构化响应校验、HTTP 429重试分类;真实PostgreSQL组合验证模型调用期间不持有通话行锁,模型返回后正常生成候选,敏感话轮不会调用外部模型。
- 方舟Embedding适配器轻量验证覆盖Bearer鉴权、`/embeddings`请求、模型/维度参数、1536维结果和错误维度拒绝;真实PostgreSQL组合验证向量生成期间不持有索引行锁,返回后可激活索引。 - 方舟Embedding适配器轻量验证覆盖Bearer鉴权、`/embeddings`请求、模型/维度参数、1536维结果和错误维度拒绝;真实PostgreSQL组合验证向量生成期间不持有索引行锁,返回后可激活索引。
- 端到端通话冒烟验证最终话轮去重、两类整理结果、故事来源、人工确认前零记忆、确认后记忆生成及索引生效;下一次通话冻结1个记忆版本。 - 端到端通话冒烟验证最终话轮去重、两类整理结果、故事来源、人工确认前零记忆、确认后记忆生成及索引生效;下一次通话冻结1个记忆版本。
- 记忆修订生成第2版并重新索引;停用后索引立即失效且新通话召回0条;重新启用再次进入索引流程。 - 记忆修订生成第2版并重新索引;停用后索引立即失效且新通话召回0条;重新启用再次进入索引流程。
- 永久删除验证覆盖`PENDING_DELETE → DELETED`、删除中重新激活稳定拒绝、向量清理、数据库正文替换为删除标记和删除后新通话召回0条。 - 永久删除验证覆盖`PENDING_DELETE → DELETED`、删除中重新激活稳定拒绝、向量清理、数据库正文替换为删除标记和删除后新通话召回0条。
- 通话中记忆工具验证覆盖短期令牌签名、固定通话/任务边界、跨长者不可选、结果长度上限、实际使用追踪和Embedding异常降级。
- 供应商记忆适配器验证覆盖Bearer鉴权、幂等同步、旧版本删除、单条删除、主体隐私删除前对账和失败状态保留;当前只使用模拟HTTP/内存适配器,真实供应商账号仍待外部验收。
- 内容质量反馈验证覆盖九类问题、事实不可覆盖版本、按模型/提示词聚合和跨租户拒绝;维护端只展示汇总,不显示故事、事实或记忆正文。
- React故事与记忆页以页面内管理区提供正文修订、启停和永久删除;未修改正文、空原因、未确认删除均不能提交,删除中和历史终态只读,不再依赖浏览器弹窗。 - React故事与记忆页以页面内管理区提供正文修订、启停和永久删除;未修改正文、空原因、未确认删除均不能提交,删除中和历史终态只读,不再依赖浏览器弹窗。
- OIDC、SaaS、设备、成员和通话五组冒烟全部通过;测试结束后内容表残留为0,Outbox无待处理、处理中或死信事件。 - OIDC、SaaS、设备、成员和通话五组冒烟全部通过;测试结束后内容表残留为0,Outbox无待处理、处理中或死信事件。
- 当前方舟内容与Embedding协议均使用本地模拟端点验证,尚未使用业务方真实API Key核验模型效果、召回质量、额度和数据条款;真实语音供应商与真实设备也仍待验收。 - 当前方舟内容与Embedding协议均使用本地模拟端点验证,尚未使用业务方真实API Key核验模型效果、召回质量、额度和数据条款;真实语音供应商与真实设备也仍待验收。
+31
View File
@@ -0,0 +1,31 @@
# 聊天总结模型厂商接入
聊天总结任务不绑定某个模型。管理员在“大模型管理”中选择厂商、填写对应 API Key 与模型 ID、完成连接测试并激活后,Worker 会在下一条任务执行时读取当前激活配置。
## 已适配厂商
| 页面分类 | `provider` | 默认 Base URL | 需要填写 |
|---|---|---|---|
| 火山方舟 / 豆包 | `ark` | `https://ark.cn-beijing.volces.com/api/v3` | API Key、模型 ID 或 Endpoint ID |
| 阿里云百炼 / 千问 | `dashscope` | `https://dashscope.aliyuncs.com/compatible-mode/v1` | 同地域 API Key、模型 ID |
| DeepSeek | `deepseek` | `https://api.deepseek.com` | API Key、模型 ID |
| 月之暗面 / Kimi | `moonshot` | `https://api.moonshot.cn/v1` | API Key、模型 ID |
| 智谱 AI / GLM | `zhipu` | `https://open.bigmodel.cn/api/paas/v4` | API Key、模型 ID |
| 百度千帆 / 文心 | `qianfan` | `https://qianfan.baidubce.com/v2` | Bearer Token、模型 ID |
| 腾讯混元 | `hunyuan` | `https://api.hunyuan.cloud.tencent.com/v1` | API Key、模型 ID |
| MiniMax | `minimax` | `https://api.minimaxi.com/v1` | API Key、模型 ID |
| 百川智能 | `baichuan` | `https://api.baichuan-ai.com/v1` | API Key、模型 ID |
| 硅基流动 | `siliconflow` | `https://api.siliconflow.cn/v1` | API Key、模型 ID |
| 本地 Ollama | `ollama` | `http://ollama:11434/v1` | 本地模型 ID;密钥使用占位值 |
| 自定义兼容服务 | `openai_compatible` | 管理员填写 | HTTPS Base URL、API Key、模型 ID |
所有云服务使用 Bearer Token 鉴权和 `/chat/completions` 请求。适配器只对官方已明确支持或项目已验证的厂商发送 `response_format`;其他厂商通过严格提示、JSON 提取和响应校验取得结构化结果,避免某些兼容接口因不认识 JSON Mode 参数而拒绝请求。带 `<think>` 思考包装的返回也会在解析前清理。
## 使用流程
1. 选择厂商,页面会预填官方 Base URL 和常用模型名。
2. 按厂商控制台填写 API Key 与准确模型 ID。使用地域或业务空间专属域名时,可修改 Base URL。
3. 保存后执行“测试连接”。测试内容是固定的无长者数据文本。
4. 测试通过后设为当前模型。下一条聊天总结任务立即使用,不需要重启 Worker。
API Key 通过 AES-256-GCM 加密保存,读取接口仅返回末四位提示。连接测试、激活与移除均写入审计日志。
+27
View File
@@ -0,0 +1,27 @@
# 用户端手机适配
适配范围为 `CustomerPortal` 的全部 11 个页面:首页、聊天记录、故事与记忆、长者资料、提醒与待办、家庭管理、陪伴设备、内容分享、数据与授权、服务用量、设置与权限,以及这些页面的资料编辑、邀请、设备配置、审核和授权弹窗。
- 900px 及以下使用抽屉导航,支持搜索、遮罩关闭、Escape、关闭后恢复焦点和窗口尺寸变化。桌面仍使用侧栏。
- 600px 及以下将表单、列表操作和详情改为单列,长名称和资源编号可换行。家庭切换、年度活动日历与权限表在各自区域内横向滚动,手机日历默认定位到最近日期。
- 聊天采用列表与摘要上下排列,选中后定位摘要,可返回聊天列表;手机上的“更多筛选”保留语音方式和起止日期筛选。
- 表单控件使用适合触屏的尺寸,手机输入文字至少为 16px;年度日历日期格为 24px。弹窗使用动态视口高度和安全区域边距,长表单可以滚动到底部。
- 手机覆盖规则集中在 `web/src/customer-responsive.css`,在现有主题之后加载,限定于用户端。
## 回归检查
在 `web` 目录执行 `npm run dev -- --host 127.0.0.1`。在项目根目录使用已安装的 Playwright 运行:
```powershell
$env:PLAYWRIGHT_REQUIRE_FROM = '包含 Playwright 依赖的目录/package.json'
$env:VISUAL_WEB_URL = 'http://127.0.0.1:5173'
node test/e2e/visual-customer-mobile.mjs
```
默认使用本机 Google Chrome;可通过 `CHROME_PATH` 指定浏览器可执行文件。未安装 Playwright 时,可先用 `npm install --prefix .tmp-mobile-qa --no-audit --no-fund playwright` 安装到忽略的临时目录,再把 `PLAYWRIGHT_REQUIRE_FROM` 设置为该目录下 `package.json` 的绝对路径。
脚本覆盖 320、375、390、430、768、900、1024、1440px 的所有页面,以及横屏、登录、空数据、加载失败、短视口表单、导航交互和主要操作弹窗。`MOBILE_WIDTHS` 可指定逗号分隔的宽度;`MOBILE_INTERACTIONS_ONLY=1` 只执行对应宽度的交互检查。截图和检查结果保存在 `.design-evidence/customer-mobile/`。
测试使用合成家庭数据拦截全部业务 API,非 GET 请求会直接失败,不依赖后端或修改业务数据。该检查验证布局和前端交互,不代替真实身份认证、后端业务验收或 iPhone Safari / Android 真机软键盘测试。
构建检查:在 `web` 目录执行 `npm run build`。
+50
View File
@@ -0,0 +1,50 @@
# 安心陪伴客户门户 v0.74 页面功能与设计基准
日期:2026-08-30
设计方向:沿用已确认的“回忆时间轴”方案,以长者、聊天、心情与家庭协作为核心;暖白纸张底色、雾蓝主操作、陶土色提醒、低阴影和清晰留白。
## 全局信息架构
客户门户只保留家庭用户真正需要的 12 个一级任务。当前家庭用页首标签直接切换;家庭成员、角色和邀请恢复为独立的“家庭管理”页面。客户门户不再出现“客户组织”概念。
全局页面结构固定为:
1. 68px 顶部栏:品牌、全局搜索、提醒、账号。
2. 228px 左侧导航:围绕长者陪伴任务排列。
3. 统一页面首部:标题、用途说明、家庭权限状态。
4. 统一家庭上下文栏:家庭 Tab、添加家庭、实时状态、刷新和页面主操作。
5. 页面内容:不再使用包住整页的大卡片;只在独立对象或需要分组时使用边框容器。
## 逐页功能总结
| 页面 | 用户要解决的问题 | 主要功能 | 首屏重点 | 关键状态 |
|---|---|---|---|---|
| 首页 | 最近有没有陪伴、长者心情如何、哪里需要家人关注 | 长者切换、7 天聊天时长、最近聊天摘要、心情线索、关注提醒、最近动态 | 每位长者最近一次聊天和心情变化 | 无聊天、设备离线、待关注 |
| 聊天记录 | 找到某次聊天并理解聊天内容 | 按长者/日期/状态筛选、AI 摘要、关键词、故事候选、下次话题、录音授权 | 时间轴和当前聊天 AI 总结 | 总结中、总结失败、无授权 |
| 故事与记忆 | 审核模型整理的故事,形成可信长期记忆 | 故事候选审核、修订、确认/拒绝、记忆检索和停用 | 待审核故事与来源聊天 | 空状态、审核中、已确认 |
| 长者资料 | 让豆包知道如何更适合地与这位长者聊天 | 头像、背景、兴趣、优先/回避话题、交流节奏、长期陪伴方向、授权 | 长者卡片和资料完整度 | 头像上传、只读、资料不完整 |
| 日程与提醒 | 知道什么时候聊、哪些事情需要家人处理 | 陪伴计划、提醒时间、设备/故事/隐私待办、事件状态 | 今日安排和待办 | 已完成、即将开始、已错过 |
| 家庭管理 | 管理家庭空间和可以协作的家人 | 家庭 Tab、成员、邀请、角色、所有权、家庭名称 | 家庭成员与角色 | 待接受邀请、只读、危险操作确认 |
| 陪伴设备 | 确认设备能否正常陪伴 | 绑定、在线状态、固件、电量/音量、诊断、解除绑定 | 设备健康和所属长者 | 离线、待同步、诊断中 |
| 内容分享 | 把指定故事/文字/录音安全分享给指定家人 | 资源选择、对象、权限、有效期、撤销 | 当前有效分享 | 无分享、即将过期、已撤销 |
| 数据与授权 | 查看和控制长者数据如何被使用 | 五类分项授权、数据访问/更正/撤回/删除、私有导出 | 授权清单,不以按钮堆叠呈现 | 生效、处理中、已撤回 |
| 服务用量 | 理解聊天使用情况,而不是看无意义设备统计 | 时长指标、活跃日、长者/设备筛选、热力图、最近通话 | 最近一年聊天活动 | 无数据、费用未核价 |
| 大模型管理 | 选择哪个模型用于聊天总结 | 新增配置、连通性测试、激活、停用、参数和生成历史 | 当前激活模型及健康状态 | 未配置、测试失败、已激活 |
| 设置与权限 | 理解当前账号能做什么 | 账号资料、家庭角色、15 项权限矩阵、敏感录音独立授权说明 | 当前身份和权限全表 | 无家庭、权限不足 |
## 统一视觉规格
- 桌面基准:1440 × 1024;内容最大宽度 1360px。
- 页面左右 padding:桌面 28px;平板 20px;手机 14px。
- 页面首部上下 padding:24px / 18px;与家庭上下文栏间距 16px。
- 家庭上下文栏高度不固定,最小 54px;禁止通过空白占位制造层级。
- 页面 section 间距 16px;对象卡片 padding 20–24px;列表行 14–16px。
- 字号:页面标题 28px;区块标题 20px;正文 14px;辅助信息 12px。
- 圆角:页面对象 14px、按钮/输入框 9px、状态胶囊 999px。
- 主色 `#315D9B`;强调 `#C45F3B`;成功 `#5F8F69`;底色 `#FBFAF7`;边线 `#E6E0D8`。
- 禁用按钮必须不可点击且鼠标为等待光标;加载按钮显示明确进行中动词。
- 所有日期基于 Asia/Shanghai,设计数据锚点为 2026-08-30。
## 设计验收规则
每个页面必须在 1440 × 1024 同一视口完成“设计基准截图 / 实际页面截图”对照。P0–P2 的布局、padding、信息层级、文字、色彩、状态与响应式差异必须修复;最终记录写入项目根目录 `design-qa.md`。
+2 -2
View File
@@ -437,9 +437,9 @@
## 9. 当前实施状态与后续顺序 ## 9. 当前实施状态与后续顺序
截至2026-08-25,本地工程已经完成阶段1—7的本地开发与验证闭环。1000活跃通话、50 RPS持续30分钟、500回调/秒、500条SSE、10000媒体分片、100并发删除、隔离数据库恢复、进程/Redis恢复、供应商故障、安全扫描、真实Chrome/Edge兼容、全仓检查、跨功能烟测和无测试残留对账均已完成。后续顺序为: 截至2026-08-26,本地工程已经完成阶段1—7的本地开发与验证闭环,并在阿里云单机环境部署标准HTTPS、OIDC、加密本机备份、主机健康签名上报和八项发布候选技术门禁。生产快照不能用演示部署模式、未验证的异地备份、未配置的外部告警或仍在运行的Mock服务冒充可上线。1000活跃通话、50 RPS持续30分钟、500回调/秒、500条SSE、10000媒体分片、100并发删除、隔离数据库恢复、进程/Redis恢复、供应商故障、安全扫描、真实Chrome/Edge兼容、全仓检查、跨功能烟测和无测试残留对账均已完成。后续顺序为:
1. 外部基础设施探针、运营预检台账、联调问题门禁、可归档技术验收快照、试点日/周证据和v3候选证据包均已完成;生产内容安全已具备本地前置拦截、托管HTTP策略、输入/输出双检和安全失败语义;远程CI供应链已具备绑定提交、运行、14项检查、双镜像摘要和三份SBOM的不可豁免门禁。`PILOT`周期、远程CI、真机固件、真实终端RTC六通矩阵、真实Agent生成/Embedding/内容安全质量、真实告警网关、云KMS/HSM、四类生产等价恢复、七类生产基础设施和Safari精简矩阵均不可由本地模拟报告放行。下一步在目标环境逐项替换证据索引中的`PENDING`。 1. 外部基础设施探针、运营预检台账、联调问题门禁、可归档技术验收快照、试点日/周证据和v3候选证据包均已完成;生产内容安全已具备本地前置拦截、托管HTTP策略、输入/输出双检和安全失败语义;远程CI供应链已具备绑定提交、运行、14项检查、双镜像摘要和三份SBOM的不可豁免门禁。`PILOT`周期、远程CI、真机固件、真实终端RTC六通矩阵、真实Agent生成/Embedding/内容安全质量、真实告警网关、云KMS/HSM、四类生产等价恢复、七类生产基础设施和Safari精简矩阵均不可由本地模拟报告放行。下一步在目标环境逐项替换证据索引中的`PENDING`。
2. 待真实火山RTC/豆包、大模型/Embedding账号、终端硬件和生产资源到位后,完成阶段8真机联调、家庭试点、预生产恢复与灰度发布。 2. OSS异地备份已完成无长期AK的IMDSv2/ECS RAM角色适配;待云账号侧绑定角色、创建私有Bucket后执行真实上传回读验收。随后待真实火山RTC/豆包、大模型/Embedding账号、终端硬件、外部告警接收器和其余生产资源到位,完成阶段8真机联调、家庭试点、预生产恢复与灰度发布。
实际完成证据与未完成项持续维护在[开发状态与门禁记录](development-status.md)中;本计划只定义执行顺序和退出条件,不用计划状态替代测试证据。 实际完成证据与未完成项持续维护在[开发状态与门禁记录](development-status.md)中;本计划只定义执行顺序和退出条件,不用计划状态替代测试证据。
+10 -3
View File
@@ -1,7 +1,7 @@
# 开发状态与门禁记录 # 开发状态与门禁记录
> 更新日期:2026-08-25 > 更新日期:2026-08-26
> 当前接口契约版本:0.64.0 > 当前接口契约版本:0.68.0
> 当前主里程碑:M8目标环境联调、试点与上线验收(外部条件待提供) > 当前主里程碑:M8目标环境联调、试点与上线验收(外部条件待提供)
## 阶段状态 ## 阶段状态
@@ -15,7 +15,7 @@
| 第五阶段:SaaS核心业务开发 | 本地业务闭环完成 | 已完成家庭成员邀请与角色协作、长者、稳定事实、五类分项授权、设备读取/配置/绑定、可靠命令、签名固件灰度/暂停/回滚和逐设备升级进度、通话查询、双轨录音状态/授权/播放下载的数据库、服务、HTTP契约与React工作台;服务端隔离、ETag、审计、纵向烟测及Chrome/Edge兼容检查均通过 | 固件生产签名设施、macOS Safari和类生产容量仍待外部环境验证 | | 第五阶段:SaaS核心业务开发 | 本地业务闭环完成 | 已完成家庭成员邀请与角色协作、长者、稳定事实、五类分项授权、设备读取/配置/绑定、可靠命令、签名固件灰度/暂停/回滚和逐设备升级进度、通话查询、双轨录音状态/授权/播放下载的数据库、服务、HTTP契约与React工作台;服务端隔离、ETag、审计、纵向烟测及Chrome/Edge兼容检查均通过 | 固件生产签名设施、macOS Safari和类生产容量仍待外部环境验证 |
| 第六阶段:Agent内容与记忆闭环 | 本地开发闭环完成,生产模型协议已接入 | 最终话轮、可切换本地/火山方舟内容生成器与Embedding、事务外模型调用、摘要、开放话题、故事/事实候选、人工版本审核、授权门禁、已确认记忆、索引失效、固定边界召回及React审核页面均已实现;端到端烟测通过 | 真实方舟API Key、内容效果与召回质量验收、模型评测和真实内容安全策略待完成 | | 第六阶段:Agent内容与记忆闭环 | 本地开发闭环完成,生产模型协议已接入 | 最终话轮、可切换本地/火山方舟内容生成器与Embedding、事务外模型调用、摘要、开放话题、故事/事实候选、人工版本审核、授权门禁、已确认记忆、索引失效、固定边界召回及React审核页面均已实现;端到端烟测通过 | 真实方舟API Key、内容效果与召回质量验收、模型评测和真实内容安全策略待完成 |
| 第七阶段:生产加固 | 本地证据持续收敛 | 记忆可靠删除、隐私访问/导出/删除、运营配置灰度、用量费用额度、告警处置闭环、可重试HMAC告警Webhook及不污染正式告警的接收器联通测试、三类Grafana仪表盘、证书到期监测、OIDC/方舟/RTC外部基础设施探针、本地密钥轮换、生产配置拒绝启动门禁、全部本地专项容量、隔离恢复、供应商故障、安全扫描及Chrome/Edge兼容均通过;主要业务核心包已达到70%覆盖率,供应商客户端达到关键分支85%目标 | 全仓覆盖率及HTTP公共层仍低于需求目标;真实告警网关和值班组映射、生产等价容量、云密钥托管和Safari仍待完成 | | 第七阶段:生产加固 | 本地证据持续收敛 | 记忆可靠删除、隐私访问/导出/删除、运营配置灰度、用量费用额度、告警处置闭环、可重试HMAC告警Webhook及不污染正式告警的接收器联通测试、三类Grafana仪表盘、证书到期监测、OIDC/方舟/RTC外部基础设施探针、本地密钥轮换、生产配置拒绝启动门禁、全部本地专项容量、隔离恢复、供应商故障、安全扫描及Chrome/Edge兼容均通过;主要业务核心包已达到70%覆盖率,供应商客户端达到关键分支85%目标 | 全仓覆盖率及HTTP公共层仍低于需求目标;真实告警网关和值班组映射、生产等价容量、云密钥托管和Safari仍待完成 |
| 第八阶段:联调试点与上线 | 本地准备完成,外部联调待执行 | 外部依赖预检、联调问题门禁、候选验收快照、阻断告警定位、接收器联通测试、内容安全TLS预检和按业务时区生成的试点日/周报均已形成可执行入口;本地真实OIDC、Worker、HMAC验签接收、六项技术门禁及日/周报下载通过 | 远程CI、真实投诉证据、告警网关和值班组映射、真实供应商/真机联调、家庭试点、预生产演练和生产发布 | | 第八阶段:联调试点与上线 | 本地准备完成,外部联调待执行 | 外部依赖预检、联调问题门禁、候选验收快照、阻断告警定位、接收器联通测试、内容安全TLS预检和按业务时区生成的试点日/周报均已形成可执行入口;本地真实OIDC、Worker、HMAC验签接收、八项技术门禁及日/周报下载通过;生产快照不能再用演示模式、未启用异地备份、未配置外部告警或仍运行Mock服务冒充可上线 | 远程CI、真实投诉证据、告警网关和值班组映射、真实供应商/真机联调、家庭试点、预生产演练和生产发布 |
## 本轮验证记录 ## 本轮验证记录
@@ -180,6 +180,9 @@
| 媒体清单容量 | 10批×1000分片准确落库和关联;批量SQL优化后平均0.12秒、p95/最大0.16秒,API容器内存峰值15.8MiB | | 媒体清单容量 | 10批×1000分片准确落库和关联;批量SQL优化后平均0.12秒、p95/最大0.16秒,API容器内存峰值15.8MiB |
| 隐私删除并发 | 100个并发受理全部成功、HTTP失败率0%、提交p95 385.15ms;11.58秒内600个子任务、100个墓碑和100个主体冻结全部对账 | | 隐私删除并发 | 100个并发受理全部成功、HTTP失败率0%、提交p95 385.15ms;11.58秒内600个子任务、100个墓碑和100个主体冻结全部对账 |
| PostgreSQL隔离恢复 | 实际`pg_dump`/`pg_restore`到随机隔离库,迁移`22|false`、66张表、190个外键和关键计数一致;本地约5.14秒,临时库/备份已清理 | | PostgreSQL隔离恢复 | 实际`pg_dump`/`pg_restore`到随机隔离库,迁移`22|false`、66张表、190个外键和关键计数一致;本地约5.14秒,临时库/备份已清理 |
| 阿里云单机加密备份与恢复 | 目标服务器已安装每日systemd备份任务,覆盖PostgreSQL、认证Redis RDB、MinIO全部桶和暂停后的Keycloak数据目录;归档使用AES-256-CBC/PBKDF2加密、逐文件SHA-256并保留14天。首次实际归档成功,数据库从V38备份恢复到随机隔离库后`38|false|2|3|73|215`六项证据一致,归档内Redis RDB格式有效、Keycloak 3个文件一致、临时恢复库为0;同机备份尚不作为跨故障域生产恢复证据 |
| 阿里云单机健康监测 | 目标服务器已安装五分钟systemd健康检查,结构化状态覆盖部署模式、实际应用运行环境、身份服务模式、外部告警配置、HTTPS、宿主机公网监听端口、OIDC签发地址、核心容器、模拟服务、本机/异地备份状态与摘要、备份执行结果、三个运行timer、短期IP证书剩余时间和磁盘阈值。监听策略只允许非回环TCP 22/80/443,回环数据库端口不误报,额外端口会进入critical。演示正常路径28项全部healthy;生产模式要求三个业务进程实际运行在`prod`、Keycloak不使用`start-dev`并退出模拟服务。可选Webhook仅在状态变化时发送带时间戳和HMAC-SHA256的脱敏状态;真实接收器尚未配置,因此不作为值班通知送达证据 |
| 阿里云OSS异地备份适配 | 目标服务器已按官方SHA-256安装ossutil 2.3.0,新增每日OSS同步任务、加密归档及摘要上传、远端回读复算、原子状态和健康强制门禁。ECS正式路径现使用IMDSv2和`EcsRamRole`自动短期凭据,不保存长期AK;隔离配置模式验证上传/回读2个文件并得到`verified`,目标实例缺角色时退出4并记录`failed/ecs_ram_role_missing`。2026-08-26实测地域`cn-hangzhou`且角色元数据404;当前仍无RAM角色和Bucket,正式任务保持`disabled/not_enabled`且健康状态为`not_required`,不能作为真实异地备份证据 |
| 进程与Redis恢复 | API/Callbacks/Worker重启恢复;停机期间Outbox仅执行一次;Redis 14个热状态键清空后由签名设备请求重建,PostgreSQL权威计数不变 | | 进程与Redis恢复 | API/Callbacks/Worker重启恢复;停机期间Outbox仅执行一次;Redis 14个热状态键清空后由签名设备请求重建,PostgreSQL权威计数不变 |
| 供应商故障恢复 | V23持久化按租户/供应商的可用状态;连续临时失败暂停新启动但停止任务继续尝试;演练确认暂停后新外呼0、停止尝试2次、历史接口200、CRITICAL告警出现,恢复后全部收敛;V23单版本回退/重升通过 | | 供应商故障恢复 | V23持久化按租户/供应商的可用状态;连续临时失败暂停新启动但停止任务继续尝试;演练确认暂停后新外呼0、停止尝试2次、历史接口200、CRITICAL告警出现,恢复后全部收敛;V23单版本回退/重升通过 |
| 安全专项 | Go 1.26.6容器`govulncheck`可达漏洞0、npm高危审计0、最终镜像HIGH/CRITICAL已修复漏洞0、Gitleaks无泄漏;HTTP认证/授权/注入式输入/1MiB正文/回调签名/Nonce重放/安全头/日志过滤烟测通过 | | 安全专项 | Go 1.26.6容器`govulncheck`可达漏洞0、npm高危审计0、最终镜像HIGH/CRITICAL已修复漏洞0、Gitleaks无泄漏;HTTP认证/授权/注入式输入/1MiB正文/回调签名/Nonce重放/安全头/日志过滤烟测通过 |
@@ -246,6 +249,10 @@
| 真实终端RTC/语音供应商精简矩阵与发布门禁 | 新增`real-device-rtc-capture-v1`、`real-device-rtc-conformance-report-v1`和`cmd/rtc-conformance`。真实报告以至少3台候选终端执行6通:`s2s_pure/s2s_hybrid/cascade`各正常完成1通,再覆盖凭证续签、网络切换恢复和受控停止;每通必须证明房间/用户/任务绑定、双向媒体、固定状态序列、供应商启停、最终字幕、用量、质量通过、任务去重和资源回收。协议模拟固定为`PROTOCOL_SIMULATION/external_validation_required=true`。v3证据包新增`real_device_rtc_validation`,该门禁不可豁免且首个文件必须为合格真实报告。按低强度策略执行采集器/证据包定向测试和一条协议结构工具链冒烟:1台模拟终端1通结构结果生成`PASS/PROTOCOL_SIMULATION`并继续外部待验;证据包保留真实报告、设备/通话数量、三方式和四场景共10项缺口,临时文件已清理。 | | 真实终端RTC/语音供应商精简矩阵与发布门禁 | 新增`real-device-rtc-capture-v1`、`real-device-rtc-conformance-report-v1`和`cmd/rtc-conformance`。真实报告以至少3台候选终端执行6通:`s2s_pure/s2s_hybrid/cascade`各正常完成1通,再覆盖凭证续签、网络切换恢复和受控停止;每通必须证明房间/用户/任务绑定、双向媒体、固定状态序列、供应商启停、最终字幕、用量、质量通过、任务去重和资源回收。协议模拟固定为`PROTOCOL_SIMULATION/external_validation_required=true`。v3证据包新增`real_device_rtc_validation`,该门禁不可豁免且首个文件必须为合格真实报告。按低强度策略执行采集器/证据包定向测试和一条协议结构工具链冒烟:1台模拟终端1通结构结果生成`PASS/PROTOCOL_SIMULATION`并继续外部待验;证据包保留真实报告、设备/通话数量、三方式和四场景共10项缺口,临时文件已清理。 |
| 真实Agent生成、Embedding和内容安全质量门禁 | 新增`real-agent-models-capture-v1`、`real-agent-models-conformance-report-v1`和`cmd/agent-models-conformance`。内容生成复用现有固定10类Agent评测,并使用评测集、脱敏观测和报告重新计算防篡改;Embedding按版本化阈值覆盖精确/改写召回、跨长者隔离、撤销/敏感排除和无匹配降级六类;内容安全覆盖验证码、金融账户、提示注入、敏感医疗、正常内容和模型输出六类。真实证据还必须绑定受控提供器、账号/区域、数据条款、额度和审计引用;本地生成器、哈希向量和规则固定为`LOCAL_AGENT_TOOLCHAIN/external_validation_required=true`。v3证据包新增`real_agent_models_validation`,该门禁不可豁免。按低强度策略执行聚合器/证据包定向测试和一条重算现有模拟报告的本地工具链冒烟:既有10类模拟报告重新计算一致,两组六类结构完整,输出`PASS/LOCAL_AGENT_TOOLCHAIN`并继续外部待验;证据包保留真实报告、三提供器和12类别共16项缺口,临时文件已清理。 | | 真实Agent生成、Embedding和内容安全质量门禁 | 新增`real-agent-models-capture-v1`、`real-agent-models-conformance-report-v1`和`cmd/agent-models-conformance`。内容生成复用现有固定10类Agent评测,并使用评测集、脱敏观测和报告重新计算防篡改;Embedding按版本化阈值覆盖精确/改写召回、跨长者隔离、撤销/敏感排除和无匹配降级六类;内容安全覆盖验证码、金融账户、提示注入、敏感医疗、正常内容和模型输出六类。真实证据还必须绑定受控提供器、账号/区域、数据条款、额度和审计引用;本地生成器、哈希向量和规则固定为`LOCAL_AGENT_TOOLCHAIN/external_validation_required=true`。v3证据包新增`real_agent_models_validation`,该门禁不可豁免。按低强度策略执行聚合器/证据包定向测试和一条重算现有模拟报告的本地工具链冒烟:既有10类模拟报告重新计算一致,两组六类结构完整,输出`PASS/LOCAL_AGENT_TOOLCHAIN`并继续外部待验;证据包保留真实报告、三提供器和12类别共16项缺口,临时文件已清理。 |
| 托管内容安全外部预检与告警 | OpenAPI 0.64.0把`content_safety_tls`纳入外部依赖查询和候选验收快照。生产Worker仅执行DNS、TCP和受信TLS握手,不发送待分类正文;结果复用依赖可用性指标、数据库脱敏状态和运营诊断页面,连续5分钟不可用进入统一CRITICAL告警,记录缺失/失败/过期均可阻断候选技术快照但不驱逐Pod。按低强度策略执行App/探针/诊断定向测试、前端类型生成/类型检查/生产构建和Prometheus规则校验。 | | 托管内容安全外部预检与告警 | OpenAPI 0.64.0把`content_safety_tls`纳入外部依赖查询和候选验收快照。生产Worker仅执行DNS、TCP和受信TLS握手,不发送待分类正文;结果复用依赖可用性指标、数据库脱敏状态和运营诊断页面,连续5分钟不可用进入统一CRITICAL告警,记录缺失/失败/过期均可阻断候选技术快照但不驱逐Pod。按低强度策略执行App/探针/诊断定向测试、前端类型生成/类型检查/生产构建和Prometheus规则校验。 |
| 云端主机健康状态接入 | OpenAPI 0.65.0新增独立HMAC签名的主机健康回调、30天脱敏快照表和运营只读状态页面。五分钟周期上报覆盖公网入口、容器、备份、证书、磁盘和定时器;无签名、过期、重复检查及综合状态不一致报告被拒绝,页面超过15分钟未更新明确标记过期。该能力不替代整机、Callbacks或数据库失联时必须使用的外部探针。 |
| 云端主机健康发布门禁 | OpenAPI 0.66.0把最新主机健康快照纳入发布候选技术验收:无上报、超过15分钟、综合预警或严重异常均阻断,只有当前有效且23项检查全部正常才通过,其中包括宿主机公网监听端口策略。快照仅归档状态、时间和检查数量,不复制磁盘、证书、监听地址或备份明细值。 |
| 生产基础设施强制门禁 | OpenAPI 0.67.0把生产基础设施状态作为第八项技术门禁。`PRODUCTION`快照必须从当前主机健康报告同时证明`production`部署模式、OSS异地备份上传回读校验不过期、外部告警接收器已配置及两个Mock服务均已退出;演示机继续明确上报`demo`,因此生产快照会被如实阻断。`LAB/STAGING`标记为不适用,不影响开发联调。前端同步展示部署模式、告警和Mock状态;该门禁只证明配置与当前进程状态,不替代真实告警送达演练和生产基础设施一致性报告。 |
| 生产运行时完整性门禁 | OpenAPI 0.68.0继续收紧第八项门禁:主机健康脚本直接读取API、Callbacks和Worker容器的`APP_ENV`及Keycloak实际启动命令。生产快照新增`application_runtime_ready`和`identity_service_ready`,只有三个进程均为`prod`且身份服务不使用`start-dev`才可通过;只修改`HOST_DEPLOYMENT_PROFILE`会同时触发运行环境和身份模式严重检查。演示路径保持`local/development`且功能可用,前端用业务化中文展示两项状态。 |
| 阶段8外部执行轻量预检 | 新增`scripts/stage8-preflight.ps1`,以单一只读入口核对候选Git提交、远程仓库、Compose本地基线、外部配置变量名称和十类证据索引结构/状态;报告明确不读取密钥值,也不把环境变量或本地服务可用性当作验收证据。默认仅报告缺口,`-Strict`可供自动化阻断未就绪交接;当前已建立本地`main`候选提交,本地运行基线就绪且索引结构有效,仍缺远程仓库,样例构建摘要为占位值,十类外部证据均为`PENDING`。 | | 阶段8外部执行轻量预检 | 新增`scripts/stage8-preflight.ps1`,以单一只读入口核对候选Git提交、远程仓库、Compose本地基线、外部配置变量名称和十类证据索引结构/状态;报告明确不读取密钥值,也不把环境变量或本地服务可用性当作验收证据。默认仅报告缺口,`-Strict`可供自动化阻断未就绪交接;当前已建立本地`main`候选提交,本地运行基线就绪且索引结构有效,仍缺远程仓库,样例构建摘要为占位值,十类外部证据均为`PENDING`。 |
| 候选构建身份贯通 | Compose新增统一候选构建参数,后端二进制和后端/Web镜像OCI标签共享版本、完整Git提交及UTC构建时间;远程CI从OpenAPI版本和运行编号形成候选版本,构建后重新读取两个镜像标签并与`GITHUB_SHA`和构建时间逐项比对,再把候选版本、提交、时间和双镜像摘要写入构建元数据。修复了此前Dockerfile虽支持注入但Compose/CI未传参、运行版本仍为`dev/unknown`的问题。 | | 候选构建身份贯通 | Compose新增统一候选构建参数,后端二进制和后端/Web镜像OCI标签共享版本、完整Git提交及UTC构建时间;远程CI从OpenAPI版本和运行编号形成候选版本,构建后重新读取两个镜像标签并与`GITHUB_SHA`和构建时间逐项比对,再把候选版本、提交、时间和双镜像摘要写入构建元数据。修复了此前Dockerfile虽支持注入但Compose/CI未传参、运行版本仍为`dev/unknown`的问题。 |
+15 -1
View File
@@ -1,7 +1,7 @@
# 设备 API 兼容性约定 # 设备 API 兼容性约定
> 当前设备接口主版本:`/device/v2` > 当前设备接口主版本:`/device/v2`
> 当前接口契约:OpenAPI 0.64.0 > 当前接口契约:OpenAPI 0.78.0
> 适用对象:云端 API、终端固件、模拟终端和设备联调工具 > 适用对象:云端 API、终端固件、模拟终端和设备联调工具
## 1. 兼容目标 ## 1. 兼容目标
@@ -18,6 +18,10 @@
## 3. 必须升级到`/device/v3`的变更 ## 3. 必须升级到`/device/v3`的变更
当前`/device/v3/hardware-agent-credentials:provision`和`/device/v3/agent-sessions`承载火山硬件对话智能体控制面:前者由平台代设备动态注册且不下发Product Secret,后者在设备直连供应商音频前预取冻结的人设、记忆和授权配置,并回报会话事件与终稿。现有RTC/VoiceChat设备继续使用完整的`/device/v2`接口,平台不得将v3字段或状态语义反向强加给v2固件。
v3硬件会话必须在创建时冻结SDK版本、音频格式、可靠字幕、动态配置、凭证轮换和双路PCM录制能力,并以`session_updated`作为平台存活检查。`session_started`只有在设备证明本次`session.updated`已应用且Bot、传输、配置摘要匹配后才会激活。事件响应中的`action=stop_session`是强制控制动作:设备先断开供应商,再以新的单调序号回报结束或失败;`continue`才允许保持连接。低负载传输不具备供应商服务端状态回调,因此不得把设备事件和终稿上报能力声明为可选。硬件v3仍处于目标板卡接入期,本次新增必填能力字段不改变已经量产的v2契约。
以下任一变化都属于破坏性变更:删除或重命名字段、改变字段类型或业务含义、给既有请求增加必填字段、缩小已发布枚举导致旧值失效、改变签名算法或规范化规则、让旧固件收到无法安全忽略的新控制动作、改变既有状态机或重试/幂等语义。此类变更必须新增`/device/v3`,并保留v2迁移期和独立契约。 以下任一变化都属于破坏性变更:删除或重命名字段、改变字段类型或业务含义、给既有请求增加必填字段、缩小已发布枚举导致旧值失效、改变签名算法或规范化规则、让旧固件收到无法安全忽略的新控制动作、改变既有状态机或重试/幂等语义。此类变更必须新增`/device/v3`,并保留v2迁移期和独立契约。
CI在拉取请求中以目标分支OpenAPI为基线执行破坏性变更检查。若确实需要破坏性变更,应先建立新主版本路径和迁移方案,不得通过修改或删除旧版契约绕过门禁。 CI在拉取请求中以目标分支OpenAPI为基线执行破坏性变更检查。若确实需要破坏性变更,应先建立新主版本路径和迁移方案,不得通过修改或删除旧版契约绕过门禁。
@@ -42,6 +46,12 @@ CI在拉取请求中以目标分支OpenAPI为基线执行破坏性变更检查
| OpenAPI版本 | 设备兼容变更 | 兼容结论 | | OpenAPI版本 | 设备兼容变更 | 兼容结论 |
|---|---|---| |---|---|---|
| 0.78.0 | 新增独立 `/api/v1/device/*` 首次接入协议;physical_confirm设备使用DEVREQ1签名和v3首通许可;旧设备的HMAC、v2路径和一键认领规则保留 | 新库存显式选择新协议;旧设备兼容 |
| 0.77.0 | 新增v3平台代注册端点;Windows硬件智能体客户端不再接收Product Secret,既有v2接口不变 | v2完全兼容;硬件v3客户端升级 |
| 0.68.0 | 生产基础设施状态增加实际应用运行环境与身份服务模式证明;未修改设备端点、字段、状态机或终端行为 | v2完全兼容 |
| 0.67.0 | 发布候选快照增加生产基础设施强制门禁及主机部署模式、异地备份、外部告警、模拟服务状态;未修改设备端点、字段、状态机或终端行为 | v2完全兼容 |
| 0.66.0 | 发布候选技术验收快照增加主机健康当前性门禁;未修改设备端点、字段、状态机或终端行为 | v2完全兼容 |
| 0.65.0 | 新增主机健康签名回调和运营只读查询;未修改设备端点、字段、状态机或终端行为 | v2完全兼容 |
| 0.64.0 | 运营外部预检增加`content_safety_tls`枚举和候选快照第四类可选必需依赖;未修改设备端点、字段、状态机或终端行为 | v2完全兼容 | | 0.64.0 | 运营外部预检增加`content_safety_tls`枚举和候选快照第四类可选必需依赖;未修改设备端点、字段、状态机或终端行为 | v2完全兼容 |
| 0.63.0 | 增加设备固件阶段事件查询及云端证据字段;设备已有命令回执仅记录同一结果的不可变事件,未增加旧终端必填字段 | v2兼容,旧终端行为不变 | | 0.63.0 | 增加设备固件阶段事件查询及云端证据字段;设备已有命令回执仅记录同一结果的不可变事件,未增加旧终端必填字段 | v2兼容,旧终端行为不变 |
| 0.62.0 | 命令结果增加A/B写入、待重启、启动健康和回退可选字段,省略时保持原有单分区结果语义 | v2兼容,旧终端可继续省略 | | 0.62.0 | 命令结果增加A/B写入、待重启、启动健康和回退可选字段,省略时保持原有单分区结果语义 | v2兼容,旧终端可继续省略 |
@@ -70,3 +80,7 @@ CI在拉取请求中以目标分支OpenAPI为基线执行破坏性变更检查
| 0.34.0 | 增加短期RTC凭证续签动作 | 由0.41.0补充能力协商,消除旧终端收到未知动作的风险 | | 0.34.0 | 增加短期RTC凭证续签动作 | 由0.41.0补充能力协商,消除旧终端收到未知动作的风险 |
每次OpenAPI版本更新都必须同步本表;设备端破坏性变更还必须给出v2保留期限、v3升级方式、灰度范围和回滚条件。 每次OpenAPI版本更新都必须同步本表;设备端破坏性变更还必须给出v2保留期限、v3升级方式、灰度范围和回滚条件。
## 首次接入协议隔离
`ONBOARDING-1.2-SAAS` 仅适用于生产登记为 `physical_confirm` 的新设备,不依靠固件版本猜测协议。工装写入该契约标识、物理编号、raw32字节认证密钥和版本。新设备调用 `/device/v3/agent-sessions`;同一路径新旧认证由已登记身份区分,认证失败不降级。旧 `/device/v2/conversations` 创建不向新设备开放。新设备恢复出厂前须授权解绑,重置响应不含旧claim码。
+847
View File
@@ -0,0 +1,847 @@
# 设备相关接口完整说明
> 版本:2026-09-05
> 契约来源:`internal/httpapi/router.go`、实际处理器、`openapi/openapi.yaml`
> 生产 Base URL:`https://121.41.46.238`
> 适用对象:EchoEar/ESP32-S3 固件、Windows 模拟设备、用户 App/Web、平台运维端。
## 1. 文档范围
本文覆盖当前代码中全部直接或间接管理设备的接口:
| 接口面 | 方法数 | 鉴权 | 用途 |
|---|---:|---|---|
| 设备直连接口 `/device/v2`、`/device/v3` | 20 | 设备 HMAC | 启动、配置、命令、遥测、固件下载、会话、字幕、媒体、火山凭据 |
| 用户设备管理 `/api/v1/devices*` | 10 | OIDC Bearer | 绑定、设备列表、在线状态、影子、命令 |
| 运维设备管理 `/api/v1/ops/devices*` | 9 | OIDC Bearer + `X-Tenant-Id` | 登记、密钥轮换、绑定码、激活状态机 |
| 家庭端与运维端固件接口 | 31 | OIDC Bearer | 签名密钥、版本、上传、灰度、暂停、回滚、下载 |
| 设备诊断处置 | 2 | OIDC Bearer + `X-Tenant-Id` | 强制刷新配置、吊销设备 |
火山硬件智能体 WebSocket 不是平台 HTTP API,但设备聊天必须使用,故在第 10 节单独说明。
## 2. 通用约定
### 2.1 平台响应包络
成功:
```json
{
"request_id": "req-01J...",
"data": {}
}
```
失败:
```json
{
"request_id": "req-01J...",
"error": {
"code": "VALIDATION_FAILED",
"message": "请求参数不正确",
"retryable": false,
"field_errors": [{ "field": "hardware_id", "message": "不能为空" }],
"details": {}
}
}
```
### 2.2 ID、时间和分页
- `{id}`、`device_id`、`conversation_id`、`household_id`、`subject_id` 均为 UUID。
- 时间均为 RFC3339 UTC,例如 `2026-09-05T06:30:00Z`。
- `Idempotency-Key` 长度为 16—128 字符;业务重试保留原正文和原幂等键。
- `If-Match` 使用接口返回的强 ETag,例如 `"7"`。
- 密钥、Token、绑定码和下载地址不得写入普通日志。
### 2.3 设备 HMAC 请求头
所有 `/device/v2/*` 和 `/device/v3/*` 请求必须携带:
| Header | 输入说明 |
|---|---|
| `X-Device-Id` | 平台设备公开 UUID |
| `X-Device-Timestamp` | RFC3339 时间,允许时钟偏差默认 ±5 分钟 |
| `X-Device-Nonce` | 16—128 字符,一次请求只能使用一次 |
| `X-Device-Content-SHA256` | 原始请求正文 SHA-256 小写十六进制;空正文也计算 |
| `X-Device-Signature` | HMAC-SHA256 的 Base64URL 无填充编码 |
签名原文:
```text
HTTP_METHOD大写\n
完整REQUEST_URI(含查询字符串)\n
X-Device-Timestamp\n
X-Device-Nonce\n
X-Device-Content-SHA256
```
## 3. 设备启动、配置、命令和遥测
### 3.1 GET `/device/v2/bootstrap`
介绍:设备上电后报告固件和硬件信息,取得平台权威设备状态、绑定信息、配置版本和服务器时间。
输入:
| 位置 | 字段 | 必填 | 说明 |
|---|---|---:|---|
| Query | `firmware_version` | 是 | 当前固件版本,最长 80 |
| Query | `device_time` | 是 | RFC3339 设备时间 |
| Query | `board_rev` | 否 | 硬件版本 |
| Query | `rtc_sdk_version` | 否 | SDK 版本 |
| Query | `codec` | 否 | 编码,例如 `pcm_s16le` |
| Query | `sample_rate` | 否 | 采样率,例如 `16000` |
| Query | `network` | 否 | 网络类型,例如 `wifi` |
输出:`server_time`、`clock_skew_ms`、设备状态、最低版本、兼容性、`config_revision` 和绑定对象。
示例:
```http
GET /device/v2/bootstrap?firmware_version=1.0.0&board_rev=echoear-v1&codec=pcm_s16le&sample_rate=16000&device_time=2026-09-05T06%3A30%3A00Z&network=wifi
```
```json
{
"request_id": "req-1",
"data": {
"server_time": "2026-09-05T06:30:01Z",
"clock_skew_ms": 1000,
"device": {
"device_id": "10000000-0000-0000-0000-000000000512",
"status": "registered",
"model": "echoear",
"firmware_compatible": true,
"config_revision": 7,
"binding": { "bound": true, "household_id": "11111111-1111-1111-1111-111111111111", "subject_id": "22222222-2222-2222-2222-222222222222", "subject_name": "张大妈" }
}
}
}
```
### 3.2 GET `/device/v2/config`
介绍:读取云端期望配置和当前绑定。设备保存响应 `ETag`,后续用 `If-None-Match` 增量检查。
输入:无正文;可选 Header `If-None-Match: "7"`。
输出:HTTP `200` 返回 `device_id`、`revision`、`desired_config`、`reported_revision`、`binding`;配置未变化返回 `304` 且无正文。
```json
{
"request_id": "req-2",
"data": {
"device_id": "10000000-0000-0000-0000-000000000512",
"revision": 8,
"desired_config": { "voice_runtime": "volc_hardware_agent", "volume": { "speaker": 70 } },
"reported_revision": 7,
"binding": { "bound": true, "subject_id": "22222222-2222-2222-2222-222222222222" }
}
}
```
### 3.3 GET `/device/v2/commands`
介绍:拉取未过期的待执行命令,并增加投递次数。
输入:Query `limit`,范围 1—20,默认 10。
输出:`items[]`;每项包含 `id`、`type`、`payload`、`status`、`expires_at`、`delivery_attempts`、`created_at`。
```json
{
"request_id": "req-3",
"data": {
"items": [{
"id": "33333333-3333-3333-3333-333333333333",
"type": "apply_config",
"payload": { "revision": 8 },
"status": "DELIVERED",
"expires_at": "2026-09-05T07:30:00Z",
"delivery_attempts": 1,
"created_at": "2026-09-05T06:30:00Z"
}]
}
}
```
### 3.4 POST `/device/v2/commands/{id}/ack`
介绍:设备回报命令已接收、成功或失败。
输入:`status` 为 `ACKNOWLEDGED`、`SUCCEEDED`、`FAILED`;`result_summary` 只能包含有限诊断字段,不得含 Secret/Token。
```json
{
"status": "SUCCEEDED",
"result_summary": { "applied_revision": 8 }
}
```
输出:更新后的命令投递对象。
```json
{
"request_id": "req-4",
"data": { "id": "33333333-3333-3333-3333-333333333333", "type": "apply_config", "status": "SUCCEEDED", "result_summary": { "applied_revision": 8 } }
}
```
### 3.5 POST `/device/v2/telemetry:batch`
介绍:批量上报状态、脱敏错误或命令结果;按 `event_key` 和 `sequence_no` 去重,并返回平台动作。
输入:`events` 为 1—500 项;`event_type` 为 `status`、`error` 或 `command_result`,且只携带对应同名对象。
```json
{
"events": [{
"event_key": "boot.000008",
"sequence_no": 8,
"occurred_at": "2026-09-05T06:31:00Z",
"event_type": "status",
"status": {
"firmware_version": "1.0.0",
"reported_config_revision": 8,
"network": { "type": "wifi", "rssi": -51 },
"rtc": { "joined": false, "uplink": false, "downlink": false },
"recording_queue": { "pending_segments": 0, "failed_segments": 0 }
}
}]
}
```
输出:接收/重复数、最新期望配置版本和动作列表。
```json
{
"request_id": "req-5",
"data": { "accepted_count": 1, "duplicate_count": 0, "desired_config_revision": 8, "actions": [{ "type": "pull_commands" }] }
}
```
## 4. 火山硬件智能体平台控制面
### 4.1 POST `/device/v3/hardware-agent-credentials:provision`
介绍:平台使用 Product Secret 代设备调用火山动态注册,只向当前设备返回它自己的一机一密。允许状态:`BOUND`、`PROVISIONING`、`ACTIVE`。
输入:Header `Idempotency-Key`;正文只包含稳定 `hardware_id`。
```json
{ "hardware_id": "echoear-b43a451634a8" }
```
输出:`instance_id`、`product_key`、`device_name`、`hardware_id`、一次性敏感的 `device_secret`、版本和来源。响应必须 `Cache-Control: no-store`。
```json
{
"request_id": "req-6",
"data": {
"instance_id": "6a92945cc0c4689234aef656",
"product_key": "6a9568047dd4b23eacab9b1a",
"device_name": "echoear-100000000512",
"hardware_id": "echoear-b43a451634a8",
"device_secret": "<仅保存到设备安全区>",
"credential_ref": "cred-01J...",
"credential_version": 1,
"credential_source": "platform_dynamic_registration",
"issued_at": "2026-09-05T06:32:00Z"
}
}
```
### 4.2 POST `/device/v3/agent-sessions`
介绍:每次聊天前创建一次平台会话,冻结本次 Bot、授权、人设、记忆、隐私策略和会话证明。仅 `ACTIVE` 设备可调用。
输入:
```json
{
"device_time": "2026-09-05T06:35:00Z",
"monotonic_ms": 123456,
"capabilities": {
"hardware_agent": true,
"sdk_version": "1.1.0-echoear",
"transports": ["low_load"],
"input_audio_format": "pcm_s16le_16000_mono",
"output_audio_format": "pcm_s16le_16000_mono",
"server_event_callback": false,
"transcript_upload": true,
"dynamic_session_config": true,
"credential_rotation": true,
"near_pcm_recording": false,
"far_pcm_recording": false
},
"network": { "type": "wifi", "rssi": -52 },
"requested_goal": "open_conversation",
"config_revision": 8
}
```
输出:
```json
{
"request_id": "req-7",
"data": {
"conversation_id": "44444444-4444-4444-4444-444444444444",
"status": "SESSION_READY",
"runtime_type": "volc_hardware_agent",
"runtime": {
"provider": "volcengine",
"bot_id": "botM4gWQ9A9y",
"transport": "low_load",
"provider_session_ref": "provider-ref",
"session_nonce": "<仅内存保存>",
"config_digest": "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef",
"wait_for_session_update": true
},
"session_config": {
"dialog_id": "55555555-5555-5555-5555-555555555555",
"system_prompt": "<本次人设>",
"persona_tone": "亲切、自然",
"authorized_context": "<本次授权背景与记忆>",
"prompt_bundle_version": "minimal-v2"
},
"privacy": { "retention_mode": "transcript_only", "record_near": false, "record_far": false, "upload_allowed": false },
"session": { "engine_mode": "s2s_hybrid", "heartbeat_interval_sec": 15, "allocation_expires_at": "2026-09-05T06:40:00Z" }
}
}
```
### 4.3 POST `/device/v3/agent-sessions/{id}/events`
介绍:设备回流 `session_started`、`session_updated`、`session_ended`、`session_failed`。`sequence_no` 严格递增。
开始事件输入:
```json
{
"event_id": "evt-start-1",
"sequence_no": 1,
"event_type": "session_started",
"occurred_at": "2026-09-05T06:35:03Z",
"provider_session_ref": "provider-ref",
"session_nonce": "<创建会话返回值>",
"provider_connection_ref": "event_LUVjX_wvR",
"bot_id": "botM4gWQ9A9y",
"transport": "low_load",
"config_digest": "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef",
"dynamic_config_applied": true
}
```
输出:`status`、平台动作、是否重复和接收时间。
```json
{
"request_id": "req-8",
"data": { "conversation_id": "44444444-4444-4444-4444-444444444444", "status": "ACTIVE", "action": "continue", "duplicate": false, "accepted_at": "2026-09-05T06:35:03Z" }
}
```
`action=stop_session` 时必须立即停麦、断开火山,再上报 `session_ended` 或 `session_failed`。
### 4.4 POST `/device/v3/agent-sessions/{id}/turns:batch`
介绍:可靠上传最终字幕;禁止上传增量字幕。单批 1—500 条。
输入:
```json
{
"turns": [{
"turn_key": "user-final-1",
"sequence_no": 1,
"speaker": "subject",
"start_at": "2026-09-05T06:35:10Z",
"end_at": "2026-09-05T06:35:12Z",
"final_text": "你好。",
"interrupted": false
}]
}
```
输出:
```json
{ "request_id": "req-9", "data": { "conversation_id": "44444444-4444-4444-4444-444444444444", "accepted_count": 1, "duplicate_count": 0 } }
```
## 5. 旧 RTC 会话兼容接口
新 EchoEar 使用硬件智能体 v3,不应调用本节的创建/激活接口。本节仅供存量 RTC 设备。
### 5.1 POST `/device/v2/conversations`
介绍:分配 RTC 房间和短期 Token,不启动智能体。
输入:
```json
{
"device_time": "2026-09-05T06:40:00Z",
"monotonic_ms": 130000,
"capabilities": { "rtc": true, "audio_tracks": ["near", "far"], "upload_resume": true, "rtc_credential_renewal": true },
"network": { "type": "wifi", "rssi": -50 },
"requested_goal": "open_conversation"
}
```
输出:`conversation_id`、`ROOM_ISSUED`、RTC 房间/用户/Token、隐私策略和心跳配置。
```json
{
"request_id": "req-10",
"data": {
"conversation_id": "66666666-6666-6666-6666-666666666666",
"status": "ROOM_ISSUED",
"rtc": { "app_id": "app", "room_id": "room", "user_id": "device-user", "agent_user_id": "agent-user", "token": "<仅内存>", "expires_at": "2026-09-05T06:45:00Z" },
"privacy": { "retention_mode": "none", "record_near": false, "record_far": false, "upload_allowed": false },
"session": { "engine_mode": "cascade", "heartbeat_interval_sec": 10, "allocation_expires_at": "2026-09-05T06:45:00Z" }
}
}
```
### 5.2 POST `/device/v2/conversations/{id}/activate`
介绍:设备确认已成功进入指定 RTC 房间后,触发平台启动旧语音任务。
输入:
```json
{ "joined_at": "2026-09-05T06:40:02Z", "room_id": "room", "rtc_user_id": "device-user", "rtc_join_result": "success", "config_revision": 8, "connection_reference": "rtc-conn-1" }
```
输出:
```json
{ "request_id": "req-11", "data": { "conversation_id": "66666666-6666-6666-6666-666666666666", "status": "STARTING_AGENT", "accepted_at": "2026-09-05T06:40:02Z" } }
```
### 5.3 GET `/device/v2/conversations/{id}`
介绍:读取平台权威会话状态和设备下一步动作。
输入:Path `id`。
输出示例:
```json
{ "request_id": "req-12", "data": { "conversation_id": "66666666-6666-6666-6666-666666666666", "status": "ACTIVE", "action": "continue" } }
```
失败时可能返回 `status=FAILED`、`action=leave_room`、稳定脱敏的 `failure_code`。
### 5.4 POST `/device/v2/conversations/{id}/heartbeat`
介绍:刷新会话租约、设备在线状态和配置状态;返回停止、续签或配置动作。
输入:
```json
{
"sequence_no": 2,
"device_time": "2026-09-05T06:40:12Z",
"monotonic_ms": 142000,
"conversation_status": "ACTIVE",
"rtc": { "joined": true, "uplink": true, "downlink": true },
"network": { "type": "wifi", "rssi": -49 },
"firmware_version": "1.0.0",
"reported_config_revision": 8,
"recording_queue": { "pending_segments": 0, "failed_segments": 0 },
"command_results": []
}
```
输出:
```json
{ "request_id": "req-13", "data": { "lease_expires_at": "2026-09-05T06:40:42Z", "actions": [{ "type": "continue" }], "desired_config_revision": 8, "commands": [] } }
```
### 5.5 POST `/device/v2/conversations/{id}/stop`
介绍:设备挂断并可靠停止旧供应商任务;支持幂等重放。
输入:
```json
{ "reason": "user_hangup", "device_time": "2026-09-05T06:42:00Z", "last_heartbeat_sequence": 5, "rtc_left": true, "recording": { "near_segments": 0, "far_segments": 0 } }
```
输出:
```json
{ "request_id": "req-14", "data": { "conversation_id": "66666666-6666-6666-6666-666666666666", "status": "STOPPING", "accepted_at": "2026-09-05T06:42:00Z" } }
```
### 5.6 POST `/device/v2/conversations/{id}/turns:batch`
介绍:旧 RTC 回调缺少终稿时,由设备补传最终字幕。输入输出与 4.4 完全相同。
```json
{ "turns": [{ "turn_key": "agent-final-2", "sequence_no": 2, "speaker": "agent", "start_at": "2026-09-05T06:41:00Z", "end_at": "2026-09-05T06:41:02Z", "final_text": "您好。", "interrupted": false }] }
```
## 6. 设备录音上传接口
仅当本次 `privacy.upload_allowed=true` 且对应轨道获授权时使用。
### 6.1 POST `/device/v2/conversations/{id}/upload-batches`
介绍:声明待上传录音分片,平台返回每个对象的短期 PUT 地址;业务 API 不接收音频字节。
输入:
```json
{
"segments": [{
"track": "near",
"sequence_no": 0,
"codec": "wav_pcm_s16le",
"content_type": "audio/wav",
"duration_ms": 10000,
"expected_size": 320044,
"expected_digest": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
}]
}
```
输出:批次状态和分片;创建响应中的分片包含 `upload_url`、`upload_until`。
```json
{
"request_id": "req-15",
"data": {
"id": "77777777-7777-7777-7777-777777777777",
"conversation_id": "66666666-6666-6666-6666-666666666666",
"status": "UPLOADING",
"segment_count": 1,
"expected_total_size": 320044,
"verified_count": 0,
"rejected_count": 0,
"verification_attempts": 0,
"expires_at": "2026-09-05T07:02:00Z",
"segments": [{ "id": "88888888-8888-8888-8888-888888888888", "track": "near", "sequence_no": 0, "status": "AUTHORIZED", "upload_url": "https://object.example/put?...", "upload_until": "2026-09-05T07:02:00Z" }]
}
}
```
### 6.2 POST `/device/v2/upload-batches/{id}/complete`
介绍:声明分片已上传,平台开始异步对象大小、摘要和容器校验。
输入:无正文;Path `id` 为上传批次 UUID。
输出:HTTP `202`,返回批次,`status` 通常为 `VERIFYING`。
```json
{ "request_id": "req-16", "data": { "id": "77777777-7777-7777-7777-777777777777", "status": "VERIFYING", "segment_count": 1, "verified_count": 0, "rejected_count": 0, "segments": [] } }
```
### 6.3 GET `/device/v2/upload-batches/{id}`
介绍:轮询后台校验结果。只有 `VERIFIED` 分片返回 `local_delete_after`。
输入:Path `id`。
输出示例:
```json
{ "request_id": "req-17", "data": { "id": "77777777-7777-7777-7777-777777777777", "status": "COMPLETED", "segment_count": 1, "verified_count": 1, "rejected_count": 0, "segments": [{ "id": "88888888-8888-8888-8888-888888888888", "status": "VERIFIED", "local_delete_after": "2026-09-05T07:10:00Z" }] } }
```
### 6.4 POST `/device/v2/media-segments/{id}/local-delete-ack`
介绍:到达平台允许删除时间后,设备删除本地密文并回报;不会删除云端对象。
输入:
```json
{ "status": "DELETED" }
```
失败示例:`{ "status": "FAILED", "error_code": "FLASH_DELETE_FAILED" }`。
输出:更新后的分片对象,包含 `local_delete_status` 和 `local_delete_reported_at`。
## 7. 设备固件下载接口
### GET `/device/v2/firmware-releases/{id}/download`
介绍:仅向当前灰度批次中的目标设备签发 5 分钟私有下载地址。
输入:Path `id` 为固件版本 UUID,无正文。
输出:
```json
{ "request_id": "req-18", "data": { "download_url": "https://object.example/firmware.bin?...", "download_until": "2026-09-05T07:05:00Z" } }
```
非目标设备或暂停/退役版本返回 `403 FIRMWARE_DOWNLOAD_DENIED`。
## 8. 用户 App/Web 设备管理接口
统一输入 Header:`Authorization: Bearer <OIDC access token>`。以下表中“示例输入/输出”均省略通用包络字段。
| 方法与路径 | 接口介绍 | 输入 | 输出 | 示例 |
|---|---|---|---|---|
| `POST /api/v1/device-claims` | 消费一次性绑定码 | Body:`claim_code`、`household_id`、`subject_id` | `binding_id`、`device_id`、归属、`bound_at` | 入:`{"claim_code":"CLAIM-...","household_id":"<uuid>","subject_id":"<uuid>"}`;出:`{"binding_id":"<uuid>","device_id":"<uuid>","bound_at":"2026-09-05T07:00:00Z"}` |
| `GET /api/v1/devices` | 列出当前账号可管理设备 | 无 | `items: ManagedDevice[]` | `{"items":[{"id":"<uuid>","model":"echoear","status":"registered","desired_revision":8}]}` |
| `GET /api/v1/devices/{id}` | 查询设备与绑定摘要 | Path:设备 UUID | `ManagedDevice` | `{"id":"<uuid>","binding_id":"<uuid>","subject_name":"张大妈","firmware_version":"1.0.0","version":3}` |
| `GET /api/v1/devices/{id}/binding-impact` | 解绑/转移前检查阻断项 | Query:`action=unbind\|transfer`;转移时 `target_subject_id` | 影响、`allowed`、`blockers`、当前 `binding_id` | `{"action":"transfer","allowed":true,"pending_command_count":0,"active_firmware_task_count":0}` |
| `POST /api/v1/devices/{id}/binding-change` | 执行解绑或转移 | Body:`action`、`expected_binding_id`、`reason`、`acknowledged=true`;转移时 `target_subject_id` | 旧/新绑定和变更时间 | 入:`{"action":"unbind","expected_binding_id":"<uuid>","reason":"用户解除绑定","acknowledged":true}`;出:`{"device_id":"<uuid>","action":"unbind","changed_at":"..."}` |
| `GET /api/v1/devices/{id}/presence` | 查询实时在线状态 | Path:设备 UUID | `online`、`status`、`last_seen_at`、`source` | `{"device_id":"<uuid>","online":true,"status":"ACTIVE","last_seen_at":"...","source":"redis"}` |
| `GET /api/v1/devices/{id}/twin` | 查询设备影子 | Path:设备 UUID | 期望/上报配置、版本、能力、同步状态;Header `ETag` | `{"device_id":"<uuid>","desired_revision":8,"desired_config":{"volume":{"speaker":70}},"sync_state":"applied"}` |
| `PATCH /api/v1/devices/{id}/twin` | 修改非敏感期望配置 | Header:`If-Match`;Body:`desired_config`,顶层仅 `volume/interaction/upload/rtc/policy_refs` | 更新后的 `DeviceTwin` 和新 `ETag` | `{"desired_config":{"volume":{"speaker":60}}}` |
| `GET /api/v1/devices/{id}/commands` | 查询最近 100 条命令 | Path:设备 UUID | `items: DeviceCommand[]` | `{"items":[{"id":"<uuid>","type":"reboot","status":"SUCCEEDED"}]}` |
| `POST /api/v1/devices/{id}/commands` | 下发可靠命令 | Header:`Idempotency-Key`;Body:`type`、`payload`、`expires_in_sec` | `DeviceCommand`,首次 `202`,重放 `200` | 入:`{"type":"reboot","payload":{"reason":"maintenance"},"expires_in_sec":300}`;出:`{"id":"<uuid>","status":"QUEUED","delivery_attempts":0}` |
`ManagedDevice` 主要输出字段:`id`、`binding_id`、`household_id`、`subject_id`、`subject_name`、`model`、`hardware_revision`、`status`、固件/配置版本、`last_reported_at`、`bound_at`、`registered_at`、`version`。
## 9. 平台运维设备管理接口
统一输入 Header:`Authorization: Bearer <operator token>`、`X-Tenant-Id: <tenant UUID>`。只有独立运维账号可调用。
| 方法与路径 | 接口介绍 | 输入 | 输出 | 示例 |
|---|---|---|---|---|
| `GET /api/v1/ops/devices` | 查询租户全部设备 | 无 | `items: OpsDevice[]`,不返回密钥 | `{"items":[{"id":"<uuid>","model":"echoear","activation_state":"ACTIVE","credential_status":"active","voice_runtime":"volc_hardware_agent"}]}` |
| `POST /api/v1/ops/devices` | 登记新设备并生成平台 HMAC 密钥 | Body:`model` 必填;可选硬件/最低版本/归属/运行时/初始激活状态 | `device`、只显示一次的 `credential`;可能同时返回 `claim` | 入:`{"model":"echoear","hardware_revision":"esp32-s3","voice_runtime":"volc_hardware_agent","activation_state":"FACTORY"}`;出:`{"device":{"id":"<uuid>"},"credential":{"device_id":"<uuid>","secret":"<64位hex>","key_version":1}}` |
| `GET /api/v1/ops/device-assignment-options` | 查询可分配家庭和长者 | 无 | `items[]` 含家庭/长者 ID 和名称 | `{"items":[{"household_id":"<uuid>","household_name":"张家","subject_id":"<uuid>","subject_name":"张大妈"}]}` |
| `PATCH /api/v1/ops/devices/{id}` | 更新设备型号、版本策略、状态、归属或运行时 | Header:`If-Match`;Body 至少一个可修改字段 | 更新后的 `OpsDevice` | `{"minimum_firmware_version":"1.1.0","status":"registered"}` |
| `DELETE /api/v1/ops/devices/{id}` | 软删除并吊销设备/凭据/绑定 | Header:`If-Match` | 吊销后的 `OpsDevice` | `{"id":"<uuid>","status":"revoked","activation_state":"REVOKED","credential_status":"revoked"}` |
| `POST /api/v1/ops/devices/{id}/credentials/rotate` | 轮换平台 HMAC 密钥,旧密钥立即失效 | 无正文 | 只显示一次的新 `secret`、`key_version`、`issued_at` | `{"device_id":"<uuid>","secret":"<64位hex>","key_version":2,"issued_at":"..."}` |
| `POST /api/v1/ops/devices/{id}/claims` | 为未绑定设备生成 24 小时绑定码 | 无正文 | 只显示一次的 `claim_code` 和过期时间 | `{"device_id":"<uuid>","claim_code":"CLAIM-...","issued_at":"...","expires_at":"..."}` |
| `POST /api/v1/ops/devices/{id}/activation-transitions` | 执行受控激活状态迁移 | Body:`target_state`、`reason`、`expected_revision` | 更新后的 `OpsDevice` | `{"target_state":"CLAIMABLE","reason":"工厂质检通过","expected_revision":1}` |
| `GET /api/v1/ops/devices/{id}/activation-transitions` | 查询完整激活审计历史 | Path:设备 UUID | `items: DeviceActivationTransition[]` | `{"items":[{"from_state":"FACTORY","to_state":"CLAIMABLE","reason":"工厂质检通过","activation_revision":2,"created_at":"..."}]}` |
激活状态:`FACTORY → CLAIMABLE → BOUND → PROVISIONING → ACTIVE → LOST/REVOKED`。设备不能自行修改该状态。
## 10. 火山硬件智能体 WebSocket
### 10.1 建连
```text
wss://ai-gateway.vei.volces.com/v1/realtime?bot=botM4gWQ9A9y&wait_for_session_update=true
```
输入 Header:
| Header | 说明 |
|---|---|
| `X-Auth-Type` | 固定 `1` |
| `X-Product-Key` | 平台预配返回的 Product Key |
| `X-Device-Name` | 平台预配返回的 Device Name |
| `X-Random-Num` | 本次随机整数 |
| `X-Timestamp` | Unix 秒 |
| `X-Instance-Id` | 平台预配返回的 Instance ID |
| `X-Signature` | 使用火山 Device Secret 计算的 HMAC-SHA256 Base64 |
| `X-Hardware-Id` | 稳定 Hardware ID |
签名正文顺序固定:
```text
auth_type=1&device_name={name}&random_num={random}&product_key={key}&timestamp={unix}&instance_id={instance}
```
### 10.2 动态配置输入
连接打开后立即发送:
```json
{
"event_id": "windows-session-update-<uuid>",
"type": "session.update",
"session": {
"object": "realtime.session",
"model": "",
"input_audio_format": "pcm16",
"output_audio_format": "pcm16",
"config": {
"ASRConfig": { "TurnDetectionMode": 0 },
"LLMConfig": { "SystemMessages": ["<平台冻结的人设与授权上下文>"] }
},
"agent_config": { "EnableConversationStateCallback": false }
}
}
```
输出:`session.created` 或 `session.updated`;在收到确认前不得开启麦克风。
### 10.3 音频和文本输入
上行音频:PCM16 little-endian、16kHz、单声道,建议每 20ms 320 个采样。
```json
{ "event_id": "audio-<uuid>", "type": "input_audio_buffer.append", "audio": "<PCM Base64>" }
```
文本测试:
```json
{
"event_id": "text-<uuid>",
"type": "conversation.item.create",
"item": { "type": "message", "role": "user", "content": [{ "type": "input_text", "text": "你好" }], "interrupt_mode": 1 }
}
```
### 10.4 设备消费的输出事件
| 事件 | 输出内容 | 设备动作 |
|---|---|---|
| `input_audio_buffer.speech_started/stopped` | 服务端 VAD 状态 | 更新交互状态 |
| `conversation.item.input_audio_transcription.completed` | 用户最终字幕 | 上传平台 `turns:batch` |
| `response.audio.delta` | Base64 PCM16、16kHz、单声道 | 连续缓冲并按声卡采样率重采样播放 |
| `response.audio.done` | 本轮音频结束 | 排空播放缓冲 |
| `response.audio_transcript.delta/done` | 智能体增量/最终字幕 | 只把最终字幕上传平台 |
| `response.function_call_arguments.done` | 工具名、参数和 `call_id` | 调用平台受控工具并回传结果 |
| `response.done` | 本轮完成 | 继续监听 |
| `error` | 供应商错误 | 脱敏后上报 `session_failed` |
注意:low-load WebSocket 下行是 16kHz。把它按 24kHz 播放会出现约 1.5 倍速、音调升高和明显电子感。
## 11. 固件管理接口
### 11.1 家庭/租户授权接口
| 方法与路径 | 介绍 | 输入 | 输出 |
|---|---|---|---|
| `GET /api/v1/households/{id}/firmware-signing-keys` | 查询可信 Ed25519 公钥 | Path `household id` | `FirmwareSigningKey[]` |
| `POST /api/v1/households/{id}/firmware-signing-keys` | 登记签名公钥 | `key_id`、`public_key_base64` | `201`,创建结果 |
| `POST /api/v1/households/{id}/firmware-signing-keys/revoke` | 吊销公钥 | `key_id` | `FirmwareSigningKey` |
| `GET /api/v1/households/{id}/firmware-releases` | 查询固件版本 | Path `household id` | `FirmwareRelease[]` |
| `POST /api/v1/households/{id}/firmware-releases` | 创建 `UPLOADING` 版本 | `PrepareFirmwareReleaseRequest` | 版本和短期上传地址 |
| `GET /api/v1/firmware-releases/{id}/impact` | 发布/暂停/退役前影响评估 | Path `release id` | `FirmwareReleaseImpact` |
| `POST /api/v1/firmware-releases/{id}/upload-session` | 重新签发上传地址 | 无正文 | `FirmwareRelease` |
| `POST /api/v1/firmware-releases/{id}/finalize` | 校验大小、SHA-256、Ed25519 后发布 | 无正文 | `FirmwareRelease` |
| `POST /api/v1/firmware-releases/{id}/pause` | 暂停版本和未投递命令 | 无正文 | `FirmwareRelease` |
| `POST /api/v1/firmware-releases/{id}/resume` | 恢复版本分发 | 无正文 | `FirmwareRelease` |
| `POST /api/v1/firmware-releases/{id}/retire` | 退役版本并保留审计 | 无正文 | `FirmwareRelease` |
| `GET /api/v1/households/{id}/firmware-rollouts` | 查询 OTA 批次 | Path `household id` | `FirmwareRollout[]` |
| `POST /api/v1/households/{id}/firmware-rollouts` | 启动灰度升级 | `StartFirmwareRolloutRequest` | `202 FirmwareRollout` |
| `GET /api/v1/firmware-rollouts/{id}` | 查询逐设备升级进度 | Path `rollout id` | `FirmwareRollout` |
| `POST /api/v1/firmware-rollouts/{id}/pause` | 暂停批次 | 无正文 | `FirmwareRollout` |
| `POST /api/v1/firmware-rollouts/{id}/rollback` | 创建回滚批次 | `rollback_release_id` | 新的 `FirmwareRollout` |
### 11.2 运维固件接口
以下接口全部额外要求 `X-Tenant-Id`:
| 方法与路径 | 介绍 | 输入 | 输出 |
|---|---|---|---|
| `GET /api/v1/ops/firmware-signing-keys` | 查询租户可信公钥 | 无 | `FirmwareSigningKey[]` |
| `POST /api/v1/ops/firmware-signing-keys` | 登记可信公钥 | `key_id`、`public_key_base64` | `201` |
| `POST /api/v1/ops/firmware-signing-keys/revoke` | 吊销公钥 | `key_id` | `FirmwareSigningKey` |
| `GET /api/v1/ops/firmware-releases` | 按 `model/status` 查询版本 | Query `model`、`status` 可选 | `FirmwareRelease[]` |
| `POST /api/v1/ops/firmware-releases` | 创建租户级版本 | `PrepareOpsFirmwareReleaseRequest` | `201 FirmwareRelease` |
| `POST /api/v1/ops/firmware-releases/{id}/upload-session` | 续签上传地址 | 无正文 | `FirmwareRelease` |
| `POST /api/v1/ops/firmware-releases/{id}/finalize` | 正式发布 | 无正文 | `FirmwareRelease` |
| `POST /api/v1/ops/firmware-releases/{id}/pause` | 暂停 | 无正文 | `FirmwareRelease` |
| `POST /api/v1/ops/firmware-releases/{id}/resume` | 恢复 | 无正文 | `FirmwareRelease` |
| `POST /api/v1/ops/firmware-releases/{id}/retire` | 退役 | 无正文 | `FirmwareRelease` |
| `GET /api/v1/ops/firmware-releases/{id}/download` | 运维下载制品 | Path `release id` | 下载地址、摘要、签名元数据 |
| `GET /api/v1/ops/firmware-rollouts` | 查询 OTA 批次 | 无 | `FirmwareRollout[]` |
| `POST /api/v1/ops/firmware-rollouts` | 启动 OTA | `StartFirmwareRolloutRequest` | `202 FirmwareRollout` |
| `GET /api/v1/ops/firmware-rollouts/{id}` | 查询逐设备进度 | Path `rollout id` | `FirmwareRollout` |
| `POST /api/v1/ops/firmware-rollouts/{id}/pause` | 暂停 OTA | 无正文 | `FirmwareRollout` |
固件请求示例:
```json
{ "key_id": "release-key-2026-01", "public_key_base64": "<32字节Ed25519公钥的Base64>" }
```
```json
{
"model": "echoear",
"version": "1.1.0",
"expected_size": 2097152,
"expected_digest": "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
"signing_key_id": "release-key-2026-01",
"signature": "<firmware-manifest-v1签名Base64>"
}
```
```json
{ "release_id": "<uuid>", "name": "第一批灰度", "target_percentage": 10, "device_ids": ["<device-uuid>"] }
```
典型版本输出:
```json
{ "request_id": "req-fw", "data": { "id": "<uuid>", "model": "echoear", "version": "1.1.0", "status": "READY", "sha256": "bbbb...", "created_at": "2026-09-05T07:00:00Z" } }
```
典型批次输出:
```json
{ "request_id": "req-rollout", "data": { "id": "<uuid>", "release_id": "<uuid>", "name": "第一批灰度", "status": "RUNNING", "target_percentage": 10, "targeted": 1, "succeeded": 0, "failed": 0 } }
```
## 12. 运维诊断设备动作
| 方法与路径 | 介绍 | 输入 | 输出 | 示例 |
|---|---|---|---|---|
| `POST /api/v1/ops/diagnostics/devices/{id}/config-refresh` | 强制设备重新拉取配置 | `X-Tenant-Id`、`Idempotency-Key`、设备 UUID;Body:`reason` | `action=refresh_config`、命令 ID/状态、`queued` | 入:`{"reason":"远端配置与影子不一致"}`;出:`{"action":"refresh_config","device_id":"<uuid>","command_id":"<uuid>","command_status":"QUEUED","queued":true}` |
| `POST /api/v1/ops/diagnostics/devices/{id}/revoke` | 紧急吊销异常设备 | `X-Tenant-Id`、`Idempotency-Key`、设备 UUID;Body:`reason` | `action=revoke_device`、设备 ID、`queued` | 入:`{"reason":"设备密钥疑似泄露"}`;出:`{"action":"revoke_device","device_id":"<uuid>","queued":false}` |
## 13. 主要错误码
| HTTP | 错误码 | 含义与处理 |
|---:|---|---|
| 400 | `VALIDATION_FAILED` | 字段、Header、时间或幂等键不符合契约 |
| 401 | `DEVICE_AUTHENTICATION_FAILED` | 检查平台 Device Secret、时钟、URI、正文摘要和 Base64URL |
| 403 | `DEVICE_ACTIVATION_REQUIRED` | 当前激活状态不允许该操作 |
| 403 | `TRANSCRIPT_STORAGE_NOT_ALLOWED` | 当前授权不允许保存字幕 |
| 403 | `FIRMWARE_DOWNLOAD_DENIED` | 设备不是灰度目标或版本不可下载 |
| 404 | `RESOURCE_NOT_FOUND` | 资源不存在或不属于当前主体/设备 |
| 409 | `IDEMPOTENCY_KEY_REUSED` | 同一幂等键对应不同正文,客户端必须换业务请求 |
| 409 | `DEVICE_ACTIVATION_CONFLICT` | 激活状态或修订号冲突,刷新后再处理 |
| 409 | `HARDWARE_AGENT_HARDWARE_MISMATCH` | Hardware ID 与已注册记录不一致,禁止覆盖 |
| 409 | `HARDWARE_SESSION_PROOF_INVALID` | `session_nonce`/配置摘要/Bot/传输不匹配,丢弃本次会话重新创建 |
| 412 | `VERSION_CONFLICT` | `If-Match` 过期,先重新 GET |
| 426 | `FIRMWARE_UPDATE_REQUIRED` | 当前固件低于最低版本 |
| 429 | `USAGE_QUOTA_EXCEEDED` | 额度或并发限制达到上限 |
| 503 | `HARDWARE_AGENT_PROVISIONING_UNAVAILABLE` | 平台 Product Secret/产品配置不可用 |
| 503 | `HARDWARE_AGENT_PROVISIONING_FAILED` | 火山动态注册失败;按 `request_id` 排障,可退避重试 |
## 14. 设备端推荐调用顺序
```text
上电
→ GET /device/v2/bootstrap
→ GET /device/v2/config
→ GET /device/v2/commands
→ POST /device/v2/telemetry:batch
首次绑定后
→ POST /device/v3/hardware-agent-credentials:provision
→ 安全保存火山 Device Secret
每次聊天
→ POST /device/v3/agent-sessions
→ 连接火山 WebSocket
→ session.update
→ 收到 session.created/session.updated
→ POST /device/v3/agent-sessions/{id}/events (session_started)
→ 16k PCM 双向音频
→ POST /device/v3/agent-sessions/{id}/turns:batch
→ 周期 events (session_updated)
→ events (session_ended/session_failed)
```
## 15. 机器可读契约
- 完整 OpenAPI:[`../openapi/openapi.yaml`](../openapi/openapi.yaml)
- 设备实现与安全说明:[`device-api-reference.md`](device-api-reference.md)
- 设备接入说明:[`device-integration.md`](device-integration.md)
- 固件灰度说明:[`firmware-rollout.md`](firmware-rollout.md)
若本文与 OpenAPI 或运行时代码不一致,以同一发布版本的 OpenAPI 和处理器为准,并应同步修正文档。
+576
View File
@@ -0,0 +1,576 @@
# 设备接口文档
> 适用范围:平台设备管理、EchoEar/ESP32-S3 设备接入、火山引擎硬件对话智能体。
> 契约基线:当前工作区实际路由、处理器与 `openapi/openapi.yaml`,更新日期 2026-09-05。
> 生产 API:`https://121.41.46.238`(正式域名启用后只替换 Base URL,不改变路径)。
## 1. 接入模型
设备同时使用两套彼此独立的身份:
| 身份/密钥 | 用途 | 生成与保存位置 | 是否下发设备 |
|---|---|---|---|
| 平台 `device_id` | 标识平台登记设备 | 平台生成;设备保存公开编号 | 是 |
| 平台 Device Secret | 调用 `/device/*` 时做 HMAC 鉴权 | 平台逐设备生成;平台加密保存,设备安全区保存 | 是,只在登记/轮换时显示一次 |
| 火山 `instance_id` / `product_key` | 标识火山实例和产品 | 平台配置 | 可随预配响应返回,非秘密 |
| 火山 Product Secret | 调用 DynamicRegister 的产品级秘密 | 只保存在平台 KMS/Secret | **否** |
| 火山 `device_name` / `device_secret` | 设备直连火山时使用的一机一密 | 平台代设备动态注册;平台加密保存,设备安全区保存 | 是,只返回给已认证的对应设备 |
| Bot ID | 选择硬件对话智能体 | 平台按设备绑定 | 会话响应返回,非秘密 |
核心原则:设备固件不内置 Product Secret。设备先用平台 Device Secret 认证,由平台代设备调用火山动态注册,再把该设备专属的火山 Device Secret 返回给设备。
### 1.1 火山 License 的准确含义
火山硬件对话智能体的 License 是产品侧可用额度。平台调用 DynamicRegister 时,火山自动消耗当前产品的一份有效额度并返回一机一密;设备不接收一段可安装的 License 明文,也不需要向平台提交 License 安装回执。
因此,当前有效契约中:
- 没有 `/device/v3/hardware-agent-license:ack` 路由;
- 没有设备端 `license` 字段;
- 没有 `AVAILABLE → ALLOCATED → ACTIVE` 的设备 License 安装状态机;
- 平台可以维护额度监控或影子账本,但不能用它替代火山的真实额度,也不能阻断已正确取得一机一密的设备。
火山返回额度耗尽、过期等错误时,平台统一转换为 `HARDWARE_AGENT_PROVISIONING_FAILED`,同时通过内部日志和请求编号保留供应商错误供运维定位。
## 2. 通用响应格式
成功响应:
```json
{
"request_id": "req-...",
"data": {}
}
```
失败响应:
```json
{
"request_id": "req-...",
"error": {
"code": "VALIDATION_FAILED",
"message": "请求参数不正确",
"retryable": false,
"field_errors": [],
"details": {}
}
}
```
排障时应记录 `request_id`、HTTP 状态码和稳定错误码,不记录任何 Secret、Token、完整人设或完整对话正文。
## 3. 设备 HMAC 鉴权
所有 `/device/v2/*` 和 `/device/v3/*` 请求必须携带:
| 请求头 | 说明 |
|---|---|
| `X-Device-Id` | 平台设备公开 UUID |
| `X-Device-Timestamp` | RFC3339 UTC 时间,默认允许前后 5 分钟 |
| `X-Device-Nonce` | 本次请求唯一的 16—128 字符随机值 |
| `X-Device-Content-SHA256` | 原始 HTTP 正文的 SHA-256 小写十六进制;空正文也要计算 |
| `X-Device-Signature` | HMAC-SHA256 结果的无填充 Base64URL |
签名原文必须逐字使用实际请求 URI(含查询字符串):
```text
UPPERCASE_HTTP_METHOD
REQUEST_URI_WITH_EXACT_QUERY_STRING
X-Device-Timestamp原值
X-Device-Nonce原值
X-Device-Content-SHA256原值
```
伪代码:
```text
bodyHash = hexLower(SHA256(rawBody))
canonical = methodUpper + "\n" + requestURI + "\n" + timestamp + "\n" + nonce + "\n" + bodyHash
signature = base64UrlNoPadding(HMAC_SHA256(platformDeviceSecret, canonical))
```
注意事项:
- JSON 签名后不得重新格式化或改变字段顺序;服务端校验的是收到的原始字节。
- 每次网络重试都生成新的 `timestamp`、`nonce` 和签名。
- 需要幂等的业务请求保留原 `Idempotency-Key` 和原正文。
- 时间未同步时先通过安全时间源校时;不能关闭时间窗或重放保护。
## 4. 设备激活状态机
主流程:
```text
FACTORY → CLAIMABLE → BOUND → PROVISIONING → ACTIVE
│
├─→ LOST
└─→ REVOKED
```
| 状态 | 含义 | 允许的设备操作 |
|---|---|---|
| `FACTORY` | 只有出厂身份,等待质检放行 | 不允许设备 API 鉴权 |
| `CLAIMABLE` | 可生成/消费绑定码 | 启动、配置、命令、遥测;不能通话 |
| `BOUND` | 已绑定家庭和使用人 | 可申请火山一机一密;不能创建会话 |
| `PROVISIONING` | 正在注册或恢复火山运行凭据 | 可重复预配;不能创建会话 |
| `ACTIVE` | 平台身份、绑定和运行凭据就绪 | 全部设备接口 |
| `LOST` | 设备挂失 | 设备鉴权和新会话被拒绝 |
| `REVOKED` | 永久吊销 | 终态,不可恢复 |
代码允许的恢复/解绑边包括:`BOUND → CLAIMABLE`、`PROVISIONING → BOUND`、`ACTIVE → CLAIMABLE/PROVISIONING/LOST`、`LOST → CLAIMABLE/ACTIVE`;任意非 `REVOKED` 状态均可受控转为 `REVOKED`。状态变更必须由平台业务或运维接口执行,设备不能自行声明已激活。
每次变更增加 `activation_revision` 并保存审计历史。进入 `ACTIVE` 时,硬件智能体设备必须已经绑定且存在有效火山一机一密。
## 5. 首次激活完整流程
1. 运维调用 `POST /api/v1/ops/devices` 登记设备,取得一次性平台 Device Secret。
2. 工厂烧录/安全注入 `device_id`、平台 Device Secret、API Base URL 和稳定 `hardware_id`;不得烧录 Product Secret。
3. 运维将设备从 `FACTORY` 放行到 `CLAIMABLE`,并生成绑定码。
4. 用户 App/Web 调用 `POST /api/v1/device-claims` 消费绑定码,平台把设备变为 `BOUND`。
5. 设备使用平台 HMAC 调用 `POST /device/v3/hardware-agent-credentials:provision`。
6. 平台把状态推进到 `PROVISIONING`,自动建立产品默认 Bot/传输/最低 SDK 绑定,再使用服务端 Product Secret 调用火山 DynamicRegister。
7. 火山消耗产品 License 额度并返回该 `device_name` 对应的 `device_secret`。
8. 平台加密保存火山 Device Secret,并通过 `no-store` 响应返回给同一设备,然后自动完成到 `ACTIVE` 的迁移。
9. 设备把火山一机一密写入 NVS 加密分区/安全芯片;以后重复预配会返回已保存的同一凭据,不重复注册。
10. 每次用户发起聊天时,设备先创建平台硬件智能体会话,注入本次人设和上下文后,再直连火山。
## 6. 设备端接口总览
### 6.1 启动、配置、命令和固件(v2 公共能力)
| 方法 | 路径 | 状态要求 | 幂等键 | 作用 |
|---|---|---|---|---|
| `GET` | `/device/v2/bootstrap` | `CLAIMABLE/BOUND/PROVISIONING/ACTIVE` | 否 | 上报启动版本并取得绑定和兼容性 |
| `GET` | `/device/v2/config` | 同上 | 否 | 拉取期望配置;支持 `If-None-Match` |
| `GET` | `/device/v2/commands` | 同上 | 否 | 拉取待执行命令 |
| `POST` | `/device/v2/commands/{id}/ack` | 同上 | 否 | 上报命令阶段或终态 |
| `POST` | `/device/v2/telemetry:batch` | 同上 | 是 | 批量上报设备遥测 |
| `GET` | `/device/v2/firmware-releases/{id}/download` | 同上 | 否 | 下载受控固件制品 |
`bootstrap` 示例:
```http
GET /device/v2/bootstrap?firmware_version=1.0.0&board_rev=echoear-v1&rtc_sdk_version=1.0.0&codec=pcm_s16le&sample_rate=16000&device_time=2026-09-03T08%3A00%3A00Z&network=wifi
```
配置拉取建议:首次不带 `If-None-Match`;保存响应 `ETag`,后续携带该值。HTTP `304` 表示继续使用本地有效配置。
### 6.2 旧 RTC 通话(v2,仅兼容旧设备)
| 方法 | 路径 | 说明 |
|---|---|---|
| `POST` | `/device/v2/conversations` | 创建旧 RTC 通话 |
| `POST` | `/device/v2/conversations/{id}/activate` | 入房成功后激活供应商任务 |
| `GET` | `/device/v2/conversations/{id}` | 查询通话权威状态 |
| `POST` | `/device/v2/conversations/{id}/heartbeat` | 通话心跳 |
| `POST` | `/device/v2/conversations/{id}/stop` | 停止通话 |
| `POST` | `/device/v2/conversations/{id}/turns:batch` | 补传最终字幕 |
新 EchoEar 固件的 `voice_runtime=volc_hardware_agent` 时不得调用这组 RTC 创建/激活接口。
### 6.3 录音上传(v2 公共媒体能力)
| 方法 | 路径 | 作用 |
|---|---|---|
| `POST` | `/device/v2/conversations/{id}/upload-batches` | 创建分片上传批次 |
| `GET` | `/device/v2/upload-batches/{id}` | 查询上传校验状态 |
| `POST` | `/device/v2/upload-batches/{id}/complete` | 通知平台分片已上传 |
| `POST` | `/device/v2/media-segments/{id}/local-delete-ack` | 回报本地密文已删除 |
只有 `ACTIVE` 设备且本次隐私策略允许时才能使用;没有录音授权不得采集、缓存或上传 PCM。
## 7. 火山硬件智能体接口(v3)
### 7.1 申请火山一机一密
```http
POST /device/v3/hardware-agent-credentials:provision
Idempotency-Key: 2ee3b675-59c0-4d93-a189-6db7f8db3750
Cache-Control: no-store
Content-Type: application/json
```
状态要求:`BOUND`、`PROVISIONING` 或 `ACTIVE`。
请求:
```json
{
"hardware_id": "echoear-b43a451634a8"
}
```
`hardware_id` 必须来自稳定硬件身份(如 eFuse/Chip ID 的规范化值),设备重启、升级和恢复出厂后保持不变。
成功响应:
```json
{
"request_id": "req-...",
"data": {
"instance_id": "<instance-id>",
"product_key": "<product-key>",
"device_name": "echoear-...",
"hardware_id": "echoear-b43a451634a8",
"device_secret": "<volc-device-secret>",
"credential_ref": "<non-secret-ref>",
"credential_version": 1,
"credential_source": "platform_dynamic_registration",
"issued_at": "2026-09-03T08:00:00Z"
}
}
```
安全要求:
- 响应带 `Cache-Control: no-store` 和 `Pragma: no-cache`。
- `device_secret` 只写入设备安全存储,不进入普通配置文件、串口日志、崩溃转储或平台遥测。
- `credential_source=platform_stored` 表示幂等返回平台已保存的凭据,不代表重新消耗 License。
- `hardware_id` 与已注册记录不一致时返回冲突,不允许覆盖原凭据。
常见错误:
| HTTP | 错误码 | 处理 |
|---|---|---|
| `400` | `VALIDATION_FAILED` | 修正 `hardware_id` 或幂等键 |
| `409` | `HARDWARE_AGENT_BINDING_REQUIRED` | 先在平台绑定 Bot/产品配置 |
| `409` | `HARDWARE_AGENT_PRODUCT_MISMATCH` | 运维核对实例、产品和设备绑定 |
| `409` | `HARDWARE_AGENT_HARDWARE_MISMATCH` | 停止自动重试,核对硬件身份 |
| `409` | `DEVICE_ACTIVATION_REQUIRED` | 先完成用户绑定,进入 `BOUND` |
| `503` | `HARDWARE_AGENT_PROVISIONING_UNAVAILABLE` | 平台未配置动态注册;联系运维 |
| `503` | `HARDWARE_AGENT_PROVISIONING_FAILED` | 使用原幂等键、原正文和新签名退避重试;由 `request_id` 查火山原始错误 |
### 7.2 创建本次硬件智能体会话
```http
POST /device/v3/agent-sessions
Idempotency-Key: d7f08e7a-9e1d-455f-b8ec-a745a34d8790
Content-Type: application/json
```
状态要求:仅 `ACTIVE`。每次聊天创建一个新会话,不复用上一通响应。
请求示例:
```json
{
"device_time": "2026-09-03T08:05:00Z",
"monotonic_ms": 123456,
"capabilities": {
"hardware_agent": true,
"sdk_version": "1.0.0-echoear-platform.1",
"transports": ["low_load"],
"input_audio_format": "pcm_s16le_16000_mono",
"output_audio_format": "pcm_s16le_16000_mono",
"server_event_callback": false,
"transcript_upload": true,
"dynamic_session_config": true,
"credential_rotation": true,
"near_pcm_recording": false,
"far_pcm_recording": false
},
"network": {
"type": "wifi",
"rssi": -52
},
"requested_goal": "open_conversation",
"config_revision": 1
}
```
约束:
- `transports` 只能包含 `low_load`、`high_quality`,至少一个。
- `transcript_upload`、`dynamic_session_config`、`credential_rotation` 必须为 `true`。
- EchoEar 当前使用 `low_load`;`server_event_callback=false` 时设备事件和最终字幕上传是权威回流。
- `config_revision` 必须使用最近一次 `/device/v2/config` 返回的版本。
成功响应核心字段:
```json
{
"request_id": "req-...",
"data": {
"conversation_id": "<uuid>",
"status": "SESSION_READY",
"runtime_type": "volc_hardware_agent",
"runtime": {
"provider": "volcengine",
"bot_id": "<bot-id>",
"transport": "low_load",
"provider_session_ref": "<provider-session-ref>",
"session_nonce": "<memory-only-secret>",
"config_digest": "<sha256-hex>",
"wait_for_session_update": true
},
"session_config": {
"dialog_id": "<uuid>",
"system_prompt": "<本次冻结的人设提示>",
"persona_tone": "<语气>",
"authorized_context": "<已授权背景/近期摘要/长期记忆>",
"prompt_bundle_version": "<version>"
},
"privacy": {
"retention_mode": "transcript_only",
"record_near": false,
"record_far": false,
"upload_allowed": false
},
"session": {
"engine_mode": "volc_hardware_agent",
"heartbeat_interval_sec": 10,
"allocation_expires_at": "2026-09-03T08:10:00Z"
}
}
}
```
设备处理顺序:
1. 只在运行内存保存 `session_nonce`。
2. 使用预配取得的一机一密连接响应指定的 Bot 和传输模式。
3. 以 `dialog_id` 作为本次业务对话标识。
4. 在开启麦克风前通过 SDK 动态配置或 WebSocket `session.update` 注入 `system_prompt`、`persona_tone` 和 `authorized_context`。
5. 收到火山配置已应用确认后,再上报 `session_started`。
`tool_scope_token` 如存在也只能保存在运行内存,并且仅用于响应指定的 `tool_url` 和本次会话。
### 7.3 上报会话事件
```http
POST /device/v3/agent-sessions/{conversation_id}/events
Content-Type: application/json
```
开始事件:
```json
{
"event_id": "evt-start-0001",
"sequence_no": 1,
"event_type": "session_started",
"occurred_at": "2026-09-03T08:05:03Z",
"provider_session_ref": "<response-runtime-provider-session-ref>",
"session_nonce": "<response-runtime-session-nonce>",
"provider_connection_ref": "<actual-volc-connection-ref>",
"bot_id": "<response-runtime-bot-id>",
"transport": "low_load",
"config_digest": "<response-runtime-config-digest>",
"dynamic_config_applied": true
}
```
后续事件类型:
- `session_updated`:按 `heartbeat_interval_sec` 上报存活;`sequence_no` 严格递增。
- `session_ended`:正常结束。
- `session_failed`:异常结束,可附加稳定、脱敏的 `error_code`。
响应:
```json
{
"request_id": "req-...",
"data": {
"conversation_id": "<uuid>",
"status": "ACTIVE",
"action": "continue",
"duplicate": false,
"accepted_at": "2026-09-03T08:05:03Z"
}
}
```
`action=stop_session` 时,设备必须立即关闭麦克风和火山连接,然后补报 `session_ended` 或 `session_failed`。相同 `event_id` 重放时必须保持原正文一致。
### 7.4 批量上传最终话轮
```http
POST /device/v3/agent-sessions/{conversation_id}/turns:batch
Idempotency-Key: 1710b8cc-9bf2-45e8-a16d-f32c3b80cbbe
Content-Type: application/json
```
请求:
```json
{
"turns": [
{
"turn_key": "user-final-0001",
"sequence_no": 1,
"speaker": "subject",
"start_at": "2026-09-03T08:05:10Z",
"end_at": "2026-09-03T08:05:12Z",
"final_text": "今天天气怎么样?",
"interrupted": false
},
{
"turn_key": "agent-final-0002",
"sequence_no": 2,
"speaker": "agent",
"start_at": "2026-09-03T08:05:12Z",
"end_at": "2026-09-03T08:05:15Z",
"final_text": "我来帮您看看。",
"interrupted": false
}
]
}
```
只上传最终字幕;不得上传增量识别结果。单批 1—500 条,`speaker` 只能为 `subject` 或 `agent`。同一 `turn_key`/`sequence_no` 的重试必须保持正文完全一致。
成功响应:
```json
{
"request_id": "req-...",
"data": {
"conversation_id": "<uuid>",
"accepted_count": 2,
"duplicate_count": 0
}
}
```
## 8. App/Web 的设备接口
以下接口使用用户 OIDC Bearer Token,不使用设备 HMAC:
| 方法 | 路径 | 作用 |
|---|---|---|
| `POST` | `/api/v1/device-claims` | 消费绑定码并绑定设备 |
| `GET` | `/api/v1/devices` | 列出当前用户可管理设备 |
| `GET` | `/api/v1/devices/{id}` | 查看设备详情 |
| `GET` | `/api/v1/devices/{id}/binding-impact` | 变更绑定前评估影响 |
| `POST` | `/api/v1/devices/{id}/binding-change` | 解绑或改绑 |
| `GET` | `/api/v1/devices/{id}/presence` | 查看在线状态 |
| `GET` | `/api/v1/devices/{id}/twin` | 查看设备影子 |
| `PATCH` | `/api/v1/devices/{id}/twin` | 修改期望配置;必须携带 `If-Match` |
| `GET` | `/api/v1/devices/{id}/commands` | 查看命令历史 |
| `POST` | `/api/v1/devices/{id}/commands` | 下发设备命令 |
绑定码由平台/运维为 `CLAIMABLE` 设备生成,App 只消费,不自行生成。解绑会影响设备激活状态和火山运行凭据,必须先调用 `binding-impact` 展示影响并要求用户确认。
## 9. 平台运维设备接口
所有接口要求 OIDC Bearer Token 和有效的 `X-Tenant-Id: <tenant-uuid>`,且仅独立运维账号可调用。
| 方法 | 路径 | 作用 |
|---|---|---|
| `GET` | `/api/v1/ops/devices` | 设备列表 |
| `POST` | `/api/v1/ops/devices` | 登记设备并一次性返回平台 Device Secret |
| `GET` | `/api/v1/ops/device-assignment-options` | 可选家庭/长者列表 |
| `PATCH` | `/api/v1/ops/devices/{id}` | 修改设备;必须携带 `If-Match` |
| `DELETE` | `/api/v1/ops/devices/{id}` | 吊销设备;必须携带 `If-Match` |
| `POST` | `/api/v1/ops/devices/{id}/credentials/rotate` | 轮换平台 Device Secret,只返回一次明文 |
| `POST` | `/api/v1/ops/devices/{id}/claims` | 生成一次性绑定码 |
| `POST` | `/api/v1/ops/devices/{id}/activation-transitions` | 人工迁移激活状态 |
| `GET` | `/api/v1/ops/devices/{id}/activation-transitions` | 查询激活迁移历史 |
### 9.1 登记硬件智能体设备
```json
{
"model": "echoear",
"hardware_revision": "esp32-s3",
"minimum_security_version": "1",
"minimum_firmware_version": "1.0.0",
"household_id": "",
"subject_id": "",
"voice_runtime": "volc_hardware_agent",
"activation_state": "FACTORY"
}
```
Instance ID、Product Key、默认 Bot、传输模式和最低 SDK 由平台产品配置统一持有。设备第一次预配时自动生成绑定,运维创建设备或设备自身都不能覆盖这些字段。
响应中的 `data.credential.secret` 只显示一次。运维系统应立即通过受控注入工具写入设备安全区,不得复制到工单、聊天记录或普通数据库字段。
### 9.2 人工迁移激活状态
```json
{
"target_state": "CLAIMABLE",
"reason": "工厂质检通过",
"expected_revision": 1
}
```
并发控制:列表/详情返回的设备版本对应 `ETag`;修改和删除使用 `If-Match`。激活迁移单独使用 `expected_revision`。版本不一致返回 `412 VERSION_CONFLICT` 或 `409 DEVICE_ACTIVATION_CONFLICT`,刷新后再决定,不能强制覆盖。
## 10. 幂等、重试和本地持久化
| 数据 | 本地策略 |
|---|---|
| 平台 Device Secret | 安全区长期保存;轮换时原子替换 |
| 火山 Device Secret | 安全区长期保存;不要每次启动都预配 |
| `session_nonce` | 仅本次通话运行内存 |
| `tool_scope_token` | 仅本次通话运行内存 |
| 最终字幕待上传队列 | 授权允许时加密持久化,成功确认后删除 |
| 增量字幕/原始音频 | 默认不长期保存 |
重试规则:
- `Idempotency-Key` 长度 16—128 字符。
- HTTP 超时或 `retryable=true`:原业务正文 + 原幂等键 + 新 HMAC nonce/signature,指数退避。
- `401`:先检查平台 Device Secret、时钟、请求 URI、正文摘要和 Base64URL 编码,不要盲目轮换密钥。
- `409 IDEMPOTENCY_KEY_REUSED`:同一幂等键配了不同正文;这是客户端错误。
- `HARDWARE_SESSION_PROOF_INVALID`:丢弃当前会话内存数据,重新创建新会话,不尝试其他设备的 nonce。
- `action=stop_session`:优先停止音频和供应商连接,再做结束确认及本地队列收尾。
## 11. EchoEar 固件接入清单
出厂只写入:
- 平台 `device_id`;
- 平台 Device Secret;
- `hardware_id`;
- API Base URL 和服务器信任锚;
- 固件签名信任锚。
首次绑定后设备取得并安全保存:
- 火山 `instance_id`;
- 火山 `product_key`;
- 火山 `device_name`;
- 火山 Device Secret;
- `credential_version`。
绝不写入设备:
- 火山 Product Secret;
- 平台数据库主密钥/KMS 密钥;
- 其他设备的 Device Secret;
- 固定的会话 nonce 或工具 Token;
- 人设、长期记忆和聊天记录的永久明文副本。
会话侧必须做到:
- 每通先请求平台会话配置,再连接火山;
- 人设、近期摘要和长期记忆以本次 `session_config` 为准;
- 动态配置确认前不打开麦克风;
- 设备事件和最终字幕可靠回流平台;
- 平台下发 `stop_session` 时可在一个心跳周期内停止。
## 12. 当前实现边界
- 当前 API 已实现平台 HMAC、激活状态机、火山动态注册、v3 会话控制面、事件回流和最终字幕上传。
- 设备音频是设备直连火山,平台不代理每一帧音频;平台通过设备事件和字幕回流维护会话权威状态。
- Windows/EchoEar 客户端仍需按目标 SDK 完成真实音频 HAL、火山协议、断线重连和安全存储验收。
- `/device/v3/hardware-agent-license:ack` 处理器代码虽曾存在,但没有路由、没有生产契约,也不应由设备调用。
- 完整字段定义以 [`openapi/openapi.yaml`](../openapi/openapi.yaml) 为机器可读基准;本文件是设备研发和联调的实施说明。
## 13. 联调最小验收用例
1. 错误平台签名返回 `401`,日志不泄露设备是否存在。
2. 同 nonce 重放被拒绝;相同业务幂等键配新 nonce 可安全重试。
3. `CLAIMABLE` 设备不能预配,`BOUND` 设备可以预配但不能创建会话。
4. 首次预配成功取得火山一机一密;重复预配不重复注册。
5. Product Secret 在设备固件、NVS、日志和网络响应中均不存在。
6. `ACTIVE` 设备创建 v3 会话并得到正确 Bot ID、人设和授权上下文。
7. 未确认动态配置的 `session_started` 被拒绝。
8. 正常开始、心跳、最终字幕、结束全部回流平台。
9. 平台返回 `stop_session` 后设备关闭连接并补报结束。
10. 火山 License 额度耗尽时预配失败,扩容后使用原幂等请求可恢复;设备无需安装 License 或 ACK。
+68 -3
View File
@@ -1,5 +1,21 @@
# 设备接入与启动配置联调说明 # 设备接入与启动配置联调说明
## 设备激活状态机
平台从V50开始独立维护 `activation_state`,不再用通用的 `device.status` 代替激活生命周期:
`FACTORY → CLAIMABLE → BOUND → PROVISIONING → ACTIVE → LOST/REVOKED`
- `FACTORY`:只有设备身份和平台HMAC凭据,生产质检完成后由运维放行。
- `CLAIMABLE`:可以生成一次性绑定码;设备可同步基础配置,但不能发起对话。
- `BOUND`:已经绑定家庭和长者,可以调用火山设备凭据预配接口。
- `PROVISIONING`:平台正在代设备调用火山 DynamicRegister 并保存一机一密;动态注册成功、有效绑定和设备凭据就绪后进入 `ACTIVE`。火山 License 是产品侧动态注册额度,不是设备安装包,也没有设备回执。
- `ACTIVE`:允许创建和上报对话。硬件智能体会话接口只接受该状态。
- `LOST`:设备身份认证、对话和火山运行凭据立即停用;核验归属和硬件身份后可恢复到 `ACTIVE`。
- `REVOKED`:终态。设备平台凭据、火山凭据、绑定和未使用绑定码全部吊销,不能恢复。
每次迁移都会增加 `activation_revision` 并写入 `device_activation_transition`。运维人工迁移接口为 `POST /api/v1/ops/devices/{id}/activation-transitions`,请求必须携带当前 `expected_revision`;用户认领、设备预配和删除由对应业务接口自动迁移,不应由客户端伪造状态。
## 1. 当前接口 ## 1. 当前接口
| 方法 | 路径 | 作用 | | 方法 | 路径 | 作用 |
@@ -47,13 +63,21 @@ X-Device-Content-SHA256原值
云端先核验时间窗、正文摘要和签名,再使用Redis `SET NX`登记随机数。相同设备和随机数只能使用一次;Redis或数据库不可用时设备鉴权采用fail-closed并返回可重试的503,不绕过身份验证。 云端先核验时间窗、正文摘要和签名,再使用Redis `SET NX`登记随机数。相同设备和随机数只能使用一次;Redis或数据库不可用时设备鉴权采用fail-closed并返回可重试的503,不绕过身份验证。
本地环境使用静态HMAC密钥模拟联调。量产采用每设备独立HMAC时,密钥应写入硬件安全区并通过Secrets/KMS CSI受控挂载到云端;如硬件与PKI条件允许,仍可进一步切换每设备mTLS证书。不能把本地合成密钥烧录到真实设备。 本地环境允许把既有静态HMAC密钥一次性导入数据库以兼容测试包。量产采用每设备独立HMAC时,设备侧密钥应写入硬件安全区,云端只保存主密钥加密后的密文;如硬件与PKI条件允许,仍可进一步切换每设备mTLS证书。不能把本地合成密钥烧录到真实设备。
### 2.1 逐设备生产密钥解析 ### 2.1 逐设备生产密钥解析
dev、stage和prod把设备密钥通过Docker/Kubernetes Secrets或云KMS CSI驱动挂载到API容器的受控只读目录,并设置`DEVICE_SECRET_DIRECTORY`。设备表不保存密钥正文,只保存`device-secret://<name>`引用;解析器只接受单层安全文件名,解析符号链接后仍必须位于配置根目录内,文件最大4KiB且有效密钥至少32字节。 设备表保存公开身份和`credential_ref`,`device_credential`表按设备保存AES-256-GCM加密后的HMAC密钥、独立随机nonce、密钥版本与状态,不保存明文。API使用数据库外独立托管的`DEVICE_CREDENTIAL_MASTER_KEY`解密;生产环境必须通过Docker/Kubernetes Secret或云KMS CSI挂载该主密钥。现有演示环境的`DEVICE_LOCAL_SECRET`只用于首次启动时导入缺失记录,导入不会覆盖已轮换的数据库凭据。
每次验签都按设备记录读取对应文件,因此不同设备使用不同密钥,原子替换挂载文件即可完成轮换,不需要重新构建镜像或重启API。文件缺失、过短、引用格式错误或越界时统一按无效凭据拒绝,响应与日志不暴露文件路径及密钥内容。生产启动门禁拒绝退回单一`DEVICE_LOCAL_SECRET`。 每次验签都按`credential_ref`读取当前活动数据库凭据并执行AEAD认证解密;记录缺失、已吊销、主密钥错误或密文被篡改时统一按无效凭据拒绝,响应与日志不暴露密钥内容。`device-credential`维护命令提供生命周期操作:
```text
device-credential provision <设备公开UUID>
device-credential rotate <设备公开UUID>
device-credential revoke <设备公开UUID>
```
`provision`和`rotate`生成32字节随机密钥,明文只在标准输出显示一次;数据库仅保存密文。轮换提交后旧密钥立即失效,必须通过受控渠道同步更新设备安全区。吊销后设备的所有签名请求立即失败。生产启动门禁拒绝默认主加密密钥。
## 3. 启动参数 ## 3. 启动参数
@@ -228,6 +252,8 @@ docker compose --profile simulation run --rm sim-device
| `media` | 为`SIM_CONVERSATION_ID`指定且具有当前录音授权的通话生成或读取WAV PCM16文件,直传对象存储,轮询至云端校验完成,并在`local_delete_after`到达后回报本地密文已删除;可设置`SIM_MEDIA_FILE`、`SIM_MEDIA_DURATION_MS`、`SIM_MEDIA_TRACK`、`SIM_MEDIA_VERIFY_TIMEOUT`和`SIM_MEDIA_DELETE_WAIT_MAX` | | `media` | 为`SIM_CONVERSATION_ID`指定且具有当前录音授权的通话生成或读取WAV PCM16文件,直传对象存储,轮询至云端校验完成,并在`local_delete_after`到达后回报本地密文已删除;可设置`SIM_MEDIA_FILE`、`SIM_MEDIA_DURATION_MS`、`SIM_MEDIA_TRACK`、`SIM_MEDIA_VERIFY_TIMEOUT`和`SIM_MEDIA_DELETE_WAIT_MAX` |
| `all` | 依次执行启动、完整通话、字幕、遥测和命令;录音因需要单独授权和可访问的对象存储公开地址而不自动包含 | | `all` | 依次执行启动、完整通话、字幕、遥测和命令;录音因需要单独授权和可访问的对象存储公开地址而不自动包含 |
Windows真实RTC模拟设备会读取创建通话响应中的`privacy.record_near`、`record_far`和`upload_allowed`。只有相应授权有效时,它才通过RTC SDK音频帧回调分别采集长者近端和智能体远端PCM;结束通话时下混为单声道WAV PCM16,按不超过9分钟且不超过64MiB自动分片。设备提交停止请求并等待通话进入`COMPLETED`或`FAILED`后,自动创建录音上传批次、直传短期对象地址并轮询至云端校验完成。未授权轨道不会采集,上传失败最多自动重试三次并在模拟器事件流中显示稳定错误。
例如执行联合模拟闭环: 例如执行联合模拟闭环:
```powershell ```powershell
@@ -260,6 +286,41 @@ powershell -NoProfile -ExecutionPolicy Bypass -File .\scripts\smoke-sim-firmware
脚本会验证创建、首次激活、相同幂等键重放、Worker调用、两次签名回调去重和最终`ACTIVE`状态,并断言供应商仅启动一次。脚本只使用合成数据,验证结束后按本次通话编号精确清理数据库痕迹。 脚本会验证创建、首次激活、相同幂等键重放、Worker调用、两次签名回调去重和最终`ACTIVE`状态,并断言供应商仅启动一次。脚本只使用合成数据,验证结束后按本次通话编号精确清理数据库痕迹。
## 11.5 火山硬件对话智能体 v3 控制面
设备首次准备火山连接凭据时,先使用既有平台设备签名调用`POST /device/v3/hardware-agent-credentials:provision`并提交稳定`hardware_id`。平台从产品级配置解析Instance ID、Product Key、默认Bot、传输方式和最低SDK版本,自动建立当前设备的运行绑定;再从KMS或只读Secret文件加载Product Secret,以平台生成的Device Name代设备调用火山DynamicRegister。Product Secret永不到达设备。平台把返回的`device_secret`用独立的`VOLC_HARDWARE_DEVICE_CREDENTIAL_MASTER_KEY`执行AES-256-GCM加密后保存,只在设备认证且`Cache-Control: no-store`的响应中返回给同一设备。设备使用安全芯片、系统KeyStore或Windows Safe Storage保存这份一机一密,后续用于火山WebSocket签名。
火山硬件对话智能体 License 是产品侧可用额度。平台调用 DynamicRegister 时由火山自动消耗一份有效额度并返回对应设备的一机一密;设备不会收到 License 明文,也不需要提交安装回执。当前有效路由不包含`POST /device/v3/hardware-agent-license:ack`,预配响应也不包含`license`字段。平台可以维护额度监控或影子账本,但不得把自建池作为设备运行凭据来源。额度耗尽或到期时,应根据火山动态注册错误告警并扩容,扩容后重试原预配请求。
同一设备和Hardware ID重复调用时,平台解密返回既有一机一密,不重复注册;Hardware ID、产品或实例不一致时返回409,必须走受控轮换而不是覆盖。Product Secret、Device Secret明文不得进入日志、审计详情、错误响应、通话记录或测试报告。自动回归的mock密钥仅允许在隔离测试进程中注入,不得写入正式设备缓存。
设备期望配置中的`voice_runtime`为`volc_hardware_agent`时,不再调用`POST /device/v2/conversations`创建RTC房间。首次预配后设备要重新拉取配置版本,并读取平台发布的`minimum_sdk_version`执行本地预检;云端会以产品默认绑定中的同一版本再次门禁,过低时返回HTTP 426和`HARDWARE_AGENT_SDK_UPDATE_REQUIRED`。每次用户开始聊天必须先调用`POST /device/v3/agent-sessions`,不得复用上一次响应。创建请求必须声明SDK版本、输入/输出音频格式、传输模式、可靠字幕队列、动态会话配置、凭证轮换以及近/远端PCM录制能力。平台会在同一事务中冻结这些能力、有效授权、已发布人设、档案、最近三次已完成通话摘要和最多五条人工确认的长期记忆,并返回:
- `runtime.bot_id`、传输模式和仅本次有效的`session_nonce`;
- `session_config.dialog_id`,必须作为火山会话的业务对话标识;
- `system_prompt`、`persona_tone`和`authorized_context`,必须在开始发送音频前通过SDK动态配置或标准WebSocket的`session.update`注入;
- `config_digest`,用于设备日志和平台诊断比对,不得当作供应商凭据;
- 可选的`tool_url`和`tool_scope_token`,只允许配置为本次火山会话的只读记忆工具;令牌绑定本次通话和供应商会话引用,设备不得持久化或转给其他会话;
- 本次冻结的隐私策略和会话存活间隔。
设备收到火山`session.created`后必须先应用本次`session.update`,收到确认后才向`POST /device/v3/agent-sessions/{id}/events`提交`session_started`。开始事件必须同时提交火山实际会话关联号、`bot_id`、传输模式、平台原样返回的`config_digest`和`dynamic_config_applied=true`;任何一项不一致都不会进入`ACTIVE`,此时不得打开麦克风。通话期间按响应中的心跳间隔提交`session_updated`;响应`action=continue`时保持会话,收到`action=stop_session`时必须立即关闭火山连接,然后提交`session_ended`或`session_failed`。这样,家庭撤回授权、运营停话和设备吊销即使发生在设备直连火山期间,也会收敛到设备执行。
平台不是靠轮询火山控制台才知道通话状态:v3设备事件在同一事务内更新权威通话状态,既有数据库触发器随`ACTIVE/COMPLETED/FAILED`写入最小`web_event`,已登录SaaS页面通过认证SSE收到`conversation.started/completed/failed`并刷新列表。`session_updated`只续租并更新硬件事件水位,不向家庭页面逐心跳广播,避免每10秒放大为整页查询;运维诊断仍可读取最后事件时间和序号。
录音能力采用保守门禁:只有创建请求同时声明`near_pcm_recording=true`和`far_pcm_recording=true`时,平台才可能依据既有授权返回`transcript_and_audio`。缺任一能力都会把本次会话强制降级为`transcript_only`(没有字幕授权时为`none`),并把`record_near`、`record_far`、`upload_allowed`全部置为`false`,设备不得以混音或供应商未知格式冒充双轨录音。
最终字幕写入本地加密重试队列,使用`POST /device/v3/agent-sessions/{id}/turns:batch`上传。只上传用户最终识别事件和智能体`response.audio_transcript.done`,不得长期保存或上传增量字幕;同一`turn_key`或`sequence_no`重放必须保持正文完全一致。高质量传输可额外配置火山服务端状态/字幕回调作为旁路证据;低负载传输按火山当前约束没有该服务端回调,因此设备事件与终稿上传是强制能力,创建请求必须声明`transcript_upload=true`。
Product Secret只保存在平台KMS/Secret挂载中;平台注册结果中的`device_secret`在平台数据库加密保存,并只下发给对应设备的安全区或供应商SDK凭据存储。普通会话响应和日志不得包含该值。当前Windows标准WebSocket测试程序覆盖平台代理注册和真实低负载协议;高质量方案的火山硬件SDK二进制和板级音频HAL仍必须由目标芯片工程集成后执行真机验收。
本地控制面闭环可执行:
```powershell
powershell -NoProfile -ExecutionPolicy Bypass -File .\scripts\smoke-hardware-agent.ps1
```
脚本先证明过低SDK被HTTP 426拒绝、未完成动态配置不能激活,再验证产品默认Bot绑定、每次会话上下文预取、录音能力安全降级、开始事件、最终字幕、平台停话、`stop_session`下发、设备结束确认、能力快照和数据库状态收敛,并精确清理临时会话及绑定。
## 12. 错误处理 ## 12. 错误处理
| HTTP状态 | 错误码 | 处理建议 | | HTTP状态 | 错误码 | 处理建议 |
@@ -269,6 +330,8 @@ powershell -NoProfile -ExecutionPolicy Bypass -File .\scripts\smoke-sim-firmware
| 503 | `DEVICE_AUTHENTICATION_UNAVAILABLE` | 退避后使用新随机数重试,不关闭鉴权 | | 503 | `DEVICE_AUTHENTICATION_UNAVAILABLE` | 退避后使用新随机数重试,不关闭鉴权 |
| 503 | `DEVICE_CONFIG_UNAVAILABLE` / `DEVICE_BOOTSTRAP_FAILED` | 退避重试并保留上一次有效配置 | | 503 | `DEVICE_CONFIG_UNAVAILABLE` / `DEVICE_BOOTSTRAP_FAILED` | 退避重试并保留上一次有效配置 |
| 503 | `RUNTIME_CONFIG_UNAVAILABLE` | 生产环境没有命中活动语音路由;停止创建通话,由运营发布或恢复目标范围配置 | | 503 | `RUNTIME_CONFIG_UNAVAILABLE` | 生产环境没有命中活动语音路由;停止创建通话,由运营发布或恢复目标范围配置 |
| 426 | `HARDWARE_AGENT_SDK_UPDATE_REQUIRED` | 当前火山硬件智能体SDK低于设备绑定门禁;停止创建会话并升级固件/SDK |
| 503 | `HARDWARE_AGENT_PROVISIONING_FAILED` | 火山动态注册失败;用原幂等键、原正文和新签名退避重试,并由运维根据`request_id`检查产品License额度、有效期和火山原始错误 |
| 403 | `CONSENT_REQUIRED` | 不重试,等待有效授权 | | 403 | `CONSENT_REQUIRED` | 不重试,等待有效授权 |
| 403 | `TRANSCRIPT_STORAGE_NOT_ALLOWED` | 停止补传,等待留存策略或授权恢复,不得本地绕过 | | 403 | `TRANSCRIPT_STORAGE_NOT_ALLOWED` | 停止补传,等待留存策略或授权恢复,不得本地绕过 |
| 409 | `DEVICE_NOT_BOUND` / `CONVERSATION_ALREADY_ACTIVE` | 修复绑定或查询已有通话 | | 409 | `DEVICE_NOT_BOUND` / `CONVERSATION_ALREADY_ACTIVE` | 修复绑定或查询已有通话 |
@@ -278,6 +341,8 @@ powershell -NoProfile -ExecutionPolicy Bypass -File .\scripts\smoke-sim-firmware
| 409 | `CONFIG_VERSION_CONFLICT` | 先重新获取设备配置再尝试创建新通话 | | 409 | `CONFIG_VERSION_CONFLICT` | 先重新获取设备配置再尝试创建新通话 |
| 409 | `PROVIDER_EVENT_CONFLICT` | 供应商事件编号复用且正文不同,转人工排查 | | 409 | `PROVIDER_EVENT_CONFLICT` | 供应商事件编号复用且正文不同,转人工排查 |
| 409 | `CONVERSATION_STATE_CONFLICT` | 查询云端当前通话状态,不要强制覆盖状态机 | | 409 | `CONVERSATION_STATE_CONFLICT` | 查询云端当前通话状态,不要强制覆盖状态机 |
| 401 | `HARDWARE_SESSION_PROOF_INVALID` | 丢弃本地会话,不得尝试其他设备或其他会话的nonce;重新创建会话 |
| 409 | `HARDWARE_AGENT_EVENT_CONFLICT` | 保留原事件和单调序号,查询当前会话状态并人工核对,不得覆盖 |
| 409 | `TRANSCRIPT_TURN_CONFLICT` | 保留本地终稿和冲突明细,禁止覆盖云端终稿,转入人工对账 | | 409 | `TRANSCRIPT_TURN_CONFLICT` | 保留本地终稿和冲突明细,禁止覆盖云端终稿,转入人工对账 |
| 409 | `TELEMETRY_EVENT_CONFLICT` | 保留冲突事件并停止推进本地确认点,人工核对事件键和单调序号 | | 409 | `TELEMETRY_EVENT_CONFLICT` | 保留冲突事件并停止推进本地确认点,人工核对事件键和单调序号 |
| 404 | `TELEMETRY_CONVERSATION_NOT_FOUND` | 删除错误通话引用,不得改用任意长者编号重试 | | 404 | `TELEMETRY_CONVERSATION_NOT_FOUND` | 删除错误通话引用,不得改用任意长者编号重试 |
+857
View File
@@ -0,0 +1,857 @@
# 设备注册到聊天记录上传接口文档
## 1. 设备请求平台时的请求头
运营后台和用户 App 调用 `/api/v1/*` 时使用:
```http
Authorization: Bearer <用户访问令牌>
```
设备调用 `/device/*` 时使用以下 HMAC 请求头:
| 请求头 | 中文含义 | 计算方式 |
|---|---|---|
| `X-Device-Id` | 平台设备编号 | 平台登记设备时返回的 `credential.device_id` |
| `X-Device-Timestamp` | 请求时间 | 当前 UTC 时间,RFC3339 格式,例如 `2026-09-05T09:20:00Z` |
| `X-Device-Nonce` | 防重放随机数 | 每个请求重新生成,长度 16–128 字符,不得重复 |
| `X-Device-Content-SHA256` | 请求正文摘要 | `hex_lower(SHA256(rawBody))`;GET 使用空字节正文 |
| `X-Device-Signature` | 请求签名 | `base64url_no_padding(HMAC-SHA256(platformDeviceSecret, canonical))` |
| `Idempotency-Key` | 业务幂等键 | 仅指定接口需要;同一次请求重试必须保持不变,长度 16–128 字符 |
| `Content-Type` | 正文类型 | 有 JSON 正文时为 `application/json` |
签名原文 `canonical`:
```text
{大写HTTP方法}\n{请求路径和查询参数}\n{X-Device-Timestamp}\n{X-Device-Nonce}\n{X-Device-Content-SHA256}
```
计算示例:
```text
body = UTF8('{"hardware_id":"echoear-a1b2c3"}')
bodySha256 = hex_lower(SHA256(body))
timestamp = "2026-09-05T09:20:00Z"
nonce = "d8b81b916ddd4f4584a80a82"
canonical =
"POST\n" +
"/device/v3/hardware-agent-credentials:provision\n" +
timestamp + "\n" +
nonce + "\n" +
bodySha256
signature = base64url_no_padding(
HMAC-SHA256(platformDeviceSecret, UTF8(canonical))
)
```
注意:
- 摘要必须针对实际发送的原始字节计算,签名后不能再次格式化 JSON。
- 查询参数属于签名内容,例如 `/device/v2/commands?limit=10`。
- GET 空正文 SHA-256 固定为 `e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855`。
- `platformDeviceSecret` 是平台注册时返回的设备密钥,不是火山 `Product Secret`,也不是火山 `device_secret`。
- 平台统一响应:成功为 `{"request_id":"...","data":...}`;失败为 `{"request_id":"...","error":...}`。
## 2. 完整流程
| 顺序 | 调用方 | 动作 | 接口/地址 | 结果 |
|---:|---|---|---|---|
| 1 | 运营平台 | 登记设备 | `POST /api/v1/ops/devices` | 返回平台设备 ID、平台设备密钥和绑定码 |
| 2 | 用户 App | 绑定设备 | `POST /api/v1/device-claims` | 状态变为 `BOUND` |
| 3 | 设备 | 获取火山运行凭据 | `POST /device/v3/hardware-agent-credentials:provision` | 平台动态注册火山设备,消耗一个 License,状态变为 `ACTIVE` |
| 4 | 设备 | 检查固件 | `GET /device/v2/bootstrap` | 得到 `firmware_compatible` |
| 5 | 设备 | 获取设备配置 | `GET /device/v2/config` | 得到配置版本和运行时配置 |
| 6 | 设备 | 判断是否继续 | `firmware_compatible=false` 时停止聊天并进入 OTA 流程 | 只有兼容固件可以继续 |
| 7 | 设备 | 创建对话会话 | `POST /device/v3/agent-sessions` | 获取 Bot ID、系统提示词、人设和授权记忆 |
| 8 | 设备 | 连接火山 | 火山 WebSocket + `session.update` | 注入本次人设并建立 16 kHz PCM 对话 |
| 9 | 设备 | 上报会话开始 | `POST /device/v3/agent-sessions/{id}/events` | 平台会话变为 `ACTIVE` |
| 10 | 设备 | 上传最终聊天文本 | `POST /device/v3/agent-sessions/{id}/turns:batch` | 用户和智能体最终文本入库 |
| 11 | 设备 | 上报会话结束 | 同一个 events 接口,`session_ended` | 平台会话变为 `COMPLETED` |
激活状态:
```text
FACTORY → CLAIMABLE → BOUND → PROVISIONING → ACTIVE → LOST/REVOKED
```
设备重置流程:
```text
停止语音 → 上传剩余聊天记录 → 上报session_ended → POST /device/v3/factory-reset
→ 平台解除绑定并暂停火山凭据 → 设备按local_actions清理 → 重启进入CLAIMABLE
```
## 3. 接口
### 3.1 登记设备
地址:
```http
POST /api/v1/ops/devices
```
输入:
| 字段 | 类型 | 必填 | 中文含义 |
|---|---|---:|---|
| `model` | string | 是 | 设备型号 |
| `hardware_revision` | string | 否 | 硬件版本 |
| `minimum_security_version` | string | 否 | 最低安全版本 |
| `minimum_firmware_version` | string | 否 | 最低固件版本 |
| `household_id` | UUID | 否 | 直接绑定的家庭编号;走绑定码时不传 |
| `subject_id` | UUID | 否 | 直接绑定的服务对象编号;走绑定码时不传 |
| `voice_runtime` | enum | 是 | 传 `volc_hardware_agent` |
| `hardware_agent_bot_id` | string | 否 | 兼容字段;当前方案不传,由平台产品配置决定 |
| `hardware_agent_transport` | enum | 否 | `low_load` 或 `high_quality`;建议由平台配置 |
| `hardware_agent_minimum_sdk_version` | string | 否 | 最低硬件智能体 SDK 版本 |
| `activation_state` | enum | 是 | `FACTORY`、`CLAIMABLE` 或 `BOUND`;一般传 `CLAIMABLE` |
输出:
| 字段 | 类型 | 中文含义 |
|---|---|---|
| `device.id` | UUID | 平台设备编号 |
| `device.status` | string | 设备业务状态 |
| `device.activation_state` | enum | 激活状态 |
| `device.activation_revision` | integer | 激活状态版本 |
| `device.model` | string | 设备型号 |
| `device.hardware_revision` | string/null | 硬件版本 |
| `device.minimum_security_version` | string/null | 最低安全版本 |
| `device.minimum_firmware_version` | string/null | 最低固件版本 |
| `device.voice_runtime` | enum | 语音运行时 |
| `device.version` | integer | 设备记录版本 |
| `credential.device_id` | UUID | 平台设备编号 |
| `credential.secret` | string | 设备调用平台所用的 HMAC 密钥,仅明文返回一次 |
| `credential.key_version` | integer | 平台设备密钥版本 |
| `credential.issued_at` | datetime | 密钥签发时间 |
| `claim.claim_code` | string/null | 一次性绑定码 |
| `claim.issued_at` | datetime/null | 绑定码签发时间 |
| `claim.expires_at` | datetime/null | 绑定码过期时间 |
示例:
```http
POST /api/v1/ops/devices
Authorization: Bearer <ops-token>
Content-Type: application/json
{
"model": "echoear-esp32s3",
"hardware_revision": "v1.0",
"minimum_firmware_version": "1.0.0",
"voice_runtime": "volc_hardware_agent",
"activation_state": "CLAIMABLE"
}
```
```json
{
"request_id": "req-register-001",
"data": {
"device": {
"id": "10000000-0000-0000-0000-000000000512",
"status": "registered",
"activation_state": "CLAIMABLE",
"activation_revision": 1,
"model": "echoear-esp32s3",
"voice_runtime": "volc_hardware_agent",
"version": 1
},
"credential": {
"device_id": "10000000-0000-0000-0000-000000000512",
"secret": "<platform-device-secret>",
"key_version": 1,
"issued_at": "2026-09-05T09:00:00Z"
},
"claim": {
"claim_code": "ABCD-EFGH-IJKL-MNOP",
"issued_at": "2026-09-05T09:00:00Z",
"expires_at": "2026-09-06T09:00:00Z"
}
}
}
```
### 3.2 用户绑定设备
地址:
```http
POST /api/v1/device-claims
```
输入:
| 字段 | 类型 | 必填 | 中文含义 |
|---|---|---:|---|
| `claim_code` | string | 是 | 一次性绑定码,不能填写设备 UUID |
| `household_id` | UUID | 是 | 家庭编号 |
| `subject_id` | UUID | 是 | 设备服务对象编号 |
输出:
| 字段 | 类型 | 中文含义 |
|---|---|---|
| `binding_id` | UUID | 设备绑定关系编号 |
| `device_id` | UUID | 平台设备编号 |
| `household_id` | UUID | 家庭编号 |
| `subject_id` | UUID | 服务对象编号 |
| `bound_at` | datetime | 绑定时间 |
示例:
```http
POST /api/v1/device-claims
Authorization: Bearer <user-token>
Content-Type: application/json
{
"claim_code": "ABCD-EFGH-IJKL-MNOP",
"household_id": "10000000-0000-0000-0000-000000000211",
"subject_id": "10000000-0000-0000-0000-000000000411"
}
```
```json
{
"request_id": "req-bind-001",
"data": {
"binding_id": "10000000-0000-0000-0000-000000000611",
"device_id": "10000000-0000-0000-0000-000000000512",
"household_id": "10000000-0000-0000-0000-000000000211",
"subject_id": "10000000-0000-0000-0000-000000000411",
"bound_at": "2026-09-05T09:10:00Z"
}
}
```
### 3.3 占用 License 并获取火山设备凭据
地址:
```http
POST /device/v3/hardware-agent-credentials:provision
```
输入:
| 字段 | 位置 | 类型 | 必填 | 中文含义 |
|---|---|---|---:|---|
| `Idempotency-Key` | Header | string | 是 | 凭据申请幂等键,16–128 字符 |
| `hardware_id` | Body | string | 是 | 不随固件升级变化的物理硬件标识 |
输出:
| 字段 | 类型 | 中文含义 |
|---|---|---|
| `instance_id` | string | 火山硬件智能体实例编号 |
| `product_key` | string | 火山产品标识 |
| `device_name` | string | 火山设备名称 |
| `hardware_id` | string | 已绑定的物理硬件标识 |
| `device_secret` | string | 火山设备专属密钥,需要加密保存 |
| `credential_ref` | string | 平台内部凭据引用 |
| `credential_version` | integer | 火山设备凭据版本 |
| `credential_source` | enum | `platform_dynamic_registration` 为新注册,`platform_stored` 为复用已有凭据 |
| `issued_at` | datetime | 凭据签发时间 |
示例:
```http
POST /device/v3/hardware-agent-credentials:provision
Idempotency-Key: provision-device-00000512-v1
Content-Type: application/json
<设备HMAC请求头>
{"hardware_id":"echoear-esp32s3-a1b2c3d4"}
```
```json
{
"request_id": "req-provision-001",
"data": {
"instance_id": "6a92945cc0c4689234aef656",
"product_key": "6a9568047dd4b23eacab9b1a",
"device_name": "dt-10000000000000000000000000000512",
"hardware_id": "echoear-esp32s3-a1b2c3d4",
"device_secret": "<volc-device-secret>",
"credential_ref": "volc-device-secret://10000000-0000-0000-0000-000000000512",
"credential_version": 1,
"credential_source": "platform_dynamic_registration",
"issued_at": "2026-09-05T09:12:00Z"
}
}
```
### 3.4 设备恢复出厂
地址:
```http
POST /device/v3/factory-reset
```
输入:
| 字段 | 类型 | 必填 | 中文含义 |
|---|---|---:|---|
| Header `Idempotency-Key` | string | 是 | 本次重置幂等键,16–128 字符;重试保持不变 |
| `reset_id` | UUID | 是 | 本次重置唯一编号 |
| `reset_counter` | integer | 是 | 设备安全存储中的单调递增重置次数 |
| `occurred_at` | datetime | 是 | 设备确认重置的 UTC 时间;不得晚于请求时间 5 分钟 |
| `trigger` | enum | 是 | `physical_long_press`、`device_menu` 或 `service_tool` |
| `firmware_version` | string | 是 | 当前固件版本 |
| `pending_turn_count` | integer | 是 | 未上传聊天记录数;必须为 `0` |
| `active_conversation_id` | UUID | 否 | 设备认为仍在运行的会话编号 |
输出:
| 字段 | 类型 | 中文含义 |
|---|---|---|
| `reset_id` | UUID | 重置编号 |
| `device_id` | UUID | 平台设备编号 |
| `activation_state` | enum | 重置后固定为 `CLAIMABLE` |
| `reset_counter` | integer | 平台接受的重置次数 |
| `binding_removed` | boolean | 是否关闭了原用户绑定 |
| `previous_binding_id` | UUID/null | 原绑定关系编号 |
| `provider_credential_status` | enum | 火山设备凭据状态,通常为 `SUSPENDED` |
| `license_action` | enum | 固定为 `RETAINED`,重置不释放火山 License |
| `ended_conversation_count` | integer | 平台强制结束的残留会话数 |
| `cancelled_command_count` | integer | 平台取消的待执行命令数 |
| `config_revision` | integer | 重置后的设备配置版本 |
| `claim.code` | string | 新的一次性绑定码 |
| `claim.issued_at` | datetime | 绑定码签发时间 |
| `claim.expires_at` | datetime | 绑定码过期时间 |
| `local_actions` | string[] | 设备收到成功响应后必须执行的本地清理动作 |
| `reset_at` | datetime | 平台完成重置的时间 |
平台同时把旧用户侧的设备状态置为离线,并推送 `device.binding.reset` 事件。设备收到成功响应后,按 `local_actions` 清理本地火山凭据、用户上下文、会话与网络配置;设备平台身份和 HMAC 密钥不得删除。
示例:
```http
POST /device/v3/factory-reset
Idempotency-Key: factory-reset-00000512-0001
Content-Type: application/json
<设备HMAC请求头>
{
"reset_id": "10000000-0000-0000-0000-000000000901",
"reset_counter": 1,
"occurred_at": "2026-09-05T09:30:00Z",
"trigger": "physical_long_press",
"firmware_version": "1.1.0",
"pending_turn_count": 0
}
```
```json
{
"request_id": "req-reset-001",
"data": {
"reset_id": "10000000-0000-0000-0000-000000000901",
"device_id": "10000000-0000-0000-0000-000000000512",
"activation_state": "CLAIMABLE",
"reset_counter": 1,
"binding_removed": true,
"previous_binding_id": "10000000-0000-0000-0000-000000000611",
"provider_credential_status": "SUSPENDED",
"license_action": "RETAINED",
"ended_conversation_count": 0,
"cancelled_command_count": 0,
"config_revision": 9,
"claim": {
"code": "QRST-UVWX-YZ23-4567",
"issued_at": "2026-09-05T09:30:01Z",
"expires_at": "2026-09-06T09:30:01Z"
},
"local_actions": [
"clear_volc_credential",
"clear_user_context",
"clear_session_state",
"clear_network_credentials",
"reboot_to_claimable"
],
"reset_at": "2026-09-05T09:30:01Z"
}
}
```
设备必须保留平台 `device_id`、平台 HMAC `secret` 和 `hardware_id`;其余用户上下文、火山凭据、会话状态和网络配置按 `local_actions` 清除。
### 3.5 检查固件版本
地址:
```http
GET /device/v2/bootstrap
```
输入:
| 字段 | 位置 | 类型 | 必填 | 中文含义 |
|---|---|---|---:|---|
| `firmware_version` | Query | string | 是 | 当前固件版本 |
| `device_time` | Query | datetime | 是 | 设备 UTC 时间,RFC3339 |
| `board_rev` | Query | string | 否 | 硬件版本 |
| `rtc_sdk_version` | Query | string | 否 | 火山 SDK 版本 |
| `codec` | Query | string | 否 | 音频格式,例如 `pcm16` |
| `sample_rate` | Query | integer | 否 | 音频采样率,当前为 `16000` |
| `network` | Query | string | 否 | 网络类型,例如 `wifi` |
输出:
| 字段 | 类型 | 中文含义 |
|---|---|---|
| `server_time` | datetime | 平台 UTC 时间 |
| `clock_skew_ms` | integer | 平台时间减设备时间的毫秒差 |
| `device.device_id` | UUID | 平台设备编号 |
| `device.status` | string | 设备业务状态 |
| `device.model` | string | 设备型号 |
| `device.hardware_revision` | string/null | 硬件版本 |
| `device.minimum_security_version` | string/null | 最低安全版本 |
| `device.minimum_firmware_version` | string/null | 最低固件版本 |
| `device.firmware_compatible` | boolean | 当前固件是否允许继续聊天 |
| `device.config_revision` | integer | 当前平台配置版本 |
| `device.binding.bound` | boolean | 是否已绑定 |
| `device.binding.household_id` | UUID/null | 家庭编号 |
| `device.binding.subject_id` | UUID/null | 服务对象编号 |
| `device.binding.subject_name` | string/null | 服务对象名称 |
示例:
```http
GET /device/v2/bootstrap?firmware_version=1.0.0&board_rev=v1.0&rtc_sdk_version=1.1.0&codec=pcm16&sample_rate=16000&network=wifi&device_time=2026-09-05T09%3A15%3A00Z
<设备HMAC请求头>
```
```json
{
"request_id": "req-bootstrap-001",
"data": {
"server_time": "2026-09-05T09:15:01Z",
"clock_skew_ms": 1000,
"device": {
"device_id": "10000000-0000-0000-0000-000000000512",
"status": "registered",
"model": "echoear-esp32s3",
"minimum_firmware_version": "1.0.0",
"firmware_compatible": true,
"config_revision": 8,
"binding": {
"bound": true,
"household_id": "10000000-0000-0000-0000-000000000211",
"subject_id": "10000000-0000-0000-0000-000000000411",
"subject_name": "测试用户"
}
}
}
}
```
### 3.6 获取设备配置
地址:
```http
GET /device/v2/config
```
输入:
| 字段 | 位置 | 类型 | 必填 | 中文含义 |
|---|---|---|---:|---|
| `If-None-Match` | Header | string | 否 | 本地配置 ETag;未变化时返回 `304` |
输出:
| 字段 | 类型 | 中文含义 |
|---|---|---|
| Response Header `ETag` | string | 当前配置 ETag |
| `device_id` | UUID | 平台设备编号 |
| `revision` | integer | 配置版本 |
| `desired_config` | object | 平台期望设备应用的配置 |
| `desired_config.voice_runtime` | enum | 语音运行时 |
| `desired_config.hardware_agent_transport` | enum | 火山传输方式 |
| `desired_config.minimum_sdk_version` | string | 最低 SDK 版本 |
| `reported_revision` | integer/null | 设备此前报告的配置版本 |
| `binding` | object | 当前绑定信息,字段与 bootstrap 相同 |
示例:
```http
GET /device/v2/config
If-None-Match: "7"
<设备HMAC请求头>
```
```json
{
"request_id": "req-config-001",
"data": {
"device_id": "10000000-0000-0000-0000-000000000512",
"revision": 8,
"desired_config": {
"voice_runtime": "volc_hardware_agent",
"hardware_agent_transport": "low_load",
"minimum_sdk_version": "1.1.0"
},
"reported_revision": 7,
"binding": {
"bound": true,
"household_id": "10000000-0000-0000-0000-000000000211",
"subject_id": "10000000-0000-0000-0000-000000000411"
}
}
}
```
### 3.7 创建会话并获取系统提示词
地址:
```http
POST /device/v3/agent-sessions
```
输入:
| 字段 | 类型 | 必填 | 中文含义 |
|---|---|---:|---|
| Header `Idempotency-Key` | string | 是 | 创建会话幂等键,16–128 字符 |
| `device_time` | datetime | 是 | 设备 UTC 时间 |
| `monotonic_ms` | integer | 否 | 开机后的单调时钟毫秒数 |
| `capabilities.hardware_agent` | boolean | 是 | 必须为 `true` |
| `capabilities.sdk_version` | string | 是 | 硬件智能体 SDK 版本 |
| `capabilities.transports` | string[] | 是 | 支持的传输方式 |
| `capabilities.input_audio_format` | string | 是 | 上行音频能力,当前为 `pcm_s16le_16000_mono` |
| `capabilities.output_audio_format` | string | 是 | 下行音频能力,当前为 `pcm_s16le_16000_mono` |
| `capabilities.server_event_callback` | boolean | 是 | 当前传 `false` |
| `capabilities.transcript_upload` | boolean | 是 | 必须为 `true` |
| `capabilities.dynamic_session_config` | boolean | 是 | 必须为 `true` |
| `capabilities.credential_rotation` | boolean | 是 | 必须为 `true` |
| `capabilities.near_pcm_recording` | boolean | 是 | 是否支持近端录音 |
| `capabilities.far_pcm_recording` | boolean | 是 | 是否支持远端录音 |
| `network.type` | string | 是 | 网络类型 |
| `network.rssi` | integer | 否 | Wi-Fi RSSI |
| `requested_goal` | string | 是 | 固定为 `open_conversation` |
| `config_revision` | integer | 是 | `/device/v2/config` 返回的配置版本 |
输出:
| 字段 | 类型 | 中文含义 |
|---|---|---|
| `conversation_id` | UUID | 平台会话编号 |
| `status` | string | 正常为 `SESSION_READY` |
| `runtime_type` | string | `volc_hardware_agent` |
| `runtime.provider` | string | 服务商,当前为 `volcengine` |
| `runtime.bot_id` | string | 本次使用的 Bot ID |
| `runtime.transport` | string | 本次传输方式 |
| `runtime.provider_session_ref` | string | 火山会话引用 |
| `runtime.session_nonce` | string | 会话证明随机值 |
| `runtime.config_digest` | string | 本次动态配置摘要 |
| `runtime.wait_for_session_update` | boolean | 是否必须先应用动态人设 |
| `session_config.dialog_id` | string | 本次对话标识 |
| `session_config.system_prompt` | string | 系统提示词/核心人设 |
| `session_config.persona_tone` | string | 人设语气和表达风格 |
| `session_config.authorized_context` | string | 授权的用户背景、近期摘要和长期记忆 |
| `session_config.prompt_bundle_version` | string | 提示词版本 |
| `session_config.tool_url` | URL/null | 工具调用地址 |
| `session_config.tool_scope_token` | string/null | 本次会话工具令牌 |
| `privacy.retention_mode` | string | 数据保留模式 |
| `privacy.record_near` | boolean | 是否允许近端录音 |
| `privacy.record_far` | boolean | 是否允许远端录音 |
| `privacy.upload_allowed` | boolean | 是否允许上传聊天记录 |
| `session.engine_mode` | string | 会话引擎模式 |
| `session.heartbeat_interval_sec` | integer | 会话状态上报间隔 |
| `session.allocation_expires_at` | datetime | 会话分配过期时间 |
示例:
```http
POST /device/v3/agent-sessions
Idempotency-Key: session-device-00000512-0001
Content-Type: application/json
<设备HMAC请求头>
{
"device_time": "2026-09-05T09:20:00Z",
"monotonic_ms": 812345,
"capabilities": {
"hardware_agent": true,
"sdk_version": "1.1.0-echoear",
"transports": ["low_load"],
"input_audio_format": "pcm_s16le_16000_mono",
"output_audio_format": "pcm_s16le_16000_mono",
"server_event_callback": false,
"transcript_upload": true,
"dynamic_session_config": true,
"credential_rotation": true,
"near_pcm_recording": false,
"far_pcm_recording": false
},
"network": {"type":"wifi","rssi":-52},
"requested_goal": "open_conversation",
"config_revision": 8
}
```
```json
{
"request_id": "req-session-001",
"data": {
"conversation_id": "10000000-0000-0000-0000-000000000801",
"status": "SESSION_READY",
"runtime_type": "volc_hardware_agent",
"runtime": {
"provider": "volcengine",
"bot_id": "botM4gWQ9A9y",
"transport": "low_load",
"provider_session_ref": "provider-session-ref",
"session_nonce": "<session-nonce>",
"config_digest": "<config-digest>",
"wait_for_session_update": true
},
"session_config": {
"dialog_id": "dialog-00000801",
"system_prompt": "你是面向长者的陪伴助手……",
"persona_tone": "率真、自然",
"authorized_context": "经授权的背景和记忆……",
"prompt_bundle_version": "persona-v12"
},
"privacy": {
"retention_mode": "transcript_only",
"record_near": false,
"record_far": false,
"upload_allowed": true
},
"session": {
"engine_mode": "hardware_agent",
"heartbeat_interval_sec": 30,
"allocation_expires_at": "2026-09-05T09:25:00Z"
}
}
}
```
### 3.8 连接火山并注入人设
地址:
```text
wss://ai-gateway.vei.volces.com/v1/realtime?bot={bot_id}&wait_for_session_update=true
```
输入:
| 字段 | 位置 | 中文含义 | 来源/计算 |
|---|---|---|---|
| `bot` | Query | Bot ID | 创建会话返回的 `runtime.bot_id` |
| `X-Auth-Type` | Header | 鉴权类型 | 固定为 `1` |
| `X-Product-Key` | Header | 火山产品标识 | provision 返回的 `product_key` |
| `X-Device-Name` | Header | 火山设备名称 | provision 返回的 `device_name` |
| `X-Random-Num` | Header | 随机数 | 每次连接重新生成 |
| `X-Timestamp` | Header | Unix 秒 | 当前时间 |
| `X-Instance-Id` | Header | 火山实例编号 | provision 返回的 `instance_id` |
| `X-Hardware-Id` | Header | 物理硬件标识 | provision 使用的 `hardware_id` |
| `X-Signature` | Header | 火山鉴权签名 | `base64(HMAC-SHA256(device_secret, canonical))` |
| `session.update` | WebSocket JSON | 动态会话配置 | 注入平台返回的人设和记忆 |
输出:
| 事件 | 中文含义 |
|---|---|
| `session.created` / `session.updated` | 火山会话创建/动态人设生效 |
| `conversation.item.input_audio_transcription.completed` | 用户最终识别文本 |
| `response.audio_transcript.done` | 智能体最终回复文本 |
| 音频数据 | 智能体返回的 PCM16 音频,按 16 kHz 单声道播放 |
| error 事件 | 火山连接或会话失败 |
示例:
```text
canonical = "auth_type=1&device_name={device_name}&random_num={random_num}&product_key={product_key}&timestamp={timestamp}&instance_id={instance_id}"
X-Signature = base64(HMAC-SHA256(device_secret, canonical))
```
```json
{
"event_id": "evt-session-update-001",
"type": "session.update",
"session": {
"object": "realtime.session",
"model": "",
"input_audio_format": "pcm16",
"output_audio_format": "pcm16",
"config": {
"ASRConfig": {"TurnDetectionMode": 0},
"LLMConfig": {
"SystemMessages": [
"<system_prompt>\n表达风格:<persona_tone>\n授权上下文:<authorized_context>\n会话标识:<dialog_id>"
]
}
},
"agent_config": {"EnableConversationStateCallback": false}
}
}
```
### 3.9 上报会话事件
地址:
```http
POST /device/v3/agent-sessions/{conversation_id}/events
```
输入:
| 字段 | 类型 | 必填 | 中文含义 |
|---|---|---:|---|
| Path `conversation_id` | UUID | 是 | 平台会话编号 |
| `event_id` | string | 是 | 事件唯一编号 |
| `sequence_no` | integer | 是 | 事件顺序号,从 1 开始递增 |
| `event_type` | enum | 是 | `session_started`、`session_updated`、`session_ended` 或 `session_failed` |
| `occurred_at` | datetime | 是 | 事件发生时间 |
| `provider_session_ref` | string | 是 | 火山会话引用 |
| `session_nonce` | string | 是 | 会话证明随机值 |
| `provider_connection_ref` | string | 启动时必填 | WebSocket 连接标识 |
| `bot_id` | string | 启动时必填 | 实际连接的 Bot ID |
| `transport` | string | 启动时必填 | 实际传输方式 |
| `config_digest` | string | 启动时必填 | 已应用的人设配置摘要 |
| `dynamic_config_applied` | boolean | 启动时必填 | 必须为 `true` |
| `error_code` | string | 失败时填写 | 失败错误码 |
输出:
| 字段 | 类型 | 中文含义 |
|---|---|---|
| `conversation_id` | UUID | 平台会话编号 |
| `status` | string | 会话状态,开始后为 `ACTIVE`,正常结束后为 `COMPLETED` |
| `action` | enum | `continue` 或 `stop_session` |
| `duplicate` | boolean | 是否为重复事件 |
| `accepted_at` | datetime | 平台接收时间 |
示例:
开始请求:
```json
{
"event_id": "device-event-start-001",
"sequence_no": 1,
"event_type": "session_started",
"occurred_at": "2026-09-05T09:20:03Z",
"provider_session_ref": "provider-session-ref",
"session_nonce": "<session-nonce>",
"provider_connection_ref": "ws-connection-001",
"bot_id": "botM4gWQ9A9y",
"transport": "low_load",
"config_digest": "<config-digest>",
"dynamic_config_applied": true
}
```
结束示例:
```json
{
"event_id": "device-event-end-001",
"sequence_no": 2,
"event_type": "session_ended",
"occurred_at": "2026-09-05T09:25:00Z",
"provider_session_ref": "provider-session-ref",
"session_nonce": "<session-nonce>"
}
```
响应示例:
```json
{
"request_id": "req-event-001",
"data": {
"conversation_id": "10000000-0000-0000-0000-000000000801",
"status": "ACTIVE",
"action": "continue",
"duplicate": false,
"accepted_at": "2026-09-05T09:20:03Z"
}
}
```
### 3.10 上传聊天记录
地址:
```http
POST /device/v3/agent-sessions/{conversation_id}/turns:batch
```
输入:
| 字段 | 类型 | 必填 | 中文含义 |
|---|---|---:|---|
| Path `conversation_id` | UUID | 是 | 平台会话编号 |
| Header `Idempotency-Key` | string | 是 | 本批记录幂等键,16–128 字符 |
| `turns` | array | 是 | 最终聊天记录数组,每批 1–500 条 |
| `turns[].turn_key` | string | 是 | 会话内唯一的轮次标识,最长 255 字符 |
| `turns[].sequence_no` | integer | 是 | 对话顺序号,会话内唯一且大于 0 |
| `turns[].speaker` | enum | 是 | 用户为 `subject`,智能体为 `agent` |
| `turns[].start_at` | datetime | 是 | 本轮开始时间 |
| `turns[].end_at` | datetime | 是 | 本轮结束时间,不早于开始时间 |
| `turns[].final_text` | string | 是 | 最终文本,最长 20000 字符 |
| `turns[].interrupted` | boolean | 是 | 本轮是否被打断 |
输出:
| 字段 | 类型 | 中文含义 |
|---|---|---|
| `conversation_id` | UUID | 平台会话编号 |
| `accepted_count` | integer | 本次新写入的记录数 |
| `duplicate_count` | integer | 本次幂等重复的记录数 |
示例:
```http
POST /device/v3/agent-sessions/10000000-0000-0000-0000-000000000801/turns:batch
Idempotency-Key: turns-conversation-00000801-0001
Content-Type: application/json
<设备HMAC请求头>
{
"turns": [
{
"turn_key": "user-final-001",
"sequence_no": 1,
"speaker": "subject",
"start_at": "2026-09-05T09:20:10Z",
"end_at": "2026-09-05T09:20:12Z",
"final_text": "今天天气怎么样?",
"interrupted": false
},
{
"turn_key": "agent-final-001",
"sequence_no": 2,
"speaker": "agent",
"start_at": "2026-09-05T09:20:12Z",
"end_at": "2026-09-05T09:20:16Z",
"final_text": "我来帮你看看。",
"interrupted": false
}
]
}
```
```json
{
"request_id": "req-turns-001",
"data": {
"conversation_id": "10000000-0000-0000-0000-000000000801",
"accepted_count": 2,
"duplicate_count": 0
}
}
```
File diff suppressed because one or more lines are too long
@@ -0,0 +1,49 @@
{
"schema_version": 1,
"diagram_type": "sequence",
"meta": {
"title": "设备从注册到聊天结束的完整流程",
"locale": "zh-CN",
"output": "docs/diagrams/device-conversation-lifecycle.html",
"viewBox": [1080, 590],
"quality_profile": "showcase",
"column_fit": "spread"
},
"participants": [
{ "id": "ops", "type": "frontend", "label": "运维平台", "sublabel": "登记 / 放行" },
{ "id": "app", "type": "frontend", "label": "用户 App", "sublabel": "扫码绑定" },
{ "id": "device", "type": "external", "label": "EchoEar", "sublabel": "ESP32-S3" },
{ "id": "saas", "type": "backend", "label": "自有 SaaS", "sublabel": "设备控制面" },
{ "id": "volcIot", "type": "cloud", "label": "火山 IoT", "sublabel": "DynamicRegister" },
{ "id": "volcAgent", "type": "cloud", "label": "火山智能体", "sublabel": "实时对话" }
],
"segments": [
{ "from": 150, "to": 226, "label": "一、平台注册与绑定" },
{ "from": 232, "to": 310, "label": "二、首次预配(仅一次)" },
{ "from": 316, "to": 424, "label": "三、每次开始聊天" },
{ "from": 430, "to": 520, "label": "四、聊天与结束" }
],
"messages": [
{ "id": "register", "from": "ops", "to": "saas", "y": 166, "label": "登记设备,生成平台 Device Secret", "variant": "emphasis" },
{ "id": "inject", "from": "saas", "to": "device", "y": 194, "label": "安全注入平台身份(无 Product Secret)", "variant": "security" },
{ "id": "claim", "from": "app", "to": "saas", "y": 222, "label": "消费绑定码:CLAIMABLE → BOUND", "variant": "emphasis" },
{ "id": "provision", "from": "device", "to": "saas", "y": 250, "label": "请求 hardware-agent-credentials:provision", "variant": "emphasis" },
{ "id": "dynamicRegister", "from": "saas", "to": "volcIot", "y": 278, "label": "DynamicRegister:新设备消耗 1 个 License", "variant": "security" },
{ "id": "active", "from": "saas", "to": "device", "y": 306, "label": "加密保存并下发一机一密 → ACTIVE", "variant": "return" },
{ "id": "start", "from": "device", "to": "saas", "y": 334, "label": "POST /device/v3/agent-sessions", "variant": "emphasis" },
{ "id": "config", "from": "saas", "to": "device", "y": 362, "label": "返回 Bot、人设、授权上下文与 nonce", "variant": "return" },
{ "id": "connect", "from": "device", "to": "volcAgent", "y": 390, "label": "一机一密建连并注入 session.update", "variant": "security" },
{ "id": "started", "from": "device", "to": "saas", "y": 418, "label": "配置确认后上报 session_started", "variant": "emphasis" },
{ "id": "audio", "from": "device", "to": "volcAgent", "y": 446, "label": "双向音频 / 打断 / 工具调用", "variant": "default" },
{ "id": "sync", "from": "device", "to": "saas", "y": 474, "label": "session_updated + 最终字幕回流", "variant": "dashed" },
{ "id": "ended", "from": "device", "to": "saas", "y": 506, "label": "断开火山并上报 ended / failed → 摘要记忆", "variant": "emphasis" }
],
"activations": [
{ "participant": "saas", "from": 160, "to": 228, "type": "backend" },
{ "participant": "saas", "from": 240, "to": 312, "type": "backend" },
{ "participant": "volcIot", "from": 268, "to": 286, "type": "cloud" },
{ "participant": "saas", "from": 324, "to": 424, "type": "backend" },
{ "participant": "volcAgent", "from": 380, "to": 512, "type": "cloud" },
{ "participant": "saas", "from": 468, "to": 514, "type": "backend" }
]
}
Binary file not shown.

After

Width:  |  Height:  |  Size: 126 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 122 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 164 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 161 KiB

@@ -0,0 +1,32 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width,initial-scale=1">
<title>Archify visual-check · device-conversation-lifecycle.html</title>
<style>
*{box-sizing:border-box}body{margin:0;padding:24px;background:#e9eef5;color:#172033;font:14px/1.5 ui-monospace,SFMono-Regular,Menlo,Consolas,monospace}header{max-width:1500px;margin:0 auto 18px}h1{margin:0 0 6px;font-size:20px}p{margin:0;color:#526176}.grid{max-width:1500px;margin:auto;display:grid;grid-template-columns:repeat(2,minmax(0,1fr));gap:18px}figure{margin:0;padding:10px;background:white;border:1px solid #c9d4e3;border-radius:12px;box-shadow:0 10px 30px rgba(15,23,42,.08)}img{display:block;width:100%;height:auto;border:1px solid #e2e8f0}figcaption{padding:9px 4px 2px;color:#526176}@media(max-width:900px){.grid{grid-template-columns:1fr}}
</style>
</head>
<body>
<header><h1>Archify visual-check</h1><p>device-conversation-lifecycle.html · automated containment pass · visual review pending</p></header>
<main class="grid">
<figure>
<img src="device-conversation-lifecycle.visual-check.1440x900.light.png" alt="light 1440 by 900">
<figcaption><strong>LIGHT</strong> · 1440×900 · containment pass</figcaption>
</figure>
<figure>
<img src="device-conversation-lifecycle.visual-check.1440x900.dark.png" alt="dark 1440 by 900">
<figcaption><strong>DARK</strong> · 1440×900 · containment pass</figcaption>
</figure>
<figure>
<img src="device-conversation-lifecycle.visual-check.2048x1320.light.png" alt="light 2048 by 1320">
<figcaption><strong>LIGHT</strong> · 2048×1320 · containment pass</figcaption>
</figure>
<figure>
<img src="device-conversation-lifecycle.visual-check.2048x1320.dark.png" alt="dark 2048 by 1320">
<figcaption><strong>DARK</strong> · 2048×1320 · containment pass</figcaption>
</figure>
</main>
</body>
</html>

Some files were not shown because too many files have changed in this diff Show More