feat: establish v0.70 platform milestone
This commit is contained in:
Codex Local
@@ -20,3 +20,5 @@ web/node_modules/
|
||||
web/dist/
|
||||
.gocache/
|
||||
test/e2e/screenshots/
|
||||
.design-evidence/
|
||||
device-simulator-windows/story-continuity-rtc/
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
# 变更记录
|
||||
|
||||
本项目使用语义化版本号。正式生产放行仍以仓库内的发布候选证据和外部门禁为准；里程碑标签只表示代码、迁移、接口和本地质量门禁形成了可追溯基线。
|
||||
|
||||
## v0.70.0-rc.1 — 第一阶段里程碑
|
||||
|
||||
第一阶段把家庭陪伴业务、真实语音接入基础能力和运营维护能力整理为第一个可运行候选版本。
|
||||
|
||||
主要内容：
|
||||
|
||||
- 家庭端、维护端和Windows语音终端模拟器形成完整入口。
|
||||
- 长者档案、分项授权、设备绑定、通话记录、故事、已确认记忆和隐私控制形成闭环。
|
||||
- 火山RTC/豆包语音、方舟内容摘要、通话内记忆工具和供应商记忆适配具备受控配置入口。
|
||||
- 运营配置、Agent质量证据、内容质量、设备维护、费用额度和诊断能力形成版本化管理闭环。
|
||||
- 数据库迁移整理至V42，并把客户账号与运营账号彻底隔离。
|
||||
- 本地质量门禁覆盖Go格式、静态检查、单元测试、隔离数据库集成测试、OpenAPI类型生成、Web类型检查与构建、Compose配置校验。
|
||||
|
||||
已知边界：
|
||||
|
||||
- 该标签是发布候选里程碑，不代表生产外部证据门禁已经通过。
|
||||
- 真实终端、真实供应商、Safari、云基础设施和试点运行仍按对应证据文档验收。
|
||||
@@ -2,6 +2,8 @@
|
||||
|
||||
这是依据《数据采集终端—云端系统研发需求规格说明书》建立的正式产品工程。当前已完成工程框架、公共能力、本地模拟语音闭环、家庭用户SaaS核心业务、字幕到人工确认记忆闭环，以及隐私、可靠设备命令、签名固件灰度、双轨录音、认证SSE状态事件流、具体资源分享、运营配置灰度发布、用量费用额度、告警处置、集成诊断和敏感数据临时提权闭环。
|
||||
|
||||
当前代码基线为`v0.70.0-rc.1`第一阶段里程碑。版本范围、主要能力和生产放行边界见[变更记录](CHANGELOG.md)。
|
||||
|
||||
## 当前已实现
|
||||
|
||||
- Go模块化单体的`api`、`callbacks`和`worker`三个进程入口。
|
||||
@@ -34,11 +36,15 @@
|
||||
- 角色化业务导航：家庭用户只加载家庭业务，运营角色可按需切换运行配置、费用额度、告警处置和联调诊断，避免无权限页面与全部工作台同时加载。
|
||||
- 家庭成员协作：成员列表、一次性邀请/撤销/接受、跨客户组织受邀账号映射、角色变更、成员移除和家庭所有权转移。
|
||||
- 通话查询API：长者维度游标分页、状态过滤、脱敏详情、冻结快照摘要和完整状态历史。
|
||||
- 长者扩展档案、稳定事实、授权记录、设备绑定、最终话轮、摘要、故事版本和已确认记忆数据库模型；V8迁移已完成空库升降级验证。
|
||||
- 长者扩展档案、稳定事实、授权记录、设备绑定、最终话轮、摘要、故事版本、已确认记忆、通话内记忆工具和内容质量反馈数据库模型；迁移已覆盖到V41并执行隔离数据库升降级验证。
|
||||
- 模型内容候选化：只使用可追溯长者话轮生成候选；人工确认和当前有效记忆授权共同控制长期记忆写入。
|
||||
- 可切换内容生成器：本地使用确定性适配器，生产可通过火山方舟OpenAI兼容接口生成结构化摘要、开放话题、故事和事实候选；模型网络调用不占用数据库事务，写入前重新核对通话状态、授权和话轮指纹，敏感内容在本地拦截后不发送外部模型。
|
||||
- 新通话在数据库层按租户、长者、当前授权、状态、敏感级别、有效期和索引状态检索已确认记忆，并把实际版本冻结到背景快照。
|
||||
- 在`story_extraction`与`personalized_memory`均有效时，新通话还会按同租户同长者带入最近3次已完成通话摘要；它是可撤回的短期连续上下文，不会绕过人工审核成为长期记忆。长者档案可配置最多8个“避免主动提及”话题，并从下一次新通话起随上下文快照生效。
|
||||
- 可切换记忆向量：本地使用1536维确定性向量，生产可通过火山方舟OpenAI兼容`/embeddings`接口完成索引和查询；两端共用同一模型及维度，外部调用均位于数据库事务外，查询服务临时不可用时通话可无记忆降级启动。
|
||||
- 通话中受控记忆工具：平台为固定通话和供应商任务签发短期令牌，服务端从数据库反查租户与长者边界；模型不能自行选择查询范围，检索失败返回空结果且不打断语音通话，实际使用按版本留痕。
|
||||
- 可选供应商原生记忆：数据库始终是主数据，`managed_http`适配器通过可靠任务同步；修订、停用、授权撤回、单条删除和主体隐私删除均清理外部引用并对账，未完成真实账号验证前默认关闭。
|
||||
- 细粒度内容审核与维护质量页：故事、单条事实和已确认记忆支持九类标准问题反馈；不可覆盖版本和最小质量信号按模型/提示词版本聚合到独立`/maintenance`维护端，不展示私密正文。
|
||||
- 隐私数据访问请求：所有者幂等提交、Outbox/Worker子任务、八类数据范围盘点、无正文结果和审计。
|
||||
- 私有数据导出：按长者和分类生成JSON，MinIO私有存储、SHA-256校验、短期签名下载和过期对象清理。
|
||||
- 主体级隐私删除：精确称呼确认、活动通话保护、立即冻结读取/上传/召回，以及PostgreSQL、Redis、对象存储、pgvector、外部索引和私有导出六子系统的可重试对账与最小删除证明。
|
||||
@@ -103,13 +109,15 @@ Set-Location E:\Codex\story\platform
|
||||
|
||||
Web也允许通过`http://127.0.0.1:3100`访问；本地Keycloak客户端为`localhost`和`127.0.0.1`分别登记了3100生产预览端口与5173前端开发端口，登录回调必须保持当前浏览器来源一致。
|
||||
|
||||
本地Web点击“登录研发环境”后可使用`owner`、`editor`或`outsider`三个合成账号，密码均为`LocalOnly123!`。账号只用于本机联调，权限说明见[家庭角色权限矩阵](docs/permission-matrix.md)。
|
||||
本地家庭端可使用`owner`、`editor`或`outsider`三个合成客户账号，密码均为`LocalOnly123!`。维护中心使用完全独立、且没有家庭成员关系的`operator`（`ops_admin`）或`operator-approver`（`ops_approver`），本地密码均为`OpsLocalOnly123!`。数据库通过`user_account.account_kind`约束客户账号只能是`customer`角色、运维账号只能是运营角色，禁止同一账号同时进入家庭端和维护端。账号只用于本机联调。
|
||||
|
||||
维护中心地址为`/maintenance`。其中“设备管理”允许独立运维操作员查看客户组织内全部设备及当前家庭/长者归属，登记或修改设备、调整归属、隔离或软删除设备，并在创建或轮换时自动下载仅显示一次的随机设备密钥。删除会吊销设备和密钥、关闭当前绑定，但保留历史通话与审计记录。
|
||||
|
||||
Web镜像在容器启动时根据`WEB_OIDC_AUTHORITY`和`WEB_OIDC_CLIENT_ID`生成不缓存的`runtime-config.js`。dev、preprod和prod可复用同一镜像，只需为各环境设置对应OIDC地址和客户端编号；这些值是浏览器可见的公开配置，不能用于保存客户端密钥。
|
||||
|
||||
真实火山RTC模式由两组独立开关启用：API设置`RTC_TOKEN_ISSUER=volcengine`、`RTC_APP_ID`和`RTC_APP_KEY`，Worker设置`VOICE_PROVIDER=volcengine`、`VOLC_ACCESS_KEY_ID`、`VOLC_SECRET_ACCESS_KEY`、区域、接口版本以及`VOLC_VOICE_CHAT_TEMPLATE_FILE`。生产模板采用`voice-chat-template-bundle-v1`，分别提供`s2s_pure`、`s2s_hybrid`和`cascade`三个完整VoiceChat `Config/AgentConfig`；平台按通话冻结模式选择模板，再覆盖AppId、RoomId、TaskId、目标设备用户、智能体用户，并追加本次已授权且脱敏的背景摘要。可从[模板示例](deploy/volcengine/voice-chat-template.example.json)复制结构并按照已开通的官方能力替换每个完整`Config`对象；不要把AppKey、AK/SK、ASR/TTS Token提交到仓库。
|
||||
真实火山RTC模式由两组独立开关启用：API设置`RTC_TOKEN_ISSUER=volcengine`、`RTC_APP_ID`和`RTC_APP_KEY`，Worker设置`VOICE_PROVIDER=volcengine`、`VOLC_ACCESS_KEY_ID`、`VOLC_SECRET_ACCESS_KEY`、区域、接口版本以及`VOLC_VOICE_CHAT_TEMPLATE_FILE`。生产模板采用`voice-chat-template-bundle-v1`，分别提供`s2s_pure`、`s2s_hybrid`和`cascade`三个完整VoiceChat `Config/AgentConfig`；平台按通话冻结模式选择模板，再覆盖AppId、RoomId、TaskId、目标设备用户、智能体用户，并追加本次已授权且脱敏的背景摘要。级联模式写入`LLMConfig.SystemMessages`，纯端到端模式写入`S2SConfig.ProviderParams.dialog.system_role`。可从[模板示例](deploy/volcengine/voice-chat-template.example.json)复制结构并按照已开通的官方能力替换每个完整`Config`对象；不要把AppKey、AK/SK、ASR/TTS Token提交到仓库。
|
||||
|
||||
生产API必须设置`DEVICE_SECRET_DIRECTORY`为容器内绝对路径，并把Docker/Kubernetes Secret或云KMS CSI卷只读挂载到该目录。每台设备的`credential_ref`写为`device-secret://terminal-001`，对应文件名为`terminal-001`，文件内容为至少32字节的独立随机密钥。解析器拒绝相对路径、子目录、越界符号、缺失文件和短密钥；每次验签重新读取文件，因此密钥文件原子轮换后立即生效。`DEVICE_LOCAL_SECRET`只允许本地模拟使用。
|
||||
设备HMAC密钥保存在`device_credential`表中，并使用AES-256-GCM和每条记录独立随机nonce加密；数据库不保存明文。API通过`DEVICE_CREDENTIAL_MASTER_KEY`（64位十六进制、解码后32字节）解密验签，该主密钥必须由Docker/Kubernetes Secret或云KMS CSI独立托管，不能进入数据库。`DEVICE_LOCAL_SECRET`只用于把既有演示设备凭据一次性导入数据库，完成迁移后可从运行环境移除。
|
||||
|
||||
数据库连接串、Redis、RTC AppKey、火山AK/SK、方舟API Key、对象存储凭据、回调/告警签名等敏感配置同时支持`NAME_FILE=/run/secrets/name`。文件路径必须是容器内绝对路径、目标必须是普通文件且不超过64KiB；读取时只去掉末尾换行，不在错误中打印内容。`NAME`与`NAME_FILE`同时存在会拒绝启动，防止轮换时仍意外使用旧的明文环境变量。
|
||||
|
||||
|
||||
@@ -0,0 +1,85 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"strings"
|
||||
|
||||
"data-terminal-cloud/internal/deviceauth"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
)
|
||||
|
||||
func main() {
|
||||
if err := run(context.Background(), os.Args[1:]); err != nil {
|
||||
fmt.Fprintln(os.Stderr, err)
|
||||
os.Exit(1)
|
||||
}
|
||||
}
|
||||
|
||||
func run(ctx context.Context, args []string) error {
|
||||
if len(args) != 2 || (args[0] != "provision" && args[0] != "rotate" && args[0] != "revoke") {
|
||||
return errors.New("usage: device-credential <provision|rotate|revoke> <device-public-uuid>")
|
||||
}
|
||||
if _, err := uuid.Parse(args[1]); err != nil {
|
||||
return errors.New("device public id must be a UUID")
|
||||
}
|
||||
databaseURL, err := secretEnvironment("DATABASE_URL")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
masterKey, err := secretEnvironment("DEVICE_CREDENTIAL_MASTER_KEY")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
pool, err := pgxpool.New(ctx, databaseURL)
|
||||
if err != nil {
|
||||
return fmt.Errorf("open database: %w", err)
|
||||
}
|
||||
defer pool.Close()
|
||||
resolver, err := deviceauth.NewDatabaseSecretResolver(pool, masterKey)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if args[0] == "revoke" {
|
||||
if err := resolver.Revoke(ctx, args[1]); err != nil {
|
||||
return err
|
||||
}
|
||||
return json.NewEncoder(os.Stdout).Encode(map[string]any{"device_id": args[1], "status": "revoked"})
|
||||
}
|
||||
random := make([]byte, 32)
|
||||
if _, err := rand.Read(random); err != nil {
|
||||
return fmt.Errorf("generate device credential: %w", err)
|
||||
}
|
||||
secret := hex.EncodeToString(random)
|
||||
version, err := resolver.Provision(ctx, args[1], []byte(secret), args[0] == "rotate")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
fmt.Fprintln(os.Stderr, "device credential plaintext is shown once; transfer it through a controlled channel")
|
||||
return json.NewEncoder(os.Stdout).Encode(map[string]any{
|
||||
"device_id": args[1], "key_version": version, "secret": secret,
|
||||
})
|
||||
}
|
||||
|
||||
func secretEnvironment(name string) (string, error) {
|
||||
if file := strings.TrimSpace(os.Getenv(name + "_FILE")); file != "" {
|
||||
value, err := os.ReadFile(file)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("read %s_FILE: %w", name, err)
|
||||
}
|
||||
if result := strings.TrimSpace(string(value)); result != "" {
|
||||
return result, nil
|
||||
}
|
||||
}
|
||||
if value := strings.TrimSpace(os.Getenv(name)); value != "" {
|
||||
return value, nil
|
||||
}
|
||||
return "", fmt.Errorf("%s or %s_FILE is required", name, name)
|
||||
}
|
||||
+11
-2
@@ -13,7 +13,7 @@ x-backend-environment: &backend-environment
|
||||
AUTH_ISSUER_URL: http://localhost:58180/realms/voice-local
|
||||
AUTH_JWKS_URL: http://keycloak:8080/realms/voice-local/protocol/openid-connect/certs
|
||||
AUTH_AUDIENCE: voice-platform-web
|
||||
OTEL_EXPORTER_OTLP_ENDPOINT: otel-collector:4317
|
||||
OTEL_EXPORTER_OTLP_ENDPOINT: ${OTEL_EXPORTER_OTLP_ENDPOINT:-}
|
||||
RATE_LIMIT_RPS: "50"
|
||||
RATE_LIMIT_BURST: "100"
|
||||
RATE_LIMIT_USER_RPS: "20"
|
||||
@@ -25,8 +25,17 @@ x-backend-environment: &backend-environment
|
||||
PROVIDER_START_RPS: "20"
|
||||
PROVIDER_START_BURST: "40"
|
||||
PROVIDER_MAX_ACTIVE_TASKS: "100"
|
||||
MEMORY_TOOL_BASE_URL: ${MEMORY_TOOL_BASE_URL:-http://api:8080/internal/v1/tools/memory-search}
|
||||
MEMORY_TOOL_SIGNING_SECRET: ${MEMORY_TOOL_SIGNING_SECRET:-local-memory-tool-signing-secret-32-bytes-minimum}
|
||||
MEMORY_TOOL_TOKEN_TTL: 4h
|
||||
MEMORY_TOOL_RATE_RPS: "2"
|
||||
MEMORY_TOOL_RATE_BURST: "5"
|
||||
PROVIDER_MEMORY_MODE: ${PROVIDER_MEMORY_MODE:-disabled}
|
||||
PROVIDER_MEMORY_BASE_URL: ${PROVIDER_MEMORY_BASE_URL:-}
|
||||
PROVIDER_MEMORY_API_KEY: ${PROVIDER_MEMORY_API_KEY:-}
|
||||
PROVIDER_MEMORY_TIMEOUT: ${PROVIDER_MEMORY_TIMEOUT:-5s}
|
||||
DEVICE_CLOCK_SKEW: 5m
|
||||
DEVICE_SECRET_DIRECTORY: ${DEVICE_SECRET_DIRECTORY:-}
|
||||
DEVICE_CREDENTIAL_MASTER_KEY: ${DEVICE_CREDENTIAL_MASTER_KEY:-000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f}
|
||||
RTC_APP_ID: local-public-rtc-app
|
||||
RTC_APP_KEY: ${RTC_APP_KEY:-}
|
||||
RTC_TOKEN_ISSUER: ${RTC_TOKEN_ISSUER:-local}
|
||||
|
||||
@@ -20,6 +20,7 @@ RUN CGO_ENABLED=0 go build -trimpath -ldflags="-s -w \
|
||||
-X data-terminal-cloud/internal/buildinfo.Commit=${COMMIT} \
|
||||
-X data-terminal-cloud/internal/buildinfo.BuildTime=${BUILD_TIME}" -o /out/worker ./cmd/worker && \
|
||||
CGO_ENABLED=0 go build -trimpath -o /out/sim-device ./cmd/sim-device && \
|
||||
CGO_ENABLED=0 go build -trimpath -o /out/device-credential ./cmd/device-credential && \
|
||||
CGO_ENABLED=0 go build -trimpath -o /out/firmware-conformance ./cmd/firmware-conformance && \
|
||||
CGO_ENABLED=0 go build -trimpath -o /out/alert-receiver-conformance ./cmd/alert-receiver-conformance && \
|
||||
CGO_ENABLED=0 go build -trimpath -o /out/kms-rotation-conformance ./cmd/kms-rotation-conformance && \
|
||||
|
||||
@@ -79,6 +79,26 @@
|
||||
"emailVerified": true,
|
||||
"enabled": true,
|
||||
"credentials": [{"type": "password", "value": "LocalOnly123!", "temporary": false}]
|
||||
},
|
||||
{
|
||||
"id": "00000000-0000-0000-0000-000000000104",
|
||||
"username": "operator",
|
||||
"firstName": "平台",
|
||||
"lastName": "运维管理员",
|
||||
"email": "operator@example.invalid",
|
||||
"emailVerified": true,
|
||||
"enabled": true,
|
||||
"credentials": [{"type": "password", "value": "OpsLocalOnly123!", "temporary": false}]
|
||||
},
|
||||
{
|
||||
"id": "00000000-0000-0000-0000-000000000105",
|
||||
"username": "operator-approver",
|
||||
"firstName": "平台",
|
||||
"lastName": "运维审批员",
|
||||
"email": "operator-approver@example.invalid",
|
||||
"emailVerified": true,
|
||||
"enabled": true,
|
||||
"credentials": [{"type": "password", "value": "OpsLocalOnly123!", "temporary": false}]
|
||||
}
|
||||
]
|
||||
}
|
||||
|
||||
@@ -60,6 +60,11 @@ data:
|
||||
ARK_CONTENT_MODEL: replace-with-ark-content-endpoint-id
|
||||
ARK_EMBEDDING_MODEL: replace-with-ark-embedding-endpoint-id
|
||||
ARK_REQUEST_TIMEOUT: 30s
|
||||
MEMORY_TOOL_TOKEN_TTL: 4h
|
||||
MEMORY_TOOL_RATE_RPS: "2"
|
||||
MEMORY_TOOL_RATE_BURST: "5"
|
||||
PROVIDER_MEMORY_MODE: disabled
|
||||
PROVIDER_MEMORY_TIMEOUT: 5s
|
||||
WORKER_POLL_INTERVAL: 2s
|
||||
ALERT_WEBHOOK_TIMEOUT: 5s
|
||||
---
|
||||
|
||||
@@ -55,14 +55,16 @@ spec:
|
||||
value: /var/run/secrets/voice-platform/common/database-url
|
||||
- name: REDIS_PASSWORD_FILE
|
||||
value: /var/run/secrets/voice-platform/common/redis-password
|
||||
- name: DEVICE_SECRET_DIRECTORY
|
||||
value: /var/run/secrets/voice-platform/devices
|
||||
- name: DEVICE_CREDENTIAL_MASTER_KEY_FILE
|
||||
value: /var/run/secrets/voice-platform/api/device-credential-master-key
|
||||
- name: RTC_APP_KEY_FILE
|
||||
value: /var/run/secrets/voice-platform/api/rtc-app-key
|
||||
- name: CURSOR_SIGNING_SECRET_FILE
|
||||
value: /var/run/secrets/voice-platform/api/cursor-signing-secret
|
||||
- name: SHARE_TOKEN_SIGNING_SECRET_FILE
|
||||
value: /var/run/secrets/voice-platform/api/share-token-signing-secret
|
||||
- name: MEMORY_TOOL_SIGNING_SECRET_FILE
|
||||
value: /var/run/secrets/voice-platform/model-storage/memory-tool-signing-secret
|
||||
- name: OBJECT_STORE_ACCESS_KEY_FILE
|
||||
value: /var/run/secrets/voice-platform/model-storage/object-store-access-key
|
||||
- name: OBJECT_STORE_SECRET_KEY_FILE
|
||||
@@ -114,9 +116,6 @@ spec:
|
||||
- name: model-storage-secrets
|
||||
mountPath: /var/run/secrets/voice-platform/model-storage
|
||||
readOnly: true
|
||||
- name: device-secrets
|
||||
mountPath: /var/run/secrets/voice-platform/devices
|
||||
readOnly: true
|
||||
volumes:
|
||||
- name: common-secrets
|
||||
secret:
|
||||
@@ -127,9 +126,6 @@ spec:
|
||||
- name: model-storage-secrets
|
||||
secret:
|
||||
secretName: voice-platform-model-storage-secrets
|
||||
- name: device-secrets
|
||||
secret:
|
||||
secretName: voice-platform-device-secrets
|
||||
---
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
@@ -311,6 +307,8 @@ spec:
|
||||
value: /var/run/secrets/voice-platform/model-storage/ark-api-key
|
||||
- name: CONTENT_SAFETY_API_KEY_FILE
|
||||
value: /var/run/secrets/voice-platform/model-storage/content-safety-api-key
|
||||
- name: MEMORY_TOOL_SIGNING_SECRET_FILE
|
||||
value: /var/run/secrets/voice-platform/model-storage/memory-tool-signing-secret
|
||||
ports:
|
||||
- name: http
|
||||
containerPort: 8082
|
||||
|
||||
@@ -19,7 +19,7 @@ metadata:
|
||||
spec:
|
||||
provider: replace-with-installed-provider
|
||||
parameters:
|
||||
objects: replace-with-provider-specific-rtc-app-key-cursor-and-share-secrets
|
||||
objects: replace-with-provider-specific-rtc-app-key-cursor-share-and-device-credential-master-key
|
||||
---
|
||||
apiVersion: secrets-store.csi.x-k8s.io/v1
|
||||
kind: SecretProviderClass
|
||||
@@ -49,14 +49,4 @@ metadata:
|
||||
spec:
|
||||
provider: replace-with-installed-provider
|
||||
parameters:
|
||||
objects: replace-with-provider-specific-object-store-ark-and-content-safety-credentials
|
||||
---
|
||||
apiVersion: secrets-store.csi.x-k8s.io/v1
|
||||
kind: SecretProviderClass
|
||||
metadata:
|
||||
name: voice-platform-devices
|
||||
namespace: voice-platform
|
||||
spec:
|
||||
provider: replace-with-installed-provider
|
||||
parameters:
|
||||
objects: replace-with-provider-specific-per-device-secrets
|
||||
objects: replace-with-provider-specific-object-store-ark-content-safety-and-memory-tool-credentials
|
||||
|
||||
@@ -26,12 +26,6 @@ spec:
|
||||
readOnly: true
|
||||
volumeAttributes:
|
||||
secretProviderClass: voice-platform-model-storage
|
||||
- name: device-secrets
|
||||
csi:
|
||||
driver: secrets-store.csi.k8s.io
|
||||
readOnly: true
|
||||
volumeAttributes:
|
||||
secretProviderClass: voice-platform-devices
|
||||
---
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
|
||||
@@ -10,6 +10,21 @@
|
||||
.\deploy\local-rtc\configure.ps1
|
||||
```
|
||||
|
||||
如果还要同时验证方舟大模型总结，可改用：
|
||||
|
||||
```powershell
|
||||
.\deploy\local-rtc\configure.ps1 -EnableArkSummary
|
||||
```
|
||||
|
||||
该模式会额外隐藏读取方舟API Key和内容模型Endpoint ID，并通过只读密钥文件传给Worker。
|
||||
如果 RTC 配置已经存在，只需要补充大模型总结，不必使用`-Force`重录全部密钥。先在[方舟 API Key 控制台](https://console.volcengine.com/ark/region:ark+cn-beijing/apikey)创建密钥，再运行：
|
||||
|
||||
```powershell
|
||||
.\deploy\local-rtc\configure-ark-summary.ps1
|
||||
```
|
||||
|
||||
该脚本只更新`ark-api-key`、`CONTENT_GENERATOR`和`ARK_CONTENT_MODEL`，不会修改已有RTC、语音或设备凭据。当前Chat API既支持模型Model ID，也支持Endpoint ID；脚本默认使用`doubao-seed-2-0-lite-260215`，如果账号未开通该模型，可在提示中填写已开通的Model ID或Endpoint ID。
|
||||
|
||||
脚本会以隐藏输入方式依次读取：
|
||||
|
||||
- RTC AppKey；
|
||||
@@ -34,8 +49,16 @@
|
||||
1. 保持默认本地平台地址、设备 ID 和本地测试设备密钥；
|
||||
2. 点击“检测平台和音频设备”；
|
||||
3. 选择麦克风和扬声器；
|
||||
4. 点击“开始语音聊天”；
|
||||
5. 看到“豆包语音任务已激活”后直接说话；
|
||||
6. 点击“结束通话”验证`StopVoiceChat`和RTC离房。
|
||||
4. 确认同一个RTC AppID已在RTC控制台的“实时字幕”功能中启用流式语音识别；
|
||||
5. 点击“开始语音聊天”；
|
||||
6. 看到“RTC 实时字幕已启动”和“豆包语音任务已激活”后直接说话；
|
||||
7. 确认“实时字幕与上传”区域同时出现长者和豆包话轮；
|
||||
8. 点击“结束通话”验证字幕补交、`StopVoiceChat`和RTC离房。
|
||||
|
||||
实时字幕要求先在豆包语音控制台创建服务类型为“流式语音识别”的应用，再在RTC控制台的实时字幕配置中填写该应用的APP ID、Access Token和Cluster ID。未开通或配置不匹配时SDK返回`ServiceAccessDenied`；这与“实时对话式AI > 回调设置”无关，不需要为字幕上传填写回调URL。
|
||||
|
||||
通话结束后，Worker会根据最终话轮生成通话摘要、关键词、开放话题和待审核故事。只有人工确认故事后才会进入长期记忆，这是既有授权与审核边界，模拟器不会绕过审核自动写入长期记忆。本地默认`CONTENT_GENERATOR=local`仅用于确定性闭环验证；要验证真实大模型总结，必须另外为Worker配置`CONTENT_GENERATOR=ark`、`ARK_API_KEY_FILE`和`ARK_CONTENT_MODEL`。
|
||||
|
||||
当新通话冻结快照中的`story_extraction`和`personalized_memory`授权均有效时，平台会把最近3次已完成通话的就绪摘要作为短期连续上下文注入豆包；S2S-O模式使用`dialog.system_role`，无需开启供应商侧`dialog_id`历史留存。长者档案中的“避免主动提及”话题也会从下一次开房起一并生效。
|
||||
|
||||
RTC AppKey、火山 AK/SK 和豆包 Access Token 均不进入模拟器安装包，也不会通过渲染页面读取。
|
||||
|
||||
@@ -21,7 +21,12 @@ services:
|
||||
# Quote date-shaped API versions so YAML does not coerce them into timestamps.
|
||||
VOLC_RTC_API_VERSION: "2024-12-01"
|
||||
VOLC_VOICE_CHAT_TEMPLATE_FILE: /run/secrets/local-rtc/voice-chat-template.json
|
||||
CONTENT_GENERATOR: ${CONTENT_GENERATOR:-local}
|
||||
ARK_API_KEY: ""
|
||||
ARK_API_KEY_FILE: /run/secrets/local-rtc/ark-api-key
|
||||
ARK_CONTENT_MODEL: ${ARK_CONTENT_MODEL:-}
|
||||
volumes:
|
||||
- ./deploy/local-rtc/runtime/volc-access-key-id:/run/secrets/local-rtc/volc-access-key-id:ro
|
||||
- ./deploy/local-rtc/runtime/volc-secret-access-key:/run/secrets/local-rtc/volc-secret-access-key:ro
|
||||
- ./deploy/local-rtc/runtime/voice-chat-template.json:/run/secrets/local-rtc/voice-chat-template.json:ro
|
||||
- ./deploy/local-rtc/runtime/ark-api-key:/run/secrets/local-rtc/ark-api-key:ro
|
||||
|
||||
@@ -0,0 +1,52 @@
|
||||
[CmdletBinding()]
|
||||
param(
|
||||
[string]$Model = "doubao-seed-2-0-lite-260215"
|
||||
)
|
||||
|
||||
$ErrorActionPreference = "Stop"
|
||||
$scriptDirectory = Split-Path -Parent $MyInvocation.MyCommand.Path
|
||||
$runtimeDirectory = Join-Path $scriptDirectory "runtime"
|
||||
$environmentFile = Join-Path $runtimeDirectory "local-rtc.env"
|
||||
$apiKeyFile = Join-Path $runtimeDirectory "ark-api-key"
|
||||
$utf8NoBom = [System.Text.UTF8Encoding]::new($false)
|
||||
|
||||
if (-not (Test-Path -LiteralPath $environmentFile -PathType Leaf)) {
|
||||
throw "Run .\deploy\local-rtc\configure.ps1 before configuring Ark summaries."
|
||||
}
|
||||
|
||||
$secureKey = Read-Host "Ark API Key (input is hidden)" -AsSecureString
|
||||
$pointer = [Runtime.InteropServices.Marshal]::SecureStringToBSTR($secureKey)
|
||||
try {
|
||||
$apiKey = [Runtime.InteropServices.Marshal]::PtrToStringBSTR($pointer)
|
||||
}
|
||||
finally {
|
||||
[Runtime.InteropServices.Marshal]::ZeroFreeBSTR($pointer)
|
||||
}
|
||||
if ([string]::IsNullOrWhiteSpace($apiKey)) {
|
||||
throw "Ark API Key is required."
|
||||
}
|
||||
|
||||
$selectedModel = Read-Host "Ark content Model ID or Endpoint ID [$Model]"
|
||||
if ([string]::IsNullOrWhiteSpace($selectedModel)) {
|
||||
$selectedModel = $Model
|
||||
}
|
||||
$selectedModel = $selectedModel.Trim()
|
||||
if ([string]::IsNullOrWhiteSpace($selectedModel)) {
|
||||
throw "Ark content model is required."
|
||||
}
|
||||
|
||||
$settings = [ordered]@{}
|
||||
foreach ($line in Get-Content -LiteralPath $environmentFile) {
|
||||
if ($line -match '^([^#=][^=]*)=(.*)$') {
|
||||
$settings[$matches[1].Trim()] = $matches[2]
|
||||
}
|
||||
}
|
||||
$settings["CONTENT_GENERATOR"] = "ark"
|
||||
$settings["ARK_CONTENT_MODEL"] = $selectedModel
|
||||
|
||||
$environmentContent = ($settings.GetEnumerator() | ForEach-Object { "$($_.Key)=$($_.Value)" }) -join "`n"
|
||||
[IO.File]::WriteAllText($apiKeyFile, $apiKey.Trim(), $utf8NoBom)
|
||||
[IO.File]::WriteAllText($environmentFile, $environmentContent + "`n", $utf8NoBom)
|
||||
|
||||
Write-Host "Ark summary configuration updated. Existing RTC and voice credentials were not changed." -ForegroundColor Green
|
||||
Write-Host "Next: .\deploy\local-rtc\start.ps1" -ForegroundColor Green
|
||||
@@ -1,6 +1,8 @@
|
||||
[CmdletBinding()]
|
||||
param(
|
||||
[string]$RtcAppId = "6a8f92ca43726c017d34cce1",
|
||||
[switch]$EnableArkSummary,
|
||||
[string]$ArkContentModel = "",
|
||||
[switch]$Force
|
||||
)
|
||||
|
||||
@@ -57,6 +59,11 @@ $volcSecretAccessKey = Read-RequiredSecret "火山引擎 Secret AccessKey"
|
||||
$doubaoAppId = Read-RequiredText "豆包端到端语音 APP ID"
|
||||
$doubaoAccessToken = Read-RequiredSecret "豆包端到端语音 Access Token"
|
||||
$speaker = Read-RequiredText "豆包音色 ID" "zh_female_vv_jupiter_bigtts"
|
||||
$arkApiKey = ""
|
||||
if ($EnableArkSummary) {
|
||||
$arkApiKey = Read-RequiredSecret "方舟 API Key"
|
||||
$ArkContentModel = Read-RequiredText "方舟内容模型 Endpoint ID" $ArkContentModel
|
||||
}
|
||||
|
||||
$template = [ordered]@{
|
||||
schema_version = "voice-chat-template-bundle-v1"
|
||||
@@ -103,6 +110,7 @@ $template = [ordered]@{
|
||||
Write-SecretFile "rtc-app-key" $rtcAppKey
|
||||
Write-SecretFile "volc-access-key-id" $volcAccessKeyId
|
||||
Write-SecretFile "volc-secret-access-key" $volcSecretAccessKey
|
||||
Write-SecretFile "ark-api-key" $arkApiKey
|
||||
[IO.File]::WriteAllText(
|
||||
(Join-Path $runtimeDirectory "voice-chat-template.json"),
|
||||
($template | ConvertTo-Json -Depth 20),
|
||||
@@ -110,7 +118,7 @@ Write-SecretFile "volc-secret-access-key" $volcSecretAccessKey
|
||||
)
|
||||
[IO.File]::WriteAllText(
|
||||
(Join-Path $runtimeDirectory "local-rtc.env"),
|
||||
"RTC_APP_ID=$RtcAppId`n",
|
||||
"RTC_APP_ID=$RtcAppId`nCONTENT_GENERATOR=$(if ($EnableArkSummary) { 'ark' } else { 'local' })`nARK_CONTENT_MODEL=$ArkContentModel`n",
|
||||
$utf8NoBom
|
||||
)
|
||||
|
||||
|
||||
@@ -10,13 +10,14 @@ $requiredFiles = @(
|
||||
"rtc-app-key",
|
||||
"volc-access-key-id",
|
||||
"volc-secret-access-key",
|
||||
"ark-api-key",
|
||||
"voice-chat-template.json",
|
||||
"local-rtc.env"
|
||||
)
|
||||
|
||||
foreach ($name in $requiredFiles) {
|
||||
if (-not (Test-Path -LiteralPath (Join-Path $runtimeDirectory $name) -PathType Leaf)) {
|
||||
throw "缺少 $name，请先运行 .\deploy\local-rtc\configure.ps1"
|
||||
throw "Missing $name. Run .\deploy\local-rtc\configure.ps1 first."
|
||||
}
|
||||
}
|
||||
|
||||
@@ -35,9 +36,9 @@ try {
|
||||
"callbacks", "mock-alert-receiver", "mock-provider", "api", "worker"
|
||||
)
|
||||
& docker @arguments
|
||||
if ($LASTEXITCODE -ne 0) { throw "Docker Compose 启动失败" }
|
||||
if ($LASTEXITCODE -ne 0) { throw "Docker Compose startup failed." }
|
||||
|
||||
Write-Host "`n等待本地 API 就绪..." -ForegroundColor Cyan
|
||||
Write-Host "`nWaiting for local API readiness..." -ForegroundColor Cyan
|
||||
$deadline = [DateTime]::UtcNow.AddMinutes(3)
|
||||
do {
|
||||
Start-Sleep -Seconds 2
|
||||
@@ -48,9 +49,9 @@ try {
|
||||
catch { }
|
||||
} while ([DateTime]::UtcNow -lt $deadline)
|
||||
|
||||
if (-not $health) { throw "本地 API 未在三分钟内就绪，请检查 docker compose logs api worker" }
|
||||
Write-Host "本地真实 RTC 后端已就绪：http://localhost:58080" -ForegroundColor Green
|
||||
Write-Host "现在可以启动 Windows 语音终端模拟器。" -ForegroundColor Green
|
||||
if (-not $health) { throw "Local API did not become ready within three minutes. Check docker compose logs api worker." }
|
||||
Write-Host "Local real RTC backend is ready: http://localhost:58080" -ForegroundColor Green
|
||||
Write-Host "You can now start the Windows voice device simulator." -ForegroundColor Green
|
||||
}
|
||||
finally {
|
||||
Pop-Location
|
||||
|
||||
@@ -15,14 +15,25 @@
|
||||
载入同一候选版本的`data-terminal-cloud-backend`和`data-terminal-cloud-web`镜像后执行：
|
||||
|
||||
```sh
|
||||
chmod 0755 deploy/server/*.sh
|
||||
docker compose -f compose.yaml -f deploy/server/compose.override.yaml \
|
||||
--env-file deploy/server/.env up -d --no-build \
|
||||
postgres redis minio keycloak migrate seed mock-alert-receiver callbacks \
|
||||
mock-provider api worker web gateway
|
||||
docker compose -f compose.yaml -f deploy/server/compose.override.yaml \
|
||||
--env-file deploy/server/.env up -d --no-build --force-recreate --no-deps gateway
|
||||
```
|
||||
|
||||
从Windows生成的压缩包解开后必须恢复脚本执行位。若`current`是切换到新发布目录的符号链接，还必须强制重建Gateway；否则Docker会继续使用旧发布目录已解析的Nginx绑定挂载，新增路由不会生效。
|
||||
|
||||
安全组和主机防火墙只保留TCP 80、443以及受限来源的SSH 22。HTTP 80仅用于跳转到标准HTTPS 443。正式环境仍应配置自有域名、生产OIDC和真实外部服务，并切换到Kubernetes生产覆盖层。
|
||||
|
||||
需要在演示机接入真实RTC、豆包语音和方舟摘要时，把凭据文件以`0600`权限放入`/etc/data-terminal-cloud/voice-runtime`，并在`deploy/server/.env`设置`RTC_TOKEN_ISSUER=volcengine`、`VOICE_PROVIDER=volcengine`、`RTC_APP_ID`、`CONTENT_GENERATOR`和`ARK_CONTENT_MODEL`。Compose只读挂载该目录，敏感值不进入源码、镜像或环境文件。
|
||||
|
||||
设备HMAC密钥由`device_credential`表以AES-256-GCM密文保存。`deploy/server/.env`中的`DEVICE_CREDENTIAL_MASTER_KEY`只负责数据库密文的认证解密，不得写入数据库或设备；既有`DEVICE_LOCAL_SECRET`只用于首次部署升级时导入两台演示设备，确认数据库凭据记录完整后可移除。新增、轮换和吊销设备密钥使用后端镜像内的`/app/device-credential`维护命令，生成的设备明文密钥只显示一次。
|
||||
|
||||
通话内记忆工具只通过网关精确开放`/internal/v1/tools/memory-search`，请求必须携带平台为当前通话签发的短期任务令牌。`MEMORY_TOOL_SIGNING_SECRET`必须是服务器独立随机密钥，不得与游标签名、分享签名或RTC密钥复用；网关不会开放其他`/internal`路径。
|
||||
|
||||
## 自动备份与恢复校验
|
||||
|
||||
首次安装定时任务：
|
||||
|
||||
@@ -33,9 +33,11 @@ umask 077
|
||||
printf 'KEYCLOAK_ADMIN_USERNAME=platform-admin\n'
|
||||
printf 'KEYCLOAK_ADMIN_PASSWORD=%s\n' "$(openssl rand -hex 24)"
|
||||
printf 'DEVICE_LOCAL_SECRET=%s\n' "$(openssl rand -hex 32)"
|
||||
printf 'DEVICE_CREDENTIAL_MASTER_KEY=%s\n' "$(openssl rand -hex 32)"
|
||||
printf 'RTC_TOKEN_SIGNING_SECRET=%s\n' "$(openssl rand -hex 32)"
|
||||
printf 'CURSOR_SIGNING_SECRET=%s\n' "$(openssl rand -hex 32)"
|
||||
printf 'SHARE_TOKEN_SIGNING_SECRET=%s\n' "$(openssl rand -hex 32)"
|
||||
printf 'MEMORY_TOOL_SIGNING_SECRET=%s\n' "$(openssl rand -hex 32)"
|
||||
printf 'PROVIDER_CALLBACK_SECRET=%s\n' "$(openssl rand -hex 32)"
|
||||
printf 'PLATFORM_HEALTH_CALLBACK_SECRET=%s\n' "$(openssl rand -hex 32)"
|
||||
printf 'MOCK_PROVIDER_API_KEY=%s\n' "$(openssl rand -hex 24)"
|
||||
|
||||
@@ -19,6 +19,8 @@ x-server-backend-environment: &server-backend-environment
|
||||
OBJECT_STORE_PUBLIC_SECURE: "true"
|
||||
CURSOR_SIGNING_SECRET: ${CURSOR_SIGNING_SECRET}
|
||||
SHARE_TOKEN_SIGNING_SECRET: ${SHARE_TOKEN_SIGNING_SECRET}
|
||||
MEMORY_TOOL_BASE_URL: ${PUBLIC_ORIGIN}/internal/v1/tools/memory-search
|
||||
MEMORY_TOOL_SIGNING_SECRET: ${MEMORY_TOOL_SIGNING_SECRET}
|
||||
EMBEDDING_PROVIDER: local
|
||||
VOICE_PROVIDER: ${VOICE_PROVIDER:-mock}
|
||||
|
||||
@@ -92,10 +94,15 @@ services:
|
||||
environment:
|
||||
<<: *server-backend-environment
|
||||
HTTP_ADDR: 0.0.0.0:8080
|
||||
DEVICE_LOCAL_SECRET: ${DEVICE_LOCAL_SECRET}
|
||||
DEVICE_LOCAL_SECRET: ${DEVICE_LOCAL_SECRET:-}
|
||||
DEVICE_CREDENTIAL_MASTER_KEY: ${DEVICE_CREDENTIAL_MASTER_KEY}
|
||||
RTC_TOKEN_SIGNING_SECRET: ${RTC_TOKEN_SIGNING_SECRET}
|
||||
RTC_TOKEN_ISSUER: ${RTC_TOKEN_ISSUER:-local}
|
||||
RTC_APP_ID: ${RTC_APP_ID:-local-public-rtc-app}
|
||||
RTC_APP_KEY: ""
|
||||
RTC_APP_KEY_FILE: ${RTC_APP_KEY_FILE:-/run/secrets/voice-runtime/rtc-app-key}
|
||||
volumes:
|
||||
- /etc/data-terminal-cloud/voice-runtime:/run/secrets/voice-runtime:ro
|
||||
restart: unless-stopped
|
||||
|
||||
callbacks:
|
||||
@@ -122,8 +129,21 @@ services:
|
||||
MOCK_PROVIDER_API_KEY: ${MOCK_PROVIDER_API_KEY}
|
||||
ALERT_WEBHOOK_URL: http://mock-alert-receiver:8084/webhooks/alerts
|
||||
ALERT_WEBHOOK_SECRET: ${ALERT_WEBHOOK_SECRET}
|
||||
CONTENT_GENERATOR: local
|
||||
RTC_APP_ID: ${RTC_APP_ID:-local-public-rtc-app}
|
||||
VOLC_ACCESS_KEY_ID: ""
|
||||
VOLC_ACCESS_KEY_ID_FILE: ${VOLC_ACCESS_KEY_ID_FILE:-/run/secrets/voice-runtime/volc-access-key-id}
|
||||
VOLC_SECRET_ACCESS_KEY: ""
|
||||
VOLC_SECRET_ACCESS_KEY_FILE: ${VOLC_SECRET_ACCESS_KEY_FILE:-/run/secrets/voice-runtime/volc-secret-access-key}
|
||||
VOLC_RTC_REGION: ${VOLC_RTC_REGION:-cn-north-1}
|
||||
VOLC_RTC_API_VERSION: ${VOLC_RTC_API_VERSION:-2024-12-01}
|
||||
VOLC_VOICE_CHAT_TEMPLATE_FILE: ${VOLC_VOICE_CHAT_TEMPLATE_FILE:-/run/secrets/voice-runtime/voice-chat-template.json}
|
||||
CONTENT_GENERATOR: ${CONTENT_GENERATOR:-local}
|
||||
ARK_API_KEY: ""
|
||||
ARK_API_KEY_FILE: ${ARK_API_KEY_FILE:-/run/secrets/voice-runtime/ark-api-key}
|
||||
ARK_CONTENT_MODEL: ${ARK_CONTENT_MODEL:-}
|
||||
CONTENT_SAFETY_PROVIDER: local
|
||||
volumes:
|
||||
- /etc/data-terminal-cloud/voice-runtime:/run/secrets/voice-runtime:ro
|
||||
restart: unless-stopped
|
||||
|
||||
mock-provider:
|
||||
|
||||
@@ -44,6 +44,14 @@ server {
|
||||
proxy_buffering off;
|
||||
}
|
||||
|
||||
# The voice provider can call only this signed, conversation-scoped tool.
|
||||
# Do not expose the rest of the /internal namespace.
|
||||
location = /internal/v1/tools/memory-search {
|
||||
proxy_pass $api_backend;
|
||||
proxy_buffering off;
|
||||
proxy_request_buffering off;
|
||||
}
|
||||
|
||||
location /callbacks/ {
|
||||
proxy_pass $callbacks_backend;
|
||||
proxy_buffering off;
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 360 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 72 KiB |
+59
-42
@@ -1,52 +1,69 @@
|
||||
# Design QA
|
||||
# 故事与记忆设计 QA
|
||||
|
||||
- source visual truth path: `C:\Users\19926\AppData\Local\Temp\codex-clipboard-22a7b04e-194a-47ee-9fef-f98b31a70009.png`
|
||||
- implementation URL: `http://localhost:3100/`
|
||||
- implementation screenshot path: pending browser-capture permission
|
||||
- intended viewport: 3120 x 1810 CSS px, deviceScaleFactor 1
|
||||
- source pixels: 3120 x 1810
|
||||
- implementation pixels: pending
|
||||
- density normalization: pending
|
||||
- state: independent family customer portal, existing browser login session if available
|
||||
|
||||
**Full-view comparison evidence**
|
||||
|
||||
- The source visual was opened at original resolution and used to define the target shell: fixed white header, light-gray sidebar, white content cards, compact enterprise spacing, blue-purple primary color, restrained borders and shadows.
|
||||
- The customer implementation now uses a fixed Ant Design header/sidebar and a family-only information architecture. Operations menus, platform process cards, dependency checks, build version and milestone content are not present in the served bundle.
|
||||
- The implementation is compiled and served successfully, but a browser-rendered screenshot has not yet been captured. A code-only comparison is not sufficient for visual approval.
|
||||
|
||||
**Focused region comparison evidence**
|
||||
|
||||
- Pending browser evidence for the customer header/sidebar, family dashboard, business forms, tables, empty states, authenticated profile and responsive layout.
|
||||
- source visual truth: `C:\Users\19926\AppData\Local\Temp\codex-clipboard-1f63ed7f-0c86-4167-9ac0-3b8755196941.png`
|
||||
- source implementation code: `C:\Users\19926\.codex\attachments\8ae4c28d-4e71-4dfa-aaac-c01ce9b49597\pasted-text.txt`（第 410 行起“故事与记忆-重置版”）
|
||||
- earlier failing implementation: `C:\Users\19926\AppData\Local\Temp\codex-clipboard-c4313a8a-bc8f-49d6-9a82-69137a0984d9.png`
|
||||
- implementation screenshot: `E:\Codex\story\platform\design-qa-story-implementation.png`
|
||||
- combined comparison: `E:\Codex\story\platform\design-qa-story-comparison.png`
|
||||
- browser route: `http://127.0.0.1:5173/?section=stories`
|
||||
- browser CSS viewport: 1348 x 936 CSS px, device scale factor 1
|
||||
- source pixels: 923 x 655
|
||||
- implementation pixels: 1333 x 926
|
||||
- normalization: source 保持 923 x 655；实现截图按宽度缩放至 923 px 并在 923 x 655 面板中居中；两个面板并排写入同一张比较图。由于参考图是设计导出图而不是可控浏览器视口，本次同时以内容区域宽度、相同登录态和相同空记忆状态校验，不对门户外壳的像素比例做伪精确判断。
|
||||
- state: 已登录的家庭所有者；当前家庭“合成数据一号家庭”；当前长者“测试长者”；5 个真实候选故事；0 条已确认记忆；状态流已连接。
|
||||
|
||||
**Findings**
|
||||
|
||||
- [P1] Browser-rendered comparison evidence is missing.
|
||||
Location: full implementation at `http://localhost:3100/`.
|
||||
Evidence: the source image is available, but no implementation screenshot has been captured at the matching viewport.
|
||||
Impact: typography, spacing, overflow, border radii, and above-the-fold density cannot be visually verified.
|
||||
Fix: capture the implementation in the user's chosen local browser at 3120 x 1810, combine it with the source image, compare visible differences, fix any P0/P1/P2 issues, and recapture.
|
||||
- 没有剩余 P0/P1/P2 视觉问题。
|
||||
- 参考图只绘制了 4 张示例候选卡，但标题计数为 5；真实实现按现有接口返回 5 条数据，因此第 5 张卡正常换到下一行。该差异是数据驱动的预期行为，不能通过隐藏真实候选来追求静态截图一致。
|
||||
- 门户保留现有“安心陪伴”顶栏、完整业务导航和登录账户入口；设计 QA 的精确匹配范围是本次修改的故事与记忆主体区。这样不会删除现有路由或改变业务边界。
|
||||
|
||||
**Open Questions**
|
||||
**Required Fidelity Surfaces**
|
||||
|
||||
- Permission is required before using Playwright directly for the local screenshot capture.
|
||||
|
||||
**Implementation Checklist**
|
||||
|
||||
- Capture the implementation at the matching desktop viewport and state.
|
||||
- Compare the source and implementation in one combined visual input.
|
||||
- Verify typography, layout rhythm, color tokens, icon fidelity, and app-specific copy.
|
||||
- Test sidebar navigation, refresh, login CTA, and the main workspace controls.
|
||||
- Check browser console errors and horizontal overflow.
|
||||
- Apply and re-capture any P0/P1/P2 fixes.
|
||||
|
||||
**Follow-up Polish**
|
||||
|
||||
- Evaluate whether the production bundle should be code-split after visual QA; this is a performance refinement and does not block the current style redesign.
|
||||
- Fonts and typography: 标题、区块标题、候选正文、状态与版本号的层级、字重、换行和行高与参考一致；中文继续使用项目既有 `PingFang SC` / `Microsoft YaHei` 回退，不额外引入网络字体。
|
||||
- Spacing and layout rhythm: 长者筛选与家庭状态合并为一条全宽工具栏；候选区为桌面四列；记忆标题、搜索框和虚线空状态按垂直顺序整行排列；旧的 17rem 左栏已移除。
|
||||
- Colors and visual tokens: 候选卡恢复暖白底、浅灰边框、琥珀审核状态和轻阴影；不再被全局主按钮规则染成整块砖红色。连接状态仍使用真实语义绿色，而不是照抄参考图的静态红点。
|
||||
- Image quality and asset fidelity: 主体区没有照片或插画资产；审核、标签、搜索、刷新和空状态使用项目已有 Ant Design 图标库，没有占位符、手绘 SVG 或文本图标替代。
|
||||
- Copy and content: 标题、说明、筛选标签、故事候选、已确认记忆、搜索占位和空状态文案与设计源一致；候选标题和数量来自真实接口数据。
|
||||
|
||||
**Comparison History**
|
||||
|
||||
- Iteration 0: combined SaaS console build and HTTP smoke test passed; visual comparison blocked because browser capture has not been authorized.
|
||||
- Iteration 1: family customer portal built and served at port 3100; production bundle contains the customer brand and excludes the operations menu labels; TypeScript/Vite build, Docker rebuild, HTTP 200 and API dependency readiness passed. Visual comparison remains blocked pending browser-capture authorization.
|
||||
1. Earlier comparison — blocked
|
||||
- P1: 旧 `.story-layout` 的 `17rem 1fr` 网格仍生效，长者筛选、候选区和记忆区被拆成错误的左右栏。
|
||||
- P1: 全局客户按钮样式覆盖候选卡，所有故事卡显示为整块砖红色。
|
||||
- P2: `Layout.Content` 作为 `main` 继承了全局 80px 上下内边距，所有客户页面顶部出现大块空白。
|
||||
2. Fixes applied
|
||||
- 将长者筛选放入同一条工作台工具栏；故事主体显式重置为单列文档流。
|
||||
- 候选卡从全局主按钮规则中排除并固定为桌面四列白卡。
|
||||
- 客户内容容器显式清除全局 `main` 的宽度、外边距和 80px 内边距，所有客户页主体上移。
|
||||
- 补齐沙漏、标签和刷新图标，正文恢复常规字重；保留既有审核详情弹窗和数据接口。
|
||||
3. Post-fix evidence — passed
|
||||
- 并排比较图中，页面标题、工具栏、四列首行候选、记忆搜索和空状态的结构与视觉层级一致。
|
||||
- 浏览器实测首张候选可打开“故事详情”弹窗，来源操作存在，关闭按钮可恢复列表状态。
|
||||
- 浏览器开发日志仅包含 Vite 连接、热更新和 React DevTools 提示，没有 error 级别日志。
|
||||
|
||||
final result: blocked
|
||||
**Primary Interactions Tested**
|
||||
|
||||
- 打开故事候选详情：通过。
|
||||
- 关闭故事详情弹窗：通过。
|
||||
- 详情中的“查看完整原始通话”入口存在：通过。
|
||||
- 长者下拉、当前家庭下拉和刷新按钮均可见且保持原有事件处理：通过。
|
||||
|
||||
**Open Questions**
|
||||
|
||||
- 无。
|
||||
|
||||
**Implementation Checklist**
|
||||
|
||||
- [x] 工具栏合并并横向占满。
|
||||
- [x] 候选卡恢复四列白卡样式。
|
||||
- [x] 已确认记忆移到候选区下方并整行占满。
|
||||
- [x] 修复所有客户页面顶部公共空白。
|
||||
- [x] 保留真实第 5 条故事和现有审核/记忆接口。
|
||||
- [x] 类型检查、生产构建、浏览器交互和控制台日志检查通过。
|
||||
|
||||
**Follow-up Polish**
|
||||
|
||||
- 无阻塞项。
|
||||
|
||||
final result: passed
|
||||
|
||||
@@ -1,6 +1,8 @@
|
||||
# Windows 语音终端模拟器
|
||||
|
||||
该程序使用Electron和火山引擎Web RTC SDK模拟一台真实语音终端。它采集Windows麦克风、加入平台分配的RTC房间、发布本地音频、订阅并播放豆包智能体音频，并通过设备v2接口完成激活、状态等待、心跳、Token续签和结束通话。
|
||||
该程序使用Electron和火山引擎Web RTC SDK模拟一台真实语音终端。它采集Windows麦克风、加入平台分配的RTC房间、发布本地音频、订阅并播放豆包智能体音频，并通过设备v2接口完成激活、状态等待、心跳、Token续签和结束通话。进房后还会启动RTC实时字幕，收集长者与智能体的最终话轮并通过现有`turns:batch`设备接口可靠上传；字幕未成功启动时拒绝继续激活通话，避免生成没有聊天记录的完成通话。
|
||||
|
||||
创建通话返回的隐私策略允许原录音时，模拟器同时通过RTC原始音频帧回调采集长者`near`和智能体`far`轨道。结束通话后自动封装为单声道WAV PCM16、按平台限制分片、申请短期对象地址直传，并等待后台完整性与容器校验完成。家庭工作台的通话记录会标记“可听聊天语音”，获得独立PLAY授权的账号可在详情中选择“听长者语音”或“听智能体语音”。
|
||||
|
||||
## 构建
|
||||
|
||||
@@ -12,12 +14,34 @@ npm run build
|
||||
|
||||
输出：`release/voice-device-simulator-0.1.0-windows-x64.exe`。
|
||||
|
||||
连接公网测试服务器时，可以直接双击EXE，也可以使用`launch-cloud-device.ps1`启动。Electron主进程优先读取启动脚本传入的临时环境变量，否则从当前Windows用户的`%LOCALAPPDATA%\DataTerminalVoiceDevice\server-device-secret`读取设备签名密钥。密钥不写入源码、安装包或浏览器存储，也不会回传到渲染进程；输入框留空并显示“已从Windows安全凭据文件加载”即表示可直接检测。
|
||||
|
||||
## 自动连续性验证
|
||||
|
||||
本地回归时可让模拟器自动启动通话、播放一段测试音频，并把双方最终字幕写入JSON结果文件。该入口只接受本机文件和回环地址，不影响正常交互使用：
|
||||
|
||||
```powershell
|
||||
$env:SIMULATOR_AUTO_TEST = '1'
|
||||
$env:SIMULATOR_AUTOMATED_AUDIO_PATH = 'C:\path\to\question.wav'
|
||||
$env:SIMULATOR_AUTO_RESULT_PATH = 'C:\path\to\result.json'
|
||||
& .\release\voice-device-simulator-0.1.0-windows-x64.exe
|
||||
```
|
||||
|
||||
测试会等待长者与智能体的最终字幕，结束通话后退出；结果文件包含通话ID、最终话轮和通过/失败状态。测试音频支持WAV、MP3和OGG，大小不得超过10 MB。
|
||||
|
||||
## 安全边界
|
||||
|
||||
- RTC AppKey、火山AK/SK和豆包Access Token只存在本地后端的受保护文件中；
|
||||
- 模拟器安装包只接收平台签发的短期RTC Token；
|
||||
- 设备签名密钥只驻留Electron主进程内存，不写入浏览器存储；
|
||||
- 当前构建只允许设备API地址指向`localhost`或`127.0.0.1`，避免误连服务器环境；
|
||||
- 当前测试构建只允许设备API地址指向`localhost`、`127.0.0.1`或固定测试服务器`https://121.41.46.238`，避免误连其他环境；
|
||||
- 关闭软件或结束通话时停止音频采集并销毁RTC引擎。
|
||||
- 未取得`record_near`或`record_far`授权的轨道不会注册PCM回调；`upload_allowed=false`时不会采集或上传任何原录音。
|
||||
|
||||
## 实时字幕前提
|
||||
|
||||
测试所用RTC AppID必须在火山RTC控制台开启“实时字幕”和“流式语音识别”。流式语音识别凭据来自豆包语音控制台创建的“流式语音识别”应用，包括APP ID、Access Token和Cluster ID；“音视频字幕生成”类型不能替代该应用。未开通时程序会显示`ServiceAccessDenied`并清理本次未激活通话。
|
||||
|
||||
最终字幕会在程序的“实时字幕与上传”区域显示。每条最终字幕使用通话内稳定键和严格递增序号上传；网络失败时留在内存队列，并在心跳及结束通话前重试。存在未上传字幕时，程序不会结束并丢弃当前通话。
|
||||
|
||||
完整本地启动顺序见[本地真实RTC联调说明](../deploy/local-rtc/README.md)。
|
||||
|
||||
@@ -2,8 +2,10 @@ const { app, BrowserWindow, ipcMain, session } = require("electron");
|
||||
const crypto = require("node:crypto");
|
||||
const fs = require("node:fs");
|
||||
const path = require("node:path");
|
||||
const tls = require("node:tls");
|
||||
|
||||
app.commandLine.appendSwitch("autoplay-policy", "no-user-gesture-required");
|
||||
app.disableHardwareAcceleration();
|
||||
|
||||
let deviceConfig = null;
|
||||
let desiredConfigRevision = 1;
|
||||
@@ -27,6 +29,29 @@ function createWindow() {
|
||||
|
||||
const devArgument = process.argv.find((argument) => argument.startsWith("--dev-server-url="));
|
||||
const devServerURL = devArgument ? devArgument.slice("--dev-server-url=".length) : "";
|
||||
window.webContents.on("did-fail-load", (_event, errorCode, errorDescription, validatedURL) => {
|
||||
console.error("renderer load failed", { errorCode, errorDescription, validatedURL });
|
||||
});
|
||||
window.webContents.on("render-process-gone", (_event, details) => {
|
||||
console.error("renderer process exited", details);
|
||||
});
|
||||
window.webContents.on("console-message", (_event, ...args) => {
|
||||
const details = args[0];
|
||||
const message = typeof details === "object" && details ? details.message : String(args[1] ?? details ?? "");
|
||||
console.log(`[renderer] ${message}`);
|
||||
});
|
||||
window.webContents.once("did-finish-load", async () => {
|
||||
try {
|
||||
const diagnostics = await window.webContents.executeJavaScript(`JSON.stringify({readyState: document.readyState, bodyLength: document.body.innerText.length, appChildren: document.querySelector('#app')?.childElementCount || 0})`);
|
||||
console.log(`[renderer-ready] ${diagnostics}`);
|
||||
} catch (error) {
|
||||
console.error("renderer diagnostics failed", error);
|
||||
}
|
||||
if (process.env.SIMULATOR_AUTO_TEST === "1") {
|
||||
void runAutomatedSmoke(window);
|
||||
}
|
||||
});
|
||||
|
||||
if (devServerURL) {
|
||||
window.loadURL(devServerURL);
|
||||
} else {
|
||||
@@ -45,29 +70,123 @@ function createWindow() {
|
||||
}
|
||||
}
|
||||
|
||||
function ensureLoopbackURL(value) {
|
||||
async function runAutomatedSmoke(window) {
|
||||
const delay = (milliseconds) => new Promise((resolve) => setTimeout(resolve, milliseconds));
|
||||
await delay(800);
|
||||
await window.webContents.executeJavaScript(`document.querySelector('#checkButton')?.click()`);
|
||||
for (let attempt = 0; attempt < 30; attempt += 1) {
|
||||
await delay(500);
|
||||
const state = await window.webContents.executeJavaScript(`JSON.stringify({phase: document.querySelector('#phaseBadge')?.textContent, canStart: !document.querySelector('#startButton')?.disabled, detail: document.querySelector('#apiDetail')?.textContent})`);
|
||||
console.log(`[auto-test] ${state}`);
|
||||
const parsed = JSON.parse(state);
|
||||
if (parsed.canStart) {
|
||||
await window.webContents.executeJavaScript(`document.querySelector('#startButton')?.click()`);
|
||||
console.log("[auto-test] start requested");
|
||||
await collectAutomatedConversationResult(window);
|
||||
return;
|
||||
}
|
||||
if (parsed.phase === "异常") {
|
||||
return;
|
||||
}
|
||||
}
|
||||
console.error("[auto-test] environment check did not become ready");
|
||||
}
|
||||
|
||||
async function collectAutomatedConversationResult(window) {
|
||||
const delay = (milliseconds) => new Promise((resolve) => setTimeout(resolve, milliseconds));
|
||||
const deadline = Date.now() + 120_000;
|
||||
let snapshot = {};
|
||||
while (Date.now() < deadline) {
|
||||
await delay(1000);
|
||||
const encoded = await window.webContents.executeJavaScript(`JSON.stringify({
|
||||
phase: document.querySelector('#phaseBadge')?.textContent || '',
|
||||
conversation_id: document.querySelector('#conversationID')?.textContent || '',
|
||||
subject: [...document.querySelectorAll('.transcript-turn--subject span')].map((item) => item.textContent || ''),
|
||||
agent: [...document.querySelectorAll('.transcript-turn--agent span')].map((item) => item.textContent || ''),
|
||||
turns: [...document.querySelectorAll('.transcript-turn')].map((item) => ({speaker: item.classList.contains('transcript-turn--subject') ? 'subject' : 'agent', text: item.querySelector('span')?.textContent || ''})),
|
||||
events: [...document.querySelectorAll('#eventLog li span')].map((item) => item.textContent || '')
|
||||
})`);
|
||||
snapshot = JSON.parse(encoded);
|
||||
console.log(`[auto-test] conversation ${encoded}`);
|
||||
const subjectIndex = snapshot.turns?.findIndex((turn) => turn.speaker === "subject") ?? -1;
|
||||
const hasAgentReply = subjectIndex >= 0 && snapshot.turns.slice(subjectIndex + 1).some((turn) => turn.speaker === "agent");
|
||||
if (hasAgentReply) {
|
||||
await window.webContents.executeJavaScript(`document.querySelector('#stopButton')?.click()`);
|
||||
snapshot.result = "PASS";
|
||||
break;
|
||||
}
|
||||
if (snapshot.phase === "异常") {
|
||||
snapshot.result = "ERROR";
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (!snapshot.result) snapshot.result = "TIMEOUT";
|
||||
const resultPath = process.env.SIMULATOR_AUTO_RESULT_PATH;
|
||||
if (resultPath) {
|
||||
fs.writeFileSync(path.resolve(resultPath), JSON.stringify(snapshot, null, 2), "utf8");
|
||||
}
|
||||
await delay(4000);
|
||||
app.quit();
|
||||
}
|
||||
|
||||
function ensureAllowedAPIURL(value) {
|
||||
const parsed = new URL(String(value || ""));
|
||||
const loopback = parsed.hostname === "127.0.0.1" || parsed.hostname === "localhost" || parsed.hostname === "[::1]";
|
||||
if (!loopback || (parsed.protocol !== "http:" && parsed.protocol !== "https:")) {
|
||||
throw new Error("本地测试只允许连接 localhost 或 127.0.0.1");
|
||||
const deployedTestServer = parsed.protocol === "https:" && parsed.hostname === "121.41.46.238" && parsed.port === "";
|
||||
if ((!loopback || (parsed.protocol !== "http:" && parsed.protocol !== "https:")) && !deployedTestServer) {
|
||||
throw new Error("测试设备只允许连接本机平台或已部署的测试服务器");
|
||||
}
|
||||
return parsed.origin;
|
||||
}
|
||||
|
||||
function configure(input) {
|
||||
const apiURL = ensureLoopbackURL(input?.apiURL);
|
||||
const apiURL = ensureAllowedAPIURL(input?.apiURL);
|
||||
const deviceID = String(input?.deviceID || "").trim();
|
||||
const deviceSecret = String(input?.deviceSecret || "");
|
||||
const deviceSecret = String(input?.deviceSecret || "") || loadDefaultDeviceSecret();
|
||||
if (!/^[0-9a-f-]{36}$/i.test(deviceID)) {
|
||||
throw new Error("设备 ID 必须是 UUID");
|
||||
}
|
||||
if (deviceSecret.length < 32) {
|
||||
throw new Error("设备密钥至少需要 32 个字符");
|
||||
throw new Error("未找到有效的设备密钥。请使用云端设备启动脚本，或在设备签名密钥中输入至少32个字符");
|
||||
}
|
||||
deviceConfig = { apiURL, deviceID, deviceSecret };
|
||||
return { apiURL, deviceID };
|
||||
}
|
||||
|
||||
function loadDefaultDeviceSecret() {
|
||||
const environmentSecret = String(process.env.SIMULATOR_DEFAULT_DEVICE_SECRET || "").trim();
|
||||
if (environmentSecret) return environmentSecret;
|
||||
const candidates = [];
|
||||
const localAppData = String(process.env.LOCALAPPDATA || "").trim();
|
||||
const userProfile = String(process.env.USERPROFILE || "").trim();
|
||||
const portableDirectory = String(process.env.PORTABLE_EXECUTABLE_DIR || "").trim();
|
||||
if (localAppData) candidates.push(path.join(localAppData, "DataTerminalVoiceDevice", "server-device-secret"));
|
||||
if (userProfile) candidates.push(path.join(userProfile, "AppData", "Local", "DataTerminalVoiceDevice", "server-device-secret"));
|
||||
try {
|
||||
const home = app.getPath("home");
|
||||
if (home) candidates.push(path.join(home, "AppData", "Local", "DataTerminalVoiceDevice", "server-device-secret"));
|
||||
} catch { /* app path is unavailable before ready */ }
|
||||
if (portableDirectory) candidates.push(path.join(portableDirectory, ".server-device-secret"));
|
||||
candidates.push(path.join(path.dirname(process.execPath), ".server-device-secret"));
|
||||
|
||||
for (const secretPath of new Set(candidates)) {
|
||||
try {
|
||||
const stats = fs.statSync(secretPath);
|
||||
if (!stats.isFile() || stats.size < 32 || stats.size > 4096) continue;
|
||||
const value = fs.readFileSync(secretPath, "utf8").trim();
|
||||
if (value.length >= 32) return value;
|
||||
} catch { /* try the next fixed credential location */ }
|
||||
}
|
||||
return "";
|
||||
}
|
||||
|
||||
ipcMain.handle("sim:get-runtime-defaults", () => ({
|
||||
apiURL: process.env.SIMULATOR_DEFAULT_API_URL || "https://121.41.46.238",
|
||||
deviceID: process.env.SIMULATOR_DEFAULT_DEVICE_ID || "10000000-0000-0000-0000-000000000511",
|
||||
deviceSecret: "",
|
||||
deviceSecretConfigured: loadDefaultDeviceSecret().length >= 32,
|
||||
}));
|
||||
|
||||
function nonce() {
|
||||
return crypto.randomBytes(16).toString("hex");
|
||||
}
|
||||
@@ -130,6 +249,53 @@ ipcMain.handle("sim:probe", async () => {
|
||||
return { bootstrap, configuration, desiredConfigRevision };
|
||||
});
|
||||
|
||||
const rtcWebEndpoints = [
|
||||
"access.rtc.volcvideo.com",
|
||||
"access2-hl.rtc.volcvideo.com",
|
||||
"common.rtc.volcvideo.com",
|
||||
"rtcg-access.volcvideos.com",
|
||||
"rtcg.volcvideos.com",
|
||||
"web-log-report.rtc.volcvideo.com",
|
||||
"web-log-report.volcvideos.com",
|
||||
];
|
||||
|
||||
function probeTLSEndpoint(hostname) {
|
||||
return new Promise((resolve) => {
|
||||
const socket = tls.connect({ host: hostname, port: 443, servername: hostname, rejectUnauthorized: true });
|
||||
let finished = false;
|
||||
const finish = (result) => {
|
||||
if (finished) return;
|
||||
finished = true;
|
||||
socket.destroy();
|
||||
resolve(result);
|
||||
};
|
||||
socket.setTimeout(5_000, () => finish({ hostname, ok: false, error: "连接超时" }));
|
||||
socket.once("secureConnect", () => finish({ hostname, ok: true }));
|
||||
socket.once("error", (error) => finish({ hostname, ok: false, error: String(error?.code || error?.message || "连接失败") }));
|
||||
});
|
||||
}
|
||||
|
||||
ipcMain.handle("sim:probe-rtc-network", async () => {
|
||||
const endpoints = await Promise.all(rtcWebEndpoints.map(probeTLSEndpoint));
|
||||
return { endpoints };
|
||||
});
|
||||
|
||||
ipcMain.handle("sim:get-automation-audio", async () => {
|
||||
const configuredPath = String(process.env.SIMULATOR_AUTOMATED_AUDIO_PATH || "").trim();
|
||||
if (!configuredPath) return undefined;
|
||||
const resolvedPath = path.resolve(configuredPath);
|
||||
const stats = fs.statSync(resolvedPath);
|
||||
if (!stats.isFile() || stats.size < 1 || stats.size > 10 * 1024 * 1024) {
|
||||
throw new Error("自动验证音频必须是 1 字节到 10MB 的文件");
|
||||
}
|
||||
const extension = path.extname(resolvedPath).toLowerCase();
|
||||
const mime = extension === ".mp3" ? "audio/mpeg" : extension === ".ogg" ? "audio/ogg" : "audio/wav";
|
||||
return {
|
||||
data_url: `data:${mime};base64,${fs.readFileSync(resolvedPath).toString("base64")}`,
|
||||
source_name: path.basename(resolvedPath),
|
||||
};
|
||||
});
|
||||
|
||||
ipcMain.handle("sim:create-conversation", async () => deviceRequest("POST", "/device/v2/conversations", {
|
||||
device_time: new Date().toISOString(),
|
||||
monotonic_ms: Math.max(0, Math.round(performance.now())),
|
||||
@@ -180,6 +346,22 @@ ipcMain.handle("sim:heartbeat", async (_event, input) => deviceRequest(
|
||||
},
|
||||
));
|
||||
|
||||
ipcMain.handle("sim:submit-transcript-batch", async (_event, input) => {
|
||||
const conversationID = String(input?.conversationID || "");
|
||||
const turns = Array.isArray(input?.turns) ? input.turns : [];
|
||||
if (!conversationID || turns.length === 0) {
|
||||
throw new Error("通话编号和最终话轮不能为空");
|
||||
}
|
||||
const payload = { turns };
|
||||
const digest = crypto.createHash("sha256").update(JSON.stringify(payload)).digest("hex").slice(0, 32);
|
||||
return deviceRequest(
|
||||
"POST",
|
||||
`/device/v2/conversations/${encodeURIComponent(conversationID)}/turns:batch`,
|
||||
payload,
|
||||
`windows-turns-${digest}`,
|
||||
);
|
||||
});
|
||||
|
||||
ipcMain.handle("sim:stop-conversation", async (_event, input) => deviceRequest(
|
||||
"POST",
|
||||
`/device/v2/conversations/${encodeURIComponent(input.conversationID)}/stop`,
|
||||
@@ -188,11 +370,62 @@ ipcMain.handle("sim:stop-conversation", async (_event, input) => deviceRequest(
|
||||
device_time: new Date().toISOString(),
|
||||
last_heartbeat_sequence: input.lastHeartbeatSequence || 0,
|
||||
rtc_left: true,
|
||||
recording: { near_segments: 0, far_segments: 0 },
|
||||
recording: { near_segments: Number(input.nearSegments || 0), far_segments: Number(input.farSegments || 0) },
|
||||
},
|
||||
`windows-stop-${nonce()}`,
|
||||
));
|
||||
|
||||
ipcMain.handle("sim:upload-conversation-recording", async (_event, input) => {
|
||||
const conversationID = String(input?.conversationID || "");
|
||||
const supplied = Array.isArray(input?.segments) ? input.segments : [];
|
||||
if (!conversationID || supplied.length === 0) throw new Error("通话编号和录音分片不能为空");
|
||||
|
||||
const segments = supplied.map((segment) => {
|
||||
const bytes = Buffer.from(segment.bytes);
|
||||
if (!bytes.length || bytes.length > 64 * 1024 * 1024) throw new Error("单个录音分片必须在 1 字节到 64MiB 之间");
|
||||
return {
|
||||
track: segment.track,
|
||||
sequence_no: Number(segment.sequence_no),
|
||||
codec: segment.codec,
|
||||
content_type: segment.content_type,
|
||||
duration_ms: Number(segment.duration_ms),
|
||||
expected_size: bytes.length,
|
||||
expected_digest: crypto.createHash("sha256").update(bytes).digest("hex"),
|
||||
bytes,
|
||||
};
|
||||
});
|
||||
const manifest = { segments: segments.map(({ bytes: _bytes, ...segment }) => segment) };
|
||||
const manifestDigest = crypto.createHash("sha256").update(JSON.stringify(manifest)).digest("hex").slice(0, 40);
|
||||
const batch = await deviceRequest(
|
||||
"POST",
|
||||
`/device/v2/conversations/${encodeURIComponent(conversationID)}/upload-batches`,
|
||||
manifest,
|
||||
`windows-media-${manifestDigest}`,
|
||||
);
|
||||
|
||||
for (const local of segments) {
|
||||
const remote = batch.segments?.find((segment) => segment.track === local.track && Number(segment.sequence_no) === local.sequence_no);
|
||||
if (!remote) throw new Error(`平台未返回${local.track}轨道第${local.sequence_no}段录音`);
|
||||
if (remote.status === "VERIFIED" || remote.status === "VERIFYING") continue;
|
||||
if (!remote.upload_url) throw new Error(`平台未签发${local.track}轨道第${local.sequence_no}段上传地址`);
|
||||
const response = await fetch(remote.upload_url, { method: "PUT", headers: { "Content-Type": local.content_type }, body: local.bytes });
|
||||
if (!response.ok) throw new Error(`录音对象上传失败：HTTP ${response.status}`);
|
||||
}
|
||||
|
||||
await deviceRequest("POST", `/device/v2/upload-batches/${encodeURIComponent(batch.id)}/complete`, {});
|
||||
const deadline = Date.now() + 90_000;
|
||||
while (Date.now() < deadline) {
|
||||
const current = await deviceRequest("GET", `/device/v2/upload-batches/${encodeURIComponent(batch.id)}`);
|
||||
if (current.status === "COMPLETED") return current;
|
||||
if (current.status === "FAILED" || current.status === "EXPIRED") {
|
||||
const rejected = current.segments?.find((segment) => segment.status === "REJECTED");
|
||||
throw new Error(rejected?.last_error_code || current.last_error_code || `录音批次${current.status}`);
|
||||
}
|
||||
await new Promise((resolve) => setTimeout(resolve, 750));
|
||||
}
|
||||
throw new Error("录音已上传，但等待服务器校验超时");
|
||||
});
|
||||
|
||||
app.whenReady().then(() => {
|
||||
session.defaultSession.setPermissionCheckHandler((_webContents, permission) => permission === "media");
|
||||
session.defaultSession.setPermissionRequestHandler((_webContents, permission, callback) => callback(permission === "media"));
|
||||
|
||||
@@ -1,11 +1,16 @@
|
||||
const { contextBridge, ipcRenderer } = require("electron");
|
||||
|
||||
contextBridge.exposeInMainWorld("deviceSimulator", {
|
||||
getRuntimeDefaults: () => ipcRenderer.invoke("sim:get-runtime-defaults"),
|
||||
configure: (input) => ipcRenderer.invoke("sim:configure", input),
|
||||
probe: () => ipcRenderer.invoke("sim:probe"),
|
||||
probeRTCNetwork: () => ipcRenderer.invoke("sim:probe-rtc-network"),
|
||||
getAutomationAudio: () => ipcRenderer.invoke("sim:get-automation-audio"),
|
||||
createConversation: () => ipcRenderer.invoke("sim:create-conversation"),
|
||||
activateConversation: (input) => ipcRenderer.invoke("sim:activate-conversation", input),
|
||||
getConversation: (conversationID) => ipcRenderer.invoke("sim:get-conversation", conversationID),
|
||||
heartbeat: (input) => ipcRenderer.invoke("sim:heartbeat", input),
|
||||
submitTranscriptBatch: (input) => ipcRenderer.invoke("sim:submit-transcript-batch", input),
|
||||
stopConversation: (input) => ipcRenderer.invoke("sim:stop-conversation", input),
|
||||
uploadConversationRecording: (input) => ipcRenderer.invoke("sim:upload-conversation-recording", input),
|
||||
});
|
||||
|
||||
@@ -3,7 +3,7 @@
|
||||
<head>
|
||||
<meta charset="UTF-8" />
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||
<meta http-equiv="Content-Security-Policy" content="default-src 'self'; script-src 'self'; style-src 'self'; connect-src 'self' https: wss: http://127.0.0.1:* http://localhost:*; media-src 'self' blob:; worker-src 'self' blob:; img-src 'self' data:;" />
|
||||
<meta http-equiv="Content-Security-Policy" content="default-src 'self'; script-src 'self' data:; style-src 'self'; connect-src 'self' data: https://121.41.46.238 https://*.rtc.volcvideo.com https://*.volcvideos.com wss: http://127.0.0.1:* http://localhost:*; media-src 'self' blob:; worker-src 'self' blob: data:; img-src 'self' data:;" />
|
||||
<title>语音终端模拟器</title>
|
||||
</head>
|
||||
<body>
|
||||
|
||||
@@ -0,0 +1,20 @@
|
||||
@echo off
|
||||
setlocal
|
||||
set "SECRET_FILE=%~dp0.server-device-secret"
|
||||
set "DEVICE_EXE=%~dp0voice-device-simulator-0.1.0-windows-x64.exe"
|
||||
|
||||
if not exist "%SECRET_FILE%" (
|
||||
echo Cloud device credential is missing: %SECRET_FILE%
|
||||
pause
|
||||
exit /b 1
|
||||
)
|
||||
if not exist "%DEVICE_EXE%" (
|
||||
echo Voice device simulator is missing: %DEVICE_EXE%
|
||||
pause
|
||||
exit /b 1
|
||||
)
|
||||
|
||||
set /p SIMULATOR_DEFAULT_DEVICE_SECRET=<"%SECRET_FILE%"
|
||||
set "SIMULATOR_DEFAULT_API_URL=https://121.41.46.238"
|
||||
set "SIMULATOR_DEFAULT_DEVICE_ID=10000000-0000-0000-0000-000000000511"
|
||||
start "" "%DEVICE_EXE%"
|
||||
@@ -0,0 +1,25 @@
|
||||
[CmdletBinding()]
|
||||
param()
|
||||
|
||||
$ErrorActionPreference = "Stop"
|
||||
$secretPath = Join-Path $env:LOCALAPPDATA "DataTerminalVoiceDevice\server-device-secret"
|
||||
$executablePath = Join-Path $PSScriptRoot "voice-device-simulator-0.1.0-windows-x64.exe"
|
||||
|
||||
if (-not (Test-Path -LiteralPath $secretPath -PathType Leaf)) {
|
||||
throw "Cloud device credential is missing: $secretPath"
|
||||
}
|
||||
if (-not (Test-Path -LiteralPath $executablePath -PathType Leaf)) {
|
||||
throw "Voice device simulator is missing: $executablePath"
|
||||
}
|
||||
|
||||
$env:SIMULATOR_DEFAULT_API_URL = "https://121.41.46.238"
|
||||
$env:SIMULATOR_DEFAULT_DEVICE_ID = "10000000-0000-0000-0000-000000000511"
|
||||
$env:SIMULATOR_DEFAULT_DEVICE_SECRET = [IO.File]::ReadAllText($secretPath).Trim()
|
||||
try {
|
||||
Start-Process -FilePath $executablePath
|
||||
}
|
||||
finally {
|
||||
Remove-Item Env:SIMULATOR_DEFAULT_API_URL -ErrorAction SilentlyContinue
|
||||
Remove-Item Env:SIMULATOR_DEFAULT_DEVICE_ID -ErrorAction SilentlyContinue
|
||||
Remove-Item Env:SIMULATOR_DEFAULT_DEVICE_SECRET -ErrorAction SilentlyContinue
|
||||
}
|
||||
+40
-1
@@ -4,6 +4,10 @@ type SimulatorConfiguration = {
|
||||
deviceSecret: string;
|
||||
};
|
||||
|
||||
type SimulatorRuntimeDefaults = SimulatorConfiguration & {
|
||||
deviceSecretConfigured: boolean;
|
||||
};
|
||||
|
||||
type RTCGrant = {
|
||||
app_id: string;
|
||||
room_id: string;
|
||||
@@ -17,6 +21,12 @@ type CreatedConversation = {
|
||||
conversation_id: string;
|
||||
status: string;
|
||||
rtc: RTCGrant;
|
||||
privacy: {
|
||||
retention_mode: "none" | "transcript_only" | "transcript_and_audio";
|
||||
record_near: boolean;
|
||||
record_far: boolean;
|
||||
upload_allowed: boolean;
|
||||
};
|
||||
session: {
|
||||
engine_mode: string;
|
||||
heartbeat_interval_sec: number;
|
||||
@@ -24,10 +34,23 @@ type CreatedConversation = {
|
||||
};
|
||||
};
|
||||
|
||||
type DeviceFinalTurn = {
|
||||
turn_key: string;
|
||||
sequence_no: number;
|
||||
speaker: "subject" | "agent";
|
||||
start_at: string;
|
||||
end_at: string;
|
||||
final_text: string;
|
||||
interrupted: boolean;
|
||||
};
|
||||
|
||||
interface Window {
|
||||
deviceSimulator: {
|
||||
getRuntimeDefaults(): Promise<SimulatorRuntimeDefaults>;
|
||||
configure(input: SimulatorConfiguration): Promise<{ apiURL: string; deviceID: string }>;
|
||||
probe(): Promise<Record<string, unknown>>;
|
||||
probeRTCNetwork(): Promise<{ endpoints: Array<{ hostname: string; ok: boolean; error?: string }> }>;
|
||||
getAutomationAudio(): Promise<{ data_url: string; source_name: string } | undefined>;
|
||||
createConversation(): Promise<CreatedConversation>;
|
||||
activateConversation(input: { conversationID: string; roomID: string; userID: string }): Promise<Record<string, unknown>>;
|
||||
getConversation(conversationID: string): Promise<{ status: string; action: string; failure_code?: string }>;
|
||||
@@ -35,6 +58,22 @@ interface Window {
|
||||
actions?: Array<{ type: string; rtc?: RTCGrant }>;
|
||||
desired_config_revision?: number;
|
||||
}>;
|
||||
stopConversation(input: { conversationID: string; reason: string; lastHeartbeatSequence: number }): Promise<Record<string, unknown>>;
|
||||
submitTranscriptBatch(input: { conversationID: string; turns: DeviceFinalTurn[] }): Promise<{
|
||||
conversation_id: string;
|
||||
accepted_count: number;
|
||||
duplicate_count: number;
|
||||
}>;
|
||||
stopConversation(input: { conversationID: string; reason: string; lastHeartbeatSequence: number; nearSegments: number; farSegments: number }): Promise<Record<string, unknown>>;
|
||||
uploadConversationRecording(input: {
|
||||
conversationID: string;
|
||||
segments: Array<{
|
||||
track: "near" | "far";
|
||||
sequence_no: number;
|
||||
codec: "wav_pcm_s16le";
|
||||
content_type: "audio/wav";
|
||||
duration_ms: number;
|
||||
bytes: Uint8Array;
|
||||
}>;
|
||||
}): Promise<{ id: string; status: string; segments: Array<{ id: string; track: "near" | "far"; sequence_no: number; status: string }> }>;
|
||||
};
|
||||
}
|
||||
|
||||
@@ -1,10 +1,20 @@
|
||||
import VERTC, {
|
||||
AudioSourceType,
|
||||
AudioFrameData,
|
||||
IRTCEngine,
|
||||
LocalAudioPropertiesInfo,
|
||||
MediaType,
|
||||
RemoteAudioPropertiesInfo,
|
||||
RoomProfileType,
|
||||
RoomProfileType,
|
||||
StreamIndex,
|
||||
SUBTITLE_MODE,
|
||||
SubtitleEvent,
|
||||
SubtitleEventType,
|
||||
SubtitleMessage,
|
||||
} from "@volcengine/rtc";
|
||||
import { PCMTrackRecorder, RecordedAudioSegment } from "./recording";
|
||||
|
||||
VERTC.setLogConfig({ logLevel: "trace", LogfileSize: 20 });
|
||||
import "./style.css";
|
||||
|
||||
const EngineEventsTypes = VERTC.events;
|
||||
@@ -25,6 +35,19 @@ const state: {
|
||||
agentSpeaking: boolean;
|
||||
microphoneVolume: number;
|
||||
agentVolume: number;
|
||||
subtitleStarted: boolean;
|
||||
transcriptSequence: number;
|
||||
transcriptFirstSeen: Map<string, number>;
|
||||
collectedTurnKeys: Set<string>;
|
||||
pendingTurns: DeviceFinalTurn[];
|
||||
transcriptUpload?: Promise<void>;
|
||||
subtitleStartWaiter?: { resolve: () => void; reject: (error: Error) => void };
|
||||
automationAudio?: HTMLAudioElement;
|
||||
automationAudioContext?: AudioContext;
|
||||
automationAudioTrack?: MediaStreamTrack;
|
||||
automationAudioObjectURL?: string;
|
||||
nearRecorder?: PCMTrackRecorder;
|
||||
farRecorder?: PCMTrackRecorder;
|
||||
} = {
|
||||
phase: "idle",
|
||||
heartbeatSequence: 0,
|
||||
@@ -32,6 +55,11 @@ const state: {
|
||||
agentSpeaking: false,
|
||||
microphoneVolume: 0,
|
||||
agentVolume: 0,
|
||||
subtitleStarted: false,
|
||||
transcriptSequence: 0,
|
||||
transcriptFirstSeen: new Map(),
|
||||
collectedTurnKeys: new Set(),
|
||||
pendingTurns: [],
|
||||
};
|
||||
|
||||
const app = document.querySelector<HTMLElement>("#app")!;
|
||||
@@ -47,7 +75,7 @@ app.innerHTML = `
|
||||
</header>
|
||||
|
||||
<section class="status-grid" aria-label="链路状态">
|
||||
<article class="status-card"><span>本地平台</span><strong id="apiStatus">未检测</strong><small id="apiDetail">localhost</small></article>
|
||||
<article class="status-card"><span>云端平台</span><strong id="apiStatus">未检测</strong><small id="apiDetail">121.41.46.238</small></article>
|
||||
<article class="status-card"><span>RTC 房间</span><strong id="rtcStatus">未加入</strong><small id="rtcDetail">等待开始</small></article>
|
||||
<article class="status-card"><span>豆包智能体</span><strong id="agentStatus">未进房</strong><small id="agentDetail">等待 StartVoiceChat</small></article>
|
||||
<article class="status-card"><span>通话时长</span><strong id="duration">00:00</strong><small id="modeDetail">处理方式：—</small></article>
|
||||
@@ -55,16 +83,16 @@ app.innerHTML = `
|
||||
|
||||
<section class="content-grid">
|
||||
<article class="panel setup-panel">
|
||||
<div class="panel-heading"><div><p class="eyebrow">LOCAL SETUP</p><h2>本地设备配置</h2></div><span class="safe">密钥仅驻留内存</span></div>
|
||||
<label>本地平台地址<input id="apiURL" value="http://localhost:58080" autocomplete="off" /></label>
|
||||
<div class="panel-heading"><div><p class="eyebrow">DEVICE SETUP</p><h2>测试设备配置</h2></div><span class="safe" id="credentialStatus">正在检查凭据</span></div>
|
||||
<label>云端平台地址<input id="apiURL" value="https://121.41.46.238" autocomplete="off" /></label>
|
||||
<label>模拟设备 ID<input id="deviceID" value="10000000-0000-0000-0000-000000000511" autocomplete="off" /></label>
|
||||
<label>设备签名密钥<input id="deviceSecret" type="password" value="local-simulated-device-secret-32-bytes-minimum" autocomplete="off" /></label>
|
||||
<label>设备签名密钥<input id="deviceSecret" type="password" value="" autocomplete="off" /></label>
|
||||
<div class="row">
|
||||
<label>麦克风<select id="microphone"><option value="">默认麦克风</option></select></label>
|
||||
<label>扬声器<select id="speaker"><option value="">默认扬声器</option></select></label>
|
||||
</div>
|
||||
<button class="secondary" id="checkButton">检测平台和音频设备</button>
|
||||
<p class="hint">RTC AppKey、火山 AK/SK 和豆包 Token 应配置在本地后端，本软件不会读取或保存它们。</p>
|
||||
<p class="hint">RTC AppKey、火山 AK/SK 和豆包 Token 配置在云端后台，本软件不会读取或保存它们。</p>
|
||||
</article>
|
||||
|
||||
<article class="panel call-panel">
|
||||
@@ -83,14 +111,20 @@ app.innerHTML = `
|
||||
<div><dt>RTC 房间</dt><dd id="roomID">—</dd></div>
|
||||
<div><dt>终端用户</dt><dd id="userID">—</dd></div>
|
||||
<div><dt>智能体用户</dt><dd id="agentUserID">—</dd></div>
|
||||
<div><dt>聊天语音</dt><dd id="recordingStatus">等待通话</dd></div>
|
||||
</dl>
|
||||
</article>
|
||||
</section>
|
||||
|
||||
<section class="panel log-panel">
|
||||
<div class="panel-heading"><div><p class="eyebrow">EVENT STREAM</p><h2>实时事件</h2></div><button class="text-button" id="clearLog">清空</button></div>
|
||||
<div class="panel-heading"><div><p class="eyebrow">EVENT STREAM</p><h2>实时事件</h2></div><div><button class="text-button" id="downloadRtcLog">导出 RTC 日志</button><button class="text-button" id="clearLog">清空</button></div></div>
|
||||
<ol id="eventLog" aria-live="polite"><li><time>系统</time><span>模拟器已启动，等待检测本地环境。</span></li></ol>
|
||||
</section>
|
||||
|
||||
<section class="panel transcript-panel">
|
||||
<div class="panel-heading"><div><p class="eyebrow">LIVE TRANSCRIPT</p><h2>实时字幕与上传</h2></div><span class="phase-badge" id="transcriptStatus">未启动</span></div>
|
||||
<ol id="liveTranscript" aria-live="polite"><li class="transcript-empty">通话开始后，这里会显示双方的最终字幕。</li></ol>
|
||||
</section>
|
||||
</section>
|
||||
`;
|
||||
|
||||
@@ -106,11 +140,25 @@ const controls = {
|
||||
stop: $<HTMLButtonElement>("stopButton"),
|
||||
};
|
||||
|
||||
void window.deviceSimulator.getRuntimeDefaults().then((defaults) => {
|
||||
controls.apiURL.value = defaults.apiURL;
|
||||
controls.deviceID.value = defaults.deviceID;
|
||||
controls.deviceSecret.value = defaults.deviceSecret;
|
||||
if (defaults.deviceSecretConfigured) {
|
||||
controls.deviceSecret.placeholder = "已从Windows安全凭据文件加载";
|
||||
setText("credentialStatus", "凭据已安全加载");
|
||||
} else {
|
||||
controls.deviceSecret.placeholder = "请输入至少32字符的设备密钥";
|
||||
setText("credentialStatus", "凭据未加载");
|
||||
}
|
||||
}).catch((error) => log(`读取测试设备启动配置失败：${errorMessage(error)}`, "error"));
|
||||
|
||||
function setText(id: string, value: string) {
|
||||
$(id).textContent = value;
|
||||
}
|
||||
|
||||
function log(message: string, level: "info" | "success" | "error" = "info") {
|
||||
console.log(`[${level}] ${message}`);
|
||||
const item = document.createElement("li");
|
||||
item.className = level;
|
||||
const time = document.createElement("time");
|
||||
@@ -154,10 +202,16 @@ async function configureAndProbe() {
|
||||
if (!permission.audio) throw permission.audioExceptionError || new Error("未获得麦克风权限");
|
||||
await refreshAudioDevices();
|
||||
await window.deviceSimulator.probe();
|
||||
const rtcNetwork = await window.deviceSimulator.probeRTCNetwork();
|
||||
const blockedEndpoints = rtcNetwork.endpoints.filter((endpoint) => !endpoint.ok);
|
||||
if (blockedEndpoints.length > 0) {
|
||||
const details = blockedEndpoints.map((endpoint) => `${endpoint.hostname}（${endpoint.error || "连接失败"}）`).join("、");
|
||||
throw new Error(`RTC 网络检测失败：${details}。请关闭代理/VPN、放行 TCP 443，或改用手机热点后重试`);
|
||||
}
|
||||
setText("apiStatus", "连接正常");
|
||||
setText("apiDetail", controls.apiURL.value);
|
||||
setPhase("ready", "可以开始通话");
|
||||
log(`环境检测通过，RTC Web SDK ${VERTC.getSdkVersion()}。`, "success");
|
||||
log(`环境检测通过，RTC Web SDK ${VERTC.getSdkVersion()}，火山 RTC 接入域名可达。`, "success");
|
||||
} catch (error) {
|
||||
setText("apiStatus", "检测失败");
|
||||
setText("apiDetail", errorMessage(error));
|
||||
@@ -184,6 +238,30 @@ function fillDeviceOptions(select: HTMLSelectElement, devices: MediaDeviceInfo[]
|
||||
}
|
||||
|
||||
function registerRTCEvents(engine: IRTCEngine) {
|
||||
engine.on(EngineEventsTypes.onSubtitleStateChanged, (event: SubtitleEvent) => {
|
||||
if (event.event === SubtitleEventType.STARTED) {
|
||||
state.subtitleStarted = true;
|
||||
state.subtitleStartWaiter?.resolve();
|
||||
setText("transcriptStatus", "识别中");
|
||||
log("RTC 实时字幕已启动，双方最终话轮将自动上传。", "success");
|
||||
return;
|
||||
}
|
||||
if (event.event === SubtitleEventType.STOPPED) {
|
||||
state.subtitleStarted = false;
|
||||
state.subtitleStartWaiter?.reject(new Error("RTC 实时字幕在启动期间被停止"));
|
||||
setText("transcriptStatus", "已停止");
|
||||
return;
|
||||
}
|
||||
if (event.event === SubtitleEventType.ERROR) {
|
||||
state.subtitleStarted = false;
|
||||
state.subtitleStartWaiter?.reject(new Error(`RTC 实时字幕启动失败：${event.errorCode || "UNKNOWN"}${event.errorMessage ? ` · ${event.errorMessage}` : ""}`));
|
||||
setText("transcriptStatus", "识别失败");
|
||||
log(`RTC 实时字幕错误：${event.errorCode || "UNKNOWN"}${event.errorMessage ? ` · ${event.errorMessage}` : ""}`, "error");
|
||||
}
|
||||
});
|
||||
engine.on(EngineEventsTypes.onSubtitleMessageReceived, (messages: SubtitleMessage[]) => {
|
||||
collectSubtitleMessages(messages);
|
||||
});
|
||||
engine.on(EngineEventsTypes.onUserJoined, (event) => {
|
||||
const userID = event.userInfo.userId;
|
||||
log(`远端用户已进房：${userID}`, "success");
|
||||
@@ -227,7 +305,17 @@ function registerRTCEvents(engine: IRTCEngine) {
|
||||
setPhase(state.phase, state.phase === "active" ? (state.agentSpeaking ? "豆包正在说话" : "请开始说话") : undefined);
|
||||
});
|
||||
engine.on(EngineEventsTypes.onConnectionStateChanged, (event) => {
|
||||
log(`RTC 连接状态：${String(event.state)}`);
|
||||
const names: Record<number, string> = {
|
||||
0: "准备连接",
|
||||
1: "已断开",
|
||||
2: "连接中",
|
||||
3: "已连接",
|
||||
4: "重连中",
|
||||
5: "已重连",
|
||||
6: "连接丢失",
|
||||
};
|
||||
const level = event.state === 1 || event.state === 6 ? "error" : "info";
|
||||
log(`RTC 连接状态：${String(event.state)}（${names[event.state] || "未知"}）${event.reason !== undefined ? `，原因 ${String(event.reason)}` : ""}`, level);
|
||||
});
|
||||
engine.on(EngineEventsTypes.onError, (event) => {
|
||||
log(`RTC 错误：${event.errorCode}`, "error");
|
||||
@@ -237,6 +325,106 @@ function registerRTCEvents(engine: IRTCEngine) {
|
||||
});
|
||||
}
|
||||
|
||||
function subtitleIdentity(message: SubtitleMessage) {
|
||||
const conversation = state.conversation;
|
||||
if (!conversation) return undefined;
|
||||
if (message.userId === conversation.rtc.user_id) return { speaker: "subject" as const, label: "长者" };
|
||||
if (message.userId === conversation.rtc.agent_user_id) return { speaker: "agent" as const, label: "豆包" };
|
||||
return undefined;
|
||||
}
|
||||
|
||||
function subtitleKey(message: SubtitleMessage) {
|
||||
return `${message.userId}:${message.sequence}`;
|
||||
}
|
||||
|
||||
function collectSubtitleMessages(messages: SubtitleMessage[]) {
|
||||
const now = Date.now();
|
||||
for (const message of messages) {
|
||||
const identity = subtitleIdentity(message);
|
||||
const text = message.text.trim();
|
||||
if (!identity || !text || message.mode !== SUBTITLE_MODE.ASR_ONLY) continue;
|
||||
const sourceKey = subtitleKey(message);
|
||||
if (!state.transcriptFirstSeen.has(sourceKey)) state.transcriptFirstSeen.set(sourceKey, now);
|
||||
if (!message.definite || state.collectedTurnKeys.has(sourceKey)) continue;
|
||||
|
||||
state.collectedTurnKeys.add(sourceKey);
|
||||
state.transcriptSequence += 1;
|
||||
const turn: DeviceFinalTurn = {
|
||||
turn_key: `rtc-subtitle-${identity.speaker}-${message.sequence}`,
|
||||
sequence_no: state.transcriptSequence,
|
||||
speaker: identity.speaker,
|
||||
start_at: new Date(state.transcriptFirstSeen.get(sourceKey) || now).toISOString(),
|
||||
end_at: new Date(now).toISOString(),
|
||||
final_text: text,
|
||||
interrupted: false,
|
||||
};
|
||||
state.pendingTurns.push(turn);
|
||||
renderTranscriptTurn(turn, identity.label);
|
||||
}
|
||||
void flushTranscriptTurns();
|
||||
}
|
||||
|
||||
function renderTranscriptTurn(turn: DeviceFinalTurn, label: string) {
|
||||
const list = $("liveTranscript");
|
||||
list.querySelector(".transcript-empty")?.remove();
|
||||
const item = document.createElement("li");
|
||||
item.className = `transcript-turn transcript-turn--${turn.speaker}`;
|
||||
const speaker = document.createElement("strong");
|
||||
speaker.textContent = label;
|
||||
const text = document.createElement("span");
|
||||
text.textContent = turn.final_text;
|
||||
item.append(speaker, text);
|
||||
list.append(item);
|
||||
}
|
||||
|
||||
async function flushTranscriptTurns() {
|
||||
if (state.transcriptUpload) return state.transcriptUpload;
|
||||
if (!state.conversation || state.pendingTurns.length === 0) return;
|
||||
const conversationID = state.conversation.conversation_id;
|
||||
const batch = state.pendingTurns.slice(0, 500);
|
||||
state.transcriptUpload = (async () => {
|
||||
try {
|
||||
const result = await window.deviceSimulator.submitTranscriptBatch({ conversationID, turns: batch });
|
||||
state.pendingTurns.splice(0, batch.length);
|
||||
const uploaded = state.transcriptSequence - state.pendingTurns.length;
|
||||
setText("transcriptStatus", state.subtitleStarted ? `识别中 · 已上传 ${uploaded}` : `已上传 ${uploaded}`);
|
||||
log(`最终字幕已上传：新增 ${result.accepted_count} 条，重复 ${result.duplicate_count} 条。`, "success");
|
||||
} catch (error) {
|
||||
setText("transcriptStatus", `待重试 ${state.pendingTurns.length} 条`);
|
||||
log(`最终字幕上传失败，将在心跳或结束通话时重试：${errorMessage(error)}`, "error");
|
||||
} finally {
|
||||
state.transcriptUpload = undefined;
|
||||
}
|
||||
})();
|
||||
await state.transcriptUpload;
|
||||
if (state.pendingTurns.length > 0 && state.pendingTurns.length !== batch.length) void flushTranscriptTurns();
|
||||
}
|
||||
|
||||
async function flushTranscriptTurnsWithRetry(attempts: number) {
|
||||
for (let attempt = 1; attempt <= attempts && state.pendingTurns.length > 0; attempt += 1) {
|
||||
await flushTranscriptTurns();
|
||||
if (state.pendingTurns.length === 0) return;
|
||||
if (attempt < attempts) await new Promise((resolve) => window.setTimeout(resolve, attempt * 500));
|
||||
}
|
||||
}
|
||||
|
||||
async function startSubtitleAndWait(engine: IRTCEngine) {
|
||||
let timeoutID: number | undefined;
|
||||
const stateEvent = new Promise<void>((resolve, reject) => {
|
||||
state.subtitleStartWaiter = { resolve, reject };
|
||||
});
|
||||
const timeout = new Promise<void>((_resolve, reject) => {
|
||||
timeoutID = window.setTimeout(() => reject(new Error("等待 RTC 实时字幕启动超时")), 10_000);
|
||||
});
|
||||
try {
|
||||
await engine.startSubtitle({ mode: SUBTITLE_MODE.ASR_ONLY });
|
||||
await Promise.race([stateEvent, timeout]);
|
||||
} finally {
|
||||
if (timeoutID) window.clearTimeout(timeoutID);
|
||||
state.subtitleStartWaiter = undefined;
|
||||
}
|
||||
}
|
||||
|
||||
function updateVolume(kind: "mic" | "agent", raw: number) {
|
||||
const value = Math.max(0, Math.min(100, Math.round(raw)));
|
||||
if (kind === "mic") state.microphoneVolume = value;
|
||||
@@ -254,6 +442,15 @@ async function startConversation() {
|
||||
state.credentialExpiresAt = created.rtc.expires_at;
|
||||
state.heartbeatSequence = 0;
|
||||
state.agentJoined = false;
|
||||
state.subtitleStarted = false;
|
||||
state.transcriptSequence = 0;
|
||||
state.transcriptFirstSeen.clear();
|
||||
state.collectedTurnKeys.clear();
|
||||
state.pendingTurns = [];
|
||||
state.transcriptUpload = undefined;
|
||||
state.nearRecorder = undefined;
|
||||
state.farRecorder = undefined;
|
||||
$("liveTranscript").innerHTML = '<li class="transcript-empty">正在等待双方的最终字幕。</li>';
|
||||
renderSession(created);
|
||||
log(`平台已分配 RTC 房间 ${created.rtc.room_id}。`, "success");
|
||||
|
||||
@@ -261,18 +458,27 @@ async function startConversation() {
|
||||
state.engine = engine;
|
||||
registerRTCEvents(engine);
|
||||
engine.enableAudioPropertiesReport({ interval: 300 });
|
||||
await engine.startAudioCapture(controls.microphone.value || undefined);
|
||||
configureRecording(engine, created);
|
||||
const automatedAudio = await prepareAutomationAudio(engine);
|
||||
if (!automatedAudio) await engine.startAudioCapture(controls.microphone.value || undefined);
|
||||
if (controls.speaker.value) await engine.setAudioPlaybackDevice(controls.speaker.value);
|
||||
await engine.joinRoom(created.rtc.token, created.rtc.room_id, { userId: created.rtc.user_id }, {
|
||||
isAutoPublish: true,
|
||||
isAutoSubscribeAudio: true,
|
||||
isAutoSubscribeVideo: false,
|
||||
roomProfileType: RoomProfileType.chat,
|
||||
});
|
||||
log("正在连接火山 RTC 信令并加入房间。", "info");
|
||||
await Promise.race([
|
||||
engine.joinRoom(created.rtc.token, created.rtc.room_id, { userId: created.rtc.user_id }, {
|
||||
isAutoPublish: true,
|
||||
isAutoSubscribeAudio: true,
|
||||
isAutoSubscribeVideo: false,
|
||||
roomProfileType: RoomProfileType.chat,
|
||||
}),
|
||||
new Promise<never>((_resolve, reject) => window.setTimeout(() => reject(new Error("RTC 入房超过 15 秒。请关闭代理/VPN、允许本程序通过 Windows 防火墙，或改用手机热点后重试")), 15_000)),
|
||||
]);
|
||||
setText("rtcStatus", "已加入");
|
||||
setText("rtcDetail", created.rtc.room_id);
|
||||
log("终端已加入 RTC 房间并发布麦克风。", "success");
|
||||
|
||||
setText("transcriptStatus", "正在启动");
|
||||
await startSubtitleAndWait(engine);
|
||||
|
||||
await window.deviceSimulator.activateConversation({
|
||||
conversationID: created.conversation_id,
|
||||
roomID: created.rtc.room_id,
|
||||
@@ -285,13 +491,96 @@ async function startConversation() {
|
||||
setPhase("active", "请开始说话");
|
||||
startTimers(created.session.heartbeat_interval_sec || 15);
|
||||
log("豆包语音任务已激活，可以开始双向聊天。", "success");
|
||||
if (automatedAudio) {
|
||||
await new Promise((resolve) => window.setTimeout(resolve, 4_000));
|
||||
await state.automationAudioContext?.resume();
|
||||
await state.automationAudio?.play();
|
||||
log("自动验证音频已通过 RTC 自定义音轨发送。", "success");
|
||||
}
|
||||
} catch (error) {
|
||||
setPhase("error", "启动失败");
|
||||
log(`启动通话失败：${errorMessage(error)}`, "error");
|
||||
await releaseRTC();
|
||||
if (state.conversation) {
|
||||
try {
|
||||
await window.deviceSimulator.stopConversation({
|
||||
conversationID: state.conversation.conversation_id,
|
||||
reason: "windows_start_failed",
|
||||
lastHeartbeatSequence: state.heartbeatSequence,
|
||||
nearSegments: 0,
|
||||
farSegments: 0,
|
||||
});
|
||||
} catch (cleanupError) {
|
||||
log(`清理失败通话时发生错误：${errorMessage(cleanupError)}`, "error");
|
||||
}
|
||||
}
|
||||
resetSession();
|
||||
}
|
||||
}
|
||||
|
||||
function configureRecording(engine: IRTCEngine, conversation: CreatedConversation) {
|
||||
const privacy = conversation.privacy;
|
||||
if (!privacy.upload_allowed || (!privacy.record_near && !privacy.record_far)) {
|
||||
setText("recordingStatus", "本次未授权录音");
|
||||
log("本次通话的隐私策略未授权保存原录音，不采集音频帧。", "info");
|
||||
return;
|
||||
}
|
||||
if (privacy.record_near) {
|
||||
state.nearRecorder = new PCMTrackRecorder();
|
||||
engine.setAudioFrameCallback(StreamIndex.STREAM_INDEX_MAIN, undefined, (frame: AudioFrameData) => state.nearRecorder?.append(frame), 4096);
|
||||
}
|
||||
if (privacy.record_far) {
|
||||
state.farRecorder = new PCMTrackRecorder();
|
||||
engine.setAudioFrameCallback(StreamIndex.STREAM_INDEX_MAIN, conversation.rtc.agent_user_id, (frame: AudioFrameData) => state.farRecorder?.append(frame), 4096);
|
||||
}
|
||||
setText("recordingStatus", "录音中");
|
||||
log(`已按授权开始采集${privacy.record_near ? "长者近端" : ""}${privacy.record_near && privacy.record_far ? "和" : ""}${privacy.record_far ? "智能体远端" : ""} PCM 音频。`, "success");
|
||||
}
|
||||
|
||||
function finishRecording(engine: IRTCEngine | undefined, conversation: CreatedConversation) {
|
||||
try { engine?.setAudioFrameCallback(StreamIndex.STREAM_INDEX_MAIN, undefined, undefined); } catch { /* no-op */ }
|
||||
try { engine?.setAudioFrameCallback(StreamIndex.STREAM_INDEX_MAIN, conversation.rtc.agent_user_id, undefined); } catch { /* no-op */ }
|
||||
const segments = [
|
||||
...(state.nearRecorder?.finish("near") ?? []),
|
||||
...(state.farRecorder?.finish("far") ?? []),
|
||||
];
|
||||
state.nearRecorder = undefined;
|
||||
state.farRecorder = undefined;
|
||||
return segments;
|
||||
}
|
||||
|
||||
async function prepareAutomationAudio(engine: IRTCEngine) {
|
||||
const asset = await window.deviceSimulator.getAutomationAudio();
|
||||
if (!asset) return false;
|
||||
const comma = asset.data_url.indexOf(",");
|
||||
if (comma < 0) throw new Error("自动验证音频编码无效");
|
||||
const metadata = asset.data_url.slice(0, comma);
|
||||
const mime = metadata.match(/^data:([^;]+)/)?.[1] || "audio/wav";
|
||||
const bytes = Uint8Array.from(atob(asset.data_url.slice(comma + 1)), (value) => value.charCodeAt(0));
|
||||
const objectURL = URL.createObjectURL(new Blob([bytes], { type: mime }));
|
||||
const audio = new Audio(objectURL);
|
||||
audio.preload = "auto";
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
audio.addEventListener("canplaythrough", () => resolve(), { once: true });
|
||||
audio.addEventListener("error", () => reject(new Error(`无法加载自动验证音频：${asset.source_name}`)), { once: true });
|
||||
audio.load();
|
||||
});
|
||||
const context = new AudioContext({ sampleRate: 48_000 });
|
||||
const source = context.createMediaElementSource(audio);
|
||||
const destination = context.createMediaStreamDestination();
|
||||
source.connect(destination);
|
||||
const track = destination.stream.getAudioTracks()[0];
|
||||
if (!track) throw new Error("自动验证音频没有可发布的音轨");
|
||||
await engine.setAudioSourceType(StreamIndex.STREAM_INDEX_MAIN, AudioSourceType.AUDIO_SOURCE_TYPE_EXTERNAL);
|
||||
await engine.setExternalAudioTrack(StreamIndex.STREAM_INDEX_MAIN, track);
|
||||
state.automationAudio = audio;
|
||||
state.automationAudioContext = context;
|
||||
state.automationAudioTrack = track;
|
||||
state.automationAudioObjectURL = objectURL;
|
||||
log(`已准备自动验证音频：${asset.source_name}。`, "success");
|
||||
return true;
|
||||
}
|
||||
|
||||
function renderSession(created: CreatedConversation) {
|
||||
setText("conversationID", created.conversation_id);
|
||||
setText("roomID", created.rtc.room_id);
|
||||
@@ -340,6 +629,7 @@ async function sendHeartbeat() {
|
||||
log("RTC Token 已通过设备心跳完成续签。", "success");
|
||||
}
|
||||
}
|
||||
await flushTranscriptTurns();
|
||||
} catch (error) {
|
||||
log(`设备心跳失败：${errorMessage(error)}`, "error");
|
||||
}
|
||||
@@ -350,15 +640,59 @@ async function stopConversation(reason = "windows_user_hangup") {
|
||||
setPhase("stopping", "正在结束通话");
|
||||
stopTimers();
|
||||
const conversation = state.conversation;
|
||||
let recordingResult = "等待通话";
|
||||
try {
|
||||
if (state.subtitleStarted) state.engine?.stopSubtitle();
|
||||
await new Promise((resolve) => window.setTimeout(resolve, 800));
|
||||
await flushTranscriptTurnsWithRetry(3);
|
||||
if (state.pendingTurns.length > 0) {
|
||||
log(`仍有 ${state.pendingTurns.length} 条最终字幕未上传，通话暂不结束并继续自动重试。`, "error");
|
||||
try {
|
||||
setText("transcriptStatus", "正在恢复识别");
|
||||
await state.engine?.startSubtitle({ mode: SUBTITLE_MODE.ASR_ONLY });
|
||||
} catch (error) {
|
||||
log(`恢复实时字幕失败：${errorMessage(error)}`, "error");
|
||||
}
|
||||
setPhase("active", "字幕待上传");
|
||||
startTimers(conversation.session.heartbeat_interval_sec || 15);
|
||||
return;
|
||||
}
|
||||
setText("recordingStatus", "正在封装录音");
|
||||
const recordingSegments = finishRecording(state.engine, conversation);
|
||||
await releaseRTC();
|
||||
await window.deviceSimulator.stopConversation({
|
||||
conversationID: conversation.conversation_id,
|
||||
reason,
|
||||
lastHeartbeatSequence: state.heartbeatSequence,
|
||||
nearSegments: recordingSegments.filter((segment) => segment.track === "near").length,
|
||||
farSegments: recordingSegments.filter((segment) => segment.track === "far").length,
|
||||
});
|
||||
log("RTC 已离房，平台已提交 StopVoiceChat。", "success");
|
||||
if (recordingSegments.length > 0) {
|
||||
try {
|
||||
setText("recordingStatus", "等待通话结束");
|
||||
await waitUntilTerminal(conversation.conversation_id);
|
||||
setText("recordingStatus", "正在上传");
|
||||
const totalBytes = recordingSegments.reduce((sum, segment) => sum + segment.bytes.byteLength, 0);
|
||||
log(`正在上传 ${recordingSegments.length} 个聊天语音分片（${formatRecordingBytes(totalBytes)}）。`, "info");
|
||||
const batch = await uploadRecordingWithRetry(conversation.conversation_id, recordingSegments, 3);
|
||||
setText("recordingStatus", "已上传并校验");
|
||||
recordingResult = "本次已上传";
|
||||
log(`聊天语音已上传并通过服务器校验，批次 ${batch.id}。`, "success");
|
||||
} catch (uploadError) {
|
||||
setText("recordingStatus", "上传失败");
|
||||
recordingResult = "本次上传失败";
|
||||
log(`聊天语音上传失败：${errorMessage(uploadError)}`, "error");
|
||||
}
|
||||
} else if (conversation.privacy.upload_allowed) {
|
||||
setText("recordingStatus", "没有可上传音频");
|
||||
recordingResult = "本次没有音频";
|
||||
log("本次没有收到可封装的 PCM 音频帧，未创建录音上传批次。", "info");
|
||||
} else {
|
||||
recordingResult = "本次未授权录音";
|
||||
}
|
||||
resetSession();
|
||||
setText("recordingStatus", recordingResult);
|
||||
setPhase("ready", "可以再次通话");
|
||||
} catch (error) {
|
||||
log(`结束通话失败：${errorMessage(error)}`, "error");
|
||||
@@ -367,10 +701,53 @@ async function stopConversation(reason = "windows_user_hangup") {
|
||||
}
|
||||
}
|
||||
|
||||
async function waitUntilTerminal(conversationID: string) {
|
||||
const deadline = Date.now() + 60_000;
|
||||
while (Date.now() < deadline) {
|
||||
const result = await window.deviceSimulator.getConversation(conversationID);
|
||||
if (result.status === "COMPLETED" || result.status === "FAILED") return;
|
||||
if (result.status === "EXPIRED") throw new Error("通话已过期，平台不接受录音上传");
|
||||
await new Promise((resolve) => window.setTimeout(resolve, 750));
|
||||
}
|
||||
throw new Error("等待通话进入结束状态超时");
|
||||
}
|
||||
|
||||
async function uploadRecordingWithRetry(conversationID: string, segments: RecordedAudioSegment[], attempts: number) {
|
||||
let lastError: unknown;
|
||||
for (let attempt = 1; attempt <= attempts; attempt += 1) {
|
||||
try {
|
||||
return await window.deviceSimulator.uploadConversationRecording({ conversationID, segments });
|
||||
} catch (error) {
|
||||
lastError = error;
|
||||
if (attempt < attempts) {
|
||||
log(`录音上传第 ${attempt} 次尝试失败，正在重试：${errorMessage(error)}`, "error");
|
||||
await new Promise((resolve) => window.setTimeout(resolve, attempt * 1000));
|
||||
}
|
||||
}
|
||||
}
|
||||
throw lastError;
|
||||
}
|
||||
|
||||
function formatRecordingBytes(value: number) {
|
||||
if (value < 1024 * 1024) return `${(value / 1024).toFixed(1)} KiB`;
|
||||
return `${(value / 1024 / 1024).toFixed(1)} MiB`;
|
||||
}
|
||||
|
||||
async function releaseRTC() {
|
||||
state.automationAudio?.pause();
|
||||
state.automationAudioTrack?.stop();
|
||||
if (state.automationAudioObjectURL) URL.revokeObjectURL(state.automationAudioObjectURL);
|
||||
if (state.automationAudioContext) {
|
||||
try { await state.automationAudioContext.close(); } catch { /* no-op */ }
|
||||
}
|
||||
state.automationAudio = undefined;
|
||||
state.automationAudioTrack = undefined;
|
||||
state.automationAudioContext = undefined;
|
||||
state.automationAudioObjectURL = undefined;
|
||||
const engine = state.engine;
|
||||
state.engine = undefined;
|
||||
if (!engine) return;
|
||||
try { engine.stopSubtitle(); } catch { /* no-op */ }
|
||||
try { await engine.unpublishStream(MediaType.AUDIO); } catch { /* no-op */ }
|
||||
try { await engine.leaveRoom(); } catch { /* no-op */ }
|
||||
try { await engine.stopAudioCapture(); } catch { /* no-op */ }
|
||||
@@ -392,6 +769,15 @@ function resetSession() {
|
||||
state.startedAt = undefined;
|
||||
state.agentJoined = false;
|
||||
state.agentSpeaking = false;
|
||||
state.subtitleStarted = false;
|
||||
state.transcriptSequence = 0;
|
||||
state.transcriptFirstSeen.clear();
|
||||
state.collectedTurnKeys.clear();
|
||||
state.pendingTurns = [];
|
||||
state.transcriptUpload = undefined;
|
||||
state.subtitleStartWaiter = undefined;
|
||||
state.nearRecorder = undefined;
|
||||
state.farRecorder = undefined;
|
||||
setText("rtcStatus", "未加入");
|
||||
setText("rtcDetail", "等待开始");
|
||||
setText("agentStatus", "未进房");
|
||||
@@ -399,12 +785,19 @@ function resetSession() {
|
||||
setText("duration", "00:00");
|
||||
updateVolume("mic", 0);
|
||||
updateVolume("agent", 0);
|
||||
setText("transcriptStatus", "未启动");
|
||||
setText("recordingStatus", "等待通话");
|
||||
$("liveTranscript").innerHTML = '<li class="transcript-empty">通话开始后，这里会显示双方的最终字幕。</li>';
|
||||
}
|
||||
|
||||
controls.check.addEventListener("click", () => void configureAndProbe());
|
||||
controls.start.addEventListener("click", () => void startConversation());
|
||||
controls.stop.addEventListener("click", () => void stopConversation());
|
||||
$("clearLog").addEventListener("click", () => $("eventLog").replaceChildren());
|
||||
$("downloadRtcLog").addEventListener("click", () => {
|
||||
VERTC.downloadLog();
|
||||
log("已导出火山 RTC SDK 日志，请在 Windows“下载”目录查找最新日志文件。", "success");
|
||||
});
|
||||
window.addEventListener("beforeunload", () => {
|
||||
stopTimers();
|
||||
void releaseRTC();
|
||||
|
||||
@@ -0,0 +1,128 @@
|
||||
export type AudioFrame = {
|
||||
channels: Float32Array[];
|
||||
channelCount: number;
|
||||
sampleRate: number;
|
||||
};
|
||||
|
||||
export type RecordedAudioSegment = {
|
||||
track: "near" | "far";
|
||||
sequence_no: number;
|
||||
codec: "wav_pcm_s16le";
|
||||
content_type: "audio/wav";
|
||||
duration_ms: number;
|
||||
bytes: Uint8Array;
|
||||
};
|
||||
|
||||
const WAV_HEADER_SIZE = 44;
|
||||
const MAX_SEGMENT_DURATION_MS = 9 * 60 * 1000;
|
||||
const MAX_SEGMENT_SIZE = 64 * 1024 * 1024;
|
||||
|
||||
export class PCMTrackRecorder {
|
||||
private sampleRate = 0;
|
||||
private chunks: Float32Array[] = [];
|
||||
private sampleCount = 0;
|
||||
|
||||
append(frame: AudioFrame) {
|
||||
if (!Number.isFinite(frame.sampleRate) || frame.sampleRate < 8_000 || frame.sampleRate > 192_000) return;
|
||||
const availableChannels = frame.channels.slice(0, Math.max(1, frame.channelCount)).filter((channel) => channel.length > 0);
|
||||
if (availableChannels.length === 0) return;
|
||||
const frameLength = Math.min(...availableChannels.map((channel) => channel.length));
|
||||
if (frameLength < 1) return;
|
||||
|
||||
const mono = new Float32Array(frameLength);
|
||||
for (let index = 0; index < frameLength; index += 1) {
|
||||
let value = 0;
|
||||
for (const channel of availableChannels) value += channel[index];
|
||||
mono[index] = value / availableChannels.length;
|
||||
}
|
||||
|
||||
if (this.sampleRate === 0) this.sampleRate = frame.sampleRate;
|
||||
const normalized = frame.sampleRate === this.sampleRate ? mono : resampleLinear(mono, frame.sampleRate, this.sampleRate);
|
||||
if (normalized.length === 0) return;
|
||||
this.chunks.push(normalized);
|
||||
this.sampleCount += normalized.length;
|
||||
}
|
||||
|
||||
finish(track: "near" | "far"): RecordedAudioSegment[] {
|
||||
if (this.sampleRate === 0 || this.sampleCount === 0) return [];
|
||||
const durationLimit = Math.floor(this.sampleRate * MAX_SEGMENT_DURATION_MS / 1000);
|
||||
const sizeLimit = Math.floor((MAX_SEGMENT_SIZE - WAV_HEADER_SIZE) / 2);
|
||||
const samplesPerSegment = Math.max(1, Math.min(durationLimit, sizeLimit));
|
||||
const segments: RecordedAudioSegment[] = [];
|
||||
|
||||
for (let offset = 0, sequence = 0; offset < this.sampleCount; offset += samplesPerSegment, sequence += 1) {
|
||||
const length = Math.min(samplesPerSegment, this.sampleCount - offset);
|
||||
const durationMS = Math.floor(length * 1000 / this.sampleRate);
|
||||
if (durationMS < 1) continue;
|
||||
segments.push({
|
||||
track,
|
||||
sequence_no: sequence,
|
||||
codec: "wav_pcm_s16le",
|
||||
content_type: "audio/wav",
|
||||
duration_ms: durationMS,
|
||||
bytes: encodeMonoWAV(this.chunks, offset, length, this.sampleRate),
|
||||
});
|
||||
}
|
||||
|
||||
this.chunks = [];
|
||||
this.sampleCount = 0;
|
||||
this.sampleRate = 0;
|
||||
return segments;
|
||||
}
|
||||
}
|
||||
|
||||
function resampleLinear(input: Float32Array, sourceRate: number, targetRate: number) {
|
||||
const outputLength = Math.max(1, Math.round(input.length * targetRate / sourceRate));
|
||||
const output = new Float32Array(outputLength);
|
||||
const ratio = sourceRate / targetRate;
|
||||
for (let index = 0; index < outputLength; index += 1) {
|
||||
const position = index * ratio;
|
||||
const left = Math.min(input.length - 1, Math.floor(position));
|
||||
const right = Math.min(input.length - 1, left + 1);
|
||||
const fraction = position - left;
|
||||
output[index] = input[left] + (input[right] - input[left]) * fraction;
|
||||
}
|
||||
return output;
|
||||
}
|
||||
|
||||
function encodeMonoWAV(chunks: Float32Array[], startSample: number, sampleCount: number, sampleRate: number) {
|
||||
const dataSize = sampleCount * 2;
|
||||
const bytes = new Uint8Array(WAV_HEADER_SIZE + dataSize);
|
||||
const view = new DataView(bytes.buffer);
|
||||
writeASCII(bytes, 0, "RIFF");
|
||||
view.setUint32(4, bytes.length - 8, true);
|
||||
writeASCII(bytes, 8, "WAVE");
|
||||
writeASCII(bytes, 12, "fmt ");
|
||||
view.setUint32(16, 16, true);
|
||||
view.setUint16(20, 1, true);
|
||||
view.setUint16(22, 1, true);
|
||||
view.setUint32(24, sampleRate, true);
|
||||
view.setUint32(28, sampleRate * 2, true);
|
||||
view.setUint16(32, 2, true);
|
||||
view.setUint16(34, 16, true);
|
||||
writeASCII(bytes, 36, "data");
|
||||
view.setUint32(40, dataSize, true);
|
||||
|
||||
let skipped = 0;
|
||||
let written = 0;
|
||||
for (const chunk of chunks) {
|
||||
if (skipped + chunk.length <= startSample) {
|
||||
skipped += chunk.length;
|
||||
continue;
|
||||
}
|
||||
const chunkStart = Math.max(0, startSample - skipped);
|
||||
const available = Math.min(chunk.length - chunkStart, sampleCount - written);
|
||||
for (let index = 0; index < available; index += 1) {
|
||||
const sample = Math.max(-1, Math.min(1, chunk[chunkStart + index]));
|
||||
view.setInt16(WAV_HEADER_SIZE + (written + index) * 2, sample < 0 ? Math.round(sample * 32768) : Math.round(sample * 32767), true);
|
||||
}
|
||||
written += available;
|
||||
skipped += chunk.length;
|
||||
if (written === sampleCount) break;
|
||||
}
|
||||
return bytes;
|
||||
}
|
||||
|
||||
function writeASCII(target: Uint8Array, offset: number, value: string) {
|
||||
for (let index = 0; index < value.length; index += 1) target[offset + index] = value.charCodeAt(index);
|
||||
}
|
||||
@@ -68,6 +68,15 @@ input:focus, select:focus { border-color: #d6783f; box-shadow: 0 0 0 3px rgba(21
|
||||
#eventLog li.success span { color: #247252; }
|
||||
#eventLog li.error span { color: #ae3b2f; }
|
||||
|
||||
.transcript-panel { margin-top: 16px; }
|
||||
.transcript-panel .panel-heading { margin-bottom: 14px; }
|
||||
#liveTranscript { display: grid; gap: 10px; margin: 0; padding: 0; max-height: 260px; overflow: auto; list-style: none; }
|
||||
.transcript-empty { color: #829198; font-size: 13px; }
|
||||
.transcript-turn { display: grid; gap: 4px; max-width: 80%; padding: 11px 14px; border-radius: 15px; background: #f1f6f4; }
|
||||
.transcript-turn--agent { justify-self: end; background: #edf3fb; }
|
||||
.transcript-turn strong { color: #315966; font-size: 12px; }
|
||||
.transcript-turn span { color: #263f49; font-size: 14px; line-height: 1.55; }
|
||||
|
||||
@media (max-width: 1050px) {
|
||||
.status-grid { grid-template-columns: repeat(2, 1fr); }
|
||||
.content-grid { grid-template-columns: 1fr; }
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
云端语音测试设备使用说明
|
||||
|
||||
1. 将整个 ZIP 解压到普通文件夹，不要只从压缩包内直接运行。
|
||||
2. 双击“启动云端测试设备.cmd”，不要直接双击 EXE。
|
||||
3. 首次运行时允许 Windows 麦克风访问。
|
||||
4. 点击“检测平台和音频设备”，检测通过后即可开始语音聊天。
|
||||
|
||||
云端后台：https://121.41.46.238
|
||||
测试设备：10000000-0000-0000-0000-000000000511
|
||||
|
||||
网络需要允许 HTTPS 443、WSS 以及火山 RTC 所需的出站网络连接。
|
||||
|
||||
如果检测平台时提示 RTC 网络检测失败，或入房超过 15 秒：
|
||||
- 关闭 VPN、系统代理和网络加速器；
|
||||
- 在 Windows 防火墙/杀毒软件中允许本程序联网；
|
||||
- 优先改用手机热点复测，以排除公司、校园或酒店网络限制；
|
||||
- 需要放行 TCP 443 到火山 RTC Web SDK 4.69.0 使用的 *.rtc.volcvideo.com 和 *.volcvideos.com。
|
||||
|
||||
如果手机热点、关闭VPN/代理后仍然无法入房，请在失败后点击页面底部“导出 RTC 日志”，把 Windows“下载”目录中最新生成的 RTC 日志文件交给开发人员分析。
|
||||
|
||||
安全提示：本测试包包含服务器测试设备凭据，仅限受控测试使用，请勿公开上传或转发。测试完成后请删除其他电脑上的解压目录和 ZIP 文件。
|
||||
+28
-3
@@ -17,11 +17,17 @@
|
||||
| 故事修订 | 候选故事、人工正文、原因、强ETag | 新增人工修订版本并复制来源关系，不覆盖模型版本；终态故事禁止再改 | 新的`story_revision`和`IN_REVIEW`状态 |
|
||||
| 故事确认 | 当前故事、审核原因、强ETag | 必须有原话来源、具备审核权限且个性化记忆授权当前有效；确认故事及候选事实 | `CONFIRMED`故事和`PENDING`记忆 |
|
||||
| 故事拒绝 | 当前故事、审核原因、强ETag | 拒绝故事及仍为候选状态的事实，不创建记忆 | `REJECTED`故事 |
|
||||
| 事实候选细粒度审核 | 单条事实、修订内容、敏感级别、问题类型、原因、强ETag | 事实可独立编辑、确认、拒绝或停用；每次变化保存不可覆盖版本，不要求整篇故事一起处理 | 新的`fact_candidate_revision`与质量反馈 |
|
||||
| 记忆索引 | 待索引记忆版本 | 再次校验记忆为当前版本、状态有效且授权仍有效；不满足条件时清空向量并标记`STALE` | `ACTIVE`或`STALE`索引 |
|
||||
| 生产向量化 | 普通敏感级别的当前记忆正文或新通话检索语句 | API与Worker使用同一方舟Embedding模型；请求1536维float向量并校验维度和有限数值；网络、超时、408/425/429/5xx可靠重试 | `ark:<model-id>`索引或固定边界查询向量 |
|
||||
| 记忆修订与停用 | 当前记忆、正文或状态、原因、强ETag | 正文变化创建新版本并立即使旧索引失效；停用后不再召回；重新启用必须重新校验授权并重新索引；`PENDING_DELETE/SUPERSEDED/DELETED`均禁止修改 | 新记忆版本或状态 |
|
||||
| 记忆永久删除 | 当前记忆、删除原因、强ETag | 事务内先进入`PENDING_DELETE`并清空活动向量，使新通话立即停止召回；该状态不可通过修改接口恢复；Outbox后台把索引标记删除、清除所有记忆版本正文并记录完成审计 | `DELETED`记忆和索引 |
|
||||
| 新通话记忆检索 | 服务端已确定的客户组织、长者和通话目标 | SQL同时限制租户、长者、当前授权、活动记忆、当前版本、普通敏感级别、有效期和活动索引；最多5条 | 冻结到`context_snapshot.memory_versions`的版本引用和最小背景摘要 |
|
||||
| 通话中记忆工具 | 供应商持有的单通话短期签名令牌、自然语言查询 | 令牌只绑定服务端既有通话和任务；租户、家庭和长者边界从数据库反查，模型不能传入或扩大；单条240字、总计800字、最多5条；Embedding失败返回空结果并标记降级，不中断通话 | `/internal/v1/tools/memory-search`、`memory_tool_call`和实际使用追踪 |
|
||||
| 可选供应商原生记忆 | 当前人工确认记忆、有效授权、普通敏感级别、受控HTTP适配器 | 数据库始终是主数据；开启后用Outbox同步，修订先清理旧引用，停用、授权撤回、单条删除和主体删除均可靠删除并对账；关闭功能后通话继续走数据库检索 | `provider`索引的`PENDING/ACTIVE/FAILED/DELETED`状态 |
|
||||
| 内容质量反馈 | 故事、事实或记忆的人工确认、修改、拒绝、停用及九类标准问题 | 不复制私密正文；按生成模型和提示词版本聚合候选数、审核动作和问题类型；跨租户统一拒绝 | 维护端“内容质量”与`content_review_signal` |
|
||||
| 跨通话连续上下文 | 最近已完成通话的就绪摘要、当前授权快照、交流偏好 | 只有`story_extraction`和`personalized_memory`在新通话冻结快照中同时有效时，才按同租户同长者读取最近3次摘要；读取结果每条最多240字，供应商上下文按每条140字压缩且总计不超过1000字；撤回任一授权后不再注入 | 冻结到新通话`context_snapshot.sanitized_summary`的短期连续上下文 |
|
||||
| 避免主动提及 | 长者档案`conversation_preferences.avoided_topics` | 最多8项、每项50字；从下一次新通话起进入冻结快照；仅限制智能体主动开启话题，长者主动提及时温和转向，紧急安全响应不受限制 | 新通话的交流边界 |
|
||||
| 授权撤回 | `personalized_memory`授权 | 在同一事务内停用该长者的活动记忆，清空活动/待处理索引向量并标记失效 | `DISABLED`记忆、`STALE`索引 |
|
||||
|
||||
## 3. 业务流程
|
||||
@@ -51,6 +57,10 @@ flowchart TD
|
||||
S --> U[创建下一次通话]
|
||||
U --> V[数据库固定边界检索]
|
||||
V --> W[冻结记忆版本和背景摘要]
|
||||
S --> X{供应商记忆功能已验证并开启?}
|
||||
X -- 是 --> Y[可靠同步外部引用]
|
||||
X -- 否 --> Z[只使用数据库检索]
|
||||
Y --> AA[停用/撤回/删除时外部删除对账]
|
||||
```
|
||||
|
||||
供应商停止事件和最终转写事件可能存在先后竞争。停止事件会发布内容整理待办；如果话轮尚未到达，处理器返回可重试错误，由Outbox退避重试，避免把“暂无话轮”错误当成永久空内容。
|
||||
@@ -59,6 +69,10 @@ flowchart TD
|
||||
|
||||
记忆索引也先在无事务状态读取当前版本和授权，事务外生成Embedding，返回后才锁定记忆与索引行重新校验；修订、停用或撤回授权发生在模型调用期间时，旧结果不会被激活。新通话创建前仅在设备绑定、语音授权和个性化记忆授权均可用时生成查询向量；事务内仍以固定租户、长者、授权、状态和敏感级别条件检索。Embedding临时不可用时，本次通话以`pgvector-degraded-v1`记录降级且不注入记忆，不阻断基本语音通话。普通顺序幂等重放和同键正文冲突会先读取已完成幂等记录，不重复调用Embedding；并发首请求仍由数据库唯一键保证只创建一个通话。
|
||||
|
||||
通话中工具令牌由平台在启动智能体时签发，令牌只包含通话编号、固定供应商任务编号和过期时间。工具接口先重新读取权威通话、任务、租户和长者关系，再执行与通话前召回相同的数据库边界；请求正文没有租户、家庭或长者字段。查询摘要、结果数量、延迟和降级状态进入最小审计，原始查询和记忆正文不进入普通日志。供应商原生记忆是可选增强，不取代该工具或通话前数据库召回；外部服务失败时，本地主数据和本地召回仍可继续工作。
|
||||
|
||||
最近通话摘要是短期连续上下文，不会自动升级为长期记忆，也不会绕过故事人工审核。平台在每次开房时重新检查并冻结当前授权，只读取同一客户组织、同一长者、已完成通话的最新就绪摘要；隐私删除或授权撤回后的摘要不会继续进入后续通话。纯端到端豆包模式把这份冻结上下文写入`S2SConfig.ProviderParams.dialog.system_role`，级联模式继续写入`LLMConfig.SystemMessages`，两种链路使用同一上下文快照。
|
||||
|
||||
## 4. 接口设计
|
||||
|
||||
| 方法与路径 | 权限 | 并发/校验 | 说明 |
|
||||
@@ -70,10 +84,13 @@ flowchart TD
|
||||
| `PATCH /api/v1/stories/{id}` | `story.review` | 必须`If-Match`；正文和原因必填 | 创建人工修订版本 |
|
||||
| `POST /api/v1/stories/{id}/confirm` | `story.review` | 必须`If-Match`；来源与授权门禁 | 确认故事并创建记忆 |
|
||||
| `POST /api/v1/stories/{id}/reject` | `story.review` | 必须`If-Match`；原因必填 | 拒绝候选 |
|
||||
| `PATCH /api/v1/stories/{story_id}/facts/{fact_id}` | `story.review` | 必须`If-Match`；单条事实状态机、原因及可选标准问题类型 | 独立编辑、确认、拒绝或停用事实 |
|
||||
| `GET /api/v1/subjects/{id}/memories` | `subject.read` | 过滤已删除记录；可用`q`在固定长者边界内搜索 | 已确认记忆列表 |
|
||||
| `GET /api/v1/memories/{id}` | `subject.read` | 返回强ETag | 当前记忆版本和索引状态 |
|
||||
| `PATCH /api/v1/memories/{id}` | `story.review` | 必须`If-Match`；正文/状态至少一项且原因必填；删除中及历史终态拒绝修改 | 修订、停用或重新启用记忆 |
|
||||
| `DELETE /api/v1/memories/{id}` | `story.review` | 必须`If-Match`；删除原因必填；返回202 | 可靠受理永久删除任务 |
|
||||
| `POST /internal/v1/tools/memory-search` | 单通话短期Bearer令牌 | 固定通话/任务/租户/长者边界；限流、长度和结果上限 | 通话中受控记忆检索，失败安全降级 |
|
||||
| `GET /api/v1/ops/content-quality` | `ops_viewer`及以上 | 必须选择有权客户组织；不返回正文 | 按模型和提示词版本聚合审核反馈 |
|
||||
|
||||
统一冲突码包括`STORY_SOURCE_REQUIRED`、`PERSONALIZED_MEMORY_CONSENT_REQUIRED`和`STORY_REVIEW_STATE_CONFLICT`。跨租户或非家庭成员访问统一返回403，不借404差异泄露资源是否存在。
|
||||
|
||||
@@ -87,11 +104,15 @@ flowchart TD
|
||||
| `story_revision` | 不可覆盖的故事版本 | 故事内修订号唯一；区分模型/人工作者；保存基础版本和原因 |
|
||||
| `story_source` | 故事版本到原话的追溯关系 | 版本、话轮和文本偏移组合唯一 |
|
||||
| `fact_candidate` | 可审核事实候选 | 区分长者陈述与模型推断；保存源话轮、敏感级别和状态 |
|
||||
| `fact_candidate_revision` | 事实不可覆盖版本 | 每条事实版本号唯一；保留内容、状态、原因和标准问题类型 |
|
||||
| `curated_memory` | 长期记忆聚合根 | 当前版本、状态、有效期、敏感级别和强版本号 |
|
||||
| `memory_version` | 不可覆盖的记忆版本 | 记忆内版本号唯一；确认人、确认时间、原因、来源和内容摘要 |
|
||||
| `memory_index_entry` | 记忆检索索引状态 | 记忆版本和索引类型唯一；向量维度1536；状态可失效和删除 |
|
||||
| `conversation_memory_use` | 实际记忆使用追踪 | 按通话、版本和使用方式唯一，不复制正文 |
|
||||
| `memory_tool_call` | 通话中工具可观测性 | 只保存查询摘要、数量、状态、延迟和引擎模式 |
|
||||
| `content_review_signal` | 内容质量反馈 | 按租户和长者隔离；保存资源、动作、问题类型、模型和提示词版本，不复制正文 |
|
||||
|
||||
V8迁移已经在独立空数据库执行`up → down -all → up`，最终为`8|dirty=false`，九张内容与记忆表均存在。测试数据库随后已删除。
|
||||
内容与记忆数据库结构目前覆盖到V41。迁移在独立临时数据库执行`up → down 1 → up`并核对最终版本与`dirty=false`，测试数据库随后删除，不对开发或生产数据库做回滚验证。
|
||||
|
||||
## 6. 安全、隐私与可观测性
|
||||
|
||||
@@ -102,17 +123,21 @@ V8迁移已经在独立空数据库执行`up → down -all → up`，最终为`8
|
||||
- 只有冻结快照允许保存字幕时才落`conversation_turn`；故事整理和个性化记忆分别使用独立授权门禁。
|
||||
- 故事、事实和记忆均保留租户、长者和来源边界；所有网页接口从OIDC主体重新解析家庭成员关系，不信任客户端传入租户。
|
||||
- 人工修订、确认、拒绝和授权撤回记录审计事件；写操作使用强ETag防止并发覆盖。
|
||||
- 通话创建已接入内部数据库召回并冻结实际使用的记忆版本；调用方不能传入自由租户或长者过滤条件。当前尚未开放通话中工具检索接口。
|
||||
- 通话创建和通话中工具均使用服务端固定的租户、长者、授权和版本边界；供应商或模型不能传入自由租户、家庭或长者过滤条件。
|
||||
- 供应商原生记忆默认关闭，只有完成真实账号的数据条款、隔离、同步、删除和容量验证后才配置`PROVIDER_MEMORY_MODE=managed_http`；数据库仍是唯一主数据。
|
||||
|
||||
## 7. 验证结果
|
||||
|
||||
- Go全仓测试通过，OpenAPI 0.10.0可生成TypeScript类型，React生产构建通过。
|
||||
- 新增迁移已在隔离数据库完成升库与数据库组合测试；OpenAPI 0.70.0可生成TypeScript类型，React类型检查和生产构建通过。
|
||||
- 本地内容策略单元测试覆盖验证码、银行卡号、高敏感医疗词和普通生活经历。
|
||||
- 方舟兼容适配器轻量验证覆盖Bearer鉴权、请求路径、结构化响应校验、HTTP 429重试分类；真实PostgreSQL组合验证模型调用期间不持有通话行锁，模型返回后正常生成候选，敏感话轮不会调用外部模型。
|
||||
- 方舟Embedding适配器轻量验证覆盖Bearer鉴权、`/embeddings`请求、模型/维度参数、1536维结果和错误维度拒绝；真实PostgreSQL组合验证向量生成期间不持有索引行锁，返回后可激活索引。
|
||||
- 端到端通话冒烟验证最终话轮去重、两类整理结果、故事来源、人工确认前零记忆、确认后记忆生成及索引生效；下一次通话冻结1个记忆版本。
|
||||
- 记忆修订生成第2版并重新索引；停用后索引立即失效且新通话召回0条；重新启用再次进入索引流程。
|
||||
- 永久删除验证覆盖`PENDING_DELETE → DELETED`、删除中重新激活稳定拒绝、向量清理、数据库正文替换为删除标记和删除后新通话召回0条。
|
||||
- 通话中记忆工具验证覆盖短期令牌签名、固定通话/任务边界、跨长者不可选、结果长度上限、实际使用追踪和Embedding异常降级。
|
||||
- 供应商记忆适配器验证覆盖Bearer鉴权、幂等同步、旧版本删除、单条删除、主体隐私删除前对账和失败状态保留；当前只使用模拟HTTP/内存适配器，真实供应商账号仍待外部验收。
|
||||
- 内容质量反馈验证覆盖九类问题、事实不可覆盖版本、按模型/提示词聚合和跨租户拒绝；维护端只展示汇总，不显示故事、事实或记忆正文。
|
||||
- React故事与记忆页以页面内管理区提供正文修订、启停和永久删除；未修改正文、空原因、未确认删除均不能提交，删除中和历史终态只读，不再依赖浏览器弹窗。
|
||||
- OIDC、SaaS、设备、成员和通话五组冒烟全部通过；测试结束后内容表残留为0，Outbox无待处理、处理中或死信事件。
|
||||
- 当前方舟内容与Embedding协议均使用本地模拟端点验证，尚未使用业务方真实API Key核验模型效果、召回质量、额度和数据条款；真实语音供应商与真实设备也仍待验收。
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# 设备 API 兼容性约定
|
||||
|
||||
> 当前设备接口主版本：`/device/v2`
|
||||
> 当前接口契约：OpenAPI 0.68.0
|
||||
> 当前接口契约：OpenAPI 0.70.0
|
||||
> 适用对象：云端 API、终端固件、模拟终端和设备联调工具
|
||||
|
||||
## 1. 兼容目标
|
||||
|
||||
@@ -47,13 +47,21 @@ X-Device-Content-SHA256原值
|
||||
|
||||
云端先核验时间窗、正文摘要和签名，再使用Redis `SET NX`登记随机数。相同设备和随机数只能使用一次；Redis或数据库不可用时设备鉴权采用fail-closed并返回可重试的503，不绕过身份验证。
|
||||
|
||||
本地环境使用静态HMAC密钥模拟联调。量产采用每设备独立HMAC时，密钥应写入硬件安全区并通过Secrets/KMS CSI受控挂载到云端；如硬件与PKI条件允许，仍可进一步切换每设备mTLS证书。不能把本地合成密钥烧录到真实设备。
|
||||
本地环境允许把既有静态HMAC密钥一次性导入数据库以兼容测试包。量产采用每设备独立HMAC时，设备侧密钥应写入硬件安全区，云端只保存主密钥加密后的密文；如硬件与PKI条件允许，仍可进一步切换每设备mTLS证书。不能把本地合成密钥烧录到真实设备。
|
||||
|
||||
### 2.1 逐设备生产密钥解析
|
||||
|
||||
dev、stage和prod把设备密钥通过Docker/Kubernetes Secrets或云KMS CSI驱动挂载到API容器的受控只读目录，并设置`DEVICE_SECRET_DIRECTORY`。设备表不保存密钥正文，只保存`device-secret://<name>`引用；解析器只接受单层安全文件名，解析符号链接后仍必须位于配置根目录内，文件最大4KiB且有效密钥至少32字节。
|
||||
设备表保存公开身份和`credential_ref`，`device_credential`表按设备保存AES-256-GCM加密后的HMAC密钥、独立随机nonce、密钥版本与状态，不保存明文。API使用数据库外独立托管的`DEVICE_CREDENTIAL_MASTER_KEY`解密；生产环境必须通过Docker/Kubernetes Secret或云KMS CSI挂载该主密钥。现有演示环境的`DEVICE_LOCAL_SECRET`只用于首次启动时导入缺失记录，导入不会覆盖已轮换的数据库凭据。
|
||||
|
||||
每次验签都按设备记录读取对应文件，因此不同设备使用不同密钥，原子替换挂载文件即可完成轮换，不需要重新构建镜像或重启API。文件缺失、过短、引用格式错误或越界时统一按无效凭据拒绝，响应与日志不暴露文件路径及密钥内容。生产启动门禁拒绝退回单一`DEVICE_LOCAL_SECRET`。
|
||||
每次验签都按`credential_ref`读取当前活动数据库凭据并执行AEAD认证解密；记录缺失、已吊销、主密钥错误或密文被篡改时统一按无效凭据拒绝，响应与日志不暴露密钥内容。`device-credential`维护命令提供生命周期操作：
|
||||
|
||||
```text
|
||||
device-credential provision <设备公开UUID>
|
||||
device-credential rotate <设备公开UUID>
|
||||
device-credential revoke <设备公开UUID>
|
||||
```
|
||||
|
||||
`provision`和`rotate`生成32字节随机密钥，明文只在标准输出显示一次；数据库仅保存密文。轮换提交后旧密钥立即失效，必须通过受控渠道同步更新设备安全区。吊销后设备的所有签名请求立即失败。生产启动门禁拒绝默认主加密密钥。
|
||||
|
||||
## 3. 启动参数
|
||||
|
||||
@@ -228,6 +236,8 @@ docker compose --profile simulation run --rm sim-device
|
||||
| `media` | 为`SIM_CONVERSATION_ID`指定且具有当前录音授权的通话生成或读取WAV PCM16文件，直传对象存储，轮询至云端校验完成，并在`local_delete_after`到达后回报本地密文已删除；可设置`SIM_MEDIA_FILE`、`SIM_MEDIA_DURATION_MS`、`SIM_MEDIA_TRACK`、`SIM_MEDIA_VERIFY_TIMEOUT`和`SIM_MEDIA_DELETE_WAIT_MAX` |
|
||||
| `all` | 依次执行启动、完整通话、字幕、遥测和命令；录音因需要单独授权和可访问的对象存储公开地址而不自动包含 |
|
||||
|
||||
Windows真实RTC模拟设备会读取创建通话响应中的`privacy.record_near`、`record_far`和`upload_allowed`。只有相应授权有效时，它才通过RTC SDK音频帧回调分别采集长者近端和智能体远端PCM；结束通话时下混为单声道WAV PCM16，按不超过9分钟且不超过64MiB自动分片。设备提交停止请求并等待通话进入`COMPLETED`或`FAILED`后，自动创建录音上传批次、直传短期对象地址并轮询至云端校验完成。未授权轨道不会采集，上传失败最多自动重试三次并在模拟器事件流中显示稳定错误。
|
||||
|
||||
例如执行联合模拟闭环：
|
||||
|
||||
```powershell
|
||||
|
||||
@@ -83,14 +83,16 @@ docker push "$Registry/migrate:$Version"
|
||||
| Secret名称 | 必需key | 使用者 |
|
||||
|---|---|---|
|
||||
| `voice-platform-common-secrets` | `database-url`、`redis-password` | API、Callbacks、Worker、迁移Job |
|
||||
| `voice-platform-api-secrets` | `rtc-app-key`、`cursor-signing-secret`、`share-token-signing-secret` | API |
|
||||
| `voice-platform-api-secrets` | `device-credential-master-key`、`rtc-app-key`、`cursor-signing-secret`、`share-token-signing-secret` | API |
|
||||
| `voice-platform-callbacks-secrets` | `provider-callback-secret`、`platform-health-callback-secret` | Callbacks |
|
||||
| `voice-platform-worker-secrets` | `volc-access-key-id`、`volc-secret-access-key`、`voice-chat-template.json` | Worker |
|
||||
| `voice-platform-model-storage-secrets` | `object-store-access-key`、`object-store-secret-key`、`ark-api-key`、`content-safety-api-key` | API、Worker；内容安全Key仅由Worker引用 |
|
||||
| `voice-platform-model-storage-secrets` | `object-store-access-key`、`object-store-secret-key`、`ark-api-key`、`content-safety-api-key`、`memory-tool-signing-secret` | API、Worker；内容安全Key仅由Worker引用，记忆工具签名密钥由两者共享 |
|
||||
| `voice-platform-device-secrets` | 每台设备一个key，例如`terminal-001` | API |
|
||||
| `voice-platform-tls` | `tls.crt`、`tls.key` | Ingress |
|
||||
|
||||
游标签名密钥和分享签名密钥必须各不少于32字节且互不相同。供应商回调密钥不少于32字节。数据库连接串和Redis密码由应用通过`*_FILE`读取；应用错误日志不会输出Secret内容。
|
||||
游标签名密钥、分享签名密钥和通话内记忆工具签名密钥必须各不少于32字节且互不相同。供应商回调密钥不少于32字节。数据库连接串和Redis密码由应用通过`*_FILE`读取；应用错误日志不会输出Secret内容。
|
||||
|
||||
如启用可选的供应商原生记忆，把`PROVIDER_MEMORY_MODE`设为`managed_http`，在环境专属ConfigMap中配置正式HTTPS `PROVIDER_MEMORY_BASE_URL`，并通过仅挂载给API和Worker的Secret文件注入`PROVIDER_MEMORY_API_KEY_FILE`。默认`disabled`不会向供应商复制任何已确认记忆。通话内工具还需要在环境专属ConfigMap中设置供应商可访问的HTTPS `MEMORY_TOOL_BASE_URL`；不要填写集群内地址，除非供应商执行端确实运行在同一受控网络。
|
||||
|
||||
每台量产设备在数据库中的`credential_ref`应写为`device-secret://terminal-001`，对应`voice-platform-device-secrets`中的`terminal-001`文件。名称只允许安全的单层文件名。Kubernetes Secret或CSI卷原子更新后，API会在下一次设备验签时读取新值，无需重启。
|
||||
|
||||
@@ -118,10 +120,10 @@ kubectl -n voice-platform get secret
|
||||
| SecretProviderClass | 必须挂载的文件名 |
|
||||
|---|---|
|
||||
| `voice-platform-common` | `database-url`、`redis-password` |
|
||||
| `voice-platform-api` | `rtc-app-key`、`cursor-signing-secret`、`share-token-signing-secret` |
|
||||
| `voice-platform-api` | `device-credential-master-key`、`rtc-app-key`、`cursor-signing-secret`、`share-token-signing-secret` |
|
||||
| `voice-platform-callbacks` | `provider-callback-secret`、`platform-health-callback-secret` |
|
||||
| `voice-platform-worker` | `volc-access-key-id`、`volc-secret-access-key`、`voice-chat-template.json` |
|
||||
| `voice-platform-model-storage` | `object-store-access-key`、`object-store-secret-key`、`ark-api-key`、`content-safety-api-key` |
|
||||
| `voice-platform-model-storage` | `object-store-access-key`、`object-store-secret-key`、`ark-api-key`、`content-safety-api-key`、`memory-tool-signing-secret` |
|
||||
| `voice-platform-devices` | 每台设备一个文件，例如`terminal-001` |
|
||||
|
||||
提供器对象应通过`objectAlias`或等价字段映射为上述准确文件名。Deployment挂载整个目录，不使用`subPath`，以便CSI原子轮换能够更新容器看到的符号链接。ServiceAccount已按API、Callbacks、Worker和Web拆分；现场应只给前三个需要密钥的ServiceAccount附加相应工作负载身份，并按最小权限授予密钥读取能力。
|
||||
|
||||
@@ -10,6 +10,8 @@ V19—V20及V34已经形成内部运营配置的本地纵向闭环：运营身
|
||||
|
||||
## 2. 运营角色与租户边界
|
||||
|
||||
维护人员使用独立OIDC账号登录`/maintenance`，不得复用家庭用户身份。账号表以`account_kind`建立强制边界：`customer`只能使用`platform_role=customer`并通过家庭成员关系取得业务权限；`operator`不得具有家庭成员关系，只能使用下表的维护角色。研发环境提供`operator`和`operator-approver`两个合成账号，生产环境必须由独立员工身份源、MFA和入离职流程维护。
|
||||
|
||||
| 角色 | 权限 |
|
||||
|---|---|
|
||||
| `customer` | 无内部运营接口权限 |
|
||||
@@ -18,7 +20,16 @@ V19—V20及V34已经形成内部运营配置的本地纵向闭环：运营身
|
||||
| `ops_approver` | 具备操作员能力，并可审批发布、回滚、暂停和恢复投放 |
|
||||
| `ops_admin` | 平台管理员能力；仍受创建人与审批人分离约束 |
|
||||
|
||||
运营角色保存在`user_account.platform_role`，不会信任OIDC令牌中的自报角色。每次运营请求用OIDC签名身份与`X-Tenant-Id`共同解析当前客户组织的活动账号；家庭角色不会隐式获得运营权限，跨客户组织的试验对象和设备编号均由服务端拒绝。生产环境仍需由身份系统强制MFA和受控授权流程。
|
||||
运营角色保存在`user_account.platform_role`，账号类别保存在`user_account.account_kind`，不会信任OIDC令牌中的自报角色。每次运营请求用OIDC签名身份与`X-Tenant-Id`共同解析当前客户组织的活动operator账号；家庭角色不会隐式获得运营权限，跨客户组织的试验对象和设备编号均由服务端拒绝。生产环境仍需由身份系统强制MFA和受控授权流程。
|
||||
|
||||
### 2.1 运维设备管理
|
||||
|
||||
`/maintenance?area=devices`提供租户内设备资产清单，展示设备公开编号、型号、状态、密钥版本、最后上报时间及当前家庭/长者归属。`ops_viewer`和`ops_approver`只读；`ops_operator`与`ops_admin`可登记、修改、调整归属、轮换密钥和删除设备。
|
||||
|
||||
- 登记时服务端生成UUID设备编号、不可逆随机序列摘要和32字节随机HMAC密钥；设备、影子、可选绑定、加密凭据和审计记录在同一事务提交。
|
||||
- 密钥明文只在创建或轮换响应出现一次，网页立即下载为本地凭据文件；服务端数据库只保留AES-256-GCM密文，不提供历史密钥下载接口。
|
||||
- 编辑使用强ETag防止覆盖并发修改；家庭和长者必须来自同一客户组织且保持有效。
|
||||
- 删除采用软删除，设备进入`revoked`、当前密钥立即吊销、当前绑定关闭；历史通话、绑定历史和审计记录不物理删除。
|
||||
|
||||
## 3. 配置与投放状态
|
||||
|
||||
|
||||
@@ -33,8 +33,8 @@
|
||||
| P0160 | 分片短期上传地址 | `internal/media/service.go`、对象存储适配器、V14 | 创建上传批次和返回待上传分片地址 | 清单、授权、幂等、对象引用和直传烟测 | 本地完成 |
|
||||
| P0162 | 上传完成后校验 | `internal/media/processor.go`、V14/V15 | 上传批次完成和状态查询 | 校验值、对象存在、状态收敛和模拟本地删除闭环 | 本地完成 |
|
||||
| P0164 | 播放和下载分别授权 | `internal/media/access.go`、`recording_access_grant` | 录音访问授权、撤销及分片访问接口 | 所有者不隐式获得权限、播放/下载能力分离用例 | 本地完成 |
|
||||
| P0167 | 故事和事实候选核对 | `internal/saas/content_service.go`、`internal/extraction`、V8 | 故事列表、详情、修订、确认和拒绝接口及审核页面 | 来源必需、ETag、角色、版本留痕和真实数据库组合用例 | 本地完成 |
|
||||
| P0169 | 已确认记忆管理 | `internal/saas/content_service.go`、`internal/memorylifecycle`、V8 | 记忆查询、页面内修订/启停/删除和状态详情 | 授权门禁、索引失效、删除中不可恢复、可靠删除、召回排除及真实登录页面交互 | 本地完成 |
|
||||
| P0167 | 故事和事实候选核对 | `internal/saas/content_service.go`、`internal/extraction`、V8/V41 | 故事列表、详情、修订、确认和拒绝；单条事实编辑/确认/拒绝/停用及审核页面 | 来源必需、ETag、角色、故事与事实不可覆盖版本、九类问题标签和真实数据库组合用例 | 本地完成 |
|
||||
| P0169 | 已确认记忆管理 | `internal/saas/content_service.go`、`internal/memorylifecycle`、`internal/providermemory`、V8/V41 | 记忆查询、页面内修订/启停/删除、本地与外部同步状态详情 | 授权门禁、索引失效、删除中不可恢复、本地/外部可靠删除、召回排除及数据库组合测试 | 本地完成；真实供应商记忆账号待验 |
|
||||
| P0174 | 数据访问、导出、更正、撤回和删除请求 | `internal/privacy`、`internal/dataexport`、V9/V10 | 隐私请求、导出任务和进度接口 | 多子系统删除、导出加密引用、失败/重试和清理烟测 | 本地完成，法务口径待业务确认 |
|
||||
| P0176 | 状态实时推送且不推敏感正文 | `internal/webevents`、V16、React 2秒合并刷新 | `GET /api/v1/events`及React SSE客户端 | 重连游标、资源隔离、状态事件字段、连续8事件请求上界和运行烟测 | 本地完成 |
|
||||
| P0183 | 通话用量、费用和额度 | `internal/billing`、V21 | 通话用量、价格、额度和费用汇总接口 | 回调幂等计量、额度门禁、价格版本冻结和烟测 | 本地完成，真实账单对账待验 |
|
||||
@@ -53,7 +53,9 @@
|
||||
| P0212 | 每通话选择一种语音模式 | `internal/opsconfig`、`internal/orchestration`、组合优先及模式冻结 | Agent组合/旧路由投放和通话创建 | 三模板选择、缺失配置拒绝、组合路由解析和数据库到供应商传播用例 | 本地完成，三种真实模式待验 |
|
||||
| P0214 | 管理人设、音色、模型和路由配置 | `internal/opsconfig`、`internal/orchestration`、V19/V20/V33 | 五类配置、评测、资源引用、灰度、回滚及供应商启动交接 | Agent组合UUID、同租户、组件类型、已发布状态、运行时冻结和组件参数传递 | 本地完成，真实资源兼容性待验 |
|
||||
| P0217 | 最小必要背景资料 | `internal/orchestration/repository.go`、版本化背景快照 | 通话创建内部资料准备 | 白名单字段、非白名单排除、无效类型失败和真实数据库用例 | 本地完成 |
|
||||
| P0219 | 自有数据库检索记忆 | `internal/retrieval`、pgvector、V8 | 创建通话前固定边界召回 | 当前授权、版本、敏感级别、跨长者隔离和非法limit用例 | 本地完成，真实Embedding质量待验 |
|
||||
| P0219 | 自有数据库检索记忆和通话中工具 | `internal/retrieval`、`internal/memorytool`、pgvector、V8/V41 | 创建通话前固定边界召回及`POST /internal/v1/tools/memory-search` | 当前授权、版本、敏感级别、跨长者隔离、单通话任务令牌、长度/数量上限、使用追踪和失败降级 | 本地完成，真实Embedding与供应商工具调用待验 |
|
||||
| P0221 | 验证并可选使用供应商原生记忆 | `internal/providermemory`、`memory_index_entry(index_type=provider)`、可靠Outbox和V41状态展示 | `PROVIDER_MEMORY_MODE=managed_http`可选适配器；家庭端显示外部同步状态 | 幂等同步、修订清旧、停用/撤回/单条删除/主体删除对账、失败状态和本地降级组合测试 | 本地适配完成，真实供应商账号的数据条款、隔离、容量和删除待验后方可开启 |
|
||||
| P0223 | 通话中按需调用自有记忆检索工具 | `internal/memorytool`、单通话HMAC令牌、实际使用追踪、V41 | `POST /internal/v1/tools/memory-search`，供应商启动模板注入固定工具地址和短期令牌 | 模型不可传租户/家庭/长者、固定通话任务、限流、240/800字边界、最多5条、超时降级且不中断通话 | 本地完成，真实豆包工具调用待验 |
|
||||
| P0226 | 可靠接收供应商通知 | `internal/voiceprovider/signature.go`、Callback处理和Outbox | `POST /callbacks/v1/vertc/voice-chat` | 先验签、重放、同键冲突、乱序和快速确认用例 | 本地完成 |
|
||||
| P0228 | 最终字幕和对话轮次 | `internal/orchestration/transcript_batch.go`、V8 | 供应商最终字幕和设备`turns:batch` | 轮次业务键、整批回滚、授权留存和重复补交用例 | 本地完成 |
|
||||
| P0230 | 摘要和开放话题 | `internal/extraction/generator.go`、`processor.go` | 通话完成后的可靠内容整理任务 | 事务外模型调用、话轮指纹、重复版本和敏感拦截用例 | 本地完成，真实模型质量待验 |
|
||||
@@ -62,10 +64,11 @@
|
||||
| P0238 | 长者友好表达 | 版本化PERSONA配置、背景节奏字段和固定评测集 | 运营配置、供应商启动模板、`cmd/agent-eval` | 本地规则及模拟评测通过 | 本地完成，真实听筒体验待验 |
|
||||
| P0240 | 身份、人设和边界一致 | `internal/opsconfig`、服务端授权/上下文边界、提示注入拦截 | 版本化PERSONA/AGENT_BUNDLE、冻结快照和供应商系统消息 | 配置校验、租户/长者边界、发布人设传递和敏感提示用例 | 本地完成，真实模型红队待验 |
|
||||
| P0245 | 摘要忠于原始对话 | `internal/extraction`、来源话轮和内容指纹 | 摘要/开放话题接口 | 仅长者最终话轮、来源保存、敏感拦截和虚构门槛 | 本地完成，真实内容评分待验 |
|
||||
| P0247 | 候选可核对可修改 | V8故事、故事修订和事实候选 | 故事详情、修订、确认和拒绝接口 | 来源跳转、版本、ETag和终态重复操作拒绝 | 本地完成 |
|
||||
| P0249 | 已确认记忆准确使用 | `internal/retrieval`、Embedding适配器、记忆索引生命周期 | 记忆检索和通话背景快照 | 当前版本/授权/敏感级别过滤、跨长者隔离和停用删除排除 | 本地完成，真实召回质量待验 |
|
||||
| P0252 | 固定评测集和质量指标 | `internal/agenteval`、`internal/opsconfig`、`cmd/agent-eval-collect`、`cmd/agent-eval`、`cmd/agent-eval-register`、V33及版本化JSON夹具 | 权限内通话/话轮采集、脱敏观测、评测报告及运营配置`/evaluations`标准证据幂等登记 | 十类覆盖、唯一通话、必需评分、机器指标推导、报告防篡改、服务端标准证据校验、通用证据不得放行Agent组合、201/200重放和模拟10/10报告 | 本地完成，真实账号评测待验 |
|
||||
| P0247 | 候选可核对可修改 | V8/V41故事、故事修订、事实候选与事实修订 | 故事详情、修订、确认和拒绝；单条事实细粒度审核接口 | 来源跳转、不可覆盖版本、ETag、九类问题标签和终态重复操作拒绝 | 本地完成 |
|
||||
| P0249 | 已确认记忆准确使用 | `internal/retrieval`、`internal/memorytool`、`internal/providermemory`、Embedding适配器和记忆索引生命周期 | 通话前背景快照、通话中工具、可选供应商同步 | 当前版本/授权/敏感级别过滤、跨长者隔离、停用/撤回/删除清理、失败回退本地数据库 | 本地完成，真实召回质量与供应商数据条款待验 |
|
||||
| P0252 | 固定评测集和质量反馈指标 | `internal/agenteval`、`internal/opsconfig`、`content_review_signal`、维护端内容质量页、`cmd/agent-eval-*`、V33/V41及版本化JSON夹具 | 权限内通话/话轮采集、脱敏观测、评测报告、标准证据登记及`GET /api/v1/ops/content-quality` | 十类评测覆盖、九类人工问题反馈、按模型/提示词聚合、跨租户拒绝、报告防篡改及模拟10/10报告 | 本地完成，真实账号评测待验 |
|
||||
| P0254 | Agent配置发布、灰度和回滚 | `internal/opsconfig`、`internal/orchestration`、运营配置工作台和投放台账 | 双人审批、租户/长者/设备组灰度、自定义版本族、不可变回滚、组合优先选择 | 组件引用复核、评测/资源门禁、配置类型选择、范围与激活时间优先级、并发上限、新通话冻结及供应商启动交接 | 本地完成，真实灰度待验 |
|
||||
| P0256 | 用户核对到模型优化的反馈闭环 | `fact_candidate_revision`、`content_review_signal`、`internal/opsconfig/content_quality.go`和独立维护端 | 故事、单条事实和记忆审核收集九类问题；`GET /api/v1/ops/content-quality`按模型/提示词聚合 | 不复制私密正文、角色与租户隔离、确认/修改/拒绝/停用计数、九类问题统计和前端生产构建 | 本地完成；训练数据导出、供应商模型再训练和真实A/B效果属于后续受控流程 |
|
||||
|
||||
## 4. 关键异常和三方交接场景
|
||||
|
||||
@@ -87,7 +90,7 @@
|
||||
|
||||
| 来源 | 要求 | 当前处理 |
|
||||
|---|---|---|
|
||||
| P0414 | 首期需求、验收用例、缺陷和风险形成对应关系 | 本文完成首期功能到代码、接口、数据和验证的基线；V30问题清单由V36补充严重级别、根因、修复版本、回归证据及未关闭HIGH/CRITICAL发布阻断汇总；OpenAPI 0.50.0—0.68.0可按候选构建导出八项技术门禁及脱敏阻断告警定位快照，绑定候选版本/构建摘要后按业务时区归档日/周运行证据。第七项门禁要求主机健康快照在15分钟内且全部检查正常；第八项在`PRODUCTION`强制要求生产部署模式、三个业务进程实际`APP_ENV=prod`、身份服务非开发模式、已验证异地备份、外部告警配置及Mock服务退出，在`LAB/STAGING`明确标记不适用。`cmd/release-evidence`现使用v3清单，把同一候选的快照、接口、迁移、部署、SBOM和十类外部状态形成逐文件摘要；远程CI供应链、真机固件、真实终端RTC、真实Agent模型、真实告警网关、云KMS/HSM轮换、生产等价恢复、生产基础设施和Safari兼容分别要求结构化报告，并把14项CI检查/镜像/SBOM、设备固件场景、语音方式与恢复场景、生成/Embedding/内容安全质量、路由演练、轮换模式、恢复类型、基础设施控制项及Safari矩阵覆盖与缺口写入清单。V38及九个离线校验器把云端状态与远程构建、外部设备、RTC/语音供应商、模型提供器、网关、密钥提供器、目标环境和浏览器证据绑定；`content_safety_tls`把策略服务网络/证书状态纳入指标、告警、运营页面和候选快照。缺检查、缺制品、缺场景、缺演练、缺质量类别、缺控制项、`PENDING`和本地模拟证据均不会被误报为外部验收完成，整体发布仍必须基于真实联调、灰度和审批记录确认。 |
|
||||
| P0414 | 首期需求、验收用例、缺陷和风险形成对应关系 | 本文完成首期功能到代码、接口、数据和验证的基线；V30问题清单由V36补充严重级别、根因、修复版本、回归证据及未关闭HIGH/CRITICAL发布阻断汇总；OpenAPI 0.50.0—0.70.0可按候选构建导出八项技术门禁及脱敏阻断告警定位快照，绑定候选版本/构建摘要后按业务时区归档日/周运行证据。第七项门禁要求主机健康快照在15分钟内且全部检查正常；第八项在`PRODUCTION`强制要求生产部署模式、三个业务进程实际`APP_ENV=prod`、身份服务非开发模式、已验证异地备份、外部告警配置及Mock服务退出，在`LAB/STAGING`明确标记不适用。`cmd/release-evidence`现使用v3清单，把同一候选的快照、接口、迁移、部署、SBOM和十类外部状态形成逐文件摘要；远程CI供应链、真机固件、真实终端RTC、真实Agent模型、真实告警网关、云KMS/HSM轮换、生产等价恢复、生产基础设施和Safari兼容分别要求结构化报告，并把14项CI检查/镜像/SBOM、设备固件场景、语音方式与恢复场景、生成/Embedding/内容安全质量、路由演练、轮换模式、恢复类型、基础设施控制项及Safari矩阵覆盖与缺口写入清单。V38及九个离线校验器把云端状态与远程构建、外部设备、RTC/语音供应商、模型提供器、网关、密钥提供器、目标环境和浏览器证据绑定；`content_safety_tls`把策略服务网络/证书状态纳入指标、告警、运营页面和候选快照。缺检查、缺制品、缺场景、缺演练、缺质量类别、缺控制项、`PENDING`和本地模拟证据均不会被误报为外部验收完成，整体发布仍必须基于真实联调、灰度和审批记录确认。 |
|
||||
| P0445 | 需求编号关联代码、测试、接口或配置变更 | 本文逐项建立关联；后续功能变更必须同步更新本矩阵和`development-status.md`。 |
|
||||
|
||||
当前仍不能宣告完成的部分集中为：真实火山RTC与豆包三模式联调、真实方舟内容与Embedding质量、真实听筒的长者交流体验、量产设备/KMS注入、跨故障域生产对象存储与备份、真机异常恢复、真实账单和生产级内容安全。阿里云单机环境已经形成加密全组件归档、数据库隔离恢复和Redis RDB校验，但同机归档不能替代托管PITR、异地副本和生产恢复演练。这些剩余项属于阶段8外部验收，不应通过增加本地低价值测试伪造完成证据。
|
||||
|
||||
@@ -73,12 +73,16 @@
|
||||
| `GET /api/v1/ops/quota-policies` | 运营查看及以上 | 查询活动与历史额度 | 客户组织隔离 |
|
||||
| `POST /api/v1/ops/quota-policies` | 运营审批及以上 | 创建并激活策略版本 | `Idempotency-Key`、事务审计 |
|
||||
| `GET /api/v1/ops/cost-summary` | 运营查看及以上 | 按时间范围聚合费用 | 最大366天、不返回任务编号 |
|
||||
| `GET /api/v1/ops/usage-overview` | 运营查看及以上 | 查看整体通话时长与可用费用 | 客户组织授权；可按家庭、设备筛选 |
|
||||
| `GET /api/v1/households/{id}/usage-overview` | 活动家庭成员 | 查看本家庭通话时长与可用费用 | 服务端反查家庭关系；只能筛选本家庭设备 |
|
||||
| `GET /api/v1/conversations/{id}/usage` | 活动家庭成员 | 查看单次通话用量和费用 | 服务端反查家庭关系 |
|
||||
| `POST /callbacks/v1/vertc/voice-chat` | 签名供应商 | 接收`voice_chat.usage` | HMAC、时间窗、事件幂等 |
|
||||
| `POST /device/v2/conversations` | 签名设备 | 创建通话并冻结预算 | 创建前统一额度检查 |
|
||||
|
||||
所有Web响应使用统一`request_id + data`信封。运营和家庭汇总只返回标准化分类、指标、数量、单位、费用和币种，不返回供应商任务、供应商事件、内部价格版本或费率编号。
|
||||
|
||||
通话用量概览以平台已经持久化的`conversation.started_at / ended_at`计算时长，因此不依赖供应商账单接口。费用只有在最终用量已经核价并生成`conversation_cost_line`时才标记为可用；前端默认展示通话时长，在当前筛选范围存在费用行时开放费用视图。家庭端的家庭编号取自URL，但服务端仍以登录身份的活动家庭成员关系重新授权；传入其他家庭或其他家庭设备都会被拒绝。运营端先验证所选客户组织的运营账号，再将家庭、设备筛选限制在该客户组织内。
|
||||
|
||||
## 4. 数据库设计
|
||||
|
||||
| 表 | 作用 | 关键约束 |
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,26 @@
|
||||
# 设备相关外部接口表
|
||||
|
||||
范围仅包含设备终端以 HMAC 身份直接调用的 `/device/v2/*` 外部接口；不包含家庭/运营 Web API、内部工具和供应商回调。基于 `openapi/openapi.yaml`（OpenAPI 3.1，版本 0.70.0）生成，并与 `internal/httpapi/router.go` 的实际路由逐项核对。当前共 **16** 个已注册且纳入 OpenAPI 契约的接口。
|
||||
|
||||
说明：`*` 表示必填字段；`body` 表示请求体；响应栏只展开成功响应。全部接口都要求设备 HMAC 请求头：`X-Device-Id`、`X-Device-Timestamp`、`X-Device-Nonce`、`X-Device-Content-SHA256`、`X-Device-Signature`，表内不重复。通用错误响应为 `ErrorResponse`。
|
||||
|
||||
## 设备端 v2
|
||||
|
||||
| 接口名字 | 方法与路径 | 接口描述 | 输入参数 | 输出参数 |
|
||||
|---|---|---|---|---|
|
||||
| `getDeviceBootstrap` | `GET /device/v2/bootstrap` | 返回可信设备状态、服务器时间、绑定和最低安全版本 | query `firmware_version*`: string<br>query `board_rev`: string<br>query `rtc_sdk_version`: string<br>query `codec`: string<br>query `sample_rate`: integer<br>query `device_time*`: string(date-time)<br>query `network`: string | 200 DeviceBootstrapResponse{data*:DeviceBootstrapData{clock_skew_ms*:integer(int64), device*:DeviceBootstrap, server_time*:string(date-time)}, request_id*:string} — 返回可信设备状态、服务器时间、绑定和最低安全版本 |
|
||||
| `pullDeviceCommands` | `GET /device/v2/commands` | 返回未过期的待执行命令并记录投递次数 | query `limit`: integer | 200 DeviceCommandDeliveryListResponse{data*:object{items*:array<DeviceCommandDelivery>}, request_id*:string} — 返回未过期的待执行命令并记录投递次数 |
|
||||
| `acknowledgeDeviceCommand` | `POST /device/v2/commands/{id}/ack` | 回执已保存 | path `id*`: string(uuid)<br>body* `application/json` DeviceCommandAckRequest{result_summary*:object, status*:string[ACKNOWLEDGED/SUCCEEDED/FAILED]} | 200 DeviceCommandDeliveryResponse{data*:DeviceCommandDelivery{created_at*:string(date-time), delivery_attempts*:integer, expires_at*:string(date-time), id*:string(uuid), payload*:object, status*:DeviceCommandStatus, type*:string[apply_config/reboot/diagnostics/firmware_update]}, request_id*:string} — 回执已保存 |
|
||||
| `getDeviceConfig` | `GET /device/v2/config` | 返回设备期望配置及当前绑定 | header `If-None-Match`: string | 200 DeviceConfigResponse{data*:DeviceConfiguration{binding*:DeviceBinding, desired_config*:object, device_id*:string(uuid), reported_revision:integer(int64), revision*:integer(int64)}, request_id*:string}；headers{ETag:string} — 返回设备期望配置及当前绑定<br>304 无响应体 — 配置未变化 |
|
||||
| `createDeviceConversation` | `POST /device/v2/conversations` | 分配房间和短期入房凭证；此步骤不启动语音智能体。 | header `Idempotency-Key*`: string<br>body* `application/json` CreateConversationRequest{capabilities*:DeviceCapabilities{audio_tracks*:array<string[near/far]>, rtc*:boolean, rtc_credential_renewal:boolean, upload_resume*:boolean}, device_time*:string(date-time), monotonic_ms:integer(int64), network*:DeviceNetwork{rssi:integer, type*:string}, requested_goal*:string} | 201 CreateConversationResponse{data*:CreatedConversation{conversation_id*:string(uuid), privacy*:ConversationPrivacy, rtc*:RTCGrant, session*:ConversationSession, status*:string}, request_id*:string}；headers{Idempotency-Replayed:string[true]} — 房间资料已分配；重复请求返回相同通话和相同确定性短期Token |
|
||||
| `getDeviceConversationState` | `GET /device/v2/conversations/{id}` | 设备确认入房后读取云端权威通话状态和下一步动作；供应商永久拒绝启动或异步启动失败时返回稳定failure_code及leave_room，响应不包含供应商原始错误。 | path `id*`: string(uuid) | 200 DeviceConversationStateResponse{data*:DeviceConversationState{action*:string[wait/continue/leave_room], conversation_id*:string(uuid), failure_code:string, status*:ConversationStatus}, request_id*:string} — 返回当前设备所属通话的权威状态和设备动作 |
|
||||
| `activateDeviceConversation` | `POST /device/v2/conversations/{id}/activate` | 设备确认已进入指定RTC房间后，投递固定供应商语音任务。 | path `id*`: string(uuid)<br>header `Idempotency-Key*`: string<br>body* `application/json` ActivateConversationRequest{config_revision*:integer(int64), connection_reference:string, joined_at*:string(date-time), room_id*:string, rtc_join_result*:string, rtc_user_id*:string} | 202 ActivateConversationResponse{data*:ActivatedConversation{accepted_at*:string(date-time), conversation_id*:string(uuid), status*:string[STARTING_AGENT/ACTIVE]}, request_id*:string} — 入房确认已接受；语音智能体正在启动或已经启动 |
|
||||
| `heartbeatDeviceConversation` | `POST /device/v2/conversations/{id}/heartbeat` | 按严格递增序号刷新设备在线状态和通话短期租约；重复或乱序心跳不会延长租约。 | path `id*`: string(uuid)<br>body* `application/json` HeartbeatConversationRequest{command_results*:array<object>, conversation_status*:string[PREPARING/ROOM_ISSUED/STARTING_AGENT/ACTIVE/STOPPING/COMPLETED/FAILED/EXPIRED], device_time*:string(date-time), firmware_version*:string, last_error:string, monotonic_ms*:integer(int64), network*:DeviceNetwork{rssi:integer, type*:string}, recording_queue*:RecordingQueue{failed_segments*:integer, last_failure_at:string(date-time), last_failure_code:string, pending_segments*:integer}, reported_config_revision*:integer(int64), rtc*:RTCHeartbeat{downlink*:boolean, joined*:boolean, uplink*:boolean}, rtc_credential_expires_at:string(date-time), sequence_no*:integer(int64)} | 200 HeartbeatConversationResponse{data*:HeartbeatResult{actions*:array<HeartbeatAction>, commands*:array<object>, desired_config_revision*:integer(int64), lease_expires_at:string(date-time)}, request_id*:string} — 心跳已接受，或已按序号判定为安全重放/乱序请求 |
|
||||
| `stopDeviceConversation` | `POST /device/v2/conversations/{id}/stop` | 记录设备挂断并可靠停止固定供应商任务；重复执行不产生第二个外部副作用。 | path `id*`: string(uuid)<br>header `Idempotency-Key*`: string<br>body* `application/json` StopConversationRequest{device_time*:string(date-time), last_heartbeat_sequence*:integer(int64), reason*:string, recording*:StopRecording{far_segments*:integer, near_segments*:integer}, rtc_left*:boolean} | 202 StopConversationResponse{data*:StoppedConversation{accepted_at*:string(date-time), conversation_id*:string(uuid), status*:string[STOPPING/COMPLETED/FAILED/EXPIRED]}, request_id*:string}；headers{Idempotency-Replayed:string[true]} — 停止请求已接受，或通话已经处于停止中/终态 |
|
||||
| `submitDeviceConversationTurns` | `POST /device/v2/conversations/{id}/turns:batch` | 当供应商最终转写缺失时由当前通话所属终端批量补传终稿；平台按turn_key和sequence_no与已有供应商终稿原子去重合并，任一内容冲突会拒绝整批请求。 | path `id*`: string(uuid)<br>header `Idempotency-Key*`: string<br>body* `application/json` DeviceTranscriptBatchRequest{turns*:array<DeviceFinalTurn>} | 200 DeviceTranscriptBatchResponse{data*:DeviceTranscriptBatchResult{accepted_count*:integer, conversation_id*:string(uuid), duplicate_count*:integer}, request_id*:string}；headers{Idempotency-Replayed:string[true]} — 整批终稿已合并；完全相同的已有终稿计入duplicate_count而不重复保存 |
|
||||
| `prepareMediaUploadBatch` | `POST /device/v2/conversations/{id}/upload-batches` | 设备提交near/far录音分片清单；业务API仅返回单对象短期PUT地址，不接收录音字节。 | path `id*`: string(uuid)<br>header `Idempotency-Key*`: string<br>body* `application/json` PrepareMediaUploadBatchRequest{segments*:array<MediaSegmentDeclaration>} | 201 MediaUploadBatchResponse{data*:MediaUploadBatch{conversation_id*:string(uuid), expected_total_size*:integer(int64), expires_at*:string(date-time), id*:string(uuid), last_error_code:string, rejected_count*:integer, segment_count*:integer, segments*:array<MediaSegment>, status*:string[PENDING/UPLOADING/VERIFYING/COMPLETED/FAILED/EXPIRED], verification_attempts*:integer, verified_count*:integer}, request_id*:string} — 返回尚未验证分片的10至20分钟单对象上传地址 |
|
||||
| `getDeviceFirmwareDownload` | `GET /device/v2/firmware-releases/{id}/download` | 仅向当前灰度目标签发 5 分钟有效的私有下载地址 | path `id*`: string(uuid) | 200 application/json `{download_url*:string(uri), download_until*:string(date-time)}`；header `Cache-Control: no-store` |
|
||||
| `acknowledgeLocalMediaDelete` | `POST /device/v2/media-segments/{id}/local-delete-ack` | 设备在平台给出的local_delete_after之后删除本地密文并回报结果；该操作不删除云端已验证对象，DELETED重复回报幂等。 | path `id*`: string(uuid)<br>body* `application/json` LocalMediaDeleteReceiptRequest{error_code:string, status*:string[DELETED/FAILED]} | 200 MediaSegmentResponse{data*:MediaSegment{codec*:string, content_type*:string, duration_ms*:integer, expected_digest*:string, expected_size*:integer(int64), id*:string(uuid), last_error_code:string, local_delete_after:string(date-time), local_delete_error_code:string, local_delete_reported_at:string(date-time), local_delete_status:string[DELETED/FAILED], sequence_no*:integer, status*:string[DECLARED/AUTHORIZED/UPLOADED/VERIFYING/VERIFIED/REJECTED/DELETED], track*:string[near/far], upload_until:string(date-time), upload_url:string(uri), verified_at:string(date-time)}, request_id*:string} — 已保存本地删除回执 |
|
||||
| `submitDeviceTelemetry` | `POST /device/v2/telemetry:batch` | 当前签名设备批量上报状态、有限错误码和命令执行结果；平台以设备内event_key和sequence_no原子去重，更新设备影子、命令状态和与当前设备通话关联的异常回收信号。 | header `Idempotency-Key*`: string<br>body* `application/json` DeviceTelemetryBatchRequest{events*:array<DeviceTelemetryEvent>} | 200 DeviceTelemetryBatchResponse{data*:DeviceTelemetryBatchResult{accepted_count*:integer, actions*:array<DeviceTelemetryAction>, desired_config_revision*:integer(int64), duplicate_count*:integer}, request_id*:string}；headers{Idempotency-Replayed:string[true]} — 整批遥测已接收；完全相同的已有事件计入duplicate_count |
|
||||
| `getDeviceMediaUploadBatch` | `GET /device/v2/upload-batches/{id}` | 返回分片校验结果；只有VERIFIED分片才包含local_delete_after | path `id*`: string(uuid) | 200 MediaUploadBatchResponse{data*:MediaUploadBatch{conversation_id*:string(uuid), expected_total_size*:integer(int64), expires_at*:string(date-time), id*:string(uuid), last_error_code:string, rejected_count*:integer, segment_count*:integer, segments*:array<MediaSegment>, status*:string[PENDING/UPLOADING/VERIFYING/COMPLETED/FAILED/EXPIRED], verification_attempts*:integer, verified_count*:integer}, request_id*:string} — 返回分片校验结果；只有VERIFIED分片才包含local_delete_after |
|
||||
| `completeMediaUploadBatch` | `POST /device/v2/upload-batches/{id}/complete` | 设备声明本批上传完成，平台可靠投递后台对象校验任务；重复提交不会创建重复任务。 | path `id*`: string(uuid) | 202 MediaUploadBatchResponse{data*:MediaUploadBatch{conversation_id*:string(uuid), expected_total_size*:integer(int64), expires_at*:string(date-time), id*:string(uuid), last_error_code:string, rejected_count*:integer, segment_count*:integer, segments*:array<MediaSegment>, status*:string[PENDING/UPLOADING/VERIFYING/COMPLETED/FAILED/EXPIRED], verification_attempts*:integer, verified_count*:integer}, request_id*:string} — 批次已进入VERIFYING或已经完成 |
|
||||
@@ -413,7 +413,7 @@ func (s *Service) resolveActor(ctx context.Context, principal identity.Principal
|
||||
return actor{}, ErrInvalid
|
||||
}
|
||||
var a actor
|
||||
e := s.pool.QueryRow(ctx, `SELECT t.id::text,t.public_id::text,ua.id::text,ua.public_id::text,ua.platform_role FROM user_account ua JOIN tenant t ON t.id=ua.tenant_id AND t.status='active' WHERE t.public_id=$1::uuid AND ua.oidc_issuer=$2 AND ua.oidc_subject=$3 AND ua.status='active'`, tenantPublicID, principal.Issuer, principal.Subject).Scan(&a.tenantID, &a.tenantPublicID, &a.userID, &a.userPublicID, &a.role)
|
||||
e := s.pool.QueryRow(ctx, `SELECT t.id::text,t.public_id::text,ua.id::text,ua.public_id::text,ua.platform_role FROM user_account ua JOIN tenant t ON t.id=ua.tenant_id AND t.status='active' WHERE t.public_id=$1::uuid AND ua.oidc_issuer=$2 AND ua.oidc_subject=$3 AND ua.status='active' AND ua.account_kind='operator'`, tenantPublicID, principal.Issuer, principal.Subject).Scan(&a.tenantID, &a.tenantPublicID, &a.userID, &a.userPublicID, &a.role)
|
||||
if errors.Is(e, pgx.ErrNoRows) {
|
||||
return actor{}, ErrAccessDenied
|
||||
}
|
||||
|
||||
@@ -32,8 +32,8 @@ func TestAlertRoutingSilenceAndIncidentLifecycle(t *testing.T) {
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
admin := identity.Principal{Issuer: "http://localhost:58180/realms/voice-local", Subject: "00000000-0000-0000-0000-000000000101"}
|
||||
approver := identity.Principal{Issuer: admin.Issuer, Subject: "00000000-0000-0000-0000-000000000102"}
|
||||
admin := identity.Principal{Issuer: "http://localhost:58180/realms/voice-local", Subject: "00000000-0000-0000-0000-000000000104"}
|
||||
approver := identity.Principal{Issuer: admin.Issuer, Subject: "00000000-0000-0000-0000-000000000105"}
|
||||
outsider := identity.Principal{Issuer: admin.Issuer, Subject: "00000000-0000-0000-0000-000000000103"}
|
||||
tenantID := "10000000-0000-0000-0000-000000000010"
|
||||
routeKeys := []string{"alerts-integration-route1-" + uuid.NewString(), "alerts-integration-route2-" + uuid.NewString()}
|
||||
|
||||
+56
-12
@@ -30,14 +30,17 @@ import (
|
||||
"data-terminal-cloud/internal/media"
|
||||
"data-terminal-cloud/internal/memoryindex"
|
||||
"data-terminal-cloud/internal/memorylifecycle"
|
||||
"data-terminal-cloud/internal/memorytool"
|
||||
"data-terminal-cloud/internal/objectstore"
|
||||
"data-terminal-cloud/internal/observability"
|
||||
"data-terminal-cloud/internal/opsaccess"
|
||||
"data-terminal-cloud/internal/opsconfig"
|
||||
"data-terminal-cloud/internal/opsdevice"
|
||||
"data-terminal-cloud/internal/orchestration"
|
||||
"data-terminal-cloud/internal/outbox"
|
||||
"data-terminal-cloud/internal/pagination"
|
||||
"data-terminal-cloud/internal/privacy"
|
||||
"data-terminal-cloud/internal/providermemory"
|
||||
"data-terminal-cloud/internal/ratelimit"
|
||||
"data-terminal-cloud/internal/rtc"
|
||||
"data-terminal-cloud/internal/saas"
|
||||
@@ -75,11 +78,18 @@ func Run(ctx context.Context, serviceName string) error {
|
||||
defer func() { _ = redisClient.Close() }()
|
||||
var exportStore *objectstore.Store
|
||||
var memoryEmbedding embeddingprovider.Provider
|
||||
var providerMemoryClient providermemory.Client
|
||||
if serviceName == "api" || serviceName == "worker" {
|
||||
memoryEmbedding, err = newMemoryEmbeddingProvider(cfg)
|
||||
if err != nil {
|
||||
return fmt.Errorf("create memory embedding provider: %w", err)
|
||||
}
|
||||
if cfg.ProviderMemoryMode == "managed_http" {
|
||||
providerMemoryClient, err = providermemory.NewHTTPClient(cfg.ProviderMemoryBaseURL, cfg.ProviderMemoryAPIKey, cfg.ProviderMemoryTimeout)
|
||||
if err != nil {
|
||||
return fmt.Errorf("create provider memory client: %w", err)
|
||||
}
|
||||
}
|
||||
exportStore, err = objectstore.New(cfg.ObjectStoreEndpoint, cfg.ObjectStorePublic, cfg.ObjectStoreAccessKey, cfg.ObjectStoreSecretKey, cfg.ObjectStoreBucket, cfg.ObjectStoreRegion, cfg.ObjectStoreSecure, cfg.ObjectStorePublicSecure)
|
||||
if err != nil {
|
||||
return fmt.Errorf("create private export object store: %w", err)
|
||||
@@ -124,6 +134,9 @@ func Run(ctx context.Context, serviceName string) error {
|
||||
var alertService *alerts.Service
|
||||
var diagnosticService *diagnostics.Service
|
||||
var opsAccessService *opsaccess.Service
|
||||
var opsDeviceService *opsdevice.Service
|
||||
var memoryToolService *memorytool.Service
|
||||
var memoryToolSigner *memorytool.TokenSigner
|
||||
if serviceName == "api" || serviceName == "callbacks" {
|
||||
diagnosticService, err = diagnostics.NewService(databasePool, idempotency.NewExecutor(databasePool), audit.NewRepository())
|
||||
if err != nil {
|
||||
@@ -131,12 +144,21 @@ func Run(ctx context.Context, serviceName string) error {
|
||||
}
|
||||
}
|
||||
if serviceName == "api" {
|
||||
memoryToolSigner, err = memorytool.NewTokenSigner([]byte(cfg.MemoryToolSigningSecret))
|
||||
if err != nil {
|
||||
return fmt.Errorf("create memory tool signer: %w", err)
|
||||
}
|
||||
memoryToolService, err = memorytool.NewService(databasePool, memoryEmbedding, audit.NewRepository())
|
||||
if err != nil {
|
||||
return fmt.Errorf("create memory tool service: %w", err)
|
||||
}
|
||||
webEventService = webevents.NewService(databasePool)
|
||||
cursorCodec, cursorErr := pagination.NewCodec([]byte(cfg.CursorSigningSecret))
|
||||
if cursorErr != nil {
|
||||
return fmt.Errorf("create pagination cursor codec: %w", cursorErr)
|
||||
}
|
||||
saasService = saas.NewService(databasePool, audit.NewRepository(), redisClient, cursorCodec)
|
||||
saasService.ConfigureProviderMemory(cfg.ProviderMemoryMode == "managed_http")
|
||||
opsConfigService, err = opsconfig.NewService(databasePool, audit.NewRepository())
|
||||
if err != nil {
|
||||
return fmt.Errorf("create operations configuration service: %w", err)
|
||||
@@ -166,21 +188,22 @@ func Run(ctx context.Context, serviceName string) error {
|
||||
}
|
||||
}
|
||||
deviceIdentityRepository := deviceauth.NewRepository(databasePool)
|
||||
var deviceSecretResolver deviceauth.SecretResolver
|
||||
if cfg.DeviceSecretDirectory != "" {
|
||||
deviceSecretResolver, err = deviceauth.NewDirectorySecretResolver(cfg.DeviceSecretDirectory)
|
||||
if err != nil {
|
||||
return fmt.Errorf("create per-device secret resolver: %w", err)
|
||||
}
|
||||
} else {
|
||||
// The local environment provides one already-bound simulator and one
|
||||
// unbound simulator so the family portal's one-time claim flow can be
|
||||
// exercised without provisioning production device secrets.
|
||||
deviceSecretResolver = deviceauth.StaticSecretResolver{
|
||||
deviceSecretResolver, err := deviceauth.NewDatabaseSecretResolver(databasePool, cfg.DeviceCredentialMasterKey)
|
||||
if err != nil {
|
||||
return fmt.Errorf("create database device secret resolver: %w", err)
|
||||
}
|
||||
if len(cfg.DeviceLocalSecret) >= 32 {
|
||||
if err := deviceSecretResolver.ImportMissingLegacyCredentials(ctx, map[string][]byte{
|
||||
"local-dev://simulated-terminal-1": []byte(cfg.DeviceLocalSecret),
|
||||
"local-dev://unbound-terminal-2": []byte(cfg.DeviceLocalSecret),
|
||||
}); err != nil {
|
||||
return fmt.Errorf("import legacy device credentials: %w", err)
|
||||
}
|
||||
}
|
||||
opsDeviceService, err = opsdevice.NewService(databasePool, deviceSecretResolver, audit.NewRepository())
|
||||
if err != nil {
|
||||
return fmt.Errorf("create operations device service: %w", err)
|
||||
}
|
||||
deviceVerifier, err = deviceauth.NewVerifier(
|
||||
deviceIdentityRepository,
|
||||
deviceSecretResolver,
|
||||
@@ -259,6 +282,7 @@ func Run(ctx context.Context, serviceName string) error {
|
||||
"tenant": {RatePerSecond: cfg.RateLimitTenantRPS, Burst: cfg.RateLimitTenantBurst},
|
||||
"device": {RatePerSecond: cfg.RateLimitDeviceRPS, Burst: cfg.RateLimitDeviceBurst},
|
||||
"provider_start": {RatePerSecond: cfg.ProviderStartRPS, Burst: cfg.ProviderStartBurst},
|
||||
"memory_tool": {RatePerSecond: cfg.MemoryToolRateRPS, Burst: cfg.MemoryToolRateBurst},
|
||||
})
|
||||
if err != nil {
|
||||
return fmt.Errorf("create rate limiter: %w", err)
|
||||
@@ -313,6 +337,15 @@ func Run(ctx context.Context, serviceName string) error {
|
||||
if processorErr = startProcessor.ConfigureStartProtection(limiter, cfg.ProviderMaxActiveTasks); processorErr != nil {
|
||||
return fmt.Errorf("configure provider start protection: %w", processorErr)
|
||||
}
|
||||
if cfg.MemoryToolBaseURL != "" {
|
||||
workerMemoryToolSigner, signerErr := memorytool.NewTokenSigner([]byte(cfg.MemoryToolSigningSecret))
|
||||
if signerErr != nil {
|
||||
return fmt.Errorf("create worker memory tool signer: %w", signerErr)
|
||||
}
|
||||
if processorErr = startProcessor.ConfigureMemoryTool(workerMemoryToolSigner, cfg.MemoryToolBaseURL, cfg.MemoryToolTokenTTL); processorErr != nil {
|
||||
return fmt.Errorf("configure provider memory tool: %w", processorErr)
|
||||
}
|
||||
}
|
||||
outboxWorker.Register("conversation.agent_start_requested", startProcessor.Handle)
|
||||
outboxWorker.Register("conversation.agent_start_reconcile_requested", startProcessor.Handle)
|
||||
stopProcessor, processorErr := orchestration.NewStopProcessor(databasePool, providerClient, providerHealth)
|
||||
@@ -347,6 +380,14 @@ func Run(ctx context.Context, serviceName string) error {
|
||||
return fmt.Errorf("create memory index processor: %w", processorErr)
|
||||
}
|
||||
outboxWorker.Register("memory.index_requested", memoryIndexProcessor.Handle)
|
||||
if providerMemoryClient != nil {
|
||||
providerMemoryProcessor, providerMemoryErr := providermemory.NewProcessor(databasePool, providerMemoryClient)
|
||||
if providerMemoryErr != nil {
|
||||
return fmt.Errorf("create provider memory processor: %w", providerMemoryErr)
|
||||
}
|
||||
outboxWorker.Register("memory.provider_sync_requested", providerMemoryProcessor.HandleSync)
|
||||
outboxWorker.Register("memory.provider_delete_requested", providerMemoryProcessor.HandleDelete)
|
||||
}
|
||||
memoryDeleteProcessor, processorErr := memorylifecycle.NewDeleteProcessor(databasePool)
|
||||
if processorErr != nil {
|
||||
return fmt.Errorf("create memory delete processor: %w", processorErr)
|
||||
@@ -357,7 +398,7 @@ func Run(ctx context.Context, serviceName string) error {
|
||||
return fmt.Errorf("create privacy access processor: %w", processorErr)
|
||||
}
|
||||
outboxWorker.Register("privacy.access_requested", privacyAccessProcessor.Handle)
|
||||
privacyDeleteProcessor, processorErr := privacy.NewDeleteProcessor(databasePool, redisClient, exportStore)
|
||||
privacyDeleteProcessor, processorErr := privacy.NewDeleteProcessor(databasePool, redisClient, exportStore, providerMemoryClient)
|
||||
if processorErr != nil {
|
||||
return fmt.Errorf("create privacy delete processor: %w", processorErr)
|
||||
}
|
||||
@@ -434,6 +475,7 @@ func Run(ctx context.Context, serviceName string) error {
|
||||
AlertService: alertService,
|
||||
DiagnosticService: diagnosticService,
|
||||
OpsAccessService: opsAccessService,
|
||||
OpsDeviceService: opsDeviceService,
|
||||
ConversationCreator: conversationCreator,
|
||||
ConversationActivator: conversationCreator,
|
||||
DeviceConversationStateReader: conversationCreator,
|
||||
@@ -446,6 +488,8 @@ func Run(ctx context.Context, serviceName string) error {
|
||||
PlatformHealthCallbackSecret: []byte(cfg.PlatformHealthSecret),
|
||||
PlatformHealthCallbackSkew: cfg.PlatformHealthSkew,
|
||||
PlatformHealthProcessor: diagnosticService,
|
||||
MemoryToolService: memoryToolService,
|
||||
MemoryToolSigner: memoryToolSigner,
|
||||
})
|
||||
server := &http.Server{
|
||||
Addr: cfg.HTTPAddr,
|
||||
|
||||
@@ -0,0 +1,347 @@
|
||||
package billing
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"sort"
|
||||
"time"
|
||||
|
||||
"data-terminal-cloud/internal/identity"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5"
|
||||
)
|
||||
|
||||
const maxUsageOverviewRange = 366 * 24 * time.Hour
|
||||
|
||||
type UsageOverviewScope struct {
|
||||
TenantID string `json:"tenant_id"`
|
||||
TenantName string `json:"tenant_name"`
|
||||
HouseholdID string `json:"household_id,omitempty"`
|
||||
HouseholdName string `json:"household_name,omitempty"`
|
||||
DeviceID string `json:"device_id,omitempty"`
|
||||
DeviceName string `json:"device_name,omitempty"`
|
||||
}
|
||||
|
||||
type UsageOverviewDay struct {
|
||||
Date string `json:"date"`
|
||||
DurationMS int64 `json:"duration_ms"`
|
||||
ConversationCount int `json:"conversation_count"`
|
||||
CostMicros int64 `json:"cost_micros"`
|
||||
}
|
||||
|
||||
type UsageOverviewDevice struct {
|
||||
ID string `json:"id"`
|
||||
Name string `json:"name"`
|
||||
HouseholdID string `json:"household_id"`
|
||||
HouseholdName string `json:"household_name"`
|
||||
DurationMS int64 `json:"duration_ms"`
|
||||
ConversationCount int `json:"conversation_count"`
|
||||
CostMicros int64 `json:"cost_micros"`
|
||||
}
|
||||
|
||||
type UsageOverviewConversation struct {
|
||||
ID string `json:"id"`
|
||||
HouseholdID string `json:"household_id"`
|
||||
HouseholdName string `json:"household_name"`
|
||||
DeviceID string `json:"device_id"`
|
||||
DeviceName string `json:"device_name"`
|
||||
SubjectName string `json:"subject_name"`
|
||||
Status string `json:"status"`
|
||||
StartedAt time.Time `json:"started_at"`
|
||||
EndedAt *time.Time `json:"ended_at,omitempty"`
|
||||
DurationMS int64 `json:"duration_ms"`
|
||||
CostMicros int64 `json:"cost_micros"`
|
||||
}
|
||||
|
||||
type UsageOverview struct {
|
||||
From time.Time `json:"from"`
|
||||
To time.Time `json:"to"`
|
||||
Timezone string `json:"timezone"`
|
||||
Scope UsageOverviewScope `json:"scope"`
|
||||
TotalDurationMS int64 `json:"total_duration_ms"`
|
||||
TodayDurationMS int64 `json:"today_duration_ms"`
|
||||
ConversationCount int `json:"conversation_count"`
|
||||
AverageDurationMS int64 `json:"average_duration_ms"`
|
||||
ActiveDays int `json:"active_days"`
|
||||
CurrentStreakDays int `json:"current_streak_days"`
|
||||
LongestStreakDays int `json:"longest_streak_days"`
|
||||
CostMicros int64 `json:"cost_micros"`
|
||||
Currency string `json:"currency"`
|
||||
CostAvailable bool `json:"cost_available"`
|
||||
Daily []UsageOverviewDay `json:"daily"`
|
||||
Devices []UsageOverviewDevice `json:"devices"`
|
||||
RecentConversations []UsageOverviewConversation `json:"recent_conversations"`
|
||||
}
|
||||
|
||||
type usageOverviewTarget struct {
|
||||
tenantID, tenantPublicID, tenantName string
|
||||
householdID, householdPublicID, householdName string
|
||||
deviceID, devicePublicID, deviceName string
|
||||
}
|
||||
|
||||
type usageOverviewRow struct {
|
||||
conversationID, householdID, householdName string
|
||||
deviceID, deviceName, subjectName, status string
|
||||
startedAt time.Time
|
||||
endedAt *time.Time
|
||||
costMicros int64
|
||||
currency string
|
||||
hasCost bool
|
||||
}
|
||||
|
||||
func (s *Service) FamilyUsageOverview(ctx context.Context, principal identity.Principal, householdPublicID, devicePublicID, timezone string, from, to time.Time) (UsageOverview, error) {
|
||||
if _, err := uuid.Parse(householdPublicID); err != nil {
|
||||
return UsageOverview{}, ErrInvalid
|
||||
}
|
||||
target := usageOverviewTarget{}
|
||||
var role identity.Role
|
||||
err := s.pool.QueryRow(ctx, `
|
||||
SELECT t.id::text,t.public_id::text,t.name,h.id::text,h.public_id::text,h.name,hm.role
|
||||
FROM user_account ua
|
||||
JOIN tenant t ON t.id=ua.tenant_id AND t.status='active'
|
||||
JOIN household_member hm ON hm.user_id=ua.id AND hm.tenant_id=ua.tenant_id AND hm.status='active'
|
||||
JOIN household h ON h.id=hm.household_id AND h.tenant_id=hm.tenant_id AND h.status<>'deleted'
|
||||
WHERE h.public_id=$1::uuid AND ua.oidc_issuer=$2 AND ua.oidc_subject=$3 AND ua.status='active'`,
|
||||
householdPublicID, principal.Issuer, principal.Subject).Scan(
|
||||
&target.tenantID, &target.tenantPublicID, &target.tenantName,
|
||||
&target.householdID, &target.householdPublicID, &target.householdName, &role)
|
||||
if errors.Is(err, pgx.ErrNoRows) || (err == nil && !identity.Can(role, identity.CapabilityHouseholdRead)) {
|
||||
return UsageOverview{}, ErrAccessDenied
|
||||
}
|
||||
if err != nil {
|
||||
return UsageOverview{}, err
|
||||
}
|
||||
if err = s.resolveUsageOverviewDevice(ctx, &target, devicePublicID, true); err != nil {
|
||||
return UsageOverview{}, err
|
||||
}
|
||||
return s.buildUsageOverview(ctx, target, timezone, from, to)
|
||||
}
|
||||
|
||||
func (s *Service) OpsUsageOverview(ctx context.Context, principal identity.Principal, tenantPublicID, householdPublicID, devicePublicID, timezone string, from, to time.Time) (UsageOverview, error) {
|
||||
currentActor, err := s.resolveActor(ctx, principal, tenantPublicID, "ops_viewer", "ops_operator", "ops_approver", "ops_admin")
|
||||
if err != nil {
|
||||
return UsageOverview{}, err
|
||||
}
|
||||
target := usageOverviewTarget{tenantID: currentActor.tenantID, tenantPublicID: currentActor.tenantPublicID}
|
||||
if err = s.pool.QueryRow(ctx, `SELECT name FROM tenant WHERE id=$1::uuid`, target.tenantID).Scan(&target.tenantName); err != nil {
|
||||
return UsageOverview{}, err
|
||||
}
|
||||
if householdPublicID != "" {
|
||||
if _, parseErr := uuid.Parse(householdPublicID); parseErr != nil {
|
||||
return UsageOverview{}, ErrInvalid
|
||||
}
|
||||
err = s.pool.QueryRow(ctx, `SELECT id::text,public_id::text,name FROM household WHERE tenant_id=$1::uuid AND public_id=$2::uuid AND status<>'deleted'`, target.tenantID, householdPublicID).Scan(&target.householdID, &target.householdPublicID, &target.householdName)
|
||||
if errors.Is(err, pgx.ErrNoRows) {
|
||||
return UsageOverview{}, ErrNotFound
|
||||
}
|
||||
if err != nil {
|
||||
return UsageOverview{}, err
|
||||
}
|
||||
}
|
||||
if err = s.resolveUsageOverviewDevice(ctx, &target, devicePublicID, false); err != nil {
|
||||
return UsageOverview{}, err
|
||||
}
|
||||
return s.buildUsageOverview(ctx, target, timezone, from, to)
|
||||
}
|
||||
|
||||
func (s *Service) resolveUsageOverviewDevice(ctx context.Context, target *usageOverviewTarget, devicePublicID string, family bool) error {
|
||||
if devicePublicID == "" {
|
||||
return nil
|
||||
}
|
||||
if _, err := uuid.Parse(devicePublicID); err != nil {
|
||||
return ErrInvalid
|
||||
}
|
||||
query := `SELECT d.id::text,d.public_id::text,d.model FROM device d WHERE d.tenant_id=$1::uuid AND d.public_id=$2::uuid`
|
||||
args := []any{target.tenantID, devicePublicID}
|
||||
if target.householdID != "" {
|
||||
query += ` AND (
|
||||
EXISTS(SELECT 1 FROM device_binding b WHERE b.tenant_id=d.tenant_id AND b.device_id=d.id AND b.household_id=$3::uuid AND b.unbound_at IS NULL)
|
||||
OR EXISTS(SELECT 1 FROM conversation c WHERE c.tenant_id=d.tenant_id AND c.device_id=d.id AND c.household_id=$3::uuid)
|
||||
)`
|
||||
args = append(args, target.householdID)
|
||||
}
|
||||
err := s.pool.QueryRow(ctx, query, args...).Scan(&target.deviceID, &target.devicePublicID, &target.deviceName)
|
||||
if errors.Is(err, pgx.ErrNoRows) {
|
||||
if family {
|
||||
return ErrAccessDenied
|
||||
}
|
||||
return ErrNotFound
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func (s *Service) buildUsageOverview(ctx context.Context, target usageOverviewTarget, timezone string, from, to time.Time) (UsageOverview, error) {
|
||||
location, err := validateUsageOverviewRange(timezone, from, to)
|
||||
if err != nil {
|
||||
return UsageOverview{}, err
|
||||
}
|
||||
from = from.UTC()
|
||||
to = to.UTC()
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT c.public_id::text,h.public_id::text,h.name,d.public_id::text,d.model,sp.preferred_name,c.status,c.started_at,c.ended_at,
|
||||
COALESCE(cost.cost_micros,0)::bigint,COALESCE(cost.currency,''),COALESCE(cost.has_cost,false)
|
||||
FROM conversation c
|
||||
JOIN household h ON h.id=c.household_id AND h.tenant_id=c.tenant_id
|
||||
JOIN device d ON d.id=c.device_id AND d.tenant_id=c.tenant_id
|
||||
JOIN subject_profile sp ON sp.id=c.subject_id AND sp.tenant_id=c.tenant_id
|
||||
LEFT JOIN LATERAL (
|
||||
SELECT SUM(ccl.cost_micros)::bigint AS cost_micros,MIN(ccl.currency) AS currency,COUNT(*)>0 AS has_cost
|
||||
FROM conversation_cost_line ccl WHERE ccl.conversation_id=c.id
|
||||
) cost ON true
|
||||
WHERE c.tenant_id=$1::uuid AND c.started_at IS NOT NULL AND c.started_at>=$2 AND c.started_at<$3
|
||||
AND ($4::uuid IS NULL OR c.household_id=$4::uuid)
|
||||
AND ($5::uuid IS NULL OR c.device_id=$5::uuid)
|
||||
ORDER BY c.started_at DESC,c.public_id`, target.tenantID, from, to, nullableUUID(target.householdID), nullableUUID(target.deviceID))
|
||||
if err != nil {
|
||||
return UsageOverview{}, err
|
||||
}
|
||||
defer rows.Close()
|
||||
|
||||
result := UsageOverview{
|
||||
From: from, To: to, Timezone: timezone,
|
||||
Scope: UsageOverviewScope{TenantID: target.tenantPublicID, TenantName: target.tenantName, HouseholdID: target.householdPublicID, HouseholdName: target.householdName, DeviceID: target.devicePublicID, DeviceName: target.deviceName},
|
||||
Daily: []UsageOverviewDay{}, Devices: []UsageOverviewDevice{}, RecentConversations: []UsageOverviewConversation{},
|
||||
}
|
||||
daily := map[string]*UsageOverviewDay{}
|
||||
devices := map[string]*UsageOverviewDevice{}
|
||||
activeDates := map[string]struct{}{}
|
||||
currencies := map[string]struct{}{}
|
||||
today := time.Now().In(location).Format("2006-01-02")
|
||||
for rows.Next() {
|
||||
var row usageOverviewRow
|
||||
if err = rows.Scan(&row.conversationID, &row.householdID, &row.householdName, &row.deviceID, &row.deviceName, &row.subjectName, &row.status, &row.startedAt, &row.endedAt, &row.costMicros, &row.currency, &row.hasCost); err != nil {
|
||||
return UsageOverview{}, err
|
||||
}
|
||||
duration := conversationDurationMS(row.startedAt, row.endedAt, to)
|
||||
date := row.startedAt.In(location).Format("2006-01-02")
|
||||
day := daily[date]
|
||||
if day == nil {
|
||||
day = &UsageOverviewDay{Date: date}
|
||||
daily[date] = day
|
||||
}
|
||||
day.DurationMS += duration
|
||||
day.ConversationCount++
|
||||
day.CostMicros += row.costMicros
|
||||
activeDates[date] = struct{}{}
|
||||
|
||||
device := devices[row.deviceID]
|
||||
if device == nil {
|
||||
device = &UsageOverviewDevice{ID: row.deviceID, Name: row.deviceName, HouseholdID: row.householdID, HouseholdName: row.householdName}
|
||||
devices[row.deviceID] = device
|
||||
}
|
||||
device.DurationMS += duration
|
||||
device.ConversationCount++
|
||||
device.CostMicros += row.costMicros
|
||||
|
||||
result.TotalDurationMS += duration
|
||||
result.ConversationCount++
|
||||
result.CostMicros += row.costMicros
|
||||
if date == today {
|
||||
result.TodayDurationMS += duration
|
||||
}
|
||||
if row.hasCost {
|
||||
result.CostAvailable = true
|
||||
currencies[row.currency] = struct{}{}
|
||||
}
|
||||
if len(result.RecentConversations) < 20 {
|
||||
result.RecentConversations = append(result.RecentConversations, UsageOverviewConversation{
|
||||
ID: row.conversationID, HouseholdID: row.householdID, HouseholdName: row.householdName,
|
||||
DeviceID: row.deviceID, DeviceName: row.deviceName, SubjectName: row.subjectName, Status: row.status,
|
||||
StartedAt: row.startedAt, EndedAt: row.endedAt, DurationMS: duration, CostMicros: row.costMicros,
|
||||
})
|
||||
}
|
||||
}
|
||||
if err = rows.Err(); err != nil {
|
||||
return UsageOverview{}, err
|
||||
}
|
||||
if len(currencies) > 1 {
|
||||
return UsageOverview{}, errors.New("usage overview contains multiple currencies")
|
||||
}
|
||||
for currency := range currencies {
|
||||
result.Currency = currency
|
||||
}
|
||||
if result.ConversationCount > 0 {
|
||||
result.AverageDurationMS = result.TotalDurationMS / int64(result.ConversationCount)
|
||||
}
|
||||
for _, day := range daily {
|
||||
result.Daily = append(result.Daily, *day)
|
||||
}
|
||||
sort.Slice(result.Daily, func(i, j int) bool { return result.Daily[i].Date < result.Daily[j].Date })
|
||||
for _, device := range devices {
|
||||
result.Devices = append(result.Devices, *device)
|
||||
}
|
||||
sort.Slice(result.Devices, func(i, j int) bool {
|
||||
if result.Devices[i].DurationMS == result.Devices[j].DurationMS {
|
||||
return result.Devices[i].ID < result.Devices[j].ID
|
||||
}
|
||||
return result.Devices[i].DurationMS > result.Devices[j].DurationMS
|
||||
})
|
||||
result.ActiveDays = len(activeDates)
|
||||
result.CurrentStreakDays, result.LongestStreakDays = usageStreaks(activeDates, time.Now().In(location))
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func validateUsageOverviewRange(timezone string, from, to time.Time) (*time.Location, error) {
|
||||
if timezone == "" || from.IsZero() || !to.After(from) || to.Sub(from) > maxUsageOverviewRange || to.After(time.Now().UTC().Add(5*time.Minute)) {
|
||||
return nil, ErrInvalid
|
||||
}
|
||||
location, err := time.LoadLocation(timezone)
|
||||
if err != nil {
|
||||
return nil, ErrInvalid
|
||||
}
|
||||
return location, nil
|
||||
}
|
||||
|
||||
func conversationDurationMS(startedAt time.Time, endedAt *time.Time, rangeEnd time.Time) int64 {
|
||||
end := rangeEnd
|
||||
if endedAt != nil && endedAt.Before(end) {
|
||||
end = *endedAt
|
||||
}
|
||||
if end.Before(startedAt) {
|
||||
return 0
|
||||
}
|
||||
return end.Sub(startedAt).Milliseconds()
|
||||
}
|
||||
|
||||
func usageStreaks(active map[string]struct{}, now time.Time) (current, longest int) {
|
||||
if len(active) == 0 {
|
||||
return 0, 0
|
||||
}
|
||||
dates := make([]time.Time, 0, len(active))
|
||||
for value := range active {
|
||||
date, err := time.ParseInLocation("2006-01-02", value, now.Location())
|
||||
if err == nil {
|
||||
dates = append(dates, date)
|
||||
}
|
||||
}
|
||||
sort.Slice(dates, func(i, j int) bool { return dates[i].Before(dates[j]) })
|
||||
streak := 0
|
||||
for index, date := range dates {
|
||||
if index == 0 || date.Equal(dates[index-1].AddDate(0, 0, 1)) {
|
||||
streak++
|
||||
} else {
|
||||
streak = 1
|
||||
}
|
||||
if streak > longest {
|
||||
longest = streak
|
||||
}
|
||||
}
|
||||
latest := dates[len(dates)-1]
|
||||
today, _ := time.ParseInLocation("2006-01-02", now.Format("2006-01-02"), now.Location())
|
||||
if latest.Before(today.AddDate(0, 0, -1)) {
|
||||
return 0, longest
|
||||
}
|
||||
current = 1
|
||||
for index := len(dates) - 1; index > 0 && dates[index].Equal(dates[index-1].AddDate(0, 0, 1)); index-- {
|
||||
current++
|
||||
}
|
||||
return current, longest
|
||||
}
|
||||
|
||||
func nullableUUID(value string) any {
|
||||
if value == "" {
|
||||
return nil
|
||||
}
|
||||
return value
|
||||
}
|
||||
@@ -0,0 +1,106 @@
|
||||
package billing
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"os"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"data-terminal-cloud/internal/audit"
|
||||
"data-terminal-cloud/internal/database"
|
||||
"data-terminal-cloud/internal/identity"
|
||||
)
|
||||
|
||||
func TestValidateUsageOverviewRange(t *testing.T) {
|
||||
now := time.Now().UTC()
|
||||
if _, err := validateUsageOverviewRange("Asia/Shanghai", now.Add(-30*24*time.Hour), now); err != nil {
|
||||
t.Fatalf("valid range: %v", err)
|
||||
}
|
||||
for _, test := range []struct {
|
||||
name string
|
||||
timezone string
|
||||
from time.Time
|
||||
to time.Time
|
||||
}{
|
||||
{name: "timezone", timezone: "not/a-zone", from: now.Add(-time.Hour), to: now},
|
||||
{name: "reversed", timezone: "Asia/Shanghai", from: now, to: now.Add(-time.Hour)},
|
||||
{name: "too long", timezone: "Asia/Shanghai", from: now.Add(-367 * 24 * time.Hour), to: now},
|
||||
} {
|
||||
t.Run(test.name, func(t *testing.T) {
|
||||
if _, err := validateUsageOverviewRange(test.timezone, test.from, test.to); err == nil {
|
||||
t.Fatal("expected validation error")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestUsageStreaks(t *testing.T) {
|
||||
location, err := time.LoadLocation("Asia/Shanghai")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
now := time.Date(2026, 8, 29, 18, 0, 0, 0, location)
|
||||
active := map[string]struct{}{
|
||||
"2026-08-20": {}, "2026-08-21": {},
|
||||
"2026-08-26": {}, "2026-08-27": {}, "2026-08-28": {}, "2026-08-29": {},
|
||||
}
|
||||
current, longest := usageStreaks(active, now)
|
||||
if current != 4 || longest != 4 {
|
||||
t.Fatalf("current=%d longest=%d", current, longest)
|
||||
}
|
||||
current, longest = usageStreaks(map[string]struct{}{"2026-08-20": {}}, now)
|
||||
if current != 0 || longest != 1 {
|
||||
t.Fatalf("stale current=%d longest=%d", current, longest)
|
||||
}
|
||||
}
|
||||
|
||||
func TestConversationDurationMS(t *testing.T) {
|
||||
start := time.Date(2026, 8, 29, 10, 0, 0, 0, time.UTC)
|
||||
end := start.Add(15*time.Minute + 250*time.Millisecond)
|
||||
if got := conversationDurationMS(start, &end, start.Add(time.Hour)); got != 900250 {
|
||||
t.Fatalf("duration=%d", got)
|
||||
}
|
||||
if got := conversationDurationMS(start, nil, start.Add(3*time.Minute)); got != 180000 {
|
||||
t.Fatalf("active duration=%d", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestUsageOverviewScopeIsolationIntegration(t *testing.T) {
|
||||
databaseURL := os.Getenv("TEST_DATABASE_URL")
|
||||
if databaseURL == "" {
|
||||
t.Skip("TEST_DATABASE_URL is not set")
|
||||
}
|
||||
ctx := context.Background()
|
||||
pool, err := database.Open(ctx, databaseURL, "usage-overview-test")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer pool.Close()
|
||||
service, err := NewService(pool, audit.NewRepository())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
now := time.Now().UTC()
|
||||
owner := identity.Principal{Issuer: billingTestIssuer, Subject: "00000000-0000-0000-0000-000000000101"}
|
||||
var operatorIssuer, operatorSubject string
|
||||
if err = pool.QueryRow(ctx, `SELECT oidc_issuer,oidc_subject FROM user_account WHERE platform_role='ops_admin' AND account_kind='operator' AND status='active' ORDER BY created_at LIMIT 1`).Scan(&operatorIssuer, &operatorSubject); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
operator := identity.Principal{Issuer: operatorIssuer, Subject: operatorSubject}
|
||||
|
||||
family, err := service.FamilyUsageOverview(ctx, owner, "10000000-0000-0000-0000-000000000211", billingDevicePublicID, "Asia/Shanghai", now.Add(-365*24*time.Hour), now)
|
||||
if err != nil || family.Scope.HouseholdID != "10000000-0000-0000-0000-000000000211" || family.Scope.DeviceID != billingDevicePublicID {
|
||||
t.Fatalf("family overview=%#v err=%v", family.Scope, err)
|
||||
}
|
||||
if _, err = service.FamilyUsageOverview(ctx, owner, "20000000-0000-0000-0000-000000000211", "", "Asia/Shanghai", now.Add(-24*time.Hour), now); !errors.Is(err, ErrAccessDenied) {
|
||||
t.Fatalf("cross-household error=%v", err)
|
||||
}
|
||||
ops, err := service.OpsUsageOverview(ctx, operator, billingTenantPublicID, "10000000-0000-0000-0000-000000000211", billingDevicePublicID, "Asia/Shanghai", now.Add(-365*24*time.Hour), now)
|
||||
if err != nil || ops.Scope.TenantID != billingTenantPublicID || ops.Scope.HouseholdID == "" || ops.Scope.DeviceID == "" {
|
||||
t.Fatalf("ops overview=%#v err=%v", ops.Scope, err)
|
||||
}
|
||||
if _, err = service.OpsUsageOverview(ctx, operator, billingOtherTenantID, "", "", "Asia/Shanghai", now.Add(-24*time.Hour), now); !errors.Is(err, ErrAccessDenied) {
|
||||
t.Fatalf("cross-tenant error=%v", err)
|
||||
}
|
||||
}
|
||||
@@ -376,7 +376,7 @@ func (s *Service) resolveActor(ctx context.Context, principal identity.Principal
|
||||
return actor{}, ErrInvalid
|
||||
}
|
||||
var result actor
|
||||
err := s.pool.QueryRow(ctx, `SELECT t.id::text,t.public_id::text,ua.id::text,ua.public_id::text,ua.platform_role FROM user_account ua JOIN tenant t ON t.id=ua.tenant_id AND t.status='active' WHERE t.public_id=$1::uuid AND ua.oidc_issuer=$2 AND ua.oidc_subject=$3 AND ua.status='active'`, tenantPublicID, principal.Issuer, principal.Subject).Scan(&result.tenantID, &result.tenantPublicID, &result.userID, &result.userPublicID, &result.role)
|
||||
err := s.pool.QueryRow(ctx, `SELECT t.id::text,t.public_id::text,ua.id::text,ua.public_id::text,ua.platform_role FROM user_account ua JOIN tenant t ON t.id=ua.tenant_id AND t.status='active' WHERE t.public_id=$1::uuid AND ua.oidc_issuer=$2 AND ua.oidc_subject=$3 AND ua.status='active' AND ua.account_kind='operator'`, tenantPublicID, principal.Issuer, principal.Subject).Scan(&result.tenantID, &result.tenantPublicID, &result.userID, &result.userPublicID, &result.role)
|
||||
if errors.Is(err, pgx.ErrNoRows) {
|
||||
return actor{}, ErrAccessDenied
|
||||
}
|
||||
|
||||
@@ -19,8 +19,8 @@ const (
|
||||
billingTestIssuer = "http://localhost:58180/realms/voice-local"
|
||||
billingTenantPublicID = "10000000-0000-0000-0000-000000000010"
|
||||
billingOtherTenantID = "20000000-0000-0000-0000-000000000010"
|
||||
billingAdminSubject = "00000000-0000-0000-0000-000000000101"
|
||||
billingApproverSubject = "00000000-0000-0000-0000-000000000102"
|
||||
billingAdminSubject = "00000000-0000-0000-0000-000000000104"
|
||||
billingApproverSubject = "00000000-0000-0000-0000-000000000105"
|
||||
billingDevicePublicID = "10000000-0000-0000-0000-000000000511"
|
||||
billingTestSource = "integration://billing-service"
|
||||
billingTestChangePrefix = "费用集成测试"
|
||||
|
||||
+233
-167
@@ -2,6 +2,7 @@ package config
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/hex"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
@@ -22,91 +23,100 @@ const (
|
||||
)
|
||||
|
||||
type Config struct {
|
||||
ServiceName string
|
||||
Environment string
|
||||
LogLevel string
|
||||
HTTPAddr string
|
||||
DatabaseURL string
|
||||
RedisAddr string
|
||||
RedisPassword string
|
||||
RedisDB int
|
||||
AuthIssuerURL string
|
||||
AuthJWKSURL string
|
||||
AuthAudience string
|
||||
OTLPEndpoint string
|
||||
RateLimitRPS float64
|
||||
RateLimitBurst int
|
||||
RateLimitUserRPS float64
|
||||
RateLimitUserBurst int
|
||||
RateLimitTenantRPS float64
|
||||
RateLimitTenantBurst int
|
||||
RateLimitDeviceRPS float64
|
||||
RateLimitDeviceBurst int
|
||||
ProviderStartRPS float64
|
||||
ProviderStartBurst int
|
||||
ProviderMaxActiveTasks int
|
||||
DeviceLocalSecret string
|
||||
DeviceSecretDirectory string
|
||||
DeviceClockSkew time.Duration
|
||||
RTCAppID string
|
||||
RTCAppKey string
|
||||
RTCTokenIssuer string
|
||||
RTCTokenSecret string
|
||||
CursorSigningSecret string
|
||||
ShareTokenSecret string
|
||||
RTCTokenTTL time.Duration
|
||||
AllocationTTL time.Duration
|
||||
HeartbeatInterval int
|
||||
ProviderCallbackSecret string
|
||||
ProviderCallbackSkew time.Duration
|
||||
PlatformHealthSecret string
|
||||
PlatformHealthSkew time.Duration
|
||||
MockProviderBaseURL string
|
||||
MockProviderAPIKey string
|
||||
VoiceProvider string
|
||||
VolcRTCBaseURL string
|
||||
VolcAccessKeyID string
|
||||
VolcSecretAccessKey string
|
||||
VolcSessionToken string
|
||||
VolcRTCRegion string
|
||||
VolcRTCAPIVersion string
|
||||
VolcVoiceChatTemplate string
|
||||
ProviderRequestTimeout time.Duration
|
||||
ProviderFailureThreshold int
|
||||
ProviderCircuitCooldown time.Duration
|
||||
DependencyTimeout time.Duration
|
||||
ExternalProbeInterval time.Duration
|
||||
ShutdownTimeout time.Duration
|
||||
WorkerPollInterval time.Duration
|
||||
RecoveryScanInterval time.Duration
|
||||
RecoveryStartingAfter time.Duration
|
||||
RecoveryStoppingAfter time.Duration
|
||||
ObjectStoreEndpoint string
|
||||
ObjectStorePublic string
|
||||
ObjectStoreAccessKey string
|
||||
ObjectStoreSecretKey string
|
||||
ObjectStoreBucket string
|
||||
ObjectStoreRegion string
|
||||
ObjectStoreSecure bool
|
||||
ObjectStorePublicSecure bool
|
||||
ExportDownloadTTL time.Duration
|
||||
ExportObjectTTL time.Duration
|
||||
MediaLocalDeleteGrace time.Duration
|
||||
AlertWebhookURL string
|
||||
AlertWebhookSecret string
|
||||
AlertWebhookTimeout time.Duration
|
||||
ContentGenerator string
|
||||
ArkBaseURL string
|
||||
ArkAPIKey string
|
||||
ArkContentModel string
|
||||
ArkRequestTimeout time.Duration
|
||||
ContentSafetyProvider string
|
||||
ContentSafetyEndpoint string
|
||||
ContentSafetyAPIKey string
|
||||
ContentSafetyPolicyRef string
|
||||
ContentSafetyTimeout time.Duration
|
||||
EmbeddingProvider string
|
||||
ArkEmbeddingModel string
|
||||
ServiceName string
|
||||
Environment string
|
||||
LogLevel string
|
||||
HTTPAddr string
|
||||
DatabaseURL string
|
||||
RedisAddr string
|
||||
RedisPassword string
|
||||
RedisDB int
|
||||
AuthIssuerURL string
|
||||
AuthJWKSURL string
|
||||
AuthAudience string
|
||||
OTLPEndpoint string
|
||||
RateLimitRPS float64
|
||||
RateLimitBurst int
|
||||
RateLimitUserRPS float64
|
||||
RateLimitUserBurst int
|
||||
RateLimitTenantRPS float64
|
||||
RateLimitTenantBurst int
|
||||
RateLimitDeviceRPS float64
|
||||
RateLimitDeviceBurst int
|
||||
ProviderStartRPS float64
|
||||
ProviderStartBurst int
|
||||
ProviderMaxActiveTasks int
|
||||
DeviceLocalSecret string
|
||||
DeviceCredentialMasterKey string
|
||||
DeviceClockSkew time.Duration
|
||||
RTCAppID string
|
||||
RTCAppKey string
|
||||
RTCTokenIssuer string
|
||||
RTCTokenSecret string
|
||||
CursorSigningSecret string
|
||||
ShareTokenSecret string
|
||||
RTCTokenTTL time.Duration
|
||||
AllocationTTL time.Duration
|
||||
HeartbeatInterval int
|
||||
ProviderCallbackSecret string
|
||||
ProviderCallbackSkew time.Duration
|
||||
PlatformHealthSecret string
|
||||
PlatformHealthSkew time.Duration
|
||||
MockProviderBaseURL string
|
||||
MockProviderAPIKey string
|
||||
VoiceProvider string
|
||||
VolcRTCBaseURL string
|
||||
VolcAccessKeyID string
|
||||
VolcSecretAccessKey string
|
||||
VolcSessionToken string
|
||||
VolcRTCRegion string
|
||||
VolcRTCAPIVersion string
|
||||
VolcVoiceChatTemplate string
|
||||
ProviderRequestTimeout time.Duration
|
||||
ProviderFailureThreshold int
|
||||
ProviderCircuitCooldown time.Duration
|
||||
DependencyTimeout time.Duration
|
||||
ExternalProbeInterval time.Duration
|
||||
ShutdownTimeout time.Duration
|
||||
WorkerPollInterval time.Duration
|
||||
RecoveryScanInterval time.Duration
|
||||
RecoveryStartingAfter time.Duration
|
||||
RecoveryStoppingAfter time.Duration
|
||||
ObjectStoreEndpoint string
|
||||
ObjectStorePublic string
|
||||
ObjectStoreAccessKey string
|
||||
ObjectStoreSecretKey string
|
||||
ObjectStoreBucket string
|
||||
ObjectStoreRegion string
|
||||
ObjectStoreSecure bool
|
||||
ObjectStorePublicSecure bool
|
||||
ExportDownloadTTL time.Duration
|
||||
ExportObjectTTL time.Duration
|
||||
MediaLocalDeleteGrace time.Duration
|
||||
AlertWebhookURL string
|
||||
AlertWebhookSecret string
|
||||
AlertWebhookTimeout time.Duration
|
||||
ContentGenerator string
|
||||
ArkBaseURL string
|
||||
ArkAPIKey string
|
||||
ArkContentModel string
|
||||
ArkRequestTimeout time.Duration
|
||||
ContentSafetyProvider string
|
||||
ContentSafetyEndpoint string
|
||||
ContentSafetyAPIKey string
|
||||
ContentSafetyPolicyRef string
|
||||
ContentSafetyTimeout time.Duration
|
||||
EmbeddingProvider string
|
||||
ArkEmbeddingModel string
|
||||
MemoryToolBaseURL string
|
||||
MemoryToolSigningSecret string
|
||||
MemoryToolTokenTTL time.Duration
|
||||
MemoryToolRateRPS float64
|
||||
MemoryToolRateBurst int
|
||||
ProviderMemoryMode string
|
||||
ProviderMemoryBaseURL string
|
||||
ProviderMemoryAPIKey string
|
||||
ProviderMemoryTimeout time.Duration
|
||||
}
|
||||
|
||||
func Load(serviceName string) (Config, error) {
|
||||
@@ -198,6 +208,14 @@ func Load(serviceName string) (Config, error) {
|
||||
if err != nil {
|
||||
return Config{}, err
|
||||
}
|
||||
providerMemoryTimeout, err := durationEnv("PROVIDER_MEMORY_TIMEOUT", 5*time.Second)
|
||||
if err != nil {
|
||||
return Config{}, err
|
||||
}
|
||||
memoryToolTokenTTL, err := durationEnv("MEMORY_TOOL_TOKEN_TTL", 4*time.Hour)
|
||||
if err != nil {
|
||||
return Config{}, err
|
||||
}
|
||||
objectStoreSecure, err := boolEnv("OBJECT_STORE_SECURE", false)
|
||||
if err != nil {
|
||||
return Config{}, err
|
||||
@@ -208,15 +226,16 @@ func Load(serviceName string) (Config, error) {
|
||||
}
|
||||
environment := stringEnv("APP_ENV", EnvironmentLocal)
|
||||
secretValues, err := loadSecretValues(
|
||||
"DATABASE_URL", "REDIS_PASSWORD", "DEVICE_LOCAL_SECRET", "RTC_APP_KEY", "RTC_TOKEN_SIGNING_SECRET",
|
||||
"DATABASE_URL", "REDIS_PASSWORD", "DEVICE_LOCAL_SECRET", "DEVICE_CREDENTIAL_MASTER_KEY", "RTC_APP_KEY", "RTC_TOKEN_SIGNING_SECRET",
|
||||
"CURSOR_SIGNING_SECRET", "SHARE_TOKEN_SIGNING_SECRET", "PROVIDER_CALLBACK_SECRET", "PLATFORM_HEALTH_CALLBACK_SECRET",
|
||||
"MOCK_PROVIDER_API_KEY", "VOLC_ACCESS_KEY_ID", "VOLC_SECRET_ACCESS_KEY", "VOLC_SESSION_TOKEN",
|
||||
"OBJECT_STORE_ACCESS_KEY", "OBJECT_STORE_SECRET_KEY", "ALERT_WEBHOOK_SECRET", "ARK_API_KEY", "CONTENT_SAFETY_API_KEY",
|
||||
"OBJECT_STORE_ACCESS_KEY", "OBJECT_STORE_SECRET_KEY", "ALERT_WEBHOOK_SECRET", "ARK_API_KEY", "CONTENT_SAFETY_API_KEY", "MEMORY_TOOL_SIGNING_SECRET", "PROVIDER_MEMORY_API_KEY",
|
||||
)
|
||||
if err != nil {
|
||||
return Config{}, err
|
||||
}
|
||||
deviceLocalSecret := secretValues["DEVICE_LOCAL_SECRET"]
|
||||
deviceCredentialMasterKey := secretValues["DEVICE_CREDENTIAL_MASTER_KEY"]
|
||||
rtcTokenSecret := secretValues["RTC_TOKEN_SIGNING_SECRET"]
|
||||
cursorSigningSecret := secretValues["CURSOR_SIGNING_SECRET"]
|
||||
shareTokenSecret := secretValues["SHARE_TOKEN_SIGNING_SECRET"]
|
||||
@@ -230,6 +249,9 @@ func Load(serviceName string) (Config, error) {
|
||||
if deviceLocalSecret == "" && serviceName == "api" && (environment == EnvironmentLocal || environment == EnvironmentTest) {
|
||||
deviceLocalSecret = "local-simulated-device-secret-32-bytes-minimum"
|
||||
}
|
||||
if deviceCredentialMasterKey == "" && serviceName == "api" && (environment == EnvironmentLocal || environment == EnvironmentTest) {
|
||||
deviceCredentialMasterKey = "000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f"
|
||||
}
|
||||
if rtcTokenSecret == "" && serviceName == "api" && (environment == EnvironmentLocal || environment == EnvironmentTest) {
|
||||
rtcTokenSecret = "local-simulated-rtc-token-secret-32-bytes-minimum"
|
||||
}
|
||||
@@ -239,6 +261,10 @@ func Load(serviceName string) (Config, error) {
|
||||
if shareTokenSecret == "" && serviceName == "api" && (environment == EnvironmentLocal || environment == EnvironmentTest) {
|
||||
shareTokenSecret = "local-share-token-signing-secret-32-bytes-minimum"
|
||||
}
|
||||
memoryToolSigningSecret := secretValues["MEMORY_TOOL_SIGNING_SECRET"]
|
||||
if memoryToolSigningSecret == "" && (serviceName == "api" || serviceName == "worker") && (environment == EnvironmentLocal || environment == EnvironmentTest) {
|
||||
memoryToolSigningSecret = "local-memory-tool-signing-secret-32-bytes-minimum"
|
||||
}
|
||||
if providerCallbackSecret == "" && serviceName == "callbacks" && (environment == EnvironmentLocal || environment == EnvironmentTest) {
|
||||
providerCallbackSecret = "local-provider-callback-secret-32-bytes-minimum"
|
||||
}
|
||||
@@ -264,78 +290,85 @@ func Load(serviceName string) (Config, error) {
|
||||
}
|
||||
|
||||
cfg := Config{
|
||||
ServiceName: serviceName,
|
||||
Environment: environment,
|
||||
LogLevel: stringEnv("LOG_LEVEL", "info"),
|
||||
HTTPAddr: stringEnv("HTTP_ADDR", defaultHTTPAddr(serviceName)),
|
||||
DatabaseURL: secretDefault(secretValues["DATABASE_URL"], "postgres://voice:voice_dev_only@localhost:55432/voice_platform?sslmode=disable"),
|
||||
RedisAddr: stringEnv("REDIS_ADDR", "localhost:56379"),
|
||||
RedisPassword: secretValues["REDIS_PASSWORD"],
|
||||
RedisDB: redisDB,
|
||||
AuthIssuerURL: stringEnv("AUTH_ISSUER_URL", "http://localhost:58180/realms/voice-local"),
|
||||
AuthJWKSURL: stringEnv("AUTH_JWKS_URL", "http://localhost:58180/realms/voice-local/protocol/openid-connect/certs"),
|
||||
AuthAudience: stringEnv("AUTH_AUDIENCE", "voice-platform-web"),
|
||||
OTLPEndpoint: stringEnvAllowEmpty("OTEL_EXPORTER_OTLP_ENDPOINT", "localhost:54317"),
|
||||
DependencyTimeout: dependencyTimeout,
|
||||
ExternalProbeInterval: externalProbeInterval,
|
||||
ShutdownTimeout: shutdownTimeout,
|
||||
WorkerPollInterval: pollInterval,
|
||||
RecoveryScanInterval: recoveryScanInterval,
|
||||
RecoveryStartingAfter: recoveryStartingAfter,
|
||||
RecoveryStoppingAfter: recoveryStoppingAfter,
|
||||
DeviceLocalSecret: deviceLocalSecret,
|
||||
DeviceSecretDirectory: strings.TrimSpace(os.Getenv("DEVICE_SECRET_DIRECTORY")),
|
||||
DeviceClockSkew: deviceClockSkew,
|
||||
RTCAppID: stringEnv("RTC_APP_ID", "local-public-rtc-app"),
|
||||
RTCAppKey: secretValues["RTC_APP_KEY"],
|
||||
RTCTokenIssuer: stringEnv("RTC_TOKEN_ISSUER", "local"),
|
||||
RTCTokenSecret: rtcTokenSecret,
|
||||
CursorSigningSecret: cursorSigningSecret,
|
||||
ShareTokenSecret: shareTokenSecret,
|
||||
RTCTokenTTL: rtcTokenTTL,
|
||||
AllocationTTL: allocationTTL,
|
||||
ProviderCallbackSecret: providerCallbackSecret,
|
||||
ProviderCallbackSkew: providerCallbackSkew,
|
||||
PlatformHealthSecret: secretValues["PLATFORM_HEALTH_CALLBACK_SECRET"],
|
||||
PlatformHealthSkew: platformHealthSkew,
|
||||
MockProviderBaseURL: stringEnv("MOCK_PROVIDER_BASE_URL", "http://localhost:59200"),
|
||||
MockProviderAPIKey: secretDefault(secretValues["MOCK_PROVIDER_API_KEY"], "local-mock-provider-key"),
|
||||
VoiceProvider: stringEnv("VOICE_PROVIDER", "mock"),
|
||||
VolcRTCBaseURL: stringEnv("VOLC_RTC_BASE_URL", "https://rtc.volcengineapi.com"),
|
||||
VolcAccessKeyID: secretValues["VOLC_ACCESS_KEY_ID"],
|
||||
VolcSecretAccessKey: secretValues["VOLC_SECRET_ACCESS_KEY"],
|
||||
VolcSessionToken: secretValues["VOLC_SESSION_TOKEN"],
|
||||
VolcRTCRegion: stringEnv("VOLC_RTC_REGION", "cn-north-1"),
|
||||
VolcRTCAPIVersion: stringEnv("VOLC_RTC_API_VERSION", "2024-12-01"),
|
||||
VolcVoiceChatTemplate: strings.TrimSpace(os.Getenv("VOLC_VOICE_CHAT_TEMPLATE_FILE")),
|
||||
ProviderRequestTimeout: providerRequestTimeout,
|
||||
ProviderCircuitCooldown: providerCircuitCooldown,
|
||||
ObjectStoreEndpoint: objectStoreEndpoint,
|
||||
ObjectStorePublic: objectStorePublic,
|
||||
ObjectStoreAccessKey: objectStoreAccessKey,
|
||||
ObjectStoreSecretKey: objectStoreSecretKey,
|
||||
ObjectStoreBucket: objectStoreBucket,
|
||||
ObjectStoreRegion: objectStoreRegion,
|
||||
ObjectStoreSecure: objectStoreSecure,
|
||||
ObjectStorePublicSecure: objectStorePublicSecure,
|
||||
ExportDownloadTTL: exportDownloadTTL,
|
||||
ExportObjectTTL: exportObjectTTL,
|
||||
MediaLocalDeleteGrace: mediaLocalDeleteGrace,
|
||||
AlertWebhookURL: strings.TrimSpace(os.Getenv("ALERT_WEBHOOK_URL")),
|
||||
AlertWebhookSecret: secretValues["ALERT_WEBHOOK_SECRET"],
|
||||
AlertWebhookTimeout: alertWebhookTimeout,
|
||||
ContentGenerator: stringEnv("CONTENT_GENERATOR", "local"),
|
||||
ArkBaseURL: stringEnv("ARK_BASE_URL", "https://ark.cn-beijing.volces.com/api/v3"),
|
||||
ArkAPIKey: secretValues["ARK_API_KEY"],
|
||||
ArkContentModel: strings.TrimSpace(os.Getenv("ARK_CONTENT_MODEL")),
|
||||
ArkRequestTimeout: arkRequestTimeout,
|
||||
ContentSafetyProvider: stringEnv("CONTENT_SAFETY_PROVIDER", "local"),
|
||||
ContentSafetyEndpoint: strings.TrimSpace(os.Getenv("CONTENT_SAFETY_ENDPOINT")),
|
||||
ContentSafetyAPIKey: secretValues["CONTENT_SAFETY_API_KEY"],
|
||||
ContentSafetyPolicyRef: strings.TrimSpace(os.Getenv("CONTENT_SAFETY_POLICY_REF")),
|
||||
ContentSafetyTimeout: contentSafetyTimeout,
|
||||
EmbeddingProvider: stringEnv("EMBEDDING_PROVIDER", "local"),
|
||||
ArkEmbeddingModel: strings.TrimSpace(os.Getenv("ARK_EMBEDDING_MODEL")),
|
||||
ServiceName: serviceName,
|
||||
Environment: environment,
|
||||
LogLevel: stringEnv("LOG_LEVEL", "info"),
|
||||
HTTPAddr: stringEnv("HTTP_ADDR", defaultHTTPAddr(serviceName)),
|
||||
DatabaseURL: secretDefault(secretValues["DATABASE_URL"], "postgres://voice:voice_dev_only@localhost:55432/voice_platform?sslmode=disable"),
|
||||
RedisAddr: stringEnv("REDIS_ADDR", "localhost:56379"),
|
||||
RedisPassword: secretValues["REDIS_PASSWORD"],
|
||||
RedisDB: redisDB,
|
||||
AuthIssuerURL: stringEnv("AUTH_ISSUER_URL", "http://localhost:58180/realms/voice-local"),
|
||||
AuthJWKSURL: stringEnv("AUTH_JWKS_URL", "http://localhost:58180/realms/voice-local/protocol/openid-connect/certs"),
|
||||
AuthAudience: stringEnv("AUTH_AUDIENCE", "voice-platform-web"),
|
||||
OTLPEndpoint: stringEnvAllowEmpty("OTEL_EXPORTER_OTLP_ENDPOINT", "localhost:54317"),
|
||||
DependencyTimeout: dependencyTimeout,
|
||||
ExternalProbeInterval: externalProbeInterval,
|
||||
ShutdownTimeout: shutdownTimeout,
|
||||
WorkerPollInterval: pollInterval,
|
||||
RecoveryScanInterval: recoveryScanInterval,
|
||||
RecoveryStartingAfter: recoveryStartingAfter,
|
||||
RecoveryStoppingAfter: recoveryStoppingAfter,
|
||||
DeviceLocalSecret: deviceLocalSecret,
|
||||
DeviceCredentialMasterKey: deviceCredentialMasterKey,
|
||||
DeviceClockSkew: deviceClockSkew,
|
||||
RTCAppID: stringEnv("RTC_APP_ID", "local-public-rtc-app"),
|
||||
RTCAppKey: secretValues["RTC_APP_KEY"],
|
||||
RTCTokenIssuer: stringEnv("RTC_TOKEN_ISSUER", "local"),
|
||||
RTCTokenSecret: rtcTokenSecret,
|
||||
CursorSigningSecret: cursorSigningSecret,
|
||||
ShareTokenSecret: shareTokenSecret,
|
||||
RTCTokenTTL: rtcTokenTTL,
|
||||
AllocationTTL: allocationTTL,
|
||||
ProviderCallbackSecret: providerCallbackSecret,
|
||||
ProviderCallbackSkew: providerCallbackSkew,
|
||||
PlatformHealthSecret: secretValues["PLATFORM_HEALTH_CALLBACK_SECRET"],
|
||||
PlatformHealthSkew: platformHealthSkew,
|
||||
MockProviderBaseURL: stringEnv("MOCK_PROVIDER_BASE_URL", "http://localhost:59200"),
|
||||
MockProviderAPIKey: secretDefault(secretValues["MOCK_PROVIDER_API_KEY"], "local-mock-provider-key"),
|
||||
VoiceProvider: stringEnv("VOICE_PROVIDER", "mock"),
|
||||
VolcRTCBaseURL: stringEnv("VOLC_RTC_BASE_URL", "https://rtc.volcengineapi.com"),
|
||||
VolcAccessKeyID: secretValues["VOLC_ACCESS_KEY_ID"],
|
||||
VolcSecretAccessKey: secretValues["VOLC_SECRET_ACCESS_KEY"],
|
||||
VolcSessionToken: secretValues["VOLC_SESSION_TOKEN"],
|
||||
VolcRTCRegion: stringEnv("VOLC_RTC_REGION", "cn-north-1"),
|
||||
VolcRTCAPIVersion: stringEnv("VOLC_RTC_API_VERSION", "2024-12-01"),
|
||||
VolcVoiceChatTemplate: strings.TrimSpace(os.Getenv("VOLC_VOICE_CHAT_TEMPLATE_FILE")),
|
||||
ProviderRequestTimeout: providerRequestTimeout,
|
||||
ProviderCircuitCooldown: providerCircuitCooldown,
|
||||
ObjectStoreEndpoint: objectStoreEndpoint,
|
||||
ObjectStorePublic: objectStorePublic,
|
||||
ObjectStoreAccessKey: objectStoreAccessKey,
|
||||
ObjectStoreSecretKey: objectStoreSecretKey,
|
||||
ObjectStoreBucket: objectStoreBucket,
|
||||
ObjectStoreRegion: objectStoreRegion,
|
||||
ObjectStoreSecure: objectStoreSecure,
|
||||
ObjectStorePublicSecure: objectStorePublicSecure,
|
||||
ExportDownloadTTL: exportDownloadTTL,
|
||||
ExportObjectTTL: exportObjectTTL,
|
||||
MediaLocalDeleteGrace: mediaLocalDeleteGrace,
|
||||
AlertWebhookURL: strings.TrimSpace(os.Getenv("ALERT_WEBHOOK_URL")),
|
||||
AlertWebhookSecret: secretValues["ALERT_WEBHOOK_SECRET"],
|
||||
AlertWebhookTimeout: alertWebhookTimeout,
|
||||
ContentGenerator: stringEnv("CONTENT_GENERATOR", "local"),
|
||||
ArkBaseURL: stringEnv("ARK_BASE_URL", "https://ark.cn-beijing.volces.com/api/v3"),
|
||||
ArkAPIKey: secretValues["ARK_API_KEY"],
|
||||
ArkContentModel: strings.TrimSpace(os.Getenv("ARK_CONTENT_MODEL")),
|
||||
ArkRequestTimeout: arkRequestTimeout,
|
||||
ContentSafetyProvider: stringEnv("CONTENT_SAFETY_PROVIDER", "local"),
|
||||
ContentSafetyEndpoint: strings.TrimSpace(os.Getenv("CONTENT_SAFETY_ENDPOINT")),
|
||||
ContentSafetyAPIKey: secretValues["CONTENT_SAFETY_API_KEY"],
|
||||
ContentSafetyPolicyRef: strings.TrimSpace(os.Getenv("CONTENT_SAFETY_POLICY_REF")),
|
||||
ContentSafetyTimeout: contentSafetyTimeout,
|
||||
EmbeddingProvider: stringEnv("EMBEDDING_PROVIDER", "local"),
|
||||
ArkEmbeddingModel: strings.TrimSpace(os.Getenv("ARK_EMBEDDING_MODEL")),
|
||||
MemoryToolBaseURL: strings.TrimSpace(os.Getenv("MEMORY_TOOL_BASE_URL")),
|
||||
MemoryToolSigningSecret: memoryToolSigningSecret,
|
||||
MemoryToolTokenTTL: memoryToolTokenTTL,
|
||||
ProviderMemoryMode: stringEnv("PROVIDER_MEMORY_MODE", "disabled"),
|
||||
ProviderMemoryBaseURL: strings.TrimSpace(os.Getenv("PROVIDER_MEMORY_BASE_URL")),
|
||||
ProviderMemoryAPIKey: secretValues["PROVIDER_MEMORY_API_KEY"],
|
||||
ProviderMemoryTimeout: providerMemoryTimeout,
|
||||
}
|
||||
cfg.RateLimitRPS, err = floatEnv("RATE_LIMIT_RPS", 50)
|
||||
if err != nil {
|
||||
@@ -381,6 +414,14 @@ func Load(serviceName string) (Config, error) {
|
||||
if err != nil {
|
||||
return Config{}, err
|
||||
}
|
||||
cfg.MemoryToolRateRPS, err = floatEnv("MEMORY_TOOL_RATE_RPS", 2)
|
||||
if err != nil {
|
||||
return Config{}, err
|
||||
}
|
||||
cfg.MemoryToolRateBurst, err = intEnv("MEMORY_TOOL_RATE_BURST", 5)
|
||||
if err != nil {
|
||||
return Config{}, err
|
||||
}
|
||||
cfg.HeartbeatInterval, err = intEnv("HEARTBEAT_INTERVAL_SECONDS", 15)
|
||||
if err != nil {
|
||||
return Config{}, err
|
||||
@@ -427,14 +468,15 @@ func (c Config) Validate() error {
|
||||
c.RateLimitUserRPS <= 0 || c.RateLimitUserBurst <= 0 ||
|
||||
c.RateLimitTenantRPS <= 0 || c.RateLimitTenantBurst <= 0 ||
|
||||
c.RateLimitDeviceRPS <= 0 || c.RateLimitDeviceBurst <= 0 ||
|
||||
c.ProviderStartRPS <= 0 || c.ProviderStartBurst <= 0 || c.ProviderMaxActiveTasks <= 0 {
|
||||
c.ProviderStartRPS <= 0 || c.ProviderStartBurst <= 0 || c.ProviderMaxActiveTasks <= 0 ||
|
||||
c.MemoryToolRateRPS <= 0 || c.MemoryToolRateBurst <= 0 {
|
||||
return errors.New("rate limits, bursts and provider task capacity must be positive")
|
||||
}
|
||||
if c.ServiceName == "api" && c.DeviceSecretDirectory == "" && len(c.DeviceLocalSecret) < 32 {
|
||||
return errors.New("DEVICE_LOCAL_SECRET must contain at least 32 characters when DEVICE_SECRET_DIRECTORY is not configured")
|
||||
}
|
||||
if c.ServiceName == "api" && c.DeviceSecretDirectory != "" && !filepath.IsAbs(c.DeviceSecretDirectory) {
|
||||
return errors.New("DEVICE_SECRET_DIRECTORY must be an absolute path")
|
||||
if c.ServiceName == "api" {
|
||||
decodedMasterKey, err := hex.DecodeString(c.DeviceCredentialMasterKey)
|
||||
if err != nil || len(decodedMasterKey) != 32 {
|
||||
return errors.New("DEVICE_CREDENTIAL_MASTER_KEY must be 32 bytes encoded as 64 hexadecimal characters")
|
||||
}
|
||||
}
|
||||
if c.DeviceClockSkew <= 0 {
|
||||
return errors.New("DEVICE_CLOCK_SKEW must be positive")
|
||||
@@ -459,6 +501,18 @@ func (c Config) Validate() error {
|
||||
if c.ServiceName == "api" && len(c.ShareTokenSecret) < 32 {
|
||||
return errors.New("SHARE_TOKEN_SIGNING_SECRET must contain at least 32 characters")
|
||||
}
|
||||
if (c.ServiceName == "api" || c.ServiceName == "worker") && (len(c.MemoryToolSigningSecret) < 32 || c.MemoryToolTokenTTL <= 0) {
|
||||
return errors.New("MEMORY_TOOL_SIGNING_SECRET must contain at least 32 characters and token TTL must be positive")
|
||||
}
|
||||
if c.MemoryToolBaseURL != "" && !isAbsoluteHTTPURL(c.MemoryToolBaseURL) {
|
||||
return errors.New("MEMORY_TOOL_BASE_URL must be an absolute HTTP(S) URL")
|
||||
}
|
||||
if c.ProviderMemoryMode != "disabled" && c.ProviderMemoryMode != "managed_http" {
|
||||
return errors.New("PROVIDER_MEMORY_MODE must be disabled or managed_http")
|
||||
}
|
||||
if (c.ServiceName == "api" || c.ServiceName == "worker") && c.ProviderMemoryMode == "managed_http" && (!isAbsoluteHTTPURL(c.ProviderMemoryBaseURL) || c.ProviderMemoryAPIKey == "" || c.ProviderMemoryTimeout <= 0) {
|
||||
return errors.New("managed provider memory requires an HTTP(S) base URL, API key and positive timeout")
|
||||
}
|
||||
if c.RTCAppID == "" || c.RTCTokenTTL <= 0 || c.AllocationTTL <= 0 || c.HeartbeatInterval <= 0 {
|
||||
return errors.New("RTC app, token TTL, allocation TTL and heartbeat interval must be valid")
|
||||
}
|
||||
@@ -570,8 +624,8 @@ func (c Config) validateProduction() error {
|
||||
}
|
||||
|
||||
if c.ServiceName == "api" {
|
||||
if c.DeviceSecretDirectory == "" {
|
||||
return errors.New("production API must use DEVICE_SECRET_DIRECTORY for per-device credentials")
|
||||
if c.DeviceCredentialMasterKey == "000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f" {
|
||||
return errors.New("production API must use a non-default DEVICE_CREDENTIAL_MASTER_KEY")
|
||||
}
|
||||
if isPlaceholderSecret(c.CursorSigningSecret) || isPlaceholderSecret(c.ShareTokenSecret) {
|
||||
return errors.New("production API signing secrets must not use local or example values")
|
||||
@@ -579,6 +633,12 @@ func (c Config) validateProduction() error {
|
||||
if c.CursorSigningSecret == c.ShareTokenSecret {
|
||||
return errors.New("production API signing secrets must be distinct")
|
||||
}
|
||||
if isPlaceholderSecret(c.MemoryToolSigningSecret) || c.MemoryToolSigningSecret == c.CursorSigningSecret || c.MemoryToolSigningSecret == c.ShareTokenSecret {
|
||||
return errors.New("production memory tool signing secret must be non-placeholder and distinct")
|
||||
}
|
||||
if c.MemoryToolBaseURL != "" && !isHTTPSURL(c.MemoryToolBaseURL) {
|
||||
return errors.New("production memory tool URL must use HTTPS")
|
||||
}
|
||||
if isLocalValue(c.RTCAppID) || c.RTCTokenIssuer != "volcengine" || isPlaceholderSecret(c.RTCAppKey) {
|
||||
return errors.New("production RTC credentials must use a real app ID, Volcengine token issuer and non-placeholder app key")
|
||||
}
|
||||
@@ -610,6 +670,12 @@ func (c Config) validateProduction() error {
|
||||
}
|
||||
}
|
||||
if c.ServiceName == "api" || c.ServiceName == "worker" {
|
||||
if isPlaceholderSecret(c.MemoryToolSigningSecret) {
|
||||
return errors.New("production memory tool signing secret must be non-placeholder")
|
||||
}
|
||||
if c.ProviderMemoryMode == "managed_http" && (!isHTTPSURL(c.ProviderMemoryBaseURL) || isLocalValue(c.ProviderMemoryBaseURL) || isPlaceholderSecret(c.ProviderMemoryAPIKey)) {
|
||||
return errors.New("production provider memory must use managed HTTPS and a non-placeholder API key")
|
||||
}
|
||||
if !c.ObjectStoreSecure || !c.ObjectStorePublicSecure {
|
||||
return errors.New("production object storage must use TLS")
|
||||
}
|
||||
|
||||
@@ -46,6 +46,32 @@ func TestLoadRejectsInvalidDuration(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoadValidatesProviderMemoryConfiguration(t *testing.T) {
|
||||
t.Setenv("PROVIDER_MEMORY_MODE", "managed_http")
|
||||
t.Setenv("PROVIDER_MEMORY_BASE_URL", "https://memory.example.com")
|
||||
t.Setenv("PROVIDER_MEMORY_API_KEY", strings.Repeat("p", 40))
|
||||
cfg, err := Load("worker")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if cfg.ProviderMemoryMode != "managed_http" || cfg.ProviderMemoryTimeout <= 0 {
|
||||
t.Fatalf("provider memory config=%+v", cfg)
|
||||
}
|
||||
t.Setenv("PROVIDER_MEMORY_BASE_URL", "")
|
||||
if _, err := Load("worker"); err == nil {
|
||||
t.Fatal("managed provider memory should require an endpoint")
|
||||
}
|
||||
|
||||
// The mode is distributed through the shared ConfigMap. Services that never
|
||||
// create a provider-memory client must not require the API/Worker-only key.
|
||||
t.Setenv("PROVIDER_MEMORY_API_KEY", "")
|
||||
t.Setenv("PROVIDER_CALLBACK_SECRET", strings.Repeat("c", 32))
|
||||
t.Setenv("PLATFORM_HEALTH_CALLBACK_SECRET", strings.Repeat("h", 32))
|
||||
if _, err := Load("callbacks"); err != nil {
|
||||
t.Fatalf("callbacks should ignore provider-memory credentials: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLoadReadsSecretsFromAbsoluteFiles(t *testing.T) {
|
||||
secret := strings.Repeat("k", 40)
|
||||
path := filepath.Join(t.TempDir(), "ark-api-key")
|
||||
@@ -118,12 +144,14 @@ func TestProductionAPIAcceptsHardenedConfiguration(t *testing.T) {
|
||||
cfg.AuthIssuerURL = "https://identity.example.com/realms/voice"
|
||||
cfg.AuthJWKSURL = "https://identity.example.com/realms/voice/certs"
|
||||
cfg.DeviceLocalSecret = strings.Repeat("d", 40)
|
||||
cfg.DeviceSecretDirectory = t.TempDir()
|
||||
cfg.DeviceCredentialMasterKey = strings.Repeat("ab", 32)
|
||||
cfg.RTCTokenSecret = strings.Repeat("t", 40)
|
||||
cfg.RTCTokenIssuer = "volcengine"
|
||||
cfg.RTCAppKey = strings.Repeat("v", 40)
|
||||
cfg.CursorSigningSecret = strings.Repeat("c", 40)
|
||||
cfg.ShareTokenSecret = strings.Repeat("h", 40)
|
||||
cfg.MemoryToolSigningSecret = strings.Repeat("m", 40)
|
||||
cfg.MemoryToolBaseURL = "https://api.example.com/internal/v1/tools/memory-search"
|
||||
cfg.RTCAppID = "661e7ae5543cf00000000000"
|
||||
cfg.ObjectStoreEndpoint = "objects.internal.example.com:9000"
|
||||
cfg.ObjectStorePublic = "objects.example.com"
|
||||
@@ -151,7 +179,7 @@ func TestProductionRejectsReusedSigningSecrets(t *testing.T) {
|
||||
cfg.AuthIssuerURL = "https://identity.example.com/realms/voice"
|
||||
cfg.AuthJWKSURL = "https://identity.example.com/realms/voice/certs"
|
||||
cfg.DeviceLocalSecret = strings.Repeat("x", 40)
|
||||
cfg.DeviceSecretDirectory = t.TempDir()
|
||||
cfg.DeviceCredentialMasterKey = strings.Repeat("cd", 32)
|
||||
cfg.RTCTokenSecret = strings.Repeat("x", 40)
|
||||
cfg.RTCTokenIssuer = "volcengine"
|
||||
cfg.RTCAppKey = strings.Repeat("v", 40)
|
||||
@@ -197,6 +225,7 @@ func TestProductionWorkerRequiresRealArkContentGenerator(t *testing.T) {
|
||||
cfg.ObjectStoreSecretKey = strings.Repeat("s", 40)
|
||||
cfg.ObjectStoreSecure = true
|
||||
cfg.ObjectStorePublicSecure = true
|
||||
cfg.MemoryToolSigningSecret = strings.Repeat("m", 40)
|
||||
if err = cfg.Validate(); err == nil || !strings.Contains(err.Error(), "content generation") {
|
||||
t.Fatalf("Validate() error=%v, want local content generator production gate", err)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,178 @@
|
||||
package deviceauth
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/aes"
|
||||
"crypto/cipher"
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"strings"
|
||||
|
||||
"github.com/jackc/pgx/v5"
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
)
|
||||
|
||||
const databaseCredentialKeyBytes = 32
|
||||
|
||||
var ErrCredentialAlreadyExists = errors.New("active device credential already exists")
|
||||
|
||||
type DatabaseSecretResolver struct {
|
||||
pool *pgxpool.Pool
|
||||
aead cipher.AEAD
|
||||
}
|
||||
|
||||
func NewDatabaseSecretResolver(pool *pgxpool.Pool, masterKeyHex string) (*DatabaseSecretResolver, error) {
|
||||
if pool == nil {
|
||||
return nil, errors.New("device credential database is required")
|
||||
}
|
||||
key, err := hex.DecodeString(strings.TrimSpace(masterKeyHex))
|
||||
if err != nil || len(key) != databaseCredentialKeyBytes {
|
||||
return nil, errors.New("device credential master key must be 32 bytes encoded as 64 hexadecimal characters")
|
||||
}
|
||||
block, err := aes.NewCipher(key)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("create device credential cipher: %w", err)
|
||||
}
|
||||
aead, err := cipher.NewGCM(block)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("create device credential AEAD: %w", err)
|
||||
}
|
||||
return &DatabaseSecretResolver{pool: pool, aead: aead}, nil
|
||||
}
|
||||
|
||||
func (r *DatabaseSecretResolver) Resolve(ctx context.Context, reference string) ([]byte, error) {
|
||||
var encrypted, nonce []byte
|
||||
var version int64
|
||||
err := r.pool.QueryRow(ctx, `
|
||||
SELECT encrypted_secret,nonce,key_version
|
||||
FROM device_credential
|
||||
WHERE credential_ref=$1 AND status='active'`, strings.TrimSpace(reference),
|
||||
).Scan(&encrypted, &nonce, &version)
|
||||
if errors.Is(err, pgx.ErrNoRows) {
|
||||
return nil, ErrCredentialInvalid
|
||||
}
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read device credential: %w", err)
|
||||
}
|
||||
secret, err := r.aead.Open(nil, nonce, encrypted, credentialAAD(reference, version))
|
||||
if err != nil || len(secret) < 32 {
|
||||
return nil, ErrCredentialInvalid
|
||||
}
|
||||
return secret, nil
|
||||
}
|
||||
|
||||
// ImportMissingLegacyCredentials moves existing deployment credentials into
|
||||
// the database without replacing an already imported or rotated credential.
|
||||
func (r *DatabaseSecretResolver) ImportMissingLegacyCredentials(ctx context.Context, credentials map[string][]byte) error {
|
||||
for reference, secret := range credentials {
|
||||
if len(secret) < 32 {
|
||||
return ErrCredentialInvalid
|
||||
}
|
||||
var deviceID, tenantID string
|
||||
err := r.pool.QueryRow(ctx, `SELECT id::text,tenant_id::text FROM device WHERE credential_ref=$1`, reference).Scan(&deviceID, &tenantID)
|
||||
if errors.Is(err, pgx.ErrNoRows) {
|
||||
continue
|
||||
}
|
||||
if err != nil {
|
||||
return fmt.Errorf("resolve legacy device credential %s: %w", reference, err)
|
||||
}
|
||||
nonce := make([]byte, r.aead.NonceSize())
|
||||
if _, err := io.ReadFull(rand.Reader, nonce); err != nil {
|
||||
return fmt.Errorf("generate device credential nonce: %w", err)
|
||||
}
|
||||
const version int64 = 1
|
||||
encrypted := r.aead.Seal(nil, nonce, secret, credentialAAD(reference, version))
|
||||
if _, err := r.pool.Exec(ctx, `
|
||||
INSERT INTO device_credential(device_id,tenant_id,credential_ref,key_version,encrypted_secret,nonce)
|
||||
VALUES($1::uuid,$2::uuid,$3,$4,$5,$6)
|
||||
ON CONFLICT (device_id) DO NOTHING`, deviceID, tenantID, reference, version, encrypted, nonce); err != nil {
|
||||
return fmt.Errorf("import legacy device credential %s: %w", reference, err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (r *DatabaseSecretResolver) Provision(ctx context.Context, devicePublicID string, secret []byte, rotate bool) (int64, error) {
|
||||
if len(secret) < 32 {
|
||||
return 0, ErrCredentialInvalid
|
||||
}
|
||||
tx, err := r.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("begin device credential provisioning: %w", err)
|
||||
}
|
||||
defer tx.Rollback(ctx)
|
||||
|
||||
version, err := r.ProvisionTx(ctx, tx, devicePublicID, secret, rotate)
|
||||
if err != nil {
|
||||
return 0, err
|
||||
}
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return 0, fmt.Errorf("commit device credential provisioning: %w", err)
|
||||
}
|
||||
return version, nil
|
||||
}
|
||||
|
||||
// ProvisionTx stores a credential inside a caller-owned transaction so device
|
||||
// registration and first credential issuance can commit atomically.
|
||||
func (r *DatabaseSecretResolver) ProvisionTx(ctx context.Context, tx pgx.Tx, devicePublicID string, secret []byte, rotate bool) (int64, error) {
|
||||
if len(secret) < 32 {
|
||||
return 0, ErrCredentialInvalid
|
||||
}
|
||||
var deviceID, tenantID, reference string
|
||||
if err := tx.QueryRow(ctx, `
|
||||
SELECT id::text,tenant_id::text,credential_ref FROM device
|
||||
WHERE public_id=$1::uuid AND status <> 'revoked' FOR UPDATE`, strings.TrimSpace(devicePublicID),
|
||||
).Scan(&deviceID, &tenantID, &reference); errors.Is(err, pgx.ErrNoRows) {
|
||||
return 0, ErrDeviceNotFound
|
||||
} else if err != nil {
|
||||
return 0, fmt.Errorf("resolve device for credential provisioning: %w", err)
|
||||
}
|
||||
var currentVersion int64
|
||||
err := tx.QueryRow(ctx, `SELECT key_version FROM device_credential WHERE device_id=$1::uuid FOR UPDATE`, deviceID).Scan(¤tVersion)
|
||||
if err != nil && !errors.Is(err, pgx.ErrNoRows) {
|
||||
return 0, fmt.Errorf("read current device credential: %w", err)
|
||||
}
|
||||
if err == nil && !rotate {
|
||||
return 0, ErrCredentialAlreadyExists
|
||||
}
|
||||
version := currentVersion + 1
|
||||
nonce := make([]byte, r.aead.NonceSize())
|
||||
if _, err := io.ReadFull(rand.Reader, nonce); err != nil {
|
||||
return 0, fmt.Errorf("generate device credential nonce: %w", err)
|
||||
}
|
||||
encrypted := r.aead.Seal(nil, nonce, secret, credentialAAD(reference, version))
|
||||
if currentVersion == 0 {
|
||||
_, err = tx.Exec(ctx, `
|
||||
INSERT INTO device_credential(device_id,tenant_id,credential_ref,key_version,encrypted_secret,nonce)
|
||||
VALUES($1::uuid,$2::uuid,$3,$4,$5,$6)`, deviceID, tenantID, reference, version, encrypted, nonce)
|
||||
} else {
|
||||
_, err = tx.Exec(ctx, `
|
||||
UPDATE device_credential SET key_version=$2,encrypted_secret=$3,nonce=$4,status='active',
|
||||
rotated_at=now(),revoked_at=NULL
|
||||
WHERE device_id=$1::uuid`, deviceID, version, encrypted, nonce)
|
||||
}
|
||||
if err != nil {
|
||||
return 0, fmt.Errorf("store device credential: %w", err)
|
||||
}
|
||||
return version, nil
|
||||
}
|
||||
|
||||
func (r *DatabaseSecretResolver) Revoke(ctx context.Context, devicePublicID string) error {
|
||||
result, err := r.pool.Exec(ctx, `
|
||||
UPDATE device_credential dc SET status='revoked',revoked_at=now()
|
||||
FROM device d WHERE d.id=dc.device_id AND d.public_id=$1::uuid AND dc.status='active'`, strings.TrimSpace(devicePublicID))
|
||||
if err != nil {
|
||||
return fmt.Errorf("revoke device credential: %w", err)
|
||||
}
|
||||
if result.RowsAffected() != 1 {
|
||||
return ErrDeviceNotFound
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func credentialAAD(reference string, version int64) []byte {
|
||||
return []byte(fmt.Sprintf("device-credential-v1\n%s\n%d", strings.TrimSpace(reference), version))
|
||||
}
|
||||
@@ -0,0 +1,89 @@
|
||||
package deviceauth
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"errors"
|
||||
"os"
|
||||
"testing"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
)
|
||||
|
||||
func TestDatabaseSecretResolverImportsEncryptedCredential(t *testing.T) {
|
||||
databaseURL := os.Getenv("TEST_DATABASE_URL")
|
||||
if databaseURL == "" {
|
||||
t.Skip("TEST_DATABASE_URL is not set")
|
||||
}
|
||||
ctx := context.Background()
|
||||
pool, err := pgxpool.New(ctx, databaseURL)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer pool.Close()
|
||||
|
||||
deviceID := uuid.NewString()
|
||||
publicID := uuid.NewString()
|
||||
reference := "database-test://" + publicID
|
||||
serial := sha256.Sum256([]byte(publicID))
|
||||
_, err = pool.Exec(ctx, `
|
||||
INSERT INTO device(id,public_id,tenant_id,serial_digest,model,status,credential_ref)
|
||||
VALUES($1::uuid,$2::uuid,'10000000-0000-0000-0000-000000000001'::uuid,$3,'database-credential-test','registered',$4)`,
|
||||
deviceID, publicID, hex.EncodeToString(serial[:]), reference)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer func() {
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM device_credential WHERE device_id=$1::uuid`, deviceID)
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM device WHERE id=$1::uuid`, deviceID)
|
||||
}()
|
||||
|
||||
const masterKey = "101112131415161718191a1b1c1d1e1f202122232425262728292a2b2c2d2e2f"
|
||||
secret := []byte("database-device-secret-at-least-32-bytes")
|
||||
resolver, err := NewDatabaseSecretResolver(pool, masterKey)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := resolver.ImportMissingLegacyCredentials(ctx, map[string][]byte{reference: secret}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
resolved, err := resolver.Resolve(ctx, reference)
|
||||
if err != nil || !bytes.Equal(resolved, secret) {
|
||||
t.Fatalf("resolved credential does not match: len=%d err=%v", len(resolved), err)
|
||||
}
|
||||
var encrypted []byte
|
||||
if err := pool.QueryRow(ctx, `SELECT encrypted_secret FROM device_credential WHERE device_id=$1::uuid`, deviceID).Scan(&encrypted); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if bytes.Contains(encrypted, secret) || bytes.Equal(encrypted, secret) {
|
||||
t.Fatal("database stored plaintext device credential")
|
||||
}
|
||||
wrongResolver, err := NewDatabaseSecretResolver(pool, "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := wrongResolver.Resolve(ctx, reference); !errors.Is(err, ErrCredentialInvalid) {
|
||||
t.Fatalf("wrong master key error=%v", err)
|
||||
}
|
||||
if _, err := resolver.Provision(ctx, publicID, []byte("replacement-device-secret-at-least-32-bytes"), false); !errors.Is(err, ErrCredentialAlreadyExists) {
|
||||
t.Fatalf("duplicate provision error=%v", err)
|
||||
}
|
||||
replacement := []byte("replacement-device-secret-at-least-32-bytes")
|
||||
version, err := resolver.Provision(ctx, publicID, replacement, true)
|
||||
if err != nil || version != 2 {
|
||||
t.Fatalf("rotate version=%d err=%v", version, err)
|
||||
}
|
||||
resolved, err = resolver.Resolve(ctx, reference)
|
||||
if err != nil || !bytes.Equal(resolved, replacement) {
|
||||
t.Fatalf("rotated credential does not match: len=%d err=%v", len(resolved), err)
|
||||
}
|
||||
if err := resolver.Revoke(ctx, publicID); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := resolver.Resolve(ctx, reference); !errors.Is(err, ErrCredentialInvalid) {
|
||||
t.Fatalf("revoked credential error=%v", err)
|
||||
}
|
||||
}
|
||||
@@ -29,7 +29,7 @@ func TestAcceptanceSnapshotIsAuditedIdempotentAndTenantScoped(t *testing.T) {
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
admin := identity.Principal{Issuer: "http://localhost:58180/realms/voice-local", Subject: "00000000-0000-0000-0000-000000000101"}
|
||||
admin := identity.Principal{Issuer: "http://localhost:58180/realms/voice-local", Subject: "00000000-0000-0000-0000-000000000104"}
|
||||
tenantPublicID := "10000000-0000-0000-0000-000000000010"
|
||||
key := "acceptance-snapshot-" + uuid.NewString()
|
||||
requestID := "acceptance-snapshot-test-" + uuid.NewString()
|
||||
|
||||
@@ -31,7 +31,7 @@ func TestIntegrationIssueLifecycleIsTenantScopedIdempotentAndVersioned(t *testin
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
admin := identity.Principal{Issuer: "http://localhost:58180/realms/voice-local", Subject: "00000000-0000-0000-0000-000000000101"}
|
||||
admin := identity.Principal{Issuer: "http://localhost:58180/realms/voice-local", Subject: "00000000-0000-0000-0000-000000000104"}
|
||||
tenantPublicID := "10000000-0000-0000-0000-000000000010"
|
||||
devicePublicID := "10000000-0000-0000-0000-000000000511"
|
||||
key := "integration-issue-" + uuid.NewString()
|
||||
|
||||
@@ -156,7 +156,7 @@ func (s *Service) resolveActor(ctx context.Context, principal identity.Principal
|
||||
}
|
||||
var result actor
|
||||
var role string
|
||||
err := s.pool.QueryRow(ctx, `SELECT t.id::text,ua.id::text,ua.public_id::text,ua.platform_role FROM tenant t JOIN user_account ua ON ua.tenant_id=t.id AND ua.status='active' WHERE t.public_id=$1::uuid AND ua.oidc_issuer=$2 AND ua.oidc_subject=$3`, tenantPublicID, principal.Issuer, principal.Subject).Scan(&result.tenantID, &result.userID, &result.userPublicID, &role)
|
||||
err := s.pool.QueryRow(ctx, `SELECT t.id::text,ua.id::text,ua.public_id::text,ua.platform_role FROM tenant t JOIN user_account ua ON ua.tenant_id=t.id AND ua.status='active' AND ua.account_kind='operator' WHERE t.public_id=$1::uuid AND ua.oidc_issuer=$2 AND ua.oidc_subject=$3`, tenantPublicID, principal.Issuer, principal.Subject).Scan(&result.tenantID, &result.userID, &result.userPublicID, &role)
|
||||
if errors.Is(err, pgx.ErrNoRows) {
|
||||
return actor{}, ErrAccessDenied
|
||||
}
|
||||
|
||||
@@ -31,7 +31,7 @@ func TestDiagnosticSearchAndConfigRefreshAreTenantScopedAndIdempotent(t *testing
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
admin := identity.Principal{Issuer: "http://localhost:58180/realms/voice-local", Subject: "00000000-0000-0000-0000-000000000101"}
|
||||
admin := identity.Principal{Issuer: "http://localhost:58180/realms/voice-local", Subject: "00000000-0000-0000-0000-000000000104"}
|
||||
tenantPublicID := "10000000-0000-0000-0000-000000000010"
|
||||
devicePublicID := "10000000-0000-0000-0000-000000000511"
|
||||
key := "diagnostic-refresh-integration"
|
||||
@@ -79,8 +79,8 @@ func TestDiagnosticConversationActionsCoverStableAndProviderStates(t *testing.T)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
admin := identity.Principal{Issuer: "http://localhost:58180/realms/voice-local", Subject: "00000000-0000-0000-0000-000000000101"}
|
||||
approver := identity.Principal{Issuer: admin.Issuer, Subject: "00000000-0000-0000-0000-000000000102"}
|
||||
admin := identity.Principal{Issuer: "http://localhost:58180/realms/voice-local", Subject: "00000000-0000-0000-0000-000000000104"}
|
||||
approver := identity.Principal{Issuer: admin.Issuer, Subject: "00000000-0000-0000-0000-000000000105"}
|
||||
tenantPublicID := "10000000-0000-0000-0000-000000000010"
|
||||
tenantID := "10000000-0000-0000-0000-000000000001"
|
||||
householdID := "10000000-0000-0000-0000-000000000201"
|
||||
@@ -191,7 +191,7 @@ func TestDiagnosticJobRetryBundleAndDeviceRevocation(t *testing.T) {
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
admin := identity.Principal{Issuer: "http://localhost:58180/realms/voice-local", Subject: "00000000-0000-0000-0000-000000000101"}
|
||||
admin := identity.Principal{Issuer: "http://localhost:58180/realms/voice-local", Subject: "00000000-0000-0000-0000-000000000104"}
|
||||
tenantPublicID, tenantID := "10000000-0000-0000-0000-000000000010", "10000000-0000-0000-0000-000000000001"
|
||||
deviceID, devicePublicID := uuid.NewString(), uuid.NewString()
|
||||
jobInternalID, aggregateID := uuid.NewString(), uuid.NewString()
|
||||
|
||||
@@ -11,6 +11,7 @@ import (
|
||||
"net/url"
|
||||
"strings"
|
||||
"time"
|
||||
"unicode"
|
||||
|
||||
"data-terminal-cloud/internal/contentpolicy"
|
||||
"data-terminal-cloud/internal/outbox"
|
||||
@@ -21,15 +22,16 @@ type GenerationInput struct {
|
||||
}
|
||||
|
||||
type GenerationResult struct {
|
||||
Summary string `json:"summary"`
|
||||
OpenTopic string `json:"open_topic"`
|
||||
StoryTitle string `json:"story_title"`
|
||||
StoryContent string `json:"story_content"`
|
||||
FactContent string `json:"fact_content"`
|
||||
Confidence float64 `json:"confidence"`
|
||||
ModelRef string `json:"-"`
|
||||
PromptVersion string `json:"-"`
|
||||
AllowStory bool `json:"-"`
|
||||
Summary string `json:"summary"`
|
||||
OpenTopic string `json:"open_topic"`
|
||||
StoryTitle string `json:"story_title"`
|
||||
StoryContent string `json:"story_content"`
|
||||
FactContent string `json:"fact_content"`
|
||||
Keywords []string `json:"keywords"`
|
||||
Confidence float64 `json:"confidence"`
|
||||
ModelRef string `json:"-"`
|
||||
PromptVersion string `json:"-"`
|
||||
AllowStory bool `json:"-"`
|
||||
}
|
||||
|
||||
type Generator interface {
|
||||
@@ -49,6 +51,7 @@ func (DeterministicGenerator) Generate(_ context.Context, input GenerationInput)
|
||||
StoryTitle: "关于“" + truncateRunes(content, 24) + "”的回忆",
|
||||
StoryContent: content,
|
||||
FactContent: content,
|
||||
Keywords: deterministicKeywords(content),
|
||||
Confidence: 0.9,
|
||||
ModelRef: "local-deterministic-extractor",
|
||||
PromptVersion: "story-v1",
|
||||
@@ -117,7 +120,7 @@ func (g *ArkGenerator) Generate(ctx context.Context, input GenerationInput) (Gen
|
||||
requestBody := map[string]any{
|
||||
"model": g.model,
|
||||
"messages": []map[string]string{
|
||||
{"role": "system", "content": "你是长者口述内容整理助手。只依据输入字幕生成可人工核对的候选，不补充未出现的事实，不把推断写成事实。必须返回JSON对象，字段为summary、open_topic、story_title、story_content、fact_content、confidence。summary概括本次内容，open_topic给出一个温和的后续话题，story和fact保留原意，confidence取0到1。"},
|
||||
{"role": "system", "content": "你是长者口述内容整理助手。只依据输入字幕生成可人工核对的候选，不补充未出现的事实，不把推断写成事实。必须返回JSON对象，字段为summary、keywords、open_topic、story_title、story_content、fact_content、confidence。summary概括本次内容，keywords是1至8个简短关键词字符串组成的数组，open_topic给出一个温和的后续话题，story和fact保留原意，confidence取0到1。"},
|
||||
{"role": "user", "content": content},
|
||||
},
|
||||
"temperature": 0.2,
|
||||
@@ -174,6 +177,7 @@ func (g *ArkGenerator) Generate(ctx context.Context, input GenerationInput) (Gen
|
||||
result.StoryTitle = strings.TrimSpace(result.StoryTitle)
|
||||
result.StoryContent = strings.TrimSpace(result.StoryContent)
|
||||
result.FactContent = strings.TrimSpace(result.FactContent)
|
||||
result.Keywords = normalizedKeywords(result.Keywords)
|
||||
if !validGeneratedContent(result) {
|
||||
return GenerationResult{}, outbox.Retryable(errors.New("Ark structured content failed validation"))
|
||||
}
|
||||
@@ -189,5 +193,52 @@ func validGeneratedContent(result GenerationResult) bool {
|
||||
result.StoryTitle != "" && len([]rune(result.StoryTitle)) <= 300 &&
|
||||
result.StoryContent != "" && len([]rune(result.StoryContent)) <= 20000 &&
|
||||
result.FactContent != "" && len([]rune(result.FactContent)) <= 5000 &&
|
||||
len(result.Keywords) >= 1 && len(result.Keywords) <= 8 &&
|
||||
result.Confidence >= 0 && result.Confidence <= 1
|
||||
}
|
||||
|
||||
func normalizedKeywords(values []string) []string {
|
||||
result := make([]string, 0, min(len(values), 8))
|
||||
seen := make(map[string]struct{}, len(values))
|
||||
for _, value := range values {
|
||||
value = strings.TrimSpace(value)
|
||||
if value == "" || len([]rune(value)) > 40 {
|
||||
continue
|
||||
}
|
||||
if _, exists := seen[value]; exists {
|
||||
continue
|
||||
}
|
||||
seen[value] = struct{}{}
|
||||
result = append(result, value)
|
||||
if len(result) == 8 {
|
||||
break
|
||||
}
|
||||
}
|
||||
return result
|
||||
}
|
||||
|
||||
func deterministicKeywords(content string) []string {
|
||||
parts := strings.FieldsFunc(content, func(character rune) bool {
|
||||
return unicode.IsSpace(character) || strings.ContainsRune("，。！？；、,.!?;：:\"“”‘’（）()", character)
|
||||
})
|
||||
candidates := make([]string, 0, len(parts))
|
||||
for _, part := range parts {
|
||||
runes := []rune(strings.TrimSpace(part))
|
||||
if len(runes) < 2 {
|
||||
continue
|
||||
}
|
||||
if len(runes) > 12 {
|
||||
runes = runes[:12]
|
||||
}
|
||||
candidates = append(candidates, string(runes))
|
||||
}
|
||||
keywords := normalizedKeywords(candidates)
|
||||
if len(keywords) > 0 {
|
||||
return keywords
|
||||
}
|
||||
runes := []rune(content)
|
||||
if len(runes) > 12 {
|
||||
runes = runes[:12]
|
||||
}
|
||||
return []string{string(runes)}
|
||||
}
|
||||
|
||||
@@ -20,7 +20,7 @@ func TestArkGeneratorProducesValidatedContent(t *testing.T) {
|
||||
if err := json.NewDecoder(r.Body).Decode(&body); err != nil || body["model"] != "doubao-test" {
|
||||
t.Fatalf("request body=%v err=%v", body, err)
|
||||
}
|
||||
_ = json.NewEncoder(w).Encode(map[string]any{"choices": []any{map[string]any{"message": map[string]any{"content": `{"summary":"谈到河边小学","open_topic":"下次可以聊聊同学","story_title":"河边小学的回忆","story_content":"小时候在河边小学读书。","fact_content":"曾在河边小学读书","confidence":0.82}`}}}})
|
||||
_ = json.NewEncoder(w).Encode(map[string]any{"choices": []any{map[string]any{"message": map[string]any{"content": `{"summary":"谈到河边小学","keywords":["河边小学","童年"],"open_topic":"下次可以聊聊同学","story_title":"河边小学的回忆","story_content":"小时候在河边小学读书。","fact_content":"曾在河边小学读书","confidence":0.82}`}}}})
|
||||
}))
|
||||
defer server.Close()
|
||||
generator, err := NewArkGenerator(server.URL+"/api/v3", "test-key", "doubao-test", time.Second)
|
||||
@@ -28,7 +28,7 @@ func TestArkGeneratorProducesValidatedContent(t *testing.T) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
result, err := generator.Generate(context.Background(), GenerationInput{Content: "小时候在河边小学读书。"})
|
||||
if err != nil || result.ModelRef != "ark:doubao-test" || result.Confidence != 0.82 || !result.AllowStory {
|
||||
if err != nil || result.ModelRef != "ark:doubao-test" || result.Confidence != 0.82 || !result.AllowStory || len(result.Keywords) != 2 {
|
||||
t.Fatalf("result=%+v err=%v", result, err)
|
||||
}
|
||||
}
|
||||
@@ -47,3 +47,26 @@ func TestArkGeneratorRetriesTemporaryFailure(t *testing.T) {
|
||||
t.Fatalf("429 should be retryable: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeterministicGeneratorIncludesKeywords(t *testing.T) {
|
||||
result, err := (DeterministicGenerator{}).Generate(context.Background(), GenerationInput{Content: "小时候在河边小学读书，最喜欢和同学一起踢球。"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(result.Keywords) < 1 || len(result.Keywords) > 8 {
|
||||
t.Fatalf("keywords=%v", result.Keywords)
|
||||
}
|
||||
formatted := summaryWithKeywords(result.Summary, result.Keywords)
|
||||
if formatted == result.Summary {
|
||||
t.Fatalf("summary did not include keywords: %q", formatted)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRequestedGenerationVersion(t *testing.T) {
|
||||
if got := requestedGenerationVersion(json.RawMessage(`{"generation_version":7}`)); got != 7 {
|
||||
t.Fatalf("generation version=%d", got)
|
||||
}
|
||||
if got := requestedGenerationVersion(json.RawMessage(`{}`)); got != 1 {
|
||||
t.Fatalf("default generation version=%d", got)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -36,6 +36,7 @@ type extractionSnapshot struct {
|
||||
tenantID, subjectID, status string
|
||||
storyExtractionAllowed bool
|
||||
turns []turn
|
||||
maxSequence int64
|
||||
digest string
|
||||
}
|
||||
|
||||
@@ -84,6 +85,10 @@ func (p *Processor) Handle(ctx context.Context, event outbox.Event) error {
|
||||
if !snapshot.storyExtractionAllowed {
|
||||
return nil
|
||||
}
|
||||
generationVersion := requestedGenerationVersion(event.Payload)
|
||||
if generationVersion < snapshot.maxSequence {
|
||||
return nil
|
||||
}
|
||||
sourceTurns := subjectTurns(snapshot.turns)
|
||||
if len(sourceTurns) == 0 {
|
||||
return outbox.Retryable(ErrConversationContentNotReady)
|
||||
@@ -94,14 +99,6 @@ func (p *Processor) Handle(ctx context.Context, event outbox.Event) error {
|
||||
return err
|
||||
}
|
||||
|
||||
generationVersion := int64(1)
|
||||
var payload struct {
|
||||
GenerationVersion int64 `json:"generation_version"`
|
||||
}
|
||||
if json.Unmarshal(event.Payload, &payload) == nil && payload.GenerationVersion > 0 {
|
||||
generationVersion = payload.GenerationVersion
|
||||
}
|
||||
|
||||
// The model call above is deliberately outside a database transaction. The
|
||||
// conversation row is locked only for the short revalidation and write phase.
|
||||
tx, err := p.pool.BeginTx(ctx, pgx.TxOptions{})
|
||||
@@ -151,7 +148,7 @@ func (p *Processor) generateSafely(ctx context.Context, content string) (Generat
|
||||
}
|
||||
|
||||
func generatedSafetyContent(result GenerationResult) string {
|
||||
return strings.Join([]string{result.Summary, result.OpenTopic, result.StoryTitle, result.StoryContent, result.FactContent}, "\n")
|
||||
return strings.Join([]string{result.Summary, strings.Join(result.Keywords, "、"), result.OpenTopic, result.StoryTitle, result.StoryContent, result.FactContent}, "\n")
|
||||
}
|
||||
|
||||
func loadExtractionSnapshot(ctx context.Context, q queryer, conversationID string, lock bool) (extractionSnapshot, error) {
|
||||
@@ -179,7 +176,7 @@ func loadExtractionSnapshot(ctx context.Context, q queryer, conversationID strin
|
||||
}
|
||||
snapshot.storyExtractionAllowed = consents["story_extraction"]
|
||||
rows, err := q.Query(ctx, `
|
||||
SELECT id::text,public_id::text,speaker,final_text,interrupted
|
||||
SELECT id::text,public_id::text,speaker,final_text,interrupted,sequence_no
|
||||
FROM conversation_turn WHERE conversation_id=$1::uuid ORDER BY sequence_no,id`, conversationID)
|
||||
if err != nil {
|
||||
return extractionSnapshot{}, fmt.Errorf("read extraction turns: %w", err)
|
||||
@@ -187,9 +184,13 @@ func loadExtractionSnapshot(ctx context.Context, q queryer, conversationID strin
|
||||
defer rows.Close()
|
||||
for rows.Next() {
|
||||
var item turn
|
||||
if err = rows.Scan(&item.id, &item.publicID, &item.speaker, &item.text, &item.interrupted); err != nil {
|
||||
var sequence int64
|
||||
if err = rows.Scan(&item.id, &item.publicID, &item.speaker, &item.text, &item.interrupted, &sequence); err != nil {
|
||||
return extractionSnapshot{}, err
|
||||
}
|
||||
if sequence > snapshot.maxSequence {
|
||||
snapshot.maxSequence = sequence
|
||||
}
|
||||
snapshot.turns = append(snapshot.turns, item)
|
||||
}
|
||||
if err = rows.Err(); err != nil {
|
||||
@@ -199,6 +200,16 @@ func loadExtractionSnapshot(ctx context.Context, q queryer, conversationID strin
|
||||
return snapshot, nil
|
||||
}
|
||||
|
||||
func requestedGenerationVersion(payload json.RawMessage) int64 {
|
||||
var value struct {
|
||||
GenerationVersion int64 `json:"generation_version"`
|
||||
}
|
||||
if json.Unmarshal(payload, &value) == nil && value.GenerationVersion > 0 {
|
||||
return value.GenerationVersion
|
||||
}
|
||||
return 1
|
||||
}
|
||||
|
||||
func snapshotDigest(snapshot extractionSnapshot) string {
|
||||
type digestTurn struct {
|
||||
ID string `json:"id"`
|
||||
@@ -250,7 +261,7 @@ func storeGeneration(ctx context.Context, tx pgx.Tx, conversationID string, snap
|
||||
if _, err := tx.Exec(ctx, `
|
||||
INSERT INTO conversation_summary (tenant_id,subject_id,conversation_id,summary_type,content,source_turn_ids,model_ref,prompt_version,generation_version,confidence,status)
|
||||
VALUES ($1::uuid,$2::uuid,$3::uuid,'conversation_summary',$4,$5::jsonb,$6,$7,$8,$9,'READY')
|
||||
ON CONFLICT (conversation_id,summary_type,generation_version) DO NOTHING`, snapshot.tenantID, snapshot.subjectID, conversationID, generated.Summary, sourceJSON, generated.ModelRef, generated.PromptVersion, generationVersion, generated.Confidence); err != nil {
|
||||
ON CONFLICT (conversation_id,summary_type,generation_version) DO NOTHING`, snapshot.tenantID, snapshot.subjectID, conversationID, summaryWithKeywords(generated.Summary, generated.Keywords), sourceJSON, generated.ModelRef, generated.PromptVersion, generationVersion, generated.Confidence); err != nil {
|
||||
return fmt.Errorf("store conversation summary: %w", err)
|
||||
}
|
||||
if _, err := tx.Exec(ctx, `
|
||||
@@ -299,6 +310,14 @@ func storeGeneration(ctx context.Context, tx pgx.Tx, conversationID string, snap
|
||||
return tx.Commit(ctx)
|
||||
}
|
||||
|
||||
func summaryWithKeywords(summary string, keywords []string) string {
|
||||
keywords = normalizedKeywords(keywords)
|
||||
if len(keywords) == 0 {
|
||||
return summary
|
||||
}
|
||||
return summary + "\n关键词：" + strings.Join(keywords, "、")
|
||||
}
|
||||
|
||||
func truncateRunes(value string, limit int) string {
|
||||
characters := []rune(value)
|
||||
if len(characters) <= limit {
|
||||
|
||||
@@ -80,7 +80,7 @@ func TestProcessorExtractionIdempotencyAndSensitiveContent(t *testing.T) {
|
||||
conversationID, "extract-subject-"+conversationID, "我年轻时在河边小学教书。", "extract-agent-"+conversationID, "这是很珍贵的经历。"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
event := outbox.Event{AggregateID: conversationID, Payload: []byte(`{"generation_version":1}`)}
|
||||
event := outbox.Event{AggregateID: conversationID, Payload: []byte(`{"generation_version":2}`)}
|
||||
if err := processor.Handle(ctx, event); !errors.Is(err, ErrConversationContentNotReady) || !outbox.IsRetryable(err) {
|
||||
t.Fatalf("active conversation extraction error=%v", err)
|
||||
}
|
||||
@@ -97,7 +97,7 @@ func TestProcessorExtractionIdempotencyAndSensitiveContent(t *testing.T) {
|
||||
http.Error(w, "database lock", http.StatusServiceUnavailable)
|
||||
return
|
||||
}
|
||||
_, _ = io.WriteString(w, `{"choices":[{"message":{"content":"{\"summary\":\"长者谈到在河边小学教书\",\"open_topic\":\"下次可以聊聊当年的学生\",\"story_title\":\"河边小学的回忆\",\"story_content\":\"我年轻时在河边小学教书。\",\"fact_content\":\"年轻时曾在河边小学教书\",\"confidence\":0.88}"}}]}`)
|
||||
_, _ = io.WriteString(w, `{"choices":[{"message":{"content":"{\"summary\":\"长者谈到在河边小学教书\",\"keywords\":[\"河边小学\",\"教书经历\"],\"open_topic\":\"下次可以聊聊当年的学生\",\"story_title\":\"河边小学的回忆\",\"story_content\":\"我年轻时在河边小学教书。\",\"fact_content\":\"年轻时曾在河边小学教书\",\"confidence\":0.88}"}}]}`)
|
||||
}))
|
||||
defer server.Close()
|
||||
arkGenerator, err := NewArkGenerator(server.URL+"/api/v3", "integration-key", "doubao-integration", 2*time.Second)
|
||||
@@ -133,7 +133,7 @@ func TestProcessorExtractionIdempotencyAndSensitiveContent(t *testing.T) {
|
||||
conversationID, "extract-sensitive-"+conversationID, "我的银行卡号是6222021234567890123，短信验证码是482931。"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
event.Payload = []byte(`{"generation_version":2}`)
|
||||
event.Payload = []byte(`{"generation_version":3}`)
|
||||
if err := modelProcessor.Handle(ctx, event); err != nil {
|
||||
t.Fatalf("extract sensitive conversation: %v", err)
|
||||
}
|
||||
@@ -141,7 +141,7 @@ func TestProcessorExtractionIdempotencyAndSensitiveContent(t *testing.T) {
|
||||
t.Fatalf("model calls=%d want=2; sensitive content must not leave the local safety boundary", modelCalls.Load())
|
||||
}
|
||||
var sensitivePrompt, sensitiveSummary string
|
||||
if err := pool.QueryRow(ctx, `SELECT prompt_version,content FROM conversation_summary WHERE conversation_id=$1::uuid AND summary_type='conversation_summary' AND generation_version=2`, conversationID).Scan(&sensitivePrompt, &sensitiveSummary); err != nil {
|
||||
if err := pool.QueryRow(ctx, `SELECT prompt_version,content FROM conversation_summary WHERE conversation_id=$1::uuid AND summary_type='conversation_summary' AND generation_version=3`, conversationID).Scan(&sensitivePrompt, &sensitiveSummary); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if !strings.HasPrefix(sensitivePrompt, "summary-sensitive-filter-v1:") || strings.Contains(sensitiveSummary, "银行卡") {
|
||||
|
||||
@@ -169,6 +169,72 @@ func GetOpsCostSummaryHandler(service *billing.Service) http.Handler {
|
||||
})
|
||||
}
|
||||
|
||||
func GetFamilyUsageOverviewHandler(service *billing.Service) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
principal, ok := webPrincipal(w, r)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
from, to, timezone, ok := usageOverviewQuery(w, r)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
item, err := service.FamilyUsageOverview(
|
||||
r.Context(), principal, r.PathValue("id"), r.URL.Query().Get("device_id"), timezone, from, to,
|
||||
)
|
||||
if err != nil {
|
||||
writeBillingError(w, r, err)
|
||||
return
|
||||
}
|
||||
w.Header().Set("Cache-Control", "no-store")
|
||||
WriteJSON(w, r, http.StatusOK, item)
|
||||
})
|
||||
}
|
||||
|
||||
func GetOpsUsageOverviewHandler(service *billing.Service) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
principal, ok := webPrincipal(w, r)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
tenantID, ok := opsTenantHeader(w, r)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
from, to, timezone, ok := usageOverviewQuery(w, r)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
item, err := service.OpsUsageOverview(
|
||||
r.Context(), principal, tenantID, r.URL.Query().Get("household_id"), r.URL.Query().Get("device_id"), timezone, from, to,
|
||||
)
|
||||
if err != nil {
|
||||
writeBillingError(w, r, err)
|
||||
return
|
||||
}
|
||||
w.Header().Set("Cache-Control", "no-store")
|
||||
WriteJSON(w, r, http.StatusOK, item)
|
||||
})
|
||||
}
|
||||
|
||||
func usageOverviewQuery(w http.ResponseWriter, r *http.Request) (time.Time, time.Time, string, bool) {
|
||||
from, fromErr := time.Parse(time.RFC3339, r.URL.Query().Get("from"))
|
||||
to, toErr := time.Parse(time.RFC3339, r.URL.Query().Get("to"))
|
||||
timezone := r.URL.Query().Get("timezone")
|
||||
if timezone == "" {
|
||||
timezone = "Asia/Shanghai"
|
||||
}
|
||||
if fromErr != nil || toErr != nil {
|
||||
writeValidation(w, r, "通话用量时间范围不正确", FieldError{Field: "from/to", Message: "必须是RFC3339时间且范围不超过366天"})
|
||||
return time.Time{}, time.Time{}, "", false
|
||||
}
|
||||
if _, err := time.LoadLocation(timezone); err != nil {
|
||||
writeValidation(w, r, "通话用量时区不正确", FieldError{Field: "timezone", Message: "必须是有效的IANA时区"})
|
||||
return time.Time{}, time.Time{}, "", false
|
||||
}
|
||||
return from, to, timezone, true
|
||||
}
|
||||
|
||||
func writeBillingError(w http.ResponseWriter, r *http.Request, err error) {
|
||||
switch {
|
||||
case errors.Is(err, billing.ErrInvalid):
|
||||
|
||||
@@ -0,0 +1,29 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"strings"
|
||||
|
||||
"data-terminal-cloud/internal/opsconfig"
|
||||
)
|
||||
|
||||
func GetOpsContentQualityHandler(service *opsconfig.Service) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
principal, ok := webPrincipal(w, r)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
tenantID := strings.TrimSpace(r.Header.Get("X-Tenant-Id"))
|
||||
if !validUUID(tenantID) {
|
||||
writeValidation(w, r, "内容质量查询参数不正确", FieldError{Field: "X-Tenant-Id", Message: "必须是有效UUID"})
|
||||
return
|
||||
}
|
||||
items, err := service.ContentQuality(r.Context(), principal, tenantID, r.URL.Query().Get("model_ref"), r.URL.Query().Get("prompt_version"))
|
||||
if err != nil {
|
||||
writeOpsConfigError(w, r, err)
|
||||
return
|
||||
}
|
||||
w.Header().Set("Cache-Control", "no-store")
|
||||
WriteJSON(w, r, http.StatusOK, map[string]any{"items": items})
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,64 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"strings"
|
||||
|
||||
"data-terminal-cloud/internal/memorytool"
|
||||
)
|
||||
|
||||
func MemoryToolSearchHandler(service *memorytool.Service, signer *memorytool.TokenSigner, limiter RateLimiter, logger *slog.Logger) http.Handler {
|
||||
if logger == nil {
|
||||
logger = slog.Default()
|
||||
}
|
||||
return http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
|
||||
authorization := strings.TrimSpace(request.Header.Get("Authorization"))
|
||||
if !strings.HasPrefix(authorization, "Bearer ") {
|
||||
WriteError(writer, request, http.StatusUnauthorized, "MEMORY_TOOL_AUTHENTICATION_FAILED", "记忆工具身份验证失败")
|
||||
return
|
||||
}
|
||||
claims, err := signer.Verify(strings.TrimSpace(strings.TrimPrefix(authorization, "Bearer ")))
|
||||
if err != nil {
|
||||
WriteError(writer, request, http.StatusUnauthorized, "MEMORY_TOOL_AUTHENTICATION_FAILED", "记忆工具身份验证失败")
|
||||
return
|
||||
}
|
||||
if limiter != nil && !allowScopes(writer, request, logger, limiter, [][2]string{{"memory_tool", claims.ConversationID}}) {
|
||||
return
|
||||
}
|
||||
decoder := json.NewDecoder(request.Body)
|
||||
decoder.DisallowUnknownFields()
|
||||
var input struct {
|
||||
Query string `json:"query"`
|
||||
Limit int `json:"limit"`
|
||||
}
|
||||
if err := decoder.Decode(&input); err != nil || ensureJSONEOF(decoder) != nil {
|
||||
WriteError(writer, request, http.StatusBadRequest, "MEMORY_TOOL_INVALID_REQUEST", "记忆检索参数不正确")
|
||||
return
|
||||
}
|
||||
input.Query = strings.TrimSpace(input.Query)
|
||||
if input.Limit == 0 {
|
||||
input.Limit = 3
|
||||
}
|
||||
if input.Query == "" || len([]rune(input.Query)) > 500 || input.Limit < 1 || input.Limit > 5 {
|
||||
WriteError(writer, request, http.StatusBadRequest, "MEMORY_TOOL_INVALID_REQUEST", "记忆检索参数不正确")
|
||||
return
|
||||
}
|
||||
result, err := service.Search(request.Context(), claims, input.Query, input.Limit, RequestID(request.Context()))
|
||||
if err != nil {
|
||||
switch {
|
||||
case errors.Is(err, memorytool.ErrConversationUnavailable):
|
||||
WriteError(writer, request, http.StatusConflict, "MEMORY_TOOL_CONVERSATION_UNAVAILABLE", "本次通话当前不能使用记忆检索")
|
||||
case errors.Is(err, memorytool.ErrEngineModeUnsupported):
|
||||
WriteError(writer, request, http.StatusConflict, "MEMORY_TOOL_ENGINE_MODE_UNSUPPORTED", "本次语音处理方式未启用通话内记忆工具")
|
||||
default:
|
||||
slog.ErrorContext(request.Context(), "memory tool search failed", "conversation_id", claims.ConversationID, "error", err)
|
||||
WriteAPIError(writer, request, http.StatusServiceUnavailable, Error{Code: "MEMORY_TOOL_UNAVAILABLE", Message: "记忆检索暂时不可用，请继续普通交流", Retryable: true})
|
||||
}
|
||||
return
|
||||
}
|
||||
WriteJSON(writer, request, http.StatusOK, result)
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,219 @@
|
||||
package httpapi
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"net/http"
|
||||
"strings"
|
||||
|
||||
"data-terminal-cloud/internal/concurrency"
|
||||
"data-terminal-cloud/internal/identity"
|
||||
"data-terminal-cloud/internal/opsdevice"
|
||||
)
|
||||
|
||||
func ListOpsDevicesHandler(service *opsdevice.Service) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
principal, tenantID, ok := opsDevicePrincipal(w, r)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
items, err := service.List(r.Context(), principal, tenantID)
|
||||
if err != nil {
|
||||
writeOpsDeviceError(w, r, err)
|
||||
return
|
||||
}
|
||||
w.Header().Set("Cache-Control", "no-store")
|
||||
WriteJSON(w, r, http.StatusOK, map[string]any{"items": items})
|
||||
})
|
||||
}
|
||||
|
||||
func ListOpsDeviceOptionsHandler(service *opsdevice.Service) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
principal, tenantID, ok := opsDevicePrincipal(w, r)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
items, err := service.Options(r.Context(), principal, tenantID)
|
||||
if err != nil {
|
||||
writeOpsDeviceError(w, r, err)
|
||||
return
|
||||
}
|
||||
w.Header().Set("Cache-Control", "no-store")
|
||||
WriteJSON(w, r, http.StatusOK, map[string]any{"items": items})
|
||||
})
|
||||
}
|
||||
|
||||
func CreateOpsDeviceHandler(service *opsdevice.Service) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
principal, tenantID, ok := opsDevicePrincipal(w, r)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
var body struct {
|
||||
Model string `json:"model"`
|
||||
HardwareRevision string `json:"hardware_revision"`
|
||||
MinimumSecurityVersion string `json:"minimum_security_version"`
|
||||
MinimumFirmwareVersion string `json:"minimum_firmware_version"`
|
||||
HouseholdID string `json:"household_id"`
|
||||
SubjectID string `json:"subject_id"`
|
||||
}
|
||||
if !decodeWebJSON(w, r, &body, "设备登记参数不正确") {
|
||||
return
|
||||
}
|
||||
if (body.HouseholdID != "" && !validUUID(body.HouseholdID)) || (body.SubjectID != "" && !validUUID(body.SubjectID)) {
|
||||
writeValidation(w, r, "设备登记参数不正确", FieldError{Field: "assignment", Message: "家庭和长者必须是有效UUID"})
|
||||
return
|
||||
}
|
||||
result, err := service.Create(r.Context(), principal, tenantID, RequestID(r.Context()), opsdevice.CreateInput{
|
||||
Model: body.Model, HardwareRevision: body.HardwareRevision, MinimumSecurityVersion: body.MinimumSecurityVersion,
|
||||
MinimumFirmwareVersion: body.MinimumFirmwareVersion, HouseholdID: body.HouseholdID, SubjectID: body.SubjectID,
|
||||
})
|
||||
if err != nil {
|
||||
writeOpsDeviceError(w, r, err)
|
||||
return
|
||||
}
|
||||
w.Header().Set("Cache-Control", "no-store")
|
||||
w.Header().Set("ETag", concurrency.FormatETag(result.Device.Version))
|
||||
WriteJSON(w, r, http.StatusCreated, result)
|
||||
})
|
||||
}
|
||||
|
||||
func UpdateOpsDeviceHandler(service *opsdevice.Service) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
principal, tenantID, ok := opsDevicePrincipal(w, r)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
if !validUUID(r.PathValue("id")) {
|
||||
writeValidation(w, r, "设备编号不正确", FieldError{Field: "id", Message: "必须是UUID"})
|
||||
return
|
||||
}
|
||||
expected, err := concurrency.ParseIfMatch(r)
|
||||
if err != nil {
|
||||
writeETagError(w, r, err)
|
||||
return
|
||||
}
|
||||
var body struct {
|
||||
Model *string `json:"model"`
|
||||
HardwareRevision *string `json:"hardware_revision"`
|
||||
MinimumSecurityVersion *string `json:"minimum_security_version"`
|
||||
MinimumFirmwareVersion *string `json:"minimum_firmware_version"`
|
||||
Status *string `json:"status"`
|
||||
HouseholdID *string `json:"household_id"`
|
||||
SubjectID *string `json:"subject_id"`
|
||||
}
|
||||
if !decodeWebJSON(w, r, &body, "设备修改参数不正确") {
|
||||
return
|
||||
}
|
||||
if (body.HouseholdID != nil && *body.HouseholdID != "" && !validUUID(*body.HouseholdID)) || (body.SubjectID != nil && *body.SubjectID != "" && !validUUID(*body.SubjectID)) {
|
||||
writeValidation(w, r, "设备修改参数不正确", FieldError{Field: "assignment", Message: "家庭和长者必须是有效UUID"})
|
||||
return
|
||||
}
|
||||
item, err := service.Update(r.Context(), principal, tenantID, r.PathValue("id"), RequestID(r.Context()), expected, opsdevice.UpdateInput{
|
||||
Model: body.Model, HardwareRevision: body.HardwareRevision, MinimumSecurityVersion: body.MinimumSecurityVersion,
|
||||
MinimumFirmwareVersion: body.MinimumFirmwareVersion, Status: body.Status, HouseholdID: body.HouseholdID, SubjectID: body.SubjectID,
|
||||
})
|
||||
if err != nil {
|
||||
writeOpsDeviceError(w, r, err)
|
||||
return
|
||||
}
|
||||
w.Header().Set("ETag", concurrency.FormatETag(item.Version))
|
||||
w.Header().Set("Cache-Control", "no-store")
|
||||
WriteJSON(w, r, http.StatusOK, item)
|
||||
})
|
||||
}
|
||||
|
||||
func DeleteOpsDeviceHandler(service *opsdevice.Service) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
principal, tenantID, ok := opsDevicePrincipal(w, r)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
if !validUUID(r.PathValue("id")) {
|
||||
writeValidation(w, r, "设备编号不正确", FieldError{Field: "id", Message: "必须是UUID"})
|
||||
return
|
||||
}
|
||||
expected, err := concurrency.ParseIfMatch(r)
|
||||
if err != nil {
|
||||
writeETagError(w, r, err)
|
||||
return
|
||||
}
|
||||
item, err := service.Delete(r.Context(), principal, tenantID, r.PathValue("id"), RequestID(r.Context()), expected)
|
||||
if err != nil {
|
||||
writeOpsDeviceError(w, r, err)
|
||||
return
|
||||
}
|
||||
w.Header().Set("ETag", concurrency.FormatETag(item.Version))
|
||||
w.Header().Set("Cache-Control", "no-store")
|
||||
WriteJSON(w, r, http.StatusOK, item)
|
||||
})
|
||||
}
|
||||
|
||||
func RotateOpsDeviceCredentialHandler(service *opsdevice.Service) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
principal, tenantID, ok := opsDevicePrincipal(w, r)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
if !validUUID(r.PathValue("id")) {
|
||||
writeValidation(w, r, "设备编号不正确", FieldError{Field: "id", Message: "必须是UUID"})
|
||||
return
|
||||
}
|
||||
item, err := service.RotateCredential(r.Context(), principal, tenantID, r.PathValue("id"), RequestID(r.Context()))
|
||||
if err != nil {
|
||||
writeOpsDeviceError(w, r, err)
|
||||
return
|
||||
}
|
||||
w.Header().Set("Cache-Control", "no-store")
|
||||
WriteJSON(w, r, http.StatusCreated, item)
|
||||
})
|
||||
}
|
||||
|
||||
func IssueOpsDeviceClaimHandler(service *opsdevice.Service) http.Handler {
|
||||
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
principal, tenantID, ok := opsDevicePrincipal(w, r)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
if !validUUID(r.PathValue("id")) {
|
||||
writeValidation(w, r, "设备编号不正确", FieldError{Field: "id", Message: "必须是UUID"})
|
||||
return
|
||||
}
|
||||
item, err := service.IssueDeviceClaim(r.Context(), principal, tenantID, r.PathValue("id"), RequestID(r.Context()))
|
||||
if err != nil {
|
||||
writeOpsDeviceError(w, r, err)
|
||||
return
|
||||
}
|
||||
w.Header().Set("Cache-Control", "no-store")
|
||||
WriteJSON(w, r, http.StatusCreated, item)
|
||||
})
|
||||
}
|
||||
|
||||
func opsDevicePrincipal(w http.ResponseWriter, r *http.Request) (principal identity.Principal, tenantID string, ok bool) {
|
||||
principal, ok = webPrincipal(w, r)
|
||||
if !ok {
|
||||
return principal, "", false
|
||||
}
|
||||
tenantID = strings.TrimSpace(r.Header.Get("X-Tenant-Id"))
|
||||
if !validUUID(tenantID) {
|
||||
writeValidation(w, r, "运维设备请求参数不正确", FieldError{Field: "X-Tenant-Id", Message: "必须提供有效的客户组织UUID"})
|
||||
return principal, "", false
|
||||
}
|
||||
return principal, tenantID, true
|
||||
}
|
||||
|
||||
func writeOpsDeviceError(w http.ResponseWriter, r *http.Request, err error) {
|
||||
switch {
|
||||
case errors.Is(err, opsdevice.ErrInvalid):
|
||||
writeValidation(w, r, "设备管理参数不正确", FieldError{Field: "device", Message: "设备字段或家庭/长者归属不正确"})
|
||||
case errors.Is(err, opsdevice.ErrAccessDenied):
|
||||
WriteError(w, r, http.StatusForbidden, "OPS_ACCESS_DENIED", "仅独立运维账号可以管理设备")
|
||||
case errors.Is(err, opsdevice.ErrNotFound):
|
||||
WriteError(w, r, http.StatusNotFound, "RESOURCE_NOT_FOUND", "设备不存在")
|
||||
case errors.Is(err, opsdevice.ErrVersionConflict):
|
||||
WriteError(w, r, http.StatusPreconditionFailed, "VERSION_CONFLICT", "设备信息已变化，请刷新后重试")
|
||||
case errors.Is(err, opsdevice.ErrClaimUnavailable):
|
||||
WriteError(w, r, http.StatusConflict, "DEVICE_CLAIM_UNAVAILABLE", "只有已登记且尚未绑定的设备可以生成绑定码")
|
||||
default:
|
||||
WriteError(w, r, http.StatusInternalServerError, "INTERNAL_ERROR", "设备管理操作失败")
|
||||
}
|
||||
}
|
||||
@@ -13,8 +13,10 @@ import (
|
||||
"data-terminal-cloud/internal/health"
|
||||
"data-terminal-cloud/internal/identity"
|
||||
"data-terminal-cloud/internal/media"
|
||||
"data-terminal-cloud/internal/memorytool"
|
||||
"data-terminal-cloud/internal/opsaccess"
|
||||
"data-terminal-cloud/internal/opsconfig"
|
||||
"data-terminal-cloud/internal/opsdevice"
|
||||
"data-terminal-cloud/internal/saas"
|
||||
"data-terminal-cloud/internal/sharing"
|
||||
"data-terminal-cloud/internal/webevents"
|
||||
@@ -39,6 +41,7 @@ type RouterOptions struct {
|
||||
AlertService *alerts.Service
|
||||
DiagnosticService *diagnostics.Service
|
||||
OpsAccessService *opsaccess.Service
|
||||
OpsDeviceService *opsdevice.Service
|
||||
ConversationCreator ConversationCreator
|
||||
ConversationActivator ConversationActivator
|
||||
DeviceConversationStateReader DeviceConversationStateReader
|
||||
@@ -51,6 +54,8 @@ type RouterOptions struct {
|
||||
PlatformHealthCallbackSecret []byte
|
||||
PlatformHealthCallbackSkew time.Duration
|
||||
PlatformHealthProcessor PlatformHealthProcessor
|
||||
MemoryToolService *memorytool.Service
|
||||
MemoryToolSigner *memorytool.TokenSigner
|
||||
}
|
||||
|
||||
func NewRouter(options RouterOptions) http.Handler {
|
||||
@@ -77,6 +82,9 @@ func NewRouter(options RouterOptions) http.Handler {
|
||||
mux.HandleFunc("GET /api/v1/version", func(writer http.ResponseWriter, request *http.Request) {
|
||||
WriteJSON(writer, request, http.StatusOK, buildinfo.Current())
|
||||
})
|
||||
if options.ServiceName == "api" && options.MemoryToolService != nil && options.MemoryToolSigner != nil {
|
||||
mux.Handle("POST /internal/v1/tools/memory-search", MemoryToolSearchHandler(options.MemoryToolService, options.MemoryToolSigner, options.RateLimiter, logger))
|
||||
}
|
||||
if options.ServiceName == "api" && options.TokenVerifier != nil && options.IdentityRepository != nil {
|
||||
mux.Handle("GET /api/v1/me", BearerAuth(options.TokenVerifier, UserRateLimit(logger, options.RateLimiter, CurrentUserHandler(options.IdentityRepository))))
|
||||
}
|
||||
@@ -95,6 +103,7 @@ func NewRouter(options RouterOptions) http.Handler {
|
||||
mux.Handle("PATCH /api/v1/sharing-grants/{id}/resource", authenticated(EditSharedStoryHandler(options.SharingService)))
|
||||
}
|
||||
if options.OpsConfigService != nil {
|
||||
mux.Handle("GET /api/v1/ops/content-quality", authenticated(GetOpsContentQualityHandler(options.OpsConfigService)))
|
||||
mux.Handle("GET /api/v1/ops/resource-references", authenticated(ListOpsResourceReferencesHandler(options.OpsConfigService)))
|
||||
mux.Handle("POST /api/v1/ops/resource-references", authenticated(CreateOpsResourceReferenceHandler(options.OpsConfigService)))
|
||||
mux.Handle("PATCH /api/v1/ops/resource-references/{id}/revoke", authenticated(RevokeOpsResourceReferenceHandler(options.OpsConfigService)))
|
||||
@@ -112,12 +121,14 @@ func NewRouter(options RouterOptions) http.Handler {
|
||||
mux.Handle("POST /api/v1/ops/config-versions/{id}/rollback", authenticated(RollbackOpsConfigVersionHandler(options.OpsConfigService)))
|
||||
}
|
||||
if options.BillingService != nil {
|
||||
mux.Handle("GET /api/v1/households/{id}/usage-overview", authenticated(GetFamilyUsageOverviewHandler(options.BillingService)))
|
||||
mux.Handle("GET /api/v1/ops/price-books", authenticated(ListOpsPriceBooksHandler(options.BillingService)))
|
||||
mux.Handle("POST /api/v1/ops/price-books", authenticated(CreateOpsPriceBookHandler(options.BillingService)))
|
||||
mux.Handle("PATCH /api/v1/ops/price-books/{id}/publish", authenticated(PublishOpsPriceBookHandler(options.BillingService)))
|
||||
mux.Handle("GET /api/v1/ops/quota-policies", authenticated(ListOpsQuotaPoliciesHandler(options.BillingService)))
|
||||
mux.Handle("POST /api/v1/ops/quota-policies", authenticated(CreateOpsQuotaPolicyHandler(options.BillingService)))
|
||||
mux.Handle("GET /api/v1/ops/cost-summary", authenticated(GetOpsCostSummaryHandler(options.BillingService)))
|
||||
mux.Handle("GET /api/v1/ops/usage-overview", authenticated(GetOpsUsageOverviewHandler(options.BillingService)))
|
||||
}
|
||||
if options.DiagnosticService != nil {
|
||||
mux.Handle("GET /api/v1/ops/diagnostics/platform-health", authenticated(GetOpsPlatformHealthHandler(options.DiagnosticService)))
|
||||
@@ -144,6 +155,15 @@ func NewRouter(options RouterOptions) http.Handler {
|
||||
mux.Handle("GET /api/v1/ops/sensitive-conversations/{id}", authenticated(GetOpsSensitiveConversationHandler(options.OpsAccessService)))
|
||||
mux.Handle("POST /api/v1/ops/sensitive-media/{id}/play", authenticated(CreateOpsSensitiveRecordingAccessHandler(options.OpsAccessService)))
|
||||
}
|
||||
if options.OpsDeviceService != nil {
|
||||
mux.Handle("GET /api/v1/ops/devices", authenticated(ListOpsDevicesHandler(options.OpsDeviceService)))
|
||||
mux.Handle("POST /api/v1/ops/devices", authenticated(CreateOpsDeviceHandler(options.OpsDeviceService)))
|
||||
mux.Handle("GET /api/v1/ops/device-assignment-options", authenticated(ListOpsDeviceOptionsHandler(options.OpsDeviceService)))
|
||||
mux.Handle("PATCH /api/v1/ops/devices/{id}", authenticated(UpdateOpsDeviceHandler(options.OpsDeviceService)))
|
||||
mux.Handle("DELETE /api/v1/ops/devices/{id}", authenticated(DeleteOpsDeviceHandler(options.OpsDeviceService)))
|
||||
mux.Handle("POST /api/v1/ops/devices/{id}/credentials/rotate", authenticated(RotateOpsDeviceCredentialHandler(options.OpsDeviceService)))
|
||||
mux.Handle("POST /api/v1/ops/devices/{id}/claims", authenticated(IssueOpsDeviceClaimHandler(options.OpsDeviceService)))
|
||||
}
|
||||
mux.Handle("GET /api/v1/households", authenticated(ListHouseholdsHandler(options.SaaSService)))
|
||||
mux.Handle("POST /api/v1/households", authenticated(CreateHouseholdHandler(options.SaaSService)))
|
||||
mux.Handle("GET /api/v1/households/{id}", authenticated(GetHouseholdHandler(options.SaaSService)))
|
||||
@@ -171,6 +191,8 @@ func NewRouter(options RouterOptions) http.Handler {
|
||||
mux.Handle("POST /api/v1/device-claims", authenticated(ClaimDeviceHandler(options.SaaSService)))
|
||||
mux.Handle("GET /api/v1/devices", authenticated(ListManagedDevicesHandler(options.SaaSService)))
|
||||
mux.Handle("GET /api/v1/devices/{id}", authenticated(GetManagedDeviceHandler(options.SaaSService)))
|
||||
mux.Handle("GET /api/v1/devices/{id}/binding-impact", authenticated(GetDeviceBindingImpactHandler(options.SaaSService)))
|
||||
mux.Handle("POST /api/v1/devices/{id}/binding-change", authenticated(ChangeDeviceBindingHandler(options.SaaSService)))
|
||||
mux.Handle("GET /api/v1/devices/{id}/presence", authenticated(GetDevicePresenceHandler(options.SaaSService)))
|
||||
mux.Handle("GET /api/v1/devices/{id}/twin", authenticated(GetDeviceTwinHandler(options.SaaSService)))
|
||||
mux.Handle("PATCH /api/v1/devices/{id}/twin", authenticated(PatchDeviceTwinHandler(options.SaaSService)))
|
||||
@@ -207,6 +229,7 @@ func NewRouter(options RouterOptions) http.Handler {
|
||||
mux.Handle("GET /api/v1/subjects/{id}/stories", authenticated(ListSubjectStoriesHandler(options.SaaSService)))
|
||||
mux.Handle("GET /api/v1/stories/{id}", authenticated(GetStoryHandler(options.SaaSService)))
|
||||
mux.Handle("PATCH /api/v1/stories/{id}", authenticated(PatchStoryHandler(options.SaaSService)))
|
||||
mux.Handle("PATCH /api/v1/stories/{id}/facts/{fact_id}", authenticated(UpdateFactCandidateHandler(options.SaaSService)))
|
||||
mux.Handle("POST /api/v1/stories/{id}/confirm", authenticated(ConfirmStoryHandler(options.SaaSService)))
|
||||
mux.Handle("POST /api/v1/stories/{id}/reject", authenticated(RejectStoryHandler(options.SaaSService)))
|
||||
mux.Handle("GET /api/v1/subjects/{id}/memories", authenticated(ListSubjectMemoriesHandler(options.SaaSService)))
|
||||
|
||||
@@ -729,6 +729,9 @@ func ClaimDeviceHandler(service *saas.Service) http.Handler {
|
||||
if len(input.ClaimCode) < 16 || len(input.ClaimCode) > 128 {
|
||||
fieldErrors = append(fieldErrors, FieldError{Field: "claim_code", Message: "长度必须为16至128字符"})
|
||||
}
|
||||
if validUUID(input.ClaimCode) {
|
||||
fieldErrors = append(fieldErrors, FieldError{Field: "claim_code", Message: "不能填写设备编号，请输入设备屏幕或维护端生成的一次性绑定码"})
|
||||
}
|
||||
if !validUUID(input.HouseholdID) {
|
||||
fieldErrors = append(fieldErrors, FieldError{Field: "household_id", Message: "必须是UUID"})
|
||||
}
|
||||
@@ -766,6 +769,86 @@ func GetManagedDeviceHandler(service *saas.Service) http.Handler {
|
||||
})
|
||||
}
|
||||
|
||||
func GetDeviceBindingImpactHandler(service *saas.Service) http.Handler {
|
||||
return http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
|
||||
principal, ok := resourcePrincipal(writer, request, "id")
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
action := strings.TrimSpace(request.URL.Query().Get("action"))
|
||||
targetSubjectID := strings.TrimSpace(request.URL.Query().Get("target_subject_id"))
|
||||
fieldErrors := make([]FieldError, 0)
|
||||
if action != saas.DeviceBindingActionUnbind && action != saas.DeviceBindingActionTransfer {
|
||||
fieldErrors = append(fieldErrors, FieldError{Field: "action", Message: "必须是unbind或transfer"})
|
||||
}
|
||||
if action == saas.DeviceBindingActionTransfer && !validUUID(targetSubjectID) {
|
||||
fieldErrors = append(fieldErrors, FieldError{Field: "target_subject_id", Message: "转移时必须提供目标长者UUID"})
|
||||
}
|
||||
if len(fieldErrors) > 0 {
|
||||
WriteAPIError(writer, request, http.StatusBadRequest, Error{Code: "VALIDATION_FAILED", Message: "设备绑定影响检查参数不正确", FieldErrors: fieldErrors})
|
||||
return
|
||||
}
|
||||
result, err := service.GetDeviceBindingImpact(request.Context(), principal, request.PathValue("id"), action, targetSubjectID)
|
||||
if err != nil {
|
||||
writeSaaSError(writer, request, err)
|
||||
return
|
||||
}
|
||||
writer.Header().Set("Cache-Control", "no-store")
|
||||
WriteJSON(writer, request, http.StatusOK, result)
|
||||
})
|
||||
}
|
||||
|
||||
func ChangeDeviceBindingHandler(service *saas.Service) http.Handler {
|
||||
return http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
|
||||
principal, ok := resourcePrincipal(writer, request, "id")
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
var input struct {
|
||||
Action string `json:"action"`
|
||||
ExpectedBindingID string `json:"expected_binding_id"`
|
||||
TargetSubjectID string `json:"target_subject_id"`
|
||||
Reason string `json:"reason"`
|
||||
Acknowledged bool `json:"acknowledged"`
|
||||
}
|
||||
if !decodeWebJSON(writer, request, &input, "设备绑定变更参数不正确") {
|
||||
return
|
||||
}
|
||||
input.Action = strings.TrimSpace(input.Action)
|
||||
input.ExpectedBindingID = strings.TrimSpace(input.ExpectedBindingID)
|
||||
input.TargetSubjectID = strings.TrimSpace(input.TargetSubjectID)
|
||||
input.Reason = strings.TrimSpace(input.Reason)
|
||||
fieldErrors := make([]FieldError, 0)
|
||||
if input.Action != saas.DeviceBindingActionUnbind && input.Action != saas.DeviceBindingActionTransfer {
|
||||
fieldErrors = append(fieldErrors, FieldError{Field: "action", Message: "必须是unbind或transfer"})
|
||||
}
|
||||
if !validUUID(input.ExpectedBindingID) {
|
||||
fieldErrors = append(fieldErrors, FieldError{Field: "expected_binding_id", Message: "必须是影响检查返回的当前绑定UUID"})
|
||||
}
|
||||
if input.Action == saas.DeviceBindingActionTransfer && !validUUID(input.TargetSubjectID) {
|
||||
fieldErrors = append(fieldErrors, FieldError{Field: "target_subject_id", Message: "转移时必须提供目标长者UUID"})
|
||||
}
|
||||
if len(input.Reason) < 3 || len(input.Reason) > 300 {
|
||||
fieldErrors = append(fieldErrors, FieldError{Field: "reason", Message: "长度必须为3至300字符"})
|
||||
}
|
||||
if !input.Acknowledged {
|
||||
fieldErrors = append(fieldErrors, FieldError{Field: "acknowledged", Message: "必须确认已核对影响范围"})
|
||||
}
|
||||
if len(fieldErrors) > 0 {
|
||||
WriteAPIError(writer, request, http.StatusBadRequest, Error{Code: "VALIDATION_FAILED", Message: "设备绑定变更参数不正确", FieldErrors: fieldErrors})
|
||||
return
|
||||
}
|
||||
result, err := service.ChangeDeviceBinding(request.Context(), principal, request.PathValue("id"), RequestID(request.Context()), saas.ChangeDeviceBindingInput{
|
||||
Action: input.Action, ExpectedBindingID: input.ExpectedBindingID, TargetSubjectID: input.TargetSubjectID, Reason: input.Reason, Acknowledged: input.Acknowledged,
|
||||
})
|
||||
if err != nil {
|
||||
writeSaaSError(writer, request, err)
|
||||
return
|
||||
}
|
||||
WriteJSON(writer, request, http.StatusOK, result)
|
||||
})
|
||||
}
|
||||
|
||||
func GetDevicePresenceHandler(service *saas.Service) http.Handler {
|
||||
return http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
|
||||
principal, ok := resourcePrincipal(writer, request, "id")
|
||||
@@ -1009,21 +1092,104 @@ func PatchStoryHandler(service *saas.Service) http.Handler {
|
||||
return
|
||||
}
|
||||
var input struct {
|
||||
Content string `json:"content"`
|
||||
Reason string `json:"reason"`
|
||||
Content string `json:"content"`
|
||||
Reason string `json:"reason"`
|
||||
ErrorType string `json:"error_type"`
|
||||
}
|
||||
if !decodeWebJSON(writer, request, &input, "故事修订参数不正确") {
|
||||
return
|
||||
}
|
||||
input.Content = strings.TrimSpace(input.Content)
|
||||
input.Reason = strings.TrimSpace(input.Reason)
|
||||
input.ErrorType = strings.TrimSpace(input.ErrorType)
|
||||
if input.Content == "" || len([]rune(input.Content)) > 50000 || input.Reason == "" || len([]rune(input.Reason)) > 500 {
|
||||
writeValidation(writer, request, "故事修订参数不正确",
|
||||
FieldError{Field: "content", Message: "不能为空且最长50000字符"},
|
||||
FieldError{Field: "reason", Message: "不能为空且最长500字符"})
|
||||
return
|
||||
}
|
||||
result, err := service.ReviseStory(request.Context(), principal, request.PathValue("id"), input.Content, input.Reason, RequestID(request.Context()), expected)
|
||||
if input.ErrorType != "" && !saas.IsContentErrorType(input.ErrorType) {
|
||||
writeValidation(writer, request, "故事修订参数不正确", FieldError{Field: "error_type", Message: "质量问题类型不正确"})
|
||||
return
|
||||
}
|
||||
result, err := service.ReviseStory(request.Context(), principal, request.PathValue("id"), input.Content, input.Reason, RequestID(request.Context()), expected, input.ErrorType)
|
||||
if err != nil {
|
||||
writeSaaSError(writer, request, err)
|
||||
return
|
||||
}
|
||||
writer.Header().Set("ETag", concurrency.FormatETag(result.Version))
|
||||
WriteJSON(writer, request, http.StatusOK, result)
|
||||
})
|
||||
}
|
||||
|
||||
func UpdateFactCandidateHandler(service *saas.Service) http.Handler {
|
||||
return http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) {
|
||||
principal, ok := resourcePrincipal(writer, request, "id")
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
if !validUUID(request.PathValue("fact_id")) {
|
||||
writeValidation(writer, request, "事实候选编号不正确", FieldError{Field: "fact_id", Message: "必须是UUID"})
|
||||
return
|
||||
}
|
||||
expected, err := concurrency.ParseIfMatch(request)
|
||||
if err != nil {
|
||||
writeETagError(writer, request, err)
|
||||
return
|
||||
}
|
||||
var input struct {
|
||||
FactType *string `json:"fact_type"`
|
||||
Content *string `json:"content"`
|
||||
Sensitivity *string `json:"sensitivity"`
|
||||
Status *string `json:"status"`
|
||||
ErrorType *string `json:"error_type"`
|
||||
Reason string `json:"reason"`
|
||||
}
|
||||
if !decodeWebJSON(writer, request, &input, "事实候选审核参数不正确") {
|
||||
return
|
||||
}
|
||||
input.Reason = strings.TrimSpace(input.Reason)
|
||||
fieldErrors := make([]FieldError, 0)
|
||||
if input.Reason == "" || len([]rune(input.Reason)) > 500 {
|
||||
fieldErrors = append(fieldErrors, FieldError{Field: "reason", Message: "不能为空且最长500字符"})
|
||||
}
|
||||
if input.FactType != nil {
|
||||
value := strings.TrimSpace(*input.FactType)
|
||||
input.FactType = &value
|
||||
if value == "" || len(value) > 64 {
|
||||
fieldErrors = append(fieldErrors, FieldError{Field: "fact_type", Message: "不能为空且最长64字符"})
|
||||
}
|
||||
}
|
||||
if input.Content != nil {
|
||||
value := strings.TrimSpace(*input.Content)
|
||||
input.Content = &value
|
||||
if value == "" || len([]rune(value)) > 4000 {
|
||||
fieldErrors = append(fieldErrors, FieldError{Field: "content", Message: "不能为空且最长4000字符"})
|
||||
}
|
||||
}
|
||||
if input.Sensitivity != nil && *input.Sensitivity != "normal" && *input.Sensitivity != "sensitive" && *input.Sensitivity != "high" {
|
||||
fieldErrors = append(fieldErrors, FieldError{Field: "sensitivity", Message: "必须为normal、sensitive或high"})
|
||||
}
|
||||
if input.Status != nil && *input.Status != "CANDIDATE" && *input.Status != "CONFIRMED" && *input.Status != "REJECTED" && *input.Status != "SUPERSEDED" {
|
||||
fieldErrors = append(fieldErrors, FieldError{Field: "status", Message: "事实候选状态不正确"})
|
||||
}
|
||||
if input.ErrorType != nil {
|
||||
value := strings.TrimSpace(*input.ErrorType)
|
||||
input.ErrorType = &value
|
||||
if value == "" || !saas.IsContentErrorType(value) {
|
||||
fieldErrors = append(fieldErrors, FieldError{Field: "error_type", Message: "质量问题类型不正确"})
|
||||
}
|
||||
}
|
||||
if input.FactType == nil && input.Content == nil && input.Sensitivity == nil && input.Status == nil {
|
||||
fieldErrors = append(fieldErrors, FieldError{Field: "body", Message: "至少提供一个需要修改或审核的字段"})
|
||||
}
|
||||
if len(fieldErrors) > 0 {
|
||||
WriteAPIError(writer, request, http.StatusBadRequest, Error{Code: "VALIDATION_FAILED", Message: "事实候选审核参数不正确", FieldErrors: fieldErrors})
|
||||
return
|
||||
}
|
||||
result, err := service.UpdateFactCandidate(request.Context(), principal, request.PathValue("id"), request.PathValue("fact_id"), RequestID(request.Context()), expected, saas.UpdateFactCandidateInput{
|
||||
FactType: input.FactType, Content: input.Content, Sensitivity: input.Sensitivity, Status: input.Status, ErrorType: input.ErrorType, Reason: input.Reason,
|
||||
})
|
||||
if err != nil {
|
||||
writeSaaSError(writer, request, err)
|
||||
return
|
||||
@@ -1053,16 +1219,22 @@ func storyDecisionHandler(service *saas.Service, confirm bool) http.Handler {
|
||||
return
|
||||
}
|
||||
var input struct {
|
||||
Reason string `json:"reason"`
|
||||
Reason string `json:"reason"`
|
||||
ErrorType string `json:"error_type"`
|
||||
}
|
||||
if !decodeWebJSON(writer, request, &input, "故事审核参数不正确") {
|
||||
return
|
||||
}
|
||||
input.Reason = strings.TrimSpace(input.Reason)
|
||||
input.ErrorType = strings.TrimSpace(input.ErrorType)
|
||||
if input.Reason == "" || len([]rune(input.Reason)) > 500 {
|
||||
writeValidation(writer, request, "故事审核参数不正确", FieldError{Field: "reason", Message: "不能为空且最长500字符"})
|
||||
return
|
||||
}
|
||||
if input.ErrorType != "" && !saas.IsContentErrorType(input.ErrorType) {
|
||||
writeValidation(writer, request, "故事审核参数不正确", FieldError{Field: "error_type", Message: "质量问题类型不正确"})
|
||||
return
|
||||
}
|
||||
if confirm {
|
||||
story, memory, err := service.ConfirmStory(request.Context(), principal, request.PathValue("id"), input.Reason, RequestID(request.Context()), expected)
|
||||
if err != nil {
|
||||
@@ -1073,7 +1245,7 @@ func storyDecisionHandler(service *saas.Service, confirm bool) http.Handler {
|
||||
WriteJSON(writer, request, http.StatusOK, map[string]any{"story": story, "memory": memory})
|
||||
return
|
||||
}
|
||||
story, err := service.RejectStory(request.Context(), principal, request.PathValue("id"), input.Reason, RequestID(request.Context()), expected)
|
||||
story, err := service.RejectStory(request.Context(), principal, request.PathValue("id"), input.Reason, RequestID(request.Context()), expected, input.ErrorType)
|
||||
if err != nil {
|
||||
writeSaaSError(writer, request, err)
|
||||
return
|
||||
@@ -1131,9 +1303,10 @@ func PatchMemoryHandler(service *saas.Service) http.Handler {
|
||||
return
|
||||
}
|
||||
var input struct {
|
||||
Content *string `json:"content"`
|
||||
Status *string `json:"status"`
|
||||
Reason string `json:"reason"`
|
||||
Content *string `json:"content"`
|
||||
Status *string `json:"status"`
|
||||
ErrorType *string `json:"error_type"`
|
||||
Reason string `json:"reason"`
|
||||
}
|
||||
if !decodeWebJSON(writer, request, &input, "记忆修改参数不正确") {
|
||||
return
|
||||
@@ -1153,6 +1326,13 @@ func PatchMemoryHandler(service *saas.Service) http.Handler {
|
||||
if input.Status != nil && !oneOf(*input.Status, "ACTIVE", "DISABLED") {
|
||||
fieldErrors = append(fieldErrors, FieldError{Field: "status", Message: "只允许ACTIVE或DISABLED"})
|
||||
}
|
||||
if input.ErrorType != nil {
|
||||
value := strings.TrimSpace(*input.ErrorType)
|
||||
input.ErrorType = &value
|
||||
if value == "" || !saas.IsContentErrorType(value) {
|
||||
fieldErrors = append(fieldErrors, FieldError{Field: "error_type", Message: "质量问题类型不正确"})
|
||||
}
|
||||
}
|
||||
if input.Reason == "" || len([]rune(input.Reason)) > 500 {
|
||||
fieldErrors = append(fieldErrors, FieldError{Field: "reason", Message: "不能为空且最长500字符"})
|
||||
}
|
||||
@@ -1160,7 +1340,7 @@ func PatchMemoryHandler(service *saas.Service) http.Handler {
|
||||
WriteAPIError(writer, request, http.StatusBadRequest, Error{Code: "VALIDATION_FAILED", Message: "记忆修改参数不正确", FieldErrors: fieldErrors})
|
||||
return
|
||||
}
|
||||
result, err := service.UpdateMemory(request.Context(), principal, request.PathValue("id"), RequestID(request.Context()), expected, saas.UpdateMemoryInput{Content: input.Content, Status: input.Status, Reason: input.Reason})
|
||||
result, err := service.UpdateMemory(request.Context(), principal, request.PathValue("id"), RequestID(request.Context()), expected, saas.UpdateMemoryInput{Content: input.Content, Status: input.Status, ErrorType: input.ErrorType, Reason: input.Reason})
|
||||
if err != nil {
|
||||
writeSaaSError(writer, request, err)
|
||||
return
|
||||
@@ -1233,15 +1413,23 @@ func CreatePrivacyRequestHandler(service *saas.Service) http.Handler {
|
||||
Reason string `json:"reason"`
|
||||
Confirmation string `json:"confirmation"`
|
||||
Scope struct {
|
||||
Categories []string `json:"categories"`
|
||||
Categories []string `json:"categories"`
|
||||
ConsentTypes []string `json:"consent_types"`
|
||||
Corrections struct {
|
||||
PreferredName *string `json:"preferred_name"`
|
||||
Language *string `json:"language"`
|
||||
Timezone *string `json:"timezone"`
|
||||
ProfileData *map[string]any `json:"profile_data"`
|
||||
ConversationPreferences *map[string]any `json:"conversation_preferences"`
|
||||
} `json:"corrections"`
|
||||
} `json:"scope"`
|
||||
}
|
||||
if !decodeWebJSON(writer, request, &input, "隐私请求参数不正确") {
|
||||
return
|
||||
}
|
||||
input.RequestType = strings.ToUpper(strings.TrimSpace(input.RequestType))
|
||||
if input.RequestType != "ACCESS" && input.RequestType != "DELETE" {
|
||||
writeValidation(writer, request, "隐私请求参数不正确", FieldError{Field: "request_type", Message: "必须为ACCESS或DELETE"})
|
||||
if input.RequestType != "ACCESS" && input.RequestType != "CORRECTION" && input.RequestType != "CONSENT_WITHDRAWAL" && input.RequestType != "DELETE" {
|
||||
writeValidation(writer, request, "隐私请求参数不正确", FieldError{Field: "request_type", Message: "必须为ACCESS、CORRECTION、CONSENT_WITHDRAWAL或DELETE"})
|
||||
return
|
||||
}
|
||||
if input.RequestType == "DELETE" {
|
||||
@@ -1274,6 +1462,68 @@ func CreatePrivacyRequestHandler(service *saas.Service) http.Handler {
|
||||
if len(categories) > 0 {
|
||||
scope["categories"] = categories
|
||||
}
|
||||
if input.RequestType == "CORRECTION" {
|
||||
corrections := map[string]any{}
|
||||
if input.Scope.Corrections.PreferredName != nil {
|
||||
value := strings.TrimSpace(*input.Scope.Corrections.PreferredName)
|
||||
if len([]rune(value)) < 1 || len([]rune(value)) > 120 {
|
||||
writeValidation(writer, request, "更正申请参数不正确", FieldError{Field: "scope.corrections.preferred_name", Message: "长度必须为1至120字符"})
|
||||
return
|
||||
}
|
||||
corrections["preferred_name"] = value
|
||||
}
|
||||
if input.Scope.Corrections.Language != nil {
|
||||
value := strings.TrimSpace(*input.Scope.Corrections.Language)
|
||||
if len(value) < 2 || len(value) > 20 {
|
||||
writeValidation(writer, request, "更正申请参数不正确", FieldError{Field: "scope.corrections.language", Message: "长度必须为2至20字符"})
|
||||
return
|
||||
}
|
||||
corrections["language"] = value
|
||||
}
|
||||
if input.Scope.Corrections.Timezone != nil {
|
||||
value := strings.TrimSpace(*input.Scope.Corrections.Timezone)
|
||||
if _, err := time.LoadLocation(value); err != nil {
|
||||
writeValidation(writer, request, "更正申请参数不正确", FieldError{Field: "scope.corrections.timezone", Message: "必须是有效IANA时区"})
|
||||
return
|
||||
}
|
||||
corrections["timezone"] = value
|
||||
}
|
||||
if input.Scope.Corrections.ProfileData != nil {
|
||||
corrections["profile_data"] = *input.Scope.Corrections.ProfileData
|
||||
}
|
||||
if input.Scope.Corrections.ConversationPreferences != nil {
|
||||
corrections["conversation_preferences"] = *input.Scope.Corrections.ConversationPreferences
|
||||
}
|
||||
if len(corrections) == 0 {
|
||||
writeValidation(writer, request, "更正申请参数不正确", FieldError{Field: "scope.corrections", Message: "至少提供一个需要更正的字段"})
|
||||
return
|
||||
}
|
||||
scope = map[string]any{"corrections": corrections}
|
||||
}
|
||||
if input.RequestType == "CONSENT_WITHDRAWAL" {
|
||||
allowedConsentTypes := map[string]bool{
|
||||
saas.ConsentTranscriptStorage: true, saas.ConsentRawAudioStorage: true, saas.ConsentStoryExtraction: true,
|
||||
saas.ConsentPersonalMemory: true, saas.ConsentFamilySharing: true,
|
||||
}
|
||||
seenConsentTypes := map[string]bool{}
|
||||
consentTypes := make([]string, 0, len(input.Scope.ConsentTypes))
|
||||
for index, raw := range input.Scope.ConsentTypes {
|
||||
consentType := strings.TrimSpace(raw)
|
||||
if !allowedConsentTypes[consentType] {
|
||||
writeValidation(writer, request, "授权撤回申请参数不正确", FieldError{Field: fmt.Sprintf("scope.consent_types[%d]", index), Message: "授权类型不正确"})
|
||||
return
|
||||
}
|
||||
if !seenConsentTypes[consentType] {
|
||||
seenConsentTypes[consentType] = true
|
||||
consentTypes = append(consentTypes, consentType)
|
||||
}
|
||||
}
|
||||
if len(consentTypes) == 0 {
|
||||
writeValidation(writer, request, "授权撤回申请参数不正确", FieldError{Field: "scope.consent_types", Message: "至少选择一种授权"})
|
||||
return
|
||||
}
|
||||
scope = map[string]any{"consent_types": consentTypes}
|
||||
}
|
||||
result, created, err := service.CreatePrivacyRequest(request.Context(), principal, request.PathValue("id"), idempotencyKey, RequestID(request.Context()), saas.CreatePrivacyRequestInput{RequestType: input.RequestType, Scope: scope, Reason: input.Reason, Confirmation: input.Confirmation})
|
||||
if err != nil {
|
||||
writeSaaSError(writer, request, err)
|
||||
@@ -1633,6 +1883,10 @@ func writeSaaSError(writer http.ResponseWriter, request *http.Request, err error
|
||||
WriteError(writer, request, http.StatusConflict, "DEVICE_CLAIM_ALREADY_USED", "设备绑定码已经用于其他绑定")
|
||||
case errors.Is(err, saas.ErrDeviceAlreadyBound):
|
||||
WriteError(writer, request, http.StatusConflict, "DEVICE_ALREADY_BOUND", "设备已经存在有效绑定")
|
||||
case errors.Is(err, saas.ErrDeviceBindingBlocked):
|
||||
WriteError(writer, request, http.StatusConflict, "DEVICE_BINDING_BLOCKED", "设备仍有活动通话、未完成命令或固件任务，请先处理阻断项")
|
||||
case errors.Is(err, saas.ErrDeviceBindingStale):
|
||||
WriteError(writer, request, http.StatusConflict, "DEVICE_BINDING_STALE", "设备绑定已发生变化，请重新执行影响检查")
|
||||
case errors.Is(err, saas.ErrIdempotencyConflict):
|
||||
WriteError(writer, request, http.StatusConflict, "IDEMPOTENCY_KEY_CONFLICT", "重复请求识别码已用于不同的请求内容")
|
||||
case errors.Is(err, saas.ErrSubjectDeleteConflict):
|
||||
@@ -1665,6 +1919,8 @@ func writeSaaSError(writer http.ResponseWriter, request *http.Request, err error
|
||||
WriteError(writer, request, http.StatusConflict, "PERSONALIZED_MEMORY_CONSENT_REQUIRED", "生成个性化记忆前必须取得当前有效授权")
|
||||
case errors.Is(err, saas.ErrReviewStateConflict):
|
||||
WriteError(writer, request, http.StatusConflict, "STORY_REVIEW_STATE_CONFLICT", "当前故事审核状态不允许此操作")
|
||||
case errors.Is(err, saas.ErrFactReviewStateConflict):
|
||||
WriteError(writer, request, http.StatusConflict, "FACT_REVIEW_STATE_CONFLICT", "当前事实候选状态不允许此操作")
|
||||
case errors.Is(err, pagination.ErrInvalidCursor):
|
||||
writeValidation(writer, request, "分页游标无效", FieldError{Field: "cursor", Message: "游标无效、已损坏或不是本服务签发"})
|
||||
default:
|
||||
|
||||
@@ -103,6 +103,15 @@ func TestDecodeAndResourcePrincipal(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestClaimDeviceRejectsDeviceIdentifierAsClaimCode(t *testing.T) {
|
||||
request := authenticatedRequest(http.MethodPost, "/api/v1/device-claims", `{"claim_code":"10000000-0000-0000-0000-000000000512","household_id":"10000000-0000-0000-0000-000000000211","subject_id":"10000000-0000-0000-0000-000000000411"}`, identity.Principal{Issuer: "test", Subject: "owner"})
|
||||
recorder := httptest.NewRecorder()
|
||||
ClaimDeviceHandler(nil).ServeHTTP(recorder, request)
|
||||
if recorder.Code != http.StatusBadRequest || !strings.Contains(recorder.Body.String(), "不能填写设备编号") {
|
||||
t.Fatalf("device identifier claim status=%d body=%s", recorder.Code, recorder.Body.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestSaaSErrorAndETagMapping(t *testing.T) {
|
||||
tests := []struct {
|
||||
err error
|
||||
|
||||
@@ -28,6 +28,7 @@ type Account struct {
|
||||
TenantName string `json:"tenant_name"`
|
||||
PolicyVersion string `json:"policy_version"`
|
||||
PlatformRole string `json:"platform_role"`
|
||||
AccountKind string `json:"account_kind"`
|
||||
DisplayName string `json:"display_name"`
|
||||
Memberships []HouseholdMembership `json:"household_memberships"`
|
||||
}
|
||||
@@ -60,6 +61,7 @@ func (r *Repository) CurrentUser(ctx context.Context, principal Principal) (Curr
|
||||
t.name,
|
||||
t.policy_version,
|
||||
ua.platform_role,
|
||||
ua.account_kind,
|
||||
COALESCE(ua.display_name, ''),
|
||||
COALESCE(
|
||||
jsonb_agg(
|
||||
@@ -95,6 +97,7 @@ func (r *Repository) CurrentUser(ctx context.Context, principal Principal) (Curr
|
||||
&account.TenantName,
|
||||
&account.PolicyVersion,
|
||||
&account.PlatformRole,
|
||||
&account.AccountKind,
|
||||
&account.DisplayName,
|
||||
&membershipsJSON,
|
||||
); err != nil {
|
||||
|
||||
@@ -42,6 +42,13 @@ func (p *DeleteProcessor) Handle(ctx context.Context, event outbox.Event) error
|
||||
if status != "PENDING_DELETE" {
|
||||
return nil
|
||||
}
|
||||
var unresolvedProviderRefs int
|
||||
if err := tx.QueryRow(ctx, `SELECT count(*) FROM memory_index_entry WHERE memory_id=$1::uuid AND index_type='provider' AND provider_ref IS NOT NULL AND status<>'DELETED'`, event.AggregateID).Scan(&unresolvedProviderRefs); err != nil {
|
||||
return fmt.Errorf("check provider memory deletion: %w", err)
|
||||
}
|
||||
if unresolvedProviderRefs > 0 {
|
||||
return outbox.Retryable(errors.New("provider memory deletion has not reconciled"))
|
||||
}
|
||||
if _, err := tx.Exec(ctx, `UPDATE memory_index_entry SET embedding=NULL,status='DELETED',provider_ref=NULL,indexed_at=NULL,last_error=NULL,updated_at=now() WHERE memory_id=$1::uuid`, event.AggregateID); err != nil {
|
||||
return fmt.Errorf("delete memory indexes: %w", err)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,161 @@
|
||||
package memorytool
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
"unicode/utf8"
|
||||
|
||||
"data-terminal-cloud/internal/audit"
|
||||
"data-terminal-cloud/internal/embeddingprovider"
|
||||
"data-terminal-cloud/internal/retrieval"
|
||||
|
||||
"github.com/jackc/pgx/v5"
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
)
|
||||
|
||||
var (
|
||||
ErrConversationUnavailable = errors.New("memory tool conversation is unavailable")
|
||||
ErrEngineModeUnsupported = errors.New("memory tool is unavailable for this engine mode")
|
||||
)
|
||||
|
||||
type Result struct {
|
||||
MemoryID string `json:"memory_id"`
|
||||
MemoryVersionID string `json:"memory_version_id"`
|
||||
VersionNo int64 `json:"version_no"`
|
||||
Content string `json:"content"`
|
||||
SourceRefs json.RawMessage `json:"source_refs"`
|
||||
Relevance float64 `json:"relevance"`
|
||||
}
|
||||
|
||||
type Response struct {
|
||||
Items []Result `json:"items"`
|
||||
Degraded bool `json:"degraded"`
|
||||
}
|
||||
|
||||
type Service struct {
|
||||
pool *pgxpool.Pool
|
||||
embedder embeddingprovider.Provider
|
||||
audit *audit.Repository
|
||||
now func() time.Time
|
||||
}
|
||||
|
||||
type conversationScope struct {
|
||||
internalID, publicID, tenantID, subjectID, taskRef, engineMode, status string
|
||||
}
|
||||
|
||||
func NewService(pool *pgxpool.Pool, embedder embeddingprovider.Provider, auditRepository *audit.Repository) (*Service, error) {
|
||||
if pool == nil || embedder == nil || auditRepository == nil {
|
||||
return nil, errors.New("memory tool database, embedding provider and audit repository are required")
|
||||
}
|
||||
return &Service{pool: pool, embedder: embedder, audit: auditRepository, now: time.Now}, nil
|
||||
}
|
||||
|
||||
func (s *Service) Search(ctx context.Context, claims Claims, query string, limit int, requestID string) (Response, error) {
|
||||
started := s.now().UTC()
|
||||
query = strings.TrimSpace(query)
|
||||
if query == "" || utf8.RuneCountInString(query) > 500 || limit < 1 || limit > 5 {
|
||||
return Response{}, errors.New("memory query and limit are invalid")
|
||||
}
|
||||
scope, err := s.resolveScope(ctx, claims)
|
||||
if err != nil {
|
||||
return Response{}, err
|
||||
}
|
||||
digest := sha256.Sum256([]byte(query))
|
||||
queryDigest := hex.EncodeToString(digest[:])
|
||||
embedding, err := s.embedder.Embed(ctx, query)
|
||||
if err != nil {
|
||||
s.recordCall(ctx, scope, queryDigest, 0, "FAILED", "EMBEDDING_UNAVAILABLE", started)
|
||||
return Response{Items: []Result{}, Degraded: true}, nil
|
||||
}
|
||||
vector, err := embeddingprovider.Literal(embedding.Vector)
|
||||
if err != nil {
|
||||
s.recordCall(ctx, scope, queryDigest, 0, "FAILED", "EMBEDDING_INVALID", started)
|
||||
return Response{Items: []Result{}, Degraded: true}, nil
|
||||
}
|
||||
tx, err := s.pool.BeginTx(ctx, pgx.TxOptions{AccessMode: pgx.ReadWrite})
|
||||
if err != nil {
|
||||
return Response{}, err
|
||||
}
|
||||
defer tx.Rollback(context.Background())
|
||||
memories, err := retrieval.SelectMemoryForContextVector(ctx, tx, scope.tenantID, scope.subjectID, vector, limit)
|
||||
if err != nil {
|
||||
return Response{}, err
|
||||
}
|
||||
items := make([]Result, 0, len(memories))
|
||||
totalRunes := 0
|
||||
for _, memory := range memories {
|
||||
content := trimRunes(memory.Content, 240)
|
||||
if totalRunes+utf8.RuneCountInString(content) > 800 {
|
||||
break
|
||||
}
|
||||
totalRunes += utf8.RuneCountInString(content)
|
||||
items = append(items, Result{MemoryID: memory.MemoryID, MemoryVersionID: memory.MemoryVersionID, VersionNo: memory.VersionNo, Content: content, SourceRefs: memory.SourceRefs, Relevance: memory.Relevance})
|
||||
if _, err := tx.Exec(ctx, `INSERT INTO conversation_memory_use(tenant_id,subject_id,conversation_id,memory_id,memory_version_id,use_type,relevance,source_refs)
|
||||
SELECT $1::uuid,$2::uuid,$3::uuid,cm.id,mv.id,'in_call_tool',$6,$7::jsonb FROM curated_memory cm JOIN memory_version mv ON mv.memory_id=cm.id
|
||||
WHERE cm.public_id=$4::uuid AND mv.public_id=$5::uuid
|
||||
ON CONFLICT(conversation_id,memory_version_id,use_type) DO UPDATE SET relevance=GREATEST(conversation_memory_use.relevance,EXCLUDED.relevance),used_at=now()`,
|
||||
scope.tenantID, scope.subjectID, scope.internalID, memory.MemoryID, memory.MemoryVersionID, memory.Relevance, memory.SourceRefs); err != nil {
|
||||
return Response{}, err
|
||||
}
|
||||
}
|
||||
status := "SUCCEEDED"
|
||||
if len(items) == 0 {
|
||||
status = "EMPTY"
|
||||
}
|
||||
latency := int(s.now().UTC().Sub(started).Milliseconds())
|
||||
if latency < 0 {
|
||||
latency = 0
|
||||
}
|
||||
if _, err := tx.Exec(ctx, `INSERT INTO memory_tool_call(tenant_id,subject_id,conversation_id,query_digest,result_count,status,latency_ms,engine_mode) VALUES($1::uuid,$2::uuid,$3::uuid,$4,$5,$6,$7,$8)`, scope.tenantID, scope.subjectID, scope.internalID, queryDigest, len(items), status, latency, scope.engineMode); err != nil {
|
||||
return Response{}, err
|
||||
}
|
||||
if err := s.audit.Record(ctx, tx, audit.Event{TenantID: &scope.tenantID, ActorType: "service", Action: "memory.tool.search", ResourceType: "conversation", ResourceID: &scope.publicID, RequestID: requestID, Result: "success", Detail: map[string]any{"result_count": len(items), "engine_mode": scope.engineMode}}); err != nil {
|
||||
return Response{}, err
|
||||
}
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return Response{}, err
|
||||
}
|
||||
return Response{Items: items, Degraded: false}, nil
|
||||
}
|
||||
|
||||
func (s *Service) resolveScope(ctx context.Context, claims Claims) (conversationScope, error) {
|
||||
var scope conversationScope
|
||||
err := s.pool.QueryRow(ctx, `SELECT c.id::text,c.public_id::text,c.tenant_id::text,c.subject_id::text,rt.provider_task_ref,c.engine_mode,c.status
|
||||
FROM conversation c JOIN rtc_task rt ON rt.conversation_id=c.id WHERE c.public_id=$1::uuid`, claims.ConversationID).
|
||||
Scan(&scope.internalID, &scope.publicID, &scope.tenantID, &scope.subjectID, &scope.taskRef, &scope.engineMode, &scope.status)
|
||||
if errors.Is(err, pgx.ErrNoRows) || scope.taskRef != claims.TaskRef {
|
||||
return conversationScope{}, ErrConversationUnavailable
|
||||
}
|
||||
if err != nil {
|
||||
return conversationScope{}, fmt.Errorf("resolve memory tool conversation: %w", err)
|
||||
}
|
||||
if scope.engineMode != "s2s_hybrid" {
|
||||
return conversationScope{}, ErrEngineModeUnsupported
|
||||
}
|
||||
if scope.status != "ACTIVE" && scope.status != "STARTING_AGENT" {
|
||||
return conversationScope{}, ErrConversationUnavailable
|
||||
}
|
||||
return scope, nil
|
||||
}
|
||||
|
||||
func (s *Service) recordCall(ctx context.Context, scope conversationScope, queryDigest string, resultCount int, status, errorCode string, started time.Time) {
|
||||
latency := int(s.now().UTC().Sub(started).Milliseconds())
|
||||
if latency < 0 {
|
||||
latency = 0
|
||||
}
|
||||
_, _ = s.pool.Exec(ctx, `INSERT INTO memory_tool_call(tenant_id,subject_id,conversation_id,query_digest,result_count,status,error_code,latency_ms,engine_mode) VALUES($1::uuid,$2::uuid,$3::uuid,$4,$5,$6,$7,$8,$9)`, scope.tenantID, scope.subjectID, scope.internalID, queryDigest, resultCount, status, errorCode, latency, scope.engineMode)
|
||||
}
|
||||
|
||||
func trimRunes(value string, limit int) string {
|
||||
runes := []rune(strings.TrimSpace(value))
|
||||
if len(runes) <= limit {
|
||||
return string(runes)
|
||||
}
|
||||
return string(runes[:limit]) + "…"
|
||||
}
|
||||
@@ -0,0 +1,84 @@
|
||||
package memorytool
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"os"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"data-terminal-cloud/internal/audit"
|
||||
"data-terminal-cloud/internal/developmentembedding"
|
||||
"data-terminal-cloud/internal/embeddingprovider"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
)
|
||||
|
||||
func TestSearchDerivesScopeAndRecordsActualMemoryUse(t *testing.T) {
|
||||
databaseURL := os.Getenv("TEST_DATABASE_URL")
|
||||
if databaseURL == "" {
|
||||
t.Skip("TEST_DATABASE_URL is not set")
|
||||
}
|
||||
ctx := context.Background()
|
||||
pool, err := pgxpool.New(ctx, databaseURL)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer pool.Close()
|
||||
contextID, conversationID, conversationPublicID := uuid.NewString(), uuid.NewString(), uuid.NewString()
|
||||
taskRef := "memory-tool-task-" + uuid.NewString()
|
||||
memoryID, memoryPublicID, versionID, versionPublicID := uuid.NewString(), uuid.NewString(), uuid.NewString(), uuid.NewString()
|
||||
defer func() {
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM audit_event WHERE request_id LIKE 'memory-tool-integration-%'`)
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM memory_tool_call WHERE conversation_id=$1::uuid`, conversationID)
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM conversation_memory_use WHERE conversation_id=$1::uuid`, conversationID)
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM rtc_task WHERE conversation_id=$1::uuid`, conversationID)
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM conversation WHERE id=$1::uuid`, conversationID)
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM context_snapshot WHERE id=$1::uuid`, contextID)
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM memory_index_entry WHERE memory_id=$1::uuid`, memoryID)
|
||||
_, _ = pool.Exec(ctx, `UPDATE curated_memory SET current_version_id=NULL WHERE id=$1::uuid`, memoryID)
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM memory_version WHERE memory_id=$1::uuid`, memoryID)
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM curated_memory WHERE id=$1::uuid`, memoryID)
|
||||
}()
|
||||
if _, err := pool.Exec(ctx, `INSERT INTO context_snapshot(id,tenant_id,subject_id,profile_version,consent_snapshot_id,memory_versions,prompt_bundle_version,engine_mode,content_digest,sanitized_summary)
|
||||
VALUES($1::uuid,'10000000-0000-0000-0000-000000000001','10000000-0000-0000-0000-000000000401',1,'10000000-0000-0000-0000-000000000702','[]','memory-tool-test','s2s_hybrid',$2,'')`, contextID, strings.Repeat("1", 64)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := pool.Exec(ctx, `INSERT INTO conversation(id,public_id,tenant_id,household_id,subject_id,device_id,consent_snapshot_id,context_snapshot_id,provider_task_ref,engine_mode,retention_mode,status,allocation_expires_at,started_at)
|
||||
VALUES($1::uuid,$2::uuid,'10000000-0000-0000-0000-000000000001','10000000-0000-0000-0000-000000000201','10000000-0000-0000-0000-000000000401','10000000-0000-0000-0000-000000000501','10000000-0000-0000-0000-000000000702',$3::uuid,$4,'s2s_hybrid','transcript_only','ACTIVE',now()+interval '1 hour',now())`, conversationID, conversationPublicID, contextID, taskRef); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := pool.Exec(ctx, `INSERT INTO rtc_task(conversation_id,provider,provider_task_ref,room_id,device_rtc_user_id,agent_rtc_user_id,status) VALUES($1::uuid,'mock',$2,$3,'device-user','agent-user','ACTIVE')`, conversationID, taskRef, "memory-room-"+conversationID); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
content := "长者曾经在河边小学教书。"
|
||||
if _, err := pool.Exec(ctx, `INSERT INTO curated_memory(id,public_id,tenant_id,subject_id,status,sensitivity,valid_from) VALUES($1::uuid,$2::uuid,'10000000-0000-0000-0000-000000000001','10000000-0000-0000-0000-000000000401','ACTIVE','normal',now())`, memoryID, memoryPublicID); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := pool.Exec(ctx, `INSERT INTO memory_version(id,public_id,memory_id,version_no,content,source_refs,confirmed_by,content_digest) VALUES($1::uuid,$2::uuid,$3::uuid,1,$4,'[{"type":"story","story_id":"synthetic"}]','10000000-0000-0000-0000-000000000101',$5)`, versionID, versionPublicID, memoryID, content, strings.Repeat("2", 64)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := pool.Exec(ctx, `UPDATE curated_memory SET current_version_id=$1::uuid WHERE id=$2::uuid`, versionID, memoryID); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := pool.Exec(ctx, `INSERT INTO memory_index_entry(tenant_id,subject_id,memory_id,memory_version_id,index_type,embedding,status,provider_ref) VALUES('10000000-0000-0000-0000-000000000001','10000000-0000-0000-0000-000000000401',$1::uuid,$2::uuid,'pgvector',$3::vector,'ACTIVE','integration')`, memoryID, versionID, developmentembedding.Literal(developmentembedding.Vector(content))); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
service, err := NewService(pool, embeddingprovider.Local{}, audit.NewRepository())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
claims := Claims{ConversationID: conversationPublicID, TaskRef: taskRef}
|
||||
result, err := service.Search(ctx, claims, "河边小学", 3, "memory-tool-integration-search")
|
||||
if err != nil || result.Degraded || len(result.Items) != 1 || result.Items[0].MemoryID != memoryPublicID || result.Items[0].MemoryVersionID != versionPublicID {
|
||||
t.Fatalf("result=%+v err=%v", result, err)
|
||||
}
|
||||
var calls, uses int
|
||||
if err := pool.QueryRow(ctx, `SELECT (SELECT count(*) FROM memory_tool_call WHERE conversation_id=$1::uuid AND status='SUCCEEDED'),(SELECT count(*) FROM conversation_memory_use WHERE conversation_id=$1::uuid AND use_type='in_call_tool')`, conversationID).Scan(&calls, &uses); err != nil || calls != 1 || uses != 1 {
|
||||
t.Fatalf("calls=%d uses=%d err=%v", calls, uses, err)
|
||||
}
|
||||
if _, err := service.Search(ctx, Claims{ConversationID: conversationPublicID, TaskRef: "wrong-task"}, "河边小学", 3, "memory-tool-integration-wrong-task"); !errors.Is(err, ErrConversationUnavailable) {
|
||||
t.Fatalf("wrong task error=%v", err)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,69 @@
|
||||
package memorytool
|
||||
|
||||
import (
|
||||
"crypto/hmac"
|
||||
"crypto/sha256"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
var ErrInvalidToken = errors.New("invalid memory tool token")
|
||||
|
||||
type Claims struct {
|
||||
ConversationID string `json:"conversation_id"`
|
||||
TaskRef string `json:"task_ref"`
|
||||
ExpiresAt int64 `json:"expires_at"`
|
||||
}
|
||||
|
||||
type TokenSigner struct {
|
||||
secret []byte
|
||||
now func() time.Time
|
||||
}
|
||||
|
||||
func NewTokenSigner(secret []byte) (*TokenSigner, error) {
|
||||
if len(secret) < 32 {
|
||||
return nil, errors.New("memory tool signing secret must contain at least 32 bytes")
|
||||
}
|
||||
return &TokenSigner{secret: append([]byte(nil), secret...), now: time.Now}, nil
|
||||
}
|
||||
|
||||
func (s *TokenSigner) Issue(conversationID, taskRef string, expiresAt time.Time) (string, error) {
|
||||
if strings.TrimSpace(conversationID) == "" || strings.TrimSpace(taskRef) == "" || !expiresAt.After(s.now()) {
|
||||
return "", ErrInvalidToken
|
||||
}
|
||||
payload, err := json.Marshal(Claims{ConversationID: conversationID, TaskRef: taskRef, ExpiresAt: expiresAt.UTC().Unix()})
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
encoded := base64.RawURLEncoding.EncodeToString(payload)
|
||||
return "v1." + encoded + "." + s.signature(encoded), nil
|
||||
}
|
||||
|
||||
func (s *TokenSigner) Verify(token string) (Claims, error) {
|
||||
parts := strings.Split(token, ".")
|
||||
if len(parts) != 3 || parts[0] != "v1" {
|
||||
return Claims{}, ErrInvalidToken
|
||||
}
|
||||
expected := s.signature(parts[1])
|
||||
if !hmac.Equal([]byte(expected), []byte(parts[2])) {
|
||||
return Claims{}, ErrInvalidToken
|
||||
}
|
||||
payload, err := base64.RawURLEncoding.DecodeString(parts[1])
|
||||
if err != nil {
|
||||
return Claims{}, ErrInvalidToken
|
||||
}
|
||||
var claims Claims
|
||||
if err := json.Unmarshal(payload, &claims); err != nil || claims.ConversationID == "" || claims.TaskRef == "" || claims.ExpiresAt <= s.now().UTC().Unix() {
|
||||
return Claims{}, ErrInvalidToken
|
||||
}
|
||||
return claims, nil
|
||||
}
|
||||
|
||||
func (s *TokenSigner) signature(payload string) string {
|
||||
mac := hmac.New(sha256.New, s.secret)
|
||||
_, _ = mac.Write([]byte("memory-tool-token-v1." + payload))
|
||||
return base64.RawURLEncoding.EncodeToString(mac.Sum(nil))
|
||||
}
|
||||
@@ -0,0 +1,31 @@
|
||||
package memorytool
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestTokenSignerBindsConversationTaskAndExpiry(t *testing.T) {
|
||||
signer, err := NewTokenSigner([]byte("01234567890123456789012345678901"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
now := time.Date(2026, 8, 28, 12, 0, 0, 0, time.UTC)
|
||||
signer.now = func() time.Time { return now }
|
||||
token, err := signer.Issue("conversation-one", "task-one", now.Add(time.Minute))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
claims, err := signer.Verify(token)
|
||||
if err != nil || claims.ConversationID != "conversation-one" || claims.TaskRef != "task-one" {
|
||||
t.Fatalf("claims=%+v err=%v", claims, err)
|
||||
}
|
||||
if _, err := signer.Verify(token + "changed"); !errors.Is(err, ErrInvalidToken) {
|
||||
t.Fatalf("tampered token error=%v", err)
|
||||
}
|
||||
signer.now = func() time.Time { return now.Add(2 * time.Minute) }
|
||||
if _, err := signer.Verify(token); !errors.Is(err, ErrInvalidToken) {
|
||||
t.Fatalf("expired token error=%v", err)
|
||||
}
|
||||
}
|
||||
@@ -107,7 +107,7 @@ func (s *Service) resolveActor(ctx context.Context, principal identity.Principal
|
||||
return actor{}, ErrInvalid
|
||||
}
|
||||
var a actor
|
||||
err := s.pool.QueryRow(ctx, `SELECT t.id::text,ua.id::text,ua.public_id::text,ua.platform_role FROM tenant t JOIN user_account ua ON ua.tenant_id=t.id AND ua.status='active' WHERE t.public_id=$1::uuid AND ua.oidc_issuer=$2 AND ua.oidc_subject=$3`, tenantPublicID, principal.Issuer, principal.Subject).Scan(&a.tenantID, &a.userID, &a.publicID, &a.role)
|
||||
err := s.pool.QueryRow(ctx, `SELECT t.id::text,ua.id::text,ua.public_id::text,ua.platform_role FROM tenant t JOIN user_account ua ON ua.tenant_id=t.id AND ua.status='active' AND ua.account_kind='operator' WHERE t.public_id=$1::uuid AND ua.oidc_issuer=$2 AND ua.oidc_subject=$3`, tenantPublicID, principal.Issuer, principal.Subject).Scan(&a.tenantID, &a.userID, &a.publicID, &a.role)
|
||||
if errors.Is(err, pgx.ErrNoRows) {
|
||||
return actor{}, ErrAccessDenied
|
||||
}
|
||||
@@ -267,7 +267,7 @@ type grant struct {
|
||||
|
||||
func (s *Service) loadGrant(ctx context.Context, tx pgx.Tx, principal identity.Principal, tenantPublicID, grantPublicID, conversationPublicID string) (grant, error) {
|
||||
var g grant
|
||||
err := tx.QueryRow(ctx, `SELECT r.id::text,r.tenant_id::text,ua.id::text,ua.public_id::text,r.requester_user_id::text,r.conversation_id::text,c.public_id::text,r.status,r.reason,r.capabilities,r.requested_expires_at FROM ops_sensitive_access_request r JOIN tenant t ON t.id=r.tenant_id JOIN conversation c ON c.id=r.conversation_id JOIN user_account ua ON ua.tenant_id=r.tenant_id AND ua.status='active' AND ua.platform_role<>'customer' AND ua.oidc_issuer=$3 AND ua.oidc_subject=$4 WHERE t.public_id=$1::uuid AND r.public_id=$2::uuid AND c.public_id=$5::uuid FOR UPDATE OF r`, tenantPublicID, grantPublicID, principal.Issuer, principal.Subject, conversationPublicID).Scan(&g.requestID, &g.tenantID, &g.actorID, &g.actorPublicID, &g.requesterID, &g.conversationID, &g.conversationPublicID, &g.status, &g.reason, &g.capabilities, &g.expiresAt)
|
||||
err := tx.QueryRow(ctx, `SELECT r.id::text,r.tenant_id::text,ua.id::text,ua.public_id::text,r.requester_user_id::text,r.conversation_id::text,c.public_id::text,r.status,r.reason,r.capabilities,r.requested_expires_at FROM ops_sensitive_access_request r JOIN tenant t ON t.id=r.tenant_id JOIN conversation c ON c.id=r.conversation_id JOIN user_account ua ON ua.tenant_id=r.tenant_id AND ua.status='active' AND ua.account_kind='operator' AND ua.oidc_issuer=$3 AND ua.oidc_subject=$4 WHERE t.public_id=$1::uuid AND r.public_id=$2::uuid AND c.public_id=$5::uuid FOR UPDATE OF r`, tenantPublicID, grantPublicID, principal.Issuer, principal.Subject, conversationPublicID).Scan(&g.requestID, &g.tenantID, &g.actorID, &g.actorPublicID, &g.requesterID, &g.conversationID, &g.conversationPublicID, &g.status, &g.reason, &g.capabilities, &g.expiresAt)
|
||||
if errors.Is(err, pgx.ErrNoRows) {
|
||||
return grant{}, ErrAccessDenied
|
||||
}
|
||||
|
||||
@@ -36,8 +36,8 @@ func TestTemporarySensitiveAccessRequiresTwoPeopleAndAuditsEveryRead(t *testing.
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
requester := identity.Principal{Issuer: "http://localhost:58180/realms/voice-local", Subject: "00000000-0000-0000-0000-000000000101"}
|
||||
approver := identity.Principal{Issuer: requester.Issuer, Subject: "00000000-0000-0000-0000-000000000102"}
|
||||
requester := identity.Principal{Issuer: "http://localhost:58180/realms/voice-local", Subject: "00000000-0000-0000-0000-000000000104"}
|
||||
approver := identity.Principal{Issuer: requester.Issuer, Subject: "00000000-0000-0000-0000-000000000105"}
|
||||
outsider := identity.Principal{Issuer: requester.Issuer, Subject: "00000000-0000-0000-0000-000000000103"}
|
||||
tenantPublicID, tenantID := "10000000-0000-0000-0000-000000000010", "10000000-0000-0000-0000-000000000001"
|
||||
conversationID, conversationPublicID := uuid.NewString(), uuid.NewString()
|
||||
|
||||
@@ -29,8 +29,8 @@ func TestAgentBundleRequiresStandardQualityEvidence(t *testing.T) {
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
admin := identity.Principal{Issuer: "http://localhost:58180/realms/voice-local", Subject: "00000000-0000-0000-0000-000000000101"}
|
||||
approver := identity.Principal{Issuer: admin.Issuer, Subject: "00000000-0000-0000-0000-000000000102"}
|
||||
admin := identity.Principal{Issuer: "http://localhost:58180/realms/voice-local", Subject: "00000000-0000-0000-0000-000000000104"}
|
||||
approver := identity.Principal{Issuer: admin.Issuer, Subject: "00000000-0000-0000-0000-000000000105"}
|
||||
tenantID := "10000000-0000-0000-0000-000000000010"
|
||||
configKey := "integration.agent.bundle." + uuid.NewString()
|
||||
invalidConfigKey := "integration.agent.bundle.invalid." + uuid.NewString()
|
||||
|
||||
@@ -0,0 +1,86 @@
|
||||
package opsconfig
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"data-terminal-cloud/internal/identity"
|
||||
)
|
||||
|
||||
type ContentQualitySummary struct {
|
||||
ModelRef string `json:"model_ref"`
|
||||
PromptVersion string `json:"prompt_version"`
|
||||
StoryCandidates int64 `json:"story_candidates"`
|
||||
FactCandidates int64 `json:"fact_candidates"`
|
||||
Confirmed int64 `json:"confirmed"`
|
||||
Edited int64 `json:"edited"`
|
||||
Rejected int64 `json:"rejected"`
|
||||
Disabled int64 `json:"disabled"`
|
||||
ErrorTypes map[string]int64 `json:"error_types"`
|
||||
}
|
||||
|
||||
func (s *Service) ContentQuality(ctx context.Context, principal identity.Principal, tenantPublicID, modelRef, promptVersion string) ([]ContentQualitySummary, error) {
|
||||
currentActor, err := s.resolveActor(ctx, principal, tenantPublicID, "ops_viewer", "ops_operator", "ops_approver", "ops_admin")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
modelRef = strings.TrimSpace(modelRef)
|
||||
promptVersion = strings.TrimSpace(promptVersion)
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
WITH generated AS (
|
||||
SELECT COALESCE(sr.model_ref,'unknown') model_ref,COALESCE(sr.prompt_version,'unknown') prompt_version,
|
||||
count(DISTINCT sd.id) story_candidates,count(DISTINCT fc.id) fact_candidates
|
||||
FROM story_draft sd JOIN story_revision sr ON sr.story_id=sd.id AND sr.revision_no=1
|
||||
LEFT JOIN fact_candidate fc ON fc.story_id=sd.id
|
||||
WHERE sd.tenant_id=$1::uuid AND (NULLIF($2,'') IS NULL OR sr.model_ref=$2) AND (NULLIF($3,'') IS NULL OR sr.prompt_version=$3)
|
||||
GROUP BY COALESCE(sr.model_ref,'unknown'),COALESCE(sr.prompt_version,'unknown')
|
||||
), signals AS (
|
||||
SELECT COALESCE(model_ref,'unknown') model_ref,COALESCE(prompt_version,'unknown') prompt_version,
|
||||
count(*) FILTER(WHERE action='CONFIRMED') confirmed,count(*) FILTER(WHERE action='EDITED') edited,
|
||||
count(*) FILTER(WHERE action='REJECTED') rejected,count(*) FILTER(WHERE action='DISABLED') disabled
|
||||
FROM content_review_signal WHERE tenant_id=$1::uuid
|
||||
AND (NULLIF($2,'') IS NULL OR model_ref=$2) AND (NULLIF($3,'') IS NULL OR prompt_version=$3)
|
||||
GROUP BY COALESCE(model_ref,'unknown'),COALESCE(prompt_version,'unknown')
|
||||
)
|
||||
SELECT COALESCE(g.model_ref,s.model_ref),COALESCE(g.prompt_version,s.prompt_version),
|
||||
COALESCE(g.story_candidates,0),COALESCE(g.fact_candidates,0),COALESCE(s.confirmed,0),COALESCE(s.edited,0),COALESCE(s.rejected,0),COALESCE(s.disabled,0)
|
||||
FROM generated g FULL JOIN signals s USING(model_ref,prompt_version)
|
||||
ORDER BY COALESCE(g.model_ref,s.model_ref),COALESCE(g.prompt_version,s.prompt_version)`, currentActor.tenantID, modelRef, promptVersion)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("query content quality summary: %w", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
items := make([]ContentQualitySummary, 0)
|
||||
lookup := make(map[string]int)
|
||||
for rows.Next() {
|
||||
var item ContentQualitySummary
|
||||
item.ErrorTypes = map[string]int64{}
|
||||
if err := rows.Scan(&item.ModelRef, &item.PromptVersion, &item.StoryCandidates, &item.FactCandidates, &item.Confirmed, &item.Edited, &item.Rejected, &item.Disabled); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
items = append(items, item)
|
||||
lookup[item.ModelRef+"\x00"+item.PromptVersion] = len(items) - 1
|
||||
}
|
||||
if err := rows.Err(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
errorRows, err := s.pool.Query(ctx, `SELECT COALESCE(model_ref,'unknown'),COALESCE(prompt_version,'unknown'),error_type,count(*) FROM content_review_signal
|
||||
WHERE tenant_id=$1::uuid AND error_type IS NOT NULL AND (NULLIF($2,'') IS NULL OR model_ref=$2) AND (NULLIF($3,'') IS NULL OR prompt_version=$3)
|
||||
GROUP BY COALESCE(model_ref,'unknown'),COALESCE(prompt_version,'unknown'),error_type`, currentActor.tenantID, modelRef, promptVersion)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer errorRows.Close()
|
||||
for errorRows.Next() {
|
||||
var model, prompt, errorType string
|
||||
var count int64
|
||||
if err := errorRows.Scan(&model, &prompt, &errorType, &count); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if index, ok := lookup[model+"\x00"+prompt]; ok {
|
||||
items[index].ErrorTypes[errorType] = count
|
||||
}
|
||||
}
|
||||
return items, errorRows.Err()
|
||||
}
|
||||
@@ -0,0 +1,53 @@
|
||||
package opsconfig
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"testing"
|
||||
|
||||
"data-terminal-cloud/internal/audit"
|
||||
"data-terminal-cloud/internal/identity"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
)
|
||||
|
||||
func TestContentQualityAggregatesOnlySelectedTenantAndVersion(t *testing.T) {
|
||||
databaseURL := os.Getenv("TEST_DATABASE_URL")
|
||||
if databaseURL == "" {
|
||||
t.Skip("TEST_DATABASE_URL is not set")
|
||||
}
|
||||
ctx := context.Background()
|
||||
pool, err := pgxpool.New(ctx, databaseURL)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer pool.Close()
|
||||
resourceIDs := []string{uuid.NewString(), uuid.NewString(), uuid.NewString()}
|
||||
defer func() {
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM content_review_signal WHERE resource_public_id=ANY($1::uuid[])`, resourceIDs)
|
||||
}()
|
||||
for index, action := range []string{"EDITED", "REJECTED", "CONFIRMED"} {
|
||||
errorType := any(nil)
|
||||
if index < 2 {
|
||||
errorType = "PERSON_ERROR"
|
||||
}
|
||||
if _, err := pool.Exec(ctx, `INSERT INTO content_review_signal(tenant_id,subject_id,resource_type,resource_public_id,action,error_type,reason,model_ref,prompt_version,generation_version,actor_id)
|
||||
VALUES('10000000-0000-0000-0000-000000000001','10000000-0000-0000-0000-000000000401','story',$1::uuid,$2,$3,$4,'quality-model-v1','quality-prompt-v2',1,'10000000-0000-0000-0000-000000000101')`, resourceIDs[index], action, errorType, "合成质量反馈"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
service, err := NewService(pool, audit.NewRepository())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
principal := identity.Principal{Issuer: "http://localhost:58180/realms/voice-local", Subject: "00000000-0000-0000-0000-000000000104"}
|
||||
items, err := service.ContentQuality(ctx, principal, "10000000-0000-0000-0000-000000000010", "quality-model-v1", "quality-prompt-v2")
|
||||
if err != nil || len(items) != 1 || items[0].Edited != 1 || items[0].Rejected != 1 || items[0].Confirmed != 1 || items[0].ErrorTypes["PERSON_ERROR"] != 2 {
|
||||
t.Fatalf("quality=%+v err=%v", items, err)
|
||||
}
|
||||
outsider := identity.Principal{Issuer: "http://localhost:58180/realms/voice-local", Subject: "00000000-0000-0000-0000-000000000103"}
|
||||
if _, err := service.ContentQuality(ctx, outsider, "10000000-0000-0000-0000-000000000010", "", ""); err != ErrAccessDenied {
|
||||
t.Fatalf("outsider quality error=%v", err)
|
||||
}
|
||||
}
|
||||
@@ -29,7 +29,7 @@ func TestResourceReferencesAndDeviceGroupsLifecycle(t *testing.T) {
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
admin := identity.Principal{Issuer: "http://localhost:58180/realms/voice-local", Subject: "00000000-0000-0000-0000-000000000101"}
|
||||
admin := identity.Principal{Issuer: "http://localhost:58180/realms/voice-local", Subject: "00000000-0000-0000-0000-000000000104"}
|
||||
outsider := identity.Principal{Issuer: admin.Issuer, Subject: "00000000-0000-0000-0000-000000000103"}
|
||||
tenantID := "10000000-0000-0000-0000-000000000010"
|
||||
deviceID := "10000000-0000-0000-0000-000000000511"
|
||||
|
||||
@@ -335,7 +335,7 @@ func (s *Service) resolveActor(ctx context.Context, principal identity.Principal
|
||||
return actor{}, ErrInvalid
|
||||
}
|
||||
var result actor
|
||||
err := s.pool.QueryRow(ctx, `SELECT t.id::text,t.public_id::text,ua.id::text,ua.public_id::text,ua.platform_role FROM user_account ua JOIN tenant t ON t.id=ua.tenant_id AND t.status='active' WHERE t.public_id=$1::uuid AND ua.oidc_issuer=$2 AND ua.oidc_subject=$3 AND ua.status='active'`, tenantPublicID, principal.Issuer, principal.Subject).Scan(&result.tenantID, &result.tenantPublicID, &result.userID, &result.userPublicID, &result.role)
|
||||
err := s.pool.QueryRow(ctx, `SELECT t.id::text,t.public_id::text,ua.id::text,ua.public_id::text,ua.platform_role FROM user_account ua JOIN tenant t ON t.id=ua.tenant_id AND t.status='active' WHERE t.public_id=$1::uuid AND ua.oidc_issuer=$2 AND ua.oidc_subject=$3 AND ua.status='active' AND ua.account_kind='operator'`, tenantPublicID, principal.Issuer, principal.Subject).Scan(&result.tenantID, &result.tenantPublicID, &result.userID, &result.userPublicID, &result.role)
|
||||
if errors.Is(err, pgx.ErrNoRows) {
|
||||
return actor{}, ErrAccessDenied
|
||||
}
|
||||
|
||||
@@ -31,8 +31,8 @@ func TestRoutingConfigurationApprovalPublishAndRollback(t *testing.T) {
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
admin := identity.Principal{Issuer: "http://localhost:58180/realms/voice-local", Subject: "00000000-0000-0000-0000-000000000101"}
|
||||
approver := identity.Principal{Issuer: admin.Issuer, Subject: "00000000-0000-0000-0000-000000000102"}
|
||||
admin := identity.Principal{Issuer: "http://localhost:58180/realms/voice-local", Subject: "00000000-0000-0000-0000-000000000104"}
|
||||
approver := identity.Principal{Issuer: admin.Issuer, Subject: "00000000-0000-0000-0000-000000000105"}
|
||||
outsider := identity.Principal{Issuer: admin.Issuer, Subject: "00000000-0000-0000-0000-000000000103"}
|
||||
tenantID := "10000000-0000-0000-0000-000000000010"
|
||||
configKey := "integration.routing." + uuid.NewString()
|
||||
|
||||
@@ -0,0 +1,542 @@
|
||||
package opsdevice
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"crypto/sha256"
|
||||
"encoding/base32"
|
||||
"encoding/hex"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"data-terminal-cloud/internal/audit"
|
||||
"data-terminal-cloud/internal/deviceauth"
|
||||
"data-terminal-cloud/internal/identity"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5"
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
)
|
||||
|
||||
var (
|
||||
ErrAccessDenied = errors.New("operator access denied")
|
||||
ErrInvalid = errors.New("invalid device management input")
|
||||
ErrNotFound = errors.New("device not found")
|
||||
ErrVersionConflict = errors.New("device version conflict")
|
||||
ErrClaimUnavailable = errors.New("device claim is unavailable")
|
||||
)
|
||||
|
||||
type Service struct {
|
||||
pool *pgxpool.Pool
|
||||
credential *deviceauth.DatabaseSecretResolver
|
||||
audit *audit.Repository
|
||||
}
|
||||
|
||||
type Device struct {
|
||||
ID string `json:"id"`
|
||||
TenantID string `json:"tenant_id"`
|
||||
Model string `json:"model"`
|
||||
HardwareRevision *string `json:"hardware_revision,omitempty"`
|
||||
Status string `json:"status"`
|
||||
MinimumSecurityVersion *string `json:"minimum_security_version,omitempty"`
|
||||
MinimumFirmwareVersion *string `json:"minimum_firmware_version,omitempty"`
|
||||
HouseholdID *string `json:"household_id,omitempty"`
|
||||
HouseholdName *string `json:"household_name,omitempty"`
|
||||
SubjectID *string `json:"subject_id,omitempty"`
|
||||
SubjectName *string `json:"subject_name,omitempty"`
|
||||
CredentialStatus string `json:"credential_status"`
|
||||
KeyVersion int64 `json:"key_version"`
|
||||
LastReportedAt *time.Time `json:"last_reported_at,omitempty"`
|
||||
RegisteredAt time.Time `json:"registered_at"`
|
||||
UpdatedAt time.Time `json:"updated_at"`
|
||||
Version int64 `json:"version"`
|
||||
}
|
||||
|
||||
type AssignmentOption struct {
|
||||
HouseholdID string `json:"household_id"`
|
||||
HouseholdName string `json:"household_name"`
|
||||
SubjectID string `json:"subject_id"`
|
||||
SubjectName string `json:"subject_name"`
|
||||
}
|
||||
|
||||
type CreateInput struct {
|
||||
Model string
|
||||
HardwareRevision string
|
||||
MinimumSecurityVersion string
|
||||
MinimumFirmwareVersion string
|
||||
HouseholdID string
|
||||
SubjectID string
|
||||
}
|
||||
|
||||
type UpdateInput struct {
|
||||
Model *string
|
||||
HardwareRevision *string
|
||||
MinimumSecurityVersion *string
|
||||
MinimumFirmwareVersion *string
|
||||
Status *string
|
||||
HouseholdID *string
|
||||
SubjectID *string
|
||||
}
|
||||
|
||||
type CredentialIssue struct {
|
||||
DeviceID string `json:"device_id"`
|
||||
Secret string `json:"secret"`
|
||||
KeyVersion int64 `json:"key_version"`
|
||||
IssuedAt time.Time `json:"issued_at"`
|
||||
}
|
||||
|
||||
type CreateResult struct {
|
||||
Device Device `json:"device"`
|
||||
Credential CredentialIssue `json:"credential"`
|
||||
Claim *DeviceClaimIssue `json:"claim,omitempty"`
|
||||
}
|
||||
|
||||
type DeviceClaimIssue struct {
|
||||
DeviceID string `json:"device_id"`
|
||||
ClaimCode string `json:"claim_code"`
|
||||
IssuedAt time.Time `json:"issued_at"`
|
||||
ExpiresAt time.Time `json:"expires_at"`
|
||||
}
|
||||
|
||||
type operatorAccess struct {
|
||||
tenantID, tenantPublicID, userID, userPublicID, role string
|
||||
}
|
||||
|
||||
type queryRow interface {
|
||||
QueryRow(context.Context, string, ...any) pgx.Row
|
||||
}
|
||||
|
||||
func NewService(pool *pgxpool.Pool, credential *deviceauth.DatabaseSecretResolver, auditRepository *audit.Repository) (*Service, error) {
|
||||
if pool == nil || credential == nil || auditRepository == nil {
|
||||
return nil, errors.New("operations device service dependencies are required")
|
||||
}
|
||||
return &Service{pool: pool, credential: credential, audit: auditRepository}, nil
|
||||
}
|
||||
|
||||
func (s *Service) List(ctx context.Context, principal identity.Principal, tenantPublicID string) ([]Device, error) {
|
||||
access, err := requireAccess(ctx, s.pool, principal, tenantPublicID, false)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
rows, err := s.pool.Query(ctx, deviceSelectSQL+` WHERE d.tenant_id=$1::uuid ORDER BY d.registered_at DESC,d.public_id`, access.tenantID)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("list operations devices: %w", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
items := make([]Device, 0)
|
||||
for rows.Next() {
|
||||
item, scanErr := scanDevice(rows)
|
||||
if scanErr != nil {
|
||||
return nil, scanErr
|
||||
}
|
||||
items = append(items, item)
|
||||
}
|
||||
return items, rows.Err()
|
||||
}
|
||||
|
||||
func (s *Service) Options(ctx context.Context, principal identity.Principal, tenantPublicID string) ([]AssignmentOption, error) {
|
||||
access, err := requireAccess(ctx, s.pool, principal, tenantPublicID, false)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT h.public_id::text,h.name,sp.public_id::text,sp.preferred_name
|
||||
FROM household h JOIN subject_profile sp ON sp.household_id=h.id AND sp.tenant_id=h.tenant_id
|
||||
WHERE h.tenant_id=$1::uuid AND h.status='active' AND sp.status IN ('active','suspended')
|
||||
ORDER BY h.name,sp.preferred_name`, access.tenantID)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("list device assignment options: %w", err)
|
||||
}
|
||||
defer rows.Close()
|
||||
items := make([]AssignmentOption, 0)
|
||||
for rows.Next() {
|
||||
var item AssignmentOption
|
||||
if err := rows.Scan(&item.HouseholdID, &item.HouseholdName, &item.SubjectID, &item.SubjectName); err != nil {
|
||||
return nil, fmt.Errorf("scan device assignment option: %w", err)
|
||||
}
|
||||
items = append(items, item)
|
||||
}
|
||||
return items, rows.Err()
|
||||
}
|
||||
|
||||
func (s *Service) Create(ctx context.Context, principal identity.Principal, tenantPublicID, requestID string, input CreateInput) (CreateResult, error) {
|
||||
input.Model = strings.TrimSpace(input.Model)
|
||||
input.HardwareRevision = strings.TrimSpace(input.HardwareRevision)
|
||||
input.MinimumSecurityVersion = strings.TrimSpace(input.MinimumSecurityVersion)
|
||||
input.MinimumFirmwareVersion = strings.TrimSpace(input.MinimumFirmwareVersion)
|
||||
if input.Model == "" || len([]rune(input.Model)) > 120 || len([]rune(input.HardwareRevision)) > 80 ||
|
||||
len([]rune(input.MinimumSecurityVersion)) > 80 || len([]rune(input.MinimumFirmwareVersion)) > 80 ||
|
||||
((input.HouseholdID == "") != (input.SubjectID == "")) {
|
||||
return CreateResult{}, ErrInvalid
|
||||
}
|
||||
secretBytes := make([]byte, 32)
|
||||
serialBytes := make([]byte, 32)
|
||||
if _, err := rand.Read(secretBytes); err != nil {
|
||||
return CreateResult{}, fmt.Errorf("generate device secret: %w", err)
|
||||
}
|
||||
if _, err := rand.Read(serialBytes); err != nil {
|
||||
return CreateResult{}, fmt.Errorf("generate device serial material: %w", err)
|
||||
}
|
||||
secret := hex.EncodeToString(secretBytes)
|
||||
serialDigest := sha256.Sum256(serialBytes)
|
||||
publicID := uuid.NewString()
|
||||
credentialRef := "device-secret://" + publicID
|
||||
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return CreateResult{}, fmt.Errorf("begin device registration: %w", err)
|
||||
}
|
||||
defer tx.Rollback(ctx)
|
||||
access, err := requireAccess(ctx, tx, principal, tenantPublicID, true)
|
||||
if err != nil {
|
||||
return CreateResult{}, err
|
||||
}
|
||||
if input.HouseholdID != "" {
|
||||
if err := validateAssignment(ctx, tx, access.tenantID, input.HouseholdID, input.SubjectID); err != nil {
|
||||
return CreateResult{}, err
|
||||
}
|
||||
}
|
||||
var deviceID string
|
||||
err = tx.QueryRow(ctx, `
|
||||
INSERT INTO device(public_id,tenant_id,serial_digest,model,hardware_revision,status,credential_ref,minimum_security_version,minimum_firmware_version)
|
||||
VALUES($1::uuid,$2::uuid,$3,$4,NULLIF($5,''),'registered',$6,NULLIF($7,''),NULLIF($8,'')) RETURNING id::text`,
|
||||
publicID, access.tenantID, hex.EncodeToString(serialDigest[:]), input.Model, input.HardwareRevision, credentialRef,
|
||||
input.MinimumSecurityVersion, input.MinimumFirmwareVersion).Scan(&deviceID)
|
||||
if err != nil {
|
||||
return CreateResult{}, fmt.Errorf("insert device: %w", err)
|
||||
}
|
||||
if _, err := tx.Exec(ctx, `INSERT INTO device_twin(device_id,tenant_id) VALUES($1::uuid,$2::uuid)`, deviceID, access.tenantID); err != nil {
|
||||
return CreateResult{}, fmt.Errorf("insert device twin: %w", err)
|
||||
}
|
||||
if input.HouseholdID != "" {
|
||||
if err := insertBinding(ctx, tx, access, deviceID, input.HouseholdID, input.SubjectID, "运维端登记设备"); err != nil {
|
||||
return CreateResult{}, err
|
||||
}
|
||||
}
|
||||
keyVersion, err := s.credential.ProvisionTx(ctx, tx, publicID, []byte(secret), false)
|
||||
if err != nil {
|
||||
return CreateResult{}, err
|
||||
}
|
||||
if err := s.audit.Record(ctx, tx, audit.Event{TenantID: &access.tenantID, ActorType: "user", ActorPublicID: &access.userPublicID,
|
||||
Action: "ops.device.create", ResourceType: "device", ResourceID: &publicID, RequestID: requestID, Result: "success",
|
||||
Detail: map[string]any{"model": input.Model, "assigned": input.HouseholdID != "", "key_version": keyVersion}}); err != nil {
|
||||
return CreateResult{}, err
|
||||
}
|
||||
var claim *DeviceClaimIssue
|
||||
if input.HouseholdID == "" {
|
||||
issued, err := s.issueDeviceClaimTx(ctx, tx, access, publicID, deviceID, requestID)
|
||||
if err != nil {
|
||||
return CreateResult{}, err
|
||||
}
|
||||
claim = &issued
|
||||
}
|
||||
item, err := readDevice(ctx, tx, access.tenantID, publicID)
|
||||
if err != nil {
|
||||
return CreateResult{}, err
|
||||
}
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return CreateResult{}, fmt.Errorf("commit device registration: %w", err)
|
||||
}
|
||||
return CreateResult{Device: item, Credential: CredentialIssue{DeviceID: publicID, Secret: secret, KeyVersion: keyVersion, IssuedAt: time.Now().UTC()}, Claim: claim}, nil
|
||||
}
|
||||
|
||||
func (s *Service) IssueDeviceClaim(ctx context.Context, principal identity.Principal, tenantPublicID, devicePublicID, requestID string) (DeviceClaimIssue, error) {
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return DeviceClaimIssue{}, fmt.Errorf("begin device claim issue: %w", err)
|
||||
}
|
||||
defer tx.Rollback(ctx)
|
||||
access, err := requireAccess(ctx, tx, principal, tenantPublicID, true)
|
||||
if err != nil {
|
||||
return DeviceClaimIssue{}, err
|
||||
}
|
||||
var deviceID, status string
|
||||
var bound bool
|
||||
if err := tx.QueryRow(ctx, `
|
||||
SELECT d.id::text,d.status,EXISTS(SELECT 1 FROM device_binding b WHERE b.device_id=d.id AND b.unbound_at IS NULL)
|
||||
FROM device d WHERE d.public_id=$1::uuid AND d.tenant_id=$2::uuid FOR UPDATE`, devicePublicID, access.tenantID).Scan(&deviceID, &status, &bound); errors.Is(err, pgx.ErrNoRows) {
|
||||
return DeviceClaimIssue{}, ErrNotFound
|
||||
} else if err != nil {
|
||||
return DeviceClaimIssue{}, fmt.Errorf("lock device for claim issue: %w", err)
|
||||
}
|
||||
if status != "registered" || bound {
|
||||
return DeviceClaimIssue{}, ErrClaimUnavailable
|
||||
}
|
||||
issued, err := s.issueDeviceClaimTx(ctx, tx, access, devicePublicID, deviceID, requestID)
|
||||
if err != nil {
|
||||
return DeviceClaimIssue{}, err
|
||||
}
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return DeviceClaimIssue{}, fmt.Errorf("commit device claim issue: %w", err)
|
||||
}
|
||||
return issued, nil
|
||||
}
|
||||
|
||||
func (s *Service) issueDeviceClaimTx(ctx context.Context, tx pgx.Tx, access operatorAccess, devicePublicID, deviceID, requestID string) (DeviceClaimIssue, error) {
|
||||
randomBytes := make([]byte, 10)
|
||||
if _, err := rand.Read(randomBytes); err != nil {
|
||||
return DeviceClaimIssue{}, fmt.Errorf("generate device claim: %w", err)
|
||||
}
|
||||
rawCode := base32.StdEncoding.WithPadding(base32.NoPadding).EncodeToString(randomBytes)
|
||||
claimCode := strings.Join([]string{rawCode[0:4], rawCode[4:8], rawCode[8:12], rawCode[12:16]}, "-")
|
||||
digest := sha256.Sum256([]byte(claimCode))
|
||||
issuedAt := time.Now().UTC()
|
||||
expiresAt := issuedAt.Add(24 * time.Hour)
|
||||
if _, err := tx.Exec(ctx, `UPDATE device_claim SET revoked_at=$1 WHERE device_id=$2::uuid AND used_at IS NULL AND revoked_at IS NULL`, issuedAt, deviceID); err != nil {
|
||||
return DeviceClaimIssue{}, fmt.Errorf("revoke previous device claim: %w", err)
|
||||
}
|
||||
if _, err := tx.Exec(ctx, `INSERT INTO device_claim(tenant_id,device_id,token_digest,expires_at,created_at) VALUES($1::uuid,$2::uuid,$3,$4,$5)`, access.tenantID, deviceID, hex.EncodeToString(digest[:]), expiresAt, issuedAt); err != nil {
|
||||
return DeviceClaimIssue{}, fmt.Errorf("insert device claim: %w", err)
|
||||
}
|
||||
if err := s.audit.Record(ctx, tx, audit.Event{TenantID: &access.tenantID, ActorType: "user", ActorPublicID: &access.userPublicID,
|
||||
Action: "ops.device.claim.issue", ResourceType: "device", ResourceID: &devicePublicID, RequestID: requestID, Result: "success",
|
||||
Detail: map[string]any{"expires_at": expiresAt}}); err != nil {
|
||||
return DeviceClaimIssue{}, err
|
||||
}
|
||||
return DeviceClaimIssue{DeviceID: devicePublicID, ClaimCode: claimCode, IssuedAt: issuedAt, ExpiresAt: expiresAt}, nil
|
||||
}
|
||||
|
||||
func (s *Service) Update(ctx context.Context, principal identity.Principal, tenantPublicID, devicePublicID, requestID string, expectedVersion int64, input UpdateInput) (Device, error) {
|
||||
if expectedVersion < 1 || (input.Model == nil && input.HardwareRevision == nil && input.MinimumSecurityVersion == nil && input.MinimumFirmwareVersion == nil && input.Status == nil && input.HouseholdID == nil && input.SubjectID == nil) {
|
||||
return Device{}, ErrInvalid
|
||||
}
|
||||
if (input.HouseholdID == nil) != (input.SubjectID == nil) {
|
||||
return Device{}, ErrInvalid
|
||||
}
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return Device{}, fmt.Errorf("begin device update: %w", err)
|
||||
}
|
||||
defer tx.Rollback(ctx)
|
||||
access, err := requireAccess(ctx, tx, principal, tenantPublicID, true)
|
||||
if err != nil {
|
||||
return Device{}, err
|
||||
}
|
||||
var deviceID string
|
||||
var currentVersion int64
|
||||
if err := tx.QueryRow(ctx, `SELECT id::text,version FROM device WHERE public_id=$1::uuid AND tenant_id=$2::uuid FOR UPDATE`, devicePublicID, access.tenantID).Scan(&deviceID, ¤tVersion); errors.Is(err, pgx.ErrNoRows) {
|
||||
return Device{}, ErrNotFound
|
||||
} else if err != nil {
|
||||
return Device{}, fmt.Errorf("lock device: %w", err)
|
||||
}
|
||||
if currentVersion != expectedVersion {
|
||||
return Device{}, ErrVersionConflict
|
||||
}
|
||||
model, modelSet := pointerValue(input.Model)
|
||||
hardware, hardwareSet := pointerValue(input.HardwareRevision)
|
||||
security, securitySet := pointerValue(input.MinimumSecurityVersion)
|
||||
firmware, firmwareSet := pointerValue(input.MinimumFirmwareVersion)
|
||||
status, statusSet := pointerValue(input.Status)
|
||||
if (modelSet && (model == "" || len([]rune(model)) > 120)) || len([]rune(hardware)) > 80 || len([]rune(security)) > 80 || len([]rune(firmware)) > 80 ||
|
||||
(statusSet && status != "registered" && status != "quarantined") {
|
||||
return Device{}, ErrInvalid
|
||||
}
|
||||
if _, err := tx.Exec(ctx, `UPDATE device SET
|
||||
model=CASE WHEN $3 THEN $4 ELSE model END,
|
||||
hardware_revision=CASE WHEN $5 THEN NULLIF($6,'') ELSE hardware_revision END,
|
||||
minimum_security_version=CASE WHEN $7 THEN NULLIF($8,'') ELSE minimum_security_version END,
|
||||
minimum_firmware_version=CASE WHEN $9 THEN NULLIF($10,'') ELSE minimum_firmware_version END,
|
||||
status=CASE WHEN $11 THEN $12 ELSE status END,version=version+1,updated_at=now()
|
||||
WHERE id=$1::uuid AND tenant_id=$2::uuid`, deviceID, access.tenantID, modelSet, model, hardwareSet, hardware, securitySet, security, firmwareSet, firmware, statusSet, status); err != nil {
|
||||
return Device{}, fmt.Errorf("update device: %w", err)
|
||||
}
|
||||
if input.HouseholdID != nil {
|
||||
householdID := strings.TrimSpace(*input.HouseholdID)
|
||||
subjectID := strings.TrimSpace(*input.SubjectID)
|
||||
if (householdID == "") != (subjectID == "") {
|
||||
return Device{}, ErrInvalid
|
||||
}
|
||||
if householdID != "" {
|
||||
if err := validateAssignment(ctx, tx, access.tenantID, householdID, subjectID); err != nil {
|
||||
return Device{}, err
|
||||
}
|
||||
}
|
||||
if _, err := tx.Exec(ctx, `UPDATE device_binding SET unbound_at=now(),unbound_by=$2::uuid,reason='运维端变更归属' WHERE device_id=$1::uuid AND unbound_at IS NULL`, deviceID, access.userID); err != nil {
|
||||
return Device{}, fmt.Errorf("close device binding: %w", err)
|
||||
}
|
||||
if householdID != "" {
|
||||
if err := insertBinding(ctx, tx, access, deviceID, householdID, subjectID, "运维端变更归属"); err != nil {
|
||||
return Device{}, err
|
||||
}
|
||||
if _, err := tx.Exec(ctx, `UPDATE device_claim SET revoked_at=now() WHERE device_id=$1::uuid AND used_at IS NULL AND revoked_at IS NULL`, deviceID); err != nil {
|
||||
return Device{}, fmt.Errorf("revoke device claims after assignment: %w", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
if err := s.audit.Record(ctx, tx, audit.Event{TenantID: &access.tenantID, ActorType: "user", ActorPublicID: &access.userPublicID,
|
||||
Action: "ops.device.update", ResourceType: "device", ResourceID: &devicePublicID, RequestID: requestID, Result: "success",
|
||||
Detail: map[string]any{"from_version": currentVersion, "to_version": currentVersion + 1}}); err != nil {
|
||||
return Device{}, err
|
||||
}
|
||||
item, err := readDevice(ctx, tx, access.tenantID, devicePublicID)
|
||||
if err != nil {
|
||||
return Device{}, err
|
||||
}
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return Device{}, fmt.Errorf("commit device update: %w", err)
|
||||
}
|
||||
return item, nil
|
||||
}
|
||||
|
||||
func (s *Service) Delete(ctx context.Context, principal identity.Principal, tenantPublicID, devicePublicID, requestID string, expectedVersion int64) (Device, error) {
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return Device{}, fmt.Errorf("begin device retirement: %w", err)
|
||||
}
|
||||
defer tx.Rollback(ctx)
|
||||
access, err := requireAccess(ctx, tx, principal, tenantPublicID, true)
|
||||
if err != nil {
|
||||
return Device{}, err
|
||||
}
|
||||
var deviceID string
|
||||
var version int64
|
||||
if err := tx.QueryRow(ctx, `SELECT id::text,version FROM device WHERE public_id=$1::uuid AND tenant_id=$2::uuid FOR UPDATE`, devicePublicID, access.tenantID).Scan(&deviceID, &version); errors.Is(err, pgx.ErrNoRows) {
|
||||
return Device{}, ErrNotFound
|
||||
} else if err != nil {
|
||||
return Device{}, fmt.Errorf("lock device for retirement: %w", err)
|
||||
}
|
||||
if version != expectedVersion {
|
||||
return Device{}, ErrVersionConflict
|
||||
}
|
||||
if _, err := tx.Exec(ctx, `UPDATE device SET status='revoked',revoked_at=COALESCE(revoked_at,now()),version=version+1,updated_at=now() WHERE id=$1::uuid`, deviceID); err != nil {
|
||||
return Device{}, fmt.Errorf("retire device: %w", err)
|
||||
}
|
||||
if _, err := tx.Exec(ctx, `UPDATE device_credential SET status='revoked',revoked_at=COALESCE(revoked_at,now()) WHERE device_id=$1::uuid`, deviceID); err != nil {
|
||||
return Device{}, fmt.Errorf("revoke device credential: %w", err)
|
||||
}
|
||||
if _, err := tx.Exec(ctx, `UPDATE device_binding SET unbound_at=now(),unbound_by=$2::uuid,reason='运维端删除设备' WHERE device_id=$1::uuid AND unbound_at IS NULL`, deviceID, access.userID); err != nil {
|
||||
return Device{}, fmt.Errorf("close retired device binding: %w", err)
|
||||
}
|
||||
if _, err := tx.Exec(ctx, `UPDATE device_claim SET revoked_at=now() WHERE device_id=$1::uuid AND used_at IS NULL AND revoked_at IS NULL`, deviceID); err != nil {
|
||||
return Device{}, fmt.Errorf("revoke retired device claims: %w", err)
|
||||
}
|
||||
if err := s.audit.Record(ctx, tx, audit.Event{TenantID: &access.tenantID, ActorType: "user", ActorPublicID: &access.userPublicID,
|
||||
Action: "ops.device.delete", ResourceType: "device", ResourceID: &devicePublicID, RequestID: requestID, Result: "success",
|
||||
Detail: map[string]any{"soft_delete": true, "credential_revoked": true}}); err != nil {
|
||||
return Device{}, err
|
||||
}
|
||||
item, err := readDevice(ctx, tx, access.tenantID, devicePublicID)
|
||||
if err != nil {
|
||||
return Device{}, err
|
||||
}
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return Device{}, fmt.Errorf("commit device retirement: %w", err)
|
||||
}
|
||||
return item, nil
|
||||
}
|
||||
|
||||
func (s *Service) RotateCredential(ctx context.Context, principal identity.Principal, tenantPublicID, devicePublicID, requestID string) (CredentialIssue, error) {
|
||||
secretBytes := make([]byte, 32)
|
||||
if _, err := rand.Read(secretBytes); err != nil {
|
||||
return CredentialIssue{}, fmt.Errorf("generate device credential: %w", err)
|
||||
}
|
||||
secret := hex.EncodeToString(secretBytes)
|
||||
tx, err := s.pool.Begin(ctx)
|
||||
if err != nil {
|
||||
return CredentialIssue{}, fmt.Errorf("begin credential rotation: %w", err)
|
||||
}
|
||||
defer tx.Rollback(ctx)
|
||||
access, err := requireAccess(ctx, tx, principal, tenantPublicID, true)
|
||||
if err != nil {
|
||||
return CredentialIssue{}, err
|
||||
}
|
||||
version, err := s.credential.ProvisionTx(ctx, tx, devicePublicID, []byte(secret), true)
|
||||
if errors.Is(err, deviceauth.ErrDeviceNotFound) {
|
||||
return CredentialIssue{}, ErrNotFound
|
||||
}
|
||||
if err != nil {
|
||||
return CredentialIssue{}, err
|
||||
}
|
||||
if err := s.audit.Record(ctx, tx, audit.Event{TenantID: &access.tenantID, ActorType: "user", ActorPublicID: &access.userPublicID,
|
||||
Action: "ops.device.credential.rotate", ResourceType: "device", ResourceID: &devicePublicID, RequestID: requestID, Result: "success",
|
||||
Detail: map[string]any{"key_version": version}}); err != nil {
|
||||
return CredentialIssue{}, err
|
||||
}
|
||||
if err := tx.Commit(ctx); err != nil {
|
||||
return CredentialIssue{}, fmt.Errorf("commit credential rotation: %w", err)
|
||||
}
|
||||
return CredentialIssue{DeviceID: devicePublicID, Secret: secret, KeyVersion: version, IssuedAt: time.Now().UTC()}, nil
|
||||
}
|
||||
|
||||
func requireAccess(ctx context.Context, q queryRow, principal identity.Principal, tenantPublicID string, mutate bool) (operatorAccess, error) {
|
||||
var access operatorAccess
|
||||
err := q.QueryRow(ctx, `SELECT t.id::text,t.public_id::text,ua.id::text,ua.public_id::text,ua.platform_role
|
||||
FROM tenant t JOIN user_account ua ON ua.tenant_id=t.id AND ua.status='active' AND ua.account_kind='operator'
|
||||
WHERE t.public_id=$1::uuid AND t.status='active' AND ua.oidc_issuer=$2 AND ua.oidc_subject=$3`, tenantPublicID, principal.Issuer, principal.Subject).
|
||||
Scan(&access.tenantID, &access.tenantPublicID, &access.userID, &access.userPublicID, &access.role)
|
||||
if errors.Is(err, pgx.ErrNoRows) {
|
||||
return operatorAccess{}, ErrAccessDenied
|
||||
}
|
||||
if err != nil {
|
||||
return operatorAccess{}, fmt.Errorf("resolve operations device access: %w", err)
|
||||
}
|
||||
if mutate && access.role != "ops_operator" && access.role != "ops_admin" {
|
||||
return operatorAccess{}, ErrAccessDenied
|
||||
}
|
||||
return access, nil
|
||||
}
|
||||
|
||||
func validateAssignment(ctx context.Context, q queryRow, tenantID, householdPublicID, subjectPublicID string) error {
|
||||
var exists bool
|
||||
err := q.QueryRow(ctx, `SELECT true FROM household h JOIN subject_profile sp ON sp.household_id=h.id AND sp.tenant_id=h.tenant_id
|
||||
WHERE h.public_id=$1::uuid AND sp.public_id=$2::uuid AND h.tenant_id=$3::uuid AND h.status='active' AND sp.status IN ('active','suspended')`,
|
||||
householdPublicID, subjectPublicID, tenantID).Scan(&exists)
|
||||
if errors.Is(err, pgx.ErrNoRows) {
|
||||
return ErrInvalid
|
||||
}
|
||||
if err != nil {
|
||||
return fmt.Errorf("validate device assignment: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func insertBinding(ctx context.Context, tx pgx.Tx, access operatorAccess, deviceID, householdPublicID, subjectPublicID, reason string) error {
|
||||
_, err := tx.Exec(ctx, `INSERT INTO device_binding(tenant_id,device_id,household_id,subject_id,bound_by,reason)
|
||||
SELECT $1::uuid,$2::uuid,h.id,sp.id,$5::uuid,$6 FROM household h JOIN subject_profile sp ON sp.household_id=h.id AND sp.tenant_id=h.tenant_id
|
||||
WHERE h.public_id=$3::uuid AND sp.public_id=$4::uuid AND h.tenant_id=$1::uuid`,
|
||||
access.tenantID, deviceID, householdPublicID, subjectPublicID, access.userID, reason)
|
||||
if err != nil {
|
||||
return fmt.Errorf("insert device binding: %w", err)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func pointerValue(value *string) (string, bool) {
|
||||
if value == nil {
|
||||
return "", false
|
||||
}
|
||||
return strings.TrimSpace(*value), true
|
||||
}
|
||||
|
||||
const deviceSelectSQL = `
|
||||
SELECT d.public_id::text,t.public_id::text,d.model,d.hardware_revision,d.status,d.minimum_security_version,d.minimum_firmware_version,
|
||||
h.public_id::text,h.name,sp.public_id::text,sp.preferred_name,COALESCE(dc.status,'missing'),COALESCE(dc.key_version,0),
|
||||
dt.last_reported_at,d.registered_at,d.updated_at,d.version
|
||||
FROM device d JOIN tenant t ON t.id=d.tenant_id
|
||||
LEFT JOIN device_twin dt ON dt.device_id=d.id AND dt.tenant_id=d.tenant_id
|
||||
LEFT JOIN device_binding b ON b.device_id=d.id AND b.tenant_id=d.tenant_id AND b.unbound_at IS NULL
|
||||
LEFT JOIN household h ON h.id=b.household_id AND h.tenant_id=b.tenant_id
|
||||
LEFT JOIN subject_profile sp ON sp.id=b.subject_id AND sp.tenant_id=b.tenant_id
|
||||
LEFT JOIN device_credential dc ON dc.device_id=d.id AND dc.tenant_id=d.tenant_id`
|
||||
|
||||
type scanner interface{ Scan(...any) error }
|
||||
|
||||
func scanDevice(row scanner) (Device, error) {
|
||||
var item Device
|
||||
if err := row.Scan(&item.ID, &item.TenantID, &item.Model, &item.HardwareRevision, &item.Status, &item.MinimumSecurityVersion, &item.MinimumFirmwareVersion,
|
||||
&item.HouseholdID, &item.HouseholdName, &item.SubjectID, &item.SubjectName, &item.CredentialStatus, &item.KeyVersion,
|
||||
&item.LastReportedAt, &item.RegisteredAt, &item.UpdatedAt, &item.Version); err != nil {
|
||||
return Device{}, fmt.Errorf("scan operations device: %w", err)
|
||||
}
|
||||
return item, nil
|
||||
}
|
||||
|
||||
func readDevice(ctx context.Context, q queryRow, tenantID, publicID string) (Device, error) {
|
||||
item, err := scanDevice(q.QueryRow(ctx, deviceSelectSQL+` WHERE d.tenant_id=$1::uuid AND d.public_id=$2::uuid`, tenantID, publicID))
|
||||
if errors.Is(err, pgx.ErrNoRows) {
|
||||
return Device{}, ErrNotFound
|
||||
}
|
||||
return item, err
|
||||
}
|
||||
@@ -0,0 +1,152 @@
|
||||
package opsdevice
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"errors"
|
||||
"os"
|
||||
"testing"
|
||||
|
||||
"data-terminal-cloud/internal/audit"
|
||||
"data-terminal-cloud/internal/deviceauth"
|
||||
"data-terminal-cloud/internal/identity"
|
||||
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
)
|
||||
|
||||
func TestOperationsDeviceLifecycle(t *testing.T) {
|
||||
databaseURL := os.Getenv("TEST_DATABASE_URL")
|
||||
if databaseURL == "" {
|
||||
t.Skip("TEST_DATABASE_URL is not set")
|
||||
}
|
||||
ctx := context.Background()
|
||||
pool, err := pgxpool.New(ctx, databaseURL)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer pool.Close()
|
||||
resolver, err := deviceauth.NewDatabaseSecretResolver(pool, "000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
service, err := NewService(pool, resolver, audit.NewRepository())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
principal := integrationOperatorPrincipal(t, ctx, pool)
|
||||
const tenantID = "10000000-0000-0000-0000-000000000010"
|
||||
created, err := service.Create(ctx, principal, tenantID, "ops-device-integration-create", CreateInput{
|
||||
Model: "integration-terminal", HardwareRevision: "rev-test",
|
||||
HouseholdID: "10000000-0000-0000-0000-000000000211", SubjectID: "10000000-0000-0000-0000-000000000411",
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
deviceID := created.Device.ID
|
||||
defer func() {
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM audit_event WHERE resource_id=$1::uuid`, deviceID)
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM device_binding WHERE device_id=(SELECT id FROM device WHERE public_id=$1::uuid)`, deviceID)
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM device_credential WHERE device_id=(SELECT id FROM device WHERE public_id=$1::uuid)`, deviceID)
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM device_twin WHERE device_id=(SELECT id FROM device WHERE public_id=$1::uuid)`, deviceID)
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM device WHERE public_id=$1::uuid`, deviceID)
|
||||
}()
|
||||
if len(created.Credential.Secret) != 64 || created.Credential.KeyVersion != 1 || created.Device.SubjectName == nil || *created.Device.SubjectName != "测试长者" {
|
||||
t.Fatalf("unexpected created result: %+v", created)
|
||||
}
|
||||
if created.Claim != nil {
|
||||
t.Fatalf("assigned device unexpectedly received claim: %+v", created.Claim)
|
||||
}
|
||||
if _, err := service.IssueDeviceClaim(ctx, principal, tenantID, deviceID, "ops-device-integration-bound-claim"); !errors.Is(err, ErrClaimUnavailable) {
|
||||
t.Fatalf("bound device claim error=%v", err)
|
||||
}
|
||||
resolved, err := resolver.Resolve(ctx, "device-secret://"+deviceID)
|
||||
if err != nil || !bytes.Equal(resolved, []byte(created.Credential.Secret)) {
|
||||
t.Fatalf("created credential did not resolve: err=%v", err)
|
||||
}
|
||||
model := "integration-terminal-v2"
|
||||
status := "quarantined"
|
||||
updated, err := service.Update(ctx, principal, tenantID, deviceID, "ops-device-integration-update", created.Device.Version, UpdateInput{Model: &model, Status: &status})
|
||||
if err != nil || updated.Model != model || updated.Status != status || updated.Version != created.Device.Version+1 {
|
||||
t.Fatalf("unexpected update: %+v err=%v", updated, err)
|
||||
}
|
||||
rotated, err := service.RotateCredential(ctx, principal, tenantID, deviceID, "ops-device-integration-rotate")
|
||||
if err != nil || rotated.KeyVersion != 2 || rotated.Secret == created.Credential.Secret {
|
||||
t.Fatalf("unexpected credential rotation: %+v err=%v", rotated, err)
|
||||
}
|
||||
resolved, err = resolver.Resolve(ctx, "device-secret://"+deviceID)
|
||||
if err != nil || !bytes.Equal(resolved, []byte(rotated.Secret)) {
|
||||
t.Fatalf("rotated credential did not resolve: err=%v", err)
|
||||
}
|
||||
retired, err := service.Delete(ctx, principal, tenantID, deviceID, "ops-device-integration-delete", updated.Version)
|
||||
if err != nil || retired.Status != "revoked" || retired.CredentialStatus != "revoked" || retired.HouseholdID != nil {
|
||||
t.Fatalf("unexpected delete result: %+v err=%v", retired, err)
|
||||
}
|
||||
if _, err := resolver.Resolve(ctx, "device-secret://"+deviceID); err == nil {
|
||||
t.Fatal("retired device credential remained active")
|
||||
}
|
||||
}
|
||||
|
||||
func TestOperationsDeviceClaimIssueAndRotation(t *testing.T) {
|
||||
databaseURL := os.Getenv("TEST_DATABASE_URL")
|
||||
if databaseURL == "" {
|
||||
t.Skip("TEST_DATABASE_URL is not set")
|
||||
}
|
||||
ctx := context.Background()
|
||||
pool, err := pgxpool.New(ctx, databaseURL)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer pool.Close()
|
||||
resolver, err := deviceauth.NewDatabaseSecretResolver(pool, "000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
service, err := NewService(pool, resolver, audit.NewRepository())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
principal := integrationOperatorPrincipal(t, ctx, pool)
|
||||
const tenantID = "10000000-0000-0000-0000-000000000010"
|
||||
created, err := service.Create(ctx, principal, tenantID, "ops-device-claim-integration-create", CreateInput{Model: "unbound-integration-terminal"})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
deviceID := created.Device.ID
|
||||
defer func() {
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM audit_event WHERE resource_id=$1::uuid`, deviceID)
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM device_claim WHERE device_id=(SELECT id FROM device WHERE public_id=$1::uuid)`, deviceID)
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM device_credential WHERE device_id=(SELECT id FROM device WHERE public_id=$1::uuid)`, deviceID)
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM device_twin WHERE device_id=(SELECT id FROM device WHERE public_id=$1::uuid)`, deviceID)
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM device WHERE public_id=$1::uuid`, deviceID)
|
||||
}()
|
||||
if created.Claim == nil || len(created.Claim.ClaimCode) != 19 || !created.Claim.ExpiresAt.After(created.Claim.IssuedAt) {
|
||||
t.Fatalf("unexpected initial claim: %+v", created.Claim)
|
||||
}
|
||||
firstDigest := sha256.Sum256([]byte(created.Claim.ClaimCode))
|
||||
var storedDigest string
|
||||
if err := pool.QueryRow(ctx, `SELECT token_digest FROM device_claim WHERE device_id=(SELECT id FROM device WHERE public_id=$1::uuid) AND revoked_at IS NULL`, deviceID).Scan(&storedDigest); err != nil || storedDigest != hex.EncodeToString(firstDigest[:]) {
|
||||
t.Fatalf("stored initial claim digest=%s err=%v", storedDigest, err)
|
||||
}
|
||||
rotated, err := service.IssueDeviceClaim(ctx, principal, tenantID, deviceID, "ops-device-claim-integration-rotate")
|
||||
if err != nil || rotated.ClaimCode == created.Claim.ClaimCode {
|
||||
t.Fatalf("rotated claim=%+v err=%v", rotated, err)
|
||||
}
|
||||
var activeClaims, revokedClaims int
|
||||
if err := pool.QueryRow(ctx, `SELECT count(*) FILTER (WHERE revoked_at IS NULL),count(*) FILTER (WHERE revoked_at IS NOT NULL) FROM device_claim WHERE device_id=(SELECT id FROM device WHERE public_id=$1::uuid)`, deviceID).Scan(&activeClaims, &revokedClaims); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if activeClaims != 1 || revokedClaims != 1 {
|
||||
t.Fatalf("claim counts active=%d revoked=%d", activeClaims, revokedClaims)
|
||||
}
|
||||
}
|
||||
|
||||
func integrationOperatorPrincipal(t *testing.T, ctx context.Context, pool *pgxpool.Pool) identity.Principal {
|
||||
t.Helper()
|
||||
var principal identity.Principal
|
||||
if err := pool.QueryRow(ctx, `SELECT oidc_issuer,oidc_subject FROM user_account WHERE public_id='10000000-0000-0000-0000-000000000114'::uuid AND status='active' AND account_kind='operator'`).Scan(&principal.Issuer, &principal.Subject); err != nil {
|
||||
t.Fatalf("resolve integration operator: %v", err)
|
||||
}
|
||||
return principal
|
||||
}
|
||||
@@ -12,6 +12,7 @@ import (
|
||||
|
||||
"data-terminal-cloud/internal/billing"
|
||||
"data-terminal-cloud/internal/conversation"
|
||||
"data-terminal-cloud/internal/memorytool"
|
||||
"data-terminal-cloud/internal/outbox"
|
||||
"data-terminal-cloud/internal/ratelimit"
|
||||
"data-terminal-cloud/internal/voiceprovider"
|
||||
@@ -40,6 +41,19 @@ type StartProcessor struct {
|
||||
startLimiter providerStartRateLimiter
|
||||
maxActiveTasks int
|
||||
now func() time.Time
|
||||
memoryTool *memorytool.TokenSigner
|
||||
memoryToolURL string
|
||||
memoryToolTTL time.Duration
|
||||
}
|
||||
|
||||
func (p *StartProcessor) ConfigureMemoryTool(signer *memorytool.TokenSigner, endpoint string, ttl time.Duration) error {
|
||||
if signer == nil || endpoint == "" || ttl <= 0 {
|
||||
return errors.New("memory tool signer, endpoint and positive token TTL are required")
|
||||
}
|
||||
p.memoryTool = signer
|
||||
p.memoryToolURL = endpoint
|
||||
p.memoryToolTTL = ttl
|
||||
return nil
|
||||
}
|
||||
|
||||
// ConfigureStartProtection enables the provider-wide RPS bucket and durable
|
||||
@@ -105,6 +119,13 @@ func (p *StartProcessor) Handle(ctx context.Context, event outbox.Event) error {
|
||||
if taskStatus != "STARTING" && taskStatus != "UNKNOWN" {
|
||||
return fmt.Errorf("provider task status %s cannot be started", taskStatus)
|
||||
}
|
||||
if request.EngineMode == "s2s_hybrid" && p.memoryTool != nil {
|
||||
request.MemoryToolURL = p.memoryToolURL
|
||||
request.MemoryToolToken, err = p.memoryTool.Issue(request.ConversationID, request.TaskRef, p.now().UTC().Add(p.memoryToolTTL))
|
||||
if err != nil {
|
||||
return fmt.Errorf("issue memory tool token: %w", err)
|
||||
}
|
||||
}
|
||||
allowed, err := p.health.AllowStart(ctx, tenantID, providerName)
|
||||
if err != nil {
|
||||
return fmt.Errorf("read provider health: %w", err)
|
||||
|
||||
@@ -38,6 +38,12 @@ var (
|
||||
|
||||
const contextPreparationFailureCode = "context_preparation_failed"
|
||||
|
||||
const (
|
||||
contextSummaryLimit = 1000
|
||||
recentSummaryLimit = 3
|
||||
recentSummaryContentLimit = 240
|
||||
)
|
||||
|
||||
type AllocationOptions struct {
|
||||
Now time.Time
|
||||
AllocationExpires time.Time
|
||||
@@ -367,19 +373,24 @@ func (r *Repository) Create(
|
||||
consentSnapshotID, memoryVersionsJSON, promptBundleVersion, engineMode,
|
||||
privacyFromConsent(effectiveConsents), options)
|
||||
}
|
||||
contextSummary := buildContextSummary(subjectName, profileContext)
|
||||
if len(memoryContents) > 0 {
|
||||
contextSummary += "；已确认记忆：" + strings.Join(memoryContents, "；")
|
||||
var recentSummaries []string
|
||||
if consentAllowsConversationContinuity(effectiveConsents) {
|
||||
recentSummaries, err = selectRecentConversationSummaries(ctx, tx, identity.TenantID, subjectID, recentSummaryLimit)
|
||||
if err != nil {
|
||||
return CreateResponse{}, "", fmt.Errorf("retrieve recent conversation summaries: %w", err)
|
||||
}
|
||||
}
|
||||
contextSummary := buildContextSummary(subjectName, profileContext, recentSummaries, memoryContents)
|
||||
contextValue := struct {
|
||||
SubjectID string `json:"subject_id"`
|
||||
PreferredName string `json:"preferred_name"`
|
||||
ProfileVersion int64 `json:"profile_version"`
|
||||
RequestedGoal string `json:"requested_goal"`
|
||||
ProfileContext authorizedSubjectContext `json:"profile_context"`
|
||||
RecentSummaries []string `json:"recent_summaries"`
|
||||
MemoryContents []string `json:"memory_contents"`
|
||||
RetrievalPolicy string `json:"retrieval_policy"`
|
||||
}{subjectPublicID, subjectName, profileVersion, input.RequestedGoal, profileContext, memoryContents, retrievalPolicy(options.MemoryPolicy)}
|
||||
}{subjectPublicID, subjectName, profileVersion, input.RequestedGoal, profileContext, recentSummaries, memoryContents, retrievalPolicy(options.MemoryPolicy)}
|
||||
contextJSON, _ := json.Marshal(contextValue)
|
||||
contextDigest := sha256.Sum256(contextJSON)
|
||||
contextSnapshotID := uuid.NewString()
|
||||
@@ -664,6 +675,7 @@ type authorizedSubjectContext struct {
|
||||
ConversationGoal string `json:"conversation_goal,omitempty"`
|
||||
Pace string `json:"pace,omitempty"`
|
||||
PreferredTopics []string `json:"preferred_topics,omitempty"`
|
||||
AvoidedTopics []string `json:"avoided_topics,omitempty"`
|
||||
}
|
||||
|
||||
func authorizedProfileContext(profileDataJSON, preferencesJSON json.RawMessage) (authorizedSubjectContext, error) {
|
||||
@@ -694,6 +706,10 @@ func authorizedProfileContext(profileDataJSON, preferencesJSON json.RawMessage)
|
||||
if err != nil {
|
||||
return authorizedSubjectContext{}, err
|
||||
}
|
||||
avoidedTopics, err := strictContextStringList(preferences, "avoided_topics", 8, 50)
|
||||
if err != nil {
|
||||
return authorizedSubjectContext{}, err
|
||||
}
|
||||
if pace != "slow" && pace != "normal" && pace != "lively" {
|
||||
pace = ""
|
||||
}
|
||||
@@ -703,30 +719,107 @@ func authorizedProfileContext(profileDataJSON, preferencesJSON json.RawMessage)
|
||||
ConversationGoal: goal,
|
||||
Pace: pace,
|
||||
PreferredTopics: preferredTopics,
|
||||
AvoidedTopics: avoidedTopics,
|
||||
}, nil
|
||||
}
|
||||
|
||||
func buildContextSummary(preferredName string, profile authorizedSubjectContext) string {
|
||||
parts := []string{"称呼：" + truncateContextText(preferredName, 120)}
|
||||
if profile.Background != "" {
|
||||
parts = append(parts, "背景资料："+profile.Background)
|
||||
func buildContextSummary(preferredName string, profile authorizedSubjectContext, recentSummaries, memoryContents []string) string {
|
||||
parts := make([]string, 0, 10)
|
||||
if len(profile.AvoidedTopics) > 0 {
|
||||
parts = append(parts, "交流边界：不要主动提及“"+strings.Join(truncateContextItems(profile.AvoidedTopics, 24), "、")+"”；若长者主动提及，先简短共情，再自然转向；紧急安全响应不受此限制")
|
||||
}
|
||||
if len(profile.Interests) > 0 {
|
||||
parts = append(parts, "兴趣："+strings.Join(profile.Interests, "、"))
|
||||
parts = append(parts, "称呼："+truncateContextField(preferredName, 60))
|
||||
for index, summary := range recentSummaries {
|
||||
parts = append(parts, fmt.Sprintf("最近第%d次通话摘要：%s", index+1, truncateContextField(summary, 140)))
|
||||
}
|
||||
if len(memoryContents) > 0 {
|
||||
parts = append(parts, "已人工确认的长期记忆："+strings.Join(truncateContextItems(memoryContents, 30), "；"))
|
||||
}
|
||||
goal := profile.ConversationGoal
|
||||
if goal == "" {
|
||||
goal = "进行一次自然、尊重的陪伴交流"
|
||||
}
|
||||
parts = append(parts, "本次交流目标："+goal)
|
||||
parts = append(parts, "本次交流目标："+truncateContextField(goal, 80))
|
||||
if profile.Pace != "" {
|
||||
paceLabels := map[string]string{"slow": "放慢语速并留出回应时间", "normal": "自然语速", "lively": "轻快但保持清晰"}
|
||||
parts = append(parts, "交流节奏："+paceLabels[profile.Pace])
|
||||
}
|
||||
if len(profile.PreferredTopics) > 0 {
|
||||
parts = append(parts, "优先话题："+strings.Join(profile.PreferredTopics, "、"))
|
||||
parts = append(parts, "优先话题："+strings.Join(truncateContextItems(profile.PreferredTopics, 15), "、"))
|
||||
}
|
||||
return strings.Join(parts, "；")
|
||||
if profile.Background != "" {
|
||||
parts = append(parts, "背景资料："+truncateContextField(profile.Background, 80))
|
||||
}
|
||||
if len(profile.Interests) > 0 {
|
||||
parts = append(parts, "兴趣："+strings.Join(truncateContextItems(profile.Interests, 15), "、"))
|
||||
}
|
||||
result := strings.Join(parts, "；")
|
||||
return truncateContextField(result, contextSummaryLimit)
|
||||
}
|
||||
|
||||
func truncateContextItems(values []string, itemLimit int) []string {
|
||||
items := make([]string, 0, len(values))
|
||||
for _, value := range values {
|
||||
if value = truncateContextField(value, itemLimit); value != "" {
|
||||
items = append(items, value)
|
||||
}
|
||||
}
|
||||
return items
|
||||
}
|
||||
|
||||
func truncateContextField(value string, limit int) string {
|
||||
runes := []rune(strings.TrimSpace(value))
|
||||
if len(runes) <= limit {
|
||||
return string(runes)
|
||||
}
|
||||
if limit <= 1 {
|
||||
return "…"
|
||||
}
|
||||
return string(runes[:limit-1]) + "…"
|
||||
}
|
||||
|
||||
func consentAllowsConversationContinuity(consents json.RawMessage) bool {
|
||||
values := map[string]bool{}
|
||||
if json.Unmarshal(consents, &values) != nil {
|
||||
return false
|
||||
}
|
||||
return values["personalized_memory"] && values["story_extraction"]
|
||||
}
|
||||
|
||||
func selectRecentConversationSummaries(ctx context.Context, tx pgx.Tx, tenantID, subjectID string, limit int) ([]string, error) {
|
||||
if limit <= 0 {
|
||||
return nil, nil
|
||||
}
|
||||
rows, err := tx.Query(ctx, `
|
||||
WITH latest_summary AS (
|
||||
SELECT DISTINCT ON (summary.conversation_id)
|
||||
summary.conversation_id, summary.content, conversation.ended_at, conversation.created_at
|
||||
FROM conversation_summary summary
|
||||
JOIN conversation ON conversation.id=summary.conversation_id
|
||||
AND conversation.tenant_id=summary.tenant_id
|
||||
WHERE summary.tenant_id=$1::uuid AND summary.subject_id=$2::uuid
|
||||
AND summary.summary_type='conversation_summary' AND summary.status='READY'
|
||||
AND conversation.status='COMPLETED'
|
||||
ORDER BY summary.conversation_id, summary.generation_version DESC, summary.created_at DESC
|
||||
)
|
||||
SELECT content FROM latest_summary
|
||||
ORDER BY COALESCE(ended_at,created_at) DESC, conversation_id DESC
|
||||
LIMIT $3`, tenantID, subjectID, limit)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
result := make([]string, 0, limit)
|
||||
for rows.Next() {
|
||||
var content string
|
||||
if err := rows.Scan(&content); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if content = truncateContextText(content, recentSummaryContentLimit); content != "" {
|
||||
result = append(result, content)
|
||||
}
|
||||
}
|
||||
return result, rows.Err()
|
||||
}
|
||||
|
||||
func strictContextString(values map[string]any, key string, limit int) (string, error) {
|
||||
|
||||
@@ -39,6 +39,9 @@ const (
|
||||
testModelConfigPublicID = "90000000-0000-0000-0000-000000000713"
|
||||
testRoutingConfigID = "90000000-0000-0000-0000-000000000704"
|
||||
testRoutingConfigPublicID = "90000000-0000-0000-0000-000000000714"
|
||||
testConsentSnapshotID = "90000000-0000-0000-0000-000000000801"
|
||||
testPriorContextID = "90000000-0000-0000-0000-000000000802"
|
||||
testPriorConversationID = "90000000-0000-0000-0000-000000000803"
|
||||
testBundleConfigKey = "orchestration-test-agent-bundle"
|
||||
testBundleCreateKey = "orchestration-agent-bundle-create"
|
||||
testBundleEvaluationKey = "orchestration-agent-bundle-evaluation"
|
||||
@@ -134,14 +137,22 @@ func TestServiceIdempotentlyCreatesConversationWithoutPersistingToken(t *testing
|
||||
{`INSERT INTO subject_profile (id, public_id, tenant_id, household_id, preferred_name, profile_data, conversation_preferences, status)
|
||||
VALUES ($1::uuid, '90000000-0000-0000-0000-000000000411', '10000000-0000-0000-0000-000000000001', '10000000-0000-0000-0000-000000000201', '服务测试长者',
|
||||
'{"background":"年轻时从事铁路工作","interests":["老照片","家乡变化"],"internal_note":"不得进入语音上下文"}',
|
||||
'{"conversation_goal":"聊聊第一次参加工作的经历","pace":"slow","preferred_topics":["工作经历"]}', 'active')`, []any{testSubjectID}},
|
||||
'{"conversation_goal":"聊聊第一次参加工作的经历","pace":"slow","preferred_topics":["工作经历"],"avoided_topics":["收入"]}', 'active')`, []any{testSubjectID}},
|
||||
{`INSERT INTO device (id, public_id, tenant_id, serial_digest, model, status, credential_ref)
|
||||
VALUES ($1::uuid, $2::uuid, '10000000-0000-0000-0000-000000000001', repeat('9',64), 'service-test', 'registered', 'test://service')`, []any{testDeviceID, testDevicePublicID}},
|
||||
{`INSERT INTO device_binding (tenant_id, device_id, household_id, subject_id, reason)
|
||||
VALUES ('10000000-0000-0000-0000-000000000001', $1::uuid, '10000000-0000-0000-0000-000000000201', $2::uuid, 'integration test')`, []any{testDeviceID, testSubjectID}},
|
||||
{`INSERT INTO device_twin (device_id, tenant_id) VALUES ($1::uuid, '10000000-0000-0000-0000-000000000001')`, []any{testDeviceID}},
|
||||
{`INSERT INTO consent_snapshot (tenant_id, subject_id, snapshot_version, effective_consents)
|
||||
VALUES ('10000000-0000-0000-0000-000000000001', $1::uuid, 1, '{"voice_chat":true,"transcript":true,"audio_recording":false}'::jsonb)`, []any{testSubjectID}},
|
||||
{`INSERT INTO consent_snapshot (id,tenant_id,subject_id,snapshot_version,effective_consents)
|
||||
VALUES ($1::uuid,'10000000-0000-0000-0000-000000000001',$2::uuid,1,'{"voice_chat":true,"transcript":true,"audio_recording":false,"story_extraction":true,"personalized_memory":true}'::jsonb)`, []any{testConsentSnapshotID, testSubjectID}},
|
||||
{`INSERT INTO context_snapshot (id,tenant_id,subject_id,profile_version,consent_snapshot_id,memory_versions,prompt_bundle_version,engine_mode,content_digest,sanitized_summary)
|
||||
VALUES ($1::uuid,'10000000-0000-0000-0000-000000000001',$2::uuid,1,$3::uuid,'[]'::jsonb,'prior-test','cascade',repeat('b',64),'此前通话上下文')`, []any{testPriorContextID, testSubjectID, testConsentSnapshotID}},
|
||||
{`INSERT INTO conversation (id,tenant_id,household_id,subject_id,device_id,consent_snapshot_id,context_snapshot_id,provider_task_ref,engine_mode,retention_mode,status,allocation_expires_at,started_at,ended_at,created_at,updated_at)
|
||||
VALUES ($1::uuid,'10000000-0000-0000-0000-000000000001','10000000-0000-0000-0000-000000000201',$2::uuid,$3::uuid,$4::uuid,$5::uuid,'prior-context-task','cascade','transcript_only','COMPLETED','2026-08-23T10:30:00Z','2026-08-23T10:00:00Z','2026-08-23T10:20:00Z','2026-08-23T10:00:00Z','2026-08-23T10:20:00Z')`, []any{testPriorConversationID, testSubjectID, testDeviceID, testConsentSnapshotID, testPriorContextID}},
|
||||
{`INSERT INTO conversation_turn (tenant_id,subject_id,conversation_id,turn_key,sequence_no,speaker,start_at,end_at,final_text,source_type)
|
||||
VALUES ('10000000-0000-0000-0000-000000000001',$1::uuid,$2::uuid,'prior-subject-1',1,'subject','2026-08-23T10:01:00Z','2026-08-23T10:01:05Z','我注册了一个个体工商户。','provider')`, []any{testSubjectID, testPriorConversationID}},
|
||||
{`INSERT INTO conversation_summary (tenant_id,subject_id,conversation_id,summary_type,content,source_turn_ids,model_ref,prompt_version,generation_version,status,created_at)
|
||||
VALUES ('10000000-0000-0000-0000-000000000001',$1::uuid,$2::uuid,'conversation_summary','上次聊到刚注册了一个个体工商户，准备开始新的工作。','["prior-subject-1"]'::jsonb,'mock-content-v1','prompt-v1',1,'READY','2026-08-23T10:21:00Z')`, []any{testSubjectID, testPriorConversationID}},
|
||||
{`INSERT INTO consent_record (tenant_id,subject_id,consent_type,policy_version,notice_digest,actor_user_id)
|
||||
VALUES ('10000000-0000-0000-0000-000000000001',$1::uuid,'personalized_memory','orchestration-service-test',repeat('a',64),'10000000-0000-0000-0000-000000000101')`, []any{testSubjectID}},
|
||||
{`INSERT INTO runtime_config_version (
|
||||
@@ -200,8 +211,8 @@ func TestServiceIdempotentlyCreatesConversationWithoutPersistingToken(t *testing
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
configAdmin := platformidentity.Principal{Issuer: "http://localhost:58180/realms/voice-local", Subject: "00000000-0000-0000-0000-000000000101"}
|
||||
configApprover := platformidentity.Principal{Issuer: configAdmin.Issuer, Subject: "00000000-0000-0000-0000-000000000102"}
|
||||
configAdmin := platformidentity.Principal{Issuer: "http://localhost:58180/realms/voice-local", Subject: "00000000-0000-0000-0000-000000000104"}
|
||||
configApprover := platformidentity.Principal{Issuer: configAdmin.Issuer, Subject: "00000000-0000-0000-0000-000000000105"}
|
||||
bundle, created, err := configService.Create(ctx, configAdmin, testBundleCreateKey, "orchestration-agent-bundle-create", opsconfig.CreateInput{
|
||||
TenantID: "10000000-0000-0000-0000-000000000010", ConfigKey: testBundleConfigKey,
|
||||
DisplayName: "编排测试智能体包", ConfigKind: "AGENT_BUNDLE", ChangeSummary: "验证运营发布的自定义版本族进入通话运行时",
|
||||
@@ -268,7 +279,7 @@ func TestServiceIdempotentlyCreatesConversationWithoutPersistingToken(t *testing
|
||||
if frozenBundleVersion != "cfg:"+bundle.ID || first.Session.EngineMode != "cascade" {
|
||||
t.Fatalf("frozen bundle=%q engine=%q", frozenBundleVersion, first.Session.EngineMode)
|
||||
}
|
||||
for _, expected := range []string{"服务测试长者", "年轻时从事铁路工作", "老照片", "聊聊第一次参加工作的经历", "放慢语速", "工作经历"} {
|
||||
for _, expected := range []string{"服务测试长者", "年轻时从事铁路工作", "老照片", "聊聊第一次参加工作的经历", "放慢语速", "工作经历", "交流边界", "收入", "最近第1次通话摘要", "个体工商户"} {
|
||||
if !strings.Contains(frozenContext, expected) {
|
||||
t.Fatalf("frozen context %q does not contain %q", frozenContext, expected)
|
||||
}
|
||||
@@ -303,7 +314,7 @@ func TestServiceIdempotentlyCreatesConversationWithoutPersistingToken(t *testing
|
||||
if err := pool.QueryRow(ctx, `SELECT EXISTS (SELECT 1 FROM idempotency_record WHERE actor_key=$1 AND response_body::text LIKE '%dev1.%')`, "device:"+testDevicePublicID).Scan(&tokenPersisted); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if conversations != 1 || histories != 2 || audits != 1 || events != 1 || tokenPersisted {
|
||||
if conversations != 2 || histories != 2 || audits != 1 || events != 1 || tokenPersisted {
|
||||
t.Fatalf("conversation=%d history=%d audit=%d event=%d tokenPersisted=%v", conversations, histories, audits, events, tokenPersisted)
|
||||
}
|
||||
activateInput := ActivateInput{
|
||||
@@ -863,6 +874,7 @@ func cleanupOrchestrationFixture(t *testing.T, ctx context.Context, pool *pgxpoo
|
||||
{`DELETE FROM runtime_config_version WHERE id=$1::uuid`, testVoiceConfigID},
|
||||
{`DELETE FROM runtime_config_version WHERE id=$1::uuid`, testModelConfigID},
|
||||
{`DELETE FROM runtime_config_version WHERE id=$1::uuid`, testRoutingConfigID},
|
||||
{`DELETE FROM conversation_summary WHERE conversation_id IN (SELECT id FROM conversation WHERE device_id=$1::uuid)`, testDeviceID},
|
||||
{`DELETE FROM conversation_turn WHERE conversation_id IN (SELECT id FROM conversation WHERE device_id=$1::uuid)`, testDeviceID},
|
||||
{`DELETE FROM provider_event WHERE conversation_id IN (SELECT id FROM conversation WHERE device_id=$1::uuid)`, testDeviceID},
|
||||
{`DELETE FROM conversation_state_history WHERE conversation_id IN (SELECT id FROM conversation WHERE device_id=$1::uuid)`, testDeviceID},
|
||||
|
||||
@@ -1,7 +1,9 @@
|
||||
package orchestration
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
@@ -52,9 +54,9 @@ func TestConstructorAndBoundaryValidation(t *testing.T) {
|
||||
func TestAuthorizedProfileContextRejectsMalformedWhitelistedFields(t *testing.T) {
|
||||
valid, err := authorizedProfileContext(
|
||||
[]byte(`{"background":"退休教师","interests":["戏曲","散步"],"internal_note":{"ignored":true}}`),
|
||||
[]byte(`{"conversation_goal":"轻松交流","pace":"slow","preferred_topics":["家乡"]}`),
|
||||
[]byte(`{"conversation_goal":"轻松交流","pace":"slow","preferred_topics":["家乡"],"avoided_topics":["收入","家庭矛盾"]}`),
|
||||
)
|
||||
if err != nil || valid.Background != "退休教师" || len(valid.Interests) != 2 || valid.Pace != "slow" {
|
||||
if err != nil || valid.Background != "退休教师" || len(valid.Interests) != 2 || valid.Pace != "slow" || len(valid.AvoidedTopics) != 2 {
|
||||
t.Fatalf("valid context=%+v err=%v", valid, err)
|
||||
}
|
||||
for _, malformed := range []struct {
|
||||
@@ -63,9 +65,53 @@ func TestAuthorizedProfileContextRejectsMalformedWhitelistedFields(t *testing.T)
|
||||
{`{"background":42}`, `{}`},
|
||||
{`{"interests":["戏曲",42]}`, `{}`},
|
||||
{`{}`, `{"preferred_topics":"家乡"}`},
|
||||
{`{}`, `{"avoided_topics":"收入"}`},
|
||||
} {
|
||||
if _, err := authorizedProfileContext([]byte(malformed.profile), []byte(malformed.preferences)); err == nil {
|
||||
t.Fatalf("malformed whitelisted context was accepted: profile=%s preferences=%s", malformed.profile, malformed.preferences)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestBuildContextSummaryIncludesContinuityAndBoundariesWithinSnapshotLimit(t *testing.T) {
|
||||
profile := authorizedSubjectContext{
|
||||
Background: "退休教师", Interests: []string{"戏曲"}, ConversationGoal: "自然陪伴", Pace: "slow",
|
||||
PreferredTopics: []string{"家乡"}, AvoidedTopics: []string{"收入", "家庭矛盾"},
|
||||
}
|
||||
result := buildContextSummary("王阿姨", profile,
|
||||
[]string{"上次聊到刚刚注册了个体工商户，准备开始新的工作。"},
|
||||
[]string{"喜欢在周末听越剧。"})
|
||||
for _, expected := range []string{"交流边界", "收入", "王阿姨", "最近第1次通话摘要", "注册了个体工商户", "已人工确认的长期记忆", "越剧"} {
|
||||
if !strings.Contains(result, expected) {
|
||||
t.Fatalf("context %q does not contain %q", result, expected)
|
||||
}
|
||||
}
|
||||
oversized := buildContextSummary(strings.Repeat("长", 120), authorizedSubjectContext{
|
||||
Background: strings.Repeat("背", 500), AvoidedTopics: []string{strings.Repeat("禁", 50), "家庭矛盾"},
|
||||
PreferredTopics: []string{strings.Repeat("优", 50)}, Interests: []string{strings.Repeat("兴", 50)},
|
||||
}, []string{strings.Repeat("摘", 500), "第二次摘要", "第三次摘要"}, []string{strings.Repeat("忆", 500), "第二条记忆", "第三条记忆", "第四条记忆", "第五条记忆"})
|
||||
if len([]rune(oversized)) > contextSummaryLimit {
|
||||
t.Fatalf("context length=%d exceeds snapshot budget", len([]rune(oversized)))
|
||||
}
|
||||
for _, expected := range []string{"交流边界", "家庭矛盾", "最近第3次通话摘要", "第三次摘要", "已人工确认的长期记忆", "第五条记忆"} {
|
||||
if !strings.Contains(oversized, expected) {
|
||||
t.Fatalf("oversized context %q lost priority field %q", oversized, expected)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestConversationContinuityRequiresBothExplicitConsents(t *testing.T) {
|
||||
for _, test := range []struct {
|
||||
value json.RawMessage
|
||||
want bool
|
||||
}{
|
||||
{json.RawMessage(`{"personalized_memory":true,"story_extraction":true}`), true},
|
||||
{json.RawMessage(`{"personalized_memory":true}`), false},
|
||||
{json.RawMessage(`{"story_extraction":true}`), false},
|
||||
{json.RawMessage(`not-json`), false},
|
||||
} {
|
||||
if got := consentAllowsConversationContinuity(test.value); got != test.want {
|
||||
t.Fatalf("consent %s allowed=%v want=%v", test.value, got, test.want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -13,6 +13,7 @@ import (
|
||||
"data-terminal-cloud/internal/audit"
|
||||
"data-terminal-cloud/internal/database"
|
||||
"data-terminal-cloud/internal/outbox"
|
||||
"data-terminal-cloud/internal/providermemory"
|
||||
|
||||
"github.com/jackc/pgx/v5"
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
@@ -24,10 +25,11 @@ type deletionObjectStore interface {
|
||||
}
|
||||
|
||||
type DeleteProcessor struct {
|
||||
pool *pgxpool.Pool
|
||||
redis *redis.Client
|
||||
store deletionObjectStore
|
||||
audit *audit.Repository
|
||||
pool *pgxpool.Pool
|
||||
redis *redis.Client
|
||||
store deletionObjectStore
|
||||
audit *audit.Repository
|
||||
providerMemory providermemory.Client
|
||||
}
|
||||
|
||||
type deletionRequest struct {
|
||||
@@ -38,11 +40,18 @@ type deletionSubtask struct {
|
||||
id, subsystem, status string
|
||||
}
|
||||
|
||||
func NewDeleteProcessor(pool *pgxpool.Pool, redisClient *redis.Client, store deletionObjectStore) (*DeleteProcessor, error) {
|
||||
func NewDeleteProcessor(pool *pgxpool.Pool, redisClient *redis.Client, store deletionObjectStore, providerClients ...providermemory.Client) (*DeleteProcessor, error) {
|
||||
if pool == nil || redisClient == nil || store == nil {
|
||||
return nil, errors.New("database, redis and object store are required")
|
||||
}
|
||||
return &DeleteProcessor{pool: pool, redis: redisClient, store: store, audit: audit.NewRepository()}, nil
|
||||
if len(providerClients) > 1 {
|
||||
return nil, errors.New("at most one provider memory client is allowed")
|
||||
}
|
||||
var providerClient providermemory.Client
|
||||
if len(providerClients) == 1 {
|
||||
providerClient = providerClients[0]
|
||||
}
|
||||
return &DeleteProcessor{pool: pool, redis: redisClient, store: store, audit: audit.NewRepository(), providerMemory: providerClient}, nil
|
||||
}
|
||||
|
||||
func (p *DeleteProcessor) Handle(ctx context.Context, event outbox.Event) error {
|
||||
@@ -196,15 +205,37 @@ func (p *DeleteProcessor) executeSubtask(ctx context.Context, request deletionRe
|
||||
_, err := p.pool.Exec(ctx, `UPDATE memory_index_entry SET embedding=NULL,provider_ref=NULL,status='DELETED',last_error=NULL,updated_at=now() WHERE tenant_id=$1::uuid AND subject_id=$2::uuid AND index_type='pgvector'`, request.tenantID, request.subjectID)
|
||||
return "COMPLETED", "", err
|
||||
case "provider_index":
|
||||
var unresolved int
|
||||
if err := p.pool.QueryRow(ctx, `SELECT count(*) FROM memory_index_entry WHERE tenant_id=$1::uuid AND subject_id=$2::uuid AND index_type='provider' AND provider_ref IS NOT NULL AND status<>'DELETED'`, request.tenantID, request.subjectID).Scan(&unresolved); err != nil {
|
||||
rows, err := p.pool.Query(ctx, `SELECT memory_version_id::text,provider_ref FROM memory_index_entry WHERE tenant_id=$1::uuid AND subject_id=$2::uuid AND index_type='provider' AND provider_ref IS NOT NULL AND status<>'DELETED' ORDER BY created_at,id`, request.tenantID, request.subjectID)
|
||||
if err != nil {
|
||||
return "", "", err
|
||||
}
|
||||
if unresolved > 0 {
|
||||
type providerRef struct{ versionID, value string }
|
||||
refs := []providerRef{}
|
||||
for rows.Next() {
|
||||
var ref providerRef
|
||||
if err := rows.Scan(&ref.versionID, &ref.value); err != nil {
|
||||
rows.Close()
|
||||
return "", "", err
|
||||
}
|
||||
refs = append(refs, ref)
|
||||
}
|
||||
rows.Close()
|
||||
if err := rows.Err(); err != nil {
|
||||
return "", "", err
|
||||
}
|
||||
if len(refs) > 0 && p.providerMemory == nil {
|
||||
return "", "", errors.New("external provider index references exist but no deletion adapter is configured")
|
||||
}
|
||||
_, err := p.pool.Exec(ctx, `UPDATE memory_index_entry SET embedding=NULL,status='DELETED',last_error=NULL,updated_at=now() WHERE tenant_id=$1::uuid AND subject_id=$2::uuid AND index_type='provider'`, request.tenantID, request.subjectID)
|
||||
return "SKIPPED", "no external provider index references were present", err
|
||||
for _, ref := range refs {
|
||||
if err := p.providerMemory.Delete(ctx, ref.value, "privacy-provider-memory-delete:"+request.id+":"+ref.versionID); err != nil {
|
||||
return "", "", outbox.Retryable(err)
|
||||
}
|
||||
}
|
||||
_, err = p.pool.Exec(ctx, `UPDATE memory_index_entry SET embedding=NULL,provider_ref=NULL,status='DELETED',last_error=NULL,updated_at=now() WHERE tenant_id=$1::uuid AND subject_id=$2::uuid AND index_type='provider'`, request.tenantID, request.subjectID)
|
||||
if len(refs) == 0 {
|
||||
return "SKIPPED", "no external provider index references were present", err
|
||||
}
|
||||
return "COMPLETED", "", err
|
||||
case "postgresql":
|
||||
return "COMPLETED", "", p.redactPostgreSQL(ctx, request)
|
||||
default:
|
||||
@@ -238,9 +269,13 @@ func (p *DeleteProcessor) redactPostgreSQL(ctx context.Context, request deletion
|
||||
`UPDATE story_revision sr SET content='[已按隐私请求删除]',edit_reason=NULL,model_ref=NULL,prompt_version=NULL FROM story_draft sd WHERE sr.story_id=sd.id AND sd.tenant_id=$1::uuid AND sd.subject_id=$2::uuid`,
|
||||
`UPDATE story_draft SET title='[已按隐私请求删除]',status='SUPERSEDED',review_reason=NULL,version=version+1,updated_at=now() WHERE tenant_id=$1::uuid AND subject_id=$2::uuid`,
|
||||
`UPDATE fact_candidate SET content='[已按隐私请求删除]',time_range='{}'::jsonb,source_turn_ids='["privacy-redacted"]'::jsonb,status='SUPERSEDED',review_reason=NULL,version=version+1,updated_at=now() WHERE tenant_id=$1::uuid AND subject_id=$2::uuid`,
|
||||
`UPDATE fact_candidate_revision fcr SET content='[已按隐私请求删除]',edit_reason=NULL FROM fact_candidate fc WHERE fcr.fact_candidate_id=fc.id AND fc.tenant_id=$1::uuid AND fc.subject_id=$2::uuid`,
|
||||
`UPDATE memory_version mv SET content='[已按隐私请求删除]',source_refs='[{"type":"privacy_tombstone"}]'::jsonb,change_reason=NULL,content_digest=encode(digest('[已按隐私请求删除]','sha256'),'hex') FROM curated_memory cm WHERE mv.memory_id=cm.id AND cm.tenant_id=$1::uuid AND cm.subject_id=$2::uuid`,
|
||||
`UPDATE curated_memory SET status='DELETED',valid_to=COALESCE(valid_to,now()),deleted_at=COALESCE(deleted_at,now()),version=version+1,updated_at=now() WHERE tenant_id=$1::uuid AND subject_id=$2::uuid`,
|
||||
`UPDATE memory_index_entry SET embedding=NULL,provider_ref=NULL,status='DELETED',last_error=NULL,updated_at=now() WHERE tenant_id=$1::uuid AND subject_id=$2::uuid`,
|
||||
`DELETE FROM conversation_memory_use WHERE tenant_id=$1::uuid AND subject_id=$2::uuid`,
|
||||
`DELETE FROM memory_tool_call WHERE tenant_id=$1::uuid AND subject_id=$2::uuid`,
|
||||
`UPDATE content_review_signal SET reason='[privacy-redacted]' WHERE tenant_id=$1::uuid AND subject_id=$2::uuid`,
|
||||
`UPDATE consent_snapshot SET effective_consents='{}'::jsonb,source_consent_ids='[]'::jsonb WHERE tenant_id=$1::uuid AND subject_id=$2::uuid`,
|
||||
`UPDATE context_snapshot SET memory_versions='[]'::jsonb,content_digest=repeat('0',64),sanitized_summary=NULL WHERE tenant_id=$1::uuid AND subject_id=$2::uuid`,
|
||||
`UPDATE recording_access_grant SET purpose='[privacy-redacted]',revoked_at=COALESCE(revoked_at,now()),updated_at=now() WHERE tenant_id=$1::uuid AND subject_id=$2::uuid`,
|
||||
|
||||
@@ -0,0 +1,116 @@
|
||||
package providermemory
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"data-terminal-cloud/internal/outbox"
|
||||
)
|
||||
|
||||
type Memory struct {
|
||||
TenantID string `json:"tenant_id"`
|
||||
HouseholdID string `json:"household_id"`
|
||||
SubjectID string `json:"subject_id"`
|
||||
MemoryID string `json:"memory_id"`
|
||||
MemoryVersion string `json:"memory_version_id"`
|
||||
VersionNo int64 `json:"version_no"`
|
||||
Content string `json:"content"`
|
||||
ContentDigest string `json:"content_digest"`
|
||||
IdempotencyKey string `json:"idempotency_key"`
|
||||
}
|
||||
|
||||
type Client interface {
|
||||
Upsert(context.Context, Memory) (string, error)
|
||||
Delete(context.Context, string, string) error
|
||||
}
|
||||
|
||||
type HTTPClient struct {
|
||||
baseURL string
|
||||
apiKey string
|
||||
client *http.Client
|
||||
}
|
||||
|
||||
func NewHTTPClient(baseURL, apiKey string, timeout time.Duration) (*HTTPClient, error) {
|
||||
parsed, err := url.Parse(strings.TrimRight(strings.TrimSpace(baseURL), "/"))
|
||||
if err != nil || parsed.Scheme == "" || parsed.Host == "" || (parsed.Scheme != "http" && parsed.Scheme != "https") {
|
||||
return nil, errors.New("provider memory base URL must be an absolute HTTP(S) URL")
|
||||
}
|
||||
if strings.TrimSpace(apiKey) == "" || timeout <= 0 {
|
||||
return nil, errors.New("provider memory API key and positive timeout are required")
|
||||
}
|
||||
return &HTTPClient{baseURL: parsed.String(), apiKey: strings.TrimSpace(apiKey), client: &http.Client{Timeout: timeout}}, nil
|
||||
}
|
||||
|
||||
func (c *HTTPClient) Upsert(ctx context.Context, memory Memory) (string, error) {
|
||||
payload, err := json.Marshal(memory)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
request, err := http.NewRequestWithContext(ctx, http.MethodPost, c.baseURL+"/v1/memories:upsert", bytes.NewReader(payload))
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
request.Header.Set("Authorization", "Bearer "+c.apiKey)
|
||||
request.Header.Set("Content-Type", "application/json")
|
||||
request.Header.Set("Idempotency-Key", memory.IdempotencyKey)
|
||||
response, err := c.client.Do(request)
|
||||
if err != nil {
|
||||
return "", outbox.Retryable(fmt.Errorf("call provider memory upsert: %w", err))
|
||||
}
|
||||
defer response.Body.Close()
|
||||
if response.StatusCode < 200 || response.StatusCode >= 300 {
|
||||
return "", providerError("upsert", response)
|
||||
}
|
||||
var result struct {
|
||||
ProviderRef string `json:"provider_ref"`
|
||||
}
|
||||
if err := json.NewDecoder(io.LimitReader(response.Body, 64*1024)).Decode(&result); err != nil {
|
||||
return "", outbox.Retryable(errors.New("provider memory upsert returned invalid JSON"))
|
||||
}
|
||||
result.ProviderRef = strings.TrimSpace(result.ProviderRef)
|
||||
if result.ProviderRef == "" || len(result.ProviderRef) > 300 {
|
||||
return "", outbox.Retryable(errors.New("provider memory upsert returned an invalid reference"))
|
||||
}
|
||||
return result.ProviderRef, nil
|
||||
}
|
||||
|
||||
func (c *HTTPClient) Delete(ctx context.Context, providerRef, idempotencyKey string) error {
|
||||
providerRef = strings.TrimSpace(providerRef)
|
||||
if providerRef == "" {
|
||||
return nil
|
||||
}
|
||||
request, err := http.NewRequestWithContext(ctx, http.MethodDelete, c.baseURL+"/v1/memories/"+url.PathEscape(providerRef), nil)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
request.Header.Set("Authorization", "Bearer "+c.apiKey)
|
||||
request.Header.Set("Idempotency-Key", idempotencyKey)
|
||||
response, err := c.client.Do(request)
|
||||
if err != nil {
|
||||
return outbox.Retryable(fmt.Errorf("call provider memory delete: %w", err))
|
||||
}
|
||||
defer response.Body.Close()
|
||||
if response.StatusCode == http.StatusNotFound || response.StatusCode == http.StatusGone {
|
||||
return nil
|
||||
}
|
||||
if response.StatusCode < 200 || response.StatusCode >= 300 {
|
||||
return providerError("delete", response)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func providerError(action string, response *http.Response) error {
|
||||
err := fmt.Errorf("provider memory %s returned HTTP %d", action, response.StatusCode)
|
||||
if response.StatusCode == http.StatusTooManyRequests || response.StatusCode >= 500 {
|
||||
return outbox.Retryable(err)
|
||||
}
|
||||
return err
|
||||
}
|
||||
@@ -0,0 +1,55 @@
|
||||
package providermemory
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestHTTPClientUpsertAndDelete(t *testing.T) {
|
||||
requests := 0
|
||||
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
requests++
|
||||
if r.Header.Get("Authorization") != "Bearer test-key" || r.Header.Get("Idempotency-Key") == "" {
|
||||
t.Fatalf("missing provider authentication or idempotency header")
|
||||
}
|
||||
switch {
|
||||
case r.Method == http.MethodPost && r.URL.Path == "/v1/memories:upsert":
|
||||
w.Header().Set("Content-Type", "application/json")
|
||||
_, _ = w.Write([]byte(`{"provider_ref":"provider-memory-1"}`))
|
||||
case r.Method == http.MethodDelete && r.URL.Path == "/v1/memories/provider-memory-1":
|
||||
w.WriteHeader(http.StatusNoContent)
|
||||
default:
|
||||
http.NotFound(w, r)
|
||||
}
|
||||
}))
|
||||
defer server.Close()
|
||||
client, err := NewHTTPClient(server.URL, "test-key", time.Second)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
providerRef, err := client.Upsert(context.Background(), Memory{TenantID: "tenant", HouseholdID: "household", SubjectID: "subject", MemoryID: "memory", MemoryVersion: "version", VersionNo: 1, Content: "content", ContentDigest: "digest", IdempotencyKey: "upsert-1"})
|
||||
if err != nil || providerRef != "provider-memory-1" {
|
||||
t.Fatalf("providerRef=%q err=%v", providerRef, err)
|
||||
}
|
||||
if err := client.Delete(context.Background(), providerRef, "delete-1"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if requests != 2 {
|
||||
t.Fatalf("requests=%d", requests)
|
||||
}
|
||||
}
|
||||
|
||||
func TestHTTPClientTreatsMissingDeleteAsSuccess(t *testing.T) {
|
||||
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, _ *http.Request) { http.NotFound(w, nil) }))
|
||||
defer server.Close()
|
||||
client, err := NewHTTPClient(server.URL, "test-key", time.Second)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := client.Delete(context.Background(), "missing", "delete-missing"); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,213 @@
|
||||
package providermemory
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
|
||||
"data-terminal-cloud/internal/audit"
|
||||
"data-terminal-cloud/internal/database"
|
||||
"data-terminal-cloud/internal/outbox"
|
||||
|
||||
"github.com/jackc/pgx/v5"
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
)
|
||||
|
||||
var ErrMemoryChangedDuringProviderSync = errors.New("memory changed during provider synchronization")
|
||||
|
||||
type Processor struct {
|
||||
pool *pgxpool.Pool
|
||||
client Client
|
||||
audit *audit.Repository
|
||||
}
|
||||
|
||||
type providerSnapshot struct {
|
||||
tenantInternalID string
|
||||
tenantID string
|
||||
householdID string
|
||||
subjectID string
|
||||
memoryID string
|
||||
memoryVersionID string
|
||||
content string
|
||||
contentDigest string
|
||||
providerRef string
|
||||
status string
|
||||
currentVersionID string
|
||||
sensitivity string
|
||||
versionNo int64
|
||||
consentEffective bool
|
||||
}
|
||||
|
||||
func NewProcessor(pool *pgxpool.Pool, client Client) (*Processor, error) {
|
||||
if pool == nil || client == nil {
|
||||
return nil, errors.New("provider memory processor requires database and client")
|
||||
}
|
||||
return &Processor{pool: pool, client: client, audit: audit.NewRepository()}, nil
|
||||
}
|
||||
|
||||
func (p *Processor) HandleSync(ctx context.Context, event outbox.Event) error {
|
||||
var memoryVersionID string
|
||||
if err := json.Unmarshal(event.Payload, &struct {
|
||||
MemoryVersionID *string `json:"memory_version_id"`
|
||||
}{MemoryVersionID: &memoryVersionID}); err != nil || memoryVersionID == "" {
|
||||
return errors.New("provider memory sync event requires memory_version_id")
|
||||
}
|
||||
before, err := p.loadSnapshot(ctx, p.pool, memoryVersionID, false)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !providerEligible(before, memoryVersionID) {
|
||||
if before.providerRef != "" {
|
||||
if err := p.client.Delete(ctx, before.providerRef, "provider-memory-stale:"+memoryVersionID); err != nil {
|
||||
return p.markFailed(ctx, memoryVersionID, err)
|
||||
}
|
||||
}
|
||||
_, err := p.pool.Exec(ctx, `UPDATE memory_index_entry SET status='STALE',provider_ref=NULL,last_attempt_at=now(),last_error='memory is not currently eligible',updated_at=now() WHERE memory_version_id=$1::uuid AND index_type='provider'`, memoryVersionID)
|
||||
return err
|
||||
}
|
||||
if err := p.deleteSupersededRefs(ctx, before.memoryID, memoryVersionID); err != nil {
|
||||
return p.markFailed(ctx, memoryVersionID, err)
|
||||
}
|
||||
providerRef, err := p.client.Upsert(ctx, Memory{
|
||||
TenantID: before.tenantID, HouseholdID: before.householdID, SubjectID: before.subjectID,
|
||||
MemoryID: before.memoryID, MemoryVersion: before.memoryVersionID, VersionNo: before.versionNo,
|
||||
Content: before.content, ContentDigest: before.contentDigest, IdempotencyKey: "provider-memory-upsert:" + memoryVersionID,
|
||||
})
|
||||
if err != nil {
|
||||
return p.markFailed(ctx, memoryVersionID, err)
|
||||
}
|
||||
|
||||
tx, err := p.pool.BeginTx(ctx, pgx.TxOptions{})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer tx.Rollback(context.Background())
|
||||
current, err := p.loadSnapshot(ctx, tx, memoryVersionID, true)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !providerEligible(current, memoryVersionID) || current.contentDigest != before.contentDigest {
|
||||
_ = p.client.Delete(ctx, providerRef, "provider-memory-rollback:"+memoryVersionID)
|
||||
return outbox.Retryable(ErrMemoryChangedDuringProviderSync)
|
||||
}
|
||||
if _, err := tx.Exec(ctx, `UPDATE memory_index_entry SET status='ACTIVE',provider_ref=$2,indexed_at=now(),last_attempt_at=now(),last_error=NULL,updated_at=now() WHERE memory_version_id=$1::uuid AND index_type='provider'`, memoryVersionID, providerRef); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := p.audit.Record(ctx, tx, audit.Event{TenantID: ¤t.tenantInternalID, ActorType: "system", Action: "memory.provider_sync.complete", ResourceType: "memory", ResourceID: ¤t.memoryID, Result: "success", Detail: map[string]any{"memory_version_id": current.memoryVersionID}}); err != nil {
|
||||
return err
|
||||
}
|
||||
return tx.Commit(ctx)
|
||||
}
|
||||
|
||||
func (p *Processor) HandleDelete(ctx context.Context, event outbox.Event) error {
|
||||
rows, err := p.pool.Query(ctx, `SELECT memory_version_id::text,COALESCE(provider_ref,'') FROM memory_index_entry WHERE memory_id=$1::uuid AND index_type='provider' AND status<>'DELETED' ORDER BY created_at,id`, event.AggregateID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
type reference struct{ versionID, providerRef string }
|
||||
refs := make([]reference, 0)
|
||||
for rows.Next() {
|
||||
var item reference
|
||||
if err := rows.Scan(&item.versionID, &item.providerRef); err != nil {
|
||||
rows.Close()
|
||||
return err
|
||||
}
|
||||
refs = append(refs, item)
|
||||
}
|
||||
rows.Close()
|
||||
if err := rows.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
for _, item := range refs {
|
||||
if item.providerRef != "" {
|
||||
if err := p.client.Delete(ctx, item.providerRef, "provider-memory-delete:"+item.versionID); err != nil {
|
||||
return p.markFailed(ctx, item.versionID, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
return database.Transact(ctx, p.pool, func(tx pgx.Tx) error {
|
||||
var tenantID, memoryPublicID string
|
||||
err := tx.QueryRow(ctx, `SELECT tenant_id::text,public_id::text FROM curated_memory WHERE id=$1::uuid FOR UPDATE`, event.AggregateID).Scan(&tenantID, &memoryPublicID)
|
||||
if errors.Is(err, pgx.ErrNoRows) {
|
||||
return nil
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := tx.Exec(ctx, `UPDATE memory_index_entry SET embedding=NULL,provider_ref=NULL,status='DELETED',indexed_at=NULL,last_attempt_at=now(),last_error=NULL,updated_at=now() WHERE memory_id=$1::uuid AND index_type='provider'`, event.AggregateID); err != nil {
|
||||
return err
|
||||
}
|
||||
return p.audit.Record(ctx, tx, audit.Event{TenantID: &tenantID, ActorType: "system", Action: "memory.provider_delete.complete", ResourceType: "memory", ResourceID: &memoryPublicID, Result: "success", Detail: map[string]any{"provider_refs_deleted": len(refs)}})
|
||||
})
|
||||
}
|
||||
|
||||
type snapshotQueryer interface {
|
||||
QueryRow(context.Context, string, ...any) pgx.Row
|
||||
}
|
||||
|
||||
func (p *Processor) loadSnapshot(ctx context.Context, q snapshotQueryer, versionID string, lock bool) (providerSnapshot, error) {
|
||||
lockClause := ""
|
||||
if lock {
|
||||
lockClause = " FOR UPDATE OF mi,cm,sp"
|
||||
}
|
||||
var s providerSnapshot
|
||||
err := q.QueryRow(ctx, `SELECT t.id::text,t.public_id::text,h.public_id::text,sp.public_id::text,cm.public_id::text,mv.public_id::text,
|
||||
mv.content,mv.content_digest,COALESCE(mi.provider_ref,''),cm.status,COALESCE(cm.current_version_id::text,''),cm.sensitivity,mv.version_no,
|
||||
EXISTS(SELECT 1 FROM consent_record cr WHERE cr.tenant_id=cm.tenant_id AND cr.subject_id=cm.subject_id AND cr.consent_type='personalized_memory' AND cr.withdrawn_at IS NULL AND (cr.expires_at IS NULL OR cr.expires_at>now()))
|
||||
FROM memory_index_entry mi JOIN memory_version mv ON mv.id=mi.memory_version_id JOIN curated_memory cm ON cm.id=mi.memory_id
|
||||
JOIN subject_profile sp ON sp.id=cm.subject_id AND sp.tenant_id=cm.tenant_id AND sp.status IN ('active','suspended')
|
||||
JOIN household h ON h.id=sp.household_id AND h.tenant_id=sp.tenant_id JOIN tenant t ON t.id=cm.tenant_id
|
||||
WHERE mi.memory_version_id=$1::uuid AND mi.index_type='provider'`+lockClause, versionID).Scan(
|
||||
&s.tenantInternalID, &s.tenantID, &s.householdID, &s.subjectID, &s.memoryID, &s.memoryVersionID, &s.content, &s.contentDigest, &s.providerRef, &s.status, &s.currentVersionID, &s.sensitivity, &s.versionNo, &s.consentEffective)
|
||||
if errors.Is(err, pgx.ErrNoRows) {
|
||||
return providerSnapshot{}, errors.New("provider memory index entry not found")
|
||||
}
|
||||
return s, err
|
||||
}
|
||||
|
||||
func providerEligible(s providerSnapshot, internalVersionID string) bool {
|
||||
return s.status == "ACTIVE" && s.currentVersionID == internalVersionID && s.consentEffective && s.sensitivity == "normal"
|
||||
}
|
||||
|
||||
func (p *Processor) deleteSupersededRefs(ctx context.Context, memoryPublicID, currentVersionID string) error {
|
||||
rows, err := p.pool.Query(ctx, `SELECT memory_version_id::text,provider_ref FROM memory_index_entry mi JOIN curated_memory cm ON cm.id=mi.memory_id WHERE cm.public_id=$1::uuid AND mi.index_type='provider' AND mi.memory_version_id<>$2::uuid AND mi.provider_ref IS NOT NULL AND mi.status<>'DELETED'`, memoryPublicID, currentVersionID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
type ref struct{ versionID, providerRef string }
|
||||
items := []ref{}
|
||||
for rows.Next() {
|
||||
var item ref
|
||||
if err := rows.Scan(&item.versionID, &item.providerRef); err != nil {
|
||||
rows.Close()
|
||||
return err
|
||||
}
|
||||
items = append(items, item)
|
||||
}
|
||||
rows.Close()
|
||||
if err := rows.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
for _, item := range items {
|
||||
if err := p.client.Delete(ctx, item.providerRef, "provider-memory-supersede:"+item.versionID); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if len(items) > 0 {
|
||||
_, err = p.pool.Exec(ctx, `UPDATE memory_index_entry mi SET status='DELETED',provider_ref=NULL,last_error=NULL,updated_at=now() FROM curated_memory cm WHERE mi.memory_id=cm.id AND cm.public_id=$1::uuid AND mi.index_type='provider' AND mi.memory_version_id<>$2::uuid`, memoryPublicID, currentVersionID)
|
||||
}
|
||||
return err
|
||||
}
|
||||
|
||||
func (p *Processor) markFailed(ctx context.Context, versionID string, cause error) error {
|
||||
message := cause.Error()
|
||||
if len(message) > 450 {
|
||||
message = message[:450]
|
||||
}
|
||||
_, _ = p.pool.Exec(ctx, `UPDATE memory_index_entry SET status='FAILED',last_attempt_at=now(),last_error=$2,updated_at=now() WHERE memory_version_id=$1::uuid AND index_type='provider'`, versionID, message)
|
||||
if outbox.IsRetryable(cause) {
|
||||
return cause
|
||||
}
|
||||
return fmt.Errorf("provider memory synchronization failed: %w", cause)
|
||||
}
|
||||
@@ -0,0 +1,86 @@
|
||||
package providermemory
|
||||
|
||||
import (
|
||||
"context"
|
||||
"os"
|
||||
"testing"
|
||||
|
||||
"data-terminal-cloud/internal/outbox"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
)
|
||||
|
||||
type recordingClient struct {
|
||||
upserts []Memory
|
||||
deletes []string
|
||||
}
|
||||
|
||||
func (c *recordingClient) Upsert(_ context.Context, memory Memory) (string, error) {
|
||||
c.upserts = append(c.upserts, memory)
|
||||
return "provider-ref-" + memory.MemoryVersion, nil
|
||||
}
|
||||
func (c *recordingClient) Delete(_ context.Context, providerRef, _ string) error {
|
||||
c.deletes = append(c.deletes, providerRef)
|
||||
return nil
|
||||
}
|
||||
|
||||
func TestProcessorSynchronizesAndDeletesProviderMemory(t *testing.T) {
|
||||
databaseURL := os.Getenv("TEST_DATABASE_URL")
|
||||
if databaseURL == "" {
|
||||
t.Skip("TEST_DATABASE_URL is not set")
|
||||
}
|
||||
ctx := context.Background()
|
||||
pool, err := pgxpool.New(ctx, databaseURL)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer pool.Close()
|
||||
memoryID, memoryPublicID, versionID, versionPublicID := uuid.NewString(), uuid.NewString(), uuid.NewString(), uuid.NewString()
|
||||
defer func() {
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM audit_event WHERE resource_id=$1::uuid`, memoryPublicID)
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM memory_index_entry WHERE memory_id=$1::uuid`, memoryID)
|
||||
_, _ = pool.Exec(ctx, `UPDATE curated_memory SET current_version_id=NULL WHERE id=$1::uuid`, memoryID)
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM memory_version WHERE memory_id=$1::uuid`, memoryID)
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM curated_memory WHERE id=$1::uuid`, memoryID)
|
||||
}()
|
||||
if _, err := pool.Exec(ctx, `INSERT INTO curated_memory(id,public_id,tenant_id,subject_id,status,sensitivity,valid_from) VALUES($1::uuid,$2::uuid,'10000000-0000-0000-0000-000000000001','10000000-0000-0000-0000-000000000401','ACTIVE','normal',now())`, memoryID, memoryPublicID); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := pool.Exec(ctx, `INSERT INTO memory_version(id,public_id,memory_id,version_no,content,source_refs,confirmed_by,change_reason,content_digest) VALUES($1::uuid,$2::uuid,$3::uuid,1,'喜欢在周末照料兰花','[{"type":"integration"}]'::jsonb,'10000000-0000-0000-0000-000000000101','integration',repeat('a',64))`, versionID, versionPublicID, memoryID); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := pool.Exec(ctx, `UPDATE curated_memory SET current_version_id=$2::uuid WHERE id=$1::uuid`, memoryID, versionID); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := pool.Exec(ctx, `INSERT INTO memory_index_entry(tenant_id,subject_id,memory_id,memory_version_id,index_type,status) VALUES('10000000-0000-0000-0000-000000000001','10000000-0000-0000-0000-000000000401',$1::uuid,$2::uuid,'provider','PENDING')`, memoryID, versionID); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
client := &recordingClient{}
|
||||
processor, err := NewProcessor(pool, client)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
payload := []byte(`{"memory_version_id":"` + versionID + `"}`)
|
||||
if err := processor.HandleSync(ctx, outbox.Event{AggregateID: memoryID, Payload: payload}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var status string
|
||||
var providerRef *string
|
||||
if err := pool.QueryRow(ctx, `SELECT status,provider_ref FROM memory_index_entry WHERE memory_version_id=$1::uuid AND index_type='provider'`, versionID).Scan(&status, &providerRef); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if status != "ACTIVE" || providerRef == nil || len(client.upserts) != 1 || client.upserts[0].SubjectID != "10000000-0000-0000-0000-000000000411" {
|
||||
t.Fatalf("status=%s provider=%v upserts=%+v", status, providerRef, client.upserts)
|
||||
}
|
||||
if err := processor.HandleDelete(ctx, outbox.Event{AggregateID: memoryID}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := pool.QueryRow(ctx, `SELECT status,provider_ref FROM memory_index_entry WHERE memory_version_id=$1::uuid AND index_type='provider'`, versionID).Scan(&status, &providerRef); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if status != "DELETED" || providerRef != nil || len(client.deletes) != 1 {
|
||||
t.Fatalf("status=%s provider=%v deletes=%v", status, providerRef, client.deletes)
|
||||
}
|
||||
}
|
||||
@@ -23,6 +23,7 @@ var (
|
||||
ErrStorySourceRequired = errors.New("story source is required")
|
||||
ErrPersonalizedMemoryConsentNeeded = errors.New("personalized memory consent is required")
|
||||
ErrReviewStateConflict = errors.New("content review state does not allow this operation")
|
||||
ErrFactReviewStateConflict = errors.New("fact candidate review state does not allow this operation")
|
||||
)
|
||||
|
||||
type ConversationTurn struct {
|
||||
@@ -74,6 +75,15 @@ type FactCandidate struct {
|
||||
Version int64 `json:"version"`
|
||||
}
|
||||
|
||||
type UpdateFactCandidateInput struct {
|
||||
FactType *string
|
||||
Content *string
|
||||
Sensitivity *string
|
||||
Status *string
|
||||
ErrorType *string
|
||||
Reason string
|
||||
}
|
||||
|
||||
type Story struct {
|
||||
ID string `json:"id"`
|
||||
SubjectID string `json:"subject_id"`
|
||||
@@ -95,29 +105,40 @@ type Story struct {
|
||||
}
|
||||
|
||||
type CuratedMemory struct {
|
||||
ID string `json:"id"`
|
||||
SubjectID string `json:"subject_id"`
|
||||
Content string `json:"content"`
|
||||
SourceRefs []map[string]any `json:"source_refs"`
|
||||
Status string `json:"status"`
|
||||
Sensitivity string `json:"sensitivity"`
|
||||
VersionNo int64 `json:"version_no"`
|
||||
Version int64 `json:"version"`
|
||||
ConfirmedBy string `json:"confirmed_by"`
|
||||
ConfirmedAt time.Time `json:"confirmed_at"`
|
||||
ChangeReason string `json:"change_reason,omitempty"`
|
||||
IndexStatus string `json:"index_status"`
|
||||
LastIndexError string `json:"last_index_error,omitempty"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
UpdatedAt time.Time `json:"updated_at"`
|
||||
ID string `json:"id"`
|
||||
SubjectID string `json:"subject_id"`
|
||||
Content string `json:"content"`
|
||||
SourceRefs []map[string]any `json:"source_refs"`
|
||||
Status string `json:"status"`
|
||||
Sensitivity string `json:"sensitivity"`
|
||||
VersionNo int64 `json:"version_no"`
|
||||
Version int64 `json:"version"`
|
||||
ConfirmedBy string `json:"confirmed_by"`
|
||||
ConfirmedAt time.Time `json:"confirmed_at"`
|
||||
ChangeReason string `json:"change_reason,omitempty"`
|
||||
IndexStatus string `json:"index_status"`
|
||||
LastIndexError string `json:"last_index_error,omitempty"`
|
||||
ProviderIndexStatus string `json:"provider_index_status"`
|
||||
ProviderIndexError string `json:"provider_index_error,omitempty"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
UpdatedAt time.Time `json:"updated_at"`
|
||||
}
|
||||
|
||||
type UpdateMemoryInput struct {
|
||||
Content *string
|
||||
Status *string
|
||||
Reason string
|
||||
Content *string
|
||||
Status *string
|
||||
ErrorType *string
|
||||
Reason string
|
||||
}
|
||||
|
||||
var contentErrorTypes = map[string]bool{
|
||||
"PERSON_ERROR": true, "TIME_ERROR": true, "PLACE_ERROR": true, "RELATION_ERROR": true,
|
||||
"OVER_INFERENCE": true, "KEY_POINT_OMISSION": true, "DUPLICATE_CONTENT": true,
|
||||
"UNNATURAL_EXPRESSION": true, "SENSITIVE_INFORMATION": true,
|
||||
}
|
||||
|
||||
func IsContentErrorType(value string) bool { return contentErrorTypes[value] }
|
||||
|
||||
func (s *Service) ListConversationTurns(ctx context.Context, principal identity.Principal, conversationPublicID string, limit int, cursor string) (ConversationTurnPage, error) {
|
||||
access, conversationID, err := s.resolveConversationContent(ctx, s.pool, principal, conversationPublicID, identity.CapabilityConversationRead, false)
|
||||
if err != nil {
|
||||
@@ -229,8 +250,92 @@ func (s *Service) GetStory(ctx context.Context, principal identity.Principal, st
|
||||
return story, nil
|
||||
}
|
||||
|
||||
func (s *Service) ReviseStory(ctx context.Context, principal identity.Principal, storyPublicID, content, reason, requestID string, expectedVersion int64) (Story, error) {
|
||||
func (s *Service) UpdateFactCandidate(ctx context.Context, principal identity.Principal, storyPublicID, factPublicID, requestID string, expectedVersion int64, input UpdateFactCandidateInput) (FactCandidate, error) {
|
||||
var result FactCandidate
|
||||
if input.ErrorType != nil && !IsContentErrorType(*input.ErrorType) {
|
||||
return result, ErrFactReviewStateConflict
|
||||
}
|
||||
err := database.Transact(ctx, s.pool, func(tx pgx.Tx) error {
|
||||
access, story, err := s.resolveStory(ctx, tx, principal, storyPublicID, identity.CapabilityStoryReview, true)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if story.Status == "CONFIRMED" || story.Status == "REJECTED" || story.Status == "SUPERSEDED" {
|
||||
return ErrFactReviewStateConflict
|
||||
}
|
||||
var internalID, factType, content, statementBasis, sensitivity, status string
|
||||
var confidence *float64
|
||||
var sourceJSON []byte
|
||||
var version int64
|
||||
if err := tx.QueryRow(ctx, `
|
||||
SELECT fc.id::text,fc.fact_type,fc.content,fc.statement_basis,fc.confidence,fc.sensitivity,fc.source_turn_ids,fc.status,fc.version
|
||||
FROM fact_candidate fc JOIN story_draft sd ON sd.id=fc.story_id AND sd.tenant_id=fc.tenant_id
|
||||
WHERE fc.public_id=$1::uuid AND sd.public_id=$2::uuid FOR UPDATE OF fc`, factPublicID, storyPublicID).Scan(
|
||||
&internalID, &factType, &content, &statementBasis, &confidence, &sensitivity, &sourceJSON, &status, &version,
|
||||
); errors.Is(err, pgx.ErrNoRows) {
|
||||
return ErrNotFound
|
||||
} else if err != nil {
|
||||
return fmt.Errorf("lock fact candidate: %w", err)
|
||||
}
|
||||
if version != expectedVersion {
|
||||
return ErrVersionConflict
|
||||
}
|
||||
if status == "REJECTED" || status == "SUPERSEDED" || (status == "CONFIRMED" && (input.Status == nil || *input.Status != "SUPERSEDED")) {
|
||||
return ErrFactReviewStateConflict
|
||||
}
|
||||
previousFactType, previousContent, previousSensitivity, previousStatus := factType, content, sensitivity, status
|
||||
if input.FactType != nil {
|
||||
factType = strings.TrimSpace(*input.FactType)
|
||||
}
|
||||
if input.Content != nil {
|
||||
content = strings.TrimSpace(*input.Content)
|
||||
}
|
||||
if input.Sensitivity != nil {
|
||||
sensitivity = *input.Sensitivity
|
||||
}
|
||||
if input.Status != nil {
|
||||
status = *input.Status
|
||||
}
|
||||
if status != "CANDIDATE" && status != "CONFIRMED" && status != "REJECTED" && status != "SUPERSEDED" {
|
||||
return ErrFactReviewStateConflict
|
||||
}
|
||||
if _, err := tx.Exec(ctx, `INSERT INTO fact_candidate_revision(fact_candidate_id,version_no,fact_type,content,sensitivity,status,edited_by,edit_reason)
|
||||
VALUES($1::uuid,$2,$3,$4,$5,$6,$7::uuid,$8) ON CONFLICT(fact_candidate_id,version_no) DO NOTHING`, internalID, version, previousFactType, previousContent, previousSensitivity, previousStatus, access.userID, input.Reason); err != nil {
|
||||
return err
|
||||
}
|
||||
row := tx.QueryRow(ctx, `
|
||||
UPDATE fact_candidate SET fact_type=$1,content=$2,sensitivity=$3,status=$4,
|
||||
reviewed_by=$5::uuid,reviewed_at=now(),review_reason=$6,version=version+1,updated_at=now()
|
||||
WHERE id=$7::uuid
|
||||
RETURNING public_id::text,fact_type,content,statement_basis,confidence,sensitivity,source_turn_ids,status,version`,
|
||||
factType, content, sensitivity, status, access.userID, input.Reason, internalID)
|
||||
if err := scanFactCandidate(row, &result); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := tx.Exec(ctx, `INSERT INTO fact_candidate_revision(fact_candidate_id,version_no,fact_type,content,sensitivity,status,edited_by,edit_reason,error_type)
|
||||
VALUES($1::uuid,$2,$3,$4,$5,$6,$7::uuid,$8,$9)`, internalID, result.Version, result.FactType, result.Content, result.Sensitivity, result.Status, access.userID, input.Reason, input.ErrorType); err != nil {
|
||||
return err
|
||||
}
|
||||
action := reviewAction(result.Status, input.Content != nil || input.FactType != nil || input.Sensitivity != nil)
|
||||
if _, err := tx.Exec(ctx, `INSERT INTO content_review_signal(tenant_id,subject_id,resource_type,resource_public_id,action,error_type,reason,model_ref,prompt_version,generation_version,actor_id)
|
||||
SELECT sd.tenant_id,sd.subject_id,'fact',$1::uuid,$2,$3,$4,sr.model_ref,sr.prompt_version,sd.generation_version,$5::uuid
|
||||
FROM story_draft sd JOIN story_revision sr ON sr.story_id=sd.id AND sr.revision_no=1 WHERE sd.public_id=$6::uuid`, factPublicID, action, input.ErrorType, input.Reason, access.userID, storyPublicID); err != nil {
|
||||
return err
|
||||
}
|
||||
return s.audit.Record(ctx, tx, audit.Event{
|
||||
TenantID: strPtr(access.tenantID), ActorType: "user", ActorPublicID: strPtr(access.userPublicID),
|
||||
Action: "fact_candidate.update", ResourceType: "fact_candidate", ResourceID: strPtr(factPublicID),
|
||||
RequestID: requestID, Result: "success", Detail: map[string]any{"story_id": storyPublicID, "status": result.Status, "reason": input.Reason},
|
||||
})
|
||||
})
|
||||
return result, err
|
||||
}
|
||||
|
||||
func (s *Service) ReviseStory(ctx context.Context, principal identity.Principal, storyPublicID, content, reason, requestID string, expectedVersion int64, errorType string) (Story, error) {
|
||||
var result Story
|
||||
if errorType != "" && !IsContentErrorType(errorType) {
|
||||
return result, ErrReviewStateConflict
|
||||
}
|
||||
err := database.Transact(ctx, s.pool, func(tx pgx.Tx) error {
|
||||
access, current, err := s.resolveStory(ctx, tx, principal, storyPublicID, identity.CapabilityStoryReview, true)
|
||||
if err != nil {
|
||||
@@ -246,7 +351,7 @@ func (s *Service) ReviseStory(ctx context.Context, principal identity.Principal,
|
||||
if err := tx.QueryRow(ctx, `SELECT current_revision_id::text FROM story_draft WHERE public_id=$1::uuid`, storyPublicID).Scan(¤tRevisionID); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := tx.QueryRow(ctx, `INSERT INTO story_revision(story_id,revision_no,content,author_type,author_id,edit_reason,base_revision_id) SELECT id,$2,$3,'user',$4::uuid,$5,$6::uuid FROM story_draft WHERE public_id=$1::uuid RETURNING id::text`, storyPublicID, current.RevisionNo+1, content, access.userID, reason, currentRevisionID).Scan(&newRevisionID); err != nil {
|
||||
if err := tx.QueryRow(ctx, `INSERT INTO story_revision(story_id,revision_no,content,author_type,author_id,edit_reason,base_revision_id,error_type) SELECT id,$2,$3,'user',$4::uuid,$5,$6::uuid,NULLIF($7,'') FROM story_draft WHERE public_id=$1::uuid RETURNING id::text`, storyPublicID, current.RevisionNo+1, content, access.userID, reason, currentRevisionID, errorType).Scan(&newRevisionID); err != nil {
|
||||
return fmt.Errorf("insert story revision: %w", err)
|
||||
}
|
||||
if _, err := tx.Exec(ctx, `INSERT INTO story_source(story_revision_id,conversation_turn_id,source_start_offset,source_end_offset,source_role) SELECT $1::uuid,conversation_turn_id,source_start_offset,source_end_offset,source_role FROM story_source WHERE story_revision_id=$2::uuid`, newRevisionID, currentRevisionID); err != nil {
|
||||
@@ -257,6 +362,9 @@ func (s *Service) ReviseStory(ctx context.Context, principal identity.Principal,
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := insertStoryReviewSignal(ctx, tx, access, storyPublicID, "EDITED", errorType, reason); err != nil {
|
||||
return err
|
||||
}
|
||||
return s.audit.Record(ctx, tx, audit.Event{TenantID: strPtr(access.tenantID), ActorType: "user", ActorPublicID: strPtr(access.userPublicID), Action: "story.revise", ResourceType: "story", ResourceID: strPtr(storyPublicID), RequestID: requestID, Result: "success", Detail: map[string]any{"revision_no": result.RevisionNo, "reason": reason}})
|
||||
})
|
||||
if err != nil {
|
||||
@@ -268,8 +376,11 @@ func (s *Service) ReviseStory(ctx context.Context, principal identity.Principal,
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func (s *Service) RejectStory(ctx context.Context, principal identity.Principal, storyPublicID, reason, requestID string, expectedVersion int64) (Story, error) {
|
||||
func (s *Service) RejectStory(ctx context.Context, principal identity.Principal, storyPublicID, reason, requestID string, expectedVersion int64, errorType string) (Story, error) {
|
||||
var result Story
|
||||
if errorType != "" && !IsContentErrorType(errorType) {
|
||||
return result, ErrReviewStateConflict
|
||||
}
|
||||
err := database.Transact(ctx, s.pool, func(tx pgx.Tx) error {
|
||||
access, current, err := s.resolveStory(ctx, tx, principal, storyPublicID, identity.CapabilityStoryReview, true)
|
||||
if err != nil {
|
||||
@@ -289,6 +400,9 @@ func (s *Service) RejectStory(ctx context.Context, principal identity.Principal,
|
||||
if _, err := tx.Exec(ctx, `UPDATE fact_candidate SET status='REJECTED',reviewed_by=$1::uuid,reviewed_at=now(),review_reason=$2,version=version+1,updated_at=now() WHERE story_id=(SELECT id FROM story_draft WHERE public_id=$3::uuid) AND status='CANDIDATE'`, access.userID, reason, storyPublicID); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := insertStoryReviewSignal(ctx, tx, access, storyPublicID, "REJECTED", errorType, reason); err != nil {
|
||||
return err
|
||||
}
|
||||
return s.audit.Record(ctx, tx, audit.Event{TenantID: strPtr(access.tenantID), ActorType: "user", ActorPublicID: strPtr(access.userPublicID), Action: "story.reject", ResourceType: "story", ResourceID: strPtr(storyPublicID), RequestID: requestID, Result: "success", Detail: map[string]any{"reason": reason}})
|
||||
})
|
||||
if err != nil {
|
||||
@@ -349,14 +463,28 @@ func (s *Service) ConfirmStory(ctx context.Context, principal identity.Principal
|
||||
if _, err := tx.Exec(ctx, `INSERT INTO memory_index_entry(tenant_id,subject_id,memory_id,memory_version_id,index_type,status) VALUES($1::uuid,$2::uuid,$3::uuid,$4::uuid,'pgvector','PENDING')`, access.tenantID, access.subjectID, memoryID, memoryVersionID); err != nil {
|
||||
return err
|
||||
}
|
||||
if s.providerMemory {
|
||||
if _, err := tx.Exec(ctx, `INSERT INTO memory_index_entry(tenant_id,subject_id,memory_id,memory_version_id,index_type,status) VALUES($1::uuid,$2::uuid,$3::uuid,$4::uuid,'provider','PENDING')`, access.tenantID, access.subjectID, memoryID, memoryVersionID); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if _, err := tx.Exec(ctx, `UPDATE fact_candidate SET status='CONFIRMED',reviewed_by=$1::uuid,reviewed_at=now(),review_reason=$2,version=version+1,updated_at=now() WHERE story_id=(SELECT id FROM story_draft WHERE public_id=$3::uuid) AND status='CANDIDATE'`, access.userID, reason, storyPublicID); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := insertStoryReviewSignal(ctx, tx, access, storyPublicID, "CONFIRMED", "", reason); err != nil {
|
||||
return err
|
||||
}
|
||||
dedupe := "memory-index-v1:" + memoryVersionID
|
||||
payload, _ := json.Marshal(map[string]any{"memory_version_id": memoryVersionID})
|
||||
if _, err := outbox.NewStore(s.pool).Enqueue(ctx, tx, outbox.NewEvent{TenantID: &access.tenantID, AggregateType: "curated_memory", AggregateID: memoryID, EventType: "memory.index_requested", DeduplicationKey: &dedupe, Payload: payload}); err != nil {
|
||||
return err
|
||||
}
|
||||
if s.providerMemory {
|
||||
providerDedupe := "memory-provider-sync-v1:" + memoryVersionID
|
||||
if _, err := outbox.NewStore(s.pool).Enqueue(ctx, tx, outbox.NewEvent{TenantID: &access.tenantID, AggregateType: "curated_memory", AggregateID: memoryID, EventType: "memory.provider_sync_requested", DeduplicationKey: &providerDedupe, Payload: payload}); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if err := s.audit.Record(ctx, tx, audit.Event{TenantID: strPtr(access.tenantID), ActorType: "user", ActorPublicID: strPtr(access.userPublicID), Action: "story.confirm", ResourceType: "story", ResourceID: strPtr(storyPublicID), RequestID: requestID, Result: "success", Detail: map[string]any{"memory_id": memory.ID, "revision_no": story.RevisionNo}}); err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -408,6 +536,9 @@ func (s *Service) GetMemory(ctx context.Context, principal identity.Principal, m
|
||||
}
|
||||
|
||||
func (s *Service) UpdateMemory(ctx context.Context, principal identity.Principal, memoryPublicID, requestID string, expectedVersion int64, input UpdateMemoryInput) (CuratedMemory, error) {
|
||||
if input.ErrorType != nil && !IsContentErrorType(*input.ErrorType) {
|
||||
return CuratedMemory{}, ErrReviewStateConflict
|
||||
}
|
||||
err := database.Transact(ctx, s.pool, func(tx pgx.Tx) error {
|
||||
var internalID, tenantID, subjectID, userID, userPublicID, role string
|
||||
var currentVersionID, currentContent, currentStatus string
|
||||
@@ -469,9 +600,9 @@ func (s *Service) UpdateMemory(ctx context.Context, principal identity.Principal
|
||||
if targetContent != currentContent {
|
||||
digest := sha256.Sum256([]byte(targetContent))
|
||||
if err := tx.QueryRow(ctx, `
|
||||
INSERT INTO memory_version(memory_id,version_no,content,source_refs,confirmed_by,change_reason,content_digest)
|
||||
VALUES($1::uuid,$2,$3,$4::jsonb,$5::uuid,$6,$7) RETURNING id::text`,
|
||||
internalID, currentVersionNo+1, targetContent, sourceRefs, userID, input.Reason, hex.EncodeToString(digest[:])).Scan(&newVersionID); err != nil {
|
||||
INSERT INTO memory_version(memory_id,version_no,content,source_refs,confirmed_by,change_reason,content_digest,error_type)
|
||||
VALUES($1::uuid,$2,$3,$4::jsonb,$5::uuid,$6,$7,$8) RETURNING id::text`,
|
||||
internalID, currentVersionNo+1, targetContent, sourceRefs, userID, input.Reason, hex.EncodeToString(digest[:]), input.ErrorType).Scan(&newVersionID); err != nil {
|
||||
return fmt.Errorf("create memory revision: %w", err)
|
||||
}
|
||||
}
|
||||
@@ -487,16 +618,48 @@ func (s *Service) UpdateMemory(ctx context.Context, principal identity.Principal
|
||||
} else if _, err := tx.Exec(ctx, `INSERT INTO memory_index_entry(tenant_id,subject_id,memory_id,memory_version_id,index_type,status) VALUES($1::uuid,$2::uuid,$3::uuid,$4::uuid,'pgvector','PENDING')`, tenantID, subjectID, internalID, newVersionID); err != nil {
|
||||
return err
|
||||
}
|
||||
if s.providerMemory {
|
||||
if _, err := tx.Exec(ctx, `INSERT INTO memory_index_entry(tenant_id,subject_id,memory_id,memory_version_id,index_type,status,last_error,updated_at)
|
||||
VALUES($1::uuid,$2::uuid,$3::uuid,$4::uuid,'provider','PENDING',NULL,now())
|
||||
ON CONFLICT(memory_version_id,index_type) DO UPDATE SET status='PENDING',last_error=NULL,updated_at=now()`, tenantID, subjectID, internalID, newVersionID); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
if _, err := tx.Exec(ctx, `UPDATE curated_memory SET current_version_id=$1::uuid,status=$2,version=version+1,updated_at=now() WHERE id=$3::uuid`, newVersionID, targetStatus, internalID); err != nil {
|
||||
return err
|
||||
}
|
||||
action := "EDITED"
|
||||
if targetStatus == "DISABLED" {
|
||||
action = "DISABLED"
|
||||
} else if targetContent == currentContent {
|
||||
action = "CONFIRMED"
|
||||
}
|
||||
if _, err := tx.Exec(ctx, `INSERT INTO content_review_signal(tenant_id,subject_id,resource_type,resource_public_id,action,error_type,reason,model_ref,prompt_version,generation_version,actor_id)
|
||||
SELECT $1::uuid,$2::uuid,'memory',$3::uuid,$4,$5,$6,sr.model_ref,sr.prompt_version,sd.generation_version,$7::uuid
|
||||
FROM memory_version mv
|
||||
LEFT JOIN story_draft sd ON sd.public_id::text=mv.source_refs->0->>'story_id'
|
||||
LEFT JOIN story_revision sr ON sr.id=sd.current_revision_id
|
||||
WHERE mv.id=$8::uuid`, tenantID, subjectID, memoryPublicID, action, input.ErrorType, input.Reason, userID, newVersionID); err != nil {
|
||||
return err
|
||||
}
|
||||
if targetStatus == "ACTIVE" {
|
||||
dedupe := fmt.Sprintf("memory-index-update-v%d:%s", resourceVersion+1, newVersionID)
|
||||
payload, _ := json.Marshal(map[string]any{"memory_version_id": newVersionID})
|
||||
if _, err := outbox.NewStore(s.pool).Enqueue(ctx, tx, outbox.NewEvent{TenantID: &tenantID, AggregateType: "curated_memory", AggregateID: internalID, EventType: "memory.index_requested", DeduplicationKey: &dedupe, Payload: payload}); err != nil {
|
||||
return err
|
||||
}
|
||||
if s.providerMemory {
|
||||
providerDedupe := fmt.Sprintf("memory-provider-sync-update-v%d:%s", resourceVersion+1, newVersionID)
|
||||
if _, err := outbox.NewStore(s.pool).Enqueue(ctx, tx, outbox.NewEvent{TenantID: &tenantID, AggregateType: "curated_memory", AggregateID: internalID, EventType: "memory.provider_sync_requested", DeduplicationKey: &providerDedupe, Payload: payload}); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
} else if s.providerMemory {
|
||||
providerDedupe := fmt.Sprintf("memory-provider-delete-disabled-v%d:%s", resourceVersion+1, internalID)
|
||||
if _, err := outbox.NewStore(s.pool).Enqueue(ctx, tx, outbox.NewEvent{TenantID: &tenantID, AggregateType: "curated_memory", AggregateID: internalID, EventType: "memory.provider_delete_requested", DeduplicationKey: &providerDedupe, Payload: []byte(`{"reason":"memory disabled"}`)}); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return s.audit.Record(ctx, tx, audit.Event{TenantID: strPtr(tenantID), ActorType: "user", ActorPublicID: strPtr(userPublicID), Action: "memory.update", ResourceType: "memory", ResourceID: strPtr(memoryPublicID), RequestID: requestID, Result: "success", Detail: map[string]any{"from_status": currentStatus, "to_status": targetStatus, "content_revised": targetContent != currentContent, "reason": input.Reason}})
|
||||
})
|
||||
@@ -542,6 +705,12 @@ func (s *Service) RequestMemoryDeletion(ctx context.Context, principal identity.
|
||||
}
|
||||
dedupe := "memory-delete:" + internalID
|
||||
payload, _ := json.Marshal(map[string]any{"reason": reason})
|
||||
if s.providerMemory {
|
||||
providerDedupe := "memory-provider-delete:" + internalID
|
||||
if _, err := outbox.NewStore(s.pool).Enqueue(ctx, tx, outbox.NewEvent{TenantID: &tenantID, AggregateType: "curated_memory", AggregateID: internalID, EventType: "memory.provider_delete_requested", DeduplicationKey: &providerDedupe, Payload: payload}); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if _, err := outbox.NewStore(s.pool).Enqueue(ctx, tx, outbox.NewEvent{TenantID: &tenantID, AggregateType: "curated_memory", AggregateID: internalID, EventType: "memory.delete_requested", DeduplicationKey: &dedupe, Payload: payload}); err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -558,6 +727,27 @@ type contentAccess struct {
|
||||
role identity.Role
|
||||
}
|
||||
|
||||
func reviewAction(status string, edited bool) string {
|
||||
if edited && status == "CANDIDATE" {
|
||||
return "EDITED"
|
||||
}
|
||||
switch status {
|
||||
case "CONFIRMED":
|
||||
return "CONFIRMED"
|
||||
case "REJECTED", "SUPERSEDED":
|
||||
return "REJECTED"
|
||||
default:
|
||||
return "EDITED"
|
||||
}
|
||||
}
|
||||
|
||||
func insertStoryReviewSignal(ctx context.Context, tx pgx.Tx, access contentAccess, storyPublicID, action, errorType, reason string) error {
|
||||
_, err := tx.Exec(ctx, `INSERT INTO content_review_signal(tenant_id,subject_id,resource_type,resource_public_id,action,error_type,reason,model_ref,prompt_version,generation_version,actor_id)
|
||||
SELECT sd.tenant_id,sd.subject_id,'story',sd.public_id,$2,NULLIF($3,''),$4,sr.model_ref,sr.prompt_version,sd.generation_version,$5::uuid
|
||||
FROM story_draft sd JOIN story_revision sr ON sr.story_id=sd.id AND sr.revision_no=1 WHERE sd.public_id=$1::uuid`, storyPublicID, action, errorType, reason, access.userID)
|
||||
return err
|
||||
}
|
||||
|
||||
func (s *Service) resolveConversationContent(ctx context.Context, q queryRow, p identity.Principal, publicID string, cap identity.Capability, lock bool) (contentAccess, string, error) {
|
||||
sql := `SELECT c.tenant_id::text,c.subject_id::text,ua.id::text,ua.public_id::text,hm.role,c.id::text FROM conversation c JOIN subject_profile sp ON sp.id=c.subject_id AND sp.tenant_id=c.tenant_id AND sp.status IN ('active','suspended') JOIN household h ON h.id=c.household_id AND h.tenant_id=c.tenant_id JOIN household_member hm ON hm.household_id=h.id AND hm.status='active' JOIN user_account ua ON ua.id=hm.user_id AND ua.tenant_id=h.tenant_id AND ua.status='active' WHERE c.public_id=$1::uuid AND ua.oidc_issuer=$2 AND ua.oidc_subject=$3`
|
||||
if lock {
|
||||
@@ -641,11 +831,7 @@ func (s *Service) loadStoryChildren(ctx context.Context, story *Story) error {
|
||||
story.Facts = []FactCandidate{}
|
||||
for rows.Next() {
|
||||
var x FactCandidate
|
||||
var source []byte
|
||||
if err := rows.Scan(&x.ID, &x.FactType, &x.Content, &x.StatementBasis, &x.Confidence, &x.Sensitivity, &source, &x.Status, &x.Version); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := json.Unmarshal(source, &x.SourceTurnIDs); err != nil {
|
||||
if err := scanFactCandidate(rows, &x); err != nil {
|
||||
return err
|
||||
}
|
||||
story.Facts = append(story.Facts, x)
|
||||
@@ -653,12 +839,20 @@ func (s *Service) loadStoryChildren(ctx context.Context, story *Story) error {
|
||||
return rows.Err()
|
||||
}
|
||||
|
||||
const memorySelectSQL = `SELECT cm.public_id::text,sp.public_id::text,mv.content,mv.source_refs,cm.status,cm.sensitivity,mv.version_no,cm.version,ua.public_id::text,mv.confirmed_at,COALESCE(mv.change_reason,''),COALESCE(mi.status,'PENDING'),COALESCE(mi.last_error,''),cm.created_at,cm.updated_at FROM curated_memory cm JOIN subject_profile sp ON sp.id=cm.subject_id AND sp.tenant_id=cm.tenant_id JOIN memory_version mv ON mv.id=cm.current_version_id JOIN user_account ua ON ua.id=mv.confirmed_by LEFT JOIN memory_index_entry mi ON mi.memory_version_id=mv.id AND mi.index_type='pgvector'`
|
||||
func scanFactCandidate(row rowScanner, result *FactCandidate) error {
|
||||
var source []byte
|
||||
if err := row.Scan(&result.ID, &result.FactType, &result.Content, &result.StatementBasis, &result.Confidence, &result.Sensitivity, &source, &result.Status, &result.Version); err != nil {
|
||||
return err
|
||||
}
|
||||
return json.Unmarshal(source, &result.SourceTurnIDs)
|
||||
}
|
||||
|
||||
const memorySelectSQL = `SELECT cm.public_id::text,sp.public_id::text,mv.content,mv.source_refs,cm.status,cm.sensitivity,mv.version_no,cm.version,ua.public_id::text,mv.confirmed_at,COALESCE(mv.change_reason,''),COALESCE(mi.status,'PENDING'),COALESCE(mi.last_error,''),COALESCE(provider_mi.status,'DISABLED'),COALESCE(provider_mi.last_error,''),cm.created_at,cm.updated_at FROM curated_memory cm JOIN subject_profile sp ON sp.id=cm.subject_id AND sp.tenant_id=cm.tenant_id JOIN memory_version mv ON mv.id=cm.current_version_id JOIN user_account ua ON ua.id=mv.confirmed_by LEFT JOIN memory_index_entry mi ON mi.memory_version_id=mv.id AND mi.index_type='pgvector' LEFT JOIN memory_index_entry provider_mi ON provider_mi.memory_version_id=mv.id AND provider_mi.index_type='provider'`
|
||||
|
||||
func scanMemory(row rowScanner) (CuratedMemory, error) {
|
||||
var m CuratedMemory
|
||||
var sourceJSON []byte
|
||||
if err := row.Scan(&m.ID, &m.SubjectID, &m.Content, &sourceJSON, &m.Status, &m.Sensitivity, &m.VersionNo, &m.Version, &m.ConfirmedBy, &m.ConfirmedAt, &m.ChangeReason, &m.IndexStatus, &m.LastIndexError, &m.CreatedAt, &m.UpdatedAt); err != nil {
|
||||
if err := row.Scan(&m.ID, &m.SubjectID, &m.Content, &sourceJSON, &m.Status, &m.Sensitivity, &m.VersionNo, &m.Version, &m.ConfirmedBy, &m.ConfirmedAt, &m.ChangeReason, &m.IndexStatus, &m.LastIndexError, &m.ProviderIndexStatus, &m.ProviderIndexError, &m.CreatedAt, &m.UpdatedAt); err != nil {
|
||||
return CuratedMemory{}, err
|
||||
}
|
||||
if err := json.Unmarshal(sourceJSON, &m.SourceRefs); err != nil {
|
||||
|
||||
@@ -0,0 +1,166 @@
|
||||
package saas
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"data-terminal-cloud/internal/audit"
|
||||
"data-terminal-cloud/internal/database"
|
||||
"data-terminal-cloud/internal/identity"
|
||||
|
||||
"github.com/jackc/pgx/v5"
|
||||
)
|
||||
|
||||
const (
|
||||
DeviceBindingActionUnbind = "unbind"
|
||||
DeviceBindingActionTransfer = "transfer"
|
||||
)
|
||||
|
||||
type DeviceBindingImpact struct {
|
||||
DeviceID string `json:"device_id"`
|
||||
BindingID string `json:"binding_id"`
|
||||
Action string `json:"action"`
|
||||
SourceHouseholdID string `json:"source_household_id"`
|
||||
SourceSubjectID string `json:"source_subject_id"`
|
||||
SourceSubjectName string `json:"source_subject_name"`
|
||||
TargetHouseholdID *string `json:"target_household_id,omitempty"`
|
||||
TargetSubjectID *string `json:"target_subject_id,omitempty"`
|
||||
TargetSubjectName *string `json:"target_subject_name,omitempty"`
|
||||
ActiveConversationID *string `json:"active_conversation_id,omitempty"`
|
||||
PendingCommandCount int `json:"pending_command_count"`
|
||||
ActiveFirmwareTaskCount int `json:"active_firmware_task_count"`
|
||||
Allowed bool `json:"allowed"`
|
||||
Blockers []string `json:"blockers"`
|
||||
EvaluatedAt time.Time `json:"evaluated_at"`
|
||||
}
|
||||
|
||||
type ChangeDeviceBindingInput struct {
|
||||
Action string
|
||||
ExpectedBindingID string
|
||||
TargetSubjectID string
|
||||
Reason string
|
||||
Acknowledged bool
|
||||
}
|
||||
|
||||
type DeviceBindingChangeResult struct {
|
||||
DeviceID string `json:"device_id"`
|
||||
Action string `json:"action"`
|
||||
PreviousBindingID string `json:"previous_binding_id"`
|
||||
BindingID *string `json:"binding_id,omitempty"`
|
||||
HouseholdID *string `json:"household_id,omitempty"`
|
||||
SubjectID *string `json:"subject_id,omitempty"`
|
||||
ChangedAt time.Time `json:"changed_at"`
|
||||
}
|
||||
|
||||
func (s *Service) GetDeviceBindingImpact(ctx context.Context, principal identity.Principal, devicePublicID, action, targetSubjectPublicID string) (DeviceBindingImpact, error) {
|
||||
access, device, err := resolveDevice(ctx, s.pool, principal, devicePublicID, identity.CapabilityDeviceManage, false)
|
||||
if err != nil {
|
||||
return DeviceBindingImpact{}, err
|
||||
}
|
||||
return s.readDeviceBindingImpact(ctx, s.pool, principal, access, device, action, targetSubjectPublicID)
|
||||
}
|
||||
|
||||
func (s *Service) ChangeDeviceBinding(ctx context.Context, principal identity.Principal, devicePublicID, requestID string, input ChangeDeviceBindingInput) (DeviceBindingChangeResult, error) {
|
||||
var result DeviceBindingChangeResult
|
||||
err := database.Transact(ctx, s.pool, func(tx pgx.Tx) error {
|
||||
access, device, err := resolveDevice(ctx, tx, principal, devicePublicID, identity.CapabilityDeviceManage, true)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
var bindingID string
|
||||
if err := tx.QueryRow(ctx, `SELECT public_id::text FROM device_binding WHERE device_id=$1::uuid AND tenant_id=$2::uuid AND unbound_at IS NULL FOR UPDATE`, access.deviceID, access.tenantID).Scan(&bindingID); errors.Is(err, pgx.ErrNoRows) {
|
||||
return ErrDeviceBindingStale
|
||||
} else if err != nil {
|
||||
return fmt.Errorf("lock active device binding: %w", err)
|
||||
}
|
||||
if bindingID != input.ExpectedBindingID {
|
||||
return ErrDeviceBindingStale
|
||||
}
|
||||
impact, err := s.readDeviceBindingImpact(ctx, tx, principal, access, device, input.Action, input.TargetSubjectID)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if !impact.Allowed {
|
||||
return ErrDeviceBindingBlocked
|
||||
}
|
||||
now := time.Now().UTC()
|
||||
if _, err := tx.Exec(ctx, `UPDATE device_binding SET unbound_at=$1,unbound_by=$2::uuid,reason=$3 WHERE public_id=$4::uuid AND unbound_at IS NULL`, now, access.userID, input.Action+":"+strings.TrimSpace(input.Reason), bindingID); err != nil {
|
||||
return fmt.Errorf("close active device binding: %w", err)
|
||||
}
|
||||
result = DeviceBindingChangeResult{DeviceID: devicePublicID, Action: input.Action, PreviousBindingID: bindingID, ChangedAt: now}
|
||||
if input.Action == DeviceBindingActionTransfer {
|
||||
targetAccess, _, err := resolveSubject(ctx, tx, principal, input.TargetSubjectID, identity.CapabilityDeviceManage, true)
|
||||
if err != nil || targetAccess.tenantID != access.tenantID {
|
||||
return ErrAccessDenied
|
||||
}
|
||||
var newBindingID string
|
||||
if err := tx.QueryRow(ctx, `
|
||||
INSERT INTO device_binding (tenant_id,device_id,household_id,subject_id,bound_by,reason,bound_at)
|
||||
VALUES ($1::uuid,$2::uuid,$3::uuid,$4::uuid,$5::uuid,$6,$7)
|
||||
RETURNING public_id::text`, access.tenantID, access.deviceID, targetAccess.householdID, targetAccess.subjectID, access.userID, "transfer:"+strings.TrimSpace(input.Reason), now).Scan(&newBindingID); err != nil {
|
||||
return fmt.Errorf("create transferred device binding: %w", err)
|
||||
}
|
||||
result.BindingID = &newBindingID
|
||||
result.HouseholdID = &targetAccess.householdPublicID
|
||||
result.SubjectID = &input.TargetSubjectID
|
||||
}
|
||||
return s.audit.Record(ctx, tx, audit.Event{
|
||||
TenantID: strPtr(access.tenantID), ActorType: "user", ActorPublicID: strPtr(access.userPublicID),
|
||||
Action: "device.binding." + input.Action, ResourceType: "device", ResourceID: strPtr(devicePublicID),
|
||||
RequestID: requestID, Result: "success", Detail: map[string]any{
|
||||
"previous_binding_id": bindingID, "new_binding_id": result.BindingID, "target_household_id": result.HouseholdID,
|
||||
"target_subject_id": result.SubjectID, "reason": strings.TrimSpace(input.Reason),
|
||||
},
|
||||
})
|
||||
})
|
||||
return result, err
|
||||
}
|
||||
|
||||
func (s *Service) readDeviceBindingImpact(ctx context.Context, q queryRow, principal identity.Principal, access deviceAccess, device ManagedDevice, action, targetSubjectPublicID string) (DeviceBindingImpact, error) {
|
||||
if action != DeviceBindingActionUnbind && action != DeviceBindingActionTransfer {
|
||||
return DeviceBindingImpact{}, ErrNotFound
|
||||
}
|
||||
impact := DeviceBindingImpact{
|
||||
DeviceID: device.ID, BindingID: device.BindingID, Action: action,
|
||||
SourceHouseholdID: device.HouseholdID, SourceSubjectID: device.SubjectID, SourceSubjectName: device.SubjectName,
|
||||
Blockers: make([]string, 0), EvaluatedAt: time.Now().UTC(),
|
||||
}
|
||||
if action == DeviceBindingActionTransfer {
|
||||
if targetSubjectPublicID == "" || targetSubjectPublicID == device.SubjectID {
|
||||
impact.Blockers = append(impact.Blockers, "target_subject_required")
|
||||
} else {
|
||||
targetAccess, target, err := resolveSubject(ctx, q, principal, targetSubjectPublicID, identity.CapabilityDeviceManage, false)
|
||||
if err != nil || targetAccess.tenantID != access.tenantID {
|
||||
return DeviceBindingImpact{}, ErrAccessDenied
|
||||
}
|
||||
impact.TargetHouseholdID = &target.HouseholdID
|
||||
impact.TargetSubjectID = &target.ID
|
||||
impact.TargetSubjectName = &target.PreferredName
|
||||
}
|
||||
}
|
||||
var activeConversation string
|
||||
err := q.QueryRow(ctx, `SELECT public_id::text FROM conversation WHERE device_id=$1::uuid AND tenant_id=$2::uuid AND status NOT IN ('COMPLETED','FAILED','EXPIRED') ORDER BY created_at DESC LIMIT 1`, access.deviceID, access.tenantID).Scan(&activeConversation)
|
||||
if err == nil {
|
||||
impact.ActiveConversationID = &activeConversation
|
||||
impact.Blockers = append(impact.Blockers, "active_conversation")
|
||||
} else if !errors.Is(err, pgx.ErrNoRows) {
|
||||
return DeviceBindingImpact{}, fmt.Errorf("check active device conversation: %w", err)
|
||||
}
|
||||
if err := q.QueryRow(ctx, `SELECT count(*) FROM device_command WHERE device_id=$1::uuid AND tenant_id=$2::uuid AND status IN ('QUEUED','DELIVERED','ACKNOWLEDGED') AND expires_at>now()`, access.deviceID, access.tenantID).Scan(&impact.PendingCommandCount); err != nil {
|
||||
return DeviceBindingImpact{}, fmt.Errorf("count pending device commands: %w", err)
|
||||
}
|
||||
if impact.PendingCommandCount > 0 {
|
||||
impact.Blockers = append(impact.Blockers, "pending_device_commands")
|
||||
}
|
||||
if err := q.QueryRow(ctx, `SELECT count(*) FROM firmware_assignment WHERE device_id=$1::uuid AND tenant_id=$2::uuid AND status IN ('PENDING','COMMAND_QUEUED','DOWNLOADING','INSTALLING')`, access.deviceID, access.tenantID).Scan(&impact.ActiveFirmwareTaskCount); err != nil {
|
||||
return DeviceBindingImpact{}, fmt.Errorf("count active firmware assignments: %w", err)
|
||||
}
|
||||
if impact.ActiveFirmwareTaskCount > 0 {
|
||||
impact.Blockers = append(impact.Blockers, "active_firmware_task")
|
||||
}
|
||||
impact.Allowed = len(impact.Blockers) == 0
|
||||
return impact, nil
|
||||
}
|
||||
@@ -0,0 +1,107 @@
|
||||
package saas
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"errors"
|
||||
"os"
|
||||
"testing"
|
||||
|
||||
"data-terminal-cloud/internal/audit"
|
||||
"data-terminal-cloud/internal/identity"
|
||||
|
||||
"github.com/google/uuid"
|
||||
"github.com/jackc/pgx/v5/pgxpool"
|
||||
)
|
||||
|
||||
func TestDeviceBindingImpactTransferAndUnbind(t *testing.T) {
|
||||
databaseURL := os.Getenv("TEST_DATABASE_URL")
|
||||
if databaseURL == "" {
|
||||
t.Skip("TEST_DATABASE_URL is not set")
|
||||
}
|
||||
ctx := context.Background()
|
||||
pool, err := pgxpool.New(ctx, databaseURL)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer pool.Close()
|
||||
|
||||
devicePublicID := uuid.NewString()
|
||||
targetSubjectPublicID := uuid.NewString()
|
||||
serialSum := sha256.Sum256([]byte(devicePublicID))
|
||||
serialDigest := hex.EncodeToString(serialSum[:])
|
||||
var deviceID, targetSubjectID string
|
||||
if err := pool.QueryRow(ctx, `
|
||||
INSERT INTO subject_profile (public_id,tenant_id,household_id,preferred_name,status,profile_version)
|
||||
SELECT $1::uuid,t.id,h.id,'绑定转移测试长者','active',1
|
||||
FROM tenant t JOIN household h ON h.tenant_id=t.id
|
||||
WHERE t.public_id=$2::uuid AND h.public_id=$3::uuid
|
||||
RETURNING id::text`, targetSubjectPublicID, testTenantPublicID, "10000000-0000-0000-0000-000000000211").Scan(&targetSubjectID); err != nil {
|
||||
t.Fatalf("insert target subject: %v", err)
|
||||
}
|
||||
if err := pool.QueryRow(ctx, `
|
||||
INSERT INTO device (public_id,tenant_id,serial_digest,model,hardware_revision,status,credential_ref,minimum_security_version)
|
||||
SELECT $1::uuid,id,$2,'binding-test-terminal','test-rev','registered','db://binding-test','test-v1'
|
||||
FROM tenant WHERE public_id=$3::uuid
|
||||
RETURNING id::text`, devicePublicID, serialDigest, testTenantPublicID).Scan(&deviceID); err != nil {
|
||||
t.Fatalf("insert device: %v", err)
|
||||
}
|
||||
if _, err := pool.Exec(ctx, `INSERT INTO device_twin (device_id,tenant_id) SELECT id,tenant_id FROM device WHERE id=$1::uuid`, deviceID); err != nil {
|
||||
t.Fatalf("insert device twin: %v", err)
|
||||
}
|
||||
var sourceBindingID string
|
||||
if err := pool.QueryRow(ctx, `
|
||||
INSERT INTO device_binding (tenant_id,device_id,household_id,subject_id,bound_by,reason)
|
||||
SELECT d.tenant_id,d.id,h.id,sp.id,ua.id,'integration-test'
|
||||
FROM device d
|
||||
JOIN household h ON h.tenant_id=d.tenant_id AND h.public_id=$2::uuid
|
||||
JOIN subject_profile sp ON sp.household_id=h.id AND sp.public_id=$3::uuid
|
||||
JOIN user_account ua ON ua.tenant_id=d.tenant_id AND ua.oidc_issuer=$4 AND ua.oidc_subject=$5
|
||||
WHERE d.id=$1::uuid
|
||||
RETURNING public_id::text`, deviceID, "10000000-0000-0000-0000-000000000211", "10000000-0000-0000-0000-000000000411", testIssuer, testOwnerSubject).Scan(&sourceBindingID); err != nil {
|
||||
t.Fatalf("insert source binding: %v", err)
|
||||
}
|
||||
defer func() {
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM audit_event WHERE request_id LIKE 'device-binding-integration-%'`)
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM device_binding WHERE device_id=$1::uuid`, deviceID)
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM device_twin WHERE device_id=$1::uuid`, deviceID)
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM device WHERE id=$1::uuid`, deviceID)
|
||||
_, _ = pool.Exec(ctx, `DELETE FROM subject_profile WHERE id=$1::uuid`, targetSubjectID)
|
||||
}()
|
||||
|
||||
service := NewService(pool, audit.NewRepository(), nil, nil)
|
||||
owner := identity.Principal{Issuer: testIssuer, Subject: testOwnerSubject}
|
||||
impact, err := service.GetDeviceBindingImpact(ctx, owner, devicePublicID, DeviceBindingActionTransfer, targetSubjectPublicID)
|
||||
if err != nil || !impact.Allowed || impact.BindingID != sourceBindingID || impact.TargetSubjectID == nil || *impact.TargetSubjectID != targetSubjectPublicID {
|
||||
t.Fatalf("unexpected transfer impact: %+v err=%v", impact, err)
|
||||
}
|
||||
if _, err := service.ChangeDeviceBinding(ctx, owner, devicePublicID, "device-binding-integration-stale", ChangeDeviceBindingInput{
|
||||
Action: DeviceBindingActionTransfer, ExpectedBindingID: uuid.NewString(), TargetSubjectID: targetSubjectPublicID, Reason: "stale review", Acknowledged: true,
|
||||
}); !errors.Is(err, ErrDeviceBindingStale) {
|
||||
t.Fatalf("stale binding error=%v", err)
|
||||
}
|
||||
transferred, err := service.ChangeDeviceBinding(ctx, owner, devicePublicID, "device-binding-integration-transfer", ChangeDeviceBindingInput{
|
||||
Action: DeviceBindingActionTransfer, ExpectedBindingID: impact.BindingID, TargetSubjectID: targetSubjectPublicID, Reason: "change assigned elder", Acknowledged: true,
|
||||
})
|
||||
if err != nil || transferred.BindingID == nil || transferred.SubjectID == nil || *transferred.SubjectID != targetSubjectPublicID {
|
||||
t.Fatalf("transfer result=%+v err=%v", transferred, err)
|
||||
}
|
||||
device, err := service.GetDevice(ctx, owner, devicePublicID)
|
||||
if err != nil || device.BindingID != *transferred.BindingID || device.SubjectID != targetSubjectPublicID {
|
||||
t.Fatalf("transferred device=%+v err=%v", device, err)
|
||||
}
|
||||
unbindImpact, err := service.GetDeviceBindingImpact(ctx, owner, devicePublicID, DeviceBindingActionUnbind, "")
|
||||
if err != nil || !unbindImpact.Allowed {
|
||||
t.Fatalf("unbind impact=%+v err=%v", unbindImpact, err)
|
||||
}
|
||||
unbound, err := service.ChangeDeviceBinding(ctx, owner, devicePublicID, "device-binding-integration-unbind", ChangeDeviceBindingInput{
|
||||
Action: DeviceBindingActionUnbind, ExpectedBindingID: unbindImpact.BindingID, Reason: "device removed from service", Acknowledged: true,
|
||||
})
|
||||
if err != nil || unbound.BindingID != nil {
|
||||
t.Fatalf("unbind result=%+v err=%v", unbound, err)
|
||||
}
|
||||
if _, err := service.GetDevice(ctx, owner, devicePublicID); !errors.Is(err, ErrAccessDenied) {
|
||||
t.Fatalf("unbound device should no longer be visible, error=%v", err)
|
||||
}
|
||||
}
|
||||
@@ -18,6 +18,7 @@ import (
|
||||
|
||||
type ManagedDevice struct {
|
||||
ID string `json:"id"`
|
||||
BindingID string `json:"binding_id"`
|
||||
HouseholdID string `json:"household_id"`
|
||||
SubjectID string `json:"subject_id"`
|
||||
SubjectName string `json:"subject_name"`
|
||||
@@ -66,7 +67,7 @@ type DevicePresence struct {
|
||||
|
||||
func (s *Service) ListDevices(ctx context.Context, principal identity.Principal) ([]ManagedDevice, error) {
|
||||
rows, err := s.pool.Query(ctx, `
|
||||
SELECT d.public_id::text,h.public_id::text,sp.public_id::text,sp.preferred_name,
|
||||
SELECT d.public_id::text,b.public_id::text,h.public_id::text,sp.public_id::text,sp.preferred_name,
|
||||
d.model,d.hardware_revision,d.status,d.minimum_security_version,d.minimum_firmware_version,dt.firmware_version,
|
||||
dt.desired_revision,dt.reported_revision,dt.last_reported_at,b.bound_at,d.registered_at,d.version,hm.role
|
||||
FROM user_account ua
|
||||
@@ -86,7 +87,7 @@ func (s *Service) ListDevices(ctx context.Context, principal identity.Principal)
|
||||
for rows.Next() {
|
||||
var item ManagedDevice
|
||||
var role identity.Role
|
||||
if err := rows.Scan(&item.ID, &item.HouseholdID, &item.SubjectID, &item.SubjectName,
|
||||
if err := rows.Scan(&item.ID, &item.BindingID, &item.HouseholdID, &item.SubjectID, &item.SubjectName,
|
||||
&item.Model, &item.HardwareRevision, &item.Status, &item.MinimumSecurityVersion, &item.MinimumFirmwareVersion, &item.FirmwareVersion,
|
||||
&item.DesiredRevision, &item.ReportedRevision, &item.LastReportedAt, &item.BoundAt, &item.RegisteredAt, &item.Version, &role); err != nil {
|
||||
return nil, fmt.Errorf("scan managed device: %w", err)
|
||||
@@ -189,7 +190,7 @@ type deviceAccess struct {
|
||||
func resolveDevice(ctx context.Context, q queryRow, principal identity.Principal, publicID string, capability identity.Capability, lock bool) (deviceAccess, ManagedDevice, error) {
|
||||
sql := `
|
||||
SELECT t.id::text,t.public_id::text,ua.id::text,ua.public_id::text,h.id::text,h.public_id::text,d.id::text,hm.role,
|
||||
d.public_id::text,h.public_id::text,sp.public_id::text,sp.preferred_name,d.model,d.hardware_revision,
|
||||
d.public_id::text,b.public_id::text,h.public_id::text,sp.public_id::text,sp.preferred_name,d.model,d.hardware_revision,
|
||||
d.status,d.minimum_security_version,d.minimum_firmware_version,dt.firmware_version,dt.desired_revision,dt.reported_revision,
|
||||
dt.last_reported_at,b.bound_at,d.registered_at,d.version
|
||||
FROM device d
|
||||
@@ -209,7 +210,7 @@ func resolveDevice(ctx context.Context, q queryRow, principal identity.Principal
|
||||
var role string
|
||||
err := q.QueryRow(ctx, sql, publicID, principal.Issuer, principal.Subject).Scan(
|
||||
&access.tenantID, &access.tenantPublicID, &access.userID, &access.userPublicID, &access.householdID, &access.householdPublicID, &access.deviceID, &role,
|
||||
&device.ID, &device.HouseholdID, &device.SubjectID, &device.SubjectName, &device.Model, &device.HardwareRevision,
|
||||
&device.ID, &device.BindingID, &device.HouseholdID, &device.SubjectID, &device.SubjectName, &device.Model, &device.HardwareRevision,
|
||||
&device.Status, &device.MinimumSecurityVersion, &device.MinimumFirmwareVersion, &device.FirmwareVersion, &device.DesiredRevision, &device.ReportedRevision,
|
||||
&device.LastReportedAt, &device.BoundAt, &device.RegisteredAt, &device.Version)
|
||||
if errors.Is(err, pgx.ErrNoRows) {
|
||||
|
||||
@@ -62,7 +62,7 @@ func (s *Service) CreatePrivacyRequest(ctx context.Context, principal identity.P
|
||||
if input.Scope == nil {
|
||||
input.Scope = map[string]any{}
|
||||
}
|
||||
if input.RequestType != "ACCESS" && input.RequestType != "DELETE" {
|
||||
if input.RequestType != "ACCESS" && input.RequestType != "CORRECTION" && input.RequestType != "CONSENT_WITHDRAWAL" && input.RequestType != "DELETE" {
|
||||
return PrivacyRequest{}, false, ErrNotFound
|
||||
}
|
||||
if input.RequestType == "DELETE" {
|
||||
@@ -152,11 +152,115 @@ func (s *Service) CreatePrivacyRequest(ctx context.Context, principal identity.P
|
||||
created = true
|
||||
eventType := "privacy.access_requested"
|
||||
action := "privacy.access.request"
|
||||
if input.RequestType == "ACCESS" {
|
||||
enqueue := true
|
||||
auditDetail := map[string]any{"subject_id": subjectPublicID, "categories": input.Scope["categories"]}
|
||||
switch input.RequestType {
|
||||
case "ACCESS":
|
||||
if _, err := tx.Exec(ctx, `INSERT INTO privacy_subtask(privacy_request_id,subsystem,target_type,target_ref) VALUES($1::uuid,'postgresql','subject_data_inventory',$2)`, internalID, subjectPublicID); err != nil {
|
||||
return fmt.Errorf("create privacy subtask: %w", err)
|
||||
}
|
||||
} else {
|
||||
case "CORRECTION":
|
||||
corrections, ok := input.Scope["corrections"].(map[string]any)
|
||||
if !ok || len(corrections) == 0 {
|
||||
return ErrNotFound
|
||||
}
|
||||
preferredName, language, timezone := subject.PreferredName, subject.Language, subject.Timezone
|
||||
profile, preferences := subject.ProfileData, subject.ConversationPreferences
|
||||
changedFields := make([]string, 0, len(corrections))
|
||||
if value, ok := corrections["preferred_name"].(string); ok {
|
||||
preferredName = value
|
||||
changedFields = append(changedFields, "preferred_name")
|
||||
}
|
||||
if value, ok := corrections["language"].(string); ok {
|
||||
language = value
|
||||
changedFields = append(changedFields, "language")
|
||||
}
|
||||
if value, ok := corrections["timezone"].(string); ok {
|
||||
timezone = value
|
||||
changedFields = append(changedFields, "timezone")
|
||||
}
|
||||
if value, ok := corrections["profile_data"].(map[string]any); ok {
|
||||
profile = value
|
||||
changedFields = append(changedFields, "profile_data")
|
||||
}
|
||||
if value, ok := corrections["conversation_preferences"].(map[string]any); ok {
|
||||
preferences = value
|
||||
changedFields = append(changedFields, "conversation_preferences")
|
||||
}
|
||||
if len(changedFields) == 0 {
|
||||
return ErrNotFound
|
||||
}
|
||||
profileJSON, _ := json.Marshal(profile)
|
||||
preferencesJSON, _ := json.Marshal(preferences)
|
||||
if _, err := tx.Exec(ctx, `UPDATE subject_profile SET preferred_name=$1,language=$2,timezone=$3,profile_data=$4::jsonb,conversation_preferences=$5::jsonb,profile_version=profile_version+1,updated_by=$6::uuid,updated_at=now() WHERE id=$7::uuid AND tenant_id=$8::uuid`, preferredName, language, timezone, profileJSON, preferencesJSON, access.userID, access.subjectID, access.tenantID); err != nil {
|
||||
return fmt.Errorf("apply privacy correction: %w", err)
|
||||
}
|
||||
if _, err := tx.Exec(ctx, `INSERT INTO privacy_subtask(privacy_request_id,subsystem,target_type,target_ref,status,attempts,completed_at) VALUES($1::uuid,'postgresql','subject_profile_correction',$2,'COMPLETED',1,now())`, internalID, subjectPublicID); err != nil {
|
||||
return fmt.Errorf("complete correction subtask: %w", err)
|
||||
}
|
||||
resultJSON, _ := json.Marshal(map[string]any{"content_included": false, "corrected_fields": changedFields, "profile_version": subject.Version + 1})
|
||||
if _, err := tx.Exec(ctx, `UPDATE privacy_request SET status='COMPLETED',result_summary=$2::jsonb,completed_at=now(),version=version+1,updated_at=now() WHERE id=$1::uuid`, internalID, resultJSON); err != nil {
|
||||
return fmt.Errorf("complete correction request: %w", err)
|
||||
}
|
||||
enqueue = false
|
||||
action = "privacy.correction.request"
|
||||
auditDetail = map[string]any{"subject_id": subjectPublicID, "corrected_fields": changedFields}
|
||||
case "CONSENT_WITHDRAWAL":
|
||||
requestedTypes := privacyScopeStrings(input.Scope, "consent_types")
|
||||
if len(requestedTypes) == 0 {
|
||||
return ErrNotFound
|
||||
}
|
||||
rows, err := tx.Query(ctx, `SELECT consent_type FROM consent_record WHERE tenant_id=$1::uuid AND subject_id=$2::uuid AND consent_type=ANY($3::text[]) AND withdrawn_at IS NULL AND granted_at<=now() AND (expires_at IS NULL OR expires_at>now()) FOR UPDATE`, access.tenantID, access.subjectID, requestedTypes)
|
||||
if err != nil {
|
||||
return fmt.Errorf("lock consents for privacy withdrawal: %w", err)
|
||||
}
|
||||
withdrawnTypes := make([]string, 0, len(requestedTypes))
|
||||
for rows.Next() {
|
||||
var consentType string
|
||||
if err := rows.Scan(&consentType); err != nil {
|
||||
rows.Close()
|
||||
return err
|
||||
}
|
||||
withdrawnTypes = append(withdrawnTypes, consentType)
|
||||
}
|
||||
if err := rows.Err(); err != nil {
|
||||
rows.Close()
|
||||
return err
|
||||
}
|
||||
rows.Close()
|
||||
if _, err := tx.Exec(ctx, `UPDATE consent_record SET withdrawn_at=now(),version=version+1,updated_at=now() WHERE tenant_id=$1::uuid AND subject_id=$2::uuid AND consent_type=ANY($3::text[]) AND withdrawn_at IS NULL AND granted_at<=now() AND (expires_at IS NULL OR expires_at>now())`, access.tenantID, access.subjectID, requestedTypes); err != nil {
|
||||
return fmt.Errorf("withdraw privacy request consents: %w", err)
|
||||
}
|
||||
if err := createConsentSnapshot(ctx, tx, access.tenantID, access.subjectID); err != nil {
|
||||
return err
|
||||
}
|
||||
if containsString(withdrawnTypes, ConsentPersonalMemory) {
|
||||
if _, err := tx.Exec(ctx, `UPDATE curated_memory SET status='DISABLED',version=version+1,updated_at=now() WHERE tenant_id=$1::uuid AND subject_id=$2::uuid AND status='ACTIVE'`, access.tenantID, access.subjectID); err != nil {
|
||||
return err
|
||||
}
|
||||
if _, err := tx.Exec(ctx, `UPDATE memory_index_entry SET embedding=NULL,status='STALE',last_error='personalized memory consent withdrawn',updated_at=now() WHERE tenant_id=$1::uuid AND subject_id=$2::uuid AND status IN ('PENDING','ACTIVE')`, access.tenantID, access.subjectID); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
stopped := make([]string, 0)
|
||||
for _, consentType := range withdrawnTypes {
|
||||
items, err := s.stopConversationsAfterConsentWithdrawal(ctx, tx, access.tenantID, access.subjectID, consentType)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
stopped = append(stopped, items...)
|
||||
}
|
||||
if _, err := tx.Exec(ctx, `INSERT INTO privacy_subtask(privacy_request_id,subsystem,target_type,target_ref,status,attempts,completed_at) VALUES($1::uuid,'postgresql','consent_withdrawal',$2,'COMPLETED',1,now())`, internalID, subjectPublicID); err != nil {
|
||||
return fmt.Errorf("complete consent withdrawal subtask: %w", err)
|
||||
}
|
||||
resultJSON, _ := json.Marshal(map[string]any{"content_included": false, "requested_consent_types": requestedTypes, "withdrawn_consent_types": withdrawnTypes, "stopped_conversation_count": len(stopped)})
|
||||
if _, err := tx.Exec(ctx, `UPDATE privacy_request SET status='COMPLETED',result_summary=$2::jsonb,completed_at=now(),version=version+1,updated_at=now() WHERE id=$1::uuid`, internalID, resultJSON); err != nil {
|
||||
return fmt.Errorf("complete consent withdrawal request: %w", err)
|
||||
}
|
||||
enqueue = false
|
||||
action = "privacy.consent_withdrawal.request"
|
||||
auditDetail = map[string]any{"subject_id": subjectPublicID, "requested_consent_types": requestedTypes, "withdrawn_consent_types": withdrawnTypes, "stopped_conversation_ids": stopped}
|
||||
case "DELETE":
|
||||
tasks := [][2]string{{"redis", "ephemeral_keys"}, {"export_files", "private_exports"}, {"object_storage", "subject_media"}, {"pgvector", "memory_embeddings"}, {"provider_index", "external_memory_index"}, {"postgresql", "subject_personal_data"}}
|
||||
for _, task := range tasks {
|
||||
if _, err := tx.Exec(ctx, `INSERT INTO privacy_subtask(privacy_request_id,subsystem,target_type,target_ref) VALUES($1::uuid,$2,$3,$4)`, internalID, task[0], task[1], subjectPublicID); err != nil {
|
||||
@@ -178,12 +282,14 @@ func (s *Service) CreatePrivacyRequest(ctx context.Context, principal identity.P
|
||||
eventType = "privacy.delete_requested"
|
||||
action = "privacy.delete.request"
|
||||
}
|
||||
dedupe := strings.ReplaceAll(eventType, ".", "-") + ":" + internalID
|
||||
payload, _ := json.Marshal(map[string]any{"request_id": internalID})
|
||||
if _, err := outbox.NewStore(s.pool).Enqueue(ctx, tx, outbox.NewEvent{TenantID: &access.tenantID, AggregateType: "privacy_request", AggregateID: internalID, EventType: eventType, DeduplicationKey: &dedupe, Payload: payload}); err != nil {
|
||||
return err
|
||||
if enqueue {
|
||||
dedupe := strings.ReplaceAll(eventType, ".", "-") + ":" + internalID
|
||||
payload, _ := json.Marshal(map[string]any{"request_id": internalID})
|
||||
if _, err := outbox.NewStore(s.pool).Enqueue(ctx, tx, outbox.NewEvent{TenantID: &access.tenantID, AggregateType: "privacy_request", AggregateID: internalID, EventType: eventType, DeduplicationKey: &dedupe, Payload: payload}); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return s.audit.Record(ctx, tx, audit.Event{TenantID: strPtr(access.tenantID), ActorType: "user", ActorPublicID: strPtr(access.userPublicID), Action: action, ResourceType: "privacy_request", ResourceID: strPtr(publicID), RequestID: requestID, Result: "success", Detail: map[string]any{"subject_id": subjectPublicID, "scope": input.Scope}})
|
||||
return s.audit.Record(ctx, tx, audit.Event{TenantID: strPtr(access.tenantID), ActorType: "user", ActorPublicID: strPtr(access.userPublicID), Action: action, ResourceType: "privacy_request", ResourceID: strPtr(publicID), RequestID: requestID, Result: "success", Detail: auditDetail})
|
||||
})
|
||||
if err != nil {
|
||||
return PrivacyRequest{}, false, err
|
||||
@@ -192,6 +298,36 @@ func (s *Service) CreatePrivacyRequest(ctx context.Context, principal identity.P
|
||||
return result, created, err
|
||||
}
|
||||
|
||||
func privacyScopeStrings(scope map[string]any, key string) []string {
|
||||
value, ok := scope[key]
|
||||
if !ok {
|
||||
return nil
|
||||
}
|
||||
switch values := value.(type) {
|
||||
case []string:
|
||||
return values
|
||||
case []any:
|
||||
result := make([]string, 0, len(values))
|
||||
for _, item := range values {
|
||||
if text, ok := item.(string); ok {
|
||||
result = append(result, text)
|
||||
}
|
||||
}
|
||||
return result
|
||||
default:
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
func containsString(values []string, target string) bool {
|
||||
for _, value := range values {
|
||||
if value == target {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func (s *Service) ListPrivacyRequests(ctx context.Context, principal identity.Principal, subjectPublicID string) ([]PrivacyRequest, error) {
|
||||
access, _, err := resolveSubject(ctx, s.pool, principal, subjectPublicID, identity.CapabilityPrivacyRead, false)
|
||||
if err != nil {
|
||||
|
||||
@@ -28,6 +28,8 @@ var (
|
||||
ErrDeviceClaimInvalid = errors.New("device claim is invalid")
|
||||
ErrDeviceClaimUsed = errors.New("device claim was used for another binding")
|
||||
ErrDeviceAlreadyBound = errors.New("device already has an active binding")
|
||||
ErrDeviceBindingBlocked = errors.New("device binding change has active blockers")
|
||||
ErrDeviceBindingStale = errors.New("device binding changed since impact review")
|
||||
ErrSubjectDeleteConflict = errors.New("subject cannot be deleted while a conversation is active")
|
||||
ErrDeleteConfirmation = errors.New("subject deletion confirmation does not match")
|
||||
)
|
||||
@@ -126,6 +128,11 @@ type Service struct {
|
||||
exportDownloadTTL time.Duration
|
||||
firmwareStore firmwareObjectStore
|
||||
firmwareURLTTL time.Duration
|
||||
providerMemory bool
|
||||
}
|
||||
|
||||
func (s *Service) ConfigureProviderMemory(enabled bool) {
|
||||
s.providerMemory = enabled
|
||||
}
|
||||
|
||||
func NewService(pool *pgxpool.Pool, auditRepository *audit.Repository, redisClient *redis.Client, cursorCodec *pagination.Codec) *Service {
|
||||
@@ -506,6 +513,32 @@ func (s *Service) WithdrawConsent(ctx context.Context, principal identity.Princi
|
||||
WHERE tenant_id=$1::uuid AND subject_id=$2::uuid AND status IN ('PENDING','ACTIVE')`, access.tenantID, access.subjectID); err != nil {
|
||||
return fmt.Errorf("invalidate memory index after consent withdrawal: %w", err)
|
||||
}
|
||||
if s.providerMemory {
|
||||
rows, err := tx.Query(ctx, `SELECT id::text FROM curated_memory WHERE tenant_id=$1::uuid AND subject_id=$2::uuid AND status<>'DELETED' ORDER BY id`, access.tenantID, access.subjectID)
|
||||
if err != nil {
|
||||
return fmt.Errorf("list provider memories after consent withdrawal: %w", err)
|
||||
}
|
||||
memoryIDs := make([]string, 0)
|
||||
for rows.Next() {
|
||||
var memoryID string
|
||||
if err := rows.Scan(&memoryID); err != nil {
|
||||
rows.Close()
|
||||
return err
|
||||
}
|
||||
memoryIDs = append(memoryIDs, memoryID)
|
||||
}
|
||||
rows.Close()
|
||||
if err := rows.Err(); err != nil {
|
||||
return err
|
||||
}
|
||||
for _, memoryID := range memoryIDs {
|
||||
dedupe := "memory-provider-delete-consent:" + memoryID + ":" + result.ID
|
||||
payload, _ := json.Marshal(map[string]string{"reason": "personalized memory consent withdrawn", "consent_id": result.ID})
|
||||
if _, err := outbox.NewStore(s.pool).Enqueue(ctx, tx, outbox.NewEvent{TenantID: &access.tenantID, AggregateType: "curated_memory", AggregateID: memoryID, EventType: "memory.provider_delete_requested", DeduplicationKey: &dedupe, Payload: payload}); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
affectedConversations, err := s.stopConversationsAfterConsentWithdrawal(ctx, tx, access.tenantID, access.subjectID, result.Type)
|
||||
if err != nil {
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user