feat: add Windows real RTC voice device simulator
This commit is contained in:
@@ -12,6 +12,7 @@ coverage*.out
|
|||||||
*.exe
|
*.exe
|
||||||
/opsconfig
|
/opsconfig
|
||||||
/deploy/server/.env
|
/deploy/server/.env
|
||||||
|
/deploy/local-rtc/runtime/
|
||||||
bin/
|
bin/
|
||||||
dist/
|
dist/
|
||||||
tmp/
|
tmp/
|
||||||
|
|||||||
@@ -185,6 +185,15 @@ docker compose --profile simulation run --rm sim-device
|
|||||||
docker compose --profile simulation run --rm -e SIM_SCENARIO=offline,conversation,transcript,telemetry,commands sim-device
|
docker compose --profile simulation run --rm -e SIM_SCENARIO=offline,conversation,transcript,telemetry,commands sim-device
|
||||||
```
|
```
|
||||||
|
|
||||||
|
在Windows本机使用真实麦克风、扬声器和火山RTC/豆包端到端语音进行联调:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
.\deploy\local-rtc\configure.ps1
|
||||||
|
.\deploy\local-rtc\start.ps1
|
||||||
|
```
|
||||||
|
|
||||||
|
然后运行`device-simulator-windows/release/voice-device-simulator-0.1.0-windows-x64.exe`。密钥配置、纯端到端首轮联调范围和操作步骤见[Windows本地真实RTC联调说明](deploy/local-rtc/README.md)。
|
||||||
|
|
||||||
还可用`SIM_SCENARIO=media`和`SIM_CONVERSATION_ID`验证已授权通话的录音直传、后台校验、延迟删除及设备删除回执;详见`docs/device-integration.md`。
|
还可用`SIM_SCENARIO=media`和`SIM_CONVERSATION_ID`验证已授权通话的录音直传、后台校验、延迟删除及设备删除回执;详见`docs/device-integration.md`。
|
||||||
|
|
||||||
执行设备状态、错误和命令结果的批量遥测、幂等、事件判重及冲突回滚烟测:
|
执行设备状态、错误和命令结果的批量遥测、幂等、事件判重及冲突回滚烟测:
|
||||||
@@ -326,6 +335,7 @@ internal/ 平台公共能力及后续领域模块
|
|||||||
migrations/ 仅追加的数据库迁移
|
migrations/ 仅追加的数据库迁移
|
||||||
openapi/ 对外接口契约基线
|
openapi/ 对外接口契约基线
|
||||||
web/ 家庭用户和运营网页
|
web/ 家庭用户和运营网页
|
||||||
|
device-simulator-windows/ Windows真实音频终端模拟器
|
||||||
deploy/ 容器构建与运行配置
|
deploy/ 容器构建与运行配置
|
||||||
docs/ 架构决策与研发基线
|
docs/ 架构决策与研发基线
|
||||||
scripts/ 本地启动和质量检查脚本
|
scripts/ 本地启动和质量检查脚本
|
||||||
|
|||||||
@@ -0,0 +1,41 @@
|
|||||||
|
# Windows 本地真实 RTC 联调
|
||||||
|
|
||||||
|
该覆盖层只用于本机联调,不访问阿里云服务器。它把 API 的 RTC Token 签发器和 Worker 的语音供应商切换到火山引擎,同时保留数据库、Redis、MinIO、Keycloak和告警模拟器在本地 Docker 中运行。
|
||||||
|
|
||||||
|
## 1. 生成本地凭据
|
||||||
|
|
||||||
|
在项目根目录运行:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
.\deploy\local-rtc\configure.ps1
|
||||||
|
```
|
||||||
|
|
||||||
|
脚本会以隐藏输入方式依次读取:
|
||||||
|
|
||||||
|
- RTC AppKey;
|
||||||
|
- 火山引擎 AccessKey ID / Secret AccessKey;
|
||||||
|
- 豆包端到端语音 APP ID / Access Token;
|
||||||
|
- 豆包音色 ID。
|
||||||
|
|
||||||
|
密钥文件只生成到`deploy/local-rtc/runtime`,该目录不会提交到 Git。当前首轮联调使用豆包 S2S-O 纯端到端模式(`OutputMode=0`),平台本地路由仍沿用`s2s_hybrid`模板槽位;这是为了只依赖豆包端到端语音凭据,先验证真实双向音频。后续接入火山方舟 Endpoint 后再切换真正的混合编排模式。
|
||||||
|
|
||||||
|
## 2. 启动本地后端
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
.\deploy\local-rtc\start.ps1
|
||||||
|
```
|
||||||
|
|
||||||
|
首次构建需要下载 Docker 基础镜像。脚本会等待`http://localhost:58080/health/ready`可用。
|
||||||
|
|
||||||
|
## 3. 启动模拟器
|
||||||
|
|
||||||
|
双击`device-simulator-windows/release/voice-device-simulator-0.1.0-windows-x64.exe`,然后:
|
||||||
|
|
||||||
|
1. 保持默认本地平台地址、设备 ID 和本地测试设备密钥;
|
||||||
|
2. 点击“检测平台和音频设备”;
|
||||||
|
3. 选择麦克风和扬声器;
|
||||||
|
4. 点击“开始语音聊天”;
|
||||||
|
5. 看到“豆包语音任务已激活”后直接说话;
|
||||||
|
6. 点击“结束通话”验证`StopVoiceChat`和RTC离房。
|
||||||
|
|
||||||
|
RTC AppKey、火山 AK/SK 和豆包 Access Token 均不进入模拟器安装包,也不会通过渲染页面读取。
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
services:
|
||||||
|
api:
|
||||||
|
environment:
|
||||||
|
RTC_APP_ID: ${RTC_APP_ID}
|
||||||
|
RTC_TOKEN_ISSUER: volcengine
|
||||||
|
RTC_APP_KEY: ""
|
||||||
|
RTC_APP_KEY_FILE: /run/secrets/local-rtc/rtc-app-key
|
||||||
|
volumes:
|
||||||
|
- ./deploy/local-rtc/runtime/rtc-app-key:/run/secrets/local-rtc/rtc-app-key:ro
|
||||||
|
|
||||||
|
worker:
|
||||||
|
environment:
|
||||||
|
VOICE_PROVIDER: volcengine
|
||||||
|
VOLC_ACCESS_KEY_ID: ""
|
||||||
|
VOLC_ACCESS_KEY_ID_FILE: /run/secrets/local-rtc/volc-access-key-id
|
||||||
|
VOLC_SECRET_ACCESS_KEY: ""
|
||||||
|
VOLC_SECRET_ACCESS_KEY_FILE: /run/secrets/local-rtc/volc-secret-access-key
|
||||||
|
VOLC_RTC_REGION: cn-north-1
|
||||||
|
VOLC_RTC_API_VERSION: 2024-12-01
|
||||||
|
VOLC_VOICE_CHAT_TEMPLATE_FILE: /run/secrets/local-rtc/voice-chat-template.json
|
||||||
|
volumes:
|
||||||
|
- ./deploy/local-rtc/runtime/volc-access-key-id:/run/secrets/local-rtc/volc-access-key-id:ro
|
||||||
|
- ./deploy/local-rtc/runtime/volc-secret-access-key:/run/secrets/local-rtc/volc-secret-access-key:ro
|
||||||
|
- ./deploy/local-rtc/runtime/voice-chat-template.json:/run/secrets/local-rtc/voice-chat-template.json:ro
|
||||||
@@ -0,0 +1,118 @@
|
|||||||
|
[CmdletBinding()]
|
||||||
|
param(
|
||||||
|
[string]$RtcAppId = "6a8f92ca43726c017d34cce1",
|
||||||
|
[switch]$Force
|
||||||
|
)
|
||||||
|
|
||||||
|
$ErrorActionPreference = "Stop"
|
||||||
|
$scriptDirectory = Split-Path -Parent $MyInvocation.MyCommand.Path
|
||||||
|
$runtimeDirectory = Join-Path $scriptDirectory "runtime"
|
||||||
|
$utf8NoBom = [System.Text.UTF8Encoding]::new($false)
|
||||||
|
|
||||||
|
function Read-RequiredText {
|
||||||
|
param([string]$Prompt, [string]$Default = "")
|
||||||
|
$suffix = if ($Default) { " [$Default]" } else { "" }
|
||||||
|
$value = Read-Host "$Prompt$suffix"
|
||||||
|
if ([string]::IsNullOrWhiteSpace($value)) { $value = $Default }
|
||||||
|
if ([string]::IsNullOrWhiteSpace($value)) { throw "$Prompt 不能为空" }
|
||||||
|
return $value.Trim()
|
||||||
|
}
|
||||||
|
|
||||||
|
function Read-RequiredSecret {
|
||||||
|
param([string]$Prompt, [int]$MinimumLength = 1)
|
||||||
|
$secureValue = Read-Host $Prompt -AsSecureString
|
||||||
|
$pointer = [Runtime.InteropServices.Marshal]::SecureStringToBSTR($secureValue)
|
||||||
|
try {
|
||||||
|
$value = [Runtime.InteropServices.Marshal]::PtrToStringBSTR($pointer)
|
||||||
|
}
|
||||||
|
finally {
|
||||||
|
[Runtime.InteropServices.Marshal]::ZeroFreeBSTR($pointer)
|
||||||
|
}
|
||||||
|
if ($value.Length -lt $MinimumLength) { throw "$Prompt 长度不足" }
|
||||||
|
return $value
|
||||||
|
}
|
||||||
|
|
||||||
|
function Write-SecretFile {
|
||||||
|
param([string]$Name, [string]$Value)
|
||||||
|
$target = Join-Path $runtimeDirectory $Name
|
||||||
|
[IO.File]::WriteAllText($target, $Value, $utf8NoBom)
|
||||||
|
}
|
||||||
|
|
||||||
|
if ($RtcAppId.Length -ne 24) {
|
||||||
|
throw "RTC AppId 必须正好为 24 个字符"
|
||||||
|
}
|
||||||
|
|
||||||
|
if ((Test-Path -LiteralPath $runtimeDirectory) -and -not $Force) {
|
||||||
|
throw "本地凭据目录已经存在。确认需要重新配置后使用 -Force。"
|
||||||
|
}
|
||||||
|
|
||||||
|
New-Item -ItemType Directory -Path $runtimeDirectory -Force | Out-Null
|
||||||
|
|
||||||
|
Write-Host "`n配置只保存在本机 deploy/local-rtc/runtime,已被 Git 忽略。" -ForegroundColor Cyan
|
||||||
|
Write-Host "输入密钥时屏幕不会显示字符。`n" -ForegroundColor DarkGray
|
||||||
|
|
||||||
|
$rtcAppKey = Read-RequiredSecret "RTC AppKey" 16
|
||||||
|
$volcAccessKeyId = Read-RequiredText "火山引擎 AccessKey ID"
|
||||||
|
$volcSecretAccessKey = Read-RequiredSecret "火山引擎 Secret AccessKey"
|
||||||
|
$doubaoAppId = Read-RequiredText "豆包端到端语音 APP ID"
|
||||||
|
$doubaoAccessToken = Read-RequiredSecret "豆包端到端语音 Access Token"
|
||||||
|
$speaker = Read-RequiredText "豆包音色 ID" "zh_female_vv_jupiter_bigtts"
|
||||||
|
|
||||||
|
$template = [ordered]@{
|
||||||
|
schema_version = "voice-chat-template-bundle-v1"
|
||||||
|
templates = [ordered]@{
|
||||||
|
s2s_hybrid = [ordered]@{
|
||||||
|
AgentConfig = [ordered]@{
|
||||||
|
TargetUserId = @()
|
||||||
|
UserId = ""
|
||||||
|
WelcomeMessage = "您好,我是豆包语音助手。今天想聊些什么?"
|
||||||
|
EnableConversationStateCallback = $false
|
||||||
|
}
|
||||||
|
Config = [ordered]@{
|
||||||
|
InterruptMode = 0
|
||||||
|
S2SConfig = [ordered]@{
|
||||||
|
Provider = "volcano"
|
||||||
|
OutputMode = 0
|
||||||
|
ProviderParams = [ordered]@{
|
||||||
|
app = [ordered]@{
|
||||||
|
appid = $doubaoAppId
|
||||||
|
token = $doubaoAccessToken
|
||||||
|
}
|
||||||
|
dialog = [ordered]@{
|
||||||
|
extra = [ordered]@{
|
||||||
|
model = "O"
|
||||||
|
strict_audit = $true
|
||||||
|
audit_response = "抱歉,这个话题我暂时无法回答。我们换个轻松的话题吧。"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
asr = [ordered]@{
|
||||||
|
extra = [ordered]@{
|
||||||
|
end_smooth_window_ms = 1200
|
||||||
|
}
|
||||||
|
}
|
||||||
|
tts = [ordered]@{
|
||||||
|
speaker = $speaker
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
Write-SecretFile "rtc-app-key" $rtcAppKey
|
||||||
|
Write-SecretFile "volc-access-key-id" $volcAccessKeyId
|
||||||
|
Write-SecretFile "volc-secret-access-key" $volcSecretAccessKey
|
||||||
|
[IO.File]::WriteAllText(
|
||||||
|
(Join-Path $runtimeDirectory "voice-chat-template.json"),
|
||||||
|
($template | ConvertTo-Json -Depth 20),
|
||||||
|
$utf8NoBom
|
||||||
|
)
|
||||||
|
[IO.File]::WriteAllText(
|
||||||
|
(Join-Path $runtimeDirectory "local-rtc.env"),
|
||||||
|
"RTC_APP_ID=$RtcAppId`n",
|
||||||
|
$utf8NoBom
|
||||||
|
)
|
||||||
|
|
||||||
|
Write-Host "`n本地真实 RTC 配置已生成。" -ForegroundColor Green
|
||||||
|
Write-Host "下一步运行: .\deploy\local-rtc\start.ps1" -ForegroundColor Green
|
||||||
@@ -0,0 +1,57 @@
|
|||||||
|
[CmdletBinding()]
|
||||||
|
param([switch]$NoBuild)
|
||||||
|
|
||||||
|
$ErrorActionPreference = "Stop"
|
||||||
|
$scriptDirectory = Split-Path -Parent $MyInvocation.MyCommand.Path
|
||||||
|
$projectRoot = Resolve-Path (Join-Path $scriptDirectory "..\..")
|
||||||
|
$runtimeDirectory = Join-Path $scriptDirectory "runtime"
|
||||||
|
$environmentFile = Join-Path $runtimeDirectory "local-rtc.env"
|
||||||
|
$requiredFiles = @(
|
||||||
|
"rtc-app-key",
|
||||||
|
"volc-access-key-id",
|
||||||
|
"volc-secret-access-key",
|
||||||
|
"voice-chat-template.json",
|
||||||
|
"local-rtc.env"
|
||||||
|
)
|
||||||
|
|
||||||
|
foreach ($name in $requiredFiles) {
|
||||||
|
if (-not (Test-Path -LiteralPath (Join-Path $runtimeDirectory $name) -PathType Leaf)) {
|
||||||
|
throw "缺少 $name,请先运行 .\deploy\local-rtc\configure.ps1"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
Push-Location $projectRoot
|
||||||
|
try {
|
||||||
|
$arguments = @(
|
||||||
|
"compose",
|
||||||
|
"-f", "compose.yaml",
|
||||||
|
"-f", "deploy/local-rtc/compose.override.yaml",
|
||||||
|
"--env-file", $environmentFile,
|
||||||
|
"up", "-d"
|
||||||
|
)
|
||||||
|
if (-not $NoBuild) { $arguments += "--build" }
|
||||||
|
$arguments += @(
|
||||||
|
"postgres", "redis", "minio", "keycloak", "migrate", "seed",
|
||||||
|
"callbacks", "mock-alert-receiver", "mock-provider", "api", "worker"
|
||||||
|
)
|
||||||
|
& docker @arguments
|
||||||
|
if ($LASTEXITCODE -ne 0) { throw "Docker Compose 启动失败" }
|
||||||
|
|
||||||
|
Write-Host "`n等待本地 API 就绪..." -ForegroundColor Cyan
|
||||||
|
$deadline = [DateTime]::UtcNow.AddMinutes(3)
|
||||||
|
do {
|
||||||
|
Start-Sleep -Seconds 2
|
||||||
|
try {
|
||||||
|
$health = Invoke-RestMethod -Uri "http://localhost:58080/health/ready" -TimeoutSec 3
|
||||||
|
if ($health) { break }
|
||||||
|
}
|
||||||
|
catch { }
|
||||||
|
} while ([DateTime]::UtcNow -lt $deadline)
|
||||||
|
|
||||||
|
if (-not $health) { throw "本地 API 未在三分钟内就绪,请检查 docker compose logs api worker" }
|
||||||
|
Write-Host "本地真实 RTC 后端已就绪:http://localhost:58080" -ForegroundColor Green
|
||||||
|
Write-Host "现在可以启动 Windows 语音终端模拟器。" -ForegroundColor Green
|
||||||
|
}
|
||||||
|
finally {
|
||||||
|
Pop-Location
|
||||||
|
}
|
||||||
@@ -0,0 +1,4 @@
|
|||||||
|
node_modules/
|
||||||
|
dist/
|
||||||
|
release/
|
||||||
|
*.log
|
||||||
@@ -0,0 +1,23 @@
|
|||||||
|
# Windows 语音终端模拟器
|
||||||
|
|
||||||
|
该程序使用Electron和火山引擎Web RTC SDK模拟一台真实语音终端。它采集Windows麦克风、加入平台分配的RTC房间、发布本地音频、订阅并播放豆包智能体音频,并通过设备v2接口完成激活、状态等待、心跳、Token续签和结束通话。
|
||||||
|
|
||||||
|
## 构建
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
Set-Location .\device-simulator-windows
|
||||||
|
npm install
|
||||||
|
npm run build
|
||||||
|
```
|
||||||
|
|
||||||
|
输出:`release/voice-device-simulator-0.1.0-windows-x64.exe`。
|
||||||
|
|
||||||
|
## 安全边界
|
||||||
|
|
||||||
|
- RTC AppKey、火山AK/SK和豆包Access Token只存在本地后端的受保护文件中;
|
||||||
|
- 模拟器安装包只接收平台签发的短期RTC Token;
|
||||||
|
- 设备签名密钥只驻留Electron主进程内存,不写入浏览器存储;
|
||||||
|
- 当前构建只允许设备API地址指向`localhost`或`127.0.0.1`,避免误连服务器环境;
|
||||||
|
- 关闭软件或结束通话时停止音频采集并销毁RTC引擎。
|
||||||
|
|
||||||
|
完整本地启动顺序见[本地真实RTC联调说明](../deploy/local-rtc/README.md)。
|
||||||
@@ -0,0 +1,208 @@
|
|||||||
|
const { app, BrowserWindow, ipcMain, session } = require("electron");
|
||||||
|
const crypto = require("node:crypto");
|
||||||
|
const fs = require("node:fs");
|
||||||
|
const path = require("node:path");
|
||||||
|
|
||||||
|
app.commandLine.appendSwitch("autoplay-policy", "no-user-gesture-required");
|
||||||
|
|
||||||
|
let deviceConfig = null;
|
||||||
|
let desiredConfigRevision = 1;
|
||||||
|
|
||||||
|
function createWindow() {
|
||||||
|
const window = new BrowserWindow({
|
||||||
|
width: 1220,
|
||||||
|
height: 820,
|
||||||
|
minWidth: 980,
|
||||||
|
minHeight: 700,
|
||||||
|
backgroundColor: "#eef3f0",
|
||||||
|
autoHideMenuBar: true,
|
||||||
|
title: "语音终端模拟器",
|
||||||
|
webPreferences: {
|
||||||
|
preload: path.join(__dirname, "preload.cjs"),
|
||||||
|
contextIsolation: true,
|
||||||
|
nodeIntegration: false,
|
||||||
|
sandbox: true,
|
||||||
|
},
|
||||||
|
});
|
||||||
|
|
||||||
|
const devArgument = process.argv.find((argument) => argument.startsWith("--dev-server-url="));
|
||||||
|
const devServerURL = devArgument ? devArgument.slice("--dev-server-url=".length) : "";
|
||||||
|
if (devServerURL) {
|
||||||
|
window.loadURL(devServerURL);
|
||||||
|
} else {
|
||||||
|
window.loadFile(path.join(__dirname, "..", "dist", "index.html"));
|
||||||
|
}
|
||||||
|
|
||||||
|
const screenshotPath = process.env.SIMULATOR_SCREENSHOT_PATH;
|
||||||
|
if (screenshotPath) {
|
||||||
|
window.webContents.once("did-finish-load", () => {
|
||||||
|
setTimeout(async () => {
|
||||||
|
const image = await window.webContents.capturePage();
|
||||||
|
fs.writeFileSync(screenshotPath, image.toPNG());
|
||||||
|
app.quit();
|
||||||
|
}, 1200);
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function ensureLoopbackURL(value) {
|
||||||
|
const parsed = new URL(String(value || ""));
|
||||||
|
const loopback = parsed.hostname === "127.0.0.1" || parsed.hostname === "localhost" || parsed.hostname === "[::1]";
|
||||||
|
if (!loopback || (parsed.protocol !== "http:" && parsed.protocol !== "https:")) {
|
||||||
|
throw new Error("本地测试只允许连接 localhost 或 127.0.0.1");
|
||||||
|
}
|
||||||
|
return parsed.origin;
|
||||||
|
}
|
||||||
|
|
||||||
|
function configure(input) {
|
||||||
|
const apiURL = ensureLoopbackURL(input?.apiURL);
|
||||||
|
const deviceID = String(input?.deviceID || "").trim();
|
||||||
|
const deviceSecret = String(input?.deviceSecret || "");
|
||||||
|
if (!/^[0-9a-f-]{36}$/i.test(deviceID)) {
|
||||||
|
throw new Error("设备 ID 必须是 UUID");
|
||||||
|
}
|
||||||
|
if (deviceSecret.length < 32) {
|
||||||
|
throw new Error("设备密钥至少需要 32 个字符");
|
||||||
|
}
|
||||||
|
deviceConfig = { apiURL, deviceID, deviceSecret };
|
||||||
|
return { apiURL, deviceID };
|
||||||
|
}
|
||||||
|
|
||||||
|
function nonce() {
|
||||||
|
return crypto.randomBytes(16).toString("hex");
|
||||||
|
}
|
||||||
|
|
||||||
|
function signedHeaders(method, requestURI, body) {
|
||||||
|
if (!deviceConfig) throw new Error("请先连接本地平台");
|
||||||
|
const timestamp = new Date().toISOString();
|
||||||
|
const requestNonce = nonce();
|
||||||
|
const digest = crypto.createHash("sha256").update(body).digest("hex");
|
||||||
|
const canonical = [method.toUpperCase(), requestURI, timestamp, requestNonce, digest].join("\n");
|
||||||
|
const signature = crypto.createHmac("sha256", deviceConfig.deviceSecret).update(canonical).digest("base64url");
|
||||||
|
return {
|
||||||
|
"X-Device-Id": deviceConfig.deviceID,
|
||||||
|
"X-Device-Timestamp": timestamp,
|
||||||
|
"X-Device-Nonce": requestNonce,
|
||||||
|
"X-Device-Content-SHA256": digest,
|
||||||
|
"X-Device-Signature": signature,
|
||||||
|
};
|
||||||
|
}
|
||||||
|
|
||||||
|
async function deviceRequest(method, requestPath, payload, idempotencyKey) {
|
||||||
|
if (!deviceConfig) throw new Error("请先连接本地平台");
|
||||||
|
const target = new URL(requestPath, `${deviceConfig.apiURL}/`);
|
||||||
|
const body = payload === undefined ? "" : JSON.stringify(payload);
|
||||||
|
const headers = signedHeaders(method, `${target.pathname}${target.search}`, body);
|
||||||
|
headers.Accept = "application/json";
|
||||||
|
if (body) headers["Content-Type"] = "application/json";
|
||||||
|
if (idempotencyKey) headers["Idempotency-Key"] = idempotencyKey;
|
||||||
|
const response = await fetch(target, { method, headers, body: body || undefined });
|
||||||
|
const text = await response.text();
|
||||||
|
let parsed;
|
||||||
|
try {
|
||||||
|
parsed = text ? JSON.parse(text) : {};
|
||||||
|
} catch {
|
||||||
|
parsed = { message: text };
|
||||||
|
}
|
||||||
|
if (!response.ok) {
|
||||||
|
const code = parsed?.error?.code || parsed?.code || `HTTP_${response.status}`;
|
||||||
|
const message = parsed?.error?.message || parsed?.message || "本地平台请求失败";
|
||||||
|
throw new Error(`${code}: ${message}`);
|
||||||
|
}
|
||||||
|
return parsed?.data ?? parsed;
|
||||||
|
}
|
||||||
|
|
||||||
|
ipcMain.handle("sim:configure", (_event, input) => configure(input));
|
||||||
|
|
||||||
|
ipcMain.handle("sim:probe", async () => {
|
||||||
|
const query = new URLSearchParams({
|
||||||
|
firmware_version: "0.3.0-windows-rtc",
|
||||||
|
board_rev: "windows-x64",
|
||||||
|
rtc_sdk_version: "web-4.69.0-electron",
|
||||||
|
codec: "opus",
|
||||||
|
sample_rate: "48000",
|
||||||
|
network: "windows-local",
|
||||||
|
device_time: new Date().toISOString(),
|
||||||
|
});
|
||||||
|
const bootstrap = await deviceRequest("GET", `/device/v2/bootstrap?${query.toString()}`);
|
||||||
|
const configuration = await deviceRequest("GET", "/device/v2/config");
|
||||||
|
desiredConfigRevision = Number(configuration.revision || bootstrap?.device?.config_revision || 1);
|
||||||
|
return { bootstrap, configuration, desiredConfigRevision };
|
||||||
|
});
|
||||||
|
|
||||||
|
ipcMain.handle("sim:create-conversation", async () => deviceRequest("POST", "/device/v2/conversations", {
|
||||||
|
device_time: new Date().toISOString(),
|
||||||
|
monotonic_ms: Math.max(0, Math.round(performance.now())),
|
||||||
|
capabilities: {
|
||||||
|
rtc: true,
|
||||||
|
audio_tracks: ["near", "far"],
|
||||||
|
upload_resume: true,
|
||||||
|
rtc_credential_renewal: true,
|
||||||
|
},
|
||||||
|
network: { type: "windows-local" },
|
||||||
|
requested_goal: "open_conversation",
|
||||||
|
}, `windows-create-${nonce()}`));
|
||||||
|
|
||||||
|
ipcMain.handle("sim:activate-conversation", async (_event, input) => deviceRequest(
|
||||||
|
"POST",
|
||||||
|
`/device/v2/conversations/${encodeURIComponent(input.conversationID)}/activate`,
|
||||||
|
{
|
||||||
|
joined_at: new Date().toISOString(),
|
||||||
|
room_id: input.roomID,
|
||||||
|
rtc_user_id: input.userID,
|
||||||
|
rtc_join_result: "success",
|
||||||
|
config_revision: desiredConfigRevision,
|
||||||
|
connection_reference: "windows-electron-web-rtc",
|
||||||
|
},
|
||||||
|
`windows-activate-${nonce()}`,
|
||||||
|
));
|
||||||
|
|
||||||
|
ipcMain.handle("sim:get-conversation", async (_event, conversationID) => deviceRequest(
|
||||||
|
"GET",
|
||||||
|
`/device/v2/conversations/${encodeURIComponent(conversationID)}`,
|
||||||
|
));
|
||||||
|
|
||||||
|
ipcMain.handle("sim:heartbeat", async (_event, input) => deviceRequest(
|
||||||
|
"POST",
|
||||||
|
`/device/v2/conversations/${encodeURIComponent(input.conversationID)}/heartbeat`,
|
||||||
|
{
|
||||||
|
sequence_no: input.sequenceNo,
|
||||||
|
device_time: new Date().toISOString(),
|
||||||
|
monotonic_ms: Math.max(0, Math.round(performance.now())),
|
||||||
|
conversation_status: "ACTIVE",
|
||||||
|
rtc: { joined: true, uplink: true, downlink: true },
|
||||||
|
network: { type: "windows-local" },
|
||||||
|
firmware_version: "0.3.0-windows-rtc",
|
||||||
|
reported_config_revision: desiredConfigRevision,
|
||||||
|
rtc_credential_expires_at: input.credentialExpiresAt,
|
||||||
|
recording_queue: { pending_segments: 0, failed_segments: 0 },
|
||||||
|
command_results: [],
|
||||||
|
},
|
||||||
|
));
|
||||||
|
|
||||||
|
ipcMain.handle("sim:stop-conversation", async (_event, input) => deviceRequest(
|
||||||
|
"POST",
|
||||||
|
`/device/v2/conversations/${encodeURIComponent(input.conversationID)}/stop`,
|
||||||
|
{
|
||||||
|
reason: input.reason || "windows_user_hangup",
|
||||||
|
device_time: new Date().toISOString(),
|
||||||
|
last_heartbeat_sequence: input.lastHeartbeatSequence || 0,
|
||||||
|
rtc_left: true,
|
||||||
|
recording: { near_segments: 0, far_segments: 0 },
|
||||||
|
},
|
||||||
|
`windows-stop-${nonce()}`,
|
||||||
|
));
|
||||||
|
|
||||||
|
app.whenReady().then(() => {
|
||||||
|
session.defaultSession.setPermissionCheckHandler((_webContents, permission) => permission === "media");
|
||||||
|
session.defaultSession.setPermissionRequestHandler((_webContents, permission, callback) => callback(permission === "media"));
|
||||||
|
createWindow();
|
||||||
|
});
|
||||||
|
|
||||||
|
app.on("window-all-closed", () => {
|
||||||
|
if (process.platform !== "darwin") app.quit();
|
||||||
|
});
|
||||||
|
|
||||||
|
app.on("activate", () => {
|
||||||
|
if (BrowserWindow.getAllWindows().length === 0) createWindow();
|
||||||
|
});
|
||||||
@@ -0,0 +1,11 @@
|
|||||||
|
const { contextBridge, ipcRenderer } = require("electron");
|
||||||
|
|
||||||
|
contextBridge.exposeInMainWorld("deviceSimulator", {
|
||||||
|
configure: (input) => ipcRenderer.invoke("sim:configure", input),
|
||||||
|
probe: () => ipcRenderer.invoke("sim:probe"),
|
||||||
|
createConversation: () => ipcRenderer.invoke("sim:create-conversation"),
|
||||||
|
activateConversation: (input) => ipcRenderer.invoke("sim:activate-conversation", input),
|
||||||
|
getConversation: (conversationID) => ipcRenderer.invoke("sim:get-conversation", conversationID),
|
||||||
|
heartbeat: (input) => ipcRenderer.invoke("sim:heartbeat", input),
|
||||||
|
stopConversation: (input) => ipcRenderer.invoke("sim:stop-conversation", input),
|
||||||
|
});
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
<!doctype html>
|
||||||
|
<html lang="zh-CN">
|
||||||
|
<head>
|
||||||
|
<meta charset="UTF-8" />
|
||||||
|
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
|
||||||
|
<meta http-equiv="Content-Security-Policy" content="default-src 'self'; script-src 'self'; style-src 'self'; connect-src 'self' https: wss: http://127.0.0.1:* http://localhost:*; media-src 'self' blob:; worker-src 'self' blob:; img-src 'self' data:;" />
|
||||||
|
<title>语音终端模拟器</title>
|
||||||
|
</head>
|
||||||
|
<body>
|
||||||
|
<main id="app"></main>
|
||||||
|
<script type="module" src="/src/main.ts"></script>
|
||||||
|
</body>
|
||||||
|
</html>
|
||||||
+4930
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,46 @@
|
|||||||
|
{
|
||||||
|
"name": "data-terminal-voice-device-simulator",
|
||||||
|
"version": "0.1.0",
|
||||||
|
"private": true,
|
||||||
|
"description": "Windows real-audio device simulator for the data terminal cloud platform",
|
||||||
|
"main": "electron/main.cjs",
|
||||||
|
"scripts": {
|
||||||
|
"dev": "concurrently -k \"vite --host 127.0.0.1 --port 5177\" \"wait-on http://127.0.0.1:5177 && electron . --dev-server-url=http://127.0.0.1:5177\"",
|
||||||
|
"typecheck": "tsc --noEmit",
|
||||||
|
"build:renderer": "vite build",
|
||||||
|
"start": "npm run build:renderer && electron .",
|
||||||
|
"build": "npm run typecheck && npm run build:renderer && electron-builder --win portable"
|
||||||
|
},
|
||||||
|
"dependencies": {
|
||||||
|
"@volcengine/rtc": "4.69.0"
|
||||||
|
},
|
||||||
|
"devDependencies": {
|
||||||
|
"concurrently": "^9.2.1",
|
||||||
|
"electron": "44.0.0",
|
||||||
|
"electron-builder": "26.15.3",
|
||||||
|
"typescript": "^5.9.2",
|
||||||
|
"vite": "^7.1.3",
|
||||||
|
"wait-on": "^8.0.4"
|
||||||
|
},
|
||||||
|
"build": {
|
||||||
|
"appId": "cn.dataterminal.voice-device-simulator",
|
||||||
|
"productName": "语音终端模拟器",
|
||||||
|
"artifactName": "voice-device-simulator-${version}-windows-${arch}.${ext}",
|
||||||
|
"directories": {
|
||||||
|
"output": "release"
|
||||||
|
},
|
||||||
|
"files": [
|
||||||
|
"dist/**/*",
|
||||||
|
"electron/**/*",
|
||||||
|
"package.json"
|
||||||
|
],
|
||||||
|
"win": {
|
||||||
|
"target": [
|
||||||
|
{
|
||||||
|
"target": "portable",
|
||||||
|
"arch": ["x64"]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
+40
@@ -0,0 +1,40 @@
|
|||||||
|
type SimulatorConfiguration = {
|
||||||
|
apiURL: string;
|
||||||
|
deviceID: string;
|
||||||
|
deviceSecret: string;
|
||||||
|
};
|
||||||
|
|
||||||
|
type RTCGrant = {
|
||||||
|
app_id: string;
|
||||||
|
room_id: string;
|
||||||
|
user_id: string;
|
||||||
|
agent_user_id: string;
|
||||||
|
token: string;
|
||||||
|
expires_at: string;
|
||||||
|
};
|
||||||
|
|
||||||
|
type CreatedConversation = {
|
||||||
|
conversation_id: string;
|
||||||
|
status: string;
|
||||||
|
rtc: RTCGrant;
|
||||||
|
session: {
|
||||||
|
engine_mode: string;
|
||||||
|
heartbeat_interval_sec: number;
|
||||||
|
allocation_expires_at: string;
|
||||||
|
};
|
||||||
|
};
|
||||||
|
|
||||||
|
interface Window {
|
||||||
|
deviceSimulator: {
|
||||||
|
configure(input: SimulatorConfiguration): Promise<{ apiURL: string; deviceID: string }>;
|
||||||
|
probe(): Promise<Record<string, unknown>>;
|
||||||
|
createConversation(): Promise<CreatedConversation>;
|
||||||
|
activateConversation(input: { conversationID: string; roomID: string; userID: string }): Promise<Record<string, unknown>>;
|
||||||
|
getConversation(conversationID: string): Promise<{ status: string; action: string; failure_code?: string }>;
|
||||||
|
heartbeat(input: { conversationID: string; sequenceNo: number; credentialExpiresAt: string }): Promise<{
|
||||||
|
actions?: Array<{ type: string; rtc?: RTCGrant }>;
|
||||||
|
desired_config_revision?: number;
|
||||||
|
}>;
|
||||||
|
stopConversation(input: { conversationID: string; reason: string; lastHeartbeatSequence: number }): Promise<Record<string, unknown>>;
|
||||||
|
};
|
||||||
|
}
|
||||||
@@ -0,0 +1,413 @@
|
|||||||
|
import VERTC, {
|
||||||
|
IRTCEngine,
|
||||||
|
LocalAudioPropertiesInfo,
|
||||||
|
MediaType,
|
||||||
|
RemoteAudioPropertiesInfo,
|
||||||
|
RoomProfileType,
|
||||||
|
} from "@volcengine/rtc";
|
||||||
|
import "./style.css";
|
||||||
|
|
||||||
|
const EngineEventsTypes = VERTC.events;
|
||||||
|
|
||||||
|
type Phase = "idle" | "checking" | "ready" | "joining" | "waiting" | "active" | "stopping" | "error";
|
||||||
|
|
||||||
|
const state: {
|
||||||
|
phase: Phase;
|
||||||
|
engine?: IRTCEngine;
|
||||||
|
conversation?: CreatedConversation;
|
||||||
|
credentialExpiresAt?: string;
|
||||||
|
heartbeatSequence: number;
|
||||||
|
heartbeatTimer?: number;
|
||||||
|
statusTimer?: number;
|
||||||
|
startedAt?: number;
|
||||||
|
durationTimer?: number;
|
||||||
|
agentJoined: boolean;
|
||||||
|
agentSpeaking: boolean;
|
||||||
|
microphoneVolume: number;
|
||||||
|
agentVolume: number;
|
||||||
|
} = {
|
||||||
|
phase: "idle",
|
||||||
|
heartbeatSequence: 0,
|
||||||
|
agentJoined: false,
|
||||||
|
agentSpeaking: false,
|
||||||
|
microphoneVolume: 0,
|
||||||
|
agentVolume: 0,
|
||||||
|
};
|
||||||
|
|
||||||
|
const app = document.querySelector<HTMLElement>("#app")!;
|
||||||
|
app.innerHTML = `
|
||||||
|
<section class="shell">
|
||||||
|
<header class="hero">
|
||||||
|
<div>
|
||||||
|
<p class="eyebrow">WINDOWS · REAL RTC DEVICE</p>
|
||||||
|
<h1>语音终端模拟器</h1>
|
||||||
|
<p class="subtitle">使用真实麦克风和扬声器,验证“终端入房—豆包进房—双向语音—打断—结束”的完整链路。</p>
|
||||||
|
</div>
|
||||||
|
<div class="overall" id="overallStatus"><span class="pulse"></span><strong>尚未连接</strong></div>
|
||||||
|
</header>
|
||||||
|
|
||||||
|
<section class="status-grid" aria-label="链路状态">
|
||||||
|
<article class="status-card"><span>本地平台</span><strong id="apiStatus">未检测</strong><small id="apiDetail">localhost</small></article>
|
||||||
|
<article class="status-card"><span>RTC 房间</span><strong id="rtcStatus">未加入</strong><small id="rtcDetail">等待开始</small></article>
|
||||||
|
<article class="status-card"><span>豆包智能体</span><strong id="agentStatus">未进房</strong><small id="agentDetail">等待 StartVoiceChat</small></article>
|
||||||
|
<article class="status-card"><span>通话时长</span><strong id="duration">00:00</strong><small id="modeDetail">处理方式:—</small></article>
|
||||||
|
</section>
|
||||||
|
|
||||||
|
<section class="content-grid">
|
||||||
|
<article class="panel setup-panel">
|
||||||
|
<div class="panel-heading"><div><p class="eyebrow">LOCAL SETUP</p><h2>本地设备配置</h2></div><span class="safe">密钥仅驻留内存</span></div>
|
||||||
|
<label>本地平台地址<input id="apiURL" value="http://localhost:58080" autocomplete="off" /></label>
|
||||||
|
<label>模拟设备 ID<input id="deviceID" value="10000000-0000-0000-0000-000000000511" autocomplete="off" /></label>
|
||||||
|
<label>设备签名密钥<input id="deviceSecret" type="password" value="local-simulated-device-secret-32-bytes-minimum" autocomplete="off" /></label>
|
||||||
|
<div class="row">
|
||||||
|
<label>麦克风<select id="microphone"><option value="">默认麦克风</option></select></label>
|
||||||
|
<label>扬声器<select id="speaker"><option value="">默认扬声器</option></select></label>
|
||||||
|
</div>
|
||||||
|
<button class="secondary" id="checkButton">检测平台和音频设备</button>
|
||||||
|
<p class="hint">RTC AppKey、火山 AK/SK 和豆包 Token 应配置在本地后端,本软件不会读取或保存它们。</p>
|
||||||
|
</article>
|
||||||
|
|
||||||
|
<article class="panel call-panel">
|
||||||
|
<div class="panel-heading"><div><p class="eyebrow">LIVE CONVERSATION</p><h2>真实双向语音</h2></div><span class="phase-badge" id="phaseBadge">待机</span></div>
|
||||||
|
<div class="conversation-orb" id="conversationOrb"><div class="orb-core"><span id="orbLabel">准备开始</span></div></div>
|
||||||
|
<div class="meters">
|
||||||
|
<div><div class="meter-label"><span>我的麦克风</span><strong id="micValue">0</strong></div><div class="meter"><i id="micMeter"></i></div></div>
|
||||||
|
<div><div class="meter-label"><span>豆包语音</span><strong id="agentValue">0</strong></div><div class="meter agent"><i id="agentMeter"></i></div></div>
|
||||||
|
</div>
|
||||||
|
<div class="call-actions">
|
||||||
|
<button class="primary" id="startButton" disabled>开始语音聊天</button>
|
||||||
|
<button class="danger" id="stopButton" disabled>结束通话</button>
|
||||||
|
</div>
|
||||||
|
<dl class="session-info">
|
||||||
|
<div><dt>通话编号</dt><dd id="conversationID">—</dd></div>
|
||||||
|
<div><dt>RTC 房间</dt><dd id="roomID">—</dd></div>
|
||||||
|
<div><dt>终端用户</dt><dd id="userID">—</dd></div>
|
||||||
|
<div><dt>智能体用户</dt><dd id="agentUserID">—</dd></div>
|
||||||
|
</dl>
|
||||||
|
</article>
|
||||||
|
</section>
|
||||||
|
|
||||||
|
<section class="panel log-panel">
|
||||||
|
<div class="panel-heading"><div><p class="eyebrow">EVENT STREAM</p><h2>实时事件</h2></div><button class="text-button" id="clearLog">清空</button></div>
|
||||||
|
<ol id="eventLog" aria-live="polite"><li><time>系统</time><span>模拟器已启动,等待检测本地环境。</span></li></ol>
|
||||||
|
</section>
|
||||||
|
</section>
|
||||||
|
`;
|
||||||
|
|
||||||
|
const $ = <T extends HTMLElement>(id: string) => document.getElementById(id) as T;
|
||||||
|
const controls = {
|
||||||
|
apiURL: $<HTMLInputElement>("apiURL"),
|
||||||
|
deviceID: $<HTMLInputElement>("deviceID"),
|
||||||
|
deviceSecret: $<HTMLInputElement>("deviceSecret"),
|
||||||
|
microphone: $<HTMLSelectElement>("microphone"),
|
||||||
|
speaker: $<HTMLSelectElement>("speaker"),
|
||||||
|
check: $<HTMLButtonElement>("checkButton"),
|
||||||
|
start: $<HTMLButtonElement>("startButton"),
|
||||||
|
stop: $<HTMLButtonElement>("stopButton"),
|
||||||
|
};
|
||||||
|
|
||||||
|
function setText(id: string, value: string) {
|
||||||
|
$(id).textContent = value;
|
||||||
|
}
|
||||||
|
|
||||||
|
function log(message: string, level: "info" | "success" | "error" = "info") {
|
||||||
|
const item = document.createElement("li");
|
||||||
|
item.className = level;
|
||||||
|
const time = document.createElement("time");
|
||||||
|
time.textContent = new Date().toLocaleTimeString("zh-CN", { hour12: false });
|
||||||
|
const content = document.createElement("span");
|
||||||
|
content.textContent = message;
|
||||||
|
item.append(time, content);
|
||||||
|
$("eventLog").prepend(item);
|
||||||
|
}
|
||||||
|
|
||||||
|
function errorMessage(error: unknown): string {
|
||||||
|
return error instanceof Error ? error.message : String(error);
|
||||||
|
}
|
||||||
|
|
||||||
|
function setPhase(phase: Phase, detail?: string) {
|
||||||
|
state.phase = phase;
|
||||||
|
const labels: Record<Phase, string> = {
|
||||||
|
idle: "待机", checking: "检测中", ready: "已就绪", joining: "正在入房", waiting: "等待豆包", active: "通话中", stopping: "结束中", error: "异常",
|
||||||
|
};
|
||||||
|
setText("phaseBadge", labels[phase]);
|
||||||
|
setText("overallStatus", detail || labels[phase]);
|
||||||
|
$("overallStatus").className = `overall phase-${phase}`;
|
||||||
|
$("conversationOrb").className = `conversation-orb phase-${phase}${state.agentSpeaking ? " speaking" : ""}`;
|
||||||
|
setText("orbLabel", detail || labels[phase]);
|
||||||
|
controls.start.disabled = phase !== "ready";
|
||||||
|
controls.stop.disabled = !["joining", "waiting", "active", "error"].includes(phase) || !state.conversation;
|
||||||
|
controls.check.disabled = ["joining", "waiting", "active", "stopping"].includes(phase);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function configureAndProbe() {
|
||||||
|
setPhase("checking", "正在检测");
|
||||||
|
try {
|
||||||
|
await window.deviceSimulator.configure({
|
||||||
|
apiURL: controls.apiURL.value,
|
||||||
|
deviceID: controls.deviceID.value,
|
||||||
|
deviceSecret: controls.deviceSecret.value,
|
||||||
|
});
|
||||||
|
const supported = await VERTC.isSupported();
|
||||||
|
if (!supported) throw new Error("当前 Electron 环境不支持火山 Web RTC SDK");
|
||||||
|
const permission = await VERTC.enableDevices({ audio: true, video: false });
|
||||||
|
if (!permission.audio) throw permission.audioExceptionError || new Error("未获得麦克风权限");
|
||||||
|
await refreshAudioDevices();
|
||||||
|
await window.deviceSimulator.probe();
|
||||||
|
setText("apiStatus", "连接正常");
|
||||||
|
setText("apiDetail", controls.apiURL.value);
|
||||||
|
setPhase("ready", "可以开始通话");
|
||||||
|
log(`环境检测通过,RTC Web SDK ${VERTC.getSdkVersion()}。`, "success");
|
||||||
|
} catch (error) {
|
||||||
|
setText("apiStatus", "检测失败");
|
||||||
|
setText("apiDetail", errorMessage(error));
|
||||||
|
setPhase("error", "检测失败");
|
||||||
|
controls.start.disabled = true;
|
||||||
|
log(`环境检测失败:${errorMessage(error)}`, "error");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function refreshAudioDevices() {
|
||||||
|
const [microphones, speakers] = await Promise.all([
|
||||||
|
VERTC.enumerateAudioCaptureDevices(),
|
||||||
|
VERTC.enumerateAudioPlaybackDevices(),
|
||||||
|
]);
|
||||||
|
fillDeviceOptions(controls.microphone, microphones, "默认麦克风");
|
||||||
|
fillDeviceOptions(controls.speaker, speakers, "默认扬声器");
|
||||||
|
}
|
||||||
|
|
||||||
|
function fillDeviceOptions(select: HTMLSelectElement, devices: MediaDeviceInfo[], fallback: string) {
|
||||||
|
const previous = select.value;
|
||||||
|
select.replaceChildren(new Option(fallback, ""));
|
||||||
|
devices.forEach((device, index) => select.add(new Option(device.label || `${fallback} ${index + 1}`, device.deviceId)));
|
||||||
|
if ([...select.options].some((option) => option.value === previous)) select.value = previous;
|
||||||
|
}
|
||||||
|
|
||||||
|
function registerRTCEvents(engine: IRTCEngine) {
|
||||||
|
engine.on(EngineEventsTypes.onUserJoined, (event) => {
|
||||||
|
const userID = event.userInfo.userId;
|
||||||
|
log(`远端用户已进房:${userID}`, "success");
|
||||||
|
if (userID === state.conversation?.rtc.agent_user_id) {
|
||||||
|
state.agentJoined = true;
|
||||||
|
setText("agentStatus", "已进房");
|
||||||
|
setText("agentDetail", userID);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
engine.on(EngineEventsTypes.onUserLeave, (event) => {
|
||||||
|
const userID = event.userInfo.userId;
|
||||||
|
log(`远端用户已离房:${userID}`);
|
||||||
|
if (userID === state.conversation?.rtc.agent_user_id) {
|
||||||
|
state.agentJoined = false;
|
||||||
|
setText("agentStatus", "已离房");
|
||||||
|
}
|
||||||
|
});
|
||||||
|
engine.on(EngineEventsTypes.onUserPublishStream, async (event) => {
|
||||||
|
if ((event.mediaType & MediaType.AUDIO) === 0) return;
|
||||||
|
log(`检测到远端音频流:${event.userId}`, "success");
|
||||||
|
try {
|
||||||
|
await engine.subscribeStream(event.userId, MediaType.AUDIO);
|
||||||
|
await engine.play(event.userId, MediaType.AUDIO);
|
||||||
|
if (event.userId === state.conversation?.rtc.agent_user_id) {
|
||||||
|
setText("agentStatus", "语音已连接");
|
||||||
|
setText("agentDetail", "正在接收豆包音频");
|
||||||
|
}
|
||||||
|
} catch (error) {
|
||||||
|
log(`播放远端音频失败:${errorMessage(error)}`, "error");
|
||||||
|
}
|
||||||
|
});
|
||||||
|
engine.on(EngineEventsTypes.onLocalAudioPropertiesReport, (reports: LocalAudioPropertiesInfo[]) => {
|
||||||
|
const volume = Math.max(0, ...reports.map((item) => item.audioPropertiesInfo.linearVolume || 0));
|
||||||
|
updateVolume("mic", volume);
|
||||||
|
});
|
||||||
|
engine.on(EngineEventsTypes.onRemoteAudioPropertiesReport, (reports: RemoteAudioPropertiesInfo[]) => {
|
||||||
|
const agentID = state.conversation?.rtc.agent_user_id;
|
||||||
|
const volume = Math.max(0, ...reports.filter((item) => !agentID || item.streamKey.userId === agentID).map((item) => item.audioPropertiesInfo.linearVolume || 0));
|
||||||
|
state.agentSpeaking = volume > 5;
|
||||||
|
updateVolume("agent", volume);
|
||||||
|
setPhase(state.phase, state.phase === "active" ? (state.agentSpeaking ? "豆包正在说话" : "请开始说话") : undefined);
|
||||||
|
});
|
||||||
|
engine.on(EngineEventsTypes.onConnectionStateChanged, (event) => {
|
||||||
|
log(`RTC 连接状态:${String(event.state)}`);
|
||||||
|
});
|
||||||
|
engine.on(EngineEventsTypes.onError, (event) => {
|
||||||
|
log(`RTC 错误:${event.errorCode}`, "error");
|
||||||
|
});
|
||||||
|
engine.on(EngineEventsTypes.onTokenWillExpire, () => {
|
||||||
|
log("RTC Token 即将过期,等待心跳续签。", "info");
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
function updateVolume(kind: "mic" | "agent", raw: number) {
|
||||||
|
const value = Math.max(0, Math.min(100, Math.round(raw)));
|
||||||
|
if (kind === "mic") state.microphoneVolume = value;
|
||||||
|
else state.agentVolume = value;
|
||||||
|
$(`${kind}Meter`).style.width = `${value}%`;
|
||||||
|
setText(`${kind}Value`, String(value));
|
||||||
|
}
|
||||||
|
|
||||||
|
async function startConversation() {
|
||||||
|
setPhase("joining", "正在申请房间");
|
||||||
|
controls.start.disabled = true;
|
||||||
|
try {
|
||||||
|
const created = await window.deviceSimulator.createConversation();
|
||||||
|
state.conversation = created;
|
||||||
|
state.credentialExpiresAt = created.rtc.expires_at;
|
||||||
|
state.heartbeatSequence = 0;
|
||||||
|
state.agentJoined = false;
|
||||||
|
renderSession(created);
|
||||||
|
log(`平台已分配 RTC 房间 ${created.rtc.room_id}。`, "success");
|
||||||
|
|
||||||
|
const engine = VERTC.createEngine(created.rtc.app_id);
|
||||||
|
state.engine = engine;
|
||||||
|
registerRTCEvents(engine);
|
||||||
|
engine.enableAudioPropertiesReport({ interval: 300 });
|
||||||
|
await engine.startAudioCapture(controls.microphone.value || undefined);
|
||||||
|
if (controls.speaker.value) await engine.setAudioPlaybackDevice(controls.speaker.value);
|
||||||
|
await engine.joinRoom(created.rtc.token, created.rtc.room_id, { userId: created.rtc.user_id }, {
|
||||||
|
isAutoPublish: true,
|
||||||
|
isAutoSubscribeAudio: true,
|
||||||
|
isAutoSubscribeVideo: false,
|
||||||
|
roomProfileType: RoomProfileType.chat,
|
||||||
|
});
|
||||||
|
setText("rtcStatus", "已加入");
|
||||||
|
setText("rtcDetail", created.rtc.room_id);
|
||||||
|
log("终端已加入 RTC 房间并发布麦克风。", "success");
|
||||||
|
|
||||||
|
await window.deviceSimulator.activateConversation({
|
||||||
|
conversationID: created.conversation_id,
|
||||||
|
roomID: created.rtc.room_id,
|
||||||
|
userID: created.rtc.user_id,
|
||||||
|
});
|
||||||
|
setPhase("waiting", "正在拉入豆包");
|
||||||
|
log("平台已提交 StartVoiceChat,等待豆包智能体激活。", "success");
|
||||||
|
await waitUntilActive(created.conversation_id);
|
||||||
|
state.startedAt = Date.now();
|
||||||
|
setPhase("active", "请开始说话");
|
||||||
|
startTimers(created.session.heartbeat_interval_sec || 15);
|
||||||
|
log("豆包语音任务已激活,可以开始双向聊天。", "success");
|
||||||
|
} catch (error) {
|
||||||
|
setPhase("error", "启动失败");
|
||||||
|
log(`启动通话失败:${errorMessage(error)}`, "error");
|
||||||
|
await releaseRTC();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function renderSession(created: CreatedConversation) {
|
||||||
|
setText("conversationID", created.conversation_id);
|
||||||
|
setText("roomID", created.rtc.room_id);
|
||||||
|
setText("userID", created.rtc.user_id);
|
||||||
|
setText("agentUserID", created.rtc.agent_user_id);
|
||||||
|
setText("modeDetail", `处理方式:${created.session.engine_mode}`);
|
||||||
|
setText("agentStatus", "等待进房");
|
||||||
|
setText("agentDetail", created.rtc.agent_user_id);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function waitUntilActive(conversationID: string) {
|
||||||
|
const deadline = Date.now() + 30_000;
|
||||||
|
while (Date.now() < deadline) {
|
||||||
|
const result = await window.deviceSimulator.getConversation(conversationID);
|
||||||
|
if (result.action === "continue" && result.status === "ACTIVE") return;
|
||||||
|
if (result.action === "leave_room") throw new Error(result.failure_code || `智能体启动失败:${result.status}`);
|
||||||
|
await new Promise((resolve) => window.setTimeout(resolve, 600));
|
||||||
|
}
|
||||||
|
throw new Error("等待豆包进房超时(30秒)");
|
||||||
|
}
|
||||||
|
|
||||||
|
function startTimers(intervalSeconds: number) {
|
||||||
|
stopTimers();
|
||||||
|
state.durationTimer = window.setInterval(() => {
|
||||||
|
if (!state.startedAt) return;
|
||||||
|
const seconds = Math.floor((Date.now() - state.startedAt) / 1000);
|
||||||
|
setText("duration", `${String(Math.floor(seconds / 60)).padStart(2, "0")}:${String(seconds % 60).padStart(2, "0")}`);
|
||||||
|
}, 1000);
|
||||||
|
state.heartbeatTimer = window.setInterval(() => void sendHeartbeat(), Math.max(5, intervalSeconds) * 1000);
|
||||||
|
void sendHeartbeat();
|
||||||
|
}
|
||||||
|
|
||||||
|
async function sendHeartbeat() {
|
||||||
|
if (!state.conversation || !state.credentialExpiresAt || state.phase !== "active") return;
|
||||||
|
try {
|
||||||
|
state.heartbeatSequence += 1;
|
||||||
|
const result = await window.deviceSimulator.heartbeat({
|
||||||
|
conversationID: state.conversation.conversation_id,
|
||||||
|
sequenceNo: state.heartbeatSequence,
|
||||||
|
credentialExpiresAt: state.credentialExpiresAt,
|
||||||
|
});
|
||||||
|
for (const action of result.actions || []) {
|
||||||
|
if (action.type === "renew_rtc_credential" && action.rtc?.token) {
|
||||||
|
await state.engine?.updateToken(action.rtc.token);
|
||||||
|
state.credentialExpiresAt = action.rtc.expires_at;
|
||||||
|
log("RTC Token 已通过设备心跳完成续签。", "success");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} catch (error) {
|
||||||
|
log(`设备心跳失败:${errorMessage(error)}`, "error");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function stopConversation(reason = "windows_user_hangup") {
|
||||||
|
if (!state.conversation) return;
|
||||||
|
setPhase("stopping", "正在结束通话");
|
||||||
|
stopTimers();
|
||||||
|
const conversation = state.conversation;
|
||||||
|
try {
|
||||||
|
await releaseRTC();
|
||||||
|
await window.deviceSimulator.stopConversation({
|
||||||
|
conversationID: conversation.conversation_id,
|
||||||
|
reason,
|
||||||
|
lastHeartbeatSequence: state.heartbeatSequence,
|
||||||
|
});
|
||||||
|
log("RTC 已离房,平台已提交 StopVoiceChat。", "success");
|
||||||
|
resetSession();
|
||||||
|
setPhase("ready", "可以再次通话");
|
||||||
|
} catch (error) {
|
||||||
|
log(`结束通话失败:${errorMessage(error)}`, "error");
|
||||||
|
resetSession();
|
||||||
|
setPhase("ready", "已释放本地音频");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function releaseRTC() {
|
||||||
|
const engine = state.engine;
|
||||||
|
state.engine = undefined;
|
||||||
|
if (!engine) return;
|
||||||
|
try { await engine.unpublishStream(MediaType.AUDIO); } catch { /* no-op */ }
|
||||||
|
try { await engine.leaveRoom(); } catch { /* no-op */ }
|
||||||
|
try { await engine.stopAudioCapture(); } catch { /* no-op */ }
|
||||||
|
try { VERTC.destroyEngine(engine); } catch { /* no-op */ }
|
||||||
|
}
|
||||||
|
|
||||||
|
function stopTimers() {
|
||||||
|
if (state.heartbeatTimer) window.clearInterval(state.heartbeatTimer);
|
||||||
|
if (state.statusTimer) window.clearInterval(state.statusTimer);
|
||||||
|
if (state.durationTimer) window.clearInterval(state.durationTimer);
|
||||||
|
state.heartbeatTimer = undefined;
|
||||||
|
state.statusTimer = undefined;
|
||||||
|
state.durationTimer = undefined;
|
||||||
|
}
|
||||||
|
|
||||||
|
function resetSession() {
|
||||||
|
state.conversation = undefined;
|
||||||
|
state.credentialExpiresAt = undefined;
|
||||||
|
state.startedAt = undefined;
|
||||||
|
state.agentJoined = false;
|
||||||
|
state.agentSpeaking = false;
|
||||||
|
setText("rtcStatus", "未加入");
|
||||||
|
setText("rtcDetail", "等待开始");
|
||||||
|
setText("agentStatus", "未进房");
|
||||||
|
setText("agentDetail", "等待 StartVoiceChat");
|
||||||
|
setText("duration", "00:00");
|
||||||
|
updateVolume("mic", 0);
|
||||||
|
updateVolume("agent", 0);
|
||||||
|
}
|
||||||
|
|
||||||
|
controls.check.addEventListener("click", () => void configureAndProbe());
|
||||||
|
controls.start.addEventListener("click", () => void startConversation());
|
||||||
|
controls.stop.addEventListener("click", () => void stopConversation());
|
||||||
|
$("clearLog").addEventListener("click", () => $("eventLog").replaceChildren());
|
||||||
|
window.addEventListener("beforeunload", () => {
|
||||||
|
stopTimers();
|
||||||
|
void releaseRTC();
|
||||||
|
});
|
||||||
|
|
||||||
|
setPhase("idle", "尚未连接");
|
||||||
@@ -0,0 +1,74 @@
|
|||||||
|
:root {
|
||||||
|
font-family: "Microsoft YaHei UI", "Segoe UI", sans-serif;
|
||||||
|
color: #112d3f;
|
||||||
|
background: #eef3f0;
|
||||||
|
font-synthesis: none;
|
||||||
|
}
|
||||||
|
|
||||||
|
* { box-sizing: border-box; }
|
||||||
|
body { margin: 0; min-width: 980px; min-height: 100vh; background: radial-gradient(circle at 88% 0%, #f5dfbd 0, transparent 30%), linear-gradient(145deg, #edf5f3 0%, #f7f4eb 100%); }
|
||||||
|
button, input, select { font: inherit; }
|
||||||
|
button { cursor: pointer; }
|
||||||
|
button:disabled { cursor: not-allowed; opacity: .45; }
|
||||||
|
|
||||||
|
.shell { width: min(1160px, calc(100vw - 48px)); margin: 0 auto; padding: 36px 0 48px; }
|
||||||
|
.hero { display: flex; align-items: flex-start; justify-content: space-between; gap: 32px; margin-bottom: 26px; }
|
||||||
|
.eyebrow { color: #c8652a; font-size: 12px; font-weight: 800; letter-spacing: .16em; margin: 0 0 9px; }
|
||||||
|
h1 { margin: 0; font-family: Georgia, "Microsoft YaHei UI", serif; font-size: 44px; letter-spacing: -.035em; }
|
||||||
|
h2 { margin: 0; font-size: 23px; }
|
||||||
|
.subtitle { max-width: 760px; color: #66808e; line-height: 1.8; margin: 12px 0 0; }
|
||||||
|
.overall { display: flex; align-items: center; min-width: 150px; justify-content: center; padding: 13px 18px; border-radius: 999px; background: rgba(255,255,255,.82); box-shadow: 0 12px 30px rgba(36,64,69,.1); color: #647a82; }
|
||||||
|
.overall.phase-ready, .overall.phase-active { color: #1a7d57; }
|
||||||
|
.overall.phase-error { color: #b63d2e; }
|
||||||
|
|
||||||
|
.status-grid { display: grid; grid-template-columns: repeat(4, 1fr); gap: 14px; margin-bottom: 16px; }
|
||||||
|
.status-card, .panel { background: rgba(255,255,255,.9); border: 1px solid rgba(23,61,69,.08); box-shadow: 0 14px 40px rgba(34,61,63,.08); }
|
||||||
|
.status-card { min-height: 118px; padding: 22px; border-radius: 20px; }
|
||||||
|
.status-card span { display: block; color: #778b93; font-size: 14px; }
|
||||||
|
.status-card strong { display: block; margin: 14px 0 6px; font-size: 21px; }
|
||||||
|
.status-card small { display: block; color: #8da0a6; white-space: nowrap; overflow: hidden; text-overflow: ellipsis; }
|
||||||
|
|
||||||
|
.content-grid { display: grid; grid-template-columns: .95fr 1.35fr; gap: 16px; }
|
||||||
|
.panel { border-radius: 24px; padding: 28px; }
|
||||||
|
.panel-heading { display: flex; justify-content: space-between; gap: 18px; align-items: flex-start; margin-bottom: 24px; }
|
||||||
|
.safe, .phase-badge { color: #287858; background: #e8f5ed; border-radius: 999px; padding: 8px 12px; font-size: 12px; font-weight: 700; }
|
||||||
|
label { display: block; color: #617985; font-size: 13px; margin-bottom: 16px; }
|
||||||
|
input, select { width: 100%; border: 1px solid #d7e2de; border-radius: 12px; padding: 12px 13px; margin-top: 7px; color: #153244; background: #f9fbfa; outline: none; }
|
||||||
|
input:focus, select:focus { border-color: #d6783f; box-shadow: 0 0 0 3px rgba(214,120,63,.12); }
|
||||||
|
.row { display: grid; grid-template-columns: 1fr 1fr; gap: 12px; }
|
||||||
|
.secondary, .primary, .danger { border: 0; border-radius: 13px; padding: 13px 18px; font-weight: 800; }
|
||||||
|
.secondary { width: 100%; color: #173b4d; background: #e9f0ee; }
|
||||||
|
.primary { flex: 1; color: white; background: #d36f32; box-shadow: 0 10px 24px rgba(211,111,50,.22); }
|
||||||
|
.danger { color: #9d382c; background: #f7e6e2; }
|
||||||
|
.hint { margin: 14px 0 0; color: #8a9aa0; font-size: 12px; line-height: 1.65; }
|
||||||
|
|
||||||
|
.conversation-orb { width: 180px; height: 180px; margin: 6px auto 24px; border-radius: 50%; display: grid; place-items: center; background: radial-gradient(circle, rgba(216,117,56,.17) 0 34%, rgba(31,88,91,.08) 35% 54%, transparent 55%); transition: .3s ease; }
|
||||||
|
.orb-core { width: 108px; height: 108px; border-radius: 50%; display: grid; place-items: center; text-align: center; padding: 12px; color: white; background: linear-gradient(145deg, #173b4d, #0d2536); box-shadow: 0 18px 34px rgba(14,43,57,.25); }
|
||||||
|
.conversation-orb.phase-active { animation: breathe 2.8s ease-in-out infinite; }
|
||||||
|
.conversation-orb.speaking { background: radial-gradient(circle, rgba(76,173,126,.25) 0 34%, rgba(31,116,91,.13) 35% 62%, transparent 63%); }
|
||||||
|
@keyframes breathe { 50% { transform: scale(1.045); } }
|
||||||
|
|
||||||
|
.meters { display: grid; grid-template-columns: 1fr 1fr; gap: 20px; }
|
||||||
|
.meter-label { display: flex; justify-content: space-between; color: #6c828b; font-size: 12px; margin-bottom: 7px; }
|
||||||
|
.meter { height: 8px; border-radius: 999px; background: #e4ebe8; overflow: hidden; }
|
||||||
|
.meter i { display: block; width: 0; height: 100%; background: linear-gradient(90deg, #d36f32, #f0ad58); transition: width .18s linear; }
|
||||||
|
.meter.agent i { background: linear-gradient(90deg, #2f8d67, #6fc591); }
|
||||||
|
.call-actions { display: flex; gap: 10px; margin: 24px 0; }
|
||||||
|
.session-info { display: grid; grid-template-columns: 1fr 1fr; gap: 10px 22px; padding-top: 18px; border-top: 1px solid #edf1ef; }
|
||||||
|
.session-info div { min-width: 0; }
|
||||||
|
.session-info dt { color: #8b9ba1; font-size: 11px; }
|
||||||
|
.session-info dd { margin: 4px 0 0; font-family: Consolas, monospace; font-size: 12px; white-space: nowrap; overflow: hidden; text-overflow: ellipsis; }
|
||||||
|
|
||||||
|
.log-panel { margin-top: 16px; padding-bottom: 18px; }
|
||||||
|
.log-panel .panel-heading { margin-bottom: 12px; }
|
||||||
|
.text-button { border: 0; color: #748991; background: transparent; }
|
||||||
|
#eventLog { list-style: none; margin: 0; padding: 0; max-height: 180px; overflow: auto; }
|
||||||
|
#eventLog li { display: grid; grid-template-columns: 82px 1fr; gap: 12px; padding: 9px 4px; border-bottom: 1px solid #edf1ef; color: #526c78; font-size: 13px; }
|
||||||
|
#eventLog time { color: #9aa8ad; font-family: Consolas, monospace; }
|
||||||
|
#eventLog li.success span { color: #247252; }
|
||||||
|
#eventLog li.error span { color: #ae3b2f; }
|
||||||
|
|
||||||
|
@media (max-width: 1050px) {
|
||||||
|
.status-grid { grid-template-columns: repeat(2, 1fr); }
|
||||||
|
.content-grid { grid-template-columns: 1fr; }
|
||||||
|
}
|
||||||
@@ -0,0 +1,13 @@
|
|||||||
|
{
|
||||||
|
"compilerOptions": {
|
||||||
|
"target": "ES2022",
|
||||||
|
"useDefineForClassFields": true,
|
||||||
|
"module": "ESNext",
|
||||||
|
"moduleResolution": "Bundler",
|
||||||
|
"strict": true,
|
||||||
|
"noEmit": true,
|
||||||
|
"lib": ["ES2022", "DOM", "DOM.Iterable"],
|
||||||
|
"skipLibCheck": true
|
||||||
|
},
|
||||||
|
"include": ["src/**/*.ts", "vite.config.ts"]
|
||||||
|
}
|
||||||
@@ -0,0 +1,10 @@
|
|||||||
|
import { defineConfig } from "vite";
|
||||||
|
|
||||||
|
export default defineConfig({
|
||||||
|
base: "./",
|
||||||
|
build: {
|
||||||
|
outDir: "dist",
|
||||||
|
emptyOutDir: true,
|
||||||
|
sourcemap: true,
|
||||||
|
},
|
||||||
|
});
|
||||||
Reference in New Issue
Block a user