wangcong/story
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

chore: checkpoint 第一版
ci / verify (push) Canceled after 0s
Details

Browse Source
This commit is contained in:
Codex Local
2026-09-06 17:04:11 +08:00
parent a8cdec98d1
commit 0c2151ace9
659 changed files with 185855 additions and 554 deletions
Download Patch File Download Diff File Expand all files Collapse all files
firmware/echoear-platform-agent/components/esp-gmf/elements/README.md
+5
View File
@@ -0,0 +1,5 @@
# ESP-GMF-Elements
- [中文版](./README_CN.md)
This is Espressif's official GMF elements component collection: [GMF Element Audio](./gmf_audio/README.md), [GMF Element Misc](./gmf_misc/README.md), GMF Element Image, [GMF IO](./gmf_io/README.md), and GMF AI Audio. Please refer to the README files of each component for detailed functionality and descriptions.
firmware/echoear-platform-agent/components/esp-gmf/elements/README_CN.md
+5
View File
@@ -0,0 +1,5 @@
# ESP-GMF-Elements
- [English](./README.md)
这是 Espressif 官方 GMF elements 组件集合：[GMF Element Audio](./gmf_audio/README_CN.md)，[GMF Element Misc](./gmf_misc/README_CN.md)，GMF Element Image, [GMF IO](./gmf_io/README_CN.md) 和 GMF AI Audio。请查看各个组件的 readme 了解具体功能和说明。
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/CHANGELOG.md
+67
View File
@@ -0,0 +1,67 @@
# Changelog
## v0.7.4
### Bug Fixes
- Fixed a crash issue that occurred when the AFE manager was destroyed.
## v0.7.3
### Features
- Added default sdkconfig for esp32p4 in wwe example
### Bug Fixes
- Fixed build error for esp32p4
## v0.7.2
### Features
- Added `esp_gmf_afe_trigger_wakeup` and `esp_gmf_afe_trigger_sleep` APIs to trigger wakeup and sleep events manually
## v0.7.1
### Features
- Increased `esp-sr` dependency to version 2.1.5
## v0.7.0
### Features
- Added `esp_gmf_afe_set_event_cb` API to register AFE element event callbacks
- Replaced the interface for decoder reconfiguration
- Introduced `esp_gmf_wn` element to support wake word detection
- Corrected return value validation for *acq_write* and *acq_release* callback function implementations
- Added `esp_gmf_afe_keep_awake` API to enable/disable the keep awake feature
- Increased `esp-sr` dependency to version 2.1.4
### Bug Fixes
- Fixed an issue where the output attribute of the `esp_gmf_afe` element was not set correctly
- Fixed a build error occurring when SPIRAM is disabled
- Integrated gmf_app_utils package for peripheral and system management
- Migrated common utilities to gmf_app_utils package
- Standardize TAG identifier format across all elements with `ai` prefix
- Fixed examples TAGs issue because of updating TAGs gmf_audio gmf_io
## v0.6.2
### Features
- Limit the version of `esp_audio_codec` and `esp_codec_dev` in `aec_rec` example
## v0.6.1
### Bug Fixes
- Fixed a build issue caused by a variable type mismatch
## v0.6.0
### Features
- Initial version of GMF AI Audio Elements
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/CMakeLists.txt
+12
View File
@@ -0,0 +1,12 @@
set(srcs "./src/esp_gmf_wn.c")
if((${IDF_TARGET} STREQUAL "esp32s3") OR (${IDF_TARGET} STREQUAL "esp32p4") OR (${IDF_TARGET} STREQUAL "esp32"))
list(APPEND srcs "./src/esp_gmf_afe_manager.c" "./src/esp_gmf_afe.c" "./src/esp_gmf_aec.c")
endif()
idf_component_register(
SRCS "${srcs}"
INCLUDE_DIRS "./include"
PRIV_INCLUDE_DIRS "./private_include"
REQUIRES esp-sr esp_timer
)
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/LICENSE
+22
View File
@@ -0,0 +1,22 @@
Espressif Modified MIT License
Copyright (c) 2025 Espressif Systems (Shanghai) CO., LTD
Permission is hereby granted for use EXCLUSIVELY with Espressif Systems products.
This includes the right to use, copy, modify, merge, publish, distribute, and sublicense
the Software, subject to the following conditions:
1. This Software MUST BE USED IN CONJUNCTION WITH ESPRESSIF SYSTEMS PRODUCTS.
2. The above copyright notice and this permission notice shall be included in all copies
or substantial portions of the Software.
3. Redistribution of the Software in source or binary form FOR USE WITH NON-ESPRESSIF PRODUCTS
is strictly prohibited.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR
PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE
FOR ANY CLAIM, DAMAGES, OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/README.md
+248
View File
@@ -0,0 +1,248 @@
# GMF AI Audio
- [![Component Registry](https://components.espressif.com/components/espressif/gmf_ai_audio/badge.svg)](https://components.espressif.com/components/espressif/gmf_ai_audio)
- [中文版](./README_CN.md)
`GMF AI Audio` is an artificial intelligence audio processing module that provides users with convenient and easy-to-use intelligent audio processing algorithms at the [GMF](https://github.com/espressif/esp-gmf) framework, such as voice wake-up, command word recognition, and echo cancellation. Currently, it offers the following modules based on `esp-sr`:
- [esp_gmf_afe_manager](./src/esp_gmf_afe_manager.c): `audio front end(afe)` manager
- [esp_gmf_aec](./src/esp_gmf_aec.c): Echo Cancellation
- [esp_gmf_wn](./src/esp_gmf_wn.c): A standalone wake word detection module that can be used independently
- [esp_gmf_afe](./src/esp_gmf_afe.c): An easy-to-use interface based on the `audio front end (afe)` from `esp-sr`, providing functionalities such as voice wake-up, command word recognition, and speech detection
| Name | Tag | Function | Method | Input Channel Number | Output Channel Number | Model Partition Dependency | Input Frame Length | Notes |
|:----:|:----:|:----:|:----:|:----:|:----:|:----:|:----:|:----|
| esp_gmf_afe | ai_afe | Audio front-end processing: Wake word detection, command word recognition, voice enhancement, echo cancellation, noise suppression, automatic gain control | `start_vcmd_det` | 1-4 | 1 | Yes | 256(sample) | Currently supports up to 2 microphone channels + 1 speaker reference signal, remaining channel selection marked as N, requires following voice command detection procedure |
| esp_gmf_aec | ai_aec | Echo cancellation: Eliminates echo interference in audio, improves voice quality | None | 1-4 | 1 | No | 256(sample) | Input channels can be set to multiple microphones, uses first microphone channel and reference channel for calculation, must include reference signal |
| esp_gmf_wn | ai_wn | Independent wake word detection: Lightweight wake word detection, independent of AFE, low resource consumption | None | 1-4 | 1 | Yes | 256(sample) | Supports up to 3 microphone channels, microphone channel count in input format must match working mode |
## AFE Manager `esp_gmf_afe_manager`
### Features
- Manages the data path and task scheduling of the Audio Front-End (AFE)
- Supports dynamic enabling/disabling of features (e.g., wake-up, AEC, VAD)
- Provides multi-task coordination to ensure real-time audio stream processing
### Key Characteristics
- Task Management: Creates independent `feed_task` and `fetch_task` for audio data input and result processing
- Feature Control: Dynamically toggle algorithm modules via `esp_afe_manager_enable_features`
- Event-Driven: Supports suspend/resume operations (`esp_afe_manager_suspend`) for low-power scenarios
### System Diagram
```mermaid
graph LR
subgraph AFE Manager
DATA_CB[Raw Data]
FEED_TASK[feed_task<br>Audio Input Task]
FETCH_TASK[fetch_task<br>Result Processing Task]
AFE_CORE[AFE Core<br>esp_afe_sr_iface]
EVENT_CB[Result Callback]
CTRL_IF[Control Interface]
end
FEED_TASK -->|Data Callback| DATA_CB
FEED_TASK -->|Raw Audio Data| AFE_CORE
AFE_CORE -->|Processing Results| FETCH_TASK
FETCH_TASK -->|Result Callback<br>Data/Wake-Up/VAD/Command Words| EVENT_CB
CTRL_IF -->|Suspend/Resume Interface| FEED_TASK
CTRL_IF -->|Suspend/Resume Interface| FETCH_TASK
CTRL_IF -->|Feature Toggle Interface<br>AEC/VAD/SE| AFE_CORE
```
## Echo Cancellation `esp_gmf_aec`
### Features
- Eliminates echo interference in audio to improve voice quality
- Supports multi-channel input and single-channel output
### Technical Specifications
- Supported Hardware: `ESP32`, `ESP32-S3`, and `ESP32-P4`
- Input Format
- Sampling Rate: 8 kHz and 16 kHz
- Bit Width: 16-bit PCM
- Channel Configuration: String identifiers (e.g., `MMNR`)
- `M`: Microphone Channel
- `R`: Reference Signal Channel
- `N`: Invalid Signal
- Output Format: 16-bit single-channel PCM
## Wake Word Detection `esp_gmf_wn`
### Features
- Runs independently without AFE dependency, low resource usage
- Supports multi-channel input and single-channel output
- Notifies wake word detection results via callback function
### Technical Specifications
- Supported Hardware: `ESP32`, `ESP32-S3`, `ESP32-C3`, `ESP32-C5` and `ESP32-P4`
- Input Format
- Sampling Rate: 16 kHz
- Bit Width: 16-bit PCM
- Channel Configuration: String identifiers (e.g., `MMNR`)
- `M`: Microphone Channel
- `R`: Reference Signal Channel
- `N`: Invalid Signal
- Output Format: 16-bit single-channel PCM
## Speech Recognition `esp_gmf_afe`
### Features
- Voice Wake-Up (Wake Word Detection)
- Command Word Recognition
- Voice Activity Detection (VAD) and State Management
### Key Characteristics
- Supported Hardware: `ESP32`, `ESP32-S3`, and `ESP32-P4`
- Input and Output: Same as described in `esp_gmf_aec`
- Unified Event Interface: Notifies events (wake-up start/end, VAD state changes, command word triggers) via callback functions
- State Machine Management: Automatically maintains transitions between wake-up, speech, and silence states
### System Diagram
```mermaid
graph TD
subgraph Audio Preprocessing
RAW[Audio Data]
end
subgraph GMF_AFE
AFE_MGR[AFE Manager<br>esp_gmf_afe_manager]
INPUT_RB[Input Buffer]
OUTPUT_RB[Output Buffer]
STATE_MACHINE[Wake-Up/VAD State Machine]
MN_DET[Command Word Detection]
DELAY[Delay Mechanism]
end
subgraph Application Layer
EVENT_CB[Event Callback]
OUT_PORT[Data Output]
end
RAW -->|Processed Audio Data| INPUT_RB
INPUT_RB -->|Data Callback| AFE_MGR
AFE_MGR -->|Result Callback<br>Data| OUTPUT_RB
AFE_MGR -->|Result Callback<br>Wake-Up/VAD Events| STATE_MACHINE
AFE_MGR -->|Result Callback<br>Data| MN_DET
OUTPUT_RB --> DELAY --> OUT_PORT
STATE_MACHINE -->|Event Notification| EVENT_CB
MN_DET -->|Event Notification| EVENT_CB
```
### `esp_gmf_afe_cfg_t` Advanced Configuration Parameters
| Parameter | Description | Default Value |
| --- | --- | --- |
| **wakeup_time** | Time in milliseconds to trigger wake-up end event if no voice activity is detected after enabling wake-up | 10000 (ms) |
| **wakeup_end** | Time in milliseconds to trigger wake-up end event after detecting silence following voice activity | 2000 (ms) |
| **vcmd_timeout** | Timeout for command word detection. After timeout, the interface must be called again to start a new detection round | 5760 (ms) |
| **delay_samples** | Output data delay to compensate for VAD detection lag. The specified data length (in samples) should be greater than or equal to `afe_config_t.vad_min_speech_ms` configured during AFE manager initialization | 2048 (samples) |
### Wake Word and VAD State Machine
The following illustrates state transitions when features are enabled. The `/` character indicates the triggered user event
- **Enable Wake Word Detection, Disable VAD**
This scenario focuses on detecting wake words and can be configured (`wakeup_time`) to trigger an end event after a certain period
Modify the configuration in the example [wwe](./examples/wwe/main/main.c) to use this scenario
```c
#define WAKENET_ENABLE (true)
#define VAD_ENABLE (false)
```
```mermaid
stateDiagram-v2
[*] --> IDLE
IDLE --> WAKEUP: Wake Word / (WAKEUP_START)
WAKEUP --> IDLE: Timer (wakeup_time) Timeout / (WAKEUP_END)
```
- **Enable VAD, Disable Wake Word Detection**
This scenario focuses on detecting voice activity and triggers corresponding events at the start and end of speech
Modify the configuration in the example [wwe](./examples/wwe/main/main.c) to use this scenario
```c
#define WAKENET_ENABLE (false)
#define VAD_ENABLE (true)
```
```mermaid
stateDiagram-v2
[*] --> IDLE
IDLE --> SPEECHING: Voice Detected / (VAD_START)
SPEECHING --> IDLE: Noise or Silence Detected / (VAD_END)
```
- **Enable Both Wake Word Detection and VAD**
This scenario combines wake word detection with voice activity detection to avoid frequent voice activity events outside the wake word interval
Modify the configuration in the example [wwe](./examples/wwe/main/main.c) to use this scenario
```c
#define WAKENET_ENABLE (true)
#define VAD_ENABLE (true)
```
```mermaid
stateDiagram-v2
[*] --> IDLE
IDLE --> WAKEUP: Wake Word / (WAKEUP_START)
WAKEUP --> SPEECHING: Voice Detected / (VAD_START)
WAKEUP --> IDLE: Timer (wakeup_time) Timeout / (WAKEUP_END)
SPEECHING --> WAIT_FOR_SLEEP: Noise or Silence Detected / (VAD_END)
WAIT_FOR_SLEEP --> SPEECHING: Voice Detected / (VAD_START)
WAIT_FOR_SLEEP --> IDLE: Timer (wakeup_end) Timeout / (WAKEUP_END)
```
### Command Word Detection
Users need to decide when to start command word detection. A typical use case is to enable detection after the state machine pushes a (WAKEUP_START) event and determine the next operation based on the detected command word index in the callback function
- Command word detection is independent of the wake word state machine
- Command word detection supports continuous detection until timeout
## Usage
For example code, refer to the [examples](./examples) folder for the Wake Word Detection and Audio Echo Cancellation demos.
You can also create and compile a project using the following commands, taking the `aec_rec` project as an example. Before starting, make sure you have a working [ESP-IDF](https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/get-started/index.html) environment.
### 1. Create the Example Project
Create the `aec_rec` example project based on the `gmf_ai_audio` component (using version v0.7.0 as an example; update the version as needed):
```shell
idf.py create-project-from-example "espressif/gmf_ai_audio=0.7.0:aec_rec"
```
### 2. Build and Flash the Project Using an ESP32-S3 Board
```shell
cd aec_rec
idf.py set-target esp32s3
idf.py -p YOUR_PORT flash monitor
```
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/README_CN.md
+248
View File
@@ -0,0 +1,248 @@
# GMF AI Audio
- [![Component Registry](https://components.espressif.com/components/espressif/gmf_ai_audio/badge.svg)](https://components.espressif.com/components/espressif/gmf_ai_audio)
- [English](./README.md)
`GMF AI Audio` 是一个人工智能语音处理模块，它在 [GMF](https://github.com/espressif/esp-gmf) 层面为用户提供方便、易用的语音唤醒、命令词识别和回声消除等常用智能语音处理算法。目前基于 `esp-sr` 提供以下模块：
- [esp_gmf_afe_manager](./src/esp_gmf_afe_manager.c): `audio front end(afe)` 管理器
- [esp_gmf_aec](./src/esp_gmf_aec.c): 回声消除
- [esp_gmf_wn](./src/esp_gmf_wn.c): 独立的唤醒词检测模块，不依赖AFE
- [esp_gmf_afe](./src/esp_gmf_afe.c): 基于 `esp-sr` 中 `audio front end (afe)` 实现的易用接口，提供了语音唤醒，命令词识别，人声检测等语音识别的功能
| 名称 | 标签 | 功能 | 函数方法 | 输入通道 | 输出通道 | 依赖模型数据分区 | 输入帧长 | 备注 |
|:----:|:----:|:----:|:----:|:----:|:----:|:----:|:----:|:----|
| esp_gmf_afe | ai_afe | 语音前处理：唤醒词检测、命令词识别、人声增强、回声消除、降噪、自动增益控制 | `start_vcmd_det` | 1-4 | 1 | 是 | 256(sample) | 目前最多支持2通道麦克风+1个扬声器的回采信号，剩余通道选择记为N，同时需要按照命令词检测步骤操作 |
| esp_gmf_aec | ai_aec | 回声消除：消除音频中的回声干扰，提升语音质量 | 无 | 1-4 | 1 | 否 | 256(sample) | 输入通道可以设置多个麦克风，采用第一个麦克风通道和回采通道进行计算，且必须包含回采信号 |
| esp_gmf_wn | ai_wn | 独立唤醒词检测：轻量级唤醒词检测，不依赖AFE，资源占用小 | 无 | 1-4 | 1 | 是 | 256(sample) | 最高支持3通道麦克风，输入格式中的麦克风通道数需要与工作模式对应 |
## AFE 管理器 `esp_gmf_afe_manager`
### 功能
- 管理音频前端（Audio Front-End, AFE）的数据通路与任务调度
- 支持动态启用/禁用特性（如唤醒、AEC、VAD）
- 提供多任务协同机制，确保实时音频流处理
### 关键特性
- 任务管理：创建独立的 `feed_task` 和 `fetch_task`，分别负责音频数据输入和结果处理
- 特性控制：通过 `esp_afe_manager_enable_features` 动态开关算法模块
- 事件驱动：支持挂起/恢复操作 `esp_afe_manager_suspend`，适配低功耗场景
### 系统框图
```mermaid
graph LR
subgraph AFE管理器
DATA_CB[原始数据]
FEED_TASK[feed_task<br>音频输入任务]
FETCH_TASK[fetch_task<br>结果处理任务]
AFE_CORE[AFE核心<br>esp_afe_sr_iface]
EVENT_CB[结果回调]
CTRL_IF[控制接口]
end
FEED_TASK -->|数据回调| DATA_CB
FEED_TASK -->|原始音频数据| AFE_CORE
AFE_CORE -->|处理结果| FETCH_TASK
FETCH_TASK -->|结果回调<br>（数据/唤醒/VAD/命令词）| EVENT_CB
CTRL_IF -->|挂起/恢复接口| FEED_TASK
CTRL_IF -->|挂起/恢复接口| FETCH_TASK
CTRL_IF -->|特性使能接口<br>（AEC/VAD/SE）| AFE_CORE
```
## 回声消除 `esp_gmf_aec`
### 功能
- 消除音频中的回声干扰，提升语音质量
- 支持多通道输入，单通道输出
### 技术规格
- 支持硬件：`ESP32`、`ESP32-S3` 和 `ESP32-P4`
- 输入格式
- 采样率：16 kHz
- 位宽：16-bit PCM
- 通道配置：字符串标识（如 `MMNR`）
- `M`：麦克风通道
- `R`：回采信号通道
- `N`：无效信号
- 输出格式：16-bit 单通道 PCM
## 唤醒词检测 `esp_gmf_wn`
### 功能
- 独立运行，不依赖AFE，资源占用小
- 支持多通道输入，单通道输出
- 通过回调函数通知唤醒词检测结果
### 技术规格
- 支持硬件：`ESP32`、`ESP32-S3`、`ESP32-C3`、`ESP32-C5` 和 `ESP32-P4`
- 输入格式
- 采样率：8 kHz, 16 kHz
- 位宽：16-bit PCM
- 通道配置：字符串标识（如 `MMNR`）
- `M`：麦克风通道
- `R`：回采信号通道
- `N`：无效信号
- 输出格式：16-bit 单通道 PCM
## 语音识别 `esp_gmf_afe`
### 功能
- 语音唤醒（Wake Word Detection）
- 命令词识别（Voice Command Detection）
- 人声检测（VAD）与状态管理
### 关键特性
- 支持硬件: `esp32`, `esp32s3` 和 `esp32p4`
- 输入与输出同上文中的 `esp_gmf_aec`
- 统一事件接口：通过回调函数通知事件（唤醒开始/结束、VAD 状态变化、命令词触发）
- 状态机管理：自动维护唤醒、语音、静音等状态切换
### 系统框图
```mermaid
graph TD
subgraph 音频前级处理
RAW[音频数据]
end
subgraph GMF_AFE
AFE_MGR[AFE管理器<br>esp_gmf_afe_manager]
INPUT_RB[输入缓存]
OUTPUT_RB[输出缓存]
STATE_MACHINE[唤醒/VAD状态机]
MN_DET[命令词检测]
DELAY[延时机制]
end
subgraph 应用层
EVENT_CB[事件回调]
OUT_PORT[数据出口]
end
RAW -->| 音频前级处理后的数据| INPUT_RB
INPUT_RB -->| 数据回调 | AFE_MGR
AFE_MGR -->| 结果回调<br>数据 | OUTPUT_RB
AFE_MGR -->| 结果回调<br>唤醒/VAD事件 | STATE_MACHINE
AFE_MGR -->| 结果回调<br>数据 | MN_DET
OUTPUT_RB --> DELAY --> OUT_PORT
STATE_MACHINE -->| 事件通知 | EVENT_CB
MN_DET -->| 事件通知 | EVENT_CB
```
### `esp_gmf_afe_cfg_t` 高级配置参数说明
| 参数 | 描述 | 默认值 |
| --- | --- | --- |
| **wakeup_time** | 唤醒功能启用后，若未检测到人声事件，则在等待 `wakeup_time` 毫秒后触发唤醒结束事件 | 10000（ms） |
| **wakeup_end** | 唤醒功能启用后，若依次检测到人声和静音，则在静音事件触发后等待 `wakeup_end` 毫秒，触发唤醒结束事件 | 2000（ms） |
| **vcmd_timeout** | 命令词检测的超时时间，超时后需重新调用接口以开启新一轮命令词检测 | 5760（ms） |
| **delay_samples** | 输出数据延迟，用于补偿 VAD 检测滞后导致的数据输出与事件回调不同步的问题。该参数指定的数据长度（以采样点为单位）换算为时间后，应大于等于初始化 AFE 管理器时配置的 `afe_config_t.vad_min_speech_ms` | 2048（samples） |
### 唤醒与VAD状态机
以下列举了功能使能情况下的状态切换，`/` 字符之后为触发的用户事件
- **使能唤醒，不使能VAD**
该场景主要关注唤醒词的检测，同时可以配置（wakeup_time），在一段时间后触发结束事件
修改示例 [wwe](./examples/wwe/main/main.c) 中的配置，从而使用该场景
```c
#define WAKENET_ENABLE (true)
#define VAD_ENABLE (false)
```
```mermaid
stateDiagram-v2
[*] --> IDLE
IDLE --> WAKEUP: 唤醒词 / （WAKEUP_START）
WAKEUP --> IDLE: 定时器（wakeup_time）超时 / （WAKEUP_END）
```
- **使VAD，不使能唤醒词**
该场景主要关注人声的检测，在检测到人说话的前后节点触发相应事件
修改示例 [wwe](./examples/wwe/main/main.c) 中的配置，从而使用该场景
```c
#define WAKENET_ENABLE (false)
#define VAD_ENABLE (true)
```
```mermaid
stateDiagram-v2
[*] --> IDLE
IDLE --> SPEECHING: 检测到人声 / （VAD_START）
SPEECHING --> IDLE: 检测到噪声或静音 / （VAD_END）
```
- **使能唤醒和VAD**
该场景结合唤醒词与人声检测，避免在唤醒间隔之外频繁触发人声检测事件
修改示例 [wwe](./examples/wwe/main/main.c) 中的配置，从而使用该场景
```c
#define WAKENET_ENABLE (true)
#define VAD_ENABLE (true)
```
```mermaid
stateDiagram-v2
[*] --> IDLE
IDLE --> WAKEUP: 唤醒词 / （WAKEUP_START）
WAKEUP --> SPEECHING: 检测到人声 / （VAD_START）
WAKEUP --> IDLE: 定时器（wakeup_time）超时 / （WAKEUP_END）
SPEECHING --> WAIT_FOR_SLEEP: 检测到噪声或静音 / （VAD_END）
WAIT_FOR_SLEEP --> SPEECHING: 检测到人声 / （VAD_START）
WAIT_FOR_SLEEP --> IDLE: 定时器（wakeup_end）超时 / （WAKEUP_END）
```
### 命令词检测
用户需决定何时开始检测命令词，如典型的使用场景：在状态机推送（WAKEUP_START）事件之后，开启检测，并在回调函数中根据检测到的命令词序号决定下一步操作
- 命令词检测与唤醒状态机独立
- 命令词支持连续检测，直到超时
## 示例
示例代码，请参阅 [示例](./examples) 文件夹中的唤醒词检测和音频回声消除程序。
此外，也可通过以下命令创建并编译工程，以 `aec_rec` 示例工程为例。开始之前需要有可运行的 [ESP-IDF](https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/get-started/index.html) 环境。
### 1. 创建示例工程
基于 `gmf_ai_audio` 组件创建 `aec_rec` 的示例（以 v0.7.0 版本为例， 请根据实际使用更新版本参数）
``` shell
idf.py create-project-from-example "espressif/gmf_ai_audio=0.7.0:aec_rec"
```
### 2. 基于 ESP32S3 编译和下载
```shell
cd aec_rec
idf.py set-target esp32s3`
idf.py -p YOUR_PORT flash monitor
```
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/examples/aec_rec/CMakeLists.txt
+7
View File
@@ -0,0 +1,7 @@
# The following lines of boilerplate have to be in your project's
# CMakeLists in this exact order for cmake to work correctly
cmake_minimum_required(VERSION 3.5)
include($ENV{IDF_PATH}/tools/cmake/project.cmake)
project(aec_rec)
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/examples/aec_rec/README.md
+244
View File
@@ -0,0 +1,244 @@
# Play Music from SDCard While Recording Audio with AEC
- [中文版](./README_CN.md)
## Example Brief
This example uses two pipelines to demonstrate the audio recording effect of AEC. One pipeline reads MP3 files stored on the SD card using File IO, then decodes the data through a decoder element. After decoding, the data undergoes audio effect processing and is output as music through the CODEC_DEV_TX IO. Simultaneously, another pipeline uses CODEC_DEV_RX IO to read PCM data from the I2S device, processes it with AEC, encodes the audio, and finally saves it as a file using File IO
This example supports compressing the audio processed by AEC into the `AAC` format, which can be enabled through the macro `ENCODER_ENABLE` in [main.c](./main/main.c):
```c
#define ENCODER_ENABLE (false)
```
- `ENCODER_ENABLE`: Enabling this option will activate the `AAC` encoder in the recording pipeline and save the encoded data as `aec.aac`
## GMF Pipeline
Below is the diagram of the pipelines used in this example:
```mermaid
graph TD
subgraph Path1 [MP3 Player]
A[File IO] --> B[Decoder: MP3] --> C[Audio Effects] --> D[CODEC_DEV_TX]
end
subgraph Path2 [AEC Recorder]
E[CODEC_DEV_RX] --> F[Rate_cvt] --> G[AEC] --> H[Encoder: AAC] --> I[File IO]
end
```
## AEC Element Initialization Example
```c
esp_gmf_element_handle_t gmf_aec_handle = NULL;
esp_gmf_aec_cfg_t gmf_aec_cfg = {
.filter_len = 4,
.type = AFE_TYPE_VC,
.mode = AFE_MODE_HIGH_PERF,
.input_format = "RMNM",
};
esp_gmf_aec_init(&gmf_aec_cfg, &gmf_aec_handle);
```
### Parameter Explanation
- `filter_len`: Filter length. Higher values increase CPU load. For `esp32s3` and `esp32p4`, a filter length of 4 is recommended. For `esp32c5`, a filter length of 2 is recommended
- `type`: AEC type, with the following options:
- `AFE_TYPE_VC`: Suitable for voice communication echo cancellation
- `AFE_TYPE_SR`: Suitable for voice recognition echo cancellation
- `mode`: AEC mode, with the following options:
- `AFE_MODE_LOW_POWER`: Low-power mode, suitable for power-sensitive scenarios
- `AFE_MODE_HIGH_PERF`: High-performance mode, suitable for scenarios requiring high-quality echo cancellation
- `input_format`: Input data format, such as `RMNM`:
- `M`: Microphone channel
- `R`: Reference signal channel
- `N`: Invalid signal
By configuring these parameters, you can adjust the performance and resource usage of AEC according to specific application scenarios. Once added to the `GMF Pool`, the AEC element can be used in the `GMF Pipeline` to process audio streams
## Example Set Up
### Default IDF Branch
This example supports IDF release/v5.3 and later branches
### Build and Flash
Before compiling this example, ensure that the ESP-IDF environment is properly configured. If it is already set up, you can proceed to the next configuration step. If not, run the following script in the root directory of ESP-IDF to set up the build environment. For detailed steps on configuring and using ESP-IDF, please refer to the [ESP-IDF Programming Guide](https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/index.html)
```
./install.sh
. ./export.sh
```
Here are the summarized steps for compilation:
- Enter the location where the test project is stored:
```
cd gmf_ai_audio/examples/aec_rec
```
- Select the target chip for compilation. For example, to use the ESP32S3:
```
idf.py set-target esp32s3
```
- Select the compilation board, taking ESP32-S3-Korvo V2 as an example:
```
idf.py menuconfig
In 'menuconfig', select 'GMF APP Configuration' -> 'Audio Board' -> 'ESP32-S3-Korvo V2', and then save and exit
```
- Build the example:
```
idf.py build
```
- Flash the program and run the monitor tool to view serial output (replace PORT with the port name):
```
idf.py -p PORT flash monitor
```
- Exit the debugging interface using ``Ctrl-]``
## How to Use the Example
- This example requires an SDCard. Prepare an MP3 file and store it on the SDCard
- Before compiling the program, modify the value of `ENCODER_ENABLE` in `main.c` to enable or disable the `AAC` encoder
- Before running the example, ensure the SDCard is correctly installed on the development board
- During the example's execution, you can speak to the development board to verify the AEC effect later
- After the example finishes running, you can export the files from the SDCard and use software to check the recording content:
- If encoding is disabled, the recording file will be named `aec_16k_16bit_1ch.pcm` with a format of 16K sampling rate, 16-bit width, and 1 channel
- If encoding is enabled, the recording file will be named `aec.aac`
### Example Functionality
- Once the example starts running, it will read MP3 audio from the SDCard and play it while recording and saving to the SDCard. After 20 seconds, it will stop and exit, printing the following:
```c
I (926) gpio: GPIO[15]| InputEn: 0| OutputEn: 0| OpenDrain: 0| Pullup: 1| Pulldown: 0| Intr:0
I (935) gpio: GPIO[7]| InputEn: 0| OutputEn: 0| OpenDrain: 0| Pullup: 1| Pulldown: 0| Intr:0
I (944) gpio: GPIO[4]| InputEn: 0| OutputEn: 0| OpenDrain: 0| Pullup: 1| Pulldown: 0| Intr:0
I (995) gpio: GPIO[17]| InputEn: 1| OutputEn: 1| OpenDrain: 1| Pullup: 1| Pulldown: 0| Intr:0
I (996) gpio: GPIO[18]| InputEn: 1| OutputEn: 1| OpenDrain: 1| Pullup: 1| Pulldown: 0| Intr:0
I (1010) ES8311: Work in Slave mode
I (1013) gpio: GPIO[48]| InputEn: 0| OutputEn: 1| OpenDrain: 0| Pullup: 0| Pulldown: 0| Intr:0
E (1017) i2s_common: i2s_channel_disable(1116): the channel has not been enabled yet
I (1025) I2S_IF: channel mode 0 bits:32/32 channel:2 mask:3
I (1031) I2S_IF: STD Mode 1 bits:32/32 channel:2 sample_rate:48000 mask:3
I (1053) Adev_Codec: Open codec device OK
I (1057) ES7210: Work in Slave mode
I (1063) ES7210: Enable ES7210_INPUT_MIC1
I (1066) ES7210: Enable ES7210_INPUT_MIC2
I (1069) ES7210: Enable ES7210_INPUT_MIC3
I (1072) ES7210: Enable TDM mode
E (1077) i2s_common: i2s_channel_disable(1116): the channel has not been enabled yet
I (1078) I2S_IF: channel mode 0 bits:32/32 channel:2 mask:3
I (1085) I2S_IF: STD Mode 0 bits:32/32 channel:2 sample_rate:48000 mask:3
I (1093) ES7210: Bits 16
I (1102) ES7210: Enable ES7210_INPUT_MIC1
I (1105) ES7210: Enable ES7210_INPUT_MIC2
I (1108) ES7210: Enable ES7210_INPUT_MIC3
I (1112) ES7210: Enable TDM mode
I (1118) ES7210: Unmuted
I (1118) Adev_Codec: Open codec device OK
I (1123) ESP_GMF_POOL: Registered items on pool:0x3c23ddf8, app_main-142
I (1129) ESP_GMF_POOL: IO, Item:0x3c23decc, H:0x3c23de0c, TAG:io_http
I (1135) ESP_GMF_POOL: IO, Item:0x3c23df9c, H:0x3c23dedc, TAG:io_http
I (1142) ESP_GMF_POOL: IO, Item:0x3c23e038, H:0x3c23dfac, TAG:io_file
I (1149) ESP_GMF_POOL: IO, Item:0x3c23e0d4, H:0x3c23e048, TAG:io_file
I (1156) ESP_GMF_POOL: IO, Item:0x3c23e204, H:0x3c23e170, TAG:io_embed_flash
I (1163) ESP_GMF_POOL: IO, Item:0x3c23ede0, H:0x3c23ed50, TAG:io_codec_dev
I (1171) ESP_GMF_POOL: IO, Item:0x3c23ee80, H:0x3c23edf0, TAG:io_codec_dev
I (1178) ESP_GMF_POOL: EL, Item:0x3c23e160, H:0x3c23e0e4, TAG:copier
I (1185) ESP_GMF_POOL: EL, Item:0x3c23e2e8, H:0x3c23e214, TAG:aud_enc
I (1193) ESP_GMF_POOL: EL, Item:0x3c23e3e8, H:0x3c23e2f8, TAG:aud_dec
I (1200) ESP_GMF_POOL: EL, Item:0x3c23e4bc, H:0x3c23e3f8, TAG:aud_alc
I (1207) ESP_GMF_POOL: EL, Item:0x3c23e570, H:0x3c23e4cc, TAG:aud_eq
I (1214) ESP_GMF_POOL: EL, Item:0x3c23e5a4, H:0x3c23e6d8, TAG:aud_ch_cvt
I (1221) ESP_GMF_POOL: EL, Item:0x3c23e5d4, H:0x3c23e774, TAG:aud_bit_cvt
I (1228) ESP_GMF_POOL: EL, Item:0x3c23e8ac, H:0x3c23e810, TAG:aud_rate_cvt
I (1235) ESP_GMF_POOL: EL, Item:0x3c23e988, H:0x3c23e8bc, TAG:aud_fade
I (1242) ESP_GMF_POOL: EL, Item:0x3c23ea84, H:0x3c23e998, TAG:aud_sonic
I (1249) ESP_GMF_POOL: EL, Item:0x3c23eb64, H:0x3c23ea94, TAG:aud_deintlv
I (1256) ESP_GMF_POOL: EL, Item:0x3c23ec40, H:0x3c23eb74, TAG:aud_intlv
I (1264) ESP_GMF_POOL: EL, Item:0x3c23ed40, H:0x3c23ec50, TAG:aud_mixer
I (1271) ESP_GMF_POOL: EL, Item:0x3c23ef60, H:0x3c23ee90, TAG:ai_aec
W (1278) ESP_GMF_PIPELINE: There is no thread for add jobs, pipe:0x3c23ef70, tsk:0x0, [el:aud_rate_cvt-0x3c23efa8]
I (1288) ESP_GMF_THREAD: The TSK_0x3fcb5404 created on internal memory
I (1288) ESP_GMF_TASK: Waiting to run... [tsk:TSK_0x3fcb5404-0x3fcb5404, wk:0x0, run:0]
I (1304) AEC_EL_2_FILE: CB: RECV Pipeline EVT: el:NULL-0x3c23ef70, type:8192, sub:ESP_GMF_EVENT_STATE_OPENING, payload:0x0, size:0,0x0
I (1328) ESP_GMF_PORT: ACQ IN, new self payload:0x3c23f35c, port:0x3c23f2a8, el:0x3c23efa8-aud_rate_cvt
I (1329) ESP_GMF_PORT: ACQ OUT SET, new self payload:0x3c23fe68, p:0x3c23f1a8, el:0x3c23efa8-aud_rate_cvt
I (1358) GMF_AEC: GMF AEC open, frame_len: 2048, nch 4, chunksize 256
I (1359) AEC_EL_2_FILE: CB: RECV Pipeline EVT: el:ai_aec-0x3c23f0c4, type:12288, sub:ESP_GMF_EVENT_STATE_INITIALIZED, payload:0x3fcb6790, size:12,0x0
I (1371) AEC_EL_2_FILE: CB: RECV Pipeline EVT: el:ai_aec-0x3c23f0c4, type:8192, sub:ESP_GMF_EVENT_STATE_RUNNING, payload:0x0, size:0,0x0
I (1383) ESP_GMF_TASK: One times job is complete, del[wk:0x3c23f3dc,ctx:0x3c23f0c4, label:aec_open]
I (1401) ESP_GMF_TASK: Waiting to run... [tsk:TSK_0x3fcca2f8-0x3fcca2f8, wk:0x0, run:0]
I (1401) ESP_GMF_THREAD: The TSK_0x3fcca2f8 created on internal memory
I (1432) ESP_GMF_TASK: Waiting to run... [tsk:TSK_0x3fcca2f8-0x3fcca2f8, wk:0x3c252ac0, run:0]
I (1433) ESP_GMF_FILE: Open, dir:1, uri:/sdcard/test.mp3
I (1446) ESP_GMF_FILE: File size: 2994349 byte, file position: 0
I (1447) AEC_EL_2_FILE: CB: RECV Pipeline EVT: el:NULL-0x3c25230c, type:8192
I (1460) ESP_GMF_TASK: One times job is complete, del[wk:0x3c252ac0,ctx:0x3c252344, label:aud_simp_dec_open]
I (1469) ESP_GMF_PORT: ACQ IN, new self payload:0x3c252ac0, port:0x3c252964, el:0x3c252344-aud_dec
I (1482) ESP_GMF_PORT: ACQ OUT SET, new self payload:0x3c252d64, p:0x3c25252c, el:0x3c252344-aud_dec
W (1491) ESP_GMF_ASMP_DEC: Not enough memory for out, need:4608, old: 1024, new: 4608
I (1505) ESP_GMF_ASMP_DEC: NOTIFY Info, rate: 0, bits: 0, ch: 0 --> rate: 44100, bits: 16, ch: 2
Audio >
I (1589) ESP_GMF_TASK: One times job is complete, del[wk:0x3c253fa4,ctx:0x3c252420, label:rate_cvt_open]
I (1590) ESP_GMF_PORT: ACQ OUT SET, new self payload:0x3c253fa4, p:0x3c2526c4, el:0x3c252420-aud_rate_cvt
I (1603) ESP_GMF_TASK: One times job is complete, del[wk:0x3c253fec,ctx:0x3c2525ac, label:ch_cvt_open]
I (1615) AEC_EL_2_FILE: CB: RECV Pipeline EVT: el:aud_bit_cvt-0x3c252744, type:12288, sub:ESP_GMF_EVENT_STATE_INITIALIZED, payload:0x3fccb680, size:12,0x0
I (1627) AEC_EL_2_FILE: CB: RECV Pipeline EVT: el:aud_bit_cvt-0x3c252744, type:8192, sub:ESP_GMF_EVENT_STATE_RUNNING, payload:0x0, size:0,0x0
I (1639) ESP_GMF_TASK: One times job is complete, del[wk:0x3c255608,ctx:0x3c252744, label:bit_cvt_open]
I (1651) ESP_GMF_PORT: ACQ OUT, new self payload:0x3c255608, port:0x3c252a34, el:0x3c252744-aud_bit_cvt
I (21462) ESP_GMF_CODEC_DEV: CLose, 0x3c23f228, pos = 7716864/0
I (21464) ESP_GMF_TASK: One times job is complete, del[wk:0x3c23f410,ctx:0x3c23efa8, label:rate_cvt_close]
I (21478) ESP_GMF_TASK: One times job is complete, del[wk:0x3c253fec,ctx:0x3c23f0c4, label:aec_close]
I (21479) AEC_EL_2_FILE: CB: RECV Pipeline EVT: el:NULL-0x3c23ef70, type:8192, sub:ESP_GMF_EVENT_STATE_STOPPED, payload:0x0, size:0,0x0
I (21501) ESP_GMF_TASK: Waiting to run... [tsk:TSK_0x3fcb5404-0x3fcb5404, wk:0x0, run:0]
I (21502) ESP_GMF_TASK: Waiting to run... [tsk:TSK_0x3fcb5404-0x3fcb5404, wk:0x0, run:0]
I (21514) ESP_GMF_FILE: CLose, 0x3c2528d8, pos = 318464/2994349
I (21525) ESP_GMF_CODEC_DEV: CLose, 0x3c2529a4, pos = 7633608/0
I (21526) ESP_GMF_TASK: One times job is complete, del[wk:0x3c25562c,ctx:0x3c252344, label:aud_simp_dec_close]
I (21538) ESP_GMF_TASK: One times job is complete, del[wk:0x3c23f410,ctx:0x3c252420, label:rate_cvt_close]
I (21549) ESP_GMF_TASK: One times job is complete, del[wk:0x3c23f380,ctx:0x3c2525ac, label:ch_cvt_close]
I (21561) ESP_GMF_TASK: One times job is complete, del[wk:0x3c23f3b8,ctx:0x3c252744, label:bit_cvt_close]
I (21572) AEC_EL_2_FILE: CB: RECV Pipeline EVT: el:NULL-0x3c25230c, type:8192, sub:ESP_GMF_EVENT_STATE_STOPPED, payload:0x0, size:0,0x0
I (21584) ESP_GMF_TASK: Waiting to run... [tsk:TSK_0x3fcca2f8-0x3fcca2f8, wk:0x0, run:0]
I (21595) ESP_GMF_TASK: Waiting to run... [tsk:TSK_0x3fcca2f8-0x3fcca2f8, wk:0x0, run:0]
E (25298) i2s_common: i2s_channel_disable(1116): the channel has not been enabled yet
E (25303) i2s_common: i2s_channel_disable(1116): the channel has not been enabled yet
I (25314) main_task: Returned from app_main()
```
## Troubleshooting
### 1. SDCard Not Recognized
- Ensure the SDCard is formatted as FAT32
- Check if the SDCard is properly inserted into the development board
### 2. MP3 File Not Playing
- Ensure the MP3 file is stored in the root directory of the SDCard
- Verify that the MP3 file is not corrupted
- Ensure the MP3 file is named `test.mp3`
### 3. No Audio Recorded
- Verify that the microphone hardware on the development board is functioning correctly
- Ensure the audio channel configuration matches the hardware design
### 4. High CPU Usage with AEC in the Pipeline
Since the AEC algorithm requires significant CPU resources, consider the following steps if issues arise:
- Identify CPU-intensive elements in the pipeline, such as `ai_aec`, `aud_enc`, or `io_file`
- Split the problematic pipeline into two separate pipelines, run them on different CPU cores, and connect them using `gmf port`
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/examples/aec_rec/README_CN.md
+244
View File
@@ -0,0 +1,244 @@
# 播放 SDCard 音乐同时 AEC 录制音频
- [English](./README.md)
## 例程简介
本例程使用两路 pipeline 演示 AEC 录制音频的效果。一路 pipeline 使用 File IO 读取存储在 SDCard 中的 MP3 文件，然后经过 decoder 元素解码，解码后数据进行音频效果处理后用 CODEC_DEV_TX IO 输出音乐。同时，另外一路 pipeline 使用 CODEC_DEV_RX IO 从 I2S 设备读取 PCM 数据，经过 AEC 处理后进行音频编码，最终通过 File IO 保存为文件
本例程支持将 AEC 处理后的音频压缩为 `AAC` 格式，通过 [main.c](./main/main.c) 中的宏 `ENCODER_ENABLE` 使能
```c
#define ENCODER_ENABLE (false)
```
- `ENCODER_ENABLE`: 使能该选项将会在录音的 pipeline 中使能 `AAC` 编码器，并保存编码后的数据为 `aec.aac`
## GMF Pipeline
以下是本例程中使用的 pipeline 的示意图：
```mermaid
graph TD
subgraph Path1 [MP3 Player]
A[File IO] --> B[Decoder: MP3] --> C[Audio Effects] --> D[CODEC_DEV_TX]
end
subgraph Path2 [AEC Recorder]
E[CODEC_DEV_RX] --> F[Rate_cvt] --> G[AEC] --> H[Encoder: AAC] --> I[File IO]
end
```
## AEC 元素初始化示例
```c
esp_gmf_element_handle_t gmf_aec_handle = NULL;
esp_gmf_aec_cfg_t gmf_aec_cfg = {
.filter_len = 4,
.type = AFE_TYPE_VC,
.mode = AFE_MODE_HIGH_PERF,
.input_format = "RMNM",
};
esp_gmf_aec_init(&gmf_aec_cfg, &gmf_aec_handle);
```
#### 参数说明
- `filter_len`: 滤波器长度。数值越大，CPU 负载越高。对于 esp32s3 和 esp32p4，建议 filter_length = 4。对于 esp32c5，建议 filter_length = 2
- `type`: AEC 类型，支持以下选项：
- `AFE_TYPE_VC`: 适用于语音通信的回声消除
- `AFE_TYPE_SR`: 适用于语音识别的回声消除
- `mode`: AEC 模式，支持以下选项：
- `AFE_MODE_LOW_POWER`: 低功耗模式，适用于功耗敏感的场景
- `AFE_MODE_HIGH_PERF`: 高性能模式，适用于需要高质量回声消除的场景
- `input_format`: 输入数据的格式, 如 `RMNM`：
- `M`：麦克风通道
- `R`：回采信号通道
- `N`：无效信号
通过配置这些参数，可以根据具体应用场景调整 AEC 的性能和资源使用。添加到 `GMF Pool` 中后，可在 `GMF Pipeline` 中使用 AEC 处理音频流
## 示例创建
### IDF 默认分支
本例程支持 IDF release/v5.3 及以后的分支
### 编译和下载
编译本例程前需要先确保已配置 ESP-IDF 的环境，如果已配置可跳到下一项配置，如果未配置需要先在 ESP-IDF 根目录运行下面脚本设置编译环境，有关配置和使用 ESP-IDF 完整步骤，请参阅 [《ESP-IDF 编程指南》](https://docs.espressif.com/projects/esp-idf/zh_CN/latest/esp32s3/index.html)
```
./install.sh
. ./export.sh
```
下面是简略编译步骤：
- 进入示例工程的位置
```
cd gmf_ai_audio/examples/aec_rec
```
- 选择编译芯片，以 esp32s3 为例：
```
idf.py set-target esp32s3
```
- 选择编译目标板，以 ESP32-S3-Korvo V2 为例：
```
idf.py menuconfig
在 `menuconfig` 中选择 `GMF APP Configuration` -> `Audio Board` -> `ESP32-S3-Korvo V2`，然后保存退出
```
- 编译例子程序
```
idf.py build
```
- 烧录程序并运行 monitor 工具来查看串口输出 (替换 PORT 为端口名称)：
```
idf.py -p PORT flash monitor
```
- 退出调试界面使用 ``Ctrl-]``
## 如何使用例程
- 本例程需要 SDCard 和 MP3 文件，用户需准备一份 MP3 文件并存入 SDCard 中
- 编译程序前，可以通过 `main.c` 中的 `ENCODER_ENABLE` 的值来决定是否启用 `AAC` 编码
- 运行例程前，请确保 SDCard 已正确安装到开发板上
- 运行例程时，用户可以对开发板说话，以便后续验证 AEC 效果
- 运行结束后，用户可以将 SDCard 中的文件导出，用软件查看录音内容：
- 如果未启用编码，录音文件名为 `aec_16k_16bit_1ch.pcm`，格式为 16K 采样率，16 bit 位宽，单声道
- 如果启用编码，录音文件名为 `aec.aac`
### 功能和用法
- 例程运行后，将从 SDCard 中读取 MP3 音频进行播放，同时录音并保存到 SDCard。运行 20 秒后停止并退出，串口打印如下日志：
```c
I (926) gpio: GPIO[15]| InputEn: 0| OutputEn: 0| OpenDrain: 0| Pullup: 1| Pulldown: 0| Intr:0
I (935) gpio: GPIO[7]| InputEn: 0| OutputEn: 0| OpenDrain: 0| Pullup: 1| Pulldown: 0| Intr:0
I (944) gpio: GPIO[4]| InputEn: 0| OutputEn: 0| OpenDrain: 0| Pullup: 1| Pulldown: 0| Intr:0
I (995) gpio: GPIO[17]| InputEn: 1| OutputEn: 1| OpenDrain: 1| Pullup: 1| Pulldown: 0| Intr:0
I (996) gpio: GPIO[18]| InputEn: 1| OutputEn: 1| OpenDrain: 1| Pullup: 1| Pulldown: 0| Intr:0
I (1010) ES8311: Work in Slave mode
I (1013) gpio: GPIO[48]| InputEn: 0| OutputEn: 1| OpenDrain: 0| Pullup: 0| Pulldown: 0| Intr:0
E (1017) i2s_common: i2s_channel_disable(1116): the channel has not been enabled yet
I (1025) I2S_IF: channel mode 0 bits:32/32 channel:2 mask:3
I (1031) I2S_IF: STD Mode 1 bits:32/32 channel:2 sample_rate:48000 mask:3
I (1053) Adev_Codec: Open codec device OK
I (1057) ES7210: Work in Slave mode
I (1063) ES7210: Enable ES7210_INPUT_MIC1
I (1066) ES7210: Enable ES7210_INPUT_MIC2
I (1069) ES7210: Enable ES7210_INPUT_MIC3
I (1072) ES7210: Enable TDM mode
E (1077) i2s_common: i2s_channel_disable(1116): the channel has not been enabled yet
I (1078) I2S_IF: channel mode 0 bits:32/32 channel:2 mask:3
I (1085) I2S_IF: STD Mode 0 bits:32/32 channel:2 sample_rate:48000 mask:3
I (1093) ES7210: Bits 16
I (1102) ES7210: Enable ES7210_INPUT_MIC1
I (1105) ES7210: Enable ES7210_INPUT_MIC2
I (1108) ES7210: Enable ES7210_INPUT_MIC3
I (1112) ES7210: Enable TDM mode
I (1118) ES7210: Unmuted
I (1118) Adev_Codec: Open codec device OK
I (1123) ESP_GMF_POOL: Registered items on pool:0x3c23ddf8, app_main-142
I (1129) ESP_GMF_POOL: IO, Item:0x3c23decc, H:0x3c23de0c, TAG:io_http
I (1135) ESP_GMF_POOL: IO, Item:0x3c23df9c, H:0x3c23dedc, TAG:io_http
I (1142) ESP_GMF_POOL: IO, Item:0x3c23e038, H:0x3c23dfac, TAG:io_file
I (1149) ESP_GMF_POOL: IO, Item:0x3c23e0d4, H:0x3c23e048, TAG:io_file
I (1156) ESP_GMF_POOL: IO, Item:0x3c23e204, H:0x3c23e170, TAG:io_embed_flash
I (1163) ESP_GMF_POOL: IO, Item:0x3c23ede0, H:0x3c23ed50, TAG:io_codec_dev
I (1171) ESP_GMF_POOL: IO, Item:0x3c23ee80, H:0x3c23edf0, TAG:io_codec_dev
I (1178) ESP_GMF_POOL: EL, Item:0x3c23e160, H:0x3c23e0e4, TAG:copier
I (1185) ESP_GMF_POOL: EL, Item:0x3c23e2e8, H:0x3c23e214, TAG:aud_enc
I (1193) ESP_GMF_POOL: EL, Item:0x3c23e3e8, H:0x3c23e2f8, TAG:aud_dec
I (1200) ESP_GMF_POOL: EL, Item:0x3c23e4bc, H:0x3c23e3f8, TAG:aud_alc
I (1207) ESP_GMF_POOL: EL, Item:0x3c23e570, H:0x3c23e4cc, TAG:aud_eq
I (1214) ESP_GMF_POOL: EL, Item:0x3c23e5a4, H:0x3c23e6d8, TAG:aud_ch_cvt
I (1221) ESP_GMF_POOL: EL, Item:0x3c23e5d4, H:0x3c23e774, TAG:aud_bit_cvt
I (1228) ESP_GMF_POOL: EL, Item:0x3c23e8ac, H:0x3c23e810, TAG:aud_rate_cvt
I (1235) ESP_GMF_POOL: EL, Item:0x3c23e988, H:0x3c23e8bc, TAG:aud_fade
I (1242) ESP_GMF_POOL: EL, Item:0x3c23ea84, H:0x3c23e998, TAG:aud_sonic
I (1249) ESP_GMF_POOL: EL, Item:0x3c23eb64, H:0x3c23ea94, TAG:aud_deintlv
I (1256) ESP_GMF_POOL: EL, Item:0x3c23ec40, H:0x3c23eb74, TAG:aud_intlv
I (1264) ESP_GMF_POOL: EL, Item:0x3c23ed40, H:0x3c23ec50, TAG:aud_mixer
I (1271) ESP_GMF_POOL: EL, Item:0x3c23ef60, H:0x3c23ee90, TAG:ai_aec
W (1278) ESP_GMF_PIPELINE: There is no thread for add jobs, pipe:0x3c23ef70, tsk:0x0, [el:aud_rate_cvt-0x3c23efa8]
I (1288) ESP_GMF_THREAD: The TSK_0x3fcb5404 created on internal memory
I (1288) ESP_GMF_TASK: Waiting to run... [tsk:TSK_0x3fcb5404-0x3fcb5404, wk:0x0, run:0]
I (1304) AEC_EL_2_FILE: CB: RECV Pipeline EVT: el:NULL-0x3c23ef70, type:8192, sub:ESP_GMF_EVENT_STATE_OPENING, payload:0x0, size:0,0x0
I (1328) ESP_GMF_PORT: ACQ IN, new self payload:0x3c23f35c, port:0x3c23f2a8, el:0x3c23efa8-aud_rate_cvt
I (1329) ESP_GMF_PORT: ACQ OUT SET, new self payload:0x3c23fe68, p:0x3c23f1a8, el:0x3c23efa8-aud_rate_cvt
I (1358) GMF_AEC: GMF AEC open, frame_len: 2048, nch 4, chunksize 256
I (1359) AEC_EL_2_FILE: CB: RECV Pipeline EVT: el:ai_aec-0x3c23f0c4, type:12288, sub:ESP_GMF_EVENT_STATE_INITIALIZED, payload:0x3fcb6790, size:12,0x0
I (1371) AEC_EL_2_FILE: CB: RECV Pipeline EVT: el:ai_aec-0x3c23f0c4, type:8192, sub:ESP_GMF_EVENT_STATE_RUNNING, payload:0x0, size:0,0x0
I (1383) ESP_GMF_TASK: One times job is complete, del[wk:0x3c23f3dc,ctx:0x3c23f0c4, label:aec_open]
I (1401) ESP_GMF_TASK: Waiting to run... [tsk:TSK_0x3fcca2f8-0x3fcca2f8, wk:0x0, run:0]
I (1401) ESP_GMF_THREAD: The TSK_0x3fcca2f8 created on internal memory
I (1432) ESP_GMF_TASK: Waiting to run... [tsk:TSK_0x3fcca2f8-0x3fcca2f8, wk:0x3c252ac0, run:0]
I (1433) ESP_GMF_FILE: Open, dir:1, uri:/sdcard/test.mp3
I (1446) ESP_GMF_FILE: File size: 2994349 byte, io_file position: 0
I (1447) AEC_EL_2_FILE: CB: RECV Pipeline EVT: el:NULL-0x3c25230c, type:8192
I (1460) ESP_GMF_TASK: One times job is complete, del[wk:0x3c252ac0,ctx:0x3c252344, label:aud_simp_dec_open]
I (1469) ESP_GMF_PORT: ACQ IN, new self payload:0x3c252ac0, port:0x3c252964, el:0x3c252344-aud_dec
I (1482) ESP_GMF_PORT: ACQ OUT SET, new self payload:0x3c252d64, p:0x3c25252c, el:0x3c252344-aud_dec
W (1491) ESP_GMF_ASMP_DEC: Not enough memory for out, need:4608, old: 1024, new: 4608
I (1505) ESP_GMF_ASMP_DEC: NOTIFY Info, rate: 0, bits: 0, ch: 0 --> rate: 44100, bits: 16, ch: 2
Audio >
I (1589) ESP_GMF_TASK: One times job is complete, del[wk:0x3c253fa4,ctx:0x3c252420, label:rate_cvt_open]
I (1590) ESP_GMF_PORT: ACQ OUT SET, new self payload:0x3c253fa4, p:0x3c2526c4, el:0x3c252420-aud_rate_cvt
I (1603) ESP_GMF_TASK: One times job is complete, del[wk:0x3c253fec,ctx:0x3c2525ac, label:ch_cvt_open]
I (1615) AEC_EL_2_FILE: CB: RECV Pipeline EVT: el:aud_bit_cvt-0x3c252744, type:12288, sub:ESP_GMF_EVENT_STATE_INITIALIZED, payload:0x3fccb680, size:12,0x0
I (1627) AEC_EL_2_FILE: CB: RECV Pipeline EVT: el:aud_bit_cvt-0x3c252744, type:8192, sub:ESP_GMF_EVENT_STATE_RUNNING, payload:0x0, size:0,0x0
I (1639) ESP_GMF_TASK: One times job is complete, del[wk:0x3c255608,ctx:0x3c252744, label:bit_cvt_open]
I (1651) ESP_GMF_PORT: ACQ OUT, new self payload:0x3c255608, port:0x3c252a34, el:0x3c252744-aud_bit_cvt
I (21462) ESP_GMF_CODEC_DEV: CLose, 0x3c23f228, pos = 7716864/0
I (21464) ESP_GMF_TASK: One times job is complete, del[wk:0x3c23f410,ctx:0x3c23efa8, label:rate_cvt_close]
I (21478) ESP_GMF_TASK: One times job is complete, del[wk:0x3c253fec,ctx:0x3c23f0c4, label:aec_close]
I (21479) AEC_EL_2_FILE: CB: RECV Pipeline EVT: el:NULL-0x3c23ef70, type:8192, sub:ESP_GMF_EVENT_STATE_STOPPED, payload:0x0, size:0,0x0
I (21501) ESP_GMF_TASK: Waiting to run... [tsk:TSK_0x3fcb5404-0x3fcb5404, wk:0x0, run:0]
I (21502) ESP_GMF_TASK: Waiting to run... [tsk:TSK_0x3fcb5404-0x3fcb5404, wk:0x0, run:0]
I (21514) ESP_GMF_FILE: CLose, 0x3c2528d8, pos = 318464/2994349
I (21525) ESP_GMF_CODEC_DEV: CLose, 0x3c2529a4, pos = 7633608/0
I (21526) ESP_GMF_TASK: One times job is complete, del[wk:0x3c25562c,ctx:0x3c252344, label:aud_simp_dec_close]
I (21538) ESP_GMF_TASK: One times job is complete, del[wk:0x3c23f410,ctx:0x3c252420, label:rate_cvt_close]
I (21549) ESP_GMF_TASK: One times job is complete, del[wk:0x3c23f380,ctx:0x3c2525ac, label:ch_cvt_close]
I (21561) ESP_GMF_TASK: One times job is complete, del[wk:0x3c23f3b8,ctx:0x3c252744, label:bit_cvt_close]
I (21572) AEC_EL_2_FILE: CB: RECV Pipeline EVT: el:NULL-0x3c25230c, type:8192, sub:ESP_GMF_EVENT_STATE_STOPPED, payload:0x0, size:0,0x0
I (21584) ESP_GMF_TASK: Waiting to run... [tsk:TSK_0x3fcca2f8-0x3fcca2f8, wk:0x0, run:0]
I (21595) ESP_GMF_TASK: Waiting to run... [tsk:TSK_0x3fcca2f8-0x3fcca2f8, wk:0x0, run:0]
E (25298) i2s_common: i2s_channel_disable(1116): the channel has not been enabled yet
E (25303) i2s_common: i2s_channel_disable(1116): the channel has not been enabled yet
I (25314) main_task: Returned from app_main()
```
## 常见问题
### 1. 无法识别 SDCard
- 确保 SDCard 已正确格式化为 FAT32 文件系统
- 检查 SDCard 是否正确插入开发板
### 2. 无法播放 MP3 文件
- 确保 MP3 文件存储在 SDCard 的根目录中
- 确保 MP3 文件未损坏
- 确保 MP3 文件名为 `test.mp3`
### 3. 录制音频无声音
- 确保开发板的麦克风硬件正常工作
- 确保语音通道配置符合硬件设计
### 4. 集成 AEC 的 Pipeline CPU 占用过高
由于 AEC 算法需要的 cpu 算力较多，所以组建包含 AEC 元素的 Pipeline 时，需合理分配资源，如遇到问题，可按如下方式处理：
- 查看 Pipeline 有哪些需要 cpu 算力较多的元素，如 `ai_aec`, `aud_enc`, `io_file` 等
- 将出现问题的 Pipeline 拆分为两个，分别运行到不同的 cpu 核心上，使用 `gmf port` 连接
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/examples/aec_rec/main/CMakeLists.txt
+2
View File
@@ -0,0 +1,2 @@
idf_component_register(SRCS "main.c"
INCLUDE_DIRS ".")
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/examples/aec_rec/main/idf_component.yml
+19
View File
@@ -0,0 +1,19 @@
dependencies:
espressif/gmf_loader:
override_path: ../../../../../packages/gmf_loader
version: "^0.7"
espressif/gmf_app_utils:
override_path: ../../../../../packages/gmf_app_utils
version: "^0.7"
espressif/gmf_core:
override_path: ../../../../../gmf_core
espressif/gmf_io:
override_path: ../../../../gmf_io
espressif/gmf_audio:
override_path: ../../../../gmf_audio
espressif/gmf_ai_audio:
override_path: ../../../../gmf_ai_audio
espressif/gmf_video:
override_path: ../../../../gmf_video
espressif/gmf_misc:
override_path: ../../../../gmf_misc
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/examples/aec_rec/main/main.c
+219
View File
@@ -0,0 +1,219 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
*
* SPDX-License-Identifier: Apache-2.0
*/
#include <stdio.h>
#include <string.h>
#include "esp_err.h"
#include "esp_log.h"
#include "esp_gmf_err.h"
#include "esp_gmf_io_codec_dev.h"
#include "esp_gmf_obj.h"
#include "esp_gmf_payload.h"
#include "esp_gmf_port.h"
#include "esp_gmf_aec.h"
#include "esp_gmf_element.h"
#include "esp_gmf_pipeline.h"
#include "esp_gmf_pool.h"
#include "esp_gmf_rate_cvt.h"
#include "esp_gmf_bit_cvt.h"
#include "esp_gmf_ch_cvt.h"
#include "esp_gmf_audio_dec.h"
#include "esp_gmf_app_setup_peripheral.h"
#include "esp_gmf_app_cli.h"
#include "gmf_loader_setup_defaults.h"
#define BOARD_LYRAT_MINI (0)
#define BOARD_KORVO_2 (1)
#if defined CONFIG_IDF_TARGET_ESP32S3
#define AUDIO_BOARD (BOARD_KORVO_2)
#elif defined CONFIG_IDF_TARGET_ESP32
#define AUDIO_BOARD (BOARD_LYRAT_MINI)
#endif /* defined CONFIG_IDF_TARGET_ESP32S3 */
#if AUDIO_BOARD == BOARD_KORVO_2
#define ADC_I2S_PORT (0)
#define ADC_I2S_CH (2)
#define ADC_I2S_BITS (32)
#define DAC_I2S_PORT (0)
#define DAC_I2S_CH (2)
#define DAC_I2S_BITS (32)
#define INPUT_CH_NUM (4)
#define INPUT_CH_BITS (16) /* For board `ESP32-S3-Korvo-2`, the es7210 is configured as 32-bit,
2-channel mode to accommodate 16-bit, 4-channel data */
#define INPUT_CH_ALLOCATION ("RMNM")
#elif AUDIO_BOARD == BOARD_LYRAT_MINI
#define ADC_I2S_PORT (1)
#define ADC_I2S_CH (2)
#define ADC_I2S_BITS (16)
#define DAC_I2S_PORT (0)
#define DAC_I2S_CH (1)
#define DAC_I2S_BITS (16)
#define INPUT_CH_NUM (ADC_I2S_CH)
#define INPUT_CH_BITS (ADC_I2S_BITS)
#define INPUT_CH_ALLOCATION ("RM")
#endif /* AUDIO_BOARD == BOARD_KORVO_2 */
#define ENCODER_ENABLE (false)
static const char *TAG = "AEC_EL_2_FILE";
static uint8_t *pcm_buffer = NULL;
const uint32_t buf_size = 600 * 1024;
static size_t pcm_received = 0;
static esp_err_t _pipeline_event(esp_gmf_event_pkt_t *event, void *ctx)
{
ESP_LOGI(TAG, "CB: RECV Pipeline EVT: el:%s-%p, type:%d, sub:%s, payload:%p, size:%d,%p",
OBJ_GET_TAG(event->from), event->from, event->type, esp_gmf_event_get_state_str(event->sub),
event->payload, event->payload_size, ctx);
return ESP_OK;
}
static esp_gmf_err_io_t pcm_buf_acq_write(void *handle, esp_gmf_payload_t *load, int wanted_size, int block_ticks)
{
return ESP_GMF_IO_OK;
}
static esp_gmf_err_io_t pcm_buf_release_write(void *handle, esp_gmf_payload_t *load, int block_ticks)
{
if (load == NULL) {
return ESP_GMF_IO_FAIL;
}
if (load->valid_size && (pcm_received + load->valid_size <= buf_size)) {
memcpy(pcm_buffer + pcm_received, load->buf, load->valid_size);
pcm_received += load->valid_size;
}
return ESP_GMF_IO_OK;
}
void app_main(void)
{
esp_log_level_set("*", ESP_LOG_INFO);
pcm_buffer = heap_caps_malloc(buf_size, MALLOC_CAP_SPIRAM);
if (!pcm_buffer) {
ESP_LOGE(TAG, "Failed to allocate SPIRAM buffer");
return;
}
pcm_received = 0;
esp_gmf_app_codec_info_t codec_info = ESP_GMF_APP_CODEC_INFO_DEFAULT();
codec_info.play_info.sample_rate = 48000;
codec_info.play_info.channel = DAC_I2S_CH;
codec_info.play_info.bits_per_sample = DAC_I2S_BITS;
codec_info.record_info.sample_rate = codec_info.play_info.sample_rate;
codec_info.record_info.channel = ADC_I2S_CH;
codec_info.record_info.bits_per_sample = ADC_I2S_BITS;
esp_gmf_app_setup_codec_dev(&codec_info);
void *sdcard_handle = NULL;
esp_gmf_app_setup_sdcard(&sdcard_handle);
esp_gmf_pool_handle_t pool = NULL;
esp_gmf_pool_init(&pool);
gmf_loader_setup_all_defaults(pool);
ESP_GMF_POOL_SHOW_ITEMS(pool);
esp_gmf_pipeline_handle_t read_pipe = NULL;
#if ENCODER_ENABLE
const char *name[] = {"aud_rate_cvt", "ai_aec", "aud_enc"};
#else
const char *name[] = {"aud_rate_cvt", "ai_aec"};
#endif /* ENCODER_ENABLE */
esp_gmf_pool_new_pipeline(pool, "io_codec_dev", name, sizeof(name) / sizeof(char *), NULL, &read_pipe);
if (read_pipe == NULL) {
ESP_LOGE(TAG, "There is no pipeline");
return;
}
esp_gmf_io_codec_dev_set_dev(ESP_GMF_PIPELINE_GET_IN_INSTANCE(read_pipe), esp_gmf_app_get_record_handle());
esp_gmf_port_handle_t out_port = NEW_ESP_GMF_PORT_OUT_BYTE(pcm_buf_acq_write, pcm_buf_release_write, NULL, NULL, 1024, portMAX_DELAY);
esp_gmf_element_register_out_port(read_pipe->last_el, out_port);
esp_gmf_obj_handle_t rate_cvt = NULL;
esp_gmf_pipeline_get_el_by_name(read_pipe, "aud_rate_cvt", &rate_cvt);
esp_gmf_rate_cvt_set_dest_rate(rate_cvt, 16000);
esp_gmf_info_sound_t info = {
.format_id = ESP_AUDIO_SIMPLE_DEC_TYPE_MP3,
.sample_rates = 48000,
.channels = INPUT_CH_NUM,
.bits = INPUT_CH_BITS,
};
esp_gmf_pipeline_report_info(read_pipe, ESP_GMF_INFO_SOUND, &info, sizeof(info));
esp_gmf_task_cfg_t cfg = DEFAULT_ESP_GMF_TASK_CONFIG();
cfg.thread.core = 1;
cfg.thread.stack = 5120;
esp_gmf_task_handle_t read_task = NULL;
esp_gmf_task_init(&cfg, &read_task);
esp_gmf_pipeline_bind_task(read_pipe, read_task);
esp_gmf_pipeline_loading_jobs(read_pipe);
esp_gmf_pipeline_set_event(read_pipe, _pipeline_event, NULL);
esp_gmf_pipeline_run(read_pipe);
esp_gmf_app_cli_init("Audio >", NULL);
// New pipeline to play 'test.mp3'
esp_gmf_pipeline_handle_t play_pipe = NULL;
const char *play_name[] = {"aud_dec", "aud_rate_cvt", "aud_ch_cvt", "aud_bit_cvt"};
esp_gmf_pool_new_pipeline(pool, "io_file", play_name, sizeof(play_name) / sizeof(char *), "io_codec_dev", &play_pipe);
if (play_pipe == NULL) {
ESP_LOGE(TAG, "There is no play pipeline");
return;
}
esp_gmf_io_codec_dev_set_dev(ESP_GMF_PIPELINE_GET_OUT_INSTANCE(play_pipe), esp_gmf_app_get_playback_handle());
esp_gmf_obj_handle_t bit_cvt = NULL;
esp_gmf_pipeline_get_el_by_name(play_pipe, "aud_bit_cvt", &bit_cvt);
esp_gmf_bit_cvt_set_dest_bits(bit_cvt, DAC_I2S_BITS);
esp_gmf_obj_handle_t ch_cvt = NULL;
esp_gmf_pipeline_get_el_by_name(play_pipe, "aud_ch_cvt", &ch_cvt);
esp_gmf_ch_cvt_set_dest_channel(ch_cvt, DAC_I2S_CH);
esp_gmf_obj_handle_t dec_el = NULL;
esp_gmf_pipeline_get_el_by_name(play_pipe, "aud_dec", &dec_el);
esp_gmf_audio_dec_reconfig_by_sound_info(dec_el, &info);
esp_gmf_pipeline_set_in_uri(play_pipe, "/sdcard/test.mp3");
esp_gmf_task_handle_t play_task = NULL;
cfg.thread.core = 0;
esp_gmf_task_init(&cfg, &play_task);
esp_gmf_pipeline_bind_task(play_pipe, play_task);
esp_gmf_pipeline_loading_jobs(play_pipe);
esp_gmf_pipeline_set_event(play_pipe, _pipeline_event, NULL);
esp_gmf_pipeline_run(play_pipe);
vTaskDelay(20000 / portTICK_PERIOD_MS);
esp_gmf_pipeline_stop(read_pipe);
esp_gmf_pipeline_stop(play_pipe);
#if ENCODER_ENABLE
FILE *file = fopen("/sdcard/aec.aac", "wb");
#else
FILE *file = fopen("/sdcard/aec_16k_16bit_1ch.pcm", "wb");
#endif /* ENCODER_ENABLE */
if (file) {
fwrite(pcm_buffer, 1, pcm_received, file);
fclose(file);
} else {
ESP_LOGE(TAG, "Failed to open file for writing");
}
heap_caps_free(pcm_buffer);
esp_gmf_task_deinit(read_task);
esp_gmf_task_deinit(play_task);
esp_gmf_pipeline_destroy(read_pipe);
esp_gmf_pipeline_destroy(play_pipe);
gmf_loader_teardown_all_defaults(pool);
esp_gmf_pool_deinit(pool);
esp_gmf_app_teardown_codec_dev();
esp_gmf_app_teardown_sdcard(sdcard_handle);
}
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/examples/aec_rec/partitions.csv
+4
View File
@@ -0,0 +1,4 @@
# Espressif ESP32 Partition Table
# Name, Type, SubType, Offset, Size
nvs, data, nvs, 0x9000, 0x6000,
factory, app, factory, 0x010000, 2500k
1 # Espressif ESP32 Partition Table
2 # Name, Type, SubType, Offset, Size
3 nvs, data, nvs, 0x9000, 0x6000,
4 factory, app, factory, 0x010000, 2500k
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/examples/aec_rec/pytest_aec_rec.py
+14
View File
@@ -0,0 +1,14 @@
# SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
#
# SPDX-License-Identifier: Apache-2.0
import pytest
import os
from pytest_embedded import Dut
@pytest.mark.esp32
@pytest.mark.esp32s3
@pytest.mark.temp_skip_ci(targets=['esp32', 'esp32s3', 'esp32p4'], reason='No running in CI')
def test_aec_rec(dut: Dut)-> None:
dut.expect(r'Audio >', timeout=20)
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/examples/aec_rec/sdkconfig.defaults
+52
View File
@@ -0,0 +1,52 @@
#
# Serial flasher config
#
CONFIG_ESPTOOLPY_FLASH_MODE_AUTO_DETECT=y
CONFIG_ESPTOOLPY_FLASHMODE_DIO=y
CONFIG_ESPTOOLPY_FLASH_SAMPLE_MODE_STR=y
CONFIG_ESPTOOLPY_FLASHMODE="dio"
CONFIG_ESPTOOLPY_FLASHFREQ_80M=y
CONFIG_ESPTOOLPY_FLASHFREQ="80m"
CONFIG_ESPTOOLPY_FLASHSIZE_8MB=y
CONFIG_ESPTOOLPY_FLASHSIZE="8MB"
#
# Partition Table
#
CONFIG_PARTITION_TABLE_CUSTOM=y
CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="partitions.csv"
CONFIG_PARTITION_TABLE_FILENAME="partitions.csv"
CONFIG_PARTITION_TABLE_OFFSET=0x8000
CONFIG_PARTITION_TABLE_MD5=y
# end of Partition Table
#
# Compiler options
#
CONFIG_COMPILER_OPTIMIZATION_PERF=y
# end of Compiler options
#
# ESP PSRAM
#
CONFIG_SPIRAM=y
CONFIG_SPIRAM_TYPE_AUTO=y
CONFIG_SPIRAM_ALLOW_STACK_EXTERNAL_MEMORY=y
CONFIG_SPIRAM_SPEED_80M=y
CONFIG_SPIRAM_SPEED=80
#
# Audio Codec Device Configuration
#
# CONFIG_CODEC_I2C_BACKWARD_COMPATIBLE is not set
# end of Audio Codec Device Configuration
CONFIG_FATFS_LFN_HEAP=y
CONFIG_FREERTOS_HZ=1000
CONFIG_FREERTOS_USE_TRACE_FACILITY=y
CONFIG_FREERTOS_USE_STATS_FORMATTING_FUNCTIONS=y
# CONFIG_FREERTOS_USE_LIST_DATA_INTEGRITY_CHECK_BYTES is not set
CONFIG_FREERTOS_VTASKLIST_INCLUDE_COREID=y
CONFIG_FREERTOS_GENERATE_RUN_TIME_STATS=y
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/examples/aec_rec/sdkconfig.defaults.esp32
+15
View File
@@ -0,0 +1,15 @@
CONFIG_IDF_CMAKE=y
CONFIG_IDF_TARGET_ARCH_XTENSA=y
CONFIG_IDF_TARGET="esp32"
CONFIG_IDF_TARGET_ESP32=y
CONFIG_IDF_FIRMWARE_CHIP_ID=0x0000
#
# ESP System Settings
#
# CONFIG_ESP_DEFAULT_CPU_FREQ_MHZ_80 is not set
# CONFIG_ESP_DEFAULT_CPU_FREQ_MHZ_160 is not set
CONFIG_ESP_DEFAULT_CPU_FREQ_MHZ_240=y
CONFIG_ESP_DEFAULT_CPU_FREQ_MHZ=240
CONFIG_AUDIO_BOARD="LYRAT_MINI_V1"
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/examples/aec_rec/sdkconfig.defaults.esp32s3
+15
View File
@@ -0,0 +1,15 @@
CONFIG_IDF_CMAKE=y
CONFIG_IDF_TARGET_ARCH_XTENSA=y
CONFIG_IDF_TARGET="esp32s3"
CONFIG_IDF_TARGET_ESP32S3=y
CONFIG_IDF_FIRMWARE_CHIP_ID=0x0009
CONFIG_COMPILER_OPTIMIZATION_PERF=y
CONFIG_SPIRAM_MODE_OCT=y
CONFIG_ESP32S3_DEFAULT_CPU_FREQ_240=y
CONFIG_ESP32S3_INSTRUCTION_CACHE_32KB=y
CONFIG_ESP32S3_DATA_CACHE_LINE_64B=y
CONFIG_AUDIO_BOARD="S3_Korvo_V2"
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/examples/wwe/CMakeLists.txt
+7
View File
@@ -0,0 +1,7 @@
# The following lines of boilerplate have to be in your project's
# CMakeLists in this exact order for cmake to work correctly
cmake_minimum_required(VERSION 3.5)
include($ENV{IDF_PATH}/tools/cmake/project.cmake)
project(gmf_wwe_demo)
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/examples/wwe/README.md
+150
View File
@@ -0,0 +1,150 @@
# Speech Recognition
- [中文版](./README_CN.md)
## Example Brief
This example demonstrates how to use the `AFE element` for wake word detection, voice activity detection (VAD), and command word recognition. It allows configuring features through the following macro definitions in [main.c](./main/main.c) to achieve different application combinations:
```c
#define VOICE2FILE (true)
#define WAKENET_ENABLE (true)
#define VAD_ENABLE (true)
#define VCMD_ENABLE (true)
```
- `VOICE2FILE`: Save the audio between VAD start and VAD end as a file
- `WAKENET_ENABLE`: Enable wake word recognition
- `VAD_ENABLE`: Enable voice activity detection
- `VCMD_ENABLE`: Enable command word recognition
## GMF Pipeline
```mermaid
graph
subgraph WWE
E[CODEC_DEV_RX] --> F[GMF_AFE] --> G[GMF_PORT]
end
```
## Example Set Up
### Default IDF Branch
This example supports IDF release/v5.3 and later branches
### Prerequisites
The voice wake word and command word detection used in this example are derived from `esp-sr`. Please familiarize yourself with its configuration and usage: [README](https://github.com/espressif/esp-sr/blob/master/README.md)
### Build and Flash
Before compiling this example, ensure that the ESP-IDF environment is properly configured. If it is already set up, you can proceed to the next configuration step. If not, run the following script in the root directory of ESP-IDF to set up the build environment. For detailed steps on configuring and using ESP-IDF, please refer to the [ESP-IDF Programming Guide](https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/index.html)
```
./install.sh
. ./export.sh
```
Here are the summarized steps for compilation:
- Enter the location where the example project is stored:
```
cd gmf_ai_audio/examples/wwe
```
- Select the target chip for compilation. For example, to use the ESP32S3:
```
idf.py set-target esp32s3
```
- Select the compilation board, taking ESP32-S3-Korvo V2 as an example:
```
idf.py menuconfig
In 'menuconfig', select 'GMF APP Configuration' -> 'Audio Board' -> 'ESP32-S3-Korvo V2', and then save and exit
```
- Build the example:
```
idf.py build
```
- Flash the program and run the monitor tool to view serial output (replace PORT with the port name):
```
idf.py -p PORT flash monitor
```
- Exit the debugging interface using `Ctrl-]`
## How to use the Example
- Before compiling the program, confirm whether wake word detection is enabled by checking the value of `WAKENET_ENABLE` in `main.c`. Also, confirm whether the `afe` output needs to be saved as a file by checking the value of `VOICE2FILE`
- If `VOICE2FILE` is set to `true`, ensure that the SDCard is correctly installed on the development board before running the example
- During the example's execution, if `WAKENET_ENABLE` is set to `true`, first say the wake word, then say the command word to verify the effects of wake word detection, voice activity detection, and command word detection. If `WAKENET_ENABLE` is set to `false`, you can directly use command words. Pay attention to the corresponding event prompts in the `log`
- If `VOICE2FILE` is set to `true`, after the example finishes running, you can export the files from the SDCard and use software to check the recording content. The recording file names are `16k_16bit_1ch_{idx}.pcm`, where `{idx}` increments. The PCM format is 16K sampling rate, 16-bit width, and 1 channel
### Example Functionality
- After the example starts running, you can say wake words and command words to the development board. The following will be printed:
```c
I (1547) AFE: AFE Version: (2MIC_V250113)
I (1550) AFE: Input PCM Config: total 4 channels(2 microphone, 1 playback), sample rate:16000
I (1560) AFE: AFE Pipeline: [input] -> |AEC(SR_HIGH_PERF)| -> |SE(BSS)| -> |VAD(WebRTC)| -> |WakeNet(wn9_hilexin,)| -> [output]
I (1572) AFE_manager: Feed task, ch 4, chunk 1024, buf size 8192
I (1579) GMF_AFE: Create AFE, ai_afe-0x3c2dcf90
I (1584) GMF_AFE: Create AFE, ai_afe-0x3c2dd0b8
I (1589) GMF_AFE: New an object,ai_afe-0x3c2dd0b8
I (1595) ESP_GMF_TASK: Waiting to run... [tsk:TSK_0x3fcc500c-0x3fcc500c, wk:0x0, run:0]
I (1603) ESP_GMF_THREAD: The TSK_0x3fcc500c created on internal memory
I (1610) ESP_GMF_TASK: Waiting to run... [tsk:TSK_0x3fcc500c-0x3fcc500c, wk:0x3c2dd17c, run:0]
I (1620) AI_AUDIO_WWE: CB: RECV Pipeline EVT: el:NULL-0x3c2dd080, type:8192, sub:ESP_GMF_EVENT_STATE_OPENING, payload:0x0, size:0,0x0
I (1632) AFE_manager: AFE Ctrl [1, 0]
I (1637) AFE_manager: VAD ctrl ret 1
Build fst from commands.
Quantized MultiNet6:rnnt_ctc_1.0, name:mn6_cn, (Feb 18 2025 12:00:53)
Quantized MultiNet6 search method: 2, time out:5.8 s
I (2639) NEW_DATA_BUS: New ringbuffer:0x3c6feeb4, num:2, item_cnt:8192, db:0x3c6e2b3c
I (2643) NEW_DATA_BUS: New ringbuffer:0x3c6fe4d0, num:1, item_cnt:20480, db:0x3c6fcbd8
I (2651) AFE_manager: AFE manager suspend 1
I (2656) AFE_manager: AFE manager suspend 0
I (2661) AI_AUDIO_WWE: CB: RECV Pipeline EVT: el:ai_afe-0x3c2dd0b8, type:12288, sub:ESP_GMF_EVENT_STATE_INITIALIZED, payload:0x3fccd920, size:12,0x0
I (2675) AI_AUDIO_WWE: CB: RECV Pipeline EVT: el:ai_afe-0x3c2dd0b8, type:8192, sub:ESP_GMF_EVENT_STATE_RUNNING, payload:0x0, size:0,0x0
I (2688) ESP_GMF_TASK: One times job is complete, del[wk:0x3c2dd17c,ctx:0x3c2dd0b8, label:gmf_afe_open]
I (2698) ESP_GMF_PORT: ACQ IN, new self payload:0x3c2dd17c, port:0x3c2dd240, el:0x3c2dd0b8-ai_afe
Type 'help' to get the list of commands.
Use UP/DOWN arrows to navigate through command history.
Press TAB when typing command name to auto-complete.
I (2893) ESP_GMF_PORT: ACQ OUT, new self payload:0x3c6fc548, port:0x3c2dd280, el:0x3c2dd0b8-ai_afe
Audio > I (5668) AFE_manager: AFE Ctrl [1, 1]
I (5669) AFE_manager: VAD ctrl ret 1
I (5674) AI_AUDIO_WWE: WAKEUP_START [1 : 1]
I (6334) AI_AUDIO_WWE: VAD_START
W (7833) AI_AUDIO_WWE: Command 25, phrase_id 25, prob 0.998051, str: da kai kong tiao
I (8110) AI_AUDIO_WWE: VAD_END
I (8128) AI_AUDIO_WWE: File closed
I (10110) AFE_manager: AFE Ctrl [1, 0]
I (10111) AFE_manager: VAD ctrl ret 1
I (10118) AI_AUDIO_WWE: WAKEUP_END
```
### Troubleshooting
1. **Wake Word Not Detected**:
- Ensure `WAKENET_ENABLE` is set to `true`
- Verify that the model files are correctly loaded
- Check if the channel configuration matches the hardware setup
2. **Recording File Not Generated**:
- Ensure the SD card is correctly installed
- Verify that `VOICE2FILE` is set to `true`
- Confirm that speech is detected after wakeup
3. **Task Watchdog Timeout**:
- Ensure `esp_afe_manager_cfg_t.feed_task_setting.core` and `esp_afe_manager_cfg_t.fetch_task_setting.core` are configured on different CPU cores
- Properly allocate tasks on the core where the timeout occurs
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/examples/wwe/README_CN.md
+156
View File
@@ -0,0 +1,156 @@
# 语音识别
- [English](./README.md)
## 例程简介
本例程介绍使用 `AFE element` 进行唤醒词、人声检测和命令词检测的方法，并通过 [main.c](./main/main.c) 中的以下宏定义来配置功能，从而实现不同的应用组合：
```c
#define VOICE2FILE (true)
#define WAKENET_ENABLE (true)
#define VAD_ENABLE (true)
#define VCMD_ENABLE (true)
```
- `VOICE2FILE`: 以文件的形式保存 VAD 开始到 VAD 结束之间的音频
- `WAKENET_ENABLE`: 唤醒词识别使能
- `VAD_ENABLE`: 人声检测使能
- `VCMD_ENABLE`: 命令词检测使能
## GMF Pipeline
```mermaid
graph
subgraph WWE
E[CODEC_DEV_RX] --> F[GMF_AFE] --> G[GMF_PORT]
end
```
## 示例创建
### IDF 默认分支
本例程支持 IDF release/v5.3 及以后的分支
### 预备知识
例程中使用的语音唤醒和命令词检测，源自于 `esp-sr`，请先了解其配置和使用: [README](https://github.com/espressif/esp-sr/blob/master/README.md)
### 编译和下载
编译本例程前需要先确保已配置 ESP-IDF 的环境，如果已配置可跳到下一项配置，如果未配置需要先在 ESP-IDF 根目录运行下面脚本设置编译环境，有关配置和使用 ESP-IDF 完整步骤，请参阅 [《ESP-IDF 编程指南》](https://docs.espressif.com/projects/esp-idf/zh_CN/latest/esp32s3/index.html)
```
./install.sh
. ./export.sh
```
下面是简略编译步骤：
- 进入示例工程的位置
```
cd gmf_ai_audio/examples/wwe
```
- 选择编译芯片，以 esp32s3 为例：
```
idf.py set-target esp32s3
```
- 选择编译目标板，以 ESP32-S3-Korvo V2 为例：
```
idf.py menuconfig
在 `menuconfig` 中选择 `GMF APP Configuration` -> `Audio Board` -> `ESP32-S3-Korvo V2`，然后保存退出
```
- 编译例子程序
```
idf.py build
```
- 烧录程序并运行 monitor 工具来查看串口输出 (替换 PORT 为端口名称)：
```
idf.py -p PORT flash monitor
```
- 退出调试界面使用 `Ctrl-]`
## 如何使用例程
1. **配置功能**: 在 `main.c` 中通过上文中说明的宏定义启用或禁用功能来选择对应的应用场景
2. **运行要求**:
- 如果启用了 `VOICE2FILE`，请确保 SD 卡已正确安装到开发板上
- 如果启用了 `WAKENET_ENABLE`，运行时需要先说出唤醒词，然后再说命令词
- 如果关闭了 `WAKENET_ENABLE`，运行时可以直接说命令词
3. **查看结果**:
- 运行过程中，串口会打印检测到的事件日志
- 如果启用了 `VOICE2FILE`，录音文件将保存到 SD 卡，文件名格式为 `16k_16bit_1ch_{idx}.pcm`
### 功能和用法
运行后，可以对开发板说唤醒词和命令词，串口会打印如下日志：
```c
I (1547) AFE: AFE Version: (2MIC_V250113)
I (1550) AFE: Input PCM Config: total 4 channels(2 microphone, 1 playback), sample rate:16000
I (1560) AFE: AFE Pipeline: [input] -> |AEC(SR_HIGH_PERF)| -> |SE(BSS)| -> |VAD(WebRTC)| -> |WakeNet(wn9_hilexin,)| -> [output]
I (1572) AFE_manager: Feed task, ch 4, chunk 1024, buf size 8192
I (1579) GMF_AFE: Create AFE, ai_afe-0x3c2dcf90
I (1584) GMF_AFE: Create AFE, ai_afe-0x3c2dd0b8
I (1589) GMF_AFE: New an object,ai_afe-0x3c2dd0b8
I (1595) ESP_GMF_TASK: Waiting to run... [tsk:TSK_0x3fcc500c-0x3fcc500c, wk:0x0, run:0]
I (1603) ESP_GMF_THREAD: The TSK_0x3fcc500c created on internal memory
I (1610) ESP_GMF_TASK: Waiting to run... [tsk:TSK_0x3fcc500c-0x3fcc500c, wk:0x3c2dd17c, run:0]
I (1620) AI_AUDIO_WWE: CB: RECV Pipeline EVT: el:NULL-0x3c2dd080, type:8192, sub:ESP_GMF_EVENT_STATE_OPENING, payload:0x0, size:0,0x0
I (1632) AFE_manager: AFE Ctrl [1, 0]
I (1637) AFE_manager: VAD ctrl ret 1
Build fst from commands.
Quantized MultiNet6:rnnt_ctc_1.0, name:mn6_cn, (Feb 18 2025 12:00:53)
Quantized MultiNet6 search method: 2, time out:5.8 s
I (2639) NEW_DATA_BUS: New ringbuffer:0x3c6feeb4, num:2, item_cnt:8192, db:0x3c6e2b3c
I (2643) NEW_DATA_BUS: New ringbuffer:0x3c6fe4d0, num:1, item_cnt:20480, db:0x3c6fcbd8
I (2651) AFE_manager: AFE manager suspend 1
I (2656) AFE_manager: AFE manager suspend 0
I (2661) AI_AUDIO_WWE: CB: RECV Pipeline EVT: el:ai_afe-0x3c2dd0b8, type:12288, sub:ESP_GMF_EVENT_STATE_INITIALIZED, payload:0x3fccd920, size:12,0x0
I (2675) AI_AUDIO_WWE: CB: RECV Pipeline EVT: el:ai_afe-0x3c2dd0b8, type:8192, sub:ESP_GMF_EVENT_STATE_RUNNING, payload:0x0, size:0,0x0
I (2688) ESP_GMF_TASK: One times job is complete, del[wk:0x3c2dd17c,ctx:0x3c2dd0b8, label:gmf_afe_open]
I (2698) ESP_GMF_PORT: ACQ IN, new self payload:0x3c2dd17c, port:0x3c2dd240, el:0x3c2dd0b8-ai_afe
Type 'help' to get the list of commands.
Use UP/DOWN arrows to navigate through command history.
Press TAB when typing command name to auto-complete.
I (2893) ESP_GMF_PORT: ACQ OUT, new self payload:0x3c6fc548, port:0x3c2dd280, el:0x3c2dd0b8-ai_afe
Audio > I (5668) AFE_manager: AFE Ctrl [1, 1]
I (5669) AFE_manager: VAD ctrl ret 1
I (5674) AI_AUDIO_WWE: WAKEUP_START [1 : 1]
I (6334) AI_AUDIO_WWE: VAD_START
W (7833) AI_AUDIO_WWE: Command 25, phrase_id 25, prob 0.998051, str: da kai kong tiao
I (8110) AI_AUDIO_WWE: VAD_END
I (8128) AI_AUDIO_WWE: File closed
I (10110) AFE_manager: AFE Ctrl [1, 0]
I (10111) AFE_manager: VAD ctrl ret 1
I (10118) AI_AUDIO_WWE: WAKEUP_END
```
### 常见问题
1. **无法检测唤醒词**:
- 确保 `WAKENET_ENABLE` 设置为 `true`
- 检查模型文件是否正确加载
- 配置的通道顺序是否符合硬件实际
2. **录音文件未生成**:
- 确保 SD 卡已正确安装。
- 检查 `VOICE2FILE` 是否设置为 `true`
- 唤醒后是否有说话
3. **Task Watch Dog**:
- 确保 `esp_afe_manager_cfg_t.feed_task_setting.core` 和 `esp_afe_manager_cfg_t.fetch_task_setting.core` 配置在不同的 cpu 核心上
- 将出现 `task wdt` 的核上任务合理分配
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/examples/wwe/main/CMakeLists.txt
+3
View File
@@ -0,0 +1,3 @@
idf_component_register(SRC_DIRS "./"
INCLUDE_DIRS "./")
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/examples/wwe/main/idf_component.yml
+19
View File
@@ -0,0 +1,19 @@
dependencies:
espressif/gmf_loader:
override_path: ../../../../../packages/gmf_loader
version: "^0.7"
espressif/gmf_app_utils:
override_path: ../../../../../packages/gmf_app_utils
version: "^0.7"
espressif/gmf_core:
override_path: ../../../../../gmf_core
espressif/gmf_io:
override_path: ../../../../gmf_io
espressif/gmf_audio:
override_path: ../../../../gmf_audio
espressif/gmf_ai_audio:
override_path: ../../../../gmf_ai_audio
espressif/gmf_video:
override_path: ../../../../gmf_video
espressif/gmf_misc:
override_path: ../../../../gmf_misc
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/examples/wwe/main/main.c
+399
View File
@@ -0,0 +1,399 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
*
* SPDX-License-Identifier: Apache-2.0
*/
#include <stdbool.h>
#include <stdio.h>
#include <string.h>
#include "esp_err.h"
#include "esp_log.h"
#include "soc/soc_caps.h"
#include "esp_console.h"
#include "argtable3/argtable3.h"
#include "esp_gmf_io.h"
#include "esp_gmf_pipeline.h"
#include "esp_gmf_pool.h"
#include "esp_gmf_app_setup_peripheral.h"
#include "esp_gmf_io_codec_dev.h"
#include "esp_gmf_app_cli.h"
#include "gmf_loader_setup_defaults.h"
#if SOC_SDMMC_HOST_SUPPORTED == 1
#define VOICE2FILE (true)
#endif /* SOC_SDMMC_HOST_SUPPORTED == 1 */
#ifdef CONFIG_GMF_AI_AUDIO_WAKEUP_ENABLE
#define WAKENET_ENABLE (true)
#else
#define WAKENET_ENABLE (false)
#endif /* CONFIG_GMF_AI_AUDIO_WAKEUP_ENABLE */
#ifdef CONFIG_GMF_AI_AUDIO_VOICE_COMMAND_ENABLE
#define VCMD_ENABLE (true)
#else
#define VCMD_ENABLE (false)
#endif /* CONFIG_GMF_AI_AUDIO_VOICE_COMMAND_ENABLE */
#define QUIT_CMD_FOUND (BIT0)
#define BOARD_LYRAT_MINI (0)
#define BOARD_KORVO_2 (1)
#define BOARD_XD_AIOT_C3 (2)
#define BOARD_ESP_SPOT (3)
#define BOARD_P4_FUN_EV (4)
#if defined CONFIG_IDF_TARGET_ESP32S3
#define WITH_AFE (true)
#define AUDIO_BOARD (BOARD_KORVO_2)
#elif defined CONFIG_IDF_TARGET_ESP32
#define WITH_AFE (true)
#define AUDIO_BOARD (BOARD_LYRAT_MINI)
#elif defined CONFIG_IDF_TARGET_ESP32C3
#define WITH_AFE (false)
#define AUDIO_BOARD (BOARD_XD_AIOT_C3)
#elif defined CONFIG_IDF_TARGET_ESP32C5
#define WITH_AFE (false)
#define AUDIO_BOARD (BOARD_ESP_SPOT)
#elif defined CONFIG_IDF_TARGET_ESP32P4
#define WITH_AFE (true)
#define AUDIO_BOARD (BOARD_P4_FUN_EV)
#endif /* defined CONFIG_IDF_TARGET_ESP32S3 */
#if AUDIO_BOARD == BOARD_KORVO_2
#define ADC_I2S_CH (2)
#define ADC_I2S_BITS (32)
#define INPUT_CH_NUM (4)
#define INPUT_CH_BITS (16) /* For board `ESP32-S3-Korvo-2`, the es7210 is configured as 32-bit,
2-channel mode to accommodate 16-bit, 4-channel data */
#elif AUDIO_BOARD == BOARD_LYRAT_MINI
#define ADC_I2S_CH (2)
#define ADC_I2S_BITS (16)
#define INPUT_CH_NUM (ADC_I2S_CH)
#define INPUT_CH_BITS (ADC_I2S_BITS)
#elif AUDIO_BOARD == BOARD_XD_AIOT_C3
#define ADC_I2S_CH (2)
#define ADC_I2S_BITS (16)
#define INPUT_CH_NUM (ADC_I2S_CH)
#define INPUT_CH_BITS (ADC_I2S_BITS)
#elif AUDIO_BOARD == BOARD_ESP_SPOT
#define ADC_I2S_CH (2)
#define ADC_I2S_BITS (16)
#define INPUT_CH_NUM (ADC_I2S_CH)
#define INPUT_CH_BITS (ADC_I2S_BITS)
#elif AUDIO_BOARD == BOARD_P4_FUN_EV
#define ADC_I2S_CH (2)
#define ADC_I2S_BITS (16)
#define INPUT_CH_NUM (ADC_I2S_CH)
#define INPUT_CH_BITS (ADC_I2S_BITS)
#endif /* AUDIO_BOARD == BOARD_KORVO_2 */
#if WITH_AFE == true
#include "esp_gmf_afe.h"
#else
#include "esp_gmf_wn.h"
#endif /* WITH_AFE == true */
static const char *TAG = "AI_AUDIO_WWE";
#if WITH_AFE == true
static bool speeching = false;
static bool wakeup = false;
#endif /* WITH_AFE == true */
static EventGroupHandle_t g_event_group = NULL;
#if WITH_AFE == true
static esp_gmf_element_handle_t g_afe = NULL;
#endif /* WITH_AFE == true */
static esp_err_t _pipeline_event(esp_gmf_event_pkt_t *event, void *ctx)
{
ESP_LOGI(TAG, "CB: RECV Pipeline EVT: el:%s-%p, type:%d, sub:%s, payload:%p, size:%d,%p",
OBJ_GET_TAG(event->from), event->from, event->type, esp_gmf_event_get_state_str(event->sub),
event->payload, event->payload_size, ctx);
return 0;
}
#if WITH_AFE == true
void esp_gmf_afe_event_cb(esp_gmf_obj_handle_t obj, esp_gmf_afe_evt_t *event, void *user_data)
{
switch (event->type) {
case ESP_GMF_AFE_EVT_WAKEUP_START: {
wakeup = true;
#if WAKENET_ENABLE == true && VCMD_ENABLE == true
esp_gmf_afe_vcmd_detection_cancel(obj);
esp_gmf_afe_vcmd_detection_begin(obj);
#endif /* WAKENET_ENABLE == true && VCMD_ENABLE == true */
if (event->event_data) {
esp_gmf_afe_wakeup_info_t *info = event->event_data;
ESP_LOGI(TAG, "WAKEUP_START [%d : %d]", info->wake_word_index, info->wakenet_model_index);
} else {
ESP_LOGI(TAG, "WAKEUP_START");
}
break;
}
case ESP_GMF_AFE_EVT_WAKEUP_END: {
wakeup = false;
#if WAKENET_ENABLE == true && VCMD_ENABLE == true
esp_gmf_afe_vcmd_detection_cancel(obj);
#endif /* WAKENET_ENABLE == true && VCMD_ENABLE == true */
ESP_LOGI(TAG, "WAKEUP_END");
break;
}
case ESP_GMF_AFE_EVT_VAD_START: {
#if WAKENET_ENABLE != true && VCMD_ENABLE == true
esp_gmf_afe_vcmd_detection_cancel(obj);
esp_gmf_afe_vcmd_detection_begin(obj);
#endif /* WAKENET_ENABLE != true && VCMD_ENABLE == true */
speeching = true;
ESP_LOGI(TAG, "VAD_START");
break;
}
case ESP_GMF_AFE_EVT_VAD_END: {
#if WAKENET_ENABLE != true && VCMD_ENABLE == true
esp_gmf_afe_vcmd_detection_cancel(obj);
#endif /* WAKENET_ENABLE != true && VCMD_ENABLE == true */
speeching = false;
ESP_LOGI(TAG, "VAD_END");
break;
}
case ESP_GMF_AFE_EVT_VCMD_DECT_TIMEOUT: {
ESP_LOGI(TAG, "VCMD_DECT_TIMEOUT");
break;
}
default: {
esp_gmf_afe_vcmd_info_t *info = event->event_data;
ESP_LOGW(TAG, "Command %d, phrase_id %d, prob %f, str: %s",
event->type, info->phrase_id, info->prob, info->str);
break;
}
}
}
#else
static void esp_gmf_wn_event_cb(esp_gmf_obj_handle_t obj, int32_t trigger_ch, void *user_ctx)
{
static int32_t cnt = 1;
ESP_LOGI(TAG, "WWE detected on channel %" PRIi32 ", cnt: %" PRIi32, trigger_ch, cnt++);
if (cnt >= 10) {
xEventGroupSetBits(g_event_group, QUIT_CMD_FOUND);
}
}
#endif /* WITH_AFE == true */
static void voice_2_file(uint8_t *buffer, int len)
{
#if VOICE2FILE == true && WITH_AFE == true
#define MAX_FNAME_LEN (50)
static FILE *fp = NULL;
static int fcnt = 0;
if (speeching) {
if (!fp) {
char fname[MAX_FNAME_LEN] = {0};
snprintf(fname, MAX_FNAME_LEN - 1, "/sdcard/16k_16bit_1ch_%d.pcm", fcnt++);
fp = fopen(fname, "wb");
if (!fp) {
ESP_LOGE(TAG, "File open failed");
return;
}
}
if (len) {
fwrite(buffer, len, 1, fp);
}
} else {
if (fp) {
ESP_LOGI(TAG, "File closed");
fclose(fp);
fp = NULL;
}
}
#endif /* VOICE2FILE == true && WITH_AFE == true */
}
static esp_gmf_err_io_t outport_acquire_write(void *handle, esp_gmf_payload_t *load, int wanted_size, int block_ticks)
{
ESP_LOGD(TAG, "Acquire write");
return ESP_GMF_IO_OK;
}
static esp_gmf_err_io_t outport_release_write(void *handle, esp_gmf_payload_t *load, int block_ticks)
{
ESP_LOGD(TAG, "Release write");
voice_2_file(load->buf, load->valid_size);
return ESP_GMF_IO_OK;
}
#if WITH_AFE == true
static struct {
struct arg_int *keep;
struct arg_end *end;
} keep_args;
static int keep_awake(int argc, char **argv)
{
int nerrors = arg_parse(argc, argv, (void **)&keep_args);
if (nerrors != 0) {
arg_print_errors(stderr, keep_args.end, argv[0]);
return ESP_FAIL;
}
if (g_afe) {
esp_gmf_afe_keep_awake(g_afe, keep_args.keep->ival[0]);
printf("Keep awake: %d\n", keep_args.keep->ival[0]);
} else {
ESP_LOGE(TAG, "AFE not found");
}
return ESP_OK;
}
static struct {
struct arg_str *cmd;
struct arg_end *end;
} trigger_args;
static int trigger(int argc, char **argv)
{
int nerrors = arg_parse(argc, argv, (void **)&trigger_args);
if (nerrors != 0) {
arg_print_errors(stderr, trigger_args.end, argv[0]);
return ESP_FAIL;
}
if (g_afe) {
if (strcmp(trigger_args.cmd->sval[0], "wakeup") == 0) {
esp_gmf_afe_trigger_wakeup(g_afe);
} else if (strcmp(trigger_args.cmd->sval[0], "sleep") == 0) {
esp_gmf_afe_trigger_sleep(g_afe);
}
}
return 0;
}
#endif /* WITH_AFE == true */
static int quit(int argc, char **argv)
{
xEventGroupSetBits(g_event_group, QUIT_CMD_FOUND);
return 0;
}
static void wwe_cmds_register(void)
{
#if WITH_AFE == true
esp_console_cmd_t cmd_keep = {
.command = "keep",
.help = "keep awake",
.hint = NULL,
.func = &keep_awake,
.argtable = &keep_args,
};
keep_args.keep = arg_int0(NULL, NULL, "<0 - 1>", "Keep awake");
keep_args.end = arg_end(1);
esp_console_cmd_register(&cmd_keep);
esp_console_cmd_t cmd_trigger_wakeup = {
.command = "trigger",
.help = "trigger wakeup or sleep",
.hint = NULL,
.func = &trigger,
.argtable = &trigger_args,
};
trigger_args.cmd = arg_str0(NULL, NULL, "<wakeup|sleep>", "Trigger wakeup or sleep");
trigger_args.end = arg_end(1);
esp_console_cmd_register(&cmd_trigger_wakeup);
#endif /* WITH_AFE == true */
esp_console_cmd_t cmd_quit = {
.command = "quit",
.help = "quit",
.hint = NULL,
.func = &quit,
.argtable = NULL,
};
esp_console_cmd_register(&cmd_quit);
}
void app_main(void)
{
esp_log_level_set("*", ESP_LOG_INFO);
esp_gmf_app_codec_info_t codec_info = ESP_GMF_APP_CODEC_INFO_DEFAULT();
codec_info.record_info.sample_rate = 16000;
codec_info.record_info.channel = ADC_I2S_CH;
codec_info.record_info.bits_per_sample = ADC_I2S_BITS;
codec_info.play_info.sample_rate = codec_info.record_info.sample_rate;
esp_gmf_app_setup_codec_dev(&codec_info);
void *sdcard_handle = NULL;
esp_gmf_app_setup_sdcard(&sdcard_handle);
g_event_group = xEventGroupCreate();
esp_gmf_pool_handle_t pool = NULL;
esp_gmf_pool_init(&pool);
gmf_loader_setup_all_defaults(pool);
esp_gmf_pipeline_handle_t pipe = NULL;
#if WITH_AFE == true
const char *name[] = {"ai_afe"};
#else
const char *name[] = {"ai_wn"};
#endif /* WITH_AFE == true */
esp_gmf_pool_new_pipeline(pool, "io_codec_dev", name, sizeof(name) / sizeof(char *), NULL, &pipe);
if (pipe == NULL) {
ESP_LOGE(TAG, "There is no pipeline");
goto __quit;
}
esp_gmf_io_codec_dev_set_dev(ESP_GMF_PIPELINE_GET_IN_INSTANCE(pipe), esp_gmf_app_get_record_handle());
#if WITH_AFE == true
esp_gmf_pipeline_get_el_by_name(pipe, "ai_afe", &g_afe);
esp_gmf_afe_set_event_cb(g_afe, esp_gmf_afe_event_cb, NULL);
#else
esp_gmf_element_handle_t wn = NULL;
esp_gmf_pipeline_get_el_by_name(pipe, "ai_wn", &wn);
esp_gmf_wn_set_detect_cb(wn, esp_gmf_wn_event_cb, NULL);
#endif /* WITH_AFE == true */
esp_gmf_port_handle_t outport = NEW_ESP_GMF_PORT_OUT_BYTE(outport_acquire_write,
outport_release_write,
NULL,
NULL,
2048,
100);
esp_gmf_pipeline_reg_el_port(pipe, name[0], ESP_GMF_IO_DIR_WRITER, outport);
esp_gmf_info_sound_t info = {
.sample_rates = 16000,
.channels = INPUT_CH_NUM,
.bits = INPUT_CH_BITS,
};
esp_gmf_pipeline_report_info(pipe, ESP_GMF_INFO_SOUND, &info, sizeof(info));
esp_gmf_task_cfg_t cfg = DEFAULT_ESP_GMF_TASK_CONFIG();
cfg.ctx = NULL;
cfg.cb = NULL;
cfg.thread.core = 0;
cfg.thread.prio = 5;
cfg.thread.stack = 5120;
esp_gmf_task_handle_t task = NULL;
esp_gmf_task_init(&cfg, &task);
esp_gmf_pipeline_bind_task(pipe, task);
esp_gmf_pipeline_loading_jobs(pipe);
esp_gmf_pipeline_set_event(pipe, _pipeline_event, NULL);
esp_gmf_pipeline_run(pipe);
esp_gmf_app_cli_init("Audio >", wwe_cmds_register);
while (1) {
EventBits_t bits = xEventGroupWaitBits(g_event_group, QUIT_CMD_FOUND, pdTRUE, pdFALSE, portMAX_DELAY);
if (bits & QUIT_CMD_FOUND) {
ESP_LOGI(TAG, "Quit command found, stopping pipeline");
break;
}
}
__quit:
esp_gmf_pipeline_stop(pipe);
esp_gmf_task_deinit(task);
esp_gmf_pipeline_destroy(pipe);
gmf_loader_teardown_all_defaults(pool);
esp_gmf_pool_deinit(pool);
esp_gmf_app_teardown_codec_dev();
esp_gmf_app_teardown_sdcard(sdcard_handle);
vEventGroupDelete(g_event_group);
ESP_LOGW(TAG, "Wake word engine demo finished");
}
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/examples/wwe/partitions.csv
+5
View File
@@ -0,0 +1,5 @@
# Espressif ESP32 Partition Table
# Name, Type, SubType, Offset, Size
nvs, data, nvs, 0x9000, 0x6000,
factory, app, factory, 0x010000, 2500k
model, data, spiffs, , 5168K,
1 # Espressif ESP32 Partition Table
2 # Name, Type, SubType, Offset, Size
3 nvs, data, nvs, 0x9000, 0x6000,
4 factory, app, factory, 0x010000, 2500k
5 model, data, spiffs, , 5168K,
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/examples/wwe/partitions_esp32.csv
+5
View File
@@ -0,0 +1,5 @@
# Espressif ESP32 Partition Table
# Name, Type, SubType, Offset, Size
nvs, data, nvs, 0x9000, 0x6000,
factory, app, factory, 0x010000, 2500k
model, data, spiffs, , 2M,
1 # Espressif ESP32 Partition Table
2 # Name, Type, SubType, Offset, Size
3 nvs, data, nvs, 0x9000, 0x6000,
4 factory, app, factory, 0x010000, 2500k
5 model, data, spiffs, , 2M,
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/examples/wwe/pytest_wwe.py
+14
View File
@@ -0,0 +1,14 @@
# SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
#
# SPDX-License-Identifier: Apache-2.0
import pytest
import os
from pytest_embedded import Dut
@pytest.mark.esp32
@pytest.mark.esp32s3
@pytest.mark.temp_skip_ci(targets=['esp32', 'esp32s3', 'esp32p4'], reason='No running in CI')
def test_wwe(dut: Dut)-> None:
dut.expect(r'Audio >', timeout=20)
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/examples/wwe/sdkconfig.defaults
+41
View File
@@ -0,0 +1,41 @@
#
# Serial flasher config
#
CONFIG_ESPTOOLPY_FLASH_MODE_AUTO_DETECT=y
CONFIG_ESPTOOLPY_FLASHMODE_DIO=y
CONFIG_ESPTOOLPY_FLASH_SAMPLE_MODE_STR=y
CONFIG_ESPTOOLPY_FLASHMODE="dio"
CONFIG_ESPTOOLPY_FLASHFREQ_80M=y
CONFIG_ESPTOOLPY_FLASHFREQ="80m"
CONFIG_ESPTOOLPY_FLASHSIZE_8MB=y
CONFIG_ESPTOOLPY_FLASHSIZE="8MB"
#
# Compiler options
#
CONFIG_COMPILER_OPTIMIZATION_PERF=y
# end of Compiler options
#
# ESP PSRAM
#
CONFIG_SPIRAM=y
CONFIG_SPIRAM_TYPE_AUTO=y
CONFIG_SPIRAM_ALLOW_STACK_EXTERNAL_MEMORY=y
CONFIG_SPIRAM_SPEED_80M=y
CONFIG_SPIRAM_SPEED=80
#
# Audio Codec Device Configuration
#
# CONFIG_CODEC_I2C_BACKWARD_COMPATIBLE is not set
# end of Audio Codec Device Configuration
CONFIG_FATFS_LFN_HEAP=y
CONFIG_FREERTOS_HZ=1000
CONFIG_FREERTOS_USE_TRACE_FACILITY=y
CONFIG_FREERTOS_USE_STATS_FORMATTING_FUNCTIONS=y
# CONFIG_FREERTOS_USE_LIST_DATA_INTEGRITY_CHECK_BYTES is not set
CONFIG_FREERTOS_VTASKLIST_INCLUDE_COREID=y
CONFIG_FREERTOS_GENERATE_RUN_TIME_STATS=y
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/examples/wwe/sdkconfig.defaults.esp32
+45
View File
@@ -0,0 +1,45 @@
CONFIG_IDF_CMAKE=y
CONFIG_IDF_TARGET_ARCH_XTENSA=y
CONFIG_IDF_TARGET="esp32"
CONFIG_IDF_TARGET_ESP32=y
CONFIG_IDF_FIRMWARE_CHIP_ID=0x0000
#
# Partition Table
#
CONFIG_PARTITION_TABLE_CUSTOM=y
CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="partitions_esp32.csv"
CONFIG_PARTITION_TABLE_FILENAME="partitions_esp32.csv"
CONFIG_PARTITION_TABLE_OFFSET=0x8000
CONFIG_PARTITION_TABLE_MD5=y
# end of Partition Table
#
# ESP Speech Recognition
#
CONFIG_AFE_INTERFACE_V1=y
CONFIG_SR_NSN_WEBRTC=y
CONFIG_SR_VADN_WEBRTC=y
#
# Load Multiple Wake Words
#
CONFIG_SR_WN_WN9_HILEXIN=y
CONFIG_SR_MN_CN_NONE=y
# CONFIG_SR_MN_CN_MULTINET2_SINGLE_RECOGNITION is not set
CONFIG_SR_MN_EN_NONE=y
# end of ESP Speech Recognition
#
# GMF AI Audio
#
CONFIG_GMF_AI_AUDIO_INIT_AEC=y
CONFIG_GMF_AI_AUDIO_INIT_AFE=y
CONFIG_GMF_AI_AUDIO_AFE_VAD_ENABLE=y
CONFIG_GMF_AI_AUDIO_VAD_MODE=3
CONFIG_GMF_AI_AUDIO_WAKEUP_ENABLE=y
# end of GMF AI Audio
CONFIG_AUDIO_BOARD="LYRAT_MINI_V1"
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/examples/wwe/sdkconfig.defaults.esp32c3
+32
View File
@@ -0,0 +1,32 @@
#
# Partition Table
#
CONFIG_PARTITION_TABLE_CUSTOM=y
CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="partitions.csv"
CONFIG_PARTITION_TABLE_FILENAME="partitions.csv"
CONFIG_PARTITION_TABLE_OFFSET=0x8000
CONFIG_PARTITION_TABLE_MD5=y
# end of Partition Table
#
# ESP Speech Recognition
#
CONFIG_AFE_INTERFACE_V1=y
CONFIG_SR_NSN_WEBRTC=y
CONFIG_SR_VADN_WEBRTC=y
#
# Load Multiple Wake Words
#
CONFIG_SR_WN_WN9S_HILEXIN=y
# CONFIG_SR_WN_WN9S_HIESP is not set
# CONFIG_SR_WN_WN9S_NIHAOXIAOZHI is not set
# CONFIG_SR_WN_WN9S_HIJASON is not set
# end of Load Multiple Wake Words
CONFIG_SR_MN_CN_NONE=y
CONFIG_SR_MN_EN_NONE=y
# end of ESP Speech Recognition
CONFIG_GMF_AI_AUDIO_INIT_WN=y
CONFIG_GMF_AI_AUDIO_WN_CH_ALLOCATION="MR"
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/examples/wwe/sdkconfig.defaults.esp32c5
+34
View File
@@ -0,0 +1,34 @@
#
# Partition Table
#
CONFIG_PARTITION_TABLE_CUSTOM=y
CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="partitions.csv"
CONFIG_PARTITION_TABLE_FILENAME="partitions.csv"
CONFIG_PARTITION_TABLE_OFFSET=0x8000
CONFIG_PARTITION_TABLE_MD5=y
# end of Partition Table
#
# ESP Speech Recognition
#
CONFIG_AFE_INTERFACE_V1=y
CONFIG_SR_NSN_WEBRTC=y
CONFIG_SR_VADN_WEBRTC=y
#
# Load Multiple Wake Words
#
CONFIG_SR_WN_WN9S_HILEXIN=y
# CONFIG_SR_WN_WN9S_HIESP is not set
# CONFIG_SR_WN_WN9S_NIHAOXIAOZHI is not set
# CONFIG_SR_WN_WN9S_HIJASON is not set
# end of Load Multiple Wake Words
CONFIG_SR_MN_CN_NONE=y
CONFIG_SR_MN_EN_NONE=y
# end of ESP Speech Recognition
CONFIG_GMF_AI_AUDIO_INIT_WN=y
CONFIG_GMF_AI_AUDIO_WN_CH_ALLOCATION="MR"
CONFIG_ESP_CONSOLE_USB_SERIAL_JTAG=y
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/examples/wwe/sdkconfig.defaults.esp32p4
+59
View File
@@ -0,0 +1,59 @@
CONFIG_IDF_TARGET="esp32p4"
CONFIG_IDF_TARGET_ESP32P4=y
CONFIG_IDF_EXPERIMENTAL_FEATURES=y
#
# Partition Table
#
CONFIG_PARTITION_TABLE_CUSTOM=y
CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="partitions.csv"
CONFIG_PARTITION_TABLE_FILENAME="partitions.csv"
# end of Partition Table
#
# ESP Speech Recognition
#
CONFIG_MODEL_IN_FLASH=y
CONFIG_AFE_INTERFACE_V1=y
CONFIG_SR_NSN_WEBRTC=y
CONFIG_SR_VADN_WEBRTC=y
CONFIG_SR_WN_WN9S_HILEXIN=y
CONFIG_SR_MN_CN_MULTINET7_QUANT=y
CONFIG_SR_MN_EN_NONE=y
# end of ESP Speech Recognition
#
# ESP PSRAM
#
CONFIG_SPIRAM=y
#
# PSRAM config
#
CONFIG_SPIRAM_MODE_HEX=y
CONFIG_SPIRAM_SPEED_200M=y
CONFIG_SPIRAM_SPEED=200
CONFIG_SPIRAM_BOOT_INIT=y
CONFIG_SPIRAM_USE_MALLOC=y
CONFIG_SPIRAM_MEMTEST=y
CONFIG_SPIRAM_MALLOC_ALWAYSINTERNAL=16384
CONFIG_SPIRAM_MALLOC_RESERVE_INTERNAL=32768
#
# GMF AI Audio
#
CONFIG_GMF_AI_AUDIO_INIT_AEC=y
CONFIG_GMF_AI_AUDIO_INIT_AFE=y
CONFIG_GMF_AI_AUDIO_AFE_VAD_ENABLE=y
CONFIG_GMF_AI_AUDIO_AFE_AEC_ENABLE=y
CONFIG_GMF_AI_AUDIO_AEC_CH_ALLOCATION="MR"
CONFIG_GMF_AI_AUDIO_AFE_CH_ALLOCATION="MR"
CONFIG_GMF_AI_AUDIO_WAKEUP_ENABLE=y
CONFIG_GMF_AI_AUDIO_VOICE_COMMAND_ENABLE=y
# end of GMF AI Audio
CONFIG_AUDIO_BOARD="ESP32_P4_DEV_V14"
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/examples/wwe/sdkconfig.defaults.esp32s3
+84
View File
@@ -0,0 +1,84 @@
CONFIG_IDF_CMAKE=y
CONFIG_IDF_TARGET_ARCH_XTENSA=y
CONFIG_IDF_TARGET="esp32s3"
CONFIG_IDF_TARGET_ESP32S3=y
CONFIG_IDF_FIRMWARE_CHIP_ID=0x0009
#
# Partition Table
#
CONFIG_PARTITION_TABLE_CUSTOM=y
CONFIG_PARTITION_TABLE_CUSTOM_FILENAME="partitions.csv"
CONFIG_PARTITION_TABLE_FILENAME="partitions.csv"
CONFIG_PARTITION_TABLE_OFFSET=0x8000
CONFIG_PARTITION_TABLE_MD5=y
# end of Partition Table
#
# ESP Speech Recognition
#
CONFIG_MODEL_IN_FLASH=y
CONFIG_AFE_INTERFACE_V1=y
CONFIG_SR_WN_WN9_HILEXIN=y
CONFIG_SR_MN_CN_MULTINET6_QUANT=y
CONFIG_SR_MN_EN_NONE=y
# end of ESP Speech Recognition
#
# ESP PSRAM
#
CONFIG_SPIRAM=y
#
# SPI RAM config
#
CONFIG_SPIRAM_MODE_OCT=y
CONFIG_SPIRAM_TYPE_AUTO=y
CONFIG_SPIRAM_ALLOW_STACK_EXTERNAL_MEMORY=y
CONFIG_SPIRAM_CLK_IO=30
CONFIG_SPIRAM_CS_IO=26
CONFIG_SPIRAM_SPEED_80M=y
CONFIG_SPIRAM_SPEED=80
CONFIG_SPIRAM_BOOT_INIT=y
CONFIG_SPIRAM_USE_MALLOC=y
CONFIG_SPIRAM_MEMTEST=y
CONFIG_SPIRAM_MALLOC_ALWAYSINTERNAL=16384
CONFIG_SPIRAM_MALLOC_RESERVE_INTERNAL=32768
# end of SPI RAM config
# end of ESP PSRAM
#
# ESP System Settings
#
CONFIG_ESP32S3_DEFAULT_CPU_FREQ_240=y
#
# Cache config
#
CONFIG_ESP32S3_INSTRUCTION_CACHE_32KB=y
CONFIG_ESP32S3_INSTRUCTION_CACHE_SIZE=0x8000
CONFIG_ESP32S3_INSTRUCTION_CACHE_8WAYS=y
CONFIG_ESP32S3_ICACHE_ASSOCIATED_WAYS=8
CONFIG_ESP32S3_INSTRUCTION_CACHE_LINE_32B=y
CONFIG_ESP32S3_INSTRUCTION_CACHE_LINE_SIZE=32
CONFIG_ESP32S3_DATA_CACHE_64KB=y
CONFIG_ESP32S3_DATA_CACHE_SIZE=0x10000
CONFIG_ESP32S3_DATA_CACHE_8WAYS=y
CONFIG_ESP32S3_DCACHE_ASSOCIATED_WAYS=8
CONFIG_ESP32S3_DATA_CACHE_LINE_64B=y
CONFIG_ESP32S3_DATA_CACHE_LINE_SIZE=64
# end of Cache config
#
# GMF AI Audio
#
CONFIG_GMF_AI_AUDIO_INIT_AEC=y
CONFIG_GMF_AI_AUDIO_INIT_AFE=y
CONFIG_GMF_AI_AUDIO_AFE_VAD_ENABLE=y
CONFIG_GMF_AI_AUDIO_AFE_AEC_ENABLE=y
CONFIG_GMF_AI_AUDIO_WAKEUP_ENABLE=y
CONFIG_GMF_AI_AUDIO_VOICE_COMMAND_ENABLE=y
# end of GMF AI Audio
CONFIG_AUDIO_BOARD="S3_Korvo_V2"
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/idf_component.yml
+15
View File
@@ -0,0 +1,15 @@
version: "0.7.4"
description: Espressif GMF AI Audio Elements Module
url: https://github.com/espressif/esp-gmf/tree/main/elements/gmf_ai_audio
documentation: "https://github.com/espressif/esp-gmf/blob/main/elements/gmf_ai_audio/README.md"
issues: "https://github.com/espressif/esp-gmf/issues"
tags:
- "gmf"
dependencies:
espressif/esp-sr:
version: "2.1.5"
espressif/gmf_core:
version: "^0.7"
require: public
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/include/esp_gmf_aec.h
+53
View File
@@ -0,0 +1,53 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#pragma once
#include "esp_gmf_err.h"
#include "esp_gmf_element.h"
#include "esp_afe_config.h"
/* This element can run with esp32, esp32s3, esp32c5 and esp32p4 */
#ifdef __cplusplus
extern "C" {
#endif /* __cplusplus */
/**
* @brief Configuration structure for AEC
*
* @note The input format, same as afe config:
* `M` to represent the microphone channel,
* `R` to represent the playback reference channel,
* `N` to represent an unknown or unused channel
* For example, input_format="MMNR" indicates that the input data consists of four channels,
* which are the microphone channel, the microphone channel, an unused channel, and the playback channel
*/
typedef struct {
uint8_t filter_len; /*!< The length of filter. The larger the filter, the higher the CPU loading
* Recommended filter_length = 4 for esp32s3 and esp32p4. Recommended filter_length = 2 for esp32c5 */
afe_type_t type; /*!< AFE type */
afe_mode_t mode; /*!< AFE mode */
char *input_format; /*!< Input format */
} esp_gmf_aec_cfg_t;
/**
* @brief Initialize the Espressif AEC element
*
* @param[in] cfg Pointer to the configuration structure
* @param[out] out_handle Pointer to the handle to be created
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_MEMORY_LACK Memory allocation failed
* - ESP_GMF_ERR_INVALID_ARG Invalid argument
*/
esp_gmf_err_t esp_gmf_aec_init(esp_gmf_aec_cfg_t *cfg, esp_gmf_obj_handle_t *out_handle);
#ifdef __cplusplus
}
#endif /* __cplusplus */
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/include/esp_gmf_afe.h
+223
View File
@@ -0,0 +1,223 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#pragma once
#include "esp_gmf_afe_manager.h"
#include "esp_gmf_err.h"
#include "esp_gmf_element.h"
#ifdef __cplusplus
extern "C" {
#endif /* __cplusplus */
/* This element can run with esp32, esp32s3 and esp32p4 */
#define ESP_GMF_AFE_VCMD_MAX_LEN (256)
#define ESP_GMF_AFE_DEFAULT_DELAY_SAMPLES (2048)
#define ESP_GMF_AFE_DEFAULT_WAKEUP_TIME_MS (10000)
#define ESP_GMF_AFE_DEFAULT_WAKEUP_END_MS (2000)
#define ESP_GMF_AFE_DEFAULT_VCMD_TIMEOUT_MS (5760)
/**
* @brief Information when wakeup state detected, event data for "ESP_GMF_AFE_EVT_WAKEUP_START"
*/
typedef struct {
float data_volume; /*!< Volume of input audio, the unit is decibel(dB) */
int wake_word_index; /*!< Wake word index which start from 1 */
int wakenet_model_index; /*!< Wakenets index which start from 1 */
} esp_gmf_afe_wakeup_info_t;
/**
* @brief Information when voice command detected, event data for `ESP_GMF_AFE_EVT_VCMD_DECTECTED`
*/
typedef struct {
int phrase_id; /*!< Phrase ID */
float prob; /*!< probability */
char str[ESP_GMF_AFE_VCMD_MAX_LEN]; /*!< Command string */
} esp_gmf_afe_vcmd_info_t;
/**
* @brief AFE manager event type
*/
typedef enum {
ESP_GMF_AFE_EVT_WAKEUP_START = -100, /*!< Wakeup start */
ESP_GMF_AFE_EVT_WAKEUP_END, /*!< Wakeup stop */
ESP_GMF_AFE_EVT_VAD_START, /*!< Vad start */
ESP_GMF_AFE_EVT_VAD_END, /*!< Vad stop */
ESP_GMF_AFE_EVT_VCMD_DECT_TIMEOUT, /*!< Voice command detect timeout */
ESP_GMF_AFE_EVT_VCMD_DECTECTED = 0 /*!< Form 0 is the id of the voice commands detected by Multinet*/
/* DO NOT add items below this line */
} esp_gmf_afe_event_t;
/**
* @brief Event structure for GMF AFE
*/
typedef struct {
esp_gmf_afe_event_t type; /*!< Event type */
void *event_data; /*!< Event data */
size_t data_len; /*!< Length of event data */
} esp_gmf_afe_evt_t;
/**
* @brief Callback type for GMF AFE events
*/
typedef void (*esp_gmf_afe_event_cb_t)(esp_gmf_element_handle_t el, esp_gmf_afe_evt_t *event, void *user_data);
/**
* @brief Configuration structure for GMF AFE wrapper
*/
typedef struct {
esp_gmf_afe_manager_handle_t afe_manager; /*!< AFE Manager handle */
uint32_t delay_samples; /*!< Number of samples to delay
Note: If the user wants to using the output of AFE only after detecting the VAD start event,
the time corresponding to the value of this parameter should not be less than the `vad_min_speech_ms` in `afe_config_t` used when creating the afe_manager,
otherwise, a small portion of the data at the beginning of the voice may be lost */
void *models; /*!< List of models */
uint32_t wakeup_time; /*!< Unit:ms. The duration that the wakeup state remains when VAD is not triggered */
uint32_t wakeup_end; /*!< Unit:ms. When the silence time after AUDIO_REC_VAD_END state exceeds this value, it is determined as AUDIO_REC_WAKEUP_END */
bool vcmd_detect_en; /*!< Enable voice command detection */
uint32_t vcmd_timeout; /*!< Timeout for voice command detection, units: ms*/
const char *mn_language; /*!< Language for the multi-net model, `cn` or `en` */
esp_gmf_afe_event_cb_t event_cb; /*!< Callback function for AI audio events */
void *event_ctx; /*!< User context to be passed to the callback function */
} esp_gmf_afe_cfg_t;
#define DEFAULT_GMF_AFE_CFG(__afe_manager, __event_cb, __event_ctx, __models) { \
.afe_manager = __afe_manager, /* */ \
.models = __models, /* */ \
.delay_samples = ESP_GMF_AFE_DEFAULT_DELAY_SAMPLES, /* Default delay samples, 64ms */ \
.wakeup_time = ESP_GMF_AFE_DEFAULT_WAKEUP_TIME_MS, /* Default wakeup duration */ \
.wakeup_end = ESP_GMF_AFE_DEFAULT_WAKEUP_END_MS, /* Default wakeup end time */ \
.vcmd_detect_en = false, /* */ \
.vcmd_timeout = ESP_GMF_AFE_DEFAULT_VCMD_TIMEOUT_MS, /* */ \
.mn_language = "cn", /* Default language `cn` */ \
.event_cb = __event_cb, /* */ \
.event_ctx = __event_ctx, /* */ \
}
/**
* @brief Initialize the GMF AFE
*
* @param[in] config Pointer to the configuration structure
* @param[out] handle Pointer to the handle to be created
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_MEMORY_LACK Memory allocation failed
* - ESP_GMF_ERR_INVALID_ARG Invalid argument
*/
esp_gmf_err_t esp_gmf_afe_init(void *config, esp_gmf_obj_handle_t *handle);
/**
* @brief Begin voice command detection
*
* @param[in] handle Handle to the GMF object
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_INVALID_ARG Invalid argument
* - ESP_GMF_ERR_INVALID_STATE Voice command not enabled
*/
esp_gmf_err_t esp_gmf_afe_vcmd_detection_begin(esp_gmf_element_handle_t handle);
/**
* @brief Cancel voice command detection
*
* @note This function is used to clear the states of voice command detection process,
* the voice command detection will stay enabled,
* and the user can call `esp_gmf_afe_vcmd_detection_begin` to start the detection again
*
* @param[in] handle Handle to the GMF object
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_INVALID_ARG Invalid argument
* - ESP_GMF_ERR_INVALID_STATE Voice command not enabled
*/
esp_gmf_err_t esp_gmf_afe_vcmd_detection_cancel(esp_gmf_element_handle_t handle);
/**
* @brief Set the event callback for the AFE (Audio Front-End) element
*
* This function registers a callback function to handle events generated by the
* AFE element. The callback will be invoked with the specified context whenever
* an event occurs
*
* @param handle The handle to the AFE element
* @param cb The callback function to handle AFE events
* @param ctx User-defined context to be passed to the callback function
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_INVALID_ARG Invalid argument
* - ESP_GMF_ERR_INVALID_STATE Config not exist
*/
esp_gmf_err_t esp_gmf_afe_set_event_cb(esp_gmf_element_handle_t handle, esp_gmf_afe_event_cb_t cb, void *ctx);
/**
* @brief Enable or disable keep-awake mode
*
* When keep-awake mode is enabled, the system will remain in the wake state
* and prevent wakeup_end events from being triggered automatically
* This is useful for scenarios where you want to keep the system active
* without automatic timeout
*
* @param handle The handle to the AFE element
* @param enable True to enable keep-awake mode, false to disable
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_INVALID_ARG Invalid argument
* - ESP_GMF_ERR_INVALID_STATE Config not exist
* - ESP_GMF_ERR_TIMEOUT Command send timeout
*/
esp_gmf_err_t esp_gmf_afe_keep_awake(esp_gmf_element_handle_t handle, bool enable);
/**
* @brief Manually trigger wakeup state
*
* This function allows manual activation of the wakeup state without waiting
* for automatic wakeword detection. It is useful in the following scenarios:
*
* 1. **Button-triggered activation**: When users press a physical button to
* activate voice interaction, bypassing the need for wakewords
* 2. **External event-driven activation**: When the system needs to enter
* wakeup state based on external triggers (sensors, timers, network events)
*
* After calling this function, the AFE will enter wakeup state and begin
* listening for voice commands (if voice command detection is enabled).
* The system will generate ESP_GMF_AFE_EVT_WAKEUP_START event and remain
* active according to the configured wakeup_time duration.
*
* @param[in] handle Handle to the GMF object
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_INVALID_ARG Invalid argument
* - ESP_GMF_ERR_INVALID_STATE Element not opened
* - ESP_GMF_ERR_TIMEOUT Command send timeout
*/
esp_gmf_err_t esp_gmf_afe_trigger_wakeup(esp_gmf_element_handle_t handle);
/**
* @brief Manually trigger sleep of wakeup state
*
* @param[in] handle Handle to the GMF object
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_INVALID_ARG Invalid argument
* - ESP_GMF_ERR_INVALID_STATE Element not opened
* - ESP_GMF_ERR_TIMEOUT Command send timeout
*/
esp_gmf_err_t esp_gmf_afe_trigger_sleep(esp_gmf_element_handle_t handle);
#ifdef __cplusplus
}
#endif /* __cplusplus */
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/include/esp_gmf_afe_manager.h
+243
View File
@@ -0,0 +1,243 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#pragma once
#include "esp_afe_sr_iface.h"
#include "esp_gmf_err.h"
/**
* The AFE Manager aims to provide users with a simple interface for managing AFE (Audio front end) functions,
* including WakeNet, VAD, AEC, SE, and more
* This component will automatically create feed and fetch tasks,
* users only need to provide data read callback functions and result processing callback functions
* Users can configure `AFE` functions through the `afe_config_t` structure
* The data fed into `AFE` must be in 16-bit PCM format with a sampling rate of 16kHz,
* the number of channels and channel arrangement are determined by the configuration in the `afe_config_init` function,
* for details, please refer to the description of the `afe_config_init` function which provide by `esp-sr`
*/
#ifdef __cplusplus
extern "C" {
#endif /* __cplusplus */
#define ESP_AFE_MANAGER_FEED_TASK_CORE (0)
#define ESP_AFE_MANAGER_FEED_TASK_PRIO (5)
#define ESP_AFE_MANAGER_FEED_TASK_STACK (3 * 1024)
#define ESP_AFE_MANAGER_FETCH_TASK_CORE (1)
#define ESP_AFE_MANAGER_FETCH_TASK_PRIO (5)
#define ESP_AFE_MANAGER_FETCH_TASK_STACK (3 * 1024)
/**
* @enum
* @brief Enumeration of AFE functions
*/
typedef enum {
ESP_AFE_FEATURE_WAKENET, /*!< WakeNet function */
ESP_AFE_FEATURE_VAD, /*!< Voice Activity Detection function */
ESP_AFE_FEATURE_AEC, /*!< Acoustic Echo Cancellation function */
ESP_AFE_FEATURE_SE, /*!< Speech Enhancement function */
} esp_gmf_afe_feature_t;
/**
* @brief Handle for the AFE manager
*/
typedef void *esp_gmf_afe_manager_handle_t;
/**
* @brief Callback function type for processing AFE results
*
* @param[in] result Pointer to the result structure
* @param[in] user_ctx User context to be passed to the callback function
*/
typedef void (*esp_gmf_afe_manager_result_cb_t)(afe_fetch_result_t *result, void *user_ctx);
/**
* @brief Callback type for reading data
*
* @param[in] buffer Pointer to the buffer to read data into
* @param[in] buf_sz Size of the buffer
* @param[in] user_ctx User context to be passed to the callback function
* @param[in] ticks Number of ticks to wait for data
*
* @return
*/
typedef int32_t (*esp_gmf_afe_manager_read_cb_t)(void *buffer, int buf_sz, void *user_ctx, uint32_t ticks);
/**
* @brief Configuration structure for the task setting
*/
typedef struct {
uint32_t stack_size; /*!< Task stack size */
uint8_t core; /*!< Task core id */
uint8_t prio; /*!< Task priority */
} esp_gmf_afe_manager_task_setting_t;
/**
* @brief Configuration structure for the AFE manager
*/
typedef struct {
afe_config_t *afe_cfg; /*!< Configuration of ESP AFE */
esp_gmf_afe_manager_task_setting_t feed_task_setting; /*!< Feed task setting */
esp_gmf_afe_manager_task_setting_t fetch_task_setting; /*!< Fetch task setting */
esp_gmf_afe_manager_read_cb_t read_cb; /*!< Callback function for reading audio data */
void *read_ctx; /*!< Context for the read callback function */
esp_gmf_afe_manager_result_cb_t result_cb; /*!< Callback function for processing AFE results */
void *result_ctx; /*!< Context for the result callback function */
} esp_gmf_afe_manager_cfg_t;
#define DEFAULT_GMF_AFE_MANAGER_CFG(_afe_cfg, _read_cb, _read_ctx, _result_cb, _result_ctx) { \
.afe_cfg = _afe_cfg, \
.feed_task_setting = { \
.stack_size = ESP_AFE_MANAGER_FEED_TASK_STACK, \
.core = ESP_AFE_MANAGER_FEED_TASK_CORE, \
.prio = ESP_AFE_MANAGER_FEED_TASK_PRIO \
}, \
.fetch_task_setting = { \
.stack_size = ESP_AFE_MANAGER_FETCH_TASK_STACK, \
.core = ESP_AFE_MANAGER_FETCH_TASK_CORE, \
.prio = ESP_AFE_MANAGER_FETCH_TASK_PRIO \
}, \
.read_cb = _read_cb, \
.read_ctx = _read_ctx, \
.result_cb = _result_cb, \
.result_ctx = _result_ctx \
}
/**
* @brief GMF AFE Manager Feature Configuration
*
* This structure defines the feature enable states for the AFE manager
* A value of `true` indicates that the feature is enabled, while `false` indicates it is disabled
*/
typedef struct {
bool wakeup; /*!< Wake-up detection */
bool vad; /*!< Voice Activity Detection (VAD) */
bool ns; /*!< Noise Suppression (NS) */
bool aec; /*!< Acoustic Echo Cancellation (AEC) */
bool se; /*!< Speech Enhancement (SE) */
} esp_gmf_afe_manager_features_t;
/**
* @brief Create an AFE Manager instance
*
* @param[in] cfg Pointer to the AFE manager configuration structure
* @param[out] handle Pointer to the created AFE manager handle
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_FAIL Failed to create the AFE manager
* - ESP_GMF_ERR_MEMORY_LACK Insufficient memory allocation
*/
esp_gmf_err_t esp_gmf_afe_manager_create(esp_gmf_afe_manager_cfg_t *cfg, esp_gmf_afe_manager_handle_t *handle);
/**
* @brief Destroy an AFE Manager instance
*
* @param[in] handle AFE manager handle to be destroyed
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_INVALID_ARG Invalid handle
*/
esp_gmf_err_t esp_gmf_afe_manager_destroy(esp_gmf_afe_manager_handle_t handle);
/**
* @brief Set the audio input read callback for the AFE Manager
*
* @note If the read callback is set to `NULL`, the AFE Manager will be suspended
*
* @param[in] handle AFE manager handle
* @param[in] read_cb Function pointer to the read callback
* @param[in] read_ctx User-defined context to be passed to the callback
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_INVALID_ARG Invalid arguments
*/
esp_gmf_err_t esp_gmf_afe_manager_set_read_cb(esp_gmf_afe_manager_handle_t handle, esp_gmf_afe_manager_read_cb_t read_cb, void *read_ctx);
/**
* @brief Register a processing result callback for the AFE Manager
*
* @param[in] handle AFE manager handle
* @param[in] proc_cb Function pointer to the result callback
* @param[in] user_ctx User-defined context to be passed to the callback
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_INVALID_ARG Invalid arguments
*/
esp_gmf_err_t esp_gmf_afe_manager_set_result_cb(esp_gmf_afe_manager_handle_t handle, esp_gmf_afe_manager_result_cb_t proc_cb, void *user_ctx);
/**
* @brief Suspend or resume the AFE Manager
*
* @param[in] handle AFE manager handle
* @param[in] suspend `true` to suspend, `false` to resume
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_INVALID_ARG Invalid arguments
*/
esp_gmf_err_t esp_gmf_afe_manager_suspend(esp_gmf_afe_manager_handle_t handle, bool suspend);
/**
* @brief Enable or disable specific features in the AFE Manager
*
* @param[in] handle AFE manager handle
* @param[in] feature Feature to be configured (see `esp_gmf_afe_feature_t`)
* @param[in] enable `true` to enable, `false` to disable
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_INVALID_ARG Invalid arguments
*/
esp_gmf_err_t esp_gmf_afe_manager_enable_features(esp_gmf_afe_manager_handle_t handle, esp_gmf_afe_feature_t feature, bool enable);
/**
* @brief Retrieve the current feature enable states of the AFE Manager
*
* @param[in] handle AFE manager handle
* @param[out] features Pointer to a structure to store the feature states
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_INVALID_ARG Invalid arguments
*/
esp_gmf_err_t esp_gmf_afe_manager_get_features(esp_gmf_afe_manager_handle_t handle, esp_gmf_afe_manager_features_t *features);
/**
* @brief Get the processing chunk size for the AFE Manager
*
* @note The chunk size represents the number of audio samples per channel.
* The AFE Manager processes data in fixed-size chunks.
*
* @param[in] handle AFE manager handle
* @param[out] size Pointer to store the chunk size (unit: samples)
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_INVALID_ARG Invalid arguments
*/
esp_gmf_err_t esp_gmf_afe_manager_get_chunk_size(esp_gmf_afe_manager_handle_t handle, size_t *size);
/**
* @brief Retrieve the number of input channels for the AFE Manager
*
* @param[in] handle AFE manager handle
* @param[out] ch_num Pointer to store the number of channels
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_INVALID_ARG Invalid arguments
*/
esp_gmf_err_t esp_gmf_afe_manager_get_input_ch_num(esp_gmf_afe_manager_handle_t handle, uint8_t *ch_num);
#ifdef __cplusplus
}
#endif /* __cplusplus */
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/include/esp_gmf_ai_audio_methods.h
+20
View File
@@ -0,0 +1,20 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#pragma once
#ifdef __cplusplus
extern "C" {
#endif /* __cplusplus */
// AFE Method
#define ESP_GMF_METHOD_AFE_START_VCMD_DET "start_vcmd_det"
#define ESP_GMF_METHOD_AFE_START_VCMD_DET_ARG_EN "vcmd_det"
#ifdef __cplusplus
}
#endif /* __cplusplus */
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/include/esp_gmf_wn.h
+77
View File
@@ -0,0 +1,77 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#pragma once
#include "esp_gmf_element.h"
#include "esp_gmf_err.h"
#include "esp_wn_iface.h"
#include "model_path.h"
#ifdef __cplusplus
extern "C" {
#endif /* __cplusplus */
/**
* @brief Callback type for WakeNet detection
*
* @param[in] handle Handle to the WakeNet object
* @param[in] trigger_ch The microphone channel that triggered the detection
* @param[in] user_ctx User context passed during initialization
*/
typedef void (*esp_wn_detect_cb_t)(esp_gmf_element_handle_t handle, int32_t trigger_ch, void *user_ctx);
/**
* @brief Configuration structure for WakeNet
*
* @note The input format, same as afe config:
* `M` to represent the microphone channel,
* `R` to represent the playback reference channel,
* `N` to represent an unknown or unused channel
* For example, input_format="MMNR" indicates that the input data consists of four channels,
* which are the microphone channel, the microphone channel, an unused channel, and the playback channel
*/
typedef struct {
srmodel_list_t *models; /*!< Model list containing wake word models */
det_mode_t det_mode; /*!< Detection mode */
char *input_format; /*!< Input format */
esp_wn_detect_cb_t detect_cb; /*!< Detection callback function */
void *user_ctx; /*!< User context to be passed to the callback function */
} esp_gmf_wn_cfg_t;
/**
* @brief Initialize the WakeNet element
*
* @param[in] config Pointer to the configuration structure
* @param[out] handle Pointer to the handle to be initialized
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_INVALID_ARG Invalid argument
* - ESP_GMF_ERR_MEMORY_LACK Memory allocation failed
* - ESP_GMF_ERR_FAIL Other failures
*/
esp_gmf_err_t esp_gmf_wn_init(esp_gmf_wn_cfg_t *config, esp_gmf_element_handle_t *handle);
/**
* @brief Set the voice trigger detection callback for WakeNet
* This function registers a user-defined callback that will be invoked
* when WakeNet detects a wake word
*
* @param[in] handle Handle to the WakeNet element
* @param[in] detect_cb Callback function to be called on wake word detection
* @param[in] ctx User-defined context to be passed to the callback
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_INVALID_ARG Invalid argument
*/
esp_gmf_err_t esp_gmf_wn_set_detect_cb(esp_gmf_element_handle_t handle, esp_wn_detect_cb_t detect_cb, void *ctx);
#ifdef __cplusplus
}
#endif /* __cplusplus */
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/private_include/esp_gmf_ch_sort.h
+56
View File
@@ -0,0 +1,56 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#pragma once
#include "esp_types.h"
#ifdef __cplusplus
extern "C" {
#endif /* __cplusplus */
/**
* @brief Sort input data according to the input format and convert data layout
*
* @note This function performs two operations:
* 1. Channel sorting: Keeps microphone (M) channels in original order, removes unused (N) channels,
* and places reference (R) channels at the end
* 2. Layout conversion: Converts data from interleaved format (M0[0]R[0]M1[0]N[0]M0[1]R[1]M1[1]N[1]...) to block format (M0[0]M0[1]...M1[0]M1[1]...R[0]R[1]...)
*
* @param[in] input_data Pointer to the interleaved input data buffer
* @param[in] input_format Input format string (e.g., "MMNR" for 2 mic channels, 1 unused, 1 reference)
* @param[in] num_samples Number of samples per channel
* @param[in] num_channels Total number of channels in the input data
* @param[out] sorted_data Pointer to the output buffer for block-formatted sorted data
* Data will be arranged as: [mic_samples][ref_samples]
*/
static inline void esp_gmf_sort_with_format(int16_t *input_data, const char *input_format, int num_samples, int num_channels, int16_t *sorted_data)
{
int mic_index = 0;
int ref_index = 0;
for (int ch = 0; ch < num_channels; ++ch) {
if (input_format[ch] == 'M') {
for (int sample = 0; sample < num_samples; ++sample) {
sorted_data[mic_index++] = input_data[sample * num_channels + ch];
}
}
}
ref_index = mic_index;
for (int ch = 0; ch < num_channels; ++ch) {
if (input_format[ch] == 'R') {
for (int sample = 0; sample < num_samples; ++sample) {
sorted_data[ref_index++] = input_data[sample * num_channels + ch];
}
}
}
}
#ifdef __cplusplus
}
#endif /* __cplusplus */
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/src/esp_gmf_aec.c
+249
View File
@@ -0,0 +1,249 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#include <stdint.h>
#include <stdio.h>
#include <string.h>
#include "esp_log.h"
#include "esp_afe_config.h"
#include "esp_afe_aec.h"
#include "esp_gmf_payload.h"
#include "esp_gmf_aec.h"
#include "esp_gmf_audio_element.h"
#include "esp_gmf_err.h"
#include "esp_gmf_job.h"
#include "esp_gmf_node.h"
#include "esp_gmf_oal_mem.h"
#include "esp_gmf_cache.h"
#include "esp_gmf_cap.h"
#include "esp_gmf_caps_def.h"
#define ESP_AEC_BYTES_PER_SAMPLE (sizeof(uint16_t)) // ESP AEC processes data at 16 kHz, 16-bit, and single-channel
typedef struct {
esp_gmf_audio_element_t parent;
afe_aec_handle_t *aec_handle;
uint32_t chunk_size;
uint32_t frame_len;
esp_gmf_cache_t *cache;
esp_gmf_payload_t *in_load;
} gmf_aec_t;
static const char *TAG = "GMF_AEC";
static esp_gmf_err_t gmf_aec_new(void *cfg, esp_gmf_obj_handle_t *handle)
{
ESP_GMF_NULL_CHECK(TAG, cfg, { return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_NULL_CHECK(TAG, handle, { return ESP_GMF_ERR_INVALID_ARG;});
*handle = NULL;
esp_gmf_aec_cfg_t *aec_cfg = (esp_gmf_aec_cfg_t *)cfg;
esp_gmf_obj_handle_t new_obj = NULL;
esp_gmf_err_t ret = esp_gmf_aec_init(aec_cfg, &new_obj);
if (ret != ESP_GMF_ERR_OK) {
return ret;
}
*handle = (void *)new_obj;
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t gmf_aec_received_event_handler(esp_gmf_event_pkt_t *evt, void *ctx)
{
ESP_GMF_NULL_CHECK(TAG, evt, { return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_NULL_CHECK(TAG, ctx, { return ESP_GMF_ERR_INVALID_ARG;});
esp_gmf_element_handle_t self = (esp_gmf_element_handle_t)ctx;
esp_gmf_element_handle_t el = evt->from;
esp_gmf_event_state_t state = ESP_GMF_EVENT_STATE_NONE;
esp_gmf_element_get_state(self, &state);
esp_gmf_element_handle_t prev = NULL;
esp_gmf_element_get_prev_el(self, &prev);
if ((state == ESP_GMF_EVENT_STATE_NONE) || (prev == el)) {
if (evt->sub == ESP_GMF_INFO_SOUND) {
esp_gmf_info_sound_t info = {0};
memcpy(&info, evt->payload, evt->payload_size);
ESP_LOGD(TAG, "RECV info, from: %s-%p, next: %p, self: %s-%p, type: %x, state: %s, rate: %d, ch: %d, bits: %d",
OBJ_GET_TAG(el), el, esp_gmf_node_for_next((esp_gmf_node_t *)el), OBJ_GET_TAG(self), self, evt->type,
esp_gmf_event_get_state_str(state), info.sample_rates, info.channels, info.bits);
if (info.sample_rates != 16000 || info.bits != 16) {
ESP_LOGE(TAG, "Unsupported format, rate: %d, bits: %d", info.sample_rates, info.bits);
return ESP_GMF_ERR_NOT_SUPPORT;
}
esp_gmf_element_set_state(self, ESP_GMF_EVENT_STATE_INITIALIZED);
}
}
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t gmf_aec_destroy(esp_gmf_audio_element_handle_t self)
{
ESP_GMF_NULL_CHECK(TAG, self, { return ESP_GMF_ERR_INVALID_ARG;});
gmf_aec_t *gmf_aec = (gmf_aec_t *)self;
if (OBJ_GET_CFG(self)) {
esp_gmf_oal_free(OBJ_GET_CFG(self));
}
if (gmf_aec->aec_handle) {
afe_aec_destroy(gmf_aec->aec_handle);
gmf_aec->aec_handle = NULL;
}
if (gmf_aec->cache) {
esp_gmf_cache_delete(gmf_aec->cache);
gmf_aec->cache = NULL;
}
ESP_LOGD(TAG, "Destroyed");
esp_gmf_audio_el_deinit(self);
esp_gmf_oal_free(self);
return ESP_GMF_ERR_OK;
}
static esp_gmf_job_err_t gmf_aec_open(esp_gmf_audio_element_handle_t self, void *para)
{
gmf_aec_t *gmf_aec = (gmf_aec_t *)self;
esp_gmf_aec_cfg_t *cfg = OBJ_GET_CFG(self);
gmf_aec->aec_handle = afe_aec_create(cfg->input_format, cfg->filter_len, cfg->type, cfg->mode);
ESP_GMF_NULL_CHECK(TAG, gmf_aec->aec_handle, { return ESP_GMF_JOB_ERR_FAIL;});
gmf_aec->chunk_size = afe_aec_get_chunksize(gmf_aec->aec_handle);
gmf_aec->frame_len = gmf_aec->chunk_size * gmf_aec->aec_handle->pcm_config.total_ch_num * ESP_AEC_BYTES_PER_SAMPLE;
esp_gmf_cache_new(gmf_aec->frame_len, &gmf_aec->cache);
ESP_GMF_NULL_CHECK(TAG, gmf_aec->cache, { return ESP_GMF_JOB_ERR_FAIL;});
ESP_GMF_ELEMENT_IN_PORT_ATTR_SET(ESP_GMF_ELEMENT_GET(self)->in_attr, ESP_GMF_EL_PORT_CAP_SINGLE, 16, 0,
ESP_GMF_PORT_TYPE_BLOCK | ESP_GMF_PORT_TYPE_BYTE, gmf_aec->frame_len);
ESP_LOGI(TAG, "GMF AEC open, frame_len: %lu, nch %d, chunksize %lu", gmf_aec->frame_len, gmf_aec->aec_handle->pcm_config.total_ch_num, gmf_aec->chunk_size);
esp_gmf_info_sound_t snd_info = {0};
snd_info.sample_rates = 16000;
snd_info.bits = 16;
snd_info.channels = 1;
esp_gmf_element_notify_snd_info(self, &snd_info);
return ESP_GMF_JOB_ERR_OK;
}
static esp_gmf_job_err_t gmf_aec_process(esp_gmf_audio_element_handle_t self, void *para)
{
int ret = ESP_GMF_JOB_ERR_OK;
gmf_aec_t *gmf_aec = (gmf_aec_t *)self;
esp_gmf_port_handle_t in_port = ESP_GMF_ELEMENT_GET(self)->in;
esp_gmf_port_handle_t out_port = ESP_GMF_ELEMENT_GET(self)->out;
esp_gmf_payload_t *out_load = NULL;
esp_gmf_payload_t *cache_load = NULL;
esp_gmf_err_io_t load_ret = 0;
bool need_load = false;
esp_gmf_cache_ready_for_load(gmf_aec->cache, &need_load);
if (need_load) {
load_ret = esp_gmf_port_acquire_in(in_port, &gmf_aec->in_load, gmf_aec->frame_len, in_port->wait_ticks);
ESP_GMF_PORT_ACQUIRE_IN_CHECK(TAG, load_ret, ret, { goto __quit;});
esp_gmf_cache_load(gmf_aec->cache, gmf_aec->in_load);
}
esp_gmf_cache_acquire(gmf_aec->cache, gmf_aec->frame_len, &cache_load);
if (cache_load->valid_size != gmf_aec->frame_len) {
if (cache_load->is_done == true) {
ret = ESP_GMF_JOB_ERR_DONE;
} else {
ret = ESP_GMF_JOB_ERR_CONTINUE;
}
ESP_LOGD(TAG, "Return Continue, size:%u", cache_load->valid_size);
goto __quit;
}
load_ret = esp_gmf_port_acquire_out(out_port, &out_load, gmf_aec->chunk_size * ESP_AEC_BYTES_PER_SAMPLE, ESP_GMF_MAX_DELAY);
ESP_GMF_PORT_ACQUIRE_OUT_CHECK(TAG, load_ret, ret, { goto __quit;});
if (out_load->buf_length < gmf_aec->chunk_size * ESP_AEC_BYTES_PER_SAMPLE) {
ret = ESP_GMF_JOB_ERR_FAIL;
ESP_LOGE(TAG, "Output buffer is not enough");
goto __quit;
}
afe_aec_process(gmf_aec->aec_handle, (int16_t *)cache_load->buf, (int16_t *)out_load->buf);
out_load->valid_size = gmf_aec->chunk_size * ESP_AEC_BYTES_PER_SAMPLE;
out_load->is_done = cache_load->is_done;
esp_gmf_cache_ready_for_load(gmf_aec->cache, &need_load);
if (need_load == false) {
ret = ESP_GMF_JOB_ERR_TRUNCATE;
} else {
ret = (out_load->is_done == true ? ESP_GMF_JOB_ERR_DONE : ESP_GMF_JOB_ERR_OK);
}
__quit:
if (out_load) {
esp_gmf_port_release_out(out_port, out_load, ESP_GMF_MAX_DELAY);
}
if (gmf_aec->in_load && (ret != ESP_GMF_JOB_ERR_TRUNCATE)) {
esp_gmf_port_release_in(in_port, gmf_aec->in_load, ESP_GMF_MAX_DELAY);
gmf_aec->in_load = NULL;
}
if (cache_load) {
esp_gmf_cache_release(gmf_aec->cache, cache_load);
}
return ret;
}
static esp_gmf_job_err_t gmf_aec_close(esp_gmf_audio_element_handle_t self, void *para)
{
gmf_aec_t *gmf_aec = (gmf_aec_t *)self;
if (gmf_aec->aec_handle) {
afe_aec_destroy(gmf_aec->aec_handle);
gmf_aec->aec_handle = NULL;
}
if (gmf_aec->cache) {
esp_gmf_cache_delete(gmf_aec->cache);
gmf_aec->cache = NULL;
}
return ESP_GMF_JOB_ERR_OK;
}
static esp_gmf_err_t _load_aec_caps_func(esp_gmf_element_handle_t handle)
{
esp_gmf_cap_t *caps = NULL;
esp_gmf_cap_t aec_caps = {0};
aec_caps.cap_eightcc = ESP_GMF_CAPS_AUDIO_AEC;
aec_caps.attr_fun = NULL;
int ret = esp_gmf_cap_append(&caps, &aec_caps);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, { return ret;}, "Failed to create capability");
esp_gmf_element_t *el = (esp_gmf_element_t *)handle;
el->caps = caps;
return ESP_GMF_ERR_OK;
}
esp_gmf_err_t esp_gmf_aec_init(esp_gmf_aec_cfg_t *config, esp_gmf_obj_handle_t *handle)
{
ESP_GMF_NULL_CHECK(TAG, config, { return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_NULL_CHECK(TAG, handle, { return ESP_GMF_ERR_INVALID_ARG;});
*handle = NULL;
gmf_aec_t *gmf_aec = esp_gmf_oal_calloc(1, sizeof(gmf_aec_t));
ESP_GMF_MEM_VERIFY(TAG, gmf_aec, { return ESP_GMF_ERR_MEMORY_LACK;}, "aec", sizeof(gmf_aec_t));
esp_gmf_obj_t *obj = (esp_gmf_obj_t *)gmf_aec;
obj->new_obj = gmf_aec_new;
obj->del_obj = gmf_aec_destroy;
esp_gmf_aec_cfg_t *obj_cfg = esp_gmf_oal_calloc(1, sizeof(esp_gmf_aec_cfg_t));
memcpy(obj_cfg, config, sizeof(esp_gmf_aec_cfg_t));
esp_gmf_err_t ret = esp_gmf_obj_set_config(obj, obj_cfg, sizeof(esp_gmf_aec_cfg_t));
ESP_GMF_RET_ON_NOT_OK(TAG, ret, goto __failed, "Failed set OBJ configuration");
ret = esp_gmf_obj_set_tag(obj, "ai_aec");
ESP_GMF_RET_ON_NOT_OK(TAG, ret, goto __failed, "Failed set OBJ tag");
esp_gmf_element_cfg_t el_cfg = {0};
ESP_GMF_ELEMENT_IN_PORT_ATTR_SET(el_cfg.in_attr, ESP_GMF_EL_PORT_CAP_SINGLE, 16, 0,
ESP_GMF_PORT_TYPE_BLOCK | ESP_GMF_PORT_TYPE_BYTE, 1024);
ESP_GMF_ELEMENT_OUT_PORT_ATTR_SET(el_cfg.out_attr, ESP_GMF_EL_PORT_CAP_SINGLE, 16, 0,
ESP_GMF_PORT_TYPE_BLOCK | ESP_GMF_PORT_TYPE_BYTE, 1024);
el_cfg.dependency = true;
esp_gmf_audio_el_init(gmf_aec, &el_cfg);
ESP_GMF_ELEMENT_GET(gmf_aec)->ops.open = gmf_aec_open;
ESP_GMF_ELEMENT_GET(gmf_aec)->ops.process = gmf_aec_process;
ESP_GMF_ELEMENT_GET(gmf_aec)->ops.close = gmf_aec_close;
ESP_GMF_ELEMENT_GET(gmf_aec)->ops.event_receiver = gmf_aec_received_event_handler;
ESP_GMF_ELEMENT_GET(gmf_aec)->ops.load_caps = _load_aec_caps_func;
*handle = obj;
return ESP_GMF_ERR_OK;
__failed:
esp_gmf_obj_delete(obj);
return ret;
}
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/src/esp_gmf_afe.c
+820
View File
@@ -0,0 +1,820 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#include <stdbool.h>
#include <stdint.h>
#include <stdio.h>
#include <string.h>
#include "esp_gmf_obj.h"
#include "freertos/idf_additions.h"
#include "freertos/queue.h"
#include "esp_log.h"
#include "esp_timer.h"
#include "esp_gmf_audio_element.h"
#include "esp_gmf_err.h"
#include "esp_gmf_job.h"
#include "esp_gmf_new_databus.h"
#include "esp_gmf_oal_mem.h"
#include "esp_wn_iface.h"
#include "esp_mn_iface.h"
#include "esp_mn_models.h"
#include "esp_process_sdkconfig.h"
#include "esp_gmf_afe_manager.h"
#include "esp_gmf_afe.h"
#include "esp_gmf_cap.h"
#include "esp_gmf_caps_def.h"
#include "esp_gmf_method.h"
#include "esp_gmf_ai_audio_methods.h"
#define AFE_DEFAULT_DATA_SIZE (2048)
#define WAKEUP_ST_SET(handle, st) (handle->wakeup_state = st)
/**
* @brief Represents the various states of the wakeup process in the system
*
* This enumeration defines the different states that the system can be in
* during the wakeup and speech processing lifecycle.
*/
typedef enum {
ST_IDLE,
ST_WAKEUP,
ST_SPEECHING,
ST_WAIT_FOR_SLEEP,
} wakeup_state_t;
/**
* @brief Enumeration representing various wakeup events in the system
*
* This enumeration defines the different types of events that can trigger
* the state changing in the system. Each event corresponds to a specific
* condition or state.
*/
typedef enum {
ET_NOISE_DECT,
ET_SPEECH_DECT,
ET_WWE_DECT,
ET_WAKEUP_TIMER_EXPIRED,
ET_KEEP_WAKE_MODIFIED,
ET_TRIGGER_WAKEUP,
ET_TRIGGER_SLEEP,
ET_UNKNOWN
} wakeup_event_t;
typedef struct {
wakeup_event_t event;
union {
struct {
bool enable;
} keep_awake;
struct {
bool state;
} trigger;
} payload;
} afe_evt_msg_t;
/**
* @brief Structure representing the ESP GMF AFE component
*/
typedef struct {
esp_gmf_audio_element_t parent;
esp_gmf_db_handle_t in_db;
esp_gmf_db_handle_t out_db;
wakeup_state_t wakeup_state;
wakeup_event_t last_event;
bool origin_vad_enable;
bool keep_wake;
esp_timer_handle_t wakeup_timer;
SemaphoreHandle_t wake_st_lock;
model_iface_data_t *mn_handle;
bool mn_detecting;
SemaphoreHandle_t mn_lock;
esp_mn_state_t mn_state;
esp_mn_iface_t *multinet;
QueueHandle_t msg_queue;
} esp_gmf_afe_t;
static const char *TAG = "GMF_AFE";
static const char *state_str[] = {
"ST_IDLE",
"ST_WAKEUP",
"ST_SPEECHING",
"ST_WAIT_FOR_SLEEP",
};
static const char *event_str[] = {
"ET_NOISE_DECT",
"ET_SPEECH_DECT",
"ET_WWE_DECT",
"ET_WAKEUP_TIMER_EXPIRED",
"ET_KEEP_WAKE_MODIFIED",
"ET_TRIGGER_WAKEUP",
"ET_TRIGGER_SLEEP",
"ET_UNKNOWN",
};
static wakeup_event_t result_2_event(afe_fetch_result_t *result)
{
if (result->wakeup_state == WAKENET_DETECTED) {
return ET_WWE_DECT;
}
if (result->vad_state == VAD_SILENCE) {
return ET_NOISE_DECT;
}
if (result->vad_state == VAD_SPEECH) {
return ET_SPEECH_DECT;
}
return ET_UNKNOWN;
}
static void event_2_user(esp_gmf_afe_t *gmf_afe, int event, void *event_data, size_t dlen)
{
esp_gmf_afe_cfg_t *cfg = OBJ_GET_CFG(gmf_afe);
if (cfg->event_cb) {
esp_gmf_afe_evt_t afe_event = {
.type = event,
.event_data = event_data,
.data_len = dlen,
};
cfg->event_cb(gmf_afe, &afe_event, cfg->event_ctx);
}
}
static void afe_result_2_wakeup_info(afe_fetch_result_t *result, esp_gmf_afe_wakeup_info_t *info)
{
info->wake_word_index = result->wake_word_index;
info->wakenet_model_index = result->wakenet_model_index;
info->data_volume = result->data_volume;
}
static void wakeup_state_reset(esp_gmf_afe_t *gmf_afe)
{
esp_timer_stop(gmf_afe->wakeup_timer);
WAKEUP_ST_SET(gmf_afe, ST_IDLE);
esp_gmf_afe_manager_features_t feat = { 0 };
esp_gmf_afe_cfg_t *cfg = OBJ_GET_CFG(gmf_afe);
if (cfg && cfg->afe_manager) {
esp_gmf_afe_manager_get_features(cfg->afe_manager, &feat);
if (feat.wakeup && gmf_afe->origin_vad_enable) {
esp_gmf_afe_manager_enable_features(cfg->afe_manager, ESP_AFE_FEATURE_VAD, false);
}
}
}
static void wakeup_timer_start(esp_gmf_afe_t *gmf_afe)
{
esp_gmf_afe_cfg_t *cfg = OBJ_GET_CFG(gmf_afe);
int timeout = 0;
if (gmf_afe->wakeup_state == ST_WAKEUP) {
timeout = cfg->wakeup_time;
} else if (gmf_afe->wakeup_state == ST_WAIT_FOR_SLEEP) {
timeout = cfg->wakeup_end;
}
esp_timer_stop(gmf_afe->wakeup_timer);
if (timeout) {
esp_timer_start_once(gmf_afe->wakeup_timer, timeout * 1000);
}
}
static void wakeup_state_update(esp_gmf_afe_t *gmf_afe, wakeup_event_t event, void *event_data, size_t len)
{
esp_gmf_afe_cfg_t *cfg = OBJ_GET_CFG(gmf_afe);
esp_gmf_afe_manager_features_t afe_feat = {0};
int32_t user_event = -1;
if (event == ET_KEEP_WAKE_MODIFIED || gmf_afe->last_event != event) {
ESP_LOGV(TAG, "Recorder update state, cur %s, event %s", state_str[gmf_afe->wakeup_state], event_str[event]);
gmf_afe->last_event = event;
} else {
return;
}
xSemaphoreTake(gmf_afe->wake_st_lock, portMAX_DELAY);
if ((event == ET_WWE_DECT || event == ET_TRIGGER_WAKEUP) && gmf_afe->wakeup_state != ST_IDLE) {
wakeup_state_reset(gmf_afe);
}
esp_gmf_afe_manager_get_features(cfg->afe_manager, &afe_feat);
switch (gmf_afe->wakeup_state) {
case ST_IDLE: {
if (event == ET_WWE_DECT || event == ET_TRIGGER_WAKEUP) {
WAKEUP_ST_SET(gmf_afe, ST_WAKEUP);
if (!gmf_afe->keep_wake) {
wakeup_timer_start(gmf_afe);
}
if (gmf_afe->origin_vad_enable) {
esp_gmf_afe_manager_enable_features(cfg->afe_manager, ESP_AFE_FEATURE_VAD, true);
}
user_event = ESP_GMF_AFE_EVT_WAKEUP_START;
} else if (event == ET_SPEECH_DECT && afe_feat.wakeup == false) {
WAKEUP_ST_SET(gmf_afe, ST_SPEECHING);
user_event = ESP_GMF_AFE_EVT_VAD_START;
}
break;
}
case ST_WAKEUP: {
if (event == ET_SPEECH_DECT) {
esp_timer_stop(gmf_afe->wakeup_timer);
WAKEUP_ST_SET(gmf_afe, ST_SPEECHING);
user_event = ESP_GMF_AFE_EVT_VAD_START;
} else if (event == ET_KEEP_WAKE_MODIFIED) {
if (gmf_afe->keep_wake) {
esp_timer_stop(gmf_afe->wakeup_timer);
} else {
wakeup_timer_start(gmf_afe);
}
} else if (event == ET_WAKEUP_TIMER_EXPIRED) {
WAKEUP_ST_SET(gmf_afe, ST_IDLE);
esp_timer_stop(gmf_afe->wakeup_timer);
user_event = ESP_GMF_AFE_EVT_WAKEUP_END;
} else if (event == ET_TRIGGER_SLEEP) {
WAKEUP_ST_SET(gmf_afe, ST_IDLE);
esp_timer_stop(gmf_afe->wakeup_timer);
if (gmf_afe->origin_vad_enable) {
esp_gmf_afe_manager_enable_features(cfg->afe_manager, ESP_AFE_FEATURE_VAD, false);
}
user_event = ESP_GMF_AFE_EVT_WAKEUP_END;
}
break;
}
case ST_SPEECHING: {
if (event == ET_NOISE_DECT) {
if (afe_feat.wakeup == true) {
if (!gmf_afe->keep_wake) {
WAKEUP_ST_SET(gmf_afe, ST_WAIT_FOR_SLEEP);
wakeup_timer_start(gmf_afe);
} else {
WAKEUP_ST_SET(gmf_afe, ST_WAKEUP);
}
} else {
WAKEUP_ST_SET(gmf_afe, ST_IDLE);
}
user_event = ESP_GMF_AFE_EVT_VAD_END;
} else if (event == ET_TRIGGER_SLEEP) {
WAKEUP_ST_SET(gmf_afe, ST_IDLE);
if (gmf_afe->origin_vad_enable) {
esp_gmf_afe_manager_enable_features(cfg->afe_manager, ESP_AFE_FEATURE_VAD, false);
}
user_event = ESP_GMF_AFE_EVT_WAKEUP_END;
}
break;
}
case ST_WAIT_FOR_SLEEP: {
if (event == ET_SPEECH_DECT) {
WAKEUP_ST_SET(gmf_afe, ST_SPEECHING);
user_event = ESP_GMF_AFE_EVT_VAD_START;
} else if (event == ET_WAKEUP_TIMER_EXPIRED) {
WAKEUP_ST_SET(gmf_afe, ST_IDLE);
if (gmf_afe->origin_vad_enable) {
esp_gmf_afe_manager_enable_features(cfg->afe_manager, ESP_AFE_FEATURE_VAD, false);
}
user_event = ESP_GMF_AFE_EVT_WAKEUP_END;
} else if (event == ET_KEEP_WAKE_MODIFIED) {
if (gmf_afe->keep_wake) {
esp_timer_stop(gmf_afe->wakeup_timer);
WAKEUP_ST_SET(gmf_afe, ST_WAKEUP);
} else {
wakeup_timer_start(gmf_afe);
}
} else if (event == ET_TRIGGER_SLEEP) {
WAKEUP_ST_SET(gmf_afe, ST_IDLE);
esp_timer_stop(gmf_afe->wakeup_timer);
if (gmf_afe->origin_vad_enable) {
esp_gmf_afe_manager_enable_features(cfg->afe_manager, ESP_AFE_FEATURE_VAD, false);
}
user_event = ESP_GMF_AFE_EVT_WAKEUP_END;
}
break;
}
default:
break;
}
xSemaphoreGive(gmf_afe->wake_st_lock);
if (user_event != -1) {
event_2_user(gmf_afe, user_event, event_data, len);
}
}
static void wakeup_afe_monitor(afe_fetch_result_t *result, void *user_ctx)
{
wakeup_event_t event = result_2_event(result);
if (event == ET_WWE_DECT) {
esp_gmf_afe_wakeup_info_t info = {0};
afe_result_2_wakeup_info(result, &info);
wakeup_state_update(user_ctx, event, &info, sizeof(esp_gmf_afe_wakeup_info_t));
} else {
wakeup_state_update(user_ctx, event, NULL, 0);
}
}
static void wakeup_timer_expired(void *arg)
{
esp_gmf_afe_t *gmf_afe = arg;
wakeup_state_update(gmf_afe, ET_WAKEUP_TIMER_EXPIRED, NULL, 0);
}
static esp_err_t vcmd_det_begin(esp_gmf_afe_t *gmf_afe)
{
if (!gmf_afe) {
return ESP_ERR_INVALID_ARG;
}
xSemaphoreTake(gmf_afe->mn_lock, portMAX_DELAY);
gmf_afe->mn_detecting = true;
gmf_afe->mn_state = ESP_MN_STATE_TIMEOUT;
xSemaphoreGive(gmf_afe->mn_lock);
return ESP_OK;
}
static esp_err_t vcmd_det_cancel(esp_gmf_afe_t *gmf_afe)
{
if (!gmf_afe) {
return ESP_ERR_INVALID_ARG;
}
xSemaphoreTake(gmf_afe->mn_lock, portMAX_DELAY);
gmf_afe->mn_detecting = false;
gmf_afe->mn_state = ESP_MN_STATE_TIMEOUT;
gmf_afe->multinet->clean(gmf_afe->mn_handle);
xSemaphoreGive(gmf_afe->mn_lock);
return ESP_OK;
}
static void mn_afe_monitor(afe_fetch_result_t *result, void *user_ctx)
{
esp_gmf_afe_t *gmf_afe = user_ctx;
if (gmf_afe->mn_detecting) {
if ((gmf_afe->mn_state != ESP_MN_STATE_DETECTING) && result->vad_cache_size) {
uint8_t *cache = (uint8_t *)result->vad_cache;
while (cache < ((uint8_t *)result->vad_cache + result->vad_cache_size)) {
gmf_afe->mn_state = gmf_afe->multinet->detect(gmf_afe->mn_handle, (int16_t *)cache);
if (gmf_afe->mn_state == ESP_MN_STATE_DETECTED) {
goto __proc_state;
}
cache += result->data_size;
}
}
gmf_afe->mn_state = gmf_afe->multinet->detect(gmf_afe->mn_handle, result->data);
__proc_state:
switch (gmf_afe->mn_state) {
case ESP_MN_STATE_DETECTED: {
esp_mn_results_t *mn_result = gmf_afe->multinet->get_results(gmf_afe->mn_handle);
esp_gmf_afe_vcmd_info_t vcmd_info = {
.phrase_id = mn_result->phrase_id[0],
.prob = mn_result->prob[0]};
strlcpy(vcmd_info.str, mn_result->string, sizeof(vcmd_info.str));
event_2_user(gmf_afe, vcmd_info.phrase_id, &vcmd_info, sizeof(esp_gmf_afe_vcmd_info_t));
break;
}
case ESP_MN_STATE_TIMEOUT: {
xSemaphoreTake(gmf_afe->mn_lock, portMAX_DELAY);
gmf_afe->mn_detecting = false;
xSemaphoreGive(gmf_afe->mn_lock);
event_2_user(gmf_afe, ESP_GMF_AFE_EVT_VCMD_DECT_TIMEOUT, NULL, 0);
break;
}
default:
break;
}
}
}
static void esp_gmf_afe_result_proc(afe_fetch_result_t *result, void *user_ctx)
{
esp_gmf_afe_t *gmf_afe = (esp_gmf_afe_t *)user_ctx;
if (result->data_size) {
esp_gmf_afe_manager_features_t feat = {0};
esp_gmf_afe_cfg_t *cfg = OBJ_GET_CFG(gmf_afe);
esp_gmf_afe_manager_get_features(cfg->afe_manager, &feat);
if (feat.wakeup || feat.vad) {
wakeup_afe_monitor(result, user_ctx);
}
if (cfg->vcmd_detect_en) {
mn_afe_monitor(result, user_ctx);
}
uint32_t available_size = 0;
esp_gmf_db_get_available(gmf_afe->out_db, &available_size);
if (available_size < result->data_size) {
esp_gmf_data_bus_block_t blk = {0};
blk.buf_length = result->data_size - available_size;
ESP_GMF_RET_ON_FAIL(TAG, esp_gmf_db_acquire_read(gmf_afe->out_db, &blk, blk.buf_length, ESP_GMF_MAX_DELAY), { return;}, "DB failed to acquire read");
ESP_GMF_RET_ON_FAIL(TAG, esp_gmf_db_release_read(gmf_afe->out_db, &blk, ESP_GMF_MAX_DELAY), { return;}, "DB failed to release read");
}
esp_gmf_data_bus_block_t blk = {0};
ESP_GMF_RET_ON_FAIL(TAG, esp_gmf_db_acquire_write(gmf_afe->out_db, &blk, result->data_size, ESP_GMF_MAX_DELAY), { return;}, "DB failed to acquire write");
blk.buf = (uint8_t *)result->data;
blk.valid_size = result->data_size;
ESP_GMF_RET_ON_FAIL(TAG, esp_gmf_db_release_write(gmf_afe->out_db, &blk, ESP_GMF_MAX_DELAY), { return;}, "DB failed to release write");
}
}
static int32_t esp_gmf_afe_read_cb(void *buffer, int buf_sz, void *user_ctx, TickType_t ticks)
{
esp_gmf_afe_t *gmf_afe = (esp_gmf_afe_t *)user_ctx;
esp_gmf_data_bus_block_t blk = {0};
blk.buf = buffer;
blk.buf_length = buf_sz;
ESP_LOGD(TAG, "Feed %zu", blk.buf_length);
ESP_GMF_RET_ON_FAIL(TAG, esp_gmf_db_acquire_read(gmf_afe->in_db, &blk, buf_sz, ticks), { return 0;}, "DB failed to acquire read");
ESP_GMF_RET_ON_FAIL(TAG, esp_gmf_db_release_read(gmf_afe->in_db, &blk, ticks), { return 0;}, "DB failed to release read");
return buf_sz;
}
static esp_gmf_err_t esp_gmf_afe_new(void *cfg, esp_gmf_obj_handle_t *handle)
{
ESP_GMF_MEM_CHECK(TAG, cfg, { return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_MEM_CHECK(TAG, handle, { return ESP_GMF_ERR_INVALID_ARG;});
esp_gmf_afe_cfg_t *gmf_afe_manager_cfg = (esp_gmf_afe_cfg_t *)cfg;
esp_gmf_obj_handle_t new_obj = NULL;
int ret = ESP_GMF_ERR_OK;
ret = esp_gmf_afe_init(gmf_afe_manager_cfg, &new_obj);
if (ret != ESP_GMF_ERR_OK) {
return ret;
}
*handle = (void *)new_obj;
ESP_LOGI(TAG, "New an object,%s-%p", OBJ_GET_TAG(new_obj), new_obj);
return ret;
}
static esp_gmf_err_t esp_gmf_afe_destroy(esp_gmf_audio_element_handle_t self)
{
if (self) {
if (OBJ_GET_CFG(self)) {
esp_gmf_oal_free(OBJ_GET_CFG(self));
}
esp_gmf_audio_el_deinit(self);
esp_gmf_oal_free(self);
}
return ESP_GMF_ERR_OK;
}
static esp_gmf_job_err_t esp_gmf_afe_open(esp_gmf_audio_element_handle_t self, void *para)
{
esp_gmf_afe_t *gmf_afe = (esp_gmf_afe_t *)self;
esp_gmf_afe_cfg_t *cfg = OBJ_GET_CFG(self);
size_t buf_size = 0, chunk_size = 0;
uint8_t ch_num = 0;
esp_gmf_afe_manager_get_input_ch_num(cfg->afe_manager, &ch_num);
esp_gmf_afe_manager_get_chunk_size(cfg->afe_manager, &chunk_size);
buf_size = chunk_size * ch_num * sizeof(uint16_t);
ESP_GMF_ELEMENT_GET(self)->in_attr.data_size = buf_size;
ESP_GMF_ELEMENT_GET(self)->out_attr.data_size = chunk_size * 2;
esp_gmf_afe_manager_features_t feat = {0};
esp_gmf_afe_manager_get_features(cfg->afe_manager, &feat);
if (feat.wakeup || feat.vad) {
gmf_afe->wake_st_lock = xSemaphoreCreateMutex();
esp_timer_create_args_t wakeup_timer_cfg = {
.callback = wakeup_timer_expired,
.arg = gmf_afe,
.dispatch_method = ESP_TIMER_TASK,
.name = "wakeup_timer",
};
esp_timer_create(&wakeup_timer_cfg, &gmf_afe->wakeup_timer);
if (feat.wakeup && feat.vad) {
esp_gmf_afe_manager_enable_features(cfg->afe_manager, ESP_AFE_FEATURE_VAD, false);
gmf_afe->origin_vad_enable = true;
}
}
if (cfg->vcmd_detect_en) {
char *mn_name = esp_srmodel_filter(cfg->models, ESP_MN_PREFIX, cfg->mn_language);
ESP_GMF_NULL_CHECK(TAG, mn_name, return ESP_GMF_JOB_ERR_FAIL);
gmf_afe->multinet = esp_mn_handle_from_name(mn_name);
if (cfg->vcmd_timeout == 0) {
ESP_LOGW(TAG, "Voice command timeout configured as 0, reset to default: 5760 ms");
cfg->vcmd_timeout = 5760;
}
gmf_afe->mn_handle = gmf_afe->multinet->create(mn_name, cfg->vcmd_timeout);
ESP_GMF_NULL_CHECK(TAG, gmf_afe->mn_handle, return ESP_GMF_JOB_ERR_FAIL);
gmf_afe->mn_lock = xSemaphoreCreateMutex();
ESP_GMF_NULL_CHECK(TAG, gmf_afe->mn_lock, return ESP_GMF_JOB_ERR_FAIL);
gmf_afe->mn_state = ESP_MN_STATE_TIMEOUT;
esp_mn_commands_update_from_sdkconfig((esp_mn_iface_t *)gmf_afe->multinet, gmf_afe->mn_handle);
}
esp_gmf_db_new_ringbuf(2, buf_size, &gmf_afe->in_db);
esp_gmf_db_new_ringbuf(1, buf_size * 2 + (cfg->delay_samples * sizeof(uint16_t)), &gmf_afe->out_db);
esp_gmf_afe_manager_set_result_cb(cfg->afe_manager, esp_gmf_afe_result_proc, self);
esp_gmf_afe_manager_set_read_cb(cfg->afe_manager, esp_gmf_afe_read_cb, self);
if (gmf_afe->msg_queue == NULL) {
gmf_afe->msg_queue = xQueueCreate(4, sizeof(afe_evt_msg_t));
ESP_GMF_NULL_CHECK(TAG, gmf_afe->msg_queue, goto __failed);
}
esp_gmf_info_sound_t snd_info = {0};
snd_info.sample_rates = 16000;
snd_info.bits = 16;
snd_info.channels = 1;
esp_gmf_element_notify_snd_info(self, &snd_info);
return ESP_GMF_JOB_ERR_OK;
__failed:
esp_gmf_obj_delete(self);
return ESP_GMF_JOB_ERR_FAIL;
}
static esp_gmf_job_err_t esp_gmf_afe_close(esp_gmf_audio_element_handle_t self, void *para)
{
esp_gmf_afe_t *gmf_afe = (esp_gmf_afe_t *)self;
esp_gmf_afe_cfg_t *cfg = OBJ_GET_CFG(self);
if (cfg->vcmd_detect_en) {
vcmd_det_cancel(gmf_afe);
}
esp_gmf_afe_manager_set_read_cb(cfg->afe_manager, NULL, NULL);
esp_gmf_afe_manager_set_result_cb(cfg->afe_manager, NULL, NULL);
if (gmf_afe->in_db) {
esp_gmf_db_deinit(gmf_afe->in_db);
gmf_afe->in_db = NULL;
}
if (gmf_afe->out_db) {
esp_gmf_db_deinit(gmf_afe->out_db);
gmf_afe->out_db = NULL;
}
if (gmf_afe->wake_st_lock) {
vSemaphoreDelete(gmf_afe->wake_st_lock);
gmf_afe->wake_st_lock = NULL;
}
if (gmf_afe->wakeup_timer) {
esp_timer_stop(gmf_afe->wakeup_timer);
esp_timer_delete(gmf_afe->wakeup_timer);
gmf_afe->wakeup_timer = NULL;
}
if (gmf_afe->mn_lock) {
vSemaphoreDelete(gmf_afe->mn_lock);
gmf_afe->mn_lock = NULL;
}
if (gmf_afe->mn_handle) {
gmf_afe->multinet->destroy(gmf_afe->mn_handle);
gmf_afe->mn_handle = NULL;
}
if (gmf_afe->msg_queue) {
vQueueDelete(gmf_afe->msg_queue);
gmf_afe->msg_queue = NULL;
}
return ESP_GMF_JOB_ERR_OK;
}
static void esp_gmf_afe_cmd_handler(esp_gmf_audio_element_handle_t self)
{
esp_gmf_afe_t *gmf_afe = (esp_gmf_afe_t *)self;
if (gmf_afe->msg_queue) {
afe_evt_msg_t msg;
while (xQueueReceive(gmf_afe->msg_queue, &msg, 0) == pdTRUE) {
switch (msg.event) {
case ET_KEEP_WAKE_MODIFIED: {
xSemaphoreTake(gmf_afe->wake_st_lock, portMAX_DELAY);
gmf_afe->keep_wake = msg.payload.keep_awake.enable;
xSemaphoreGive(gmf_afe->wake_st_lock);
wakeup_state_update(gmf_afe, msg.event, NULL, 0);
break;
}
case ET_TRIGGER_WAKEUP:
case ET_TRIGGER_SLEEP: {
wakeup_state_update(gmf_afe, msg.event, NULL, 0);
break;
}
default:
break;
}
}
}
}
static esp_gmf_job_err_t esp_gmf_afe_proc(esp_gmf_audio_element_handle_t self, void *para)
{
int ret = 0;
esp_gmf_port_handle_t in_port = ESP_GMF_ELEMENT_GET(self)->in;
esp_gmf_port_handle_t out_port = ESP_GMF_ELEMENT_GET(self)->out;
esp_gmf_payload_t *in_load = NULL;
esp_gmf_payload_t *out_load = NULL;
esp_gmf_afe_t *gmf_afe = (esp_gmf_afe_t *)self;
esp_gmf_afe_cfg_t *cfg = OBJ_GET_CFG(self);
esp_gmf_afe_cmd_handler(self);
ret = esp_gmf_port_acquire_in(in_port, &in_load, ESP_GMF_ELEMENT_GET(self)->in_attr.data_size, ESP_GMF_MAX_DELAY);
if (ret < 0) {
ESP_LOGE(TAG, "Read data error, ret:%d, line:%d", ret, __LINE__);
return ret == ESP_GMF_IO_ABORT ? ESP_GMF_JOB_ERR_OK : ESP_GMF_JOB_ERR_FAIL;
}
esp_gmf_data_bus_block_t blk = {0};
esp_gmf_db_acquire_write(gmf_afe->in_db, &blk, in_load->valid_size, ESP_GMF_MAX_DELAY);
blk.buf = in_load->buf;
blk.valid_size = in_load->valid_size;
esp_gmf_db_release_write(gmf_afe->in_db, &blk, ESP_GMF_MAX_DELAY);
uint32_t filled_size = 0;
esp_gmf_db_get_filled_size(gmf_afe->out_db, &filled_size);
if (filled_size > (cfg->delay_samples * sizeof(uint16_t))) {
uint32_t rsize = filled_size - (cfg->delay_samples * sizeof(uint16_t));
esp_gmf_port_acquire_out(out_port, &out_load, rsize, ESP_GMF_MAX_DELAY);
esp_gmf_data_bus_block_t blk = {0};
blk.buf = out_load->buf;
blk.buf_length = rsize;
esp_gmf_db_acquire_read(gmf_afe->out_db, &blk, rsize, ESP_GMF_MAX_DELAY);
esp_gmf_db_release_read(gmf_afe->out_db, &blk, ESP_GMF_MAX_DELAY);
out_load->valid_size = rsize;
esp_gmf_port_release_out(out_port, out_load, ESP_GMF_MAX_DELAY);
ret = ESP_GMF_JOB_ERR_OK;
} else {
ret = ESP_GMF_JOB_ERR_CONTINUE;
}
if (in_load->is_done) {
ret = ESP_GMF_JOB_ERR_DONE;
}
esp_gmf_port_release_in(in_port, in_load, ESP_GMF_MAX_DELAY);
return ret;
}
static esp_gmf_err_t _load_afe_caps_func(esp_gmf_element_handle_t handle)
{
esp_gmf_cap_t *caps = NULL;
esp_gmf_cap_t afe_caps = {0};
afe_caps.cap_eightcc = ESP_GMF_CAPS_AUDIO_AEC;
afe_caps.attr_fun = NULL;
int ret = esp_gmf_cap_append(&caps, &afe_caps);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, { return ret;}, "Failed to create AEC capability");
afe_caps.cap_eightcc = ESP_GMF_CAPS_AUDIO_AGC;
ret = esp_gmf_cap_append(&caps, &afe_caps);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, { return ret;}, "Failed to create AGC capability");
afe_caps.cap_eightcc = ESP_GMF_CAPS_AUDIO_NS;
afe_caps.attr_fun = NULL;
ret = esp_gmf_cap_append(&caps, &afe_caps);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, { return ret;}, "Failed to create NS capability");
afe_caps.cap_eightcc = ESP_GMF_CAPS_AUDIO_VAD;
afe_caps.attr_fun = NULL;
ret = esp_gmf_cap_append(&caps, &afe_caps);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, { return ret;}, "Failed to create VAD capability");
afe_caps.cap_eightcc = ESP_GMF_CAPS_AUDIO_WWE;
afe_caps.attr_fun = NULL;
ret = esp_gmf_cap_append(&caps, &afe_caps);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, { return ret;}, "Failed to create WWE capability");
afe_caps.cap_eightcc = ESP_GMF_CAPS_AUDIO_VCMD;
afe_caps.attr_fun = NULL;
ret = esp_gmf_cap_append(&caps, &afe_caps);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, { return ret;}, "Failed to create VCMD capability");
esp_gmf_element_t *el = (esp_gmf_element_t *)handle;
el->caps = caps;
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t __afe_set_vcmd_det(esp_gmf_audio_element_handle_t handle, esp_gmf_args_desc_t *arg_desc,
uint8_t *buf, int buf_len)
{
ESP_GMF_NULL_CHECK(TAG, handle, {return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_NULL_CHECK(TAG, arg_desc, {return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_NULL_CHECK(TAG, buf, {return ESP_GMF_ERR_INVALID_ARG;});
esp_gmf_afe_t *gmf_afe = (esp_gmf_afe_t *)handle;
esp_gmf_afe_cfg_t *cfg = OBJ_GET_CFG(gmf_afe);
uint8_t vcmd_det_start = buf[0];
if (cfg->vcmd_detect_en) {
if (vcmd_det_start) {
vcmd_det_begin(gmf_afe);
} else {
vcmd_det_cancel(gmf_afe);
}
return ESP_GMF_ERR_OK;
}
return ESP_GMF_ERR_INVALID_STATE;
}
static esp_gmf_err_t _load_afe_methods_func(esp_gmf_element_handle_t handle)
{
esp_gmf_method_t *method = NULL;
esp_gmf_args_desc_t *set_args = NULL;
esp_gmf_err_t ret = esp_gmf_args_desc_append(&set_args, ESP_GMF_METHOD_AFE_START_VCMD_DET_ARG_EN,
ESP_GMF_ARGS_TYPE_UINT8, sizeof(uint8_t), 0);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to append vcmd det argument");
ret = esp_gmf_method_append(&method, ESP_GMF_METHOD_AFE_START_VCMD_DET, __afe_set_vcmd_det, set_args);
ESP_GMF_RET_ON_ERROR(TAG, ret, {return ret;}, "Failed to register %s method", ESP_GMF_METHOD_AFE_START_VCMD_DET);
esp_gmf_element_t *el = (esp_gmf_element_t *)handle;
el->method = method;
return ESP_GMF_ERR_OK;
}
esp_gmf_err_t esp_gmf_afe_init(void *config, esp_gmf_obj_handle_t *handle)
{
ESP_GMF_MEM_CHECK(TAG, config, { return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_MEM_CHECK(TAG, handle, { return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_NULL_CHECK(TAG, ((esp_gmf_afe_cfg_t *)config)->afe_manager, { return ESP_GMF_ERR_INVALID_ARG;})
esp_gmf_afe_t *gmf_afe = esp_gmf_oal_calloc(1, sizeof(esp_gmf_afe_t));
ESP_GMF_MEM_CHECK(TAG, gmf_afe, { return ESP_GMF_ERR_MEMORY_LACK;});
esp_gmf_obj_t *obj = (esp_gmf_obj_t *)gmf_afe;
obj->new_obj = esp_gmf_afe_new;
obj->del_obj = esp_gmf_afe_destroy;
esp_gmf_afe_cfg_t *obj_cfg = esp_gmf_oal_calloc(1, sizeof(esp_gmf_afe_cfg_t));
memcpy(obj_cfg, config, sizeof(esp_gmf_afe_cfg_t));
esp_gmf_obj_set_config(obj, obj_cfg, sizeof(esp_gmf_afe_cfg_t));
int ret = esp_gmf_obj_set_tag(obj, "ai_afe");
ESP_GMF_RET_ON_NOT_OK(TAG, ret, goto __failed, "Failed set OBJ tag");
esp_gmf_element_cfg_t el_cfg = {0};
ESP_GMF_ELEMENT_IN_PORT_ATTR_SET(el_cfg.in_attr, ESP_GMF_EL_PORT_CAP_SINGLE, 16, 0,
ESP_GMF_PORT_TYPE_BLOCK | ESP_GMF_PORT_TYPE_BYTE, AFE_DEFAULT_DATA_SIZE);
ESP_GMF_ELEMENT_OUT_PORT_ATTR_SET(el_cfg.out_attr, ESP_GMF_EL_PORT_CAP_SINGLE, 16, 0,
ESP_GMF_PORT_TYPE_BLOCK | ESP_GMF_PORT_TYPE_BYTE, AFE_DEFAULT_DATA_SIZE);
el_cfg.dependency = false;
ret = esp_gmf_audio_el_init(gmf_afe, &el_cfg);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, goto __failed, "Failed to initialize audio element");
gmf_afe->last_event = ET_UNKNOWN;
ESP_GMF_ELEMENT_GET(gmf_afe)->ops.open = esp_gmf_afe_open;
ESP_GMF_ELEMENT_GET(gmf_afe)->ops.process = esp_gmf_afe_proc;
ESP_GMF_ELEMENT_GET(gmf_afe)->ops.close = esp_gmf_afe_close;
ESP_GMF_ELEMENT_GET(gmf_afe)->ops.load_caps = _load_afe_caps_func;
ESP_GMF_ELEMENT_GET(gmf_afe)->ops.load_methods = _load_afe_methods_func;
*handle = obj;
ESP_LOGI(TAG, "Create AFE, %s-%p", OBJ_GET_TAG(obj), obj);
return ESP_GMF_ERR_OK;
__failed:
esp_gmf_afe_destroy(gmf_afe);
return ESP_GMF_ERR_FAIL;
}
static esp_gmf_err_t esp_gmf_afe_set_vcmd_detection(esp_gmf_element_handle_t handle, bool enable)
{
ESP_GMF_NULL_CHECK(TAG, handle, {return ESP_GMF_ERR_INVALID_ARG;});
const esp_gmf_method_t *method_head = NULL;
const esp_gmf_method_t *method = NULL;
esp_gmf_element_get_method((esp_gmf_element_handle_t)handle, &method_head);
esp_gmf_method_found(method_head, ESP_GMF_METHOD_AFE_START_VCMD_DET, &method);
uint8_t buf[1] = { enable };
esp_gmf_args_set_value(method->args_desc, ESP_GMF_METHOD_AFE_START_VCMD_DET_ARG_EN, buf, buf, sizeof(uint8_t));
return esp_gmf_element_exe_method((esp_gmf_element_handle_t)handle, ESP_GMF_METHOD_AFE_START_VCMD_DET, buf, sizeof(buf));
}
esp_gmf_err_t esp_gmf_afe_vcmd_detection_begin(esp_gmf_element_handle_t handle)
{
return esp_gmf_afe_set_vcmd_detection(handle, true);
}
esp_gmf_err_t esp_gmf_afe_vcmd_detection_cancel(esp_gmf_element_handle_t handle)
{
return esp_gmf_afe_set_vcmd_detection(handle, false);
}
esp_gmf_err_t esp_gmf_afe_set_event_cb(esp_gmf_element_handle_t handle, esp_gmf_afe_event_cb_t cb, void *ctx)
{
ESP_GMF_NULL_CHECK(TAG, handle, return ESP_GMF_ERR_INVALID_ARG;);
esp_gmf_afe_cfg_t *cfg = OBJ_GET_CFG(handle);
ESP_GMF_NULL_CHECK(TAG, cfg, return ESP_GMF_ERR_INVALID_STATE;)
cfg->event_cb = cb;
cfg->event_ctx = ctx;
return ESP_GMF_ERR_OK;
}
esp_gmf_err_t esp_gmf_afe_keep_awake(esp_gmf_element_handle_t handle, bool enable)
{
ESP_GMF_NULL_CHECK(TAG, handle, return ESP_GMF_ERR_INVALID_ARG;);
esp_gmf_afe_t *gmf_afe = (esp_gmf_afe_t *)handle;
if (gmf_afe->msg_queue == NULL) {
ESP_LOGE(TAG, "GMF AFE Message queue not initialized");
return ESP_GMF_ERR_INVALID_STATE;
}
afe_evt_msg_t msg = { 0 };
msg.event = ET_KEEP_WAKE_MODIFIED;
msg.payload.keep_awake.enable = enable;
BaseType_t ok = xQueueSend(gmf_afe->msg_queue, &msg, pdMS_TO_TICKS(1000));
return ok == pdTRUE ? ESP_GMF_ERR_OK : ESP_GMF_ERR_TIMEOUT;
}
esp_gmf_err_t esp_gmf_afe_trigger_wakeup(esp_gmf_element_handle_t handle)
{
ESP_GMF_NULL_CHECK(TAG, handle, return ESP_GMF_ERR_INVALID_ARG;);
esp_gmf_afe_t *gmf_afe = (esp_gmf_afe_t *)handle;
if (gmf_afe->msg_queue == NULL) {
ESP_LOGE(TAG, "GMF AFE Message queue not initialized");
return ESP_GMF_ERR_INVALID_STATE;
}
afe_evt_msg_t msg = { 0 };
msg.event = ET_TRIGGER_WAKEUP;
msg.payload.trigger.state = true;
BaseType_t ok = xQueueSend(gmf_afe->msg_queue, &msg, pdMS_TO_TICKS(1000));
return ok == pdTRUE ? ESP_GMF_ERR_OK : ESP_GMF_ERR_TIMEOUT;
}
esp_gmf_err_t esp_gmf_afe_trigger_sleep(esp_gmf_element_handle_t handle)
{
ESP_GMF_NULL_CHECK(TAG, handle, return ESP_GMF_ERR_INVALID_ARG;);
esp_gmf_afe_t *gmf_afe = (esp_gmf_afe_t *)handle;
if (gmf_afe->msg_queue == NULL) {
ESP_LOGE(TAG, "GMF AFE Message queue not initialized");
return ESP_GMF_ERR_INVALID_STATE;
}
afe_evt_msg_t msg = { 0 };
msg.event = ET_TRIGGER_SLEEP;
msg.payload.trigger.state = false;
BaseType_t ok = xQueueSend(gmf_afe->msg_queue, &msg, pdMS_TO_TICKS(1000));
return ok == pdTRUE ? ESP_GMF_ERR_OK : ESP_GMF_ERR_TIMEOUT;
}
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/src/esp_gmf_afe_manager.c
+372
View File
@@ -0,0 +1,372 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#include <stdbool.h>
#include <stdint.h>
#include <string.h>
#include "freertos/FreeRTOS.h"
#include "freertos/event_groups.h"
#include "freertos/portmacro.h"
#include "freertos/task.h"
#include "freertos/idf_additions.h"
#include "freertos/semphr.h"
#include "esp_private/freertos_idf_additions_priv.h"
#include "esp_afe_sr_models.h"
#include "esp_check.h"
#include "esp_err.h"
#include "esp_heap_caps.h"
#include "esp_gmf_afe_manager.h"
#define AFE_RUN_EVENT (BIT0)
#define AFE_DESTROYING_EVENT (BIT1)
#define FEED_TASK_DESTROYED (BIT2)
#define FETCH_TASK_DESTROYED (BIT3)
/**
* @brief Information about an AFE (Audio Front-End) task.
*/
typedef struct {
TaskHandle_t task; /*!< Task handle */
bool running; /*!< Task running status */
} afe_task_info_t;
/**
* @brief AFE (Audio Front-End) manager.
*/
typedef struct __afe {
afe_task_info_t feed; /*!< Feed task information */
afe_task_info_t fetch; /*!< Fetch task information */
esp_gmf_afe_manager_read_cb_t read_cb; /*!< Read callback function */
void *read_ctx; /*!< Read callback context */
esp_afe_sr_data_t *afe_data; /*!< AFE data */
esp_gmf_afe_manager_result_cb_t result_proc; /*!< Result processing callback function */
void *result_ctx; /*!< Result processing callback context */
esp_gmf_afe_manager_features_t feat; /*!< AFE feature state */
EventGroupHandle_t ctrl_events; /*!< Control events */
SemaphoreHandle_t read_cb_lock; /*!< Mutex for read callback */
SemaphoreHandle_t result_cb_lock;/*!< Mutex for result callback */
esp_afe_sr_iface_t *esp_afe; /*!< AFE interface */
} esp_gmf_afe_manager_t;
static const char *TAG = "AFE_MANAGER";
static void feed_task(void *arg)
{
esp_gmf_afe_manager_t *afe_manager = (esp_gmf_afe_manager_t *)arg;
int chan_num = afe_manager->esp_afe->get_feed_channel_num(afe_manager->afe_data);
int chunksize = afe_manager->esp_afe->get_feed_chunksize(afe_manager->afe_data);
const int buf_size = chunksize * chan_num * sizeof(uint16_t);
ESP_LOGI(TAG, "Feed task, ch %d, chunk %d, buf size %d", chan_num, chunksize, buf_size);
int16_t *buf = heap_caps_calloc_prefer(1, buf_size, 2, MALLOC_CAP_SPIRAM, MALLOC_CAP_INTERNAL);
if (buf == NULL) {
ESP_LOGE(TAG, "Feed task calloc failed, task quit");
goto __quit;
}
afe_manager->feed.running = true;
while (afe_manager->feed.running) {
EventBits_t bits = xEventGroupWaitBits(afe_manager->ctrl_events, AFE_RUN_EVENT | AFE_DESTROYING_EVENT, false, false, portMAX_DELAY);
if (bits & AFE_DESTROYING_EVENT) {
break;
}
int rlen = 0;
xSemaphoreTake(afe_manager->read_cb_lock, portMAX_DELAY);
if (afe_manager->read_cb) {
rlen = afe_manager->read_cb(buf, buf_size, afe_manager->read_ctx, pdMS_TO_TICKS(500));
}
xSemaphoreGive(afe_manager->read_cb_lock);
if (rlen == buf_size) {
afe_manager->esp_afe->feed(afe_manager->afe_data, buf);
} else {
ESP_LOGE(TAG, "AFE read failed %d", rlen);
}
}
__quit:
if (buf) {
heap_caps_free(buf);
}
xEventGroupSetBits(afe_manager->ctrl_events, FEED_TASK_DESTROYED);
vTaskDelete(NULL);
}
static void fetch_task(void *arg)
{
esp_gmf_afe_manager_t *afe_manager = (esp_gmf_afe_manager_t *)arg;
afe_manager->fetch.running = true;
while (afe_manager->fetch.running) {
EventBits_t bits = xEventGroupWaitBits(afe_manager->ctrl_events, AFE_RUN_EVENT | AFE_DESTROYING_EVENT, false, false, portMAX_DELAY);
if (bits & AFE_DESTROYING_EVENT) {
break;
}
afe_fetch_result_t *result = afe_manager->esp_afe->fetch(afe_manager->afe_data);
xSemaphoreTake(afe_manager->result_cb_lock, portMAX_DELAY);
if (afe_manager->result_proc != NULL) {
afe_manager->result_proc(result, afe_manager->result_ctx);
}
xSemaphoreGive(afe_manager->result_cb_lock);
}
xEventGroupSetBits(afe_manager->ctrl_events, FETCH_TASK_DESTROYED);
vTaskDelete(NULL);
}
esp_gmf_err_t esp_gmf_afe_manager_set_result_cb(esp_gmf_afe_manager_handle_t handle, esp_gmf_afe_manager_result_cb_t proc, void *ctx)
{
esp_gmf_afe_manager_t *afe_manager = (esp_gmf_afe_manager_t *)handle;
ESP_RETURN_ON_FALSE(afe_manager, ESP_GMF_ERR_INVALID_ARG, TAG, "AFE set result cb invalid handle");
xSemaphoreTake(afe_manager->result_cb_lock, portMAX_DELAY);
afe_manager->result_proc = proc;
afe_manager->result_ctx = ctx;
xSemaphoreGive(afe_manager->result_cb_lock);
return ESP_GMF_ERR_OK;
}
esp_gmf_err_t esp_gmf_afe_manager_destroy(esp_gmf_afe_manager_handle_t handle)
{
esp_gmf_afe_manager_t *afe_manager = (esp_gmf_afe_manager_t *)handle;
ESP_RETURN_ON_FALSE(afe_manager, ESP_GMF_ERR_INVALID_ARG, TAG, "AFE destroy: handle NULL");
xEventGroupSetBits(afe_manager->ctrl_events, AFE_DESTROYING_EVENT);
EventBits_t wait_bits = 0;
if (afe_manager->feed.task) {
afe_manager->feed.running = false;
wait_bits |= FEED_TASK_DESTROYED;
}
if (afe_manager->fetch.task) {
afe_manager->fetch.running = false;
wait_bits |= FETCH_TASK_DESTROYED;
}
EventBits_t bits = xEventGroupWaitBits(afe_manager->ctrl_events, wait_bits, true, true, portMAX_DELAY);
if ((bits & wait_bits) != wait_bits) {
ESP_LOGE(TAG, "AFE destroy wait bits timeout %" PRIu32, bits);
return ESP_GMF_ERR_TIMEOUT;
}
if (afe_manager->ctrl_events) {
vEventGroupDelete(afe_manager->ctrl_events);
}
if (afe_manager->read_cb_lock) {
vSemaphoreDelete(afe_manager->read_cb_lock);
}
if (afe_manager->result_cb_lock) {
vSemaphoreDelete(afe_manager->result_cb_lock);
}
if (afe_manager->afe_data) {
afe_manager->esp_afe->destroy(afe_manager->afe_data);
afe_manager->afe_data = NULL;
}
heap_caps_free(afe_manager);
ESP_LOGI(TAG, "AFE manager destroy");
return ESP_GMF_ERR_OK;
}
esp_gmf_err_t esp_gmf_afe_manager_create(esp_gmf_afe_manager_cfg_t *cfg, esp_gmf_afe_manager_handle_t *handle)
{
esp_gmf_err_t ret = ESP_GMF_ERR_OK;
*handle = NULL;
ESP_RETURN_ON_FALSE(cfg != NULL, ESP_GMF_ERR_INVALID_ARG, TAG, "AFE manager create: cfg NULL");
ESP_RETURN_ON_FALSE(handle != NULL, ESP_GMF_ERR_INVALID_ARG, TAG, "AFE manager create: handle NULL");
ESP_RETURN_ON_FALSE(cfg->afe_cfg != NULL, ESP_GMF_ERR_INVALID_ARG, TAG, "AFE manager create: cfg->afe_cfg NULL");
esp_gmf_afe_manager_t *afe_manager = heap_caps_calloc_prefer(1, sizeof(esp_gmf_afe_manager_t), 2, MALLOC_CAP_SPIRAM, MALLOC_CAP_INTERNAL);
ESP_GOTO_ON_FALSE(afe_manager, ESP_GMF_ERR_MEMORY_LACK, __err, TAG, "AFE manager create no memory");
afe_manager->read_cb = cfg->read_cb;
afe_manager->read_ctx = cfg->read_ctx;
afe_config_print(cfg->afe_cfg);
afe_manager->esp_afe = esp_afe_handle_from_config(cfg->afe_cfg);
afe_manager->afe_data = afe_manager->esp_afe->create_from_config(cfg->afe_cfg);
ESP_GOTO_ON_FALSE(afe_manager->afe_data, ESP_GMF_ERR_FAIL, __err, TAG, "AFE manager create failed");
afe_manager->ctrl_events = xEventGroupCreate();
ESP_GOTO_ON_FALSE(afe_manager->ctrl_events, ESP_GMF_ERR_MEMORY_LACK, __err, TAG, "AFE manager create events failed");
afe_manager->read_cb_lock = xSemaphoreCreateMutex();
ESP_GOTO_ON_FALSE(afe_manager->read_cb_lock, ESP_GMF_ERR_MEMORY_LACK, __err, TAG, "AFE manager create mutex failed");
afe_manager->result_cb_lock = xSemaphoreCreateMutex();
ESP_GOTO_ON_FALSE(afe_manager->result_cb_lock, ESP_GMF_ERR_MEMORY_LACK, __err, TAG, "AFE manager create mutex failed");
if (afe_manager->read_cb) {
xEventGroupSetBits(afe_manager->ctrl_events, AFE_RUN_EVENT);
}
#if (configSUPPORT_STATIC_ALLOCATION == 1) && defined(CONFIG_SPIRAM_BOOT_INIT)
prvTaskCreateDynamicPinnedToCoreWithCaps(feed_task,
"afe_feed",
cfg->feed_task_setting.stack_size,
afe_manager,
cfg->feed_task_setting.prio,
cfg->feed_task_setting.core,
(MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT),
&afe_manager->feed.task);
ESP_GOTO_ON_FALSE(afe_manager->feed.task, ESP_GMF_ERR_MEMORY_LACK, __err, TAG, "Create afe feed task failed");
prvTaskCreateDynamicPinnedToCoreWithCaps(fetch_task,
"afe_fetch",
cfg->fetch_task_setting.stack_size,
afe_manager,
cfg->fetch_task_setting.prio,
cfg->fetch_task_setting.core,
(MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT),
&afe_manager->fetch.task);
ESP_GOTO_ON_FALSE(afe_manager->fetch.task, ESP_GMF_ERR_MEMORY_LACK, __err, TAG, "Create afe fetch task failed");
#else
ESP_LOGW(TAG, "The AFE Manager may not work when PSRAM is disabled");
xTaskCreatePinnedToCore(feed_task,
"afe_feed",
cfg->feed_task_setting.stack_size,
afe_manager,
cfg->feed_task_setting.prio,
&afe_manager->feed.task,
cfg->feed_task_setting.core);
ESP_GOTO_ON_FALSE(afe_manager->feed.task, ESP_GMF_ERR_MEMORY_LACK, __err, TAG, "Create afe feed task failed");
xTaskCreatePinnedToCore(fetch_task,
"afe_fetch",
cfg->fetch_task_setting.stack_size,
afe_manager,
cfg->fetch_task_setting.prio,
&afe_manager->fetch.task,
cfg->fetch_task_setting.core);
ESP_GOTO_ON_FALSE(afe_manager->fetch.task, ESP_GMF_ERR_MEMORY_LACK, __err, TAG, "Create afe fetch task failed");
#endif /* (configSUPPORT_STATIC_ALLOCATION == 1) */
afe_manager->feat.wakeup = cfg->afe_cfg->wakenet_init;
afe_manager->feat.aec = cfg->afe_cfg->aec_init;
afe_manager->feat.se = cfg->afe_cfg->se_init;
afe_manager->feat.vad = (cfg->afe_cfg->vad_init);
*handle = afe_manager;
return ret;
__err:
if (afe_manager) {
esp_gmf_afe_manager_destroy(afe_manager);
}
return ret;
}
esp_gmf_err_t esp_gmf_afe_manager_get_features(esp_gmf_afe_manager_handle_t handle, esp_gmf_afe_manager_features_t *features)
{
ESP_RETURN_ON_FALSE(handle, ESP_GMF_ERR_INVALID_ARG, TAG, "AFE get feature: handle NULL");
ESP_RETURN_ON_FALSE(features, ESP_GMF_ERR_INVALID_ARG, TAG, "AFE get feature: features NULL");
esp_gmf_afe_manager_t *afe_manager = (esp_gmf_afe_manager_t *)handle;
memcpy(features, &(afe_manager->feat), sizeof(esp_gmf_afe_manager_features_t));
return ESP_GMF_ERR_OK;
}
esp_gmf_err_t esp_gmf_afe_manager_suspend(esp_gmf_afe_manager_handle_t handle, bool suspend)
{
esp_gmf_afe_manager_t *afe_manager = (esp_gmf_afe_manager_t *)handle;
ESP_RETURN_ON_FALSE(afe_manager, ESP_GMF_ERR_INVALID_ARG, TAG, "AFE suspend: handle NULL");
ESP_LOGI(TAG, "AFE manager suspend %d", suspend);
if (suspend) {
xEventGroupClearBits(afe_manager->ctrl_events, AFE_RUN_EVENT);
} else {
xEventGroupSetBits(afe_manager->ctrl_events, AFE_RUN_EVENT);
}
return ESP_GMF_ERR_OK;
}
esp_gmf_err_t esp_gmf_afe_manager_enable_features(esp_gmf_afe_manager_handle_t handle, esp_gmf_afe_feature_t feature, bool enable)
{
esp_gmf_afe_manager_t *afe_manager = (esp_gmf_afe_manager_t *)handle;
ESP_RETURN_ON_FALSE(afe_manager, ESP_GMF_ERR_INVALID_ARG, TAG, "AFE suspend: handle NULL");
esp_gmf_err_t ret = ESP_GMF_ERR_OK;
ESP_LOGD(TAG, "AFE Ctrl [%u, %d]", feature, enable);
switch (feature) {
case ESP_AFE_FEATURE_WAKENET: {
if (enable) {
ret = afe_manager->esp_afe->enable_wakenet(afe_manager->afe_data);
} else {
ret = afe_manager->esp_afe->disable_wakenet(afe_manager->afe_data);
}
ESP_LOGD(TAG, "Wakenet ctrl ret %d", ret);
if (ret >= 0) {
afe_manager->feat.wakeup = ret;
}
break;
}
case ESP_AFE_FEATURE_AEC: {
if (enable) {
ret = afe_manager->esp_afe->enable_aec(afe_manager->afe_data);
} else {
ret = afe_manager->esp_afe->disable_aec(afe_manager->afe_data);
}
ESP_LOGD(TAG, "AEC ctrl ret %d", ret);
if (ret >= 0) {
afe_manager->feat.aec = ret;
}
break;
}
case ESP_AFE_FEATURE_SE: {
if (enable) {
ret = afe_manager->esp_afe->enable_se(afe_manager->afe_data);
} else {
ret = afe_manager->esp_afe->disable_se(afe_manager->afe_data);
}
ESP_LOGD(TAG, "AE ctrl ret %d", ret);
if (ret >= 0) {
afe_manager->feat.se = ret;
}
break;
}
case ESP_AFE_FEATURE_VAD: {
if (enable) {
ret = afe_manager->esp_afe->enable_vad(afe_manager->afe_data);
} else {
ret = afe_manager->esp_afe->disable_vad(afe_manager->afe_data);
}
ESP_LOGD(TAG, "VAD ctrl ret %d", ret);
if (ret >= 0) {
afe_manager->feat.vad = ret;
}
ret = afe_manager->esp_afe->reset_vad(afe_manager->afe_data);
break;
}
default:
ESP_LOGW(TAG, "AFE manager feature ctrl: %u not support", feature);
ret = ESP_GMF_ERR_INVALID_ARG;
break;
}
return ret;
}
esp_gmf_err_t esp_gmf_afe_manager_set_read_cb(esp_gmf_afe_manager_handle_t handle, esp_gmf_afe_manager_read_cb_t read_cb, void *read_ctx)
{
ESP_RETURN_ON_FALSE(handle, ESP_GMF_ERR_INVALID_ARG, TAG, "AFE Manager set read: handle NULL");
esp_gmf_afe_manager_t *afe_manager = (esp_gmf_afe_manager_t *)handle;
esp_gmf_afe_manager_suspend(afe_manager, true);
xSemaphoreTake(afe_manager->read_cb_lock, portMAX_DELAY);
afe_manager->read_cb = read_cb;
afe_manager->read_ctx = read_ctx;
if (afe_manager->read_cb) {
esp_gmf_afe_manager_suspend(afe_manager, false);
}
xSemaphoreGive(afe_manager->read_cb_lock);
return ESP_GMF_ERR_OK;
}
esp_gmf_err_t esp_gmf_afe_manager_get_chunk_size(esp_gmf_afe_manager_handle_t handle, size_t *size)
{
ESP_RETURN_ON_FALSE(handle, ESP_GMF_ERR_INVALID_ARG, TAG, "AFE Manager get feed size: handle NULL");
ESP_RETURN_ON_FALSE(size, ESP_GMF_ERR_INVALID_ARG, TAG, "AFE Manager get feed size: size NULL");
esp_gmf_afe_manager_t *afe_manager = (esp_gmf_afe_manager_t *)handle;
*size = afe_manager->esp_afe->get_feed_chunksize(afe_manager->afe_data);
return ESP_GMF_ERR_OK;
}
esp_gmf_err_t esp_gmf_afe_manager_get_input_ch_num(esp_gmf_afe_manager_handle_t handle, uint8_t *ch_num)
{
ESP_RETURN_ON_FALSE(handle, ESP_GMF_ERR_INVALID_ARG, TAG, "AFE Manager get feed size: handle NULL");
ESP_RETURN_ON_FALSE(ch_num, ESP_GMF_ERR_INVALID_ARG, TAG, "AFE Manager get feed size: ch_num NULL");
esp_gmf_afe_manager_t *afe_manager = (esp_gmf_afe_manager_t *)handle;
*ch_num = afe_manager->esp_afe->get_feed_channel_num(afe_manager->afe_data);
return ESP_GMF_ERR_OK;
}
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_ai_audio/src/esp_gmf_wn.c
+287
View File
@@ -0,0 +1,287 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#include <stdint.h>
#include <stdio.h>
#include <string.h>
#include "esp_log.h"
#include "esp_types.h"
#include "esp_gmf_audio_element.h"
#include "esp_gmf_cache.h"
#include "esp_gmf_cap.h"
#include "esp_gmf_caps_def.h"
#include "esp_gmf_err.h"
#include "esp_gmf_job.h"
#include "esp_gmf_node.h"
#include "esp_gmf_oal_mem.h"
#include "esp_gmf_payload.h"
#include "esp_wn_models.h"
#include "esp_gmf_wn.h"
#include "esp_gmf_ch_sort.h"
#define ESP_WN_BYTES_PER_SAMPLE (sizeof(uint16_t))
typedef struct {
esp_gmf_audio_element_t parent;
esp_wn_iface_t *wn_iface;
model_iface_data_t *model_data;
uint32_t sample_rate;
uint32_t ch;
uint32_t chunk_size;
uint32_t frame_len;
esp_gmf_cache_t *cache;
esp_gmf_payload_t *in_load;
} gmf_wn_t;
static const char *TAG = "GMF_WN";
static esp_gmf_err_t gmf_wn_new(void *cfg, esp_gmf_obj_handle_t *handle)
{
ESP_GMF_NULL_CHECK(TAG, cfg, { return ESP_GMF_ERR_INVALID_ARG; });
ESP_GMF_NULL_CHECK(TAG, handle, { return ESP_GMF_ERR_INVALID_ARG; });
*handle = NULL;
return esp_gmf_wn_init((esp_gmf_wn_cfg_t *)cfg, (esp_gmf_element_handle_t *)handle);
}
static esp_gmf_err_t gmf_wn_received_event_handler(esp_gmf_event_pkt_t *evt, void *ctx)
{
ESP_GMF_NULL_CHECK(TAG, evt, { return ESP_GMF_ERR_INVALID_ARG; });
ESP_GMF_NULL_CHECK(TAG, ctx, { return ESP_GMF_ERR_INVALID_ARG; });
esp_gmf_element_handle_t self = (esp_gmf_element_handle_t)ctx;
esp_gmf_element_handle_t el = evt->from;
esp_gmf_event_state_t state = ESP_GMF_EVENT_STATE_NONE;
esp_gmf_element_get_state(self, &state);
if (evt->sub == ESP_GMF_INFO_SOUND) {
esp_gmf_info_sound_t info = {0};
memcpy(&info, evt->payload, evt->payload_size);
ESP_LOGD(TAG, "RECV info, from: %s-%p, next: %p, self: %s-%p, type: %x, state: %s, rate: %d, ch: %d, bits: %d",
OBJ_GET_TAG(el), el, esp_gmf_node_for_next((esp_gmf_node_t *)el), OBJ_GET_TAG(self), self, evt->type,
esp_gmf_event_get_state_str(state), info.sample_rates, info.channels, info.bits);
if (info.sample_rates != 16000 || info.bits != 16) {
ESP_LOGE(TAG, "Unsupported format, rate: %d, bits: %d", info.sample_rates, info.bits);
return ESP_GMF_ERR_NOT_SUPPORT;
}
}
if (state == ESP_GMF_EVENT_STATE_NONE) {
esp_gmf_element_set_state(self, ESP_GMF_EVENT_STATE_INITIALIZED);
}
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t gmf_wn_destroy(esp_gmf_obj_handle_t self)
{
ESP_GMF_NULL_CHECK(TAG, self, { return ESP_GMF_ERR_INVALID_ARG; });
gmf_wn_t *gmf_wn = (gmf_wn_t *)self;
if (OBJ_GET_CFG(self)) {
esp_gmf_oal_free(OBJ_GET_CFG(self));
}
if (gmf_wn->wn_iface && gmf_wn->model_data) {
gmf_wn->wn_iface->destroy(gmf_wn->model_data);
gmf_wn->model_data = NULL;
}
if (gmf_wn->cache) {
esp_gmf_cache_delete(gmf_wn->cache);
gmf_wn->cache = NULL;
}
ESP_LOGD(TAG, "Destroyed");
esp_gmf_audio_el_deinit(self);
esp_gmf_oal_free(self);
return ESP_GMF_ERR_OK;
}
static esp_gmf_job_err_t gmf_wn_open(esp_gmf_audio_element_handle_t self, void *para)
{
gmf_wn_t *gmf_wn = (gmf_wn_t *)self;
esp_gmf_wn_cfg_t *cfg = OBJ_GET_CFG(self);
char *model_name = esp_srmodel_filter(cfg->models, ESP_WN_PREFIX, NULL);
ESP_GMF_NULL_CHECK(TAG, model_name, { return ESP_GMF_JOB_ERR_FAIL; });
gmf_wn->wn_iface = (esp_wn_iface_t *)esp_wn_handle_from_name(model_name);
ESP_GMF_NULL_CHECK(TAG, gmf_wn->wn_iface, { return ESP_GMF_JOB_ERR_FAIL; });
gmf_wn->model_data = gmf_wn->wn_iface->create(model_name, cfg->det_mode);
ESP_GMF_NULL_CHECK(TAG, gmf_wn->model_data, { return ESP_GMF_JOB_ERR_FAIL; });
int algo_ch_num = gmf_wn->wn_iface->get_channel_num(gmf_wn->model_data);
gmf_wn->ch = strlen(cfg->input_format);
if (algo_ch_num > gmf_wn->ch) {
ESP_LOGE(TAG, "Input channel number is less than needed");
return ESP_GMF_JOB_ERR_FAIL;
}
gmf_wn->sample_rate = gmf_wn->wn_iface->get_samp_rate(gmf_wn->model_data);
gmf_wn->chunk_size = gmf_wn->wn_iface->get_samp_chunksize(gmf_wn->model_data);
gmf_wn->frame_len = gmf_wn->chunk_size * gmf_wn->ch * ESP_WN_BYTES_PER_SAMPLE;
esp_gmf_cache_new(gmf_wn->frame_len, &gmf_wn->cache);
ESP_GMF_NULL_CHECK(TAG, gmf_wn->cache, { return ESP_GMF_JOB_ERR_FAIL; });
ESP_GMF_ELEMENT_IN_PORT_ATTR_SET(ESP_GMF_ELEMENT_GET(self)->in_attr, ESP_GMF_EL_PORT_CAP_SINGLE, 16, 0,
ESP_GMF_PORT_TYPE_BLOCK | ESP_GMF_PORT_TYPE_BYTE, gmf_wn->frame_len);
ESP_GMF_ELEMENT_OUT_PORT_ATTR_SET(ESP_GMF_ELEMENT_GET(self)->out_attr, ESP_GMF_EL_PORT_CAP_SINGLE, 16, 0,
ESP_GMF_PORT_TYPE_BLOCK | ESP_GMF_PORT_TYPE_BYTE, gmf_wn->chunk_size * ESP_WN_BYTES_PER_SAMPLE);
ESP_LOGI(TAG, "Open, frame_len: %" PRIu32 ", ch %" PRIu32 ", chunksize %" PRIu32, gmf_wn->frame_len, gmf_wn->ch, gmf_wn->chunk_size);
esp_gmf_info_sound_t snd_info = {0};
snd_info.sample_rates = 16000;
snd_info.bits = 16;
snd_info.channels = 1;
esp_gmf_element_notify_snd_info(self, &snd_info);
return ESP_GMF_JOB_ERR_OK;
}
static esp_gmf_job_err_t gmf_wn_process(esp_gmf_audio_element_handle_t self, void *para)
{
int ret = ESP_GMF_JOB_ERR_OK;
gmf_wn_t *gmf_wn = (gmf_wn_t *)self;
esp_gmf_port_handle_t in_port = ESP_GMF_ELEMENT_GET(self)->in;
esp_gmf_port_handle_t out_port = ESP_GMF_ELEMENT_GET(self)->out;
esp_gmf_payload_t *out_load = NULL;
esp_gmf_payload_t *cache_load = NULL;
esp_gmf_err_io_t load_ret = 0;
esp_gmf_wn_cfg_t *cfg = OBJ_GET_CFG(self);
bool need_load = false;
esp_gmf_cache_ready_for_load(gmf_wn->cache, &need_load);
if (need_load) {
load_ret = esp_gmf_port_acquire_in(in_port, &gmf_wn->in_load, gmf_wn->frame_len, in_port->wait_ticks);
ESP_GMF_PORT_ACQUIRE_IN_CHECK(TAG, load_ret, ret, { goto __quit; });
esp_gmf_cache_load(gmf_wn->cache, gmf_wn->in_load);
}
esp_gmf_cache_acquire(gmf_wn->cache, gmf_wn->frame_len, &cache_load);
if (cache_load->valid_size != gmf_wn->frame_len) {
if (cache_load->is_done == true) {
ret = ESP_GMF_JOB_ERR_DONE;
} else {
ret = ESP_GMF_JOB_ERR_CONTINUE;
ESP_LOGD(TAG, "Return Continue, size:%u", cache_load->valid_size);
}
goto __quit;
}
load_ret = esp_gmf_port_acquire_out(out_port, &out_load, gmf_wn->frame_len, ESP_GMF_MAX_DELAY);
out_load->is_done = cache_load->is_done;
ESP_GMF_PORT_ACQUIRE_OUT_CHECK(TAG, load_ret, ret, { goto __quit; });
if (out_load->buf_length < gmf_wn->frame_len) {
ret = ESP_GMF_JOB_ERR_FAIL;
ESP_LOGE(TAG, "Output buffer is not enough");
goto __quit;
}
esp_gmf_sort_with_format((int16_t *)cache_load->buf,
cfg->input_format,
gmf_wn->chunk_size,
gmf_wn->ch,
(int16_t *)out_load->buf);
int res = gmf_wn->wn_iface->detect(gmf_wn->model_data, (int16_t *)out_load->buf);
if (res > 0) {
int trigger_ch = gmf_wn->wn_iface->get_triggered_channel(gmf_wn->model_data);
if (cfg->detect_cb) {
cfg->detect_cb(self, trigger_ch, cfg->user_ctx);
}
}
out_load->valid_size = gmf_wn->chunk_size * ESP_WN_BYTES_PER_SAMPLE;
esp_gmf_cache_ready_for_load(gmf_wn->cache, &need_load);
if (need_load == false) {
ret = ESP_GMF_JOB_ERR_TRUNCATE;
} else {
ret = (out_load->is_done == true ? ESP_GMF_JOB_ERR_DONE : ESP_GMF_JOB_ERR_OK);
}
__quit:
if (out_load) {
esp_gmf_port_release_out(out_port, out_load, ESP_GMF_MAX_DELAY);
}
if (gmf_wn->in_load && (ret != ESP_GMF_JOB_ERR_TRUNCATE)) {
esp_gmf_port_release_in(in_port, gmf_wn->in_load, ESP_GMF_MAX_DELAY);
gmf_wn->in_load = NULL;
}
if (cache_load) {
esp_gmf_cache_release(gmf_wn->cache, cache_load);
}
return ret;
}
static esp_gmf_job_err_t gmf_wn_close(esp_gmf_audio_element_handle_t self, void *para)
{
gmf_wn_t *gmf_wn = (gmf_wn_t *)self;
if (gmf_wn->wn_iface && gmf_wn->model_data) {
gmf_wn->wn_iface->destroy(gmf_wn->model_data);
gmf_wn->model_data = NULL;
}
if (gmf_wn->cache) {
esp_gmf_cache_delete(gmf_wn->cache);
gmf_wn->cache = NULL;
}
return ESP_GMF_JOB_ERR_OK;
}
static esp_gmf_err_t _load_wn_caps_func(esp_gmf_element_handle_t handle)
{
esp_gmf_cap_t *caps = NULL;
esp_gmf_cap_t wn_caps = {0};
wn_caps.cap_eightcc = ESP_GMF_CAPS_AUDIO_WWE;
wn_caps.attr_fun = NULL;
int ret = esp_gmf_cap_append(&caps, &wn_caps);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, { return ret; }, "Failed to create capability");
esp_gmf_element_t *el = (esp_gmf_element_t *)handle;
el->caps = caps;
return ESP_GMF_ERR_OK;
}
esp_gmf_err_t esp_gmf_wn_init(esp_gmf_wn_cfg_t *config, esp_gmf_element_handle_t *handle)
{
esp_gmf_err_t ret = ESP_GMF_ERR_OK;
ESP_GMF_NULL_CHECK(TAG, config, { return ESP_GMF_ERR_INVALID_ARG; });
ESP_GMF_NULL_CHECK(TAG, handle, { return ESP_GMF_ERR_INVALID_ARG; });
*handle = NULL;
gmf_wn_t *gmf_wn = esp_gmf_oal_calloc(1, sizeof(gmf_wn_t));
ESP_GMF_MEM_VERIFY(TAG, gmf_wn, { return ESP_GMF_ERR_MEMORY_LACK; }, "wn", sizeof(gmf_wn_t));
esp_gmf_obj_t *obj = (esp_gmf_obj_t *)gmf_wn;
obj->new_obj = gmf_wn_new;
obj->del_obj = gmf_wn_destroy;
esp_gmf_wn_cfg_t *obj_cfg = esp_gmf_oal_calloc(1, sizeof(esp_gmf_wn_cfg_t));
ESP_GMF_NULL_CHECK(TAG, obj_cfg, { ret = ESP_GMF_ERR_MEMORY_LACK; goto __failed; });
memcpy(obj_cfg, config, sizeof(esp_gmf_wn_cfg_t));
ret = esp_gmf_obj_set_config(obj, obj_cfg, sizeof(esp_gmf_wn_cfg_t));
ESP_GMF_RET_ON_NOT_OK(TAG, ret, goto __failed, "Failed set OBJ configuration");
ret = esp_gmf_obj_set_tag(obj, "ai_wn");
ESP_GMF_RET_ON_NOT_OK(TAG, ret, goto __failed, "Failed set OBJ tag");
esp_gmf_element_cfg_t el_cfg = {0};
ESP_GMF_ELEMENT_IN_PORT_ATTR_SET(el_cfg.in_attr, ESP_GMF_EL_PORT_CAP_SINGLE, 16, 0,
ESP_GMF_PORT_TYPE_BLOCK | ESP_GMF_PORT_TYPE_BYTE, 1024);
ESP_GMF_ELEMENT_OUT_PORT_ATTR_SET(el_cfg.out_attr, ESP_GMF_EL_PORT_CAP_SINGLE, 16, 0,
ESP_GMF_PORT_TYPE_BLOCK | ESP_GMF_PORT_TYPE_BYTE, 1024);
el_cfg.dependency = true;
ret = esp_gmf_audio_el_init(gmf_wn, &el_cfg);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, goto __failed, "Failed to initialize wn element");
ESP_GMF_ELEMENT_GET(gmf_wn)->ops.open = gmf_wn_open;
ESP_GMF_ELEMENT_GET(gmf_wn)->ops.process = gmf_wn_process;
ESP_GMF_ELEMENT_GET(gmf_wn)->ops.close = gmf_wn_close;
ESP_GMF_ELEMENT_GET(gmf_wn)->ops.event_receiver = gmf_wn_received_event_handler;
ESP_GMF_ELEMENT_GET(gmf_wn)->ops.load_caps = _load_wn_caps_func;
*handle = obj;
return ESP_GMF_ERR_OK;
__failed:
esp_gmf_obj_delete(obj);
return ret;
}
esp_gmf_err_t esp_gmf_wn_set_detect_cb(esp_gmf_element_handle_t self, esp_wn_detect_cb_t cb, void *ctx)
{
ESP_GMF_NULL_CHECK(TAG, self, { return ESP_GMF_ERR_INVALID_ARG; });
ESP_GMF_NULL_CHECK(TAG, cb, { return ESP_GMF_ERR_INVALID_ARG; });
esp_gmf_wn_cfg_t *cfg = OBJ_GET_CFG(self);
cfg->detect_cb = cb;
cfg->user_ctx = ctx;
return ESP_GMF_ERR_OK;
}
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_audio/CHANGELOG.md
+105
View File
@@ -0,0 +1,105 @@
# Changelog
## v0.7.2
### Features
- Support C++ build
## v0.7.1
### Bug Fixes
- Fixed sonic bypass failed for first payload
### Features
- Added audio param setting for `aud_alc`, `aud_fade`, `aud_sonic`
## v0.7.0
### Features
- Updated `esp_audio_codec` to version v2.3.0
- Updated `esp_audio_effects` to version v1.1.0
- Added `esp_gmf_audio_enc_get_frame_size`, `esp_gmf_audio_enc_set_bitrate`, `esp_gmf_audio_enc_get_bitrate`, `esp_gmf_audio_enc_reconfig` and `esp_gmf_audio_enc_reconfig_by_sound_info` functions to `gmf_audio_enc`
- Added `SBC` and `LC3` encoders
- Added `esp_gmf_audio_dec_reconfig` and `esp_gmf_audio_dec_reconfig_by_sound_info` functions to `gmf_audio_dec`
- Added `SBC` and `LC3` decoders
- Added `meta_flag` in `gmf_audio_dec` to support audio decoder recovery status tracking
- Added common audio parameter setting interface
- Redefined audio methods name
- Removed the audio encoder and decoder reconfig interface in `esp_gmf_audio_helper.c`
- Used the `esp_gmf_element_handle_t` type handle in the `gmf_audio` module
- Use default configuration for `esp_gmf_audio_xxx_init` internally if configuration set to NULL
- Added TRUNCATE support for SONIC element to enable its flexible placement at any position within the pipeline
- Supported scenarios where the input data for audio encoding is not a complete frame
- Added PTS (Presentation Time Stamp) calculation and propagation for the `aud_alc`, `aud_eq`, `aud_fade`, `aud_enc`
- Supported user-configurable input frame length and load acquisition timeout for the mixer
### Bug Fixes
- Fixed `gmf_audio_enc` process blocked due to forget release of in_load when truncate is returned
- Fixed parameter mismatch in `audio_dec_reconfig_dec_by_sound_info`
- Standardize TAG identifier format across all audio elements with `aud` prefix
- Fixed a crash caused by passing a NULL encoder config to the init function and subsequently reporting info
- Fixed bugs in the implementation of methods for sonic, fade, and mixer
- Fixed the issue where the encoder modified the configuration during runtime
- Fixed the issue where the mixer did not forward the out_load parameter to downstream components when all input stream were no data
- Fixed bugs that not check validation of in and out buffer for mixer which will cause crash
- Fixed bugs where setting parameters before opening had no effect for `aud_fade` and `aud_mixer`
## v0.6.3
### Bug Fixes
- Fixed decoder not releasing input port on failure
- Fixed missing input size check in gmf_audio_enc when input is insufficient to encode a frame and stream has ended
- Fixed missing zero size check for acquire_in in gmf_audio_dec
- Fixed unmatched sub_cfg when reconfigured by esp_gmf_audio_helper_reconfig_dec_by_uri
## v0.6.2
### Features
- Limit the version of `esp_audio_codec` and `esp_audio_effects`
## v0.6.1
- Added missing #include "esp_gmf_audio_element.h" across all audio elements
- Fixed out-of-range parameter handling for mixer and EQ elements
- Resolved rate/bit/channel converter bypass errors caused by asynchronous modification of obj->cfg between set/event callbacks and process functions
- Deleted one unreasonable log from esp_gmf_audio_helper.c
## v0.6.0
### Features
- Updated the logic for GMF capabilities and method handling
- Added support for NULL object configuration
- Added GMF port attribution for elements
- Added gmf_cache and optimized the loop path for the audio encoder and decoder
- Renamed component to `gmf_audio`
- Updated the License
## v0.5.2
### Bug Fixes
- Fixed an issue where the decoder's output done flag was repeatedly set after the input was marked as done
- Fixed an issue where the input port was not released when the input data size was 0
## v0.5.1
### Bug Fixes
- Fixed the component requirements
## v0.5.0
### Features
- Initial version of `esp-gmf-audio`
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_audio/CMakeLists.txt
+4
View File
@@ -0,0 +1,4 @@
idf_component_register(SRC_DIRS "./"
INCLUDE_DIRS "./include"
PRIV_INCLUDE_DIRS "./private_include"
REQUIRES esp_timer)
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_audio/LICENSE
+22
View File
@@ -0,0 +1,22 @@
Espressif Modified MIT License
Copyright (c) 2025 Espressif Systems (Shanghai) CO., LTD
Permission is hereby granted for use EXCLUSIVELY with Espressif Systems products.
This includes the right to use, copy, modify, merge, publish, distribute, and sublicense
the Software, subject to the following conditions:
1. This Software MUST BE USED IN CONJUNCTION WITH ESPRESSIF SYSTEMS PRODUCTS.
2. The above copyright notice and this permission notice shall be included in all copies
or substantial portions of the Software.
3. Redistribution of the Software in source or binary form FOR USE WITH NON-ESPRESSIF PRODUCTS
is strictly prohibited.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR
PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE
FOR ANY CLAIM, DAMAGES, OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_audio/README.md
+43
View File
@@ -0,0 +1,43 @@
# ESP-GMF-Audio
- [![Component Registry](https://components.espressif.com/components/espressif/gmf_audio/badge.svg)](https://components.espressif.com/components/espressif/gmf_audio)
- [中文版](./README_CN.md)
ESP GMF Audio is a collection of GMF elements related to audio processing, including audio encoding, decoding, and audio effect processing algorithms. The currently supported audio modules are listed in the table below.
|Name|TAG|Function|Method|Input Port|Output Port|Input blocking time|Output blocking time|Dependent on Audio Information|
|:----:|:----:|:-----:|:----:|:----:|:----:|:----:|:----:|:----|
| AUDIO_DEC |aud_dec |Audio decoder: MP3,AAC,AMRNB,<br>AMRWB,FLAC,WAV,M4A,TS|Nil|Single|Single|User configurable, default value is maximum delay|User configurable, default value is maximum delay|No|
| AUDIO_ENC |aud_enc |Audio encoder: AAC,AMRNB,AMRWB,<br>ADPCM,OPUS,PCM|Nil|Single|Single|User configurable, default value is maximum delay|User configurable, default value is maximum delay|Yes|
| RATE_CVT|aud_rate_cvt |Audio sampling rate adjustment|`set_dest_rate`|Single|Single|Maximum delay|Maximum delay|Yes|
| BIT_CVT |aud_bit_cvt |Audio bit-depth conversion|`set_dest_bits`|Single|Single|Maximum delay|Maximum delay|Yes|
| CH_CVT |aud_ch_cvt |Audio channel conversion|`set_dest_ch`|Single|Single|Maximum delay|Maximum delay|Yes|
| ALC |aud_alc |Audio volume adjustment|`set_gain`<br>`get_gain`|Single|Single|Maximum delay|Maximum delay|Yes|
| EQ |aud_eq |Audio equalizer adjustment|`set_para`<br>`get_para`<br>`enable_filter`<br>`disable_filter`|Single|Single|Maximum delay|Maximum delay|Yes|
| FADE |aud_fade |Audio fade-in and fade-out effects|`set_mode`<br>`get_mode`<br>`reset_weight`|Single|Single|Maximum delay|Maximum delay|Yes|
| SONIC |aud_sonic |Audio pitch and speed shifting effects|`set_speed`<br>`get_speed`<br>`set_pitch`<br>`get_pitch`|Single|Single|Maximum delay|Maximum delay|Yes|
| MIXER |aud_mixer |Audio mixing effects|`set_info`<br>`set_mode`|Multiple|Single|The blocking time for the first channel is 0, while the blocking time for other channels is maximum delay|Maximum delay|No|
|INTERLEAVE|aud_intlv|Data interleaving|Nil|Multiple|Single|User configurable, default value is maximum delay|Maximum delay|Yes|
|DEINTERLEAVE|aud_deintlv|Data de-interleaving|Nil|Single|Multiple|Maximum delay|User configurable, default value is maximum delay|Yes|
## Usage
The ESP GMF Audio is often used in combination to form a pipeline. For example code, please refer to [test_app](../test_apps/main/elements/gmf_audio_play_el_test.c)。
You can also create and compile a project using the following commands, taking the `pipeline_play_embed_music` project as an example. Before starting, make sure you have a working [ESP-IDF](https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/get-started/index.html) environment.
### 1. Create the Example Project
Create the `pipeline_play_embed_music` example project based on the `gmf_examples` component (using version v0.7.0 as an example; update the version as needed):
```shell
idf.py create-project-from-example "espressif/gmf_examples=0.7.0:pipeline_play_embed_music"
```
### 2. Build and Flash the Project Using an ESP32-S3 Board
```shell
cd pipeline_play_embed_music
idf.py set-target esp32s3
idf.py -p YOUR_PORT flash monitor
```
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_audio/README_CN.md
+43
View File
@@ -0,0 +1,43 @@
# ESP-GMF-Audio
- [![Component Registry](https://components.espressif.com/components/espressif/gmf_audio/badge.svg)](https://components.espressif.com/components/espressif/gmf_audio)
- [English](./README.md)
ESP GMF Audio 是 GMF 音频处理相关元素的集合，包括音频编码，解码和音频效果处理算法。目前已支持的 Audio 模块参见下表。
| 名称 | 标签 | 功能 | 函数方法 | 输入端口 | 输出端口 | 输入阻塞时间 | 输出阻塞时间 |依赖音频信息 |
|:----:| :-----: | :----: | :----: |:----: |:----: |:----: |:----: |:---- |
| AUDIO_DEC | aud_dec | 音频解码：MP3,AAC,AMRNB,<br>AMRWB,FLAC,WAV,M4A,TS | 无 | 单个 | 单个 | 可用户配置，默认是最大延迟 |可用户配置，默认是最大延迟 |否 |
| AUDIO_ENC | aud_enc | 音频编码：AAC,AMRNB,AMRWB,<br>ADPCM,OPUS,PCM | 无 | 单个 | 单个 | 可用户配置，默认是最大延迟 |可用户配置，默认是最大延迟 |是 |
| RATE_CVT| aud_rate_cvt | 音频采样率调节 | `set_dest_rate` | 单个 | 单个 |最大延迟 |最大延迟| 是 |
| BIT_CVT | aud_bit_cvt | 音频比特位转换 | `set_dest_bits`| 单个 | 单个 |最大延迟 |最大延迟| 是 |
| CH_CVT | aud_ch_cvt | 音频声道数转换 | `set_dest_ch`| 单个 | 单个 |最大延迟 |最大延迟| 是 |
| ALC | aud_alc | 音频音量调节 | `set_gain`<br>`get_gain`| 单个 | 单个 |最大延迟 |最大延迟| 是 |
| EQ | aud_eq | 音频均衡器调节 |`set_para`<br>`get_para`<br>`enable_filter`<br>`disable_filter` |单个 |单个|最大延迟 |最大延迟|是 |
| FADE | aud_fade | 音频淡入淡出效果 |`set_mode`<br>`get_mode`<br>`reset_weight` | 单个 | 单个 |最大延迟 |最大延迟 |是 |
| SONIC | aud_sonic | 音频变速变调效果 |`set_speed`<br>`get_speed`<br>`set_pitch`<br>`get_pitch`| 单个 | 单个 |最大延迟 |最大延迟| 是 |
| MIXER | aud_mixer | 音频混音效果 |`set_info`<br>`set_mode`| 多个 | 单个 | 第一路阻塞时间为0，其他路阻塞时间为最大延迟 |最大延迟| 否 |
|INTERLEAVE| aud_intlv | 数据交织 | 无 | 多个 | 单个 | 可用户配置，默认是最大延迟 |最大延迟| 是 |
|DEINTERLEAVE| aud_deintlv | 数据解交织 | 无| 单个 | 多个 |最大延迟|可用户配置，默认是最大延迟 |是 |
## 示例
ESP GMF Audio 常常组合成管道使用，示例代码请参考 [test_app](../test_apps/main/elements/gmf_audio_play_el_test.c)。
此外，也可通过以下命令创建并编译工程，以 `pipeline_play_embed_music` 示例工程为例。开始之前需要有可运行的 [ESP-IDF](https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/get-started/index.html) 环境。
### 1. 创建示例工程
基于 `gmf_examples` 组件创建 `pipeline_play_embed_music` 的示例（以 v0.7.0 版本为例， 请根据实际使用更新版本参数）
``` shell
idf.py create-project-from-example "espressif/gmf_examples=0.7.0:pipeline_play_embed_music"
```
### 2. 基于 ESP32S3 编译和下载
```shell
cd pipeline_play_embed_music
idf.py set-target esp32s3`
idf.py -p YOUR_PORT flash monitor
```
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_audio/esp_gmf_alc.c
+353
View File
@@ -0,0 +1,353 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#include <string.h>
#include "esp_log.h"
#include "esp_gmf_oal_mem.h"
#include "esp_gmf_oal_mutex.h"
#include "esp_gmf_node.h"
#include "esp_gmf_alc.h"
#include "esp_gmf_args_desc.h"
#include "gmf_audio_common.h"
#include "esp_gmf_audio_methods_def.h"
#include "esp_gmf_cap.h"
#include "esp_gmf_caps_def.h"
#include "esp_gmf_audio_element.h"
#define GMF_ALC_DEFAULT_MAX_CHANNEL 2
/**
* @brief Audio ALC context in GMF
*/
typedef struct {
esp_gmf_audio_element_t parent; /*!< The GMF alc handle */
esp_ae_alc_handle_t alc_hd; /*!< The audio effects alc handle */
uint8_t bytes_per_sample; /*!< Bytes number of per sampling point */
int8_t *gain; /*!< The modified gain value of a certain channel number */
int8_t max_ch; /*!< The maximum channel number */
bool need_reopen : 1; /*!< Whether need to reopen.
True: Execute the close function first, then execute the open function
False: Do nothing */
} esp_gmf_alc_t;
static const char *TAG = "ESP_GMF_ALC";
static esp_gmf_err_t __alc_set_gain(esp_gmf_element_handle_t handle, esp_gmf_args_desc_t *arg_desc,
uint8_t *buf, int buf_len)
{
ESP_GMF_NULL_CHECK(TAG, arg_desc, { return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_NULL_CHECK(TAG, buf, { return ESP_GMF_ERR_INVALID_ARG;});
esp_gmf_args_desc_t *alc_desc = arg_desc;
uint8_t idx = (uint8_t)(*buf);
alc_desc = alc_desc->next;
int8_t gain = (int8_t)(*(buf + alc_desc->offset));
esp_gmf_err_t ret = ESP_GMF_ERR_OK;
if (idx == 0xFF) {
// Apply gain to all channels
esp_ae_alc_cfg_t *config = (esp_ae_alc_cfg_t *)OBJ_GET_CFG(handle);
if (config) {
for (uint8_t i = 0; i < config->channel; i++) {
ret |= esp_gmf_alc_set_gain(handle, i, gain);
}
}
} else {
ret = esp_gmf_alc_set_gain(handle, idx, gain);
}
return ret;
}
static esp_gmf_err_t __alc_get_gain(esp_gmf_element_handle_t handle, esp_gmf_args_desc_t *arg_desc,
uint8_t *buf, int buf_len)
{
ESP_GMF_NULL_CHECK(TAG, arg_desc, {return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_NULL_CHECK(TAG, buf, {return ESP_GMF_ERR_INVALID_ARG;});
esp_gmf_args_desc_t *alc_desc = arg_desc;
uint8_t idx = (uint8_t)(*buf);
alc_desc = alc_desc->next;
int8_t *gain = (int8_t *)(buf + alc_desc->offset);
return esp_gmf_alc_get_gain(handle, idx, gain);
}
static esp_gmf_err_t esp_gmf_alc_new(void *cfg, esp_gmf_obj_handle_t *handle)
{
return esp_gmf_alc_init(cfg, (esp_gmf_element_handle_t *)handle);
}
static esp_gmf_job_err_t esp_gmf_alc_open(esp_gmf_element_handle_t self, void *para)
{
esp_gmf_alc_t *alc = (esp_gmf_alc_t *)self;
esp_ae_alc_cfg_t *config = (esp_ae_alc_cfg_t *)OBJ_GET_CFG(self);
ESP_GMF_NULL_CHECK(TAG, config, { return ESP_GMF_JOB_ERR_FAIL;});
alc->bytes_per_sample = (config->bits_per_sample >> 3) * config->channel;
esp_ae_alc_open(config, &alc->alc_hd);
ESP_GMF_CHECK(TAG, alc->alc_hd, { return ESP_GMF_JOB_ERR_FAIL;}, "Failed to create alc handle");
GMF_AUDIO_UPDATE_SND_INFO(self, config->sample_rate, config->bits_per_sample, config->channel);
for (size_t i = 0; i < config->channel; i++) {
esp_ae_err_t ret = esp_ae_alc_set_gain(alc->alc_hd, i, alc->gain[i]);
if (ret != ESP_AE_ERR_OK) {
return ESP_GMF_JOB_ERR_FAIL;
}
}
alc->need_reopen = false;
ESP_LOGD(TAG, "Open, %p", self);
return ESP_GMF_JOB_ERR_OK;
}
static esp_gmf_job_err_t esp_gmf_alc_close(esp_gmf_element_handle_t self, void *para)
{
esp_gmf_alc_t *alc = (esp_gmf_alc_t *)self;
ESP_LOGD(TAG, "Closed, %p", self);
if (alc->alc_hd != NULL) {
esp_ae_alc_close(alc->alc_hd);
alc->alc_hd = NULL;
}
return ESP_GMF_ERR_OK;
}
static esp_gmf_job_err_t esp_gmf_alc_process(esp_gmf_element_handle_t self, void *para)
{
esp_gmf_alc_t *alc = (esp_gmf_alc_t *)self;
esp_gmf_job_err_t out_len = ESP_GMF_JOB_ERR_OK;
if (alc->need_reopen) {
esp_gmf_alc_close(self, NULL);
out_len = esp_gmf_alc_open(self, NULL);
if (out_len != ESP_GMF_JOB_ERR_OK) {
ESP_LOGE(TAG, "ALC reopen failed");
return out_len;
}
}
esp_gmf_port_handle_t in_port = ESP_GMF_ELEMENT_GET(self)->in;
esp_gmf_port_handle_t out_port = ESP_GMF_ELEMENT_GET(self)->out;
esp_gmf_payload_t *in_load = NULL;
esp_gmf_payload_t *out_load = NULL;
int samples_num = ESP_GMF_ELEMENT_GET(alc)->in_attr.data_size / (alc->bytes_per_sample);
int bytes = samples_num * alc->bytes_per_sample;
esp_gmf_err_io_t load_ret = esp_gmf_port_acquire_in(in_port, &in_load, bytes, ESP_GMF_MAX_DELAY);
samples_num = in_load->valid_size / (alc->bytes_per_sample);
bytes = samples_num * alc->bytes_per_sample;
if ((bytes != in_load->valid_size) || (load_ret < ESP_GMF_IO_OK)) {
ESP_LOGE(TAG, "Invalid in load size %d, ret %d", in_load->valid_size, load_ret);
out_len = ESP_GMF_JOB_ERR_FAIL;
goto __alc_release;
}
if (in_port->is_shared == 1) {
out_load = in_load;
}
load_ret = esp_gmf_port_acquire_out(out_port, &out_load, samples_num ? bytes : in_load->buf_length, ESP_GMF_MAX_DELAY);
ESP_GMF_PORT_ACQUIRE_OUT_CHECK(TAG, load_ret, out_len, { goto __alc_release;});
if (samples_num) {
esp_gmf_oal_mutex_lock(((esp_gmf_audio_element_t *)self)->lock);
esp_ae_err_t ret = esp_ae_alc_process(alc->alc_hd, samples_num, in_load->buf, out_load->buf);
esp_gmf_oal_mutex_unlock(((esp_gmf_audio_element_t *)self)->lock);
ESP_GMF_RET_ON_ERROR(TAG, ret, {out_len = ESP_GMF_JOB_ERR_FAIL; goto __alc_release;}, "ALC process error %d", ret);
}
ESP_LOGV(TAG, "Samples: %d, IN-PLD: %p-%p-%d-%d-%d, OUT-PLD: %p-%p-%d-%d-%d",
samples_num, in_load, in_load->buf, in_load->valid_size, in_load->buf_length, in_load->is_done,
out_load, out_load->buf, out_load->valid_size, out_load->buf_length, out_load->is_done);
out_load->valid_size = samples_num * alc->bytes_per_sample;
out_load->is_done = in_load->is_done;
out_load->pts = in_load->pts;
esp_gmf_audio_el_update_file_pos((esp_gmf_element_handle_t)self, out_load->valid_size);
if (in_load->is_done) {
out_len = ESP_GMF_JOB_ERR_DONE;
ESP_LOGD(TAG, "ALC done, out len: %d", out_load->valid_size);
}
__alc_release:
// Release in and out port
if (out_load != NULL) {
load_ret = esp_gmf_port_release_out(out_port, out_load, ESP_GMF_MAX_DELAY);
if ((load_ret < ESP_GMF_IO_OK) && (load_ret != ESP_GMF_IO_ABORT)) {
ESP_LOGE(TAG, "OUT port release error, ret:%d", load_ret);
out_len = ESP_GMF_JOB_ERR_FAIL;
}
}
if (in_load != NULL) {
load_ret = esp_gmf_port_release_in(in_port, in_load, ESP_GMF_MAX_DELAY);
if ((load_ret < ESP_GMF_IO_OK) && (load_ret != ESP_GMF_IO_ABORT)) {
ESP_LOGE(TAG, "IN port release error, ret:%d", load_ret);
out_len = ESP_GMF_JOB_ERR_FAIL;
}
}
return out_len;
}
static esp_gmf_err_t alc_received_event_handler(esp_gmf_event_pkt_t *evt, void *ctx)
{
ESP_GMF_NULL_CHECK(TAG, ctx, {return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_NULL_CHECK(TAG, evt, {return ESP_GMF_ERR_INVALID_ARG;});
if ((evt->type != ESP_GMF_EVT_TYPE_REPORT_INFO)
|| (evt->sub != ESP_GMF_INFO_SOUND)
|| (evt->payload == NULL)) {
return ESP_GMF_ERR_OK;
}
esp_gmf_element_handle_t self = (esp_gmf_element_handle_t)ctx;
esp_gmf_element_handle_t el = evt->from;
esp_gmf_event_state_t state = ESP_GMF_EVENT_STATE_NONE;
esp_gmf_element_get_state(self, &state);
esp_gmf_info_sound_t *info = (esp_gmf_info_sound_t *)evt->payload;
esp_ae_alc_cfg_t *config = (esp_ae_alc_cfg_t *)OBJ_GET_CFG(self);
ESP_GMF_NULL_CHECK(TAG, config, return ESP_GMF_ERR_FAIL);
esp_gmf_alc_t *alc = (esp_gmf_alc_t *)self;
if (info->channels > alc->max_ch) {
alc->gain = esp_gmf_oal_realloc(alc->gain, config->channel * sizeof(*alc->gain));
ESP_GMF_MEM_VERIFY(TAG, alc->gain, return ESP_GMF_ERR_MEMORY_LACK, "alc gain", config->channel * sizeof(*alc->gain));
alc->max_ch = info->channels;
}
alc->need_reopen = (config->sample_rate != info->sample_rates) || (info->channels != config->channel) || (config->bits_per_sample != info->bits);
config->sample_rate = info->sample_rates;
config->channel = info->channels;
config->bits_per_sample = info->bits;
ESP_LOGD(TAG, "RECV element info, from: %s-%p, next: %p, self: %s-%p, type: %x, state: %s, rate: %d, ch: %d, bits: %d",
OBJ_GET_TAG(el), el, esp_gmf_node_for_next((esp_gmf_node_t *)el), OBJ_GET_TAG(self), self, evt->type,
esp_gmf_event_get_state_str(state), info->sample_rates, info->channels, info->bits);
if (state == ESP_GMF_EVENT_STATE_NONE) {
esp_gmf_element_set_state(self, ESP_GMF_EVENT_STATE_INITIALIZED);
}
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t esp_gmf_alc_destroy(esp_gmf_element_handle_t self)
{
esp_gmf_alc_t *alc = (esp_gmf_alc_t *)self;
ESP_LOGD(TAG, "Destroyed, %p", self);
void *cfg = OBJ_GET_CFG(self);
if (cfg) {
esp_gmf_oal_free(cfg);
}
if (alc->gain) {
esp_gmf_oal_free(alc->gain);
alc->gain = NULL;
}
esp_gmf_audio_el_deinit(self);
esp_gmf_oal_free(alc);
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t _load_alc_caps_func(esp_gmf_element_handle_t handle)
{
esp_gmf_cap_t *caps = NULL;
esp_gmf_cap_t dec_caps = {0};
dec_caps.cap_eightcc = ESP_GMF_CAPS_AUDIO_ALC;
dec_caps.attr_fun = NULL;
int ret = esp_gmf_cap_append(&caps, &dec_caps);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to create capability");
esp_gmf_element_t *el = (esp_gmf_element_t *)handle;
el->caps = caps;
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t _load_alc_methods_func(esp_gmf_element_handle_t handle)
{
esp_gmf_method_t *method = NULL;
esp_gmf_args_desc_t *set_args = NULL;
esp_gmf_args_desc_t *get_args = NULL;
esp_gmf_err_t ret = esp_gmf_args_desc_append(&set_args, AMETHOD_ARG(ALC, SET_GAIN, IDX),
ESP_GMF_ARGS_TYPE_UINT8, sizeof(uint8_t), 0);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to append index argument");
ret = esp_gmf_args_desc_append(&set_args, AMETHOD_ARG(ALC, SET_GAIN, GAIN), ESP_GMF_ARGS_TYPE_INT8,
sizeof(int8_t), sizeof(uint8_t));
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to append gain argument");
ret = esp_gmf_method_append(&method, AMETHOD(ALC, SET_GAIN), __alc_set_gain, set_args);
ESP_GMF_RET_ON_ERROR(TAG, ret, {return ret;}, "Failed to register %s method", AMETHOD(ALC, SET_GAIN));
ret = esp_gmf_args_desc_copy(set_args, &get_args);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to copy argument");
ret = esp_gmf_method_append(&method, AMETHOD(ALC, GET_GAIN), __alc_get_gain, get_args);
ESP_GMF_RET_ON_ERROR(TAG, ret, {return ret;}, "Failed to register %s method", AMETHOD(ALC, GET_GAIN));
esp_gmf_element_t *el = (esp_gmf_element_t *)handle;
el->method = method;
return ESP_GMF_ERR_OK;
}
esp_gmf_err_t esp_gmf_alc_set_gain(esp_gmf_element_handle_t handle, uint8_t idx, int8_t gain)
{
ESP_GMF_NULL_CHECK(TAG, handle, { return ESP_GMF_ERR_INVALID_ARG;});
esp_gmf_alc_t *alc = (esp_gmf_alc_t *)handle;
if (idx >= alc->max_ch) {
ESP_LOGE(TAG, "Gain index %d is out of range", idx);
return ESP_GMF_ERR_INVALID_ARG;
}
if (alc->alc_hd) {
esp_gmf_oal_mutex_lock(((esp_gmf_audio_element_t *)handle)->lock);
esp_ae_err_t ret = esp_ae_alc_set_gain(alc->alc_hd, idx, gain);
esp_gmf_oal_mutex_unlock(((esp_gmf_audio_element_t *)handle)->lock);
if (ret != ESP_AE_ERR_OK) {
return ESP_GMF_JOB_ERR_FAIL;
}
}
alc->gain[idx] = gain;
return ESP_GMF_JOB_ERR_OK;
}
esp_gmf_err_t esp_gmf_alc_get_gain(esp_gmf_element_handle_t handle, uint8_t idx, int8_t *gain)
{
ESP_GMF_NULL_CHECK(TAG, handle, { return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_NULL_CHECK(TAG, gain, { return ESP_GMF_ERR_INVALID_ARG;});
esp_gmf_alc_t *alc = (esp_gmf_alc_t *)handle;
if(idx > alc->max_ch) {
ESP_LOGE(TAG, "Gain index %d is out of range", idx);
return ESP_GMF_ERR_INVALID_ARG;
}
if (alc->alc_hd) {
esp_ae_err_t ret = esp_ae_alc_get_gain(alc->alc_hd, idx, gain);
if (ret != ESP_AE_ERR_OK) {
return ESP_GMF_JOB_ERR_FAIL;
}
return ESP_GMF_JOB_ERR_OK;
}
*gain = alc->gain[idx];
return ESP_GMF_JOB_ERR_OK;
}
esp_gmf_err_t esp_gmf_alc_init(esp_ae_alc_cfg_t *config, esp_gmf_element_handle_t *handle)
{
ESP_GMF_NULL_CHECK(TAG, handle, {return ESP_GMF_ERR_INVALID_ARG;});
*handle = NULL;
esp_gmf_err_t ret = ESP_GMF_ERR_OK;
esp_gmf_alc_t *alc = esp_gmf_oal_calloc(1, sizeof(esp_gmf_alc_t));
ESP_GMF_MEM_VERIFY(TAG, alc, {return ESP_GMF_ERR_MEMORY_LACK;}, "ALC", sizeof(esp_gmf_alc_t));
esp_gmf_obj_t *obj = (esp_gmf_obj_t *)alc;
obj->new_obj = esp_gmf_alc_new;
obj->del_obj = esp_gmf_alc_destroy;
alc->max_ch = GMF_ALC_DEFAULT_MAX_CHANNEL;
esp_ae_alc_cfg_t *cfg = esp_gmf_oal_calloc(1, sizeof(esp_ae_alc_cfg_t));
ESP_GMF_MEM_VERIFY(TAG, cfg, ret = ESP_GMF_ERR_MEMORY_LACK; goto ALC_INIT_FAIL;, "alc configuration", sizeof(esp_ae_alc_cfg_t));
esp_gmf_obj_set_config(obj, cfg, sizeof(esp_ae_alc_cfg_t));
if (config) {
alc->max_ch = config->channel > 0 ? config->channel : GMF_ALC_DEFAULT_MAX_CHANNEL;
memcpy(cfg, config, sizeof(esp_ae_alc_cfg_t));
} else {
esp_ae_alc_cfg_t dcfg = DEFAULT_ESP_GMF_ALC_CONFIG();
alc->max_ch = dcfg.channel;
memcpy(cfg, &dcfg, sizeof(esp_ae_alc_cfg_t));
}
alc->gain = esp_gmf_oal_calloc(1, alc->max_ch * sizeof(int8_t));
ESP_GMF_MEM_VERIFY(TAG, alc->gain, goto ALC_INIT_FAIL, "alc gain", alc->max_ch * sizeof(int8_t));
ret = esp_gmf_obj_set_tag(obj, "aud_alc");
ESP_GMF_RET_ON_NOT_OK(TAG, ret, goto ALC_INIT_FAIL, "Failed to set obj tag");
esp_gmf_element_cfg_t el_cfg = {0};
ESP_GMF_ELEMENT_IN_PORT_ATTR_SET(el_cfg.in_attr, ESP_GMF_EL_PORT_CAP_SINGLE, 0, 0,
ESP_GMF_PORT_TYPE_BLOCK | ESP_GMF_PORT_TYPE_BYTE, ESP_GMF_ELEMENT_PORT_DATA_SIZE_DEFAULT);
ESP_GMF_ELEMENT_OUT_PORT_ATTR_SET(el_cfg.out_attr, ESP_GMF_EL_PORT_CAP_SINGLE, 0, 0,
ESP_GMF_PORT_TYPE_BLOCK | ESP_GMF_PORT_TYPE_BYTE, ESP_GMF_ELEMENT_PORT_DATA_SIZE_DEFAULT);
el_cfg.dependency = true;
ret = esp_gmf_audio_el_init(alc, &el_cfg);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, goto ALC_INIT_FAIL, "Failed to initialize alc element");
ESP_GMF_ELEMENT_GET(alc)->ops.open = esp_gmf_alc_open;
ESP_GMF_ELEMENT_GET(alc)->ops.process = esp_gmf_alc_process;
ESP_GMF_ELEMENT_GET(alc)->ops.close = esp_gmf_alc_close;
ESP_GMF_ELEMENT_GET(alc)->ops.event_receiver = alc_received_event_handler;
ESP_GMF_ELEMENT_GET(alc)->ops.load_caps = _load_alc_caps_func;
ESP_GMF_ELEMENT_GET(alc)->ops.load_methods = _load_alc_methods_func;
*handle = obj;
ESP_LOGD(TAG, "Initialization, %s-%p", OBJ_GET_TAG(obj), obj);
return ESP_GMF_ERR_OK;
ALC_INIT_FAIL:
esp_gmf_alc_destroy(obj);
return ret;
}
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_audio/esp_gmf_audio_dec.c
+517
View File
@@ -0,0 +1,517 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#include <string.h>
#include "esp_log.h"
#include "esp_gmf_oal_mem.h"
#include "esp_gmf_err.h"
#include "esp_gmf_audio_dec.h"
#include "esp_audio_types.h"
#include "esp_audio_simple_dec_default.h"
#include "gmf_audio_common.h"
#include "esp_gmf_cap.h"
#include "esp_fourcc.h"
#include "esp_gmf_caps_def.h"
#include "esp_gmf_audio_methods_def.h"
#include "esp_gmf_audio_element.h"
#define DEFAULT_DEC_OUTPUT_BUFFER_SIZE 1024
/**
* @brief Audio simple decoder context in GMF
*/
typedef struct {
esp_gmf_audio_element_t parent; /*!< The GMF audio decoder handle */
esp_audio_simple_dec_handle_t dec_hd; /*!< The audio simple decoder handle */
esp_audio_simple_dec_raw_t in_data; /*!< The audio simple decoder input data handle */
esp_audio_simple_dec_out_t out_data; /*!< The audio simple decoder output data handle */
int32_t buf_size; /*!< The size of decoder out buffer */
esp_gmf_payload_t *in_load; /*!< The input payload */
uint64_t pts; /*!< Audio pts */
} esp_gmf_audio_dec_t;
static const char *TAG = "ESP_GMF_ASMP_DEC";
static esp_gmf_err_t _dec_caps_iter_fun(uint32_t attr_index, esp_gmf_cap_attr_t *attr)
{
switch (attr_index) {
case 0: {
const static uint32_t support_dec_type[] = {ESP_FOURCC_MP3, ESP_FOURCC_AAC, ESP_FOURCC_OPUS, ESP_FOURCC_FLAC, ESP_FOURCC_AMRNB, ESP_FOURCC_AMRWB, ESP_FOURCC_ALAC,
ESP_FOURCC_M4A, ESP_FOURCC_ALAW, ESP_FOURCC_ULAW, ESP_FOURCC_LC3, ESP_FOURCC_SBC, ESP_FOURCC_PCM};
ESP_GMF_CAP_ATTR_SET_DISCRETE(attr, ESP_FOURCC_TO_INT('T', 'Y', 'P', 'E'), (uint32_t *) &support_dec_type,
sizeof(support_dec_type) / sizeof(uint32_t), sizeof(uint32_t));
break;
}
default:
attr->prop_type = ESP_GMF_PROP_TYPE_NONE;
return ESP_GMF_ERR_NOT_SUPPORT;
}
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t __audio_dec_reconfig(esp_gmf_element_handle_t handle, esp_gmf_args_desc_t *arg_desc,
uint8_t *buf, int buf_len)
{
ESP_GMF_NULL_CHECK(TAG, buf, return ESP_GMF_ERR_INVALID_ARG);
esp_audio_simple_dec_cfg_t *config = (esp_audio_simple_dec_cfg_t *)buf;
return esp_gmf_audio_dec_reconfig(handle, config);
}
static esp_gmf_err_t __audio_dec_reconfig_by_sound_info(esp_gmf_element_handle_t handle, esp_gmf_args_desc_t *arg_desc,
uint8_t *buf, int buf_len)
{
ESP_GMF_NULL_CHECK(TAG, buf, return ESP_GMF_ERR_INVALID_ARG);
esp_gmf_info_sound_t *snd_info = (esp_gmf_info_sound_t *)buf;
return esp_gmf_audio_dec_reconfig_by_sound_info(handle, snd_info);
}
static inline esp_gmf_err_t dupl_esp_audio_simple_cfg(esp_audio_simple_dec_cfg_t *config, esp_audio_simple_dec_cfg_t **new_config)
{
void *sub_cfg = NULL;
*new_config = esp_gmf_oal_calloc(1, sizeof(*config));
ESP_GMF_MEM_VERIFY(TAG, *new_config, {return ESP_GMF_ERR_MEMORY_LACK;}, "audio simple decoder configuration", sizeof(*config));
memcpy(*new_config, config, sizeof(*config));
if (config->dec_cfg && (config->cfg_size > 0)) {
sub_cfg = esp_gmf_oal_calloc(1, config->cfg_size);
ESP_GMF_MEM_VERIFY(TAG, sub_cfg, {esp_gmf_oal_free(*new_config); return ESP_GMF_ERR_MEMORY_LACK;},
"decoder configuration", config->cfg_size);
memcpy(sub_cfg, config->dec_cfg, config->cfg_size);
(*new_config)->dec_cfg = sub_cfg;
}
return ESP_GMF_JOB_ERR_OK;
}
static esp_gmf_err_t audio_dec_set_subcfg(esp_audio_simple_dec_cfg_t *dec_cfg, void *sub_cfg, int32_t sub_cfg_sz)
{
dec_cfg->dec_cfg = esp_gmf_oal_calloc(1, sub_cfg_sz);
ESP_GMF_MEM_CHECK(TAG, dec_cfg->dec_cfg, return ESP_GMF_ERR_MEMORY_LACK;);
dec_cfg->cfg_size = sub_cfg_sz;
memcpy(dec_cfg->dec_cfg, sub_cfg, sub_cfg_sz);
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t audio_dec_reconfig_dec_by_sound_info(esp_gmf_element_handle_t handle, esp_gmf_info_sound_t *info)
{
esp_audio_simple_dec_cfg_t *dec_cfg = (esp_audio_simple_dec_cfg_t *)OBJ_GET_CFG(handle);
if (dec_cfg == NULL) {
dec_cfg = esp_gmf_oal_calloc(1, sizeof(esp_audio_simple_dec_cfg_t));
ESP_GMF_MEM_VERIFY(TAG, dec_cfg, return ESP_GMF_ERR_MEMORY_LACK, "audio simple decoder configuration", sizeof(esp_audio_simple_dec_cfg_t));
esp_gmf_obj_set_config(handle, dec_cfg, sizeof(esp_audio_simple_dec_cfg_t));
}
esp_gmf_err_t ret = ESP_GMF_ERR_OK;
bool same_type = (info->format_id == dec_cfg->dec_type) ? (true) : (false);
dec_cfg->dec_type = info->format_id;
// free sub cfg first
if (dec_cfg->dec_cfg && (same_type == false)) {
esp_gmf_oal_free(dec_cfg->dec_cfg);
dec_cfg->dec_cfg = NULL;
dec_cfg->cfg_size = 0;
}
switch (info->format_id) {
case ESP_AUDIO_SIMPLE_DEC_TYPE_MP3:
case ESP_AUDIO_SIMPLE_DEC_TYPE_AMRWB:
case ESP_AUDIO_SIMPLE_DEC_TYPE_AMRNB:
case ESP_AUDIO_SIMPLE_DEC_TYPE_FLAC:
case ESP_AUDIO_SIMPLE_DEC_TYPE_WAV:
case ESP_AUDIO_SIMPLE_DEC_TYPE_M4A:
case ESP_AUDIO_SIMPLE_DEC_TYPE_TS:
break;
case ESP_AUDIO_SIMPLE_DEC_TYPE_AAC:
if (dec_cfg->dec_cfg == NULL) {
esp_aac_dec_cfg_t aac_cfg = {
.no_adts_header = false,
.aac_plus_enable = true,
};
ret = audio_dec_set_subcfg(dec_cfg, &aac_cfg, sizeof(esp_aac_dec_cfg_t));
}
esp_aac_dec_cfg_t *aac_cfg = (esp_aac_dec_cfg_t *)dec_cfg->dec_cfg;
aac_cfg->sample_rate = info->sample_rates;
aac_cfg->channel = info->channels;
aac_cfg->bits_per_sample = info->bits;
break;
case ESP_AUDIO_SIMPLE_DEC_TYPE_RAW_OPUS:
if (dec_cfg->dec_cfg == NULL) {
dec_cfg->dec_type = ESP_AUDIO_SIMPLE_DEC_TYPE_RAW_OPUS;
esp_opus_dec_cfg_t opus_cfg = ESP_OPUS_DEC_CONFIG_DEFAULT();
opus_cfg.frame_duration = ESP_OPUS_DEC_FRAME_DURATION_20_MS;
opus_cfg.channel = info->channels;
opus_cfg.sample_rate = info->sample_rates;
ret = audio_dec_set_subcfg(dec_cfg, &opus_cfg, sizeof(esp_opus_dec_cfg_t));
} else {
esp_opus_dec_cfg_t *opus_cfg = (esp_opus_dec_cfg_t *)dec_cfg->dec_cfg;
opus_cfg->channel = info->channels;
opus_cfg->sample_rate = info->sample_rates;
}
break;
case ESP_AUDIO_SIMPLE_DEC_TYPE_PCM:
esp_pcm_dec_cfg_t pcm_cfg = {
.sample_rate = info->sample_rates,
.channel = info->channels,
.bits_per_sample = info->bits,
};
ret = audio_dec_set_subcfg(dec_cfg, &pcm_cfg, sizeof(esp_pcm_dec_cfg_t));
break;
case ESP_AUDIO_SIMPLE_DEC_TYPE_G711A:
case ESP_AUDIO_SIMPLE_DEC_TYPE_G711U:
esp_g711_dec_cfg_t g711_cfg = {
.channel = info->channels,
};
ret = audio_dec_set_subcfg(dec_cfg, &g711_cfg, sizeof(esp_g711_dec_cfg_t));
break;
case ESP_AUDIO_SIMPLE_DEC_TYPE_ADPCM:
dec_cfg->dec_type = ESP_AUDIO_SIMPLE_DEC_TYPE_ADPCM;
esp_adpcm_dec_cfg_t adpcm_cfg = {
.bits_per_sample = 4,
.channel = info->channels,
.sample_rate = info->sample_rates,
};
ret = audio_dec_set_subcfg(dec_cfg, &adpcm_cfg, sizeof(esp_adpcm_dec_cfg_t));
break;
case ESP_AUDIO_SIMPLE_DEC_TYPE_SBC:
if (dec_cfg->dec_cfg == NULL) {
dec_cfg->dec_type = ESP_AUDIO_SIMPLE_DEC_TYPE_SBC;
esp_sbc_dec_cfg_t sbc_cfg = {
.sbc_mode = ESP_SBC_MODE_STD,
.ch_num = 2,
.enable_plc = false,
};
ret = audio_dec_set_subcfg(dec_cfg, &sbc_cfg, sizeof(esp_sbc_dec_cfg_t));
}
break;
case ESP_AUDIO_SIMPLE_DEC_TYPE_LC3:
if (dec_cfg->dec_cfg == NULL) {
dec_cfg->dec_type = ESP_AUDIO_SIMPLE_DEC_TYPE_LC3;
esp_lc3_dec_cfg_t lc3_cfg = ESP_LC3_DEC_CONFIG_DEFAULT();
lc3_cfg.channel = info->channels;
lc3_cfg.sample_rate = info->sample_rates;
lc3_cfg.bits_per_sample = info->bits;
ret = audio_dec_set_subcfg(dec_cfg, &lc3_cfg, sizeof(esp_lc3_dec_cfg_t));
} else {
esp_lc3_dec_cfg_t *lc3_cfg = (esp_lc3_dec_cfg_t *)dec_cfg->dec_cfg;
lc3_cfg->channel = info->channels;
lc3_cfg->sample_rate = info->sample_rates;
lc3_cfg->bits_per_sample = info->bits;
}
break;
default:
dec_cfg->dec_type = ESP_AUDIO_SIMPLE_DEC_TYPE_NONE;
ESP_LOGW(TAG, "Not support for simple decoder type %ld", info->format_id);
return ESP_GMF_ERR_NOT_SUPPORT;
}
ESP_LOGD(TAG, "The new dec type is %d", dec_cfg->dec_type);
return ((ret == ESP_GMF_ERR_OK) ? (ESP_GMF_ERR_OK) : (ret));
}
static inline void free_esp_audio_simple_cfg(esp_audio_simple_dec_cfg_t *config)
{
if (config) {
if (config->dec_cfg) {
esp_gmf_oal_free(config->dec_cfg);
config->cfg_size = 0;
}
esp_gmf_oal_free(config);
}
}
static esp_gmf_err_t esp_gmf_audio_dec_new(void *cfg, esp_gmf_obj_handle_t *handle)
{
return esp_gmf_audio_dec_init((esp_audio_simple_dec_cfg_t *)cfg, (esp_gmf_element_handle_t *)handle);
}
static esp_gmf_job_err_t esp_gmf_audio_dec_open(esp_gmf_element_handle_t self, void *para)
{
esp_gmf_audio_dec_t *audio_dec = (esp_gmf_audio_dec_t *)self;
esp_audio_simple_dec_cfg_t *dec_cfg = (esp_audio_simple_dec_cfg_t *)OBJ_GET_CFG(self);
ESP_GMF_NULL_CHECK(TAG, dec_cfg, return ESP_GMF_ERR_FAIL);
esp_audio_simple_dec_open(dec_cfg, &audio_dec->dec_hd);
ESP_GMF_CHECK(TAG, audio_dec->dec_hd, {return ESP_GMF_JOB_ERR_FAIL;}, "Failed to create simple decoder handle");
esp_gmf_port_enable_payload_share(ESP_GMF_ELEMENT_GET(self)->in, false);
audio_dec->buf_size = DEFAULT_DEC_OUTPUT_BUFFER_SIZE;
ESP_LOGD(TAG, "Open, el: %p, cfg: %p, type: %d", self, dec_cfg, dec_cfg->dec_type);
return ESP_GMF_JOB_ERR_OK;
}
static esp_gmf_job_err_t esp_gmf_audio_dec_process(esp_gmf_element_handle_t self, void *para)
{
esp_gmf_port_t *in_port = ESP_GMF_ELEMENT_GET(self)->in;
esp_gmf_port_t *out = ESP_GMF_ELEMENT_GET(self)->out;
esp_audio_err_t ret = ESP_AUDIO_ERR_OK;
esp_gmf_job_err_t out_len = ESP_GMF_JOB_ERR_OK;
esp_gmf_err_io_t load_ret = ESP_GMF_IO_OK;
esp_gmf_audio_dec_t *audio_dec = (esp_gmf_audio_dec_t *)self;
esp_gmf_payload_t *out_load = NULL;
esp_audio_simple_dec_info_t dec_info = {0};
esp_gmf_info_sound_t snd_info = {0};
if (audio_dec->in_data.len == 0) {
load_ret = esp_gmf_port_acquire_in(in_port, &audio_dec->in_load, ESP_GMF_ELEMENT_GET(audio_dec)->in_attr.data_size, in_port->wait_ticks);
ESP_GMF_PORT_ACQUIRE_IN_CHECK(TAG, load_ret, out_len, {goto __aud_proc_release;});
audio_dec->in_data.buffer = audio_dec->in_load->buf;
audio_dec->in_data.len = audio_dec->in_load->valid_size;
audio_dec->in_data.consumed = 0;
audio_dec->in_data.eos = audio_dec->in_load->is_done;
audio_dec->in_data.frame_recover = (audio_dec->in_load->meta_flag & ESP_GMF_META_FLAG_AUD_RECOVERY_PLC) ? 1 : 0;
}
ESP_LOGV(TAG, "Read, in_len: %ld, done: %d\r\n", audio_dec->in_data.len, audio_dec->in_load ? audio_dec->in_load->is_done : -1);
load_ret = esp_gmf_port_acquire_out(out, &out_load, audio_dec->buf_size, ESP_GMF_MAX_DELAY);
ESP_GMF_PORT_ACQUIRE_OUT_CHECK(TAG, load_ret, out_len, {goto __aud_proc_release;});
out_load->valid_size = 0;
audio_dec->out_data.buffer = out_load->buf;
audio_dec->out_data.len = out_load->buf_length;
if (audio_dec->in_data.len == 0) {
if (audio_dec->in_load->is_done == true) {
out_len = ESP_GMF_JOB_ERR_DONE;
out_load->is_done = audio_dec->in_load->is_done;
ESP_LOGD(TAG, "Return done, line:%d", __LINE__);
goto __aud_proc_release;
} else {
out_len = ESP_GMF_JOB_ERR_CONTINUE;
ESP_LOGD(TAG, "Return Continue, size:%d", audio_dec->in_load->valid_size);
goto __aud_proc_release;
}
}
while (1) {
ret = esp_audio_simple_dec_process(audio_dec->dec_hd, &audio_dec->in_data, &audio_dec->out_data);
if (ret != ESP_AUDIO_ERR_OK && ret != ESP_AUDIO_ERR_BUFF_NOT_ENOUGH) {
ESP_LOGE(TAG, "Failed to decode data, ret: %d", ret);
out_len = ESP_GMF_JOB_ERR_FAIL;
goto __aud_proc_release;
}
if (ret == ESP_AUDIO_ERR_BUFF_NOT_ENOUGH) {
load_ret = esp_gmf_port_release_out(out, out_load, out->wait_ticks);
ESP_GMF_PORT_RELEASE_OUT_CHECK(TAG, load_ret, out_len, {goto __aud_proc_release;});
load_ret = esp_gmf_port_acquire_out(out, &out_load, audio_dec->out_data.needed_size, ESP_GMF_MAX_DELAY);
ESP_GMF_PORT_ACQUIRE_OUT_CHECK(TAG, load_ret, out_len, {goto __aud_proc_release;});
ESP_LOGW(TAG, "Not enough memory for out, need:%d, old: %ld, new: %d", (int)audio_dec->out_data.needed_size,
audio_dec->out_data.len, out_load->buf_length);
audio_dec->out_data.buffer = out_load->buf;
audio_dec->out_data.len = out_load->buf_length;
audio_dec->buf_size = audio_dec->out_data.needed_size;
continue;
}
if (audio_dec->in_data.consumed <= audio_dec->in_data.len) {
audio_dec->in_data.buffer += audio_dec->in_data.consumed;
audio_dec->in_data.len -= audio_dec->in_data.consumed;
}
ESP_LOGV(TAG, "Dec, out len: %ld, need: %ld, in len: %ld, consumed: %ld, dec: %ld",
audio_dec->out_data.len, audio_dec->out_data.needed_size,
audio_dec->in_data.len, audio_dec->in_data.consumed, audio_dec->out_data.decoded_size);
ESP_LOGV(TAG, "buf: %p, sz: %d, dec: %ld", out_load->buf, out_load->valid_size, audio_dec->out_data.decoded_size);
if (audio_dec->out_data.decoded_size > 0) {
esp_audio_simple_dec_get_info(audio_dec->dec_hd, &dec_info);
esp_gmf_audio_el_get_snd_info(self, &snd_info);
if (snd_info.sample_rates != dec_info.sample_rate
|| snd_info.channels != dec_info.channel
|| snd_info.bits != dec_info.bits_per_sample) {
ESP_LOGD(TAG, "NOTIFY Info, rate: %d, bits: %d, ch: %d --> rate: %ld, bits: %d, ch: %d",
snd_info.sample_rates, snd_info.bits, snd_info.channels, dec_info.sample_rate, dec_info.bits_per_sample, dec_info.channel);
GMF_AUDIO_UPDATE_SND_INFO(self, dec_info.sample_rate, dec_info.bits_per_sample, dec_info.channel);
}
out_load->valid_size = audio_dec->out_data.decoded_size;
out_load->pts = audio_dec->pts;
audio_dec->pts += GMF_AUDIO_CALC_PTS(out_load->valid_size, dec_info.sample_rate, dec_info.channel, dec_info.bits_per_sample);
esp_gmf_audio_el_update_file_pos(self, out_load->valid_size);
if (audio_dec->in_load != NULL && audio_dec->in_data.len > 0) {
ESP_LOGD(TAG, "Return truncate, in len:%ld", audio_dec->in_data.len);
out_len = ESP_GMF_JOB_ERR_TRUNCATE;
}
} else {
if (audio_dec->in_data.len > 0) {
continue;
}
if (audio_dec->in_load && audio_dec->in_load->is_done) {
out_load->is_done = audio_dec->in_load->is_done;
ESP_LOGD(TAG, "Return done, line:%d", __LINE__);
out_len = ESP_GMF_JOB_ERR_DONE;
} else {
ESP_LOGD(TAG, "Return Continue, in len:%ld", audio_dec->in_data.len);
out_len = ESP_GMF_JOB_ERR_CONTINUE;
}
}
ESP_LOGV(TAG, "Release IN, in_len: %ld, done: %d, decoded_size: %ld",
audio_dec->in_data.len, audio_dec->in_load ? audio_dec->in_load->is_done : -1, audio_dec->out_data.decoded_size);
break;
}
__aud_proc_release:
if (out_load != NULL) {
load_ret = esp_gmf_port_release_out(out, out_load, out->wait_ticks);
if ((load_ret < ESP_GMF_IO_OK) && (load_ret != ESP_GMF_IO_ABORT)) {
ESP_LOGE(TAG, "OUT port release error, ret:%d", load_ret);
out_len = ESP_GMF_JOB_ERR_FAIL;
}
}
// If decoding fails or there is no data on input side, release input port
if ((out_len == ESP_GMF_JOB_ERR_FAIL) || (audio_dec->in_load && (audio_dec->in_data.len == 0))) {
load_ret = esp_gmf_port_release_in(in_port, audio_dec->in_load, ESP_GMF_MAX_DELAY);
if ((load_ret < ESP_GMF_IO_OK) && (load_ret != ESP_GMF_IO_ABORT)) {
ESP_LOGE(TAG, "IN port release error, ret:%d", load_ret);
out_len = ESP_GMF_JOB_ERR_FAIL;
}
audio_dec->in_load = NULL;
}
return out_len;
}
static esp_gmf_job_err_t esp_gmf_audio_dec_close(esp_gmf_element_handle_t self, void *para)
{
ESP_LOGD(TAG, "Closed, %p", self);
esp_gmf_audio_dec_t *audio_dec = (esp_gmf_audio_dec_t *)self;
if (audio_dec->dec_hd != NULL) {
esp_audio_simple_dec_close(audio_dec->dec_hd);
audio_dec->dec_hd = NULL;
}
audio_dec->pts = 0;
esp_gmf_info_sound_t snd_info = {0};
audio_dec->in_load = NULL;
audio_dec->in_data.len = 0;
esp_gmf_audio_el_set_snd_info(self, &snd_info);
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t esp_gmf_audio_dec_destroy(esp_gmf_element_handle_t self)
{
ESP_LOGD(TAG, "Destroyed, %p", self);
free_esp_audio_simple_cfg(OBJ_GET_CFG(self));
esp_gmf_audio_el_deinit(self);
esp_gmf_oal_free(self);
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t _load_dec_caps_func(esp_gmf_element_handle_t handle)
{
esp_gmf_cap_t *caps = NULL;
esp_gmf_cap_t dec_caps = {0};
dec_caps.cap_eightcc = ESP_GMF_CAPS_AUDIO_DECODER;
dec_caps.attr_fun = _dec_caps_iter_fun;
int ret = esp_gmf_cap_append(&caps, &dec_caps);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to create capability");
esp_gmf_element_t *el = (esp_gmf_element_t *)handle;
el->caps = caps;
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t _load_dec_methods_func(esp_gmf_element_handle_t handle)
{
esp_gmf_method_t *method = NULL;
esp_gmf_args_desc_t *set_args = NULL;
esp_gmf_args_desc_t *reconfig_args = NULL;
esp_gmf_args_desc_t *sndinfo_args = NULL;
esp_gmf_err_t ret = esp_gmf_args_desc_append(&sndinfo_args, AMETHOD_ARG(DECODER, RECONFIG_BY_SND_INFO, INFO_TYPE), ESP_GMF_ARGS_TYPE_UINT32,
sizeof(uint32_t), offsetof(esp_gmf_info_sound_t, format_id));
ESP_GMF_RET_ON_NOT_OK(TAG, ret, return ret, "Failed to append type argument");
ret = esp_gmf_args_desc_append(&sndinfo_args, AMETHOD_ARG(DECODER, RECONFIG_BY_SND_INFO, INFO_SAMPLERATE), ESP_GMF_ARGS_TYPE_INT32,
sizeof(int32_t), offsetof(esp_gmf_info_sound_t, sample_rates));
ESP_GMF_RET_ON_NOT_OK(TAG, ret, return ret, "Failed to append sample_rates argument");
ret = esp_gmf_args_desc_append(&sndinfo_args, AMETHOD_ARG(DECODER, RECONFIG_BY_SND_INFO, INFO_CHANNEL), ESP_GMF_ARGS_TYPE_INT8,
sizeof(int8_t), 12);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, return ret, "Failed to append channels argument");
ret = esp_gmf_args_desc_append(&sndinfo_args, AMETHOD_ARG(DECODER, RECONFIG_BY_SND_INFO, INFO_BITS), ESP_GMF_ARGS_TYPE_INT8,
sizeof(int8_t), 13);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, return ret, "Failed to append bits argument");
ret = esp_gmf_args_desc_append_array(&set_args, AMETHOD_ARG(DECODER, RECONFIG_BY_SND_INFO, INFO), sndinfo_args,
sizeof(esp_gmf_info_sound_t), 0);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, return ret, "Failed to append sound info argument");
ret = esp_gmf_method_append(&method, AMETHOD(DECODER, RECONFIG_BY_SND_INFO), __audio_dec_reconfig_by_sound_info, set_args);
ESP_GMF_RET_ON_ERROR(TAG, ret, return ret, "Failed to register %s method", AMETHOD(DECODER, RECONFIG_BY_SND_INFO));
set_args = NULL;
ret = esp_gmf_args_desc_append(&reconfig_args, AMETHOD_ARG(DECODER, RECONFIG, CFG_TYPE), ESP_GMF_ARGS_TYPE_INT32,
sizeof(int32_t), offsetof(esp_audio_simple_dec_cfg_t, dec_type));
ESP_GMF_RET_ON_NOT_OK(TAG, ret, return ret, "Failed to append type argument");
ret = esp_gmf_args_desc_append(&reconfig_args, AMETHOD_ARG(DECODER, RECONFIG, CFG_SUBCFGPTR), ESP_GMF_ARGS_TYPE_INT32,
sizeof(int32_t), offsetof(esp_audio_simple_dec_cfg_t, dec_cfg));
ESP_GMF_RET_ON_NOT_OK(TAG, ret, return ret, "Failed to append cfg argument");
ret = esp_gmf_args_desc_append(&reconfig_args, AMETHOD_ARG(DECODER, RECONFIG, CFG_SUBCFGSZ), ESP_GMF_ARGS_TYPE_UINT32,
sizeof(uint32_t), offsetof(esp_audio_simple_dec_cfg_t, cfg_size));
ESP_GMF_RET_ON_NOT_OK(TAG, ret, return ret, "Failed to append cfg_sz argument");
ret = esp_gmf_args_desc_append_array(&set_args, AMETHOD_ARG(DECODER, RECONFIG, CFG), reconfig_args,
sizeof(esp_audio_simple_dec_cfg_t), 0);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, return ret, "Failed to append config argument");
ret = esp_gmf_method_append(&method, AMETHOD(DECODER, RECONFIG), __audio_dec_reconfig, set_args);
ESP_GMF_RET_ON_ERROR(TAG, ret, return ret, "Failed to register %s method", AMETHOD(DECODER, RECONFIG));
esp_gmf_element_t *el = (esp_gmf_element_t *)handle;
el->method = method;
return ESP_GMF_ERR_OK;
}
esp_gmf_err_t esp_gmf_audio_dec_reconfig(esp_gmf_element_handle_t handle, esp_audio_simple_dec_cfg_t *config)
{
ESP_GMF_NULL_CHECK(TAG, handle, return ESP_GMF_ERR_INVALID_ARG);
ESP_GMF_NULL_CHECK(TAG, config, return ESP_GMF_ERR_INVALID_ARG);
esp_gmf_event_state_t state = ESP_GMF_EVENT_STATE_NONE;
esp_gmf_element_get_state(handle, &state);
if (state < ESP_GMF_EVENT_STATE_OPENING) {
esp_audio_simple_dec_cfg_t *new_config = NULL;
esp_gmf_err_t ret = dupl_esp_audio_simple_cfg(config, &new_config);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, return ret, "Failed to duplicate audio decoder configuration");
free_esp_audio_simple_cfg(OBJ_GET_CFG(handle));
esp_gmf_obj_set_config(handle, new_config, sizeof(*config));
return ESP_GMF_ERR_OK;
} else {
ESP_LOGE(TAG, "Failed to reconfig decoder due to invalid state: %s", esp_gmf_event_get_state_str(state));
return ESP_GMF_ERR_FAIL;
}
}
esp_gmf_err_t esp_gmf_audio_dec_reconfig_by_sound_info(esp_gmf_element_handle_t handle, esp_gmf_info_sound_t *info)
{
ESP_GMF_NULL_CHECK(TAG, handle, return ESP_GMF_ERR_INVALID_ARG);
esp_gmf_event_state_t state = ESP_GMF_EVENT_STATE_NONE;
esp_gmf_element_get_state(handle, &state);
if (state < ESP_GMF_EVENT_STATE_OPENING) {
esp_gmf_err_t ret = audio_dec_reconfig_dec_by_sound_info(handle, info);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, return ret, "Failed to reconfig simple decoder by sound information");
return ESP_GMF_ERR_OK;
} else {
ESP_LOGE(TAG, "Failed to reconfig decoder due to invalid state: %s", esp_gmf_event_get_state_str(state));
return ESP_GMF_ERR_FAIL;
}
}
esp_gmf_err_t esp_gmf_audio_dec_init(esp_audio_simple_dec_cfg_t *config, esp_gmf_element_handle_t *handle)
{
ESP_GMF_NULL_CHECK(TAG, handle, {return ESP_GMF_ERR_INVALID_ARG;});
*handle = NULL;
esp_gmf_err_t ret = ESP_GMF_ERR_OK;
esp_gmf_audio_dec_t *dec_hd = esp_gmf_oal_calloc(1, sizeof(esp_gmf_audio_dec_t));
ESP_GMF_MEM_VERIFY(TAG, dec_hd, {return ESP_GMF_ERR_MEMORY_LACK;}, "audio decoder", sizeof(esp_gmf_audio_dec_t));
esp_gmf_obj_t *obj = (esp_gmf_obj_t *)dec_hd;
obj->new_obj = esp_gmf_audio_dec_new;
obj->del_obj = esp_gmf_audio_dec_destroy;
esp_audio_simple_dec_cfg_t *cfg = NULL;
if (config) {
dupl_esp_audio_simple_cfg(config, &cfg);
} else {
esp_audio_simple_dec_cfg_t dcfg = DEFAULT_ESP_GMF_AUDIO_DEC_CONFIG();
dupl_esp_audio_simple_cfg(&dcfg, &cfg);
}
ESP_GMF_CHECK(TAG, cfg, {ret = ESP_GMF_ERR_MEMORY_LACK; goto ES_DEC_FAIL;}, "Failed to allocate audio decoder configuration");
esp_gmf_obj_set_config(obj, cfg, sizeof(esp_audio_simple_dec_cfg_t));
ret = esp_gmf_obj_set_tag(obj, "aud_dec");
ESP_GMF_RET_ON_NOT_OK(TAG, ret, goto ES_DEC_FAIL, "Failed to set obj tag");
esp_gmf_element_cfg_t el_cfg = {0};
ESP_GMF_ELEMENT_IN_PORT_ATTR_SET(el_cfg.in_attr, ESP_GMF_EL_PORT_CAP_SINGLE, 0, 0,
ESP_GMF_PORT_TYPE_BLOCK | ESP_GMF_PORT_TYPE_BYTE, 512);
ESP_GMF_ELEMENT_OUT_PORT_ATTR_SET(el_cfg.out_attr, ESP_GMF_EL_PORT_CAP_SINGLE, 0, 0,
ESP_GMF_PORT_TYPE_BLOCK | ESP_GMF_PORT_TYPE_BYTE, 512);
el_cfg.dependency = false;
ret = esp_gmf_audio_el_init(dec_hd, &el_cfg);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, goto ES_DEC_FAIL, "Failed to initialize audio decoder element");
ESP_LOGD(TAG, "Initialization, %s-%p", OBJ_GET_TAG(obj), obj);
ESP_GMF_ELEMENT_GET(dec_hd)->ops.open = esp_gmf_audio_dec_open;
ESP_GMF_ELEMENT_GET(dec_hd)->ops.process = esp_gmf_audio_dec_process;
ESP_GMF_ELEMENT_GET(dec_hd)->ops.close = esp_gmf_audio_dec_close;
ESP_GMF_ELEMENT_GET(dec_hd)->ops.load_caps = _load_dec_caps_func;
ESP_GMF_ELEMENT_GET(dec_hd)->ops.load_methods = _load_dec_methods_func;
*handle = obj;
return ESP_GMF_ERR_OK;
ES_DEC_FAIL:
esp_gmf_audio_dec_destroy(obj);
return ret;
}
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_audio/esp_gmf_audio_enc.c
+701
View File
@@ -0,0 +1,701 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#include <string.h>
#include "esp_log.h"
#include "esp_gmf_node.h"
#include "esp_gmf_oal_mem.h"
#include "esp_gmf_oal_mutex.h"
#include "esp_gmf_audio_enc.h"
#include "esp_audio_enc_default.h"
#include "esp_gmf_cache.h"
#include "esp_gmf_cap.h"
#include "esp_gmf_caps_def.h"
#include "esp_gmf_audio_methods_def.h"
#include "esp_gmf_audio_element.h"
#include "gmf_audio_common.h"
#define AUD_ENC_DEFAULT_INPUT_TIME_MS (20)
#define SET_ENC_BASIC_INFO(cfg, info) do { \
(cfg)->sample_rate = (info)->sample_rates; \
(cfg)->channel = (info)->channels; \
(cfg)->bits_per_sample = (info)->bits; \
} while (0)
/**
* @brief Audio encoder context in GMF
*/
typedef struct {
esp_gmf_audio_element_t parent; /*!< The GMF audio encoder handle */
esp_audio_enc_handle_t audio_enc_hd; /*!< The audio encoder handle */
esp_gmf_cache_t *cached_payload; /*!< A Cached payload for data concatenation */
uint32_t bitrate; /*!< The bitrate of the encoded data */
esp_gmf_payload_t *origin_in_load; /*!< The original input payload */
int64_t cur_pts; /*!< The audio Presentation Time Stamp(pts) */
} esp_gmf_audio_enc_t;
static const char *TAG = "ESP_GMF_AENC";
static esp_gmf_err_t __audio_enc_get_frame_size(esp_gmf_element_handle_t handle, esp_gmf_args_desc_t *arg_desc,
uint8_t *buf, int buf_len)
{
ESP_GMF_NULL_CHECK(TAG, arg_desc, return ESP_GMF_ERR_INVALID_ARG);
ESP_GMF_NULL_CHECK(TAG, buf, return ESP_GMF_ERR_INVALID_ARG);
esp_gmf_args_desc_t *enc_desc = arg_desc;
uint32_t *in_size = (uint32_t *)(buf);
enc_desc = enc_desc->next;
uint32_t *out_size = (uint32_t *)(buf + enc_desc->offset);
return esp_gmf_audio_enc_get_frame_size(handle, in_size, out_size);
}
static esp_gmf_err_t __audio_enc_set_bitrate(esp_gmf_element_handle_t handle, esp_gmf_args_desc_t *arg_desc,
uint8_t *buf, int buf_len)
{
ESP_GMF_NULL_CHECK(TAG, arg_desc, return ESP_GMF_ERR_INVALID_ARG);
ESP_GMF_NULL_CHECK(TAG, buf, return ESP_GMF_ERR_INVALID_ARG);
uint32_t bitrate = *((uint32_t *)buf);
return esp_gmf_audio_enc_set_bitrate(handle, bitrate);
}
static esp_gmf_err_t __audio_enc_get_bitrate(esp_gmf_element_handle_t handle, esp_gmf_args_desc_t *arg_desc,
uint8_t *buf, int buf_len)
{
ESP_GMF_NULL_CHECK(TAG, arg_desc, return ESP_GMF_ERR_INVALID_ARG);
ESP_GMF_NULL_CHECK(TAG, buf, return ESP_GMF_ERR_INVALID_ARG);
uint32_t *bitrate = (uint32_t *)buf;
return esp_gmf_audio_enc_get_bitrate(handle, bitrate);
}
static esp_gmf_err_t __audio_enc_reconfig(esp_gmf_element_handle_t handle, esp_gmf_args_desc_t *arg_desc,
uint8_t *buf, int buf_len)
{
ESP_GMF_NULL_CHECK(TAG, arg_desc, return ESP_GMF_ERR_INVALID_ARG);
ESP_GMF_NULL_CHECK(TAG, buf, return ESP_GMF_ERR_INVALID_ARG);
esp_audio_enc_config_t *config = (esp_audio_enc_config_t *)buf;
return esp_gmf_audio_enc_reconfig(handle, config);
}
static esp_gmf_err_t __audio_enc_reconfig_by_sound_info(esp_gmf_element_handle_t handle, esp_gmf_args_desc_t *arg_desc,
uint8_t *buf, int buf_len)
{
ESP_GMF_NULL_CHECK(TAG, arg_desc, return ESP_GMF_ERR_INVALID_ARG);
ESP_GMF_NULL_CHECK(TAG, buf, return ESP_GMF_ERR_INVALID_ARG);
esp_gmf_info_sound_t *snd_info = (esp_gmf_info_sound_t *)buf;
return esp_gmf_audio_enc_reconfig_by_sound_info(handle, snd_info);
}
static inline esp_gmf_err_t dupl_esp_gmf_audio_enc_cfg(esp_audio_enc_config_t *config, esp_audio_enc_config_t **new_config)
{
void *sub_cfg = NULL;
*new_config = esp_gmf_oal_calloc(1, sizeof(*config));
ESP_GMF_MEM_VERIFY(TAG, *new_config, {return ESP_GMF_ERR_MEMORY_LACK;}, "audio encoder handle configuration", sizeof(*config));
memcpy(*new_config, config, sizeof(*config));
if (config->cfg && (config->cfg_sz > 0)) {
sub_cfg = esp_gmf_oal_calloc(1, config->cfg_sz);
ESP_GMF_MEM_VERIFY(TAG, sub_cfg, {esp_gmf_oal_free(*new_config); return ESP_GMF_ERR_MEMORY_LACK;},
"audio encoder configuration", (int)config->cfg_sz);
memcpy(sub_cfg, config->cfg, config->cfg_sz);
(*new_config)->cfg = sub_cfg;
}
return ESP_GMF_JOB_ERR_OK;
}
static inline void free_esp_gmf_audio_enc_cfg(esp_audio_enc_config_t *config)
{
if (config) {
if (config->cfg) {
esp_gmf_oal_free(config->cfg);
config->cfg = NULL;
config->cfg_sz = 0;
}
esp_gmf_oal_free(config);
}
}
static inline void audio_enc_change_audio_info(esp_audio_enc_config_t *enc_cfg, esp_gmf_info_sound_t *info)
{
switch (enc_cfg->type) {
case ESP_AUDIO_TYPE_AAC: {
esp_aac_enc_config_t *aac_enc_cfg = (esp_aac_enc_config_t *)enc_cfg->cfg;
SET_ENC_BASIC_INFO(aac_enc_cfg, info);
break;
}
case ESP_AUDIO_TYPE_AMRNB: {
esp_amrnb_enc_config_t *amr_enc_cfg = (esp_amrnb_enc_config_t *)enc_cfg->cfg;
SET_ENC_BASIC_INFO(amr_enc_cfg, info);
break;
}
case ESP_AUDIO_TYPE_G711U:
case ESP_AUDIO_TYPE_G711A: {
esp_g711_enc_config_t *g711_enc_cfg = (esp_g711_enc_config_t *)enc_cfg->cfg;
SET_ENC_BASIC_INFO(g711_enc_cfg, info);
break;
}
case ESP_AUDIO_TYPE_AMRWB: {
esp_amrnb_enc_config_t *amr_enc_cfg = (esp_amrnb_enc_config_t *)enc_cfg->cfg;
SET_ENC_BASIC_INFO(amr_enc_cfg, info);
break;
}
case ESP_AUDIO_TYPE_ALAC: {
esp_alac_enc_config_t *alac_enc_cfg = (esp_alac_enc_config_t *)enc_cfg->cfg;
SET_ENC_BASIC_INFO(alac_enc_cfg, info);
break;
}
case ESP_AUDIO_TYPE_PCM: {
esp_pcm_enc_config_t *pcm_enc_cfg = (esp_pcm_enc_config_t *)enc_cfg->cfg;
SET_ENC_BASIC_INFO(pcm_enc_cfg, info);
break;
}
case ESP_AUDIO_TYPE_OPUS: {
esp_opus_enc_config_t *opus_enc_cfg = (esp_opus_enc_config_t *)enc_cfg->cfg;
SET_ENC_BASIC_INFO(opus_enc_cfg, info);
break;
}
case ESP_AUDIO_TYPE_ADPCM: {
esp_adpcm_enc_config_t *adpcm_enc_cfg = (esp_adpcm_enc_config_t *)enc_cfg->cfg;
SET_ENC_BASIC_INFO(adpcm_enc_cfg, info);
break;
}
case ESP_AUDIO_TYPE_LC3: {
esp_lc3_enc_config_t *lc3_enc_cfg = (esp_lc3_enc_config_t *)enc_cfg->cfg;
SET_ENC_BASIC_INFO(lc3_enc_cfg, info);
break;
}
case ESP_AUDIO_TYPE_SBC: {
esp_sbc_enc_config_t *sbc_enc_cfg = (esp_sbc_enc_config_t *)enc_cfg->cfg;
sbc_enc_cfg->sample_rate = info->sample_rates;
sbc_enc_cfg->bits_per_sample = info->bits;
if (info->channels == 1) {
sbc_enc_cfg->ch_mode = ESP_SBC_CH_MODE_MONO;
} else if (info->channels == 2) {
sbc_enc_cfg->ch_mode = ((sbc_enc_cfg->ch_mode > ESP_SBC_CH_MODE_MONO && sbc_enc_cfg->ch_mode <= ESP_SBC_CH_MODE_JOINT_STEREO) ? (sbc_enc_cfg->ch_mode) : (ESP_SBC_CH_MODE_DUAL));
} else {
sbc_enc_cfg->ch_mode = ESP_SBC_CH_MODE_INVALID;
}
break;
}
default:
break;
}
}
static esp_gmf_err_t audio_enc_set_subcfg(esp_audio_enc_config_t *enc_cfg, void *sub_cfg, int32_t sub_cfg_sz)
{
enc_cfg->cfg = esp_gmf_oal_calloc(1, sub_cfg_sz);
ESP_GMF_MEM_CHECK(TAG, enc_cfg->cfg, return ESP_GMF_ERR_MEMORY_LACK;);
enc_cfg->cfg_sz = sub_cfg_sz;
memcpy(enc_cfg->cfg, sub_cfg, sub_cfg_sz);
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t audio_enc_reconfig_enc_by_sound_info(esp_gmf_element_handle_t handle, esp_gmf_info_sound_t *info)
{
esp_audio_enc_config_t *cfg = (esp_audio_enc_config_t *)OBJ_GET_CFG(handle);
if (cfg == NULL) {
cfg = esp_gmf_oal_calloc(1, sizeof(esp_audio_enc_config_t));
ESP_GMF_MEM_VERIFY(TAG, cfg, return ESP_GMF_ERR_MEMORY_LACK, "audio encoder configuration", sizeof(esp_audio_enc_config_t));
esp_gmf_obj_set_config(handle, cfg, sizeof(esp_audio_enc_config_t));
}
esp_gmf_err_t ret = ESP_GMF_ERR_OK;
bool same_type = (cfg->type == info->format_id) ? true : false;
// free sub cfg first
if (cfg->cfg && (same_type == false)) {
esp_gmf_oal_free(cfg->cfg);
cfg->cfg = NULL;
cfg->cfg_sz = 0;
}
cfg->type = info->format_id;
if (same_type == true && (cfg->cfg != NULL)) {
audio_enc_change_audio_info(cfg, info);
return ESP_GMF_ERR_OK;
}
switch (info->format_id) {
case ESP_AUDIO_TYPE_AAC: {
esp_aac_enc_config_t aac_enc_cfg = ESP_AAC_ENC_CONFIG_DEFAULT();
SET_ENC_BASIC_INFO(&aac_enc_cfg, info);
aac_enc_cfg.bitrate = info->bitrate;
ret = audio_enc_set_subcfg(cfg, &aac_enc_cfg, sizeof(esp_aac_enc_config_t));
break;
}
case ESP_AUDIO_TYPE_AMRNB: {
esp_amrnb_enc_config_t amr_enc_cfg = ESP_AMRNB_ENC_CONFIG_DEFAULT();
SET_ENC_BASIC_INFO(&amr_enc_cfg, info);
amr_enc_cfg.bitrate_mode = info->bitrate;
ret = audio_enc_set_subcfg(cfg, &amr_enc_cfg, sizeof(esp_amrnb_enc_config_t));
break;
}
case ESP_AUDIO_TYPE_AMRWB: {
esp_amrnb_enc_config_t amr_enc_cfg = ESP_AMRNB_ENC_CONFIG_DEFAULT();
SET_ENC_BASIC_INFO(&amr_enc_cfg, info);
amr_enc_cfg.bitrate_mode = info->bitrate;
ret = audio_enc_set_subcfg(cfg, &amr_enc_cfg, sizeof(esp_amrnb_enc_config_t));
break;
}
case ESP_AUDIO_TYPE_G711A: {
esp_g711_enc_config_t g711_enc_cfg = ESP_G711_ENC_CONFIG_DEFAULT();
SET_ENC_BASIC_INFO(&g711_enc_cfg, info);
ret = audio_enc_set_subcfg(cfg, &g711_enc_cfg, sizeof(esp_g711_enc_config_t));
break;
}
case ESP_AUDIO_TYPE_G711U: {
esp_g711_enc_config_t g711_enc_cfg = ESP_G711_ENC_CONFIG_DEFAULT();
SET_ENC_BASIC_INFO(&g711_enc_cfg, info);
ret = audio_enc_set_subcfg(cfg, &g711_enc_cfg, sizeof(esp_g711_enc_config_t));
break;
}
case ESP_AUDIO_TYPE_ALAC: {
esp_alac_enc_config_t alac_enc_cfg = ESP_ALAC_ENC_CONFIG_DEFAULT();
SET_ENC_BASIC_INFO(&alac_enc_cfg, info);
ret = audio_enc_set_subcfg(cfg, &alac_enc_cfg, sizeof(esp_alac_enc_config_t));
break;
}
case ESP_AUDIO_TYPE_PCM: {
esp_pcm_enc_config_t pcm_enc_cfg = ESP_PCM_ENC_CONFIG_DEFAULT();
SET_ENC_BASIC_INFO(&pcm_enc_cfg, info);
ret = audio_enc_set_subcfg(cfg, &pcm_enc_cfg, sizeof(esp_pcm_enc_config_t));
break;
}
case ESP_AUDIO_TYPE_OPUS: {
esp_opus_enc_config_t opus_enc_cfg = ESP_OPUS_ENC_CONFIG_DEFAULT();
SET_ENC_BASIC_INFO(&opus_enc_cfg, info);
opus_enc_cfg.bitrate = info->bitrate;
ret = audio_enc_set_subcfg(cfg, &opus_enc_cfg, sizeof(esp_opus_enc_config_t));
break;
}
case ESP_AUDIO_TYPE_ADPCM: {
esp_adpcm_enc_config_t adpcm_enc_cfg = ESP_ADPCM_ENC_CONFIG_DEFAULT();
SET_ENC_BASIC_INFO(&adpcm_enc_cfg, info);
ret = audio_enc_set_subcfg(cfg, &adpcm_enc_cfg, sizeof(esp_adpcm_enc_config_t));
break;
}
case ESP_AUDIO_TYPE_LC3: {
esp_lc3_enc_config_t lc3_enc_cfg = ESP_LC3_ENC_CONFIG_DEFAULT();
SET_ENC_BASIC_INFO(&lc3_enc_cfg, info);
ret = audio_enc_set_subcfg(cfg, &lc3_enc_cfg, sizeof(esp_lc3_enc_config_t));
break;
}
case ESP_AUDIO_TYPE_SBC: {
esp_sbc_enc_config_t sbc_enc_cfg = ESP_SBC_STD_ENC_CONFIG_DEFAULT();
sbc_enc_cfg.sample_rate = info->sample_rates;
sbc_enc_cfg.bits_per_sample = info->bits;
if (info->channels == 1) {
sbc_enc_cfg.ch_mode = ESP_SBC_CH_MODE_MONO;
} else if (info->channels == 2) {
sbc_enc_cfg.ch_mode = ESP_SBC_CH_MODE_DUAL;
} else {
sbc_enc_cfg.ch_mode = ESP_SBC_CH_MODE_INVALID;
}
ret = audio_enc_set_subcfg(cfg, &sbc_enc_cfg, sizeof(esp_sbc_enc_config_t));
break;
}
default:
ESP_LOGE(TAG, "Not support for encoder, %ld", info->format_id);
cfg->type = ESP_AUDIO_TYPE_UNSUPPORT;
return ESP_GMF_ERR_NOT_SUPPORT;
}
return ((ret == ESP_GMF_ERR_OK) ? (ESP_GMF_ERR_OK) : (ret));
}
static esp_gmf_job_err_t gmf_audio_enc_acquire_in(esp_gmf_audio_enc_t *audio_enc, esp_gmf_port_handle_t in_port, esp_gmf_payload_t **in_load)
{
bool needed_load = false;
esp_gmf_job_err_t job_ret = ESP_GMF_JOB_ERR_OK;
esp_gmf_cache_ready_for_load(audio_enc->cached_payload, &needed_load);
if (needed_load) {
esp_gmf_err_io_t load_ret = esp_gmf_port_acquire_in(in_port, &audio_enc->origin_in_load, ESP_GMF_ELEMENT_GET(audio_enc)->in_attr.data_size, in_port->wait_ticks);
ESP_GMF_PORT_ACQUIRE_IN_CHECK(TAG, load_ret, job_ret, return job_ret);
int cache_size = 0;
esp_gmf_cache_get_cached_size(audio_enc->cached_payload, &cache_size);
audio_enc->cur_pts = audio_enc->origin_in_load->pts - GMF_AUDIO_CALC_PTS(cache_size, audio_enc->parent.snd_info.sample_rates, audio_enc->parent.snd_info.channels, audio_enc->parent.snd_info.bits);
esp_gmf_cache_load(audio_enc->cached_payload, audio_enc->origin_in_load);
}
esp_gmf_err_t ret = esp_gmf_cache_acquire(audio_enc->cached_payload, ESP_GMF_ELEMENT_GET(audio_enc)->in_attr.data_size, in_load);
if (ret != ESP_GMF_ERR_OK) {
job_ret = ((ret == ESP_GMF_ERR_NOT_ENOUGH) ? (ESP_GMF_JOB_ERR_CONTINUE) : (ESP_GMF_JOB_ERR_FAIL));
}
return job_ret;
}
static esp_gmf_err_t esp_gmf_audio_enc_new(void *cfg, esp_gmf_obj_handle_t *handle)
{
return esp_gmf_audio_enc_init(cfg, (esp_gmf_element_handle_t *)handle);
}
static esp_gmf_job_err_t esp_gmf_audio_enc_open(esp_gmf_element_handle_t self, void *para)
{
esp_gmf_audio_enc_t *enc = (esp_gmf_audio_enc_t *)self;
esp_audio_enc_config_t *enc_cfg = (esp_audio_enc_config_t *)OBJ_GET_CFG(enc);
ESP_GMF_CHECK(TAG, enc_cfg, {return ESP_GMF_JOB_ERR_FAIL;}, "There is no encoder configuration");
esp_audio_err_t ret = esp_audio_enc_open(enc_cfg, &enc->audio_enc_hd);
ESP_GMF_CHECK(TAG, enc->audio_enc_hd, {return ESP_GMF_JOB_ERR_FAIL;}, "Failed to create audio encoder handle");
if (esp_audio_enc_get_frame_size(enc->audio_enc_hd, &ESP_GMF_ELEMENT_GET(enc)->in_attr.data_size, &ESP_GMF_ELEMENT_GET(enc)->out_attr.data_size) != ESP_AUDIO_ERR_OK) {
ESP_LOGE(TAG, "Failed to obtain frame size, ret: %d", ret);
return ESP_GMF_JOB_ERR_FAIL;
}
esp_gmf_port_enable_payload_share(ESP_GMF_ELEMENT_GET(self)->in, false);
esp_gmf_cache_new(ESP_GMF_ELEMENT_GET(enc)->in_attr.data_size, &enc->cached_payload);
ESP_GMF_CHECK(TAG, enc->cached_payload, {return ESP_GMF_JOB_ERR_FAIL;}, "Failed to new a cached payload on open");
esp_audio_enc_info_t enc_info = {0};
esp_audio_enc_get_info(enc->audio_enc_hd, &enc_info);
GMF_AUDIO_UPDATE_SND_INFO(self, enc_info.sample_rate, enc_info.bits_per_sample, enc_info.channel);
ESP_LOGI(TAG, "Open, type:%s, acquire in frame: %d, out frame: %d", esp_audio_codec_get_name(enc_cfg->type), ESP_GMF_ELEMENT_GET(enc)->in_attr.data_size, ESP_GMF_ELEMENT_GET(enc)->out_attr.data_size);
return ESP_GMF_JOB_ERR_OK;
}
static esp_gmf_job_err_t esp_gmf_audio_enc_process(esp_gmf_element_handle_t self, void *para)
{
// Get audio encoder instance and initialize variables
esp_gmf_audio_enc_t *audio_enc = (esp_gmf_audio_enc_t *)self;
esp_gmf_job_err_t out_len = ESP_GMF_JOB_ERR_OK;
esp_audio_err_t ret = ESP_AUDIO_ERR_OK;
esp_gmf_port_handle_t in_port = ESP_GMF_ELEMENT_GET(self)->in;
esp_gmf_port_handle_t out_port = ESP_GMF_ELEMENT_GET(self)->out;
esp_gmf_payload_t *out_load = NULL;
esp_audio_enc_in_frame_t enc_in_frame = {0};
esp_audio_enc_out_frame_t enc_out_frame = {0};
esp_gmf_err_io_t load_ret = 0;
bool needed_load = false;
esp_gmf_payload_t *in_load = NULL;
out_len = gmf_audio_enc_acquire_in(audio_enc, in_port, &in_load);
if (out_len != ESP_GMF_JOB_ERR_OK) {
goto __audio_enc_release;
}
ESP_LOGD(TAG, "Acq cache, buf:%p, vld:%d, len:%d, done:%d", in_load->buf, in_load->valid_size, in_load->buf_length, in_load->is_done);
// Acquire output buffer for encoded data and verify output buffer size
load_ret = esp_gmf_port_acquire_out(out_port, &out_load, ESP_GMF_ELEMENT_GET(audio_enc)->out_attr.data_size, ESP_GMF_MAX_DELAY);
ESP_GMF_PORT_ACQUIRE_OUT_CHECK(TAG, load_ret, out_len, {goto __audio_enc_release;});
if (out_load->buf_length < (ESP_GMF_ELEMENT_GET(audio_enc)->out_attr.data_size)) {
ESP_LOGE(TAG, "The out payload valid size(%d) is smaller than wanted size(%d)",
out_load->buf_length, (ESP_GMF_ELEMENT_GET(audio_enc)->out_attr.data_size));
out_len = ESP_GMF_JOB_ERR_FAIL;
goto __audio_enc_release;
}
// Check if have enough input data for processing
if (in_load->valid_size != ESP_GMF_ELEMENT_GET(audio_enc)->in_attr.data_size) {
if (in_load->is_done == true) {
out_len = ESP_GMF_JOB_ERR_DONE;
out_load->valid_size = 0;
out_load->is_done = in_load->is_done;
ESP_LOGD(TAG, "Return done, line:%d", __LINE__);
goto __audio_enc_release;
} else {
out_len = ESP_GMF_JOB_ERR_CONTINUE;
ESP_LOGD(TAG, "Return Continue, line:%d", __LINE__);
goto __audio_enc_release;
}
}
// Perform audio encoding with mutex protection
enc_in_frame.buffer = in_load->buf;
enc_in_frame.len = in_load->valid_size;
enc_out_frame.buffer = out_load->buf;
enc_out_frame.len = ESP_GMF_ELEMENT_GET(audio_enc)->out_attr.data_size;
esp_gmf_oal_mutex_lock(((esp_gmf_audio_element_t *)self)->lock);
ret = esp_audio_enc_process(audio_enc->audio_enc_hd, &enc_in_frame, &enc_out_frame);
esp_gmf_oal_mutex_unlock(((esp_gmf_audio_element_t *)self)->lock);
ESP_GMF_RET_ON_ERROR(TAG, ret, {out_len = ESP_GMF_JOB_ERR_FAIL; goto __audio_enc_release;}, "Audio encoder process error %d", ret);
out_load->valid_size = enc_out_frame.encoded_bytes;
out_load->is_done = in_load->is_done;
out_load->pts = audio_enc->cur_pts;
audio_enc->cur_pts += GMF_AUDIO_CALC_PTS(enc_in_frame.len, audio_enc->parent.snd_info.sample_rates, audio_enc->parent.snd_info.channels, audio_enc->parent.snd_info.bits);
// Handle end of stream
if (in_load->is_done) {
ESP_LOGW(TAG, "Got done, out size: %d", out_load->valid_size);
out_len = ESP_GMF_JOB_ERR_DONE;
}
// Check if need to truncate
esp_gmf_cache_ready_for_load(audio_enc->cached_payload, &needed_load);
if (needed_load == false) {
out_len = ESP_GMF_JOB_ERR_TRUNCATE;
int cached_size = 0;
esp_gmf_cache_get_cached_size(audio_enc->cached_payload, &cached_size);
ESP_LOGD(TAG, "Return TRUNCATE, reminder in size: %d", cached_size);
}
__audio_enc_release:
// Cleanup resources
esp_gmf_cache_release(audio_enc->cached_payload, in_load);
if (out_load != NULL) {
load_ret = esp_gmf_port_release_out(out_port, out_load, out_port->wait_ticks);
if ((load_ret < ESP_GMF_IO_OK) && (load_ret != ESP_GMF_IO_ABORT)) {
ESP_LOGE(TAG, "OUT port release error, ret:%d", load_ret);
out_len = ESP_GMF_JOB_ERR_FAIL;
}
}
if ((audio_enc->origin_in_load != NULL) && (out_len != ESP_GMF_JOB_ERR_TRUNCATE)) {
load_ret = esp_gmf_port_release_in(in_port, audio_enc->origin_in_load, in_port->wait_ticks);
if ((load_ret < ESP_GMF_IO_OK) && (load_ret != ESP_GMF_IO_ABORT)) {
ESP_LOGE(TAG, "IN port release error, ret:%d", load_ret);
out_len = ESP_GMF_JOB_ERR_FAIL;
}
audio_enc->origin_in_load = NULL;
}
return out_len;
}
static esp_gmf_job_err_t esp_gmf_audio_enc_close(esp_gmf_element_handle_t self, void *para)
{
ESP_LOGD(TAG, "Closed, %p", self);
esp_gmf_audio_enc_t *enc = (esp_gmf_audio_enc_t *)self;
if (enc->cached_payload) {
esp_gmf_cache_delete(enc->cached_payload);
enc->cached_payload = NULL;
}
if (enc->audio_enc_hd != NULL) {
esp_audio_enc_close(enc->audio_enc_hd);
enc->audio_enc_hd = NULL;
}
return ESP_GMF_JOB_ERR_OK;
}
static esp_gmf_err_t audio_enc_received_event_handler(esp_gmf_event_pkt_t *evt, void *ctx)
{
ESP_GMF_NULL_CHECK(TAG, ctx, { return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_NULL_CHECK(TAG, evt, { return ESP_GMF_ERR_INVALID_ARG;});
if ((evt->type != ESP_GMF_EVT_TYPE_REPORT_INFO)
|| (evt->sub != ESP_GMF_INFO_SOUND)
|| (evt->payload == NULL)) {
return ESP_GMF_ERR_OK;
}
esp_gmf_element_handle_t self = (esp_gmf_element_handle_t)ctx;
esp_gmf_element_handle_t el = evt->from;
esp_gmf_event_state_t state = ESP_GMF_EVENT_STATE_NONE;
esp_gmf_element_get_state(self, &state);
if (state < ESP_GMF_EVENT_STATE_OPENING) {
esp_audio_enc_config_t *enc_cfg = (esp_audio_enc_config_t *)OBJ_GET_CFG(self);
ESP_GMF_NULL_CHECK(TAG, enc_cfg, return ESP_GMF_ERR_FAIL);
esp_gmf_info_sound_t *info = (esp_gmf_info_sound_t *)evt->payload;
audio_enc_change_audio_info(enc_cfg, info);
ESP_LOGD(TAG, "RECV info, from: %s-%p, next: %p, self: %s-%p, type: %x, state: %s, rate: %d, ch: %d, bits: %d",
OBJ_GET_TAG(el), el, esp_gmf_node_for_next((esp_gmf_node_t *)el), OBJ_GET_TAG(self), self, evt->type,
esp_gmf_event_get_state_str(state), info->sample_rates, info->channels, info->bits);
if (state == ESP_GMF_EVENT_STATE_NONE) {
esp_gmf_element_set_state(self, ESP_GMF_EVENT_STATE_INITIALIZED);
}
}
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t esp_gmf_audio_enc_destroy(esp_gmf_element_handle_t self)
{
ESP_LOGD(TAG, "Destroyed, %p", self);
esp_gmf_audio_enc_t *enc = (esp_gmf_audio_enc_t *)self;
free_esp_gmf_audio_enc_cfg(OBJ_GET_CFG(self));
esp_gmf_audio_el_deinit(self);
esp_gmf_oal_free(enc);
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t _load_enc_caps_func(esp_gmf_element_handle_t handle)
{
esp_gmf_cap_t *caps = NULL;
esp_gmf_cap_t dec_caps = {0};
dec_caps.cap_eightcc = ESP_GMF_CAPS_AUDIO_ENCODER;
dec_caps.attr_fun = NULL;
int ret = esp_gmf_cap_append(&caps, &dec_caps);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to create capability");
esp_gmf_element_t *el = (esp_gmf_element_t *)handle;
el->caps = caps;
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t _load_enc_methods_func(esp_gmf_element_handle_t handle)
{
esp_gmf_method_t *method = NULL;
esp_gmf_args_desc_t *set_args = NULL;
esp_gmf_args_desc_t *get_args = NULL;
esp_gmf_err_t ret = esp_gmf_args_desc_append(&set_args, AMETHOD_ARG(ENCODER, SET_BITRATE, BITRATE),
ESP_GMF_ARGS_TYPE_INT32, sizeof(uint32_t), 0);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, return ret, "Failed to append bitrate argument");
ret = esp_gmf_method_append(&method, AMETHOD(ENCODER, SET_BITRATE), __audio_enc_set_bitrate, set_args);
ESP_GMF_RET_ON_ERROR(TAG, ret, return ret, "Failed to register %s method", AMETHOD(ENCODER, SET_BITRATE));
ret = esp_gmf_args_desc_copy(set_args, &get_args);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, return ret, "Failed to copy argument");
ret = esp_gmf_method_append(&method, AMETHOD(ENCODER, GET_BITRATE), __audio_enc_get_bitrate, get_args);
ESP_GMF_RET_ON_ERROR(TAG, ret, return ret, "Failed to register %s method", AMETHOD(ENCODER, GET_BITRATE));
get_args = NULL;
set_args = NULL;
ret = esp_gmf_args_desc_append(&get_args, AMETHOD_ARG(ENCODER, GET_FRAME_SZ, INSIZE),
ESP_GMF_ARGS_TYPE_INT32, sizeof(uint32_t), 0);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, return ret, "Failed to append insize argument");
ret = esp_gmf_args_desc_append(&get_args, AMETHOD_ARG(ENCODER, GET_FRAME_SZ, OUTSIZE), ESP_GMF_ARGS_TYPE_INT32,
sizeof(uint32_t), sizeof(uint32_t));
ESP_GMF_RET_ON_NOT_OK(TAG, ret, return ret, "Failed to append outsize argument");
ret = esp_gmf_method_append(&method, AMETHOD(ENCODER, GET_FRAME_SZ), __audio_enc_get_frame_size, get_args);
ESP_GMF_RET_ON_ERROR(TAG, ret, return ret, "Failed to register %s method", AMETHOD(ENCODER, GET_FRAME_SZ));
set_args = NULL;
esp_gmf_args_desc_t *sndinfo_args = NULL;
ret = esp_gmf_args_desc_append(&sndinfo_args, AMETHOD_ARG(ENCODER, RECONFIG_BY_SND_INFO, INFO_TYPE), ESP_GMF_ARGS_TYPE_UINT32,
sizeof(uint32_t), offsetof(esp_gmf_info_sound_t, format_id));
ESP_GMF_RET_ON_NOT_OK(TAG, ret, return ret, "Failed to append type argument");
ret = esp_gmf_args_desc_append(&sndinfo_args, AMETHOD_ARG(ENCODER, RECONFIG_BY_SND_INFO, INFO_SAMPLERATE), ESP_GMF_ARGS_TYPE_INT32,
sizeof(int32_t), offsetof(esp_gmf_info_sound_t, sample_rates));
ESP_GMF_RET_ON_NOT_OK(TAG, ret, return ret, "Failed to append sample_rates argument");
ret = esp_gmf_args_desc_append(&sndinfo_args, AMETHOD_ARG(ENCODER, RECONFIG_BY_SND_INFO, INFO_BITRATE), ESP_GMF_ARGS_TYPE_INT32,
sizeof(int32_t), offsetof(esp_gmf_info_sound_t, bitrate));
ESP_GMF_RET_ON_NOT_OK(TAG, ret, return ret, "Failed to append bitrate argument");
ret = esp_gmf_args_desc_append(&sndinfo_args, AMETHOD_ARG(ENCODER, RECONFIG_BY_SND_INFO, INFO_CHANNEL), ESP_GMF_ARGS_TYPE_INT8,
sizeof(int8_t), 12);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, return ret, "Failed to append channels argument");
ret = esp_gmf_args_desc_append(&sndinfo_args, AMETHOD_ARG(ENCODER, RECONFIG_BY_SND_INFO, INFO_BITS), ESP_GMF_ARGS_TYPE_INT8,
sizeof(int8_t), 13);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, return ret, "Failed to append bits argument");
ret = esp_gmf_args_desc_append_array(&set_args, AMETHOD_ARG(ENCODER, RECONFIG_BY_SND_INFO, INFO), sndinfo_args,
sizeof(esp_gmf_info_sound_t), 0);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, return ret, "Failed to append sound info argument");
ret = esp_gmf_method_append(&method, AMETHOD(ENCODER, RECONFIG_BY_SND_INFO), __audio_enc_reconfig_by_sound_info, set_args);
ESP_GMF_RET_ON_ERROR(TAG, ret, return ret, "Failed to register %s method", AMETHOD(ENCODER, RECONFIG_BY_SND_INFO));
set_args = NULL;
esp_gmf_args_desc_t *reconfig_args = NULL;
ret = esp_gmf_args_desc_append(&reconfig_args, AMETHOD_ARG(ENCODER, RECONFIG, CFG_TYPE), ESP_GMF_ARGS_TYPE_INT32,
sizeof(int32_t), offsetof(esp_audio_enc_config_t, type));
ESP_GMF_RET_ON_NOT_OK(TAG, ret, return ret, "Failed to append type argument");
ret = esp_gmf_args_desc_append(&reconfig_args, AMETHOD_ARG(ENCODER, RECONFIG, CFG_SUBCFGPTR), ESP_GMF_ARGS_TYPE_INT32,
sizeof(int32_t), offsetof(esp_audio_enc_config_t, cfg));
ESP_GMF_RET_ON_NOT_OK(TAG, ret, return ret, "Failed to append cfg argument");
ret = esp_gmf_args_desc_append(&reconfig_args, AMETHOD_ARG(ENCODER, RECONFIG, CFG_SUBCFGSZ), ESP_GMF_ARGS_TYPE_UINT32,
sizeof(uint32_t), offsetof(esp_audio_enc_config_t, cfg_sz));
ESP_GMF_RET_ON_NOT_OK(TAG, ret, return ret, "Failed to append cfg_sz argument");
ret = esp_gmf_args_desc_append_array(&set_args, AMETHOD_ARG(ENCODER, RECONFIG, CFG), reconfig_args,
sizeof(esp_audio_enc_config_t), 0);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, return ret, "Failed to append argument");
ret = esp_gmf_method_append(&method, AMETHOD(ENCODER, RECONFIG), __audio_enc_reconfig, set_args);
ESP_GMF_RET_ON_ERROR(TAG, ret, return ret, "Failed to register %s method", AMETHOD(ENCODER, RECONFIG));
esp_gmf_element_t *el = (esp_gmf_element_t *)handle;
el->method = method;
return ESP_GMF_ERR_OK;
}
esp_gmf_err_t esp_gmf_audio_enc_get_frame_size(esp_gmf_element_handle_t handle, uint32_t *in_size, uint32_t *out_size)
{
ESP_GMF_NULL_CHECK(TAG, handle, return ESP_GMF_ERR_INVALID_ARG);
esp_audio_enc_config_t *cfg = (esp_audio_enc_config_t *)OBJ_GET_CFG(handle);
ESP_GMF_NULL_CHECK(TAG, cfg, return ESP_GMF_ERR_FAIL);
esp_gmf_audio_enc_t *enc = (esp_gmf_audio_enc_t *)handle;
esp_audio_err_t ret = -1;
if (enc->audio_enc_hd == NULL) {
esp_audio_enc_frame_info_t frame_info = {0};
ret = esp_audio_enc_get_frame_info_by_cfg(cfg, &frame_info);
if (ret != ESP_AUDIO_ERR_OK) {
return ESP_GMF_ERR_FAIL;
}
*in_size = (uint32_t)frame_info.in_frame_size;
*out_size = (uint32_t)frame_info.out_frame_size;
} else {
ret = esp_audio_enc_get_frame_size(enc->audio_enc_hd, (int *)in_size, (int *)out_size);
}
return ((ret == ESP_AUDIO_ERR_OK) ? (ESP_GMF_ERR_OK) : (ESP_GMF_ERR_FAIL));
}
esp_gmf_err_t esp_gmf_audio_enc_set_bitrate(esp_gmf_element_handle_t handle, uint32_t bitrate)
{
ESP_GMF_NULL_CHECK(TAG, handle, return ESP_GMF_ERR_INVALID_ARG);
esp_gmf_audio_enc_t *enc = (esp_gmf_audio_enc_t *)handle;
if (enc->audio_enc_hd) {
esp_gmf_oal_mutex_lock(((esp_gmf_audio_element_t *)handle)->lock);
esp_audio_err_t ret = esp_audio_enc_set_bitrate(enc->audio_enc_hd, (int)bitrate);
esp_gmf_oal_mutex_unlock(((esp_gmf_audio_element_t *)handle)->lock);
return ((ret == ESP_AUDIO_ERR_OK) ? (ESP_GMF_ERR_OK) : (ESP_GMF_ERR_FAIL));
}
enc->bitrate = bitrate;
return ESP_GMF_JOB_ERR_OK;
}
esp_gmf_err_t esp_gmf_audio_enc_get_bitrate(esp_gmf_element_handle_t handle, uint32_t *bitrate)
{
ESP_GMF_NULL_CHECK(TAG, handle, return ESP_GMF_ERR_INVALID_ARG);
esp_gmf_audio_enc_t *enc = (esp_gmf_audio_enc_t *)handle;
if (enc->audio_enc_hd) {
esp_audio_enc_info_t enc_info = {0};
esp_audio_err_t ret = esp_audio_enc_get_info(enc->audio_enc_hd, &enc_info);
*bitrate = enc_info.bitrate;
return ((ret == ESP_AUDIO_ERR_OK) ? (ESP_GMF_ERR_OK) : (ESP_GMF_ERR_FAIL));
}
*bitrate = enc->bitrate;
return ESP_GMF_JOB_ERR_OK;
}
esp_gmf_err_t esp_gmf_audio_enc_reconfig(esp_gmf_element_handle_t handle, esp_audio_enc_config_t *config)
{
ESP_GMF_NULL_CHECK(TAG, handle, return ESP_GMF_ERR_INVALID_ARG);
ESP_GMF_NULL_CHECK(TAG, config, return ESP_GMF_ERR_INVALID_ARG);
esp_gmf_event_state_t state = ESP_GMF_EVENT_STATE_NONE;
esp_gmf_element_get_state(handle, &state);
if (state < ESP_GMF_EVENT_STATE_OPENING) {
esp_audio_enc_config_t *new_config = NULL;
esp_gmf_err_t ret = dupl_esp_gmf_audio_enc_cfg(config, &new_config);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to duplicate config");
free_esp_gmf_audio_enc_cfg(OBJ_GET_CFG(handle));
esp_gmf_obj_set_config(handle, new_config, sizeof(esp_audio_enc_config_t));
return ESP_GMF_ERR_OK;
} else {
ESP_LOGE(TAG, "Failed to reconfig encoder due to invalid state: %s", esp_gmf_event_get_state_str(state));
return ESP_GMF_ERR_FAIL;
}
}
esp_gmf_err_t esp_gmf_audio_enc_reconfig_by_sound_info(esp_gmf_element_handle_t handle, esp_gmf_info_sound_t *info)
{
ESP_GMF_NULL_CHECK(TAG, handle, return ESP_GMF_ERR_INVALID_ARG);
esp_gmf_event_state_t state = ESP_GMF_EVENT_STATE_NONE;
esp_gmf_element_get_state(handle, &state);
if (state < ESP_GMF_EVENT_STATE_OPENING) {
esp_gmf_err_t ret = audio_enc_reconfig_enc_by_sound_info(handle, info);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, return ret, "Failed to reconfig encoder by sound information");
return ESP_GMF_ERR_OK;
} else {
ESP_LOGE(TAG, "Failed to reconfig encoder due to invalid state: %s", esp_gmf_event_get_state_str(state));
return ESP_GMF_ERR_FAIL;
}
}
esp_gmf_err_t esp_gmf_audio_enc_init(esp_audio_enc_config_t *config, esp_gmf_element_handle_t *handle)
{
ESP_GMF_NULL_CHECK(TAG, handle, {return ESP_GMF_ERR_INVALID_ARG;});
*handle = NULL;
esp_gmf_err_t ret = ESP_GMF_ERR_OK;
esp_gmf_audio_enc_t *audio_enc = esp_gmf_oal_calloc(1, sizeof(esp_gmf_audio_enc_t));
ESP_GMF_MEM_VERIFY(TAG, audio_enc, {return ESP_GMF_ERR_MEMORY_LACK;}, "audio encoder", sizeof(esp_gmf_audio_enc_t));
esp_gmf_obj_t *obj = (esp_gmf_obj_t *)audio_enc;
obj->new_obj = esp_gmf_audio_enc_new;
obj->del_obj = esp_gmf_audio_enc_destroy;
esp_audio_enc_config_t *cfg = NULL;
if (config) {
dupl_esp_gmf_audio_enc_cfg(config, &cfg);
} else {
esp_audio_enc_config_t dcfg = DEFAULT_ESP_GMF_AUDIO_ENC_CONFIG();
dupl_esp_gmf_audio_enc_cfg(&dcfg, &cfg);
}
ESP_GMF_CHECK(TAG, cfg, ret = ESP_GMF_ERR_MEMORY_LACK; goto ES_ENC_FAIL;, "Failed to allocate audio encoder configuration");
esp_gmf_obj_set_config(obj, cfg, sizeof(esp_audio_enc_config_t));
ret = esp_gmf_obj_set_tag(obj, "aud_enc");
ESP_GMF_RET_ON_NOT_OK(TAG, ret, goto ES_ENC_FAIL, "Failed to set obj tag");
esp_gmf_element_cfg_t el_cfg = {0};
ESP_GMF_ELEMENT_IN_PORT_ATTR_SET(el_cfg.in_attr, ESP_GMF_EL_PORT_CAP_SINGLE, 0, 0,
ESP_GMF_PORT_TYPE_BLOCK | ESP_GMF_PORT_TYPE_BYTE, ESP_GMF_ELEMENT_PORT_DATA_SIZE_DEFAULT);
ESP_GMF_ELEMENT_OUT_PORT_ATTR_SET(el_cfg.out_attr, ESP_GMF_EL_PORT_CAP_SINGLE, 0, 0,
ESP_GMF_PORT_TYPE_BLOCK | ESP_GMF_PORT_TYPE_BYTE, ESP_GMF_ELEMENT_PORT_DATA_SIZE_DEFAULT);
el_cfg.dependency = true;
ret = esp_gmf_audio_el_init(audio_enc, &el_cfg);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, goto ES_ENC_FAIL, "Failed to initialize audio encoder element");
audio_enc->parent.base.ops.open = esp_gmf_audio_enc_open;
audio_enc->parent.base.ops.process = esp_gmf_audio_enc_process;
audio_enc->parent.base.ops.close = esp_gmf_audio_enc_close;
audio_enc->parent.base.ops.event_receiver = audio_enc_received_event_handler;
audio_enc->parent.base.ops.load_caps = _load_enc_caps_func;
audio_enc->parent.base.ops.load_methods = _load_enc_methods_func;
*handle = obj;
ESP_LOGD(TAG, "Initialization, %s-%p", OBJ_GET_TAG(obj), obj);
return ret;
ES_ENC_FAIL:
esp_gmf_audio_enc_destroy(obj);
return ret;
}
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_audio/esp_gmf_audio_helper.c
+60
View File
@@ -0,0 +1,60 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#include <string.h>
#include "esp_log.h"
#include "esp_gmf_audio_helper.h"
#include "esp_fourcc.h"
static const char *TAG = "ESP_GMF_AUDIO_HELPER";
esp_gmf_err_t esp_gmf_audio_helper_get_audio_type_by_uri(const char *uri, uint32_t *format_id)
{
const char *ext = strrchr(uri, '.');
if (ext == NULL) {
return ESP_GMF_ERR_NOT_SUPPORT;
}
ext++;
if (strncasecmp(ext, "aac", 3) == 0) {
*format_id = ESP_FOURCC_AAC;
} else if (strncasecmp(ext, "g711a", 5) == 0) {
*format_id = ESP_FOURCC_ALAW;
} else if (strncasecmp(ext, "g711u", 5) == 0) {
*format_id = ESP_FOURCC_ULAW;
} else if (strncasecmp(ext, "amr", 3) == 0) {
*format_id = ESP_FOURCC_AMRNB;
} else if (strncasecmp(ext, "awb", 3) == 0) {
*format_id = ESP_FOURCC_AMRWB;
} else if (strncasecmp(ext, "alac", 4) == 0) {
*format_id = ESP_FOURCC_ALAC;
} else if (strncasecmp(ext, "pcm", 3) == 0) {
*format_id = ESP_FOURCC_PCM;
} else if (strncasecmp(ext, "opus", 4) == 0) {
*format_id = ESP_FOURCC_OPUS;
} else if (strncasecmp(ext, "adpcm", 5) == 0) {
*format_id = ESP_FOURCC_ADPCM;
} else if (strncasecmp(ext, "sbc", 3) == 0) {
*format_id = ESP_FOURCC_SBC;
} else if (strncasecmp(ext, "lc3", 3) == 0) {
*format_id = ESP_FOURCC_LC3;
} else if (strncasecmp(ext, "mp3", 3) == 0) {
*format_id = ESP_FOURCC_MP3;
} else if (strncasecmp(ext, "m4a", 3) == 0) {
*format_id = ESP_FOURCC_M4A;
} else if (strncasecmp(ext, "wav", 3) == 0) {
*format_id = ESP_FOURCC_WAV;
} else if (strncasecmp(ext, "ts", 2) == 0) {
*format_id = ESP_FOURCC_M2TS;
} else if (strncasecmp(ext, "flac", 4) == 0) {
*format_id = ESP_FOURCC_FLAC;
} else {
*format_id = 0;
ESP_LOGE(TAG, "Not support audio codec type, %s", uri);
return ESP_GMF_ERR_NOT_SUPPORT;
}
return ESP_GMF_ERR_OK;
}
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_audio/esp_gmf_audio_param.c
+102
View File
@@ -0,0 +1,102 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#include "esp_gmf_method_helper.h"
#include "esp_gmf_audio_param.h"
#include "esp_gmf_audio_methods_def.h"
#define PREPARE_AMETHOD_SETTING() \
esp_gmf_method_exec_ctx_t exec_ctx = {}; \
const esp_gmf_method_t* method_head = NULL; \
esp_gmf_element_get_method(self, &method_head); \
esp_gmf_err_t ret = esp_gmf_method_prepare_exec_ctx(method_head, method_name, &exec_ctx); \
if (ret != ESP_GMF_ERR_OK) { \
return ret; \
}
#define SET_AMETHOD_ARG(arg_name, value) \
esp_gmf_args_set_value(exec_ctx.method->args_desc, arg_name, exec_ctx.exec_buf, (uint8_t*) &value, sizeof(value));
#define RELEASE_AMETHOD_SETTING() \
ret = exec_ctx.method->func(self, exec_ctx.method->args_desc, exec_ctx.exec_buf, exec_ctx.buf_size); \
esp_gmf_method_release_exec_ctx(&exec_ctx); \
return ret;
esp_gmf_err_t esp_gmf_audio_param_set_dest_rate(esp_gmf_element_handle_t self, uint32_t dest_rate)
{
const char *method_name = AMETHOD(RATE_CVT, SET_DEST_RATE);
PREPARE_AMETHOD_SETTING();
SET_AMETHOD_ARG(AMETHOD_ARG(RATE_CVT, SET_DEST_RATE, RATE), dest_rate);
RELEASE_AMETHOD_SETTING();
}
esp_gmf_err_t esp_gmf_audio_param_set_dest_bits(esp_gmf_element_handle_t self, uint8_t dest_bits)
{
const char *method_name = AMETHOD(BIT_CVT, SET_DEST_BITS);
PREPARE_AMETHOD_SETTING();
SET_AMETHOD_ARG(AMETHOD_ARG(BIT_CVT, SET_DEST_BITS, BITS), dest_bits);
RELEASE_AMETHOD_SETTING();
}
esp_gmf_err_t esp_gmf_audio_param_set_dest_ch(esp_gmf_element_handle_t self, uint8_t dest_ch)
{
const char *method_name = AMETHOD(CH_CVT, SET_DEST_CH);
PREPARE_AMETHOD_SETTING();
SET_AMETHOD_ARG(AMETHOD_ARG(CH_CVT, SET_DEST_CH, CH), dest_ch);
RELEASE_AMETHOD_SETTING();
}
esp_gmf_err_t esp_gmf_audio_param_set_speed(esp_gmf_element_handle_t self, float speed)
{
const char *method_name = AMETHOD(SONIC, SET_SPEED);
PREPARE_AMETHOD_SETTING();
SET_AMETHOD_ARG(AMETHOD_ARG(SONIC, SET_SPEED, SPEED), speed);
RELEASE_AMETHOD_SETTING();
}
esp_gmf_err_t esp_gmf_audio_param_set_pitch(esp_gmf_element_handle_t self, float pitch)
{
const char *method_name = AMETHOD(SONIC, SET_PITCH);
PREPARE_AMETHOD_SETTING();
SET_AMETHOD_ARG(AMETHOD_ARG(SONIC, SET_PITCH, PITCH), pitch);
RELEASE_AMETHOD_SETTING();
}
esp_gmf_err_t esp_gmf_audio_param_set_alc_channel_gain(esp_gmf_element_handle_t self, uint8_t ch_idx, float db)
{
const char *method_name = AMETHOD(ALC, SET_GAIN);
// ALC currently only support int8 db
int8_t gain = (int8_t) db;
PREPARE_AMETHOD_SETTING();
SET_AMETHOD_ARG(AMETHOD_ARG(ALC, SET_GAIN, IDX), ch_idx);
SET_AMETHOD_ARG(AMETHOD_ARG(ALC, SET_GAIN, GAIN), gain);
RELEASE_AMETHOD_SETTING();
}
esp_gmf_err_t esp_gmf_audio_param_set_fade(esp_gmf_element_handle_t self, bool is_fade_in)
{
const char *method_name = AMETHOD(FADE, SET_MODE);
// Aligned with ESP_AE_FADE_MODE_FADE_IN = 1, ESP_AE_FADE_MODE_FADE_OUT = 2
int32_t mode = is_fade_in ? 1 : 2;
PREPARE_AMETHOD_SETTING();
SET_AMETHOD_ARG(AMETHOD_ARG(FADE, SET_MODE, MODE), mode);
RELEASE_AMETHOD_SETTING();
}
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_audio/esp_gmf_bit_cvt.c
+280
View File
@@ -0,0 +1,280 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#include <string.h>
#include "esp_log.h"
#include "esp_gmf_node.h"
#include "esp_gmf_oal_mem.h"
#include "esp_gmf_oal_mutex.h"
#include "esp_gmf_bit_cvt.h"
#include "gmf_audio_common.h"
#include "esp_gmf_audio_methods_def.h"
#include "esp_gmf_cap.h"
#include "esp_gmf_caps_def.h"
#include "esp_gmf_audio_element.h"
/**
* @brief Audio bit conversion context in GMF
*/
typedef struct {
esp_gmf_audio_element_t parent; /*!< The GMF bit cvt handle */
esp_ae_bit_cvt_handle_t bit_hd; /*!< The audio effects bit cvt handle */
uint8_t in_bytes_per_sample; /*!< Source bytes number of per sampling point */
uint8_t out_bytes_per_sample; /*!< Dest bytes number of per sampling point */
bool need_reopen : 1; /*!< Whether need to reopen.
True: Execute the close function first, then execute the open function
False: Do nothing */
bool bypass : 1; /*!< Whether bypass. True: need bypass. False: needn't bypass */
} esp_gmf_bit_cvt_t;
static const char *TAG = "ESP_GMF_BIT_CVT";
static esp_gmf_err_t __set_dest_bits(esp_gmf_element_handle_t handle, esp_gmf_args_desc_t *arg_desc,
uint8_t *buf, int buf_len)
{
ESP_GMF_NULL_CHECK(TAG, buf, {return ESP_GMF_ERR_INVALID_ARG;});
uint8_t dest_bits = (uint8_t)(*buf);
return esp_gmf_bit_cvt_set_dest_bits(handle, dest_bits);
}
static esp_gmf_err_t esp_gmf_bit_cvt_new(void *cfg, esp_gmf_obj_handle_t *handle)
{
return esp_gmf_bit_cvt_init(cfg, (esp_gmf_element_handle_t *)handle);
}
static esp_gmf_job_err_t esp_gmf_bit_cvt_open(esp_gmf_element_handle_t self, void *para)
{
esp_gmf_bit_cvt_t *bit_cvt = (esp_gmf_bit_cvt_t *)self;
esp_ae_bit_cvt_cfg_t *bit_info = (esp_ae_bit_cvt_cfg_t *)OBJ_GET_CFG(self);
ESP_GMF_NULL_CHECK(TAG, bit_info, {return ESP_GMF_JOB_ERR_FAIL;});
esp_ae_bit_cvt_open(bit_info, &bit_cvt->bit_hd);
ESP_GMF_CHECK(TAG, bit_cvt->bit_hd, {return ESP_GMF_JOB_ERR_FAIL;}, "Failed to create bit conversion handle");
bit_cvt->in_bytes_per_sample = (bit_info->src_bits >> 3) * bit_info->channel;
bit_cvt->out_bytes_per_sample = (bit_info->dest_bits >> 3) * bit_info->channel;
GMF_AUDIO_UPDATE_SND_INFO(self, bit_info->sample_rate, bit_info->dest_bits, bit_info->channel);
ESP_LOGD(TAG, "Open, rate: %ld, channel: %d, src_bits: %d, dest_bits: %d",
bit_info->sample_rate, bit_info->channel, bit_info->src_bits, bit_info->dest_bits);
bit_cvt->need_reopen = false;
bit_cvt->bypass = (bit_info->src_bits == bit_info->dest_bits);
return ESP_GMF_JOB_ERR_OK;
}
static esp_gmf_job_err_t esp_gmf_bit_cvt_close(esp_gmf_element_handle_t self, void *para)
{
esp_gmf_bit_cvt_t *bit_cvt = (esp_gmf_bit_cvt_t *)self;
ESP_LOGD(TAG, "Closed, %p", self);
if (bit_cvt->bit_hd != NULL) {
esp_ae_bit_cvt_close(bit_cvt->bit_hd);
bit_cvt->bit_hd = NULL;
}
return ESP_GMF_ERR_OK;
}
static esp_gmf_job_err_t esp_gmf_bit_cvt_process(esp_gmf_element_handle_t self, void *para)
{
esp_gmf_bit_cvt_t *bit_cvt = (esp_gmf_bit_cvt_t *)self;
esp_gmf_job_err_t out_len = ESP_GMF_JOB_ERR_OK;
if (bit_cvt->need_reopen) {
esp_gmf_bit_cvt_close(self, NULL);
out_len = esp_gmf_bit_cvt_open(self, NULL);
if (out_len != ESP_GMF_JOB_ERR_OK) {
ESP_LOGE(TAG, "Bit conversion reopen failed");
return out_len;
}
}
esp_gmf_port_handle_t in_port = ESP_GMF_ELEMENT_GET(self)->in;
esp_gmf_port_handle_t out_port = ESP_GMF_ELEMENT_GET(self)->out;
esp_gmf_payload_t *in_load = NULL;
esp_gmf_payload_t *out_load = NULL;
int samples_num = ESP_GMF_ELEMENT_GET(bit_cvt)->in_attr.data_size / (bit_cvt->in_bytes_per_sample);
int bytes = samples_num * bit_cvt->in_bytes_per_sample;
esp_gmf_err_io_t load_ret = esp_gmf_port_acquire_in(in_port, &in_load, bytes, ESP_GMF_MAX_DELAY);
if (load_ret < ESP_GMF_IO_OK) {
ESP_LOGE(TAG, "ACQ IN, ret: %d", load_ret);
out_len = ESP_GMF_JOB_ERR_FAIL;
goto __bit_release;
}
samples_num = in_load->valid_size / (bit_cvt->in_bytes_per_sample);
bytes = samples_num * bit_cvt->out_bytes_per_sample;
if ((bit_cvt->in_bytes_per_sample * samples_num != in_load->valid_size) || (load_ret < ESP_GMF_IO_OK)) {
ESP_LOGE(TAG, "Invalid in load size %d, ret %d", in_load->valid_size, load_ret);
out_len = ESP_GMF_JOB_ERR_FAIL;
goto __bit_release;
}
if ((bit_cvt->bypass) && (in_port->is_shared == 1)) {
// This case bit conversion is do bypass
out_load = in_load;
}
load_ret = esp_gmf_port_acquire_out(out_port, &out_load, samples_num ? bytes : in_load->buf_length, ESP_GMF_MAX_DELAY);
ESP_GMF_PORT_ACQUIRE_OUT_CHECK(TAG, load_ret, out_len, { goto __bit_release;});
if (samples_num) {
esp_ae_err_t ret = esp_ae_bit_cvt_process(bit_cvt->bit_hd, samples_num, (unsigned char *)in_load->buf, (unsigned char *)out_load->buf);
ESP_GMF_RET_ON_ERROR(TAG, ret, {out_len = ESP_GMF_JOB_ERR_FAIL; goto __bit_release;}, "Bit conversion process error, ret: %d", ret);
}
ESP_LOGV(TAG, "Samples: %d, IN-PLD: %p-%p-%d-%d-%d, OUT-PLD: %p-%p-%d-%d-%d",
samples_num, in_load, in_load->buf, in_load->valid_size, in_load->buf_length, in_load->is_done,
out_load, out_load->buf, out_load->valid_size, out_load->buf_length, out_load->is_done);
out_load->valid_size = samples_num * bit_cvt->out_bytes_per_sample;
out_load->pts = in_load->pts;
out_load->is_done = in_load->is_done;
if (out_load->valid_size > 0) {
esp_gmf_audio_el_update_file_pos((esp_gmf_element_handle_t)self, out_load->valid_size);
}
if (in_load->is_done) {
out_len = ESP_GMF_JOB_ERR_DONE;
ESP_LOGD(TAG, "Bit conversion is done, out len: %d", out_load->valid_size);
}
__bit_release:
// Release in and out port
if (out_load != NULL) {
load_ret = esp_gmf_port_release_out(out_port, out_load, ESP_GMF_MAX_DELAY);
if ((load_ret < ESP_GMF_IO_OK) && (load_ret != ESP_GMF_IO_ABORT)) {
ESP_LOGE(TAG, "OUT port release error, ret:%d", load_ret);
out_len = ESP_GMF_JOB_ERR_FAIL;
}
}
if (in_load != NULL) {
load_ret = esp_gmf_port_release_in(in_port, in_load, ESP_GMF_MAX_DELAY);
if ((load_ret < ESP_GMF_IO_OK) && (load_ret != ESP_GMF_IO_ABORT)) {
ESP_LOGE(TAG, "IN port release error, ret:%d", load_ret);
out_len = ESP_GMF_JOB_ERR_FAIL;
}
}
return out_len;
}
static esp_gmf_err_t bit_cvt_received_event_handler(esp_gmf_event_pkt_t *evt, void *ctx)
{
ESP_GMF_NULL_CHECK(TAG, ctx, {return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_NULL_CHECK(TAG, evt, {return ESP_GMF_ERR_INVALID_ARG;});
if ((evt->type != ESP_GMF_EVT_TYPE_REPORT_INFO)
|| (evt->sub != ESP_GMF_INFO_SOUND)
|| (evt->payload == NULL)) {
return ESP_GMF_ERR_OK;
}
esp_gmf_element_handle_t self = (esp_gmf_element_handle_t)ctx;
esp_gmf_element_handle_t el = evt->from;
esp_gmf_event_state_t state = ESP_GMF_EVENT_STATE_NONE;
esp_gmf_element_get_state(self, &state);
esp_gmf_info_sound_t *info = (esp_gmf_info_sound_t *)evt->payload;
esp_ae_bit_cvt_cfg_t *config = (esp_ae_bit_cvt_cfg_t *)OBJ_GET_CFG(self);
ESP_GMF_NULL_CHECK(TAG, config, return ESP_GMF_ERR_FAIL);
esp_gmf_bit_cvt_t *bit_cvt = (esp_gmf_bit_cvt_t *)self;
bit_cvt->need_reopen = (config->sample_rate != info->sample_rates) || (info->channels != config->channel) || (config->src_bits != info->bits);
config->sample_rate = info->sample_rates;
config->channel = info->channels;
config->src_bits = info->bits;
ESP_LOGD(TAG, "RECV info, from: %s-%p, next: %p, self: %s-%p, type: %x, state: %s, rate: %d, ch: %d, bits: %d",
OBJ_GET_TAG(el), el, esp_gmf_node_for_next((esp_gmf_node_t *)el), OBJ_GET_TAG(self), self, evt->type,
esp_gmf_event_get_state_str(state), info->sample_rates, info->channels, info->bits);
if (state == ESP_GMF_EVENT_STATE_NONE) {
esp_gmf_element_set_state(self, ESP_GMF_EVENT_STATE_INITIALIZED);
}
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t esp_gmf_bit_cvt_destroy(esp_gmf_element_handle_t self)
{
esp_gmf_bit_cvt_t *bit_cvt = (esp_gmf_bit_cvt_t *)self;
ESP_LOGD(TAG, "Destroyed, %p", self);
void *cfg = OBJ_GET_CFG(self);
if (cfg) {
esp_gmf_oal_free(cfg);
}
esp_gmf_audio_el_deinit(self);
esp_gmf_oal_free(bit_cvt);
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t _load_bit_cvt_caps_func(esp_gmf_element_handle_t handle)
{
esp_gmf_cap_t *caps = NULL;
esp_gmf_cap_t dec_caps = {0};
dec_caps.cap_eightcc = ESP_GMF_CAPS_AUDIO_BIT_CONVERT;
dec_caps.attr_fun = NULL;
int ret = esp_gmf_cap_append(&caps, &dec_caps);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to create capability");
esp_gmf_element_t *el = (esp_gmf_element_t *)handle;
el->caps = caps;
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t _load_bit_cvt_methods_func(esp_gmf_element_handle_t handle)
{
esp_gmf_method_t *method = NULL;
esp_gmf_args_desc_t *set_args = NULL;
esp_gmf_err_t ret = esp_gmf_args_desc_append(&set_args, AMETHOD_ARG(BIT_CVT, SET_DEST_BITS, BITS),
ESP_GMF_ARGS_TYPE_UINT8, sizeof(uint8_t), 0);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to append argument");
ret = esp_gmf_method_append(&method, AMETHOD(BIT_CVT, SET_DEST_BITS), __set_dest_bits, set_args);
ESP_GMF_RET_ON_ERROR(TAG, ret, {return ret;}, "Failed to register %s method", AMETHOD(BIT_CVT, SET_DEST_BITS));
esp_gmf_element_t *el = (esp_gmf_element_t *)handle;
el->method = method;
return ESP_GMF_ERR_OK;
}
esp_gmf_err_t esp_gmf_bit_cvt_set_dest_bits(esp_gmf_element_handle_t handle, uint8_t dest_bits)
{
ESP_GMF_NULL_CHECK(TAG, handle, { return ESP_GMF_ERR_INVALID_ARG;});
esp_ae_bit_cvt_cfg_t *cfg = (esp_ae_bit_cvt_cfg_t *)OBJ_GET_CFG(handle);
if (cfg) {
if (cfg->dest_bits == dest_bits) {
return ESP_GMF_ERR_OK;
}
cfg->dest_bits = dest_bits;
esp_gmf_bit_cvt_t *bit_cvt = (esp_gmf_bit_cvt_t *)handle;
bit_cvt->need_reopen = true;
return ESP_GMF_ERR_OK;
}
ESP_LOGE(TAG, "Failed to set dest bits, cfg is NULL");
return ESP_GMF_ERR_FAIL;
}
esp_gmf_err_t esp_gmf_bit_cvt_init(esp_ae_bit_cvt_cfg_t *config, esp_gmf_element_handle_t *handle)
{
ESP_GMF_NULL_CHECK(TAG, handle, {return ESP_GMF_ERR_INVALID_ARG;});
*handle = NULL;
esp_gmf_err_t ret = ESP_GMF_ERR_OK;
esp_gmf_bit_cvt_t *bit_cvt = esp_gmf_oal_calloc(1, sizeof(esp_gmf_bit_cvt_t));
ESP_GMF_MEM_VERIFY(TAG, bit_cvt, {return ESP_GMF_ERR_MEMORY_LACK;}, "bit conversion", sizeof(esp_gmf_bit_cvt_t));
esp_gmf_obj_t *obj = (esp_gmf_obj_t *)bit_cvt;
obj->new_obj = esp_gmf_bit_cvt_new;
obj->del_obj = esp_gmf_bit_cvt_destroy;
esp_ae_bit_cvt_cfg_t *cfg = esp_gmf_oal_calloc(1, sizeof(esp_ae_bit_cvt_cfg_t));
ESP_GMF_MEM_VERIFY(TAG, cfg, {ret = ESP_GMF_ERR_MEMORY_LACK; goto BIT_CVT_INIT_FAIL;}, "bit conversion configuration", sizeof(esp_ae_bit_cvt_cfg_t));
esp_gmf_obj_set_config(obj, cfg, sizeof(esp_ae_bit_cvt_cfg_t));
if (config) {
memcpy(cfg, config, sizeof(esp_ae_bit_cvt_cfg_t));
} else {
esp_ae_bit_cvt_cfg_t dcfg = DEFAULT_ESP_GMF_BIT_CVT_CONFIG();
memcpy(cfg, &dcfg, sizeof(esp_ae_bit_cvt_cfg_t));
}
ret = esp_gmf_obj_set_tag(obj, "aud_bit_cvt");
ESP_GMF_RET_ON_NOT_OK(TAG, ret, goto BIT_CVT_INIT_FAIL, "Failed to set obj tag");
esp_gmf_element_cfg_t el_cfg = {0};
ESP_GMF_ELEMENT_IN_PORT_ATTR_SET(el_cfg.in_attr, ESP_GMF_EL_PORT_CAP_SINGLE, 0, 0,
ESP_GMF_PORT_TYPE_BLOCK | ESP_GMF_PORT_TYPE_BYTE, ESP_GMF_ELEMENT_PORT_DATA_SIZE_DEFAULT);
ESP_GMF_ELEMENT_IN_PORT_ATTR_SET(el_cfg.out_attr, ESP_GMF_EL_PORT_CAP_SINGLE, 0, 0,
ESP_GMF_PORT_TYPE_BLOCK | ESP_GMF_PORT_TYPE_BYTE, ESP_GMF_ELEMENT_PORT_DATA_SIZE_DEFAULT);
el_cfg.dependency = true;
ret = esp_gmf_audio_el_init(bit_cvt, &el_cfg);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, goto BIT_CVT_INIT_FAIL, "Failed to initialize bit conversion element");
ESP_GMF_ELEMENT_GET(bit_cvt)->ops.open = esp_gmf_bit_cvt_open;
ESP_GMF_ELEMENT_GET(bit_cvt)->ops.process = esp_gmf_bit_cvt_process;
ESP_GMF_ELEMENT_GET(bit_cvt)->ops.close = esp_gmf_bit_cvt_close;
ESP_GMF_ELEMENT_GET(bit_cvt)->ops.event_receiver = bit_cvt_received_event_handler;
ESP_GMF_ELEMENT_GET(bit_cvt)->ops.load_caps = _load_bit_cvt_caps_func;
ESP_GMF_ELEMENT_GET(bit_cvt)->ops.load_methods = _load_bit_cvt_methods_func;
*handle = obj;
ESP_LOGD(TAG, "Initialization, %s-%p", OBJ_GET_TAG(obj), obj);
return ESP_GMF_ERR_OK;
BIT_CVT_INIT_FAIL:
esp_gmf_bit_cvt_destroy(obj);
return ret;
}
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_audio/esp_gmf_ch_cvt.c
+304
View File
@@ -0,0 +1,304 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#include <string.h>
#include "esp_log.h"
#include "esp_gmf_node.h"
#include "esp_gmf_oal_mem.h"
#include "esp_gmf_oal_mutex.h"
#include "esp_gmf_ch_cvt.h"
#include "gmf_audio_common.h"
#include "esp_gmf_audio_methods_def.h"
#include "esp_gmf_cap.h"
#include "esp_gmf_caps_def.h"
#include "esp_gmf_audio_element.h"
/**
* @brief Audio channel conversion context in GMF
*/
typedef struct {
esp_gmf_audio_element_t parent; /*!< The GMF channel cvt handle */
esp_ae_ch_cvt_handle_t ch_hd; /*!< The audio effects channel cvt handle */
uint8_t in_bytes_per_sample; /*!< Source bytes number of per sampling point */
uint8_t out_bytes_per_sample; /*!< Dest bytes number of per sampling point */
bool need_reopen : 1; /*!< Whether need to reopen.
True: Execute the close function first, then execute the open function
False: Do nothing */
bool bypass : 1; /*!< Whether bypass. True: need bypass. False: needn't bypass */
} esp_gmf_ch_cvt_t;
static const char *TAG = "ESP_GMF_CH_CVT";
static inline esp_gmf_err_t dupl_esp_ae_ch_cvt_cfg(esp_ae_ch_cvt_cfg_t *config, esp_ae_ch_cvt_cfg_t **new_config)
{
void *sub_cfg = NULL;
*new_config = esp_gmf_oal_calloc(1, sizeof(*config));
ESP_GMF_MEM_VERIFY(TAG, *new_config, {return ESP_GMF_ERR_MEMORY_LACK;}, "channel conversion configuration", sizeof(*config));
memcpy(*new_config, config, sizeof(*config));
if (config->weight && (config->weight_len > 0)) {
sub_cfg = esp_gmf_oal_calloc(1, config->weight_len);
ESP_GMF_MEM_VERIFY(TAG, sub_cfg, {esp_gmf_oal_free(*new_config); return ESP_GMF_ERR_MEMORY_LACK;},
"weight array", (int)config->weight_len);
memcpy(sub_cfg, config->weight, config->weight_len);
(*new_config)->weight = sub_cfg;
}
return ESP_GMF_JOB_ERR_OK;
}
static inline void free_esp_ae_ch_cvt_cfg(esp_ae_ch_cvt_cfg_t *config)
{
if (config) {
if (config->weight) {
esp_gmf_oal_free(config->weight);
config->weight = NULL;
config->weight_len = 0;
}
esp_gmf_oal_free(config);
}
}
static esp_gmf_err_t __set_dest_ch(esp_gmf_element_handle_t handle, esp_gmf_args_desc_t *arg_desc,
uint8_t *buf, int buf_len)
{
ESP_GMF_NULL_CHECK(TAG, buf, {return ESP_GMF_ERR_INVALID_ARG;});
uint8_t dest_ch = (uint8_t)(*buf);
return esp_gmf_ch_cvt_set_dest_channel(handle, dest_ch);
}
static esp_gmf_err_t esp_gmf_ch_cvt_new(void *cfg, esp_gmf_obj_handle_t *handle)
{
return esp_gmf_ch_cvt_init(cfg, (esp_gmf_element_handle_t *)handle);
}
static esp_gmf_job_err_t esp_gmf_ch_cvt_open(esp_gmf_element_handle_t self, void *para)
{
esp_gmf_ch_cvt_t *ch_cvt = (esp_gmf_ch_cvt_t *)self;
esp_ae_ch_cvt_cfg_t *ch_info = (esp_ae_ch_cvt_cfg_t *)OBJ_GET_CFG(self);
ESP_GMF_NULL_CHECK(TAG, ch_info, {return ESP_GMF_JOB_ERR_FAIL;});
esp_ae_ch_cvt_open(ch_info, &ch_cvt->ch_hd);
ESP_GMF_CHECK(TAG, ch_cvt->ch_hd, {return ESP_GMF_JOB_ERR_FAIL;}, "Failed to create channel conversion handle");
ch_cvt->in_bytes_per_sample = (ch_info->bits_per_sample >> 3) * ch_info->src_ch;
ch_cvt->out_bytes_per_sample = (ch_info->bits_per_sample >> 3) * ch_info->dest_ch;
GMF_AUDIO_UPDATE_SND_INFO(self, ch_info->sample_rate, ch_info->bits_per_sample, ch_info->dest_ch);
ESP_LOGD(TAG, "Open, rate: %ld, bits: %d, src_channel: %d, dest_channel: %d",
ch_info->sample_rate, ch_info->bits_per_sample, ch_info->src_ch, ch_info->dest_ch);
ch_cvt->need_reopen = false;
ch_cvt->bypass = ch_info->src_ch == ch_info->dest_ch;
return ESP_GMF_JOB_ERR_OK;
}
static esp_gmf_job_err_t esp_gmf_ch_cvt_close(esp_gmf_element_handle_t self, void *para)
{
esp_gmf_ch_cvt_t *ch_cvt = (esp_gmf_ch_cvt_t *)self;
ESP_LOGD(TAG, "Closed, %p", self);
if (ch_cvt->ch_hd != NULL) {
esp_ae_ch_cvt_close(ch_cvt->ch_hd);
ch_cvt->ch_hd = NULL;
}
return ESP_GMF_JOB_ERR_OK;
}
static esp_gmf_job_err_t esp_gmf_ch_cvt_process(esp_gmf_element_handle_t self, void *para)
{
esp_gmf_ch_cvt_t *ch_cvt = (esp_gmf_ch_cvt_t *)self;
esp_gmf_job_err_t out_len = ESP_GMF_JOB_ERR_OK;
if (ch_cvt->need_reopen) {
esp_gmf_ch_cvt_close(self, NULL);
out_len = esp_gmf_ch_cvt_open(self, NULL);
if (out_len != ESP_GMF_JOB_ERR_OK) {
ESP_LOGE(TAG, "Channel conversion reopen failed");
return out_len;
}
}
esp_gmf_port_handle_t in_port = ESP_GMF_ELEMENT_GET(self)->in;
esp_gmf_port_handle_t out_port = ESP_GMF_ELEMENT_GET(self)->out;
esp_gmf_payload_t *in_load = NULL;
esp_gmf_payload_t *out_load = NULL;
int samples_num = ESP_GMF_ELEMENT_GET(ch_cvt)->in_attr.data_size / (ch_cvt->in_bytes_per_sample);
int bytes = samples_num * ch_cvt->in_bytes_per_sample;
esp_gmf_err_io_t load_ret = esp_gmf_port_acquire_in(in_port, &in_load, bytes, ESP_GMF_MAX_DELAY);
if (load_ret < ESP_GMF_IO_OK) {
ESP_LOGE(TAG, "ACQ IN, ret: %d", load_ret);
out_len = ESP_GMF_JOB_ERR_FAIL;
goto __ch_release;
}
samples_num = in_load->valid_size / (ch_cvt->in_bytes_per_sample);
bytes = samples_num * ch_cvt->out_bytes_per_sample;
if ((ch_cvt->in_bytes_per_sample * samples_num != in_load->valid_size) || (load_ret < ESP_GMF_IO_OK)) {
ESP_LOGE(TAG, "Invalid in load size %d, ret %d", in_load->valid_size, load_ret);
out_len = ESP_GMF_JOB_ERR_FAIL;
goto __ch_release;
}
if (ch_cvt->bypass && (in_port->is_shared == true)) {
// This case channel conversion is do bypass
out_load = in_load;
}
load_ret = esp_gmf_port_acquire_out(out_port, &out_load, samples_num ? bytes : in_load->buf_length, ESP_GMF_MAX_DELAY);
ESP_GMF_PORT_ACQUIRE_OUT_CHECK(TAG, load_ret, out_len, { goto __ch_release;});
if (samples_num) {
esp_ae_err_t ret = esp_ae_ch_cvt_process(ch_cvt->ch_hd, samples_num, (unsigned char *)in_load->buf, (unsigned char *)out_load->buf);
ESP_GMF_RET_ON_ERROR(TAG, ret, {out_len = ESP_GMF_JOB_ERR_FAIL; goto __ch_release;}, "Channel conversion process error, ret: %d", ret);
}
out_load->valid_size = samples_num * ch_cvt->out_bytes_per_sample;
out_load->pts = in_load->pts;
out_load->is_done = in_load->is_done;
ESP_LOGV(TAG, "Samples: %d, IN-PLD: %p-%p-%d-%d-%d, OUT-PLD: %p-%p-%d-%d-%d",
samples_num, in_load, in_load->buf, in_load->valid_size, in_load->buf_length, in_load->is_done,
out_load, out_load->buf, out_load->valid_size, out_load->buf_length, out_load->is_done);
if (out_load->valid_size > 0) {
esp_gmf_audio_el_update_file_pos((esp_gmf_element_handle_t)self, out_load->valid_size);
}
if (in_load->is_done) {
out_len = ESP_GMF_JOB_ERR_DONE;
ESP_LOGD(TAG, "The channel cvt done, out len: %d", out_load->valid_size);
}
__ch_release:
// Release in and out port
if (out_load != NULL) {
load_ret = esp_gmf_port_release_out(out_port, out_load, ESP_GMF_MAX_DELAY);
if ((load_ret < ESP_GMF_IO_OK) && (load_ret != ESP_GMF_IO_ABORT)) {
ESP_LOGE(TAG, "OUT port release error, ret:%d", load_ret);
out_len = ESP_GMF_JOB_ERR_FAIL;
}
}
if (in_load != NULL) {
load_ret = esp_gmf_port_release_in(in_port, in_load, ESP_GMF_MAX_DELAY);
if ((load_ret < ESP_GMF_IO_OK) && (load_ret != ESP_GMF_IO_ABORT)) {
ESP_LOGE(TAG, "IN port release error, ret:%d", load_ret);
out_len = ESP_GMF_JOB_ERR_FAIL;
}
}
return out_len;
}
static esp_gmf_err_t ch_cvt_received_event_handler(esp_gmf_event_pkt_t *evt, void *ctx)
{
ESP_GMF_NULL_CHECK(TAG, ctx, {return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_NULL_CHECK(TAG, evt, {return ESP_GMF_ERR_INVALID_ARG;});
if ((evt->type != ESP_GMF_EVT_TYPE_REPORT_INFO)
|| (evt->sub != ESP_GMF_INFO_SOUND)
|| (evt->payload == NULL)) {
return ESP_GMF_ERR_OK;
}
esp_gmf_element_handle_t self = (esp_gmf_element_handle_t)ctx;
esp_gmf_element_handle_t el = evt->from;
esp_gmf_event_state_t state = ESP_GMF_EVENT_STATE_NONE;
esp_gmf_element_get_state(self, &state);
esp_gmf_info_sound_t *info = (esp_gmf_info_sound_t *)evt->payload;
esp_ae_ch_cvt_cfg_t *config = (esp_ae_ch_cvt_cfg_t *)OBJ_GET_CFG(self);
ESP_GMF_NULL_CHECK(TAG, config, return ESP_GMF_ERR_FAIL);
esp_gmf_ch_cvt_t *ch_cvt = (esp_gmf_ch_cvt_t *)self;
ch_cvt->need_reopen = (config->sample_rate != info->sample_rates) || (info->channels != config->src_ch) || (config->bits_per_sample != info->bits);
config->sample_rate = info->sample_rates;
config->src_ch = info->channels;
config->bits_per_sample = info->bits;
ESP_LOGD(TAG, "RECV info, from: %s-%p, next: %p, self: %s-%p, type: %x, state: %s, rate: %d, ch: %d, bits: %d",
OBJ_GET_TAG(el), el, esp_gmf_node_for_next((esp_gmf_node_t *)el), OBJ_GET_TAG(self), self, evt->type,
esp_gmf_event_get_state_str(state), info->sample_rates, info->channels, info->bits);
if (state == ESP_GMF_EVENT_STATE_NONE) {
esp_gmf_element_set_state(self, ESP_GMF_EVENT_STATE_INITIALIZED);
}
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t esp_gmf_ch_cvt_destroy(esp_gmf_element_handle_t self)
{
esp_gmf_ch_cvt_t *ch_cvt = (esp_gmf_ch_cvt_t *)self;
ESP_LOGD(TAG, "Destroyed, %p", self);
free_esp_ae_ch_cvt_cfg(OBJ_GET_CFG(self));
esp_gmf_audio_el_deinit(self);
esp_gmf_oal_free(ch_cvt);
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t _load_channel_cvt_caps_func(esp_gmf_element_handle_t handle)
{
esp_gmf_cap_t *caps = NULL;
esp_gmf_cap_t dec_caps = {0};
dec_caps.cap_eightcc = ESP_GMF_CAPS_AUDIO_CHANNEL_CONVERT;
dec_caps.attr_fun = NULL;
int ret = esp_gmf_cap_append(&caps, &dec_caps);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to create capability");
esp_gmf_element_t *el = (esp_gmf_element_t *)handle;
el->caps = caps;
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t _load_channel_cvt_methods_func(esp_gmf_element_handle_t handle)
{
esp_gmf_method_t *method = NULL;
esp_gmf_args_desc_t *set_args = NULL;
esp_gmf_err_t ret = esp_gmf_args_desc_append(&set_args, AMETHOD_ARG(CH_CVT, SET_DEST_CH, CH), ESP_GMF_ARGS_TYPE_UINT8, sizeof(uint8_t), 0);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to append argument");
ret = esp_gmf_method_append(&method, AMETHOD(CH_CVT, SET_DEST_CH), __set_dest_ch, set_args);
ESP_GMF_RET_ON_ERROR(TAG, ret, {return ret;}, "Failed to register %s method", AMETHOD(CH_CVT, SET_DEST_CH));
esp_gmf_element_t *el = (esp_gmf_element_t *)handle;
el->method = method;
return ESP_GMF_ERR_OK;
}
esp_gmf_err_t esp_gmf_ch_cvt_set_dest_channel(esp_gmf_element_handle_t handle, uint8_t dest_ch)
{
ESP_GMF_NULL_CHECK(TAG, handle, { return ESP_GMF_ERR_INVALID_ARG;});
esp_ae_ch_cvt_cfg_t *cfg = (esp_ae_ch_cvt_cfg_t *)OBJ_GET_CFG(handle);
if (cfg) {
if (cfg->dest_ch == dest_ch) {
return ESP_GMF_ERR_OK;
}
cfg->dest_ch = dest_ch;
esp_gmf_ch_cvt_t *ch_cvt = (esp_gmf_ch_cvt_t *)handle;
ch_cvt->need_reopen = true;
return ESP_GMF_ERR_OK;
}
ESP_LOGE(TAG, "Failed to set dest channel, cfg is NULL");
return ESP_GMF_ERR_FAIL;
}
esp_gmf_err_t esp_gmf_ch_cvt_init(esp_ae_ch_cvt_cfg_t *config, esp_gmf_element_handle_t *handle)
{
ESP_GMF_NULL_CHECK(TAG, handle, {return ESP_GMF_ERR_INVALID_ARG;});
*handle = NULL;
esp_gmf_err_t ret = ESP_GMF_ERR_OK;
esp_gmf_ch_cvt_t *ch_cvt = esp_gmf_oal_calloc(1, sizeof(esp_gmf_ch_cvt_t));
ESP_GMF_MEM_VERIFY(TAG, ch_cvt, {return ESP_GMF_ERR_MEMORY_LACK;}, "channel conversion", sizeof(esp_gmf_ch_cvt_t));
esp_gmf_obj_t *obj = (esp_gmf_obj_t *)ch_cvt;
obj->new_obj = esp_gmf_ch_cvt_new;
obj->del_obj = esp_gmf_ch_cvt_destroy;
esp_ae_ch_cvt_cfg_t *cfg = NULL;
if (config) {
dupl_esp_ae_ch_cvt_cfg(config, &cfg);
} else {
esp_ae_ch_cvt_cfg_t dcfg = DEFAULT_ESP_GMF_CH_CVT_CONFIG();
dupl_esp_ae_ch_cvt_cfg(&dcfg, &cfg);
}
ESP_GMF_CHECK(TAG, cfg, {ret = ESP_GMF_ERR_MEMORY_LACK; goto CH_CVT_INIT_FAIL;}, "Failed to allocate channel conversion configuration");
esp_gmf_obj_set_config(obj, cfg, sizeof(esp_ae_ch_cvt_cfg_t));
ret = esp_gmf_obj_set_tag(obj, "aud_ch_cvt");
ESP_GMF_RET_ON_NOT_OK(TAG, ret, goto CH_CVT_INIT_FAIL, "Failed to set obj tag");
esp_gmf_element_cfg_t el_cfg = {0};
ESP_GMF_ELEMENT_IN_PORT_ATTR_SET(el_cfg.in_attr, ESP_GMF_EL_PORT_CAP_SINGLE, 0, 0,
ESP_GMF_PORT_TYPE_BLOCK | ESP_GMF_PORT_TYPE_BYTE, ESP_GMF_ELEMENT_PORT_DATA_SIZE_DEFAULT);
ESP_GMF_ELEMENT_IN_PORT_ATTR_SET(el_cfg.out_attr, ESP_GMF_EL_PORT_CAP_SINGLE, 0, 0,
ESP_GMF_PORT_TYPE_BLOCK | ESP_GMF_PORT_TYPE_BYTE, ESP_GMF_ELEMENT_PORT_DATA_SIZE_DEFAULT);
el_cfg.dependency = true;
ret = esp_gmf_audio_el_init(ch_cvt, &el_cfg);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, goto CH_CVT_INIT_FAIL, "Failed to initialize channel conversion element");
ESP_GMF_ELEMENT_GET(ch_cvt)->ops.open = esp_gmf_ch_cvt_open;
ESP_GMF_ELEMENT_GET(ch_cvt)->ops.process = esp_gmf_ch_cvt_process;
ESP_GMF_ELEMENT_GET(ch_cvt)->ops.close = esp_gmf_ch_cvt_close;
ESP_GMF_ELEMENT_GET(ch_cvt)->ops.event_receiver = ch_cvt_received_event_handler;
ESP_GMF_ELEMENT_GET(ch_cvt)->ops.load_caps = _load_channel_cvt_caps_func;
ESP_GMF_ELEMENT_GET(ch_cvt)->ops.load_methods = _load_channel_cvt_methods_func;
*handle = obj;
ESP_LOGD(TAG, "Initialization, %s-%p", OBJ_GET_TAG(obj), obj);
return ESP_GMF_ERR_OK;
CH_CVT_INIT_FAIL:
esp_gmf_ch_cvt_destroy(obj);
return ret;
}
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_audio/esp_gmf_deinterleave.c
+274
View File
@@ -0,0 +1,274 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#include <string.h>
#include "esp_log.h"
#include "esp_gmf_oal_mem.h"
#include "esp_gmf_oal_mutex.h"
#include "esp_gmf_node.h"
#include "esp_gmf_deinterleave.h"
#include "esp_ae_data_weaver.h"
#include "esp_heap_caps.h"
#include "gmf_audio_common.h"
#include "esp_gmf_cap.h"
#include "esp_gmf_caps_def.h"
#include "esp_gmf_audio_element.h"
#define ESP_GMF_PROCESS_SAMPLE (256)
/**
* @brief Audio deinterleave context in GMF
*/
typedef struct {
esp_gmf_audio_element_t parent; /*!< The GMF deinterleave handle */
uint8_t bytes_per_sample; /*!< Bytes number of per sampling point */
esp_gmf_payload_t *in_load; /*!< The input payload */
esp_gmf_payload_t **out_load; /*!< The array of output payload */
uint8_t **out_arr; /*!< The array of output buffer pointer */
uint8_t channel; /*!< The audio channel */
uint8_t bits_per_sample; /*!< Bits number of per sampling point */
bool need_reopen : 1; /*!< Whether need to reopen.
True: Execute the close function first, then execute the open function
False: Do nothing */
} esp_gmf_deinterleave_t;
static const char *TAG = "ESP_GMF_DEINTLV";
static inline esp_gmf_err_t dupl_esp_ae_deinterleave_cfg(esp_gmf_deinterleave_cfg *config, esp_gmf_deinterleave_cfg **new_config)
{
*new_config = esp_gmf_oal_calloc(1, sizeof(*config));
ESP_GMF_MEM_VERIFY(TAG, *new_config, {return ESP_GMF_ERR_MEMORY_LACK;}, "deinterleave configuration", sizeof(*config));
memcpy(*new_config, config, sizeof(*config));
return ESP_GMF_JOB_ERR_OK;
}
static inline void free_esp_ae_deinterleave_cfg(esp_gmf_deinterleave_cfg *config)
{
if (config) {
esp_gmf_oal_free(config);
}
}
static esp_gmf_err_t esp_gmf_deinterleave_new(void *cfg, esp_gmf_obj_handle_t *handle)
{
return esp_gmf_deinterleave_init(cfg, (esp_gmf_element_handle_t *)handle);
}
static esp_gmf_job_err_t esp_gmf_deinterleave_open(esp_gmf_element_handle_t self, void *para)
{
esp_gmf_deinterleave_t *deinterleave = (esp_gmf_deinterleave_t *)self;
esp_gmf_deinterleave_cfg *deinterleave_info = (esp_gmf_deinterleave_cfg *)OBJ_GET_CFG(self);
ESP_GMF_NULL_CHECK(TAG, deinterleave_info, {return ESP_GMF_JOB_ERR_FAIL;})
deinterleave->bytes_per_sample = (deinterleave_info->bits_per_sample >> 3);
deinterleave->out_load = esp_gmf_oal_calloc(1, sizeof(esp_gmf_payload_t *) * deinterleave_info->channel);
ESP_GMF_MEM_VERIFY(TAG, deinterleave->out_load, {return ESP_GMF_JOB_ERR_FAIL;},
"out load", sizeof(esp_gmf_payload_t *) * deinterleave_info->channel);
deinterleave->out_arr = esp_gmf_oal_calloc(1, sizeof(uint8_t *) * deinterleave_info->channel);
ESP_GMF_MEM_VERIFY(TAG, deinterleave->out_arr, {return ESP_GMF_JOB_ERR_FAIL;},
"out buffer array", sizeof(uint8_t *) * deinterleave_info->channel);
GMF_AUDIO_UPDATE_SND_INFO(self, deinterleave_info->sample_rate, deinterleave_info->bits_per_sample, 1);
deinterleave->channel = deinterleave_info->channel;
deinterleave->bits_per_sample = deinterleave_info->bits_per_sample;
deinterleave->need_reopen = false;
ESP_LOGD(TAG, "Open, %p", self);
return ESP_GMF_ERR_OK;
}
static esp_gmf_job_err_t esp_gmf_deinterleave_close(esp_gmf_element_handle_t self, void *para)
{
esp_gmf_deinterleave_t *deinterleave = (esp_gmf_deinterleave_t *)self;
ESP_LOGD(TAG, "Closed, %p", self);
if (deinterleave->out_arr != NULL) {
esp_gmf_oal_free(deinterleave->out_arr);
deinterleave->out_arr = NULL;
}
if (deinterleave->out_load != NULL) {
esp_gmf_oal_free(deinterleave->out_load);
deinterleave->out_load = NULL;
}
return ESP_GMF_ERR_OK;
}
static esp_gmf_job_err_t esp_gmf_deinterleave_process(esp_gmf_element_handle_t self, void *para)
{
esp_gmf_deinterleave_t *deinterleave = (esp_gmf_deinterleave_t *)self;
esp_gmf_job_err_t out_len = ESP_GMF_JOB_ERR_OK;
int i = 0;
if (deinterleave->need_reopen) {
esp_gmf_deinterleave_close(self, NULL);
out_len = esp_gmf_deinterleave_open(self, NULL);
if (out_len != ESP_GMF_JOB_ERR_OK) {
ESP_LOGE(TAG, "deinterleave reopen failed");
return out_len;
}
}
esp_gmf_port_handle_t in_port = ESP_GMF_ELEMENT_GET(self)->in;
esp_gmf_port_handle_t out = ESP_GMF_ELEMENT_GET(self)->out;
esp_gmf_port_handle_t out_port = out;
deinterleave->in_load = NULL;
memset(deinterleave->out_load, 0, sizeof(esp_gmf_payload_t *) * deinterleave->channel);
int samples_num = ESP_GMF_ELEMENT_GET(deinterleave)->in_attr.data_size / (deinterleave->bytes_per_sample * deinterleave->channel);
int bytes = samples_num * deinterleave->bytes_per_sample * deinterleave->channel;
esp_gmf_err_io_t load_ret = esp_gmf_port_acquire_in(in_port, &deinterleave->in_load, bytes, ESP_GMF_MAX_DELAY);
samples_num = deinterleave->in_load->valid_size / (deinterleave->bytes_per_sample * deinterleave->channel);
bytes = samples_num * deinterleave->bytes_per_sample;
if (((bytes * deinterleave->channel) != deinterleave->in_load->valid_size) || (load_ret < ESP_GMF_IO_OK)) {
ESP_LOGE(TAG, "Invalid in load size %d, ret %d", deinterleave->in_load->valid_size, load_ret);
out_len = ESP_GMF_JOB_ERR_FAIL;
goto __deintlv_release;
}
ESP_LOGV(TAG, "IN: load: %p, buf: %p, valid size: %d, buf length: %d, done: %d",
deinterleave->in_load, deinterleave->in_load->buf, deinterleave->in_load->valid_size,
deinterleave->in_load->buf_length, deinterleave->in_load->is_done);
// Not do deinterleave anymore if one channel failed
while (out_port != NULL) {
load_ret = esp_gmf_port_acquire_out(out_port, &(deinterleave->out_load[i]),
samples_num ? bytes : deinterleave->in_load->buf_length, ESP_GMF_MAX_DELAY);
ESP_GMF_PORT_CHECK(TAG, load_ret, out_len, {out_len = ESP_GMF_JOB_ERR_FAIL; goto __deintlv_release;}, "Failed to acquire out, idx:%d, ret: %d", i, load_ret);
deinterleave->out_arr[i] = deinterleave->out_load[i]->buf;
out_port = out_port->next;
i++;
}
if (samples_num > 0) {
esp_ae_err_t proc_ret = esp_ae_deintlv_process(deinterleave->channel, deinterleave->bits_per_sample, samples_num,
deinterleave->in_load->buf, (void **)deinterleave->out_arr);
ESP_GMF_RET_ON_ERROR(TAG, proc_ret, {out_len = ESP_GMF_JOB_ERR_FAIL; goto __deintlv_release;}, "Deinterleave process error, ret: %d", proc_ret);
}
out_port = out;
i = 0;
while (out_port != NULL) {
deinterleave->out_load[i]->pts = deinterleave->in_load->pts;
deinterleave->out_load[i]->is_done = deinterleave->in_load->is_done;
deinterleave->out_load[i]->valid_size = samples_num * deinterleave->bytes_per_sample;
ESP_LOGV(TAG, "OUT: idx: %d load: %p, buf: %p, valid size: %d, buf length: %d, done: %d",
i, deinterleave->out_load[i], deinterleave->out_load[i]->buf, deinterleave->out_load[i]->valid_size,
deinterleave->out_load[i]->buf_length, deinterleave->out_load[i]->is_done);
out_port = out_port->next;
i++;
}
if (deinterleave->in_load->is_done) {
ESP_LOGD(TAG, "Deinterleave is done");
out_len = ESP_GMF_JOB_ERR_DONE;
}
__deintlv_release:
out_port = out;
i = 0;
while (out_port != NULL && deinterleave->out_load[i] != NULL) {
load_ret = esp_gmf_port_release_out(out_port, deinterleave->out_load[i], out_port->wait_ticks);
if ((load_ret < ESP_GMF_IO_OK) && (load_ret != ESP_GMF_IO_ABORT)) {
ESP_LOGE(TAG, "OUT port release error, ret:%d", load_ret);
out_len = ESP_GMF_JOB_ERR_FAIL;
}
out_port = out_port->next;
i++;
}
if (deinterleave->in_load != NULL) {
load_ret = esp_gmf_port_release_in(in_port, deinterleave->in_load, ESP_GMF_MAX_DELAY);
if ((load_ret < ESP_GMF_IO_OK) && (load_ret != ESP_GMF_IO_ABORT)) {
ESP_LOGE(TAG, "IN port release error, ret:%d", load_ret);
out_len = ESP_GMF_JOB_ERR_FAIL;
}
}
return out_len;
}
static esp_gmf_err_t deinterleave_received_event_handler(esp_gmf_event_pkt_t *evt, void *ctx)
{
ESP_GMF_NULL_CHECK(TAG, ctx, {return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_NULL_CHECK(TAG, evt, {return ESP_GMF_ERR_INVALID_ARG;});
if ((evt->type != ESP_GMF_EVT_TYPE_REPORT_INFO)
|| (evt->sub != ESP_GMF_INFO_SOUND)
|| (evt->payload == NULL)) {
return ESP_GMF_ERR_OK;
}
esp_gmf_element_handle_t self = (esp_gmf_element_handle_t)ctx;
esp_gmf_element_handle_t el = evt->from;
esp_gmf_event_state_t state = ESP_GMF_EVENT_STATE_NONE;
esp_gmf_element_get_state(self, &state);
esp_gmf_info_sound_t *info = (esp_gmf_info_sound_t *)evt->payload;
ESP_GMF_NULL_CHECK(TAG, info, { return ESP_GMF_ERR_FAIL;});
esp_gmf_deinterleave_cfg *config = (esp_gmf_deinterleave_cfg *)OBJ_GET_CFG(self);
ESP_GMF_NULL_CHECK(TAG, config, return ESP_GMF_ERR_FAIL);
esp_gmf_deinterleave_t *deinterleave = (esp_gmf_deinterleave_t *)self;
deinterleave->need_reopen = (config->sample_rate != info->sample_rates) || (info->channels != config->channel) || (config->bits_per_sample != info->bits);
config->sample_rate = info->sample_rates;
config->channel = info->channels;
config->bits_per_sample = info->bits;
ESP_LOGD(TAG, "RECV element info, from: %s-%p, next: %p, self: %s-%p, type: %x, state: %s, rate: %d, ch: %d, bits: %d",
OBJ_GET_TAG(el), el, esp_gmf_node_for_next((esp_gmf_node_t *)el), OBJ_GET_TAG(self), self, evt->type,
esp_gmf_event_get_state_str(state), info->sample_rates, info->channels, info->bits);
if (state == ESP_GMF_EVENT_STATE_NONE) {
esp_gmf_element_set_state(self, ESP_GMF_EVENT_STATE_INITIALIZED);
}
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t esp_gmf_deinterleave_destroy(esp_gmf_element_handle_t self)
{
esp_gmf_deinterleave_t *deinterleave = (esp_gmf_deinterleave_t *)self;
ESP_LOGD(TAG, "Destroyed, %p", self);
free_esp_ae_deinterleave_cfg(OBJ_GET_CFG(self));
esp_gmf_audio_el_deinit(self);
esp_gmf_oal_free(deinterleave);
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t _load_deinterleave_caps_func(esp_gmf_element_handle_t handle)
{
esp_gmf_cap_t *caps = NULL;
esp_gmf_cap_t dec_caps = {0};
dec_caps.cap_eightcc = ESP_GMF_CAPS_AUDIO_DEINTERLEAVE;
dec_caps.attr_fun = NULL;
int ret = esp_gmf_cap_append(&caps, &dec_caps);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to create capability");
esp_gmf_element_t *el = (esp_gmf_element_t *)handle;
el->caps = caps;
return ESP_GMF_ERR_OK;
}
esp_gmf_err_t esp_gmf_deinterleave_init(esp_gmf_deinterleave_cfg *config, esp_gmf_element_handle_t *handle)
{
ESP_GMF_NULL_CHECK(TAG, handle, {return ESP_GMF_ERR_INVALID_ARG;});
*handle = NULL;
esp_gmf_err_t ret = ESP_GMF_ERR_OK;
esp_gmf_deinterleave_t *deinterleave = esp_gmf_oal_calloc(1, sizeof(esp_gmf_deinterleave_t));
ESP_GMF_MEM_VERIFY(TAG, deinterleave, {return ESP_GMF_ERR_MEMORY_LACK;}, "deinterleave", sizeof(esp_gmf_deinterleave_t));
esp_gmf_obj_t *obj = (esp_gmf_obj_t *)deinterleave;
obj->new_obj = esp_gmf_deinterleave_new;
obj->del_obj = esp_gmf_deinterleave_destroy;
esp_gmf_deinterleave_cfg *cfg = NULL;
if (config) {
dupl_esp_ae_deinterleave_cfg(config, &cfg);
} else {
esp_gmf_deinterleave_cfg dcfg = DEFAULT_ESP_GMF_DEINTERLEAVE_CONFIG();
dupl_esp_ae_deinterleave_cfg(&dcfg, &cfg);
}
ESP_GMF_CHECK(TAG, cfg, ret = ESP_GMF_ERR_MEMORY_LACK; goto DEINTLV_INIT_FAIL;, "Failed to allocate deinterleave configuration");
esp_gmf_obj_set_config(obj, cfg, sizeof(esp_gmf_deinterleave_cfg));
ret = esp_gmf_obj_set_tag(obj, "aud_deintlv");
ESP_GMF_RET_ON_NOT_OK(TAG, ret, goto DEINTLV_INIT_FAIL, "Failed to set obj tag");
esp_gmf_element_cfg_t el_cfg = {0};
ESP_GMF_ELEMENT_IN_PORT_ATTR_SET(el_cfg.in_attr, ESP_GMF_EL_PORT_CAP_SINGLE, 0, 0,
ESP_GMF_PORT_TYPE_BLOCK | ESP_GMF_PORT_TYPE_BYTE, ESP_GMF_ELEMENT_PORT_DATA_SIZE_DEFAULT);
ESP_GMF_ELEMENT_IN_PORT_ATTR_SET(el_cfg.out_attr, ESP_GMF_EL_PORT_CAP_MULTI, 0, 0,
ESP_GMF_PORT_TYPE_BLOCK | ESP_GMF_PORT_TYPE_BYTE, ESP_GMF_ELEMENT_PORT_DATA_SIZE_DEFAULT);
el_cfg.dependency = true;
ret = esp_gmf_audio_el_init(deinterleave, &el_cfg);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, goto DEINTLV_INIT_FAIL, "Failed to initialize deinterleave element");
ESP_GMF_ELEMENT_GET(deinterleave)->ops.open = esp_gmf_deinterleave_open;
ESP_GMF_ELEMENT_GET(deinterleave)->ops.process = esp_gmf_deinterleave_process;
ESP_GMF_ELEMENT_GET(deinterleave)->ops.close = esp_gmf_deinterleave_close;
ESP_GMF_ELEMENT_GET(deinterleave)->ops.event_receiver = deinterleave_received_event_handler;
ESP_GMF_ELEMENT_GET(deinterleave)->ops.load_caps = _load_deinterleave_caps_func;
*handle = obj;
ESP_LOGD(TAG, "Initialization, %s-%p", OBJ_GET_TAG(obj), obj);
return ESP_GMF_ERR_OK;
DEINTLV_INIT_FAIL:
esp_gmf_deinterleave_destroy(obj);
return ret;
}
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_audio/esp_gmf_eq.c
+446
View File
@@ -0,0 +1,446 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#include <string.h>
#include "esp_log.h"
#include "esp_gmf_oal_mem.h"
#include "esp_gmf_oal_mutex.h"
#include "esp_gmf_node.h"
#include "esp_gmf_eq.h"
#include "esp_gmf_args_desc.h"
#include "gmf_audio_common.h"
#include "esp_gmf_audio_methods_def.h"
#include "esp_gmf_cap.h"
#include "esp_gmf_caps_def.h"
#include "esp_gmf_audio_element.h"
/**
* @brief Audio equalizer context in GMF
*/
typedef struct {
esp_gmf_audio_element_t parent; /*!< The GMF eq handle */
esp_ae_eq_handle_t eq_hd; /*!< The audio effects eq handle */
uint8_t bytes_per_sample; /*!< Bytes number of per sampling point */
bool *is_filter_enabled; /*!< The flag of whether eq filter is enabled */
bool need_reopen : 1; /*!< Whether need to reopen.
True: Execute the close function first, then execute the open function
False: Do nothing */
} esp_gmf_eq_t;
static const char *TAG = "ESP_GMF_EQ";
const esp_ae_eq_filter_para_t esp_gmf_default_eq_paras[10] = {
{ESP_AE_EQ_FILTER_PEAK, 31, 1.0, 0.0},
{ESP_AE_EQ_FILTER_PEAK, 62, 1.0, 0.0},
{ESP_AE_EQ_FILTER_PEAK, 125, 1.0, 0.0},
{ESP_AE_EQ_FILTER_PEAK, 250, 1.0, 1.0},
{ESP_AE_EQ_FILTER_PEAK, 500, 1.0, 2.0},
{ESP_AE_EQ_FILTER_PEAK, 1000, 1.0, 3.0},
{ESP_AE_EQ_FILTER_PEAK, 2000, 1.0, 3.0},
{ESP_AE_EQ_FILTER_PEAK, 4000, 1.0, 2.0},
{ESP_AE_EQ_FILTER_PEAK, 8000, 1.0, 1.0},
{ESP_AE_EQ_FILTER_PEAK, 16000, 1.0, 0.0},
};
static inline esp_gmf_err_t dupl_esp_ae_eq_cfg(esp_ae_eq_cfg_t *config, esp_ae_eq_cfg_t **new_config)
{
void *sub_cfg = NULL;
*new_config = esp_gmf_oal_calloc(1, sizeof(*config));
ESP_GMF_MEM_VERIFY(TAG, *new_config, {return ESP_GMF_ERR_MEMORY_LACK;}, "eq configuration", sizeof(*config));
memcpy(*new_config, config, sizeof(*config));
if (config->para && (config->filter_num > 0)) {
sub_cfg = esp_gmf_oal_calloc(1, config->filter_num * sizeof(esp_ae_eq_filter_para_t));
ESP_GMF_MEM_VERIFY(TAG, sub_cfg, {esp_gmf_oal_free(*new_config); return ESP_GMF_ERR_MEMORY_LACK;},
"filter parameter", config->filter_num * sizeof(esp_ae_eq_filter_para_t));
memcpy(sub_cfg, config->para, config->filter_num * sizeof(esp_ae_eq_filter_para_t));
(*new_config)->para = sub_cfg;
}
return ESP_GMF_JOB_ERR_OK;
}
static inline void free_esp_ae_eq_cfg(esp_ae_eq_cfg_t *config)
{
if (config) {
if (config->para && (config->para != esp_gmf_default_eq_paras)) {
esp_gmf_oal_free(config->para);
}
esp_gmf_oal_free(config);
}
}
static inline void eq_change_src_info(esp_gmf_element_handle_t self, uint32_t src_rate, uint8_t src_ch, uint8_t src_bits)
{
esp_ae_eq_cfg_t *eq_info = (esp_ae_eq_cfg_t *)OBJ_GET_CFG(self);
eq_info->channel = src_ch;
eq_info->sample_rate = src_rate;
eq_info->bits_per_sample = src_bits;
}
static esp_gmf_err_t __eq_set_para(esp_gmf_element_handle_t handle, esp_gmf_args_desc_t *arg_desc,
uint8_t *buf, int buf_len)
{
ESP_GMF_NULL_CHECK(TAG, handle, {return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_NULL_CHECK(TAG, arg_desc, {return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_NULL_CHECK(TAG, buf, {return ESP_GMF_ERR_INVALID_ARG;});
uint8_t idx = (uint8_t)(*buf);
return esp_gmf_eq_set_para(handle, idx, (esp_ae_eq_filter_para_t *)(buf + arg_desc->next->offset));
}
static esp_gmf_err_t __eq_get_para(esp_gmf_element_handle_t handle, esp_gmf_args_desc_t *arg_desc,
uint8_t *buf, int buf_len)
{
ESP_GMF_NULL_CHECK(TAG, handle, {return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_NULL_CHECK(TAG, arg_desc, {return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_NULL_CHECK(TAG, buf, {return ESP_GMF_ERR_INVALID_ARG;});
uint8_t idx = (uint8_t)(*buf);
esp_ae_eq_filter_para_t *para = (esp_ae_eq_filter_para_t *)(buf + arg_desc->next->offset);
return esp_gmf_eq_get_para(handle, idx, para);
}
static esp_gmf_err_t __eq_enable_filter(esp_gmf_element_handle_t handle, esp_gmf_args_desc_t *arg_desc,
uint8_t *buf, int buf_len)
{
ESP_GMF_NULL_CHECK(TAG, handle, {return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_NULL_CHECK(TAG, arg_desc, {return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_NULL_CHECK(TAG, buf, {return ESP_GMF_ERR_INVALID_ARG;});
esp_gmf_args_desc_t *filter_desc = arg_desc;
uint8_t idx = (uint8_t)(*buf);
filter_desc = filter_desc->next;
uint8_t is_enable = (uint8_t)(*(buf + filter_desc->offset));
return esp_gmf_eq_enable_filter(handle, idx,is_enable);
}
static esp_gmf_err_t esp_gmf_eq_new(void *cfg, esp_gmf_obj_handle_t *handle)
{
return esp_gmf_eq_init(cfg, (esp_gmf_element_handle_t *)handle);
}
static esp_gmf_job_err_t esp_gmf_eq_open(esp_gmf_element_handle_t self, void *para)
{
esp_gmf_eq_t *eq = (esp_gmf_eq_t *)self;
esp_ae_eq_cfg_t *eq_info = (esp_ae_eq_cfg_t *)OBJ_GET_CFG(self);
ESP_GMF_NULL_CHECK(TAG, eq_info, {return ESP_GMF_JOB_ERR_FAIL;});
eq->bytes_per_sample = (eq_info->bits_per_sample >> 3) * eq_info->channel;
esp_ae_eq_open(eq_info, &eq->eq_hd);
ESP_GMF_CHECK(TAG, eq->eq_hd, {return ESP_GMF_JOB_ERR_FAIL;}, "Failed to create eq handle");
GMF_AUDIO_UPDATE_SND_INFO(self, eq_info->sample_rate, eq_info->bits_per_sample, eq_info->channel);
for (int i = 0; i < eq_info->filter_num; i++) {
if (eq->is_filter_enabled[i]) {
esp_ae_eq_enable_filter(eq->eq_hd, i);
} else {
esp_ae_eq_disable_filter(eq->eq_hd, i);
}
}
eq->need_reopen = false;
ESP_LOGD(TAG, "Open, %p", eq);
return ESP_GMF_ERR_OK;
}
static esp_gmf_job_err_t esp_gmf_eq_close(esp_gmf_element_handle_t self, void *para)
{
esp_gmf_eq_t *eq = (esp_gmf_eq_t *)self;
ESP_LOGD(TAG, "Closed, %p", self);
if (eq->eq_hd != NULL) {
esp_ae_eq_close(eq->eq_hd);
eq->eq_hd = NULL;
}
return ESP_GMF_ERR_OK;
}
static esp_gmf_job_err_t esp_gmf_eq_process(esp_gmf_element_handle_t self, void *para)
{
esp_gmf_eq_t *eq = (esp_gmf_eq_t *)self;
esp_gmf_job_err_t out_len = ESP_GMF_JOB_ERR_OK;
if (eq->need_reopen) {
esp_gmf_eq_close(self, NULL);
out_len = esp_gmf_eq_open(self, NULL);
if (out_len != ESP_GMF_JOB_ERR_OK) {
ESP_LOGE(TAG, "EQ reopen failed");
return out_len;
}
}
esp_gmf_port_handle_t in_port = ESP_GMF_ELEMENT_GET(self)->in;
esp_gmf_port_handle_t out_port = ESP_GMF_ELEMENT_GET(self)->out;
esp_gmf_payload_t *in_load = NULL;
esp_gmf_payload_t *out_load = NULL;
int samples_num = ESP_GMF_ELEMENT_GET(eq)->in_attr.data_size / (eq->bytes_per_sample);
int bytes = samples_num * eq->bytes_per_sample;
esp_gmf_err_io_t load_ret = esp_gmf_port_acquire_in(in_port, &in_load, bytes, ESP_GMF_MAX_DELAY);
samples_num = in_load->valid_size / (eq->bytes_per_sample);
bytes = samples_num * eq->bytes_per_sample;
if ((bytes != in_load->valid_size) || (load_ret < ESP_GMF_IO_OK)) {
ESP_LOGE(TAG, "Invalid in load size %d, ret %d", in_load->valid_size, load_ret);
out_len = ESP_GMF_JOB_ERR_FAIL;
goto __eq_release;
}
if (in_port->is_shared == 1) {
out_load = in_load;
}
load_ret = esp_gmf_port_acquire_out(out_port, &out_load, samples_num ? bytes : in_load->buf_length, ESP_GMF_MAX_DELAY);
ESP_GMF_PORT_ACQUIRE_OUT_CHECK(TAG, load_ret, out_len, { goto __eq_release;});
if (samples_num > 0) {
esp_gmf_oal_mutex_lock(((esp_gmf_audio_element_t *)self)->lock);
esp_ae_err_t ret = esp_ae_eq_process(eq->eq_hd, samples_num, in_load->buf, out_load->buf);
esp_gmf_oal_mutex_unlock(((esp_gmf_audio_element_t *)self)->lock);
ESP_GMF_RET_ON_ERROR(TAG, ret, {out_len = ESP_GMF_JOB_ERR_FAIL; goto __eq_release;}, "Equalize process error %d", ret);
}
ESP_LOGV(TAG, "Samples: %d, IN-PLD: %p-%p-%d-%d-%d, OUT-PLD: %p-%p-%d-%d-%d",
samples_num, in_load, in_load->buf, in_load->valid_size, in_load->buf_length, in_load->is_done,
out_load, out_load->buf, out_load->valid_size, out_load->buf_length, out_load->is_done);
out_load->valid_size = samples_num * eq->bytes_per_sample;
out_load->is_done = in_load->is_done;
out_load->pts = in_load->pts;
if (out_load->valid_size > 0) {
esp_gmf_audio_el_update_file_pos((esp_gmf_element_handle_t)self, out_load->valid_size);
}
if (in_load->is_done) {
out_len = ESP_GMF_JOB_ERR_DONE;
ESP_LOGD(TAG, "Equalize done, out len: %d", out_load->valid_size);
}
__eq_release:
// Release in and out port
if (out_load != NULL) {
load_ret = esp_gmf_port_release_out(out_port, out_load, ESP_GMF_MAX_DELAY);
if ((load_ret < ESP_GMF_IO_OK) && (load_ret != ESP_GMF_IO_ABORT)) {
ESP_LOGE(TAG, "OUT port release error, ret:%d", load_ret);
out_len = ESP_GMF_JOB_ERR_FAIL;
}
}
if (in_load != NULL) {
load_ret = esp_gmf_port_release_in(in_port, in_load, ESP_GMF_MAX_DELAY);
if ((load_ret < ESP_GMF_IO_OK) && (load_ret != ESP_GMF_IO_ABORT)) {
ESP_LOGE(TAG, "IN port release error, ret:%d", load_ret);
out_len = ESP_GMF_JOB_ERR_FAIL;
}
}
return out_len;
}
static esp_gmf_err_t eq_received_event_handler(esp_gmf_event_pkt_t *evt, void *ctx)
{
ESP_GMF_NULL_CHECK(TAG, ctx, {return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_NULL_CHECK(TAG, evt, {return ESP_GMF_ERR_INVALID_ARG;});
if ((evt->type != ESP_GMF_EVT_TYPE_REPORT_INFO)
|| (evt->sub != ESP_GMF_INFO_SOUND)
|| (evt->payload == NULL)) {
return ESP_GMF_ERR_OK;
}
esp_gmf_element_handle_t self = (esp_gmf_element_handle_t)ctx;
esp_gmf_element_handle_t el = evt->from;
esp_gmf_event_state_t state = ESP_GMF_EVENT_STATE_NONE;
esp_gmf_element_get_state(self, &state);
esp_gmf_info_sound_t *info = (esp_gmf_info_sound_t *)evt->payload;
esp_ae_eq_cfg_t *config = (esp_ae_eq_cfg_t *)OBJ_GET_CFG(self);
ESP_GMF_NULL_CHECK(TAG, config, return ESP_GMF_ERR_FAIL);
esp_gmf_eq_t *eq = (esp_gmf_eq_t *)self;
eq->need_reopen = (config->sample_rate != info->sample_rates) || (info->channels != config->channel) || (config->bits_per_sample != info->bits);
config->sample_rate = info->sample_rates;
config->channel = info->channels;
config->bits_per_sample = info->bits;
ESP_LOGD(TAG, "RECV element info, from: %s-%p, next: %p, self: %s-%p, type: %x, state: %s, rate: %d, ch: %d, bits: %d",
OBJ_GET_TAG(el), el, esp_gmf_node_for_next((esp_gmf_node_t *)el), OBJ_GET_TAG(self), self, evt->type,
esp_gmf_event_get_state_str(state), info->sample_rates, info->channels, info->bits);
if (state == ESP_GMF_EVENT_STATE_NONE) {
esp_gmf_element_set_state(self, ESP_GMF_EVENT_STATE_INITIALIZED);
}
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t esp_gmf_eq_destroy(esp_gmf_element_handle_t self)
{
esp_gmf_eq_t *eq = (esp_gmf_eq_t *)self;
ESP_LOGD(TAG, "Destroyed, %p", self);
free_esp_ae_eq_cfg(OBJ_GET_CFG(self));
if(eq->is_filter_enabled){
esp_gmf_oal_free(eq->is_filter_enabled);
}
esp_gmf_audio_el_deinit(self);
esp_gmf_oal_free(eq);
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t _load_eq_caps_func(esp_gmf_element_handle_t handle)
{
esp_gmf_cap_t *caps = NULL;
esp_gmf_cap_t dec_caps = {0};
dec_caps.cap_eightcc = ESP_GMF_CAPS_AUDIO_EQUALIZER;
dec_caps.attr_fun = NULL;
int ret = esp_gmf_cap_append(&caps, &dec_caps);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to create capability");
esp_gmf_element_t *el = (esp_gmf_element_t *)handle;
el->caps = caps;
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t _load_eq_methods_func(esp_gmf_element_handle_t handle)
{
esp_gmf_method_t *method = NULL;
esp_gmf_args_desc_t *set_args = NULL;
esp_gmf_args_desc_t *get_args = NULL;
esp_gmf_args_desc_t *pointer_args = NULL;
esp_gmf_err_t ret = esp_gmf_args_desc_append(&pointer_args, AMETHOD_ARG(EQ, SET_PARA, PARA_FT), ESP_GMF_ARGS_TYPE_UINT32,
sizeof(uint32_t), offsetof(esp_ae_eq_filter_para_t, filter_type));
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to append FILTER argument");
ret = esp_gmf_args_desc_append(&pointer_args, AMETHOD_ARG(EQ, SET_PARA, PARA_FC), ESP_GMF_ARGS_TYPE_UINT32,
sizeof(uint32_t), offsetof(esp_ae_eq_filter_para_t, fc));
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to append FC argument");
ret = esp_gmf_args_desc_append(&pointer_args, AMETHOD_ARG(EQ, SET_PARA, PARA_Q), ESP_GMF_ARGS_TYPE_FLOAT,
sizeof(float), offsetof(esp_ae_eq_filter_para_t, q));
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to append Q argument");
ret = esp_gmf_args_desc_append(&pointer_args, AMETHOD_ARG(EQ, SET_PARA, PARA_GAIN), ESP_GMF_ARGS_TYPE_FLOAT,
sizeof(float), offsetof(esp_ae_eq_filter_para_t, gain));
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to append GAIN argument");
ret = esp_gmf_args_desc_append(&set_args, AMETHOD_ARG(EQ, SET_PARA, IDX), ESP_GMF_ARGS_TYPE_UINT8, sizeof(uint8_t), 0);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to append INDEX argument");
ret = esp_gmf_args_desc_append_array(&set_args, AMETHOD_ARG(EQ, SET_PARA, PARA), pointer_args,
sizeof(esp_ae_eq_filter_para_t), sizeof(uint8_t));
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to append PARA argument");
ret = esp_gmf_method_append(&method, AMETHOD(EQ, SET_PARA), __eq_set_para, set_args);
ESP_GMF_RET_ON_ERROR(TAG, ret, {return ret;}, "Failed to register %s method", AMETHOD(EQ, SET_PARA));
ret = esp_gmf_args_desc_copy(set_args, &get_args);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to copy PARA argument");
ret = esp_gmf_method_append(&method, AMETHOD(EQ, GET_PARA), __eq_get_para, get_args);
ESP_GMF_RET_ON_ERROR(TAG, ret, {return ret;}, "Failed to register %s method", AMETHOD(EQ, GET_PARA));
set_args = NULL;
get_args = NULL;
ret = esp_gmf_args_desc_append(&set_args, AMETHOD_ARG(EQ, ENABLE_FILTER, IDX), ESP_GMF_ARGS_TYPE_UINT8, sizeof(uint8_t), 0);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to append INDEX argument");
ret = esp_gmf_args_desc_append(&set_args, AMETHOD_ARG(EQ, ENABLE_FILTER, ENABLE), ESP_GMF_ARGS_TYPE_UINT8,
sizeof(uint8_t), sizeof(uint8_t));
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to append PARA argument");
ret = esp_gmf_method_append(&method, AMETHOD(EQ, ENABLE_FILTER), __eq_enable_filter, set_args);
ESP_GMF_RET_ON_ERROR(TAG, ret, {return ret;}, "Failed to register %s method", AMETHOD(EQ, ENABLE_FILTER));
esp_gmf_element_t *el = (esp_gmf_element_t *)handle;
el->method = method;
return ESP_GMF_ERR_OK;
}
esp_gmf_err_t esp_gmf_eq_set_para(esp_gmf_element_handle_t handle, uint8_t idx, esp_ae_eq_filter_para_t *para)
{
ESP_GMF_NULL_CHECK(TAG, handle, { return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_NULL_CHECK(TAG, para, { return ESP_GMF_ERR_INVALID_ARG;});
esp_gmf_eq_t *eq = (esp_gmf_eq_t *)handle;
if (eq->eq_hd) {
esp_gmf_oal_mutex_lock(((esp_gmf_audio_element_t *)handle)->lock);
esp_ae_err_t ret = esp_ae_eq_set_filter_para(eq->eq_hd, idx, para);
esp_gmf_oal_mutex_unlock(((esp_gmf_audio_element_t *)handle)->lock);
ESP_GMF_RET_ON_ERROR(TAG, ret, return ESP_GMF_JOB_ERR_FAIL;, "Equalize set error %d", ret);
}
esp_ae_eq_cfg_t *cfg = (esp_ae_eq_cfg_t *)OBJ_GET_CFG(handle);
ESP_GMF_NULL_CHECK(TAG, cfg, return ESP_GMF_ERR_FAIL);
if (cfg->para) {
if (idx >= cfg->filter_num) {
ESP_LOGE(TAG, "Invalid idx %d", idx);
return ESP_GMF_ERR_FAIL;
}
memcpy(&cfg->para[idx], para, sizeof(esp_ae_eq_filter_para_t));
return ESP_GMF_ERR_OK;
}
ESP_LOGE(TAG, "Failed to set EQ para, no para allocated");
return ESP_GMF_ERR_FAIL;
}
esp_gmf_err_t esp_gmf_eq_get_para(esp_gmf_element_handle_t handle, uint8_t idx, esp_ae_eq_filter_para_t *para)
{
ESP_GMF_NULL_CHECK(TAG, handle, { return ESP_GMF_ERR_INVALID_ARG;});
esp_gmf_eq_t *eq = (esp_gmf_eq_t *)handle;
if (eq->eq_hd) {
esp_ae_err_t ret = esp_ae_eq_get_filter_para(eq->eq_hd, idx, para);
ESP_GMF_RET_ON_ERROR(TAG, ret, return ESP_GMF_JOB_ERR_FAIL;, "Equalize set error %d", ret);
} else {
esp_ae_eq_cfg_t *cfg = (esp_ae_eq_cfg_t *)OBJ_GET_CFG(handle);
ESP_GMF_NULL_CHECK(TAG, cfg, return ESP_GMF_ERR_FAIL);
if (cfg->para) {
if (idx >= cfg->filter_num) {
ESP_LOGE(TAG, "Invalid idx %d", idx);
return ESP_GMF_ERR_FAIL;
}
memcpy(para, &cfg->para[idx], sizeof(esp_ae_eq_filter_para_t));
}
}
return ESP_GMF_ERR_OK;
}
esp_gmf_err_t esp_gmf_eq_enable_filter(esp_gmf_element_handle_t handle, uint8_t idx, bool is_enable)
{
ESP_GMF_NULL_CHECK(TAG, handle, { return ESP_GMF_ERR_INVALID_ARG;});
esp_gmf_eq_t *eq = (esp_gmf_eq_t *)handle;
esp_ae_eq_cfg_t *cfg = (esp_ae_eq_cfg_t *)OBJ_GET_CFG(handle);
ESP_GMF_NULL_CHECK(TAG, cfg, return ESP_GMF_ERR_FAIL);
if (idx >= cfg->filter_num) {
ESP_LOGE(TAG, "Filter index %d overlimit %d hd:%p", idx, cfg->filter_num, eq);
return ESP_GMF_ERR_INVALID_ARG;
}
if (eq->eq_hd) {
esp_gmf_oal_mutex_lock(((esp_gmf_audio_element_t *)handle)->lock);
esp_ae_err_t ret = 0;
if (is_enable) {
ret = esp_ae_eq_enable_filter(eq->eq_hd, idx);
} else {
ret = esp_ae_eq_disable_filter(eq->eq_hd, idx);
}
esp_gmf_oal_mutex_unlock(((esp_gmf_audio_element_t *)handle)->lock);
ESP_GMF_RET_ON_ERROR(TAG, ret, return ESP_GMF_JOB_ERR_FAIL;, "Equalize set error %d", ret);
}
eq->is_filter_enabled[idx] = is_enable;
return ESP_GMF_ERR_OK;
}
esp_gmf_err_t esp_gmf_eq_init(esp_ae_eq_cfg_t *config, esp_gmf_element_handle_t *handle)
{
ESP_GMF_NULL_CHECK(TAG, handle, {return ESP_GMF_ERR_INVALID_ARG;});
*handle = NULL;
esp_gmf_err_t ret = ESP_GMF_ERR_OK;
esp_gmf_eq_t *eq = esp_gmf_oal_calloc(1, sizeof(esp_gmf_eq_t));
ESP_GMF_MEM_VERIFY(TAG, eq, {return ESP_GMF_ERR_MEMORY_LACK;}, "eq", sizeof(esp_gmf_eq_t));
esp_gmf_obj_t *obj = (esp_gmf_obj_t *)eq;
obj->new_obj = esp_gmf_eq_new;
obj->del_obj = esp_gmf_eq_destroy;
esp_ae_eq_cfg_t *cfg = NULL;
if (config) {
if (config->para == NULL) {
config->para = (esp_ae_eq_filter_para_t *)esp_gmf_default_eq_paras;
config->filter_num = sizeof(esp_gmf_default_eq_paras) / sizeof(esp_ae_eq_filter_para_t);
}
dupl_esp_ae_eq_cfg(config, &cfg);
} else {
esp_ae_eq_cfg_t dcfg = DEFAULT_ESP_GMF_EQ_CONFIG();
dcfg.para = (esp_ae_eq_filter_para_t *)esp_gmf_default_eq_paras;
dcfg.filter_num = sizeof(esp_gmf_default_eq_paras) / sizeof(esp_ae_eq_filter_para_t);
dupl_esp_ae_eq_cfg(&dcfg, &cfg);
}
ESP_GMF_CHECK(TAG, cfg, {ret = ESP_GMF_ERR_MEMORY_LACK; goto EQ_INI_FAIL;}, "Failed to allocate eq configuration");
esp_gmf_obj_set_config(obj, cfg, sizeof(esp_ae_eq_cfg_t));
eq->is_filter_enabled = esp_gmf_oal_calloc(cfg->filter_num, sizeof(bool));
ESP_GMF_MEM_VERIFY(TAG, eq->is_filter_enabled, ret = ESP_GMF_ERR_MEMORY_LACK; goto EQ_INI_FAIL, "Rellocation failed", cfg->filter_num);
ret = esp_gmf_obj_set_tag(obj, "aud_eq");
ESP_GMF_RET_ON_NOT_OK(TAG, ret, goto EQ_INI_FAIL, "Failed to set obj tag");
esp_gmf_element_cfg_t el_cfg = {0};
ESP_GMF_ELEMENT_IN_PORT_ATTR_SET(el_cfg.in_attr, ESP_GMF_EL_PORT_CAP_SINGLE, 0, 0,
ESP_GMF_PORT_TYPE_BLOCK | ESP_GMF_PORT_TYPE_BYTE, ESP_GMF_ELEMENT_PORT_DATA_SIZE_DEFAULT);
ESP_GMF_ELEMENT_IN_PORT_ATTR_SET(el_cfg.out_attr, ESP_GMF_EL_PORT_CAP_SINGLE, 0, 0,
ESP_GMF_PORT_TYPE_BLOCK | ESP_GMF_PORT_TYPE_BYTE, ESP_GMF_ELEMENT_PORT_DATA_SIZE_DEFAULT);
el_cfg.dependency = true;
ret = esp_gmf_audio_el_init(eq, &el_cfg);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, goto EQ_INI_FAIL, "Failed to initialize eq element");
*handle = obj;
ESP_LOGD(TAG, "Initialization, %s-%p", OBJ_GET_TAG(obj), obj);
ESP_GMF_ELEMENT_GET(eq)->ops.open = esp_gmf_eq_open;
ESP_GMF_ELEMENT_GET(eq)->ops.process = esp_gmf_eq_process;
ESP_GMF_ELEMENT_GET(eq)->ops.close = esp_gmf_eq_close;
ESP_GMF_ELEMENT_GET(eq)->ops.event_receiver = eq_received_event_handler;
ESP_GMF_ELEMENT_GET(eq)->ops.load_caps = _load_eq_caps_func;
ESP_GMF_ELEMENT_GET(eq)->ops.load_methods = _load_eq_methods_func;
return ESP_GMF_ERR_OK;
EQ_INI_FAIL:
esp_gmf_eq_destroy(obj);
return ret;
}
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_audio/esp_gmf_fade.c
+329
View File
@@ -0,0 +1,329 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#include <string.h>
#include "esp_log.h"
#include "esp_gmf_oal_mem.h"
#include "esp_gmf_oal_mutex.h"
#include "esp_gmf_node.h"
#include "esp_gmf_fade.h"
#include "gmf_audio_common.h"
#include "esp_gmf_audio_methods_def.h"
#include "esp_gmf_cap.h"
#include "esp_gmf_caps_def.h"
#include "esp_gmf_audio_element.h"
/**
* @brief Audio fade context in GMF
*/
typedef struct {
esp_gmf_audio_element_t parent; /*!< The GMF fade handle */
esp_ae_fade_handle_t fade_hd; /*!< The audio effects fade handle */
uint8_t bytes_per_sample; /*!< Bytes number of per sampling point */
bool need_reopen : 1; /*!< Whether need to reopen.
True: Execute the close function first, then execute the open function
False: Do nothing */
} esp_gmf_fade_t;
static const char *TAG = "ESP_GMF_FADE";
static esp_gmf_err_t __fade_set_mode(esp_gmf_element_handle_t handle, esp_gmf_args_desc_t *arg_desc,
uint8_t *buf, int buf_len)
{
ESP_GMF_NULL_CHECK(TAG, arg_desc, {return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_NULL_CHECK(TAG, buf, {return ESP_GMF_ERR_INVALID_ARG;});
return esp_gmf_fade_set_mode(handle, *((esp_ae_fade_mode_t *)buf));
}
static esp_gmf_err_t __fade_get_mode(esp_gmf_element_handle_t handle, esp_gmf_args_desc_t *arg_desc,
uint8_t *buf, int buf_len)
{
ESP_GMF_NULL_CHECK(TAG, arg_desc, {return ESP_GMF_ERR_INVALID_ARG;});
return esp_gmf_fade_get_mode(handle, (esp_ae_fade_mode_t *)buf);
}
static esp_gmf_err_t __fade_reset(esp_gmf_element_handle_t handle, esp_gmf_args_desc_t *arg_desc,
uint8_t *buf, int buf_len)
{
return esp_gmf_fade_reset_weight(handle);
}
static esp_gmf_err_t esp_gmf_fade_new(void *cfg, esp_gmf_obj_handle_t *handle)
{
return esp_gmf_fade_init(cfg, (esp_gmf_element_handle_t *)handle);
}
static esp_gmf_job_err_t esp_gmf_fade_open(esp_gmf_element_handle_t self, void *para)
{
esp_gmf_fade_t *fade = (esp_gmf_fade_t *)self;
esp_ae_fade_cfg_t *fade_info = (esp_ae_fade_cfg_t *)OBJ_GET_CFG(self);
ESP_GMF_NULL_CHECK(TAG, fade_info, {return ESP_GMF_JOB_ERR_FAIL;});
fade->bytes_per_sample = (fade_info->bits_per_sample >> 3) * fade_info->channel;
esp_ae_fade_open(fade_info, &fade->fade_hd);
ESP_GMF_CHECK(TAG, fade->fade_hd, {return ESP_GMF_JOB_ERR_FAIL;}, "Failed to create fade handle");
GMF_AUDIO_UPDATE_SND_INFO(self, fade_info->sample_rate, fade_info->bits_per_sample, fade_info->channel);
fade->need_reopen = false;
ESP_LOGD(TAG, "Open, %p", self);
return ESP_GMF_ERR_OK;
}
static esp_gmf_job_err_t esp_gmf_fade_close(esp_gmf_element_handle_t self, void *para)
{
esp_gmf_fade_t *fade = (esp_gmf_fade_t *)self;
ESP_LOGD(TAG, "Closed, %p", self);
if (fade->fade_hd != NULL) {
esp_ae_fade_close(fade->fade_hd);
fade->fade_hd = NULL;
}
return ESP_GMF_ERR_OK;
}
static esp_gmf_job_err_t esp_gmf_fade_process(esp_gmf_element_handle_t self, void *para)
{
esp_gmf_fade_t *fade = (esp_gmf_fade_t *)self;
esp_gmf_job_err_t out_len = ESP_GMF_JOB_ERR_OK;
if (fade->need_reopen) {
esp_gmf_fade_close(self, NULL);
out_len = esp_gmf_fade_open(self, NULL);
if (out_len != ESP_GMF_JOB_ERR_OK) {
ESP_LOGE(TAG, "Fade reopen failed");
return out_len;
}
}
esp_gmf_port_handle_t in_port = ESP_GMF_ELEMENT_GET(self)->in;
esp_gmf_port_handle_t out_port = ESP_GMF_ELEMENT_GET(self)->out;
esp_gmf_payload_t *in_load = NULL;
esp_gmf_payload_t *out_load = NULL;
int samples_num = ESP_GMF_ELEMENT_GET(fade)->in_attr.data_size / (fade->bytes_per_sample);
int bytes = samples_num * fade->bytes_per_sample;
esp_gmf_err_io_t load_ret = esp_gmf_port_acquire_in(in_port, &in_load, bytes, ESP_GMF_MAX_DELAY);
samples_num = in_load->valid_size / (fade->bytes_per_sample);
bytes = samples_num * fade->bytes_per_sample;
if ((bytes != in_load->valid_size) || (load_ret < ESP_GMF_IO_OK)) {
ESP_LOGE(TAG, "Invalid in load size %d, ret %d", in_load->valid_size, load_ret);
out_len = ESP_GMF_JOB_ERR_FAIL;
goto __fade_release;
}
if (in_port->is_shared == 1) {
out_load = in_load;
}
load_ret = esp_gmf_port_acquire_out(out_port, &out_load, samples_num ? bytes : in_load->buf_length, ESP_GMF_MAX_DELAY);
ESP_GMF_PORT_ACQUIRE_OUT_CHECK(TAG, load_ret, out_len, { goto __fade_release;});
if (samples_num > 0) {
esp_gmf_oal_mutex_lock(((esp_gmf_audio_element_t *)self)->lock);
esp_ae_err_t ret = esp_ae_fade_process(fade->fade_hd, samples_num, in_load->buf, out_load->buf);
esp_gmf_oal_mutex_unlock(((esp_gmf_audio_element_t *)self)->lock);
ESP_GMF_RET_ON_ERROR(TAG, ret, {out_len = ESP_GMF_JOB_ERR_FAIL; goto __fade_release;}, "Fade process error %d", ret);
}
ESP_LOGV(TAG, "Samples: %d, IN-PLD: %p-%p-%d-%d-%d, OUT-PLD: %p-%p-%d-%d-%d",
samples_num, in_load, in_load->buf, in_load->valid_size, in_load->buf_length, in_load->is_done,
out_load, out_load->buf, out_load->valid_size, out_load->buf_length, out_load->is_done);
out_load->valid_size = samples_num * fade->bytes_per_sample;
out_load->is_done = in_load->is_done;
out_load->pts = in_load->pts;
if (out_load->valid_size > 0) {
esp_gmf_audio_el_update_file_pos((esp_gmf_element_handle_t)self, out_load->valid_size);
}
if (in_load->is_done) {
out_len = ESP_GMF_JOB_ERR_DONE;
ESP_LOGD(TAG, "Fade done, out len: %d", out_load->valid_size);
}
__fade_release:
// Release in and out port
if (out_load != NULL) {
load_ret = esp_gmf_port_release_out(out_port, out_load, ESP_GMF_MAX_DELAY);
if ((load_ret < ESP_GMF_IO_OK) && (load_ret != ESP_GMF_IO_ABORT)) {
ESP_LOGE(TAG, "OUT port release error, ret:%d", load_ret);
out_len = ESP_GMF_JOB_ERR_FAIL;
}
}
if (in_load != NULL) {
load_ret = esp_gmf_port_release_in(in_port, in_load, ESP_GMF_MAX_DELAY);
if ((load_ret < ESP_GMF_IO_OK) && (load_ret != ESP_GMF_IO_ABORT)) {
ESP_LOGE(TAG, "IN port release error, ret:%d", load_ret);
out_len = ESP_GMF_JOB_ERR_FAIL;
}
}
return out_len;
}
static esp_gmf_err_t fade_received_event_handler(esp_gmf_event_pkt_t *evt, void *ctx)
{
ESP_GMF_NULL_CHECK(TAG, ctx, {return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_NULL_CHECK(TAG, evt, {return ESP_GMF_ERR_INVALID_ARG;});
if ((evt->type != ESP_GMF_EVT_TYPE_REPORT_INFO)
|| (evt->sub != ESP_GMF_INFO_SOUND)
|| (evt->payload == NULL)) {
return ESP_GMF_ERR_OK;
}
esp_gmf_element_handle_t self = (esp_gmf_element_handle_t)ctx;
esp_gmf_element_handle_t el = evt->from;
esp_gmf_event_state_t state = ESP_GMF_EVENT_STATE_NONE;
esp_gmf_element_get_state(self, &state);
esp_gmf_info_sound_t *info = (esp_gmf_info_sound_t *)evt->payload;
esp_ae_fade_cfg_t *config = (esp_ae_fade_cfg_t *)OBJ_GET_CFG(self);
ESP_GMF_NULL_CHECK(TAG, config, return ESP_GMF_ERR_FAIL);
esp_gmf_fade_t *fade = (esp_gmf_fade_t *)self;
fade->need_reopen = (config->sample_rate != info->sample_rates) || (info->channels != config->channel) || (config->bits_per_sample != info->bits);
config->sample_rate = info->sample_rates;
config->channel = info->channels;
config->bits_per_sample = info->bits;
ESP_LOGD(TAG, "RECV element info, from: %s-%p, next: %p, self: %s-%p, type: %x, state: %s, rate: %d, ch: %d, bits: %d",
OBJ_GET_TAG(el), el, esp_gmf_node_for_next((esp_gmf_node_t *)el), OBJ_GET_TAG(self), self, evt->type,
esp_gmf_event_get_state_str(state), info->sample_rates, info->channels, info->bits);
if (state == ESP_GMF_EVENT_STATE_NONE) {
esp_gmf_element_set_state(self, ESP_GMF_EVENT_STATE_INITIALIZED);
}
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t esp_gmf_fade_destroy(esp_gmf_element_handle_t self)
{
esp_gmf_fade_t *fade = (esp_gmf_fade_t *)self;
ESP_LOGD(TAG, "Destroyed, %p", self);
void *cfg = OBJ_GET_CFG(self);
if (cfg) {
esp_gmf_oal_free(cfg);
}
esp_gmf_audio_el_deinit(self);
esp_gmf_oal_free(fade);
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t _load_fade_caps_func(esp_gmf_element_handle_t handle)
{
esp_gmf_cap_t *caps = NULL;
esp_gmf_cap_t dec_caps = {0};
dec_caps.cap_eightcc = ESP_GMF_CAPS_AUDIO_FADE;
dec_caps.attr_fun = NULL;
int ret = esp_gmf_cap_append(&caps, &dec_caps);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to create capability");
esp_gmf_element_t *el = (esp_gmf_element_t *)handle;
el->caps = caps;
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t _load_fade_methods_func(esp_gmf_element_handle_t handle)
{
esp_gmf_method_t *method = NULL;
esp_gmf_args_desc_t *set_args = NULL;
esp_gmf_args_desc_t *get_args = NULL;
esp_gmf_err_t ret = esp_gmf_args_desc_append(&set_args, AMETHOD_ARG(FADE, SET_MODE, MODE), ESP_GMF_ARGS_TYPE_INT32, sizeof(int32_t), 0);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to append MODE argument");
ret = esp_gmf_method_append(&method, AMETHOD(FADE, SET_MODE), __fade_set_mode, set_args);
ESP_GMF_RET_ON_ERROR(TAG, ret, {return ret;}, "Failed to register %s method", AMETHOD(FADE, SET_MODE));
ret = esp_gmf_args_desc_copy(set_args, &get_args);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to copy argument");
ret = esp_gmf_method_append(&method, AMETHOD(FADE, GET_MODE), __fade_get_mode, get_args);
ESP_GMF_RET_ON_ERROR(TAG, ret, {return ret;}, "Failed to register %s method", AMETHOD(FADE, GET_MODE));
ret = esp_gmf_method_append(&method, AMETHOD(FADE, RESET_WEIGHT), __fade_reset, NULL);
ESP_GMF_RET_ON_ERROR(TAG, ret, {return ret;}, "Failed to register %s method", AMETHOD(FADE, RESET_WEIGHT));
esp_gmf_element_t *el = (esp_gmf_element_t *)handle;
el->method = method;
return ESP_GMF_ERR_OK;
}
esp_gmf_err_t esp_gmf_fade_set_mode(esp_gmf_element_handle_t handle, esp_ae_fade_mode_t mode)
{
ESP_GMF_NULL_CHECK(TAG, handle, { return ESP_GMF_ERR_INVALID_ARG;});
esp_gmf_fade_t *fade = (esp_gmf_fade_t *)handle;
esp_ae_fade_cfg_t *cfg = (esp_ae_fade_cfg_t *)OBJ_GET_CFG(handle);
ESP_GMF_NULL_CHECK(TAG, cfg, return ESP_GMF_ERR_FAIL);
if (fade->fade_hd) {
esp_gmf_oal_mutex_lock(((esp_gmf_audio_element_t *)handle)->lock);
esp_ae_err_t ret = esp_ae_fade_set_mode(fade->fade_hd, mode);
esp_gmf_oal_mutex_unlock(((esp_gmf_audio_element_t *)handle)->lock);
if (ret != ESP_AE_ERR_OK) {
return ESP_GMF_JOB_ERR_FAIL;
}
}
cfg->mode = mode;
return ESP_GMF_JOB_ERR_OK;
}
esp_gmf_err_t esp_gmf_fade_get_mode(esp_gmf_element_handle_t handle, esp_ae_fade_mode_t *mode)
{
ESP_GMF_NULL_CHECK(TAG, handle, { return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_NULL_CHECK(TAG, mode, { return ESP_GMF_ERR_INVALID_ARG;});
esp_ae_fade_cfg_t *cfg = (esp_ae_fade_cfg_t *)OBJ_GET_CFG(handle);
ESP_GMF_NULL_CHECK(TAG, cfg, return ESP_GMF_ERR_FAIL);
esp_gmf_fade_t *fade = (esp_gmf_fade_t *)handle;
if (fade->fade_hd) {
esp_ae_err_t ret = esp_ae_fade_get_mode(fade->fade_hd, mode);
if (ret != ESP_AE_ERR_OK) {
return ESP_GMF_JOB_ERR_FAIL;
}
} else {
*mode = cfg->mode;
}
return ESP_GMF_JOB_ERR_OK;
}
esp_gmf_err_t esp_gmf_fade_reset_weight(esp_gmf_element_handle_t handle)
{
ESP_GMF_NULL_CHECK(TAG, handle, { return ESP_GMF_ERR_INVALID_ARG;});
esp_ae_fade_cfg_t *cfg = (esp_ae_fade_cfg_t *)OBJ_GET_CFG(handle);
ESP_GMF_NULL_CHECK(TAG, cfg, return ESP_GMF_ERR_FAIL);
esp_gmf_fade_t *fade = (esp_gmf_fade_t *)handle;
if(fade->fade_hd) {
esp_gmf_oal_mutex_lock(((esp_gmf_audio_element_t *)handle)->lock);
esp_ae_err_t ret = esp_ae_fade_reset_weight(fade->fade_hd);
esp_gmf_oal_mutex_unlock(((esp_gmf_audio_element_t *)handle)->lock);
if (ret != ESP_AE_ERR_OK) {
return ESP_GMF_JOB_ERR_FAIL;
}
esp_ae_fade_get_mode(fade->fade_hd, &cfg->mode);
}
return ESP_GMF_JOB_ERR_OK;
}
esp_gmf_err_t esp_gmf_fade_init(esp_ae_fade_cfg_t *config, esp_gmf_element_handle_t *handle)
{
ESP_GMF_NULL_CHECK(TAG, handle, {return ESP_GMF_ERR_INVALID_ARG;});
*handle = NULL;
esp_gmf_err_t ret = ESP_GMF_ERR_OK;
esp_gmf_fade_t *fade = esp_gmf_oal_calloc(1, sizeof(esp_gmf_fade_t));
ESP_GMF_MEM_VERIFY(TAG, fade, {return ESP_GMF_ERR_MEMORY_LACK;}, "fade", sizeof(esp_gmf_fade_t));
esp_gmf_obj_t *obj = (esp_gmf_obj_t *)fade;
obj->new_obj = esp_gmf_fade_new;
obj->del_obj = esp_gmf_fade_destroy;
esp_ae_fade_cfg_t *cfg = esp_gmf_oal_calloc(1, sizeof(esp_ae_fade_cfg_t));
ESP_GMF_MEM_VERIFY(TAG, cfg, {ret = ESP_GMF_ERR_MEMORY_LACK; goto FADE_INIT_FAIL;}, "fade configuration", sizeof(esp_ae_fade_cfg_t));
esp_gmf_obj_set_config(obj, cfg, sizeof(esp_ae_fade_cfg_t));
if (config) {
memcpy(cfg, config, sizeof(esp_ae_fade_cfg_t));
} else {
esp_ae_fade_cfg_t dcfg = DEFAULT_ESP_GMF_FADE_CONFIG();
memcpy(cfg, &dcfg, sizeof(esp_ae_fade_cfg_t));
}
ret = esp_gmf_obj_set_tag(obj, "aud_fade");
ESP_GMF_RET_ON_NOT_OK(TAG, ret, goto FADE_INIT_FAIL, "Failed to set obj tag");
esp_gmf_element_cfg_t el_cfg = {0};
ESP_GMF_ELEMENT_IN_PORT_ATTR_SET(el_cfg.in_attr, ESP_GMF_EL_PORT_CAP_SINGLE, 0, 0,
ESP_GMF_PORT_TYPE_BLOCK | ESP_GMF_PORT_TYPE_BYTE, ESP_GMF_ELEMENT_PORT_DATA_SIZE_DEFAULT);
ESP_GMF_ELEMENT_IN_PORT_ATTR_SET(el_cfg.out_attr, ESP_GMF_EL_PORT_CAP_SINGLE, 0, 0,
ESP_GMF_PORT_TYPE_BLOCK | ESP_GMF_PORT_TYPE_BYTE, ESP_GMF_ELEMENT_PORT_DATA_SIZE_DEFAULT);
el_cfg.dependency = true;
ret = esp_gmf_audio_el_init(fade, &el_cfg);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, goto FADE_INIT_FAIL, "Failed to initialize fade element");
ESP_GMF_ELEMENT_GET(fade)->ops.open = esp_gmf_fade_open;
ESP_GMF_ELEMENT_GET(fade)->ops.process = esp_gmf_fade_process;
ESP_GMF_ELEMENT_GET(fade)->ops.close = esp_gmf_fade_close;
ESP_GMF_ELEMENT_GET(fade)->ops.event_receiver = fade_received_event_handler;
ESP_GMF_ELEMENT_GET(fade)->ops.load_caps = _load_fade_caps_func;
ESP_GMF_ELEMENT_GET(fade)->ops.load_methods = _load_fade_methods_func;
*handle = obj;
ESP_LOGD(TAG, "Initialization, %s-%p", OBJ_GET_TAG(obj), obj);
return ESP_GMF_ERR_OK;
FADE_INIT_FAIL:
esp_gmf_fade_destroy(obj);
return ret;
}
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_audio/esp_gmf_interleave.c
+270
View File
@@ -0,0 +1,270 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#include <string.h>
#include "esp_log.h"
#include "esp_gmf_oal_mem.h"
#include "esp_gmf_oal_mutex.h"
#include "esp_gmf_node.h"
#include "esp_gmf_interleave.h"
#include "esp_ae_data_weaver.h"
#include "esp_heap_caps.h"
#include "gmf_audio_common.h"
#include "esp_gmf_cap.h"
#include "esp_gmf_caps_def.h"
#include "esp_gmf_audio_element.h"
#define ESP_GMF_PROCESS_SAMPLE (256)
/**
* @brief Audio interleave context in GMF
*/
typedef struct {
esp_gmf_audio_element_t parent; /*!< The GMF interleave handle */
uint8_t bytes_per_sample; /*!< Bytes number of per sampling point */
esp_gmf_payload_t **in_load; /*!< The array of input payload */
esp_gmf_payload_t *out_load; /*!< The output payload */
uint8_t **in_arr; /*!< The array of input buffer pointer */
uint8_t src_num; /*!< The number of input stream source */
uint8_t bits_per_sample; /*!< Bits number of per sampling point */
bool need_reopen : 1; /*!< Whether need to reopen.
True: Execute the close function first, then execute the open function
False: Do nothing */
} esp_gmf_interleave_t;
static const char *TAG = "ESP_GMF_INTLV";
static inline esp_gmf_err_t dupl_esp_ae_interleave_cfg(esp_gmf_interleave_cfg *config, esp_gmf_interleave_cfg **new_config)
{
*new_config = esp_gmf_oal_calloc(1, sizeof(*config));
ESP_GMF_MEM_VERIFY(TAG, *new_config, {return ESP_GMF_ERR_MEMORY_LACK;}, "interleave configuration", sizeof(*config));
memcpy(*new_config, config, sizeof(*config));
return ESP_GMF_JOB_ERR_OK;
}
static inline void free_esp_ae_interleave_cfg(esp_gmf_interleave_cfg *config)
{
if (config) {
esp_gmf_oal_free(config);
}
}
static esp_gmf_err_t esp_gmf_interleave_new(void *cfg, esp_gmf_obj_handle_t *handle)
{
return esp_gmf_interleave_init(cfg, (esp_gmf_element_handle_t *)handle);
}
static esp_gmf_job_err_t esp_gmf_interleave_open(esp_gmf_element_handle_t self, void *para)
{
esp_gmf_interleave_t *interleave = (esp_gmf_interleave_t *)self;
esp_gmf_interleave_cfg *interleave_info = (esp_gmf_interleave_cfg *)OBJ_GET_CFG(self);
ESP_GMF_NULL_CHECK(TAG, interleave_info, {return ESP_GMF_JOB_ERR_FAIL;})
interleave->bytes_per_sample = (interleave_info->bits_per_sample >> 3);
interleave->in_load = esp_gmf_oal_calloc(1, sizeof(esp_gmf_payload_t *) * interleave_info->src_num);
ESP_GMF_MEM_VERIFY(TAG, interleave->in_load, {return ESP_GMF_JOB_ERR_FAIL;},
"in load", sizeof(esp_gmf_payload_t *) * interleave_info->src_num);
interleave->in_arr = esp_gmf_oal_calloc(1, sizeof(int *) * interleave_info->src_num);
ESP_GMF_MEM_VERIFY(TAG, interleave->in_arr, {return ESP_GMF_JOB_ERR_FAIL;},
"in buffer array", sizeof(int *) * interleave_info->src_num);
GMF_AUDIO_UPDATE_SND_INFO(self, interleave_info->sample_rate, interleave_info->bits_per_sample, interleave_info->src_num);
interleave->src_num = interleave_info->src_num;
interleave->bits_per_sample = interleave_info->bits_per_sample;
interleave->need_reopen = false;
ESP_LOGD(TAG, "Open, %p", self);
return ESP_GMF_JOB_ERR_OK;
}
static esp_gmf_job_err_t esp_gmf_interleave_close(esp_gmf_element_handle_t self, void *para)
{
esp_gmf_interleave_t *interleave = (esp_gmf_interleave_t *)self;
ESP_LOGD(TAG, "Closed, %p", self);
if (interleave->in_arr != NULL) {
esp_gmf_oal_free(interleave->in_arr);
interleave->in_arr = NULL;
}
if (interleave->in_load != NULL) {
esp_gmf_oal_free(interleave->in_load);
interleave->in_load = NULL;
}
return ESP_GMF_ERR_OK;
}
static esp_gmf_job_err_t esp_gmf_interleave_process(esp_gmf_element_handle_t self, void *para)
{
esp_gmf_interleave_t *interleave = (esp_gmf_interleave_t *)self;
esp_gmf_job_err_t out_len = ESP_GMF_JOB_ERR_OK;
if (interleave->need_reopen) {
esp_gmf_interleave_close(self, NULL);
out_len = esp_gmf_interleave_open(self, NULL);
if (out_len != ESP_GMF_JOB_ERR_OK) {
ESP_LOGE(TAG, "Interleave reopen failed");
return out_len;
}
}
esp_gmf_port_handle_t in = ESP_GMF_ELEMENT_GET(self)->in;
esp_gmf_port_handle_t in_port = in;
esp_gmf_port_handle_t out_port = ESP_GMF_ELEMENT_GET(self)->out;
esp_gmf_interleave_cfg *interleave_info = (esp_gmf_interleave_cfg *)OBJ_GET_CFG(self);
ESP_GMF_NULL_CHECK(TAG, interleave_info, return ESP_GMF_JOB_ERR_FAIL);
int index = interleave_info->src_num;
int i = 0;
bool is_done = false;
esp_gmf_err_io_t load_ret = ESP_GMF_IO_OK;
memset(interleave->in_load, 0, sizeof(esp_gmf_payload_t *) * index);
interleave->out_load = NULL;
int samples_num = ESP_GMF_ELEMENT_GET(interleave)->in_attr.data_size / (interleave->bytes_per_sample);
int bytes = samples_num * interleave->bytes_per_sample;
while (in_port != NULL) {
load_ret = esp_gmf_port_acquire_in(in_port, &(interleave->in_load[i]), bytes, in_port->wait_ticks);
ESP_GMF_PORT_CHECK(TAG, load_ret, out_len, {out_len = ESP_GMF_JOB_ERR_FAIL; goto __intlv_release;}, "Failed to acquire in, idx:%d, ret: %d", i, load_ret);
interleave->in_arr[i] = interleave->in_load[i]->buf;
in_port = in_port->next;
// if one load is done means interleave need to done
is_done = (((is_done == 1) || (interleave->in_load[i]->is_done == 1)) ? true : false);
ESP_LOGV(TAG, "IN: idx: %d load: %p, buf: %p, valid size: %d, buf length: %d, done: %d",
i, interleave->in_load[i], interleave->in_load[i]->buf, interleave->in_load[i]->valid_size,
interleave->in_load[i]->buf_length, interleave->in_load[i]->is_done);
i++;
}
samples_num = interleave->in_load[0]->valid_size / (interleave->bytes_per_sample);
bytes = samples_num * interleave->bytes_per_sample;
load_ret = esp_gmf_port_acquire_out(out_port, &interleave->out_load,
samples_num ? bytes * interleave->src_num : interleave->in_load[0]->buf_length, ESP_GMF_MAX_DELAY);
ESP_GMF_PORT_ACQUIRE_OUT_CHECK(TAG, load_ret, out_len, { goto __intlv_release;});
if (samples_num > 0) {
esp_ae_err_t ret = esp_ae_intlv_process(interleave->src_num, interleave->bits_per_sample, samples_num,
(void **)interleave->in_arr, interleave->out_load->buf);
ESP_GMF_RET_ON_ERROR(TAG, ret, {out_len = ESP_GMF_JOB_ERR_FAIL; goto __intlv_release;}, "Interleave process error, ret: %d", ret);
}
ESP_LOGV(TAG, "OUT: load: %p, buf: %p, valid size: %d, buf length: %d",
interleave->out_load, interleave->out_load->buf, interleave->out_load->valid_size, interleave->out_load->buf_length);
interleave->out_load->valid_size = samples_num * interleave->bytes_per_sample * interleave->src_num;
if (interleave->out_load->valid_size > 0) {
esp_gmf_audio_el_update_file_pos((esp_gmf_element_handle_t)self, interleave->out_load->valid_size);
}
// Interleave finished
if (is_done == true) {
ESP_LOGD(TAG, "Interleave is done.");
interleave->out_load->is_done = true;
out_len = ESP_GMF_JOB_ERR_DONE;
}
__intlv_release:
if (interleave->out_load != NULL) {
load_ret = esp_gmf_port_release_out(out_port, interleave->out_load, ESP_GMF_MAX_DELAY);
if ((load_ret < ESP_GMF_IO_OK) && (load_ret != ESP_GMF_IO_ABORT)) {
ESP_LOGE(TAG, "OUT port release error, ret:%d", load_ret);
out_len = ESP_GMF_JOB_ERR_FAIL;
}
}
in_port = in;
i = 0;
while (in_port != NULL && interleave->in_load[i] != NULL) {
load_ret = esp_gmf_port_release_in(in_port, interleave->in_load[i], ESP_GMF_MAX_DELAY);
if ((load_ret < ESP_GMF_IO_OK) && (load_ret != ESP_GMF_IO_ABORT)) {
ESP_LOGE(TAG, "IN port release error, ret:%d", load_ret);
out_len = ESP_GMF_JOB_ERR_FAIL;
}
in_port = in_port->next;
i++;
}
return out_len;
}
static esp_gmf_err_t interleave_received_event_handler(esp_gmf_event_pkt_t *evt, void *ctx)
{
ESP_GMF_NULL_CHECK(TAG, ctx, {return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_NULL_CHECK(TAG, evt, {return ESP_GMF_ERR_INVALID_ARG;});
if ((evt->type != ESP_GMF_EVT_TYPE_REPORT_INFO)
|| (evt->sub != ESP_GMF_INFO_SOUND)
|| (evt->payload == NULL)) {
return ESP_GMF_ERR_OK;
}
esp_gmf_element_handle_t self = (esp_gmf_element_handle_t)ctx;
esp_gmf_element_handle_t el = evt->from;
esp_gmf_event_state_t state = ESP_GMF_EVENT_STATE_NONE;
esp_gmf_element_get_state(self, &state);
esp_gmf_info_sound_t *info = (esp_gmf_info_sound_t *)evt->payload;
esp_gmf_interleave_cfg *config = (esp_gmf_interleave_cfg *)OBJ_GET_CFG(self);
ESP_GMF_NULL_CHECK(TAG, config, return ESP_GMF_ERR_FAIL);
esp_gmf_interleave_t *interleave = (esp_gmf_interleave_t *)self;
interleave->need_reopen = (config->sample_rate != info->sample_rates) || (config->bits_per_sample != info->bits);
config->sample_rate = info->sample_rates;
config->bits_per_sample = info->bits;
ESP_LOGD(TAG, "RECV element info, from: %s-%p, next: %p, self: %s-%p, type: %x, state: %s, rate: %d, ch: %d, bits: %d",
OBJ_GET_TAG(el), el, esp_gmf_node_for_next((esp_gmf_node_t *)el), OBJ_GET_TAG(self), self, evt->type,
esp_gmf_event_get_state_str(state), info->sample_rates, info->channels, info->bits);
if (state == ESP_GMF_EVENT_STATE_NONE) {
esp_gmf_element_set_state(self, ESP_GMF_EVENT_STATE_INITIALIZED);
}
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t esp_gmf_interleave_destroy(esp_gmf_element_handle_t self)
{
esp_gmf_interleave_t *interleave = (esp_gmf_interleave_t *)self;
ESP_LOGD(TAG, "Destroyed, %p", self);
free_esp_ae_interleave_cfg(OBJ_GET_CFG(self));
esp_gmf_audio_el_deinit(self);
esp_gmf_oal_free(interleave);
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t _load_interleave_caps_func(esp_gmf_element_handle_t handle)
{
esp_gmf_cap_t *caps = NULL;
esp_gmf_cap_t dec_caps = {0};
dec_caps.cap_eightcc = ESP_GMF_CAPS_AUDIO_INTERLEAVE;
dec_caps.attr_fun = NULL;
int ret = esp_gmf_cap_append(&caps, &dec_caps);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to create capability");
esp_gmf_element_t *el = (esp_gmf_element_t *)handle;
el->caps = caps;
return ESP_GMF_ERR_OK;
}
esp_gmf_err_t esp_gmf_interleave_init(esp_gmf_interleave_cfg *config, esp_gmf_element_handle_t *handle)
{
ESP_GMF_NULL_CHECK(TAG, handle, {return ESP_GMF_ERR_INVALID_ARG;});
*handle = NULL;
esp_gmf_err_t ret = ESP_GMF_ERR_OK;
esp_gmf_interleave_t *interleave = esp_gmf_oal_calloc(1, sizeof(esp_gmf_interleave_t));
ESP_GMF_MEM_VERIFY(TAG, interleave, {return ESP_GMF_ERR_MEMORY_LACK;}, "interleave", sizeof(esp_gmf_interleave_t));
esp_gmf_obj_t *obj = (esp_gmf_obj_t *)interleave;
obj->new_obj = esp_gmf_interleave_new;
obj->del_obj = esp_gmf_interleave_destroy;
esp_gmf_interleave_cfg *cfg = esp_gmf_oal_calloc(1, sizeof(esp_gmf_interleave_cfg));
ESP_GMF_MEM_VERIFY(TAG, cfg, {ret = ESP_GMF_ERR_MEMORY_LACK; goto INTLV_INIT_FAIL;}, "interleave configuration", sizeof(esp_gmf_interleave_cfg));
esp_gmf_obj_set_config(obj, cfg, sizeof(esp_gmf_interleave_cfg));
if (config) {
memcpy(cfg, config, sizeof(esp_gmf_interleave_cfg));
} else {
esp_gmf_interleave_cfg dcfg = DEFAULT_ESP_GMF_INTERLEAVE_CONFIG();
memcpy(cfg, &dcfg, sizeof(esp_gmf_interleave_cfg));
}
ret = esp_gmf_obj_set_tag(obj, "aud_intlv");
ESP_GMF_RET_ON_NOT_OK(TAG, ret, goto INTLV_INIT_FAIL, "Failed to set obj tag");
esp_gmf_element_cfg_t el_cfg = {0};
ESP_GMF_ELEMENT_IN_PORT_ATTR_SET(el_cfg.in_attr, ESP_GMF_EL_PORT_CAP_MULTI, 0, 0,
ESP_GMF_PORT_TYPE_BLOCK | ESP_GMF_PORT_TYPE_BYTE, ESP_GMF_ELEMENT_PORT_DATA_SIZE_DEFAULT);
ESP_GMF_ELEMENT_IN_PORT_ATTR_SET(el_cfg.out_attr, ESP_GMF_EL_PORT_CAP_SINGLE, 0, 0,
ESP_GMF_PORT_TYPE_BLOCK | ESP_GMF_PORT_TYPE_BYTE, ESP_GMF_ELEMENT_PORT_DATA_SIZE_DEFAULT);
el_cfg.dependency = true;
ret = esp_gmf_audio_el_init(interleave, &el_cfg);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, goto INTLV_INIT_FAIL, "Failed to initialize interleave element");
ESP_GMF_ELEMENT_GET(interleave)->ops.open = esp_gmf_interleave_open;
ESP_GMF_ELEMENT_GET(interleave)->ops.process = esp_gmf_interleave_process;
ESP_GMF_ELEMENT_GET(interleave)->ops.close = esp_gmf_interleave_close;
ESP_GMF_ELEMENT_GET(interleave)->ops.event_receiver = interleave_received_event_handler;
ESP_GMF_ELEMENT_GET(interleave)->ops.load_caps = _load_interleave_caps_func;
*handle = obj;
ESP_LOGD(TAG, "Initialization, %s-%p", OBJ_GET_TAG(obj), obj);
return ESP_GMF_ERR_OK;
INTLV_INIT_FAIL:
esp_gmf_interleave_destroy(obj);
return ret;
}
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_audio/esp_gmf_mixer.c
+437
View File
@@ -0,0 +1,437 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#include <string.h>
#include "esp_log.h"
#include "esp_gmf_oal_mem.h"
#include "esp_gmf_oal_mutex.h"
#include "esp_gmf_node.h"
#include "esp_gmf_mixer.h"
#include "esp_heap_caps.h"
#include "gmf_audio_common.h"
#include "esp_gmf_audio_methods_def.h"
#include "esp_gmf_cap.h"
#include "esp_gmf_caps_def.h"
#include "esp_gmf_audio_element.h"
#ifdef MIXER_DEBUG
#include "esp_timer.h"
#endif
#define MIXER_DEFAULT_PROC_TIME_MS (10)
#define MIXER_GET_FRAME_BYTE_SIZE(info) (MIXER_DEFAULT_PROC_TIME_MS * info->sample_rate * info->channel * info->bits_per_sample / 8000)
typedef struct {
esp_gmf_audio_element_t parent; /*!< The GMF mixer handle */
esp_ae_mixer_handle_t mixer_hd; /*!< The audio effects mixer handle */
uint8_t bytes_per_sample; /*!< Bytes number of per sampling point */
uint32_t process_num; /*!< The data number of mixer processing */
uint32_t frame_time; /*!< The time of one frame, unit: ms */
esp_gmf_payload_t **in_load; /*!< The array of input payload */
esp_gmf_payload_t *out_load; /*!< The output payload */
uint8_t **in_arr; /*!< The input buffer pointer array of mixer */
esp_ae_mixer_mode_t *mode; /*!< The mixer mode */
uint8_t src_num; /*!< The number of source */
bool need_reopen : 1; /*!< Whether need to reopen.
True: Execute the close function first, then execute the open function
False: Do nothing */
} esp_gmf_mixer_t;
static const char *TAG = "ESP_GMF_MIXER";
const esp_ae_mixer_info_t esp_gmf_default_mixer_src_info[] = {
{1.0, 0.5, 500},
{0.5, 0.0, 500},
};
static inline esp_gmf_err_t dupl_esp_ae_mixer_cfg(esp_ae_mixer_cfg_t *config, esp_ae_mixer_cfg_t **new_config)
{
void *sub_cfg = NULL;
*new_config = esp_gmf_oal_calloc(1, sizeof(*config));
ESP_GMF_MEM_VERIFY(TAG, *new_config, {return ESP_GMF_ERR_MEMORY_LACK;}, "mixer configuration", sizeof(*config));
memcpy(*new_config, config, sizeof(*config));
if (config->src_info && (config->src_num > 0)) {
sub_cfg = esp_gmf_oal_calloc(1, config->src_num * sizeof(esp_ae_mixer_info_t));
ESP_GMF_MEM_VERIFY(TAG, sub_cfg, {esp_gmf_oal_free(*new_config); return ESP_GMF_ERR_MEMORY_LACK;},
"source information", config->src_num * sizeof(esp_ae_mixer_info_t));
memcpy(sub_cfg, config->src_info, config->src_num * sizeof(esp_ae_mixer_info_t));
(*new_config)->src_info = sub_cfg;
}
return ESP_GMF_JOB_ERR_OK;
}
static inline void free_esp_ae_mixer_cfg(esp_ae_mixer_cfg_t *config)
{
if (config) {
if (config->src_info && (config->src_info != esp_gmf_default_mixer_src_info)) {
esp_gmf_oal_free(config->src_info);
}
esp_gmf_oal_free(config);
}
}
static esp_gmf_err_t __mixer_set_mode(esp_gmf_element_handle_t handle, esp_gmf_args_desc_t *arg_desc,
uint8_t *buf, int buf_len)
{
ESP_GMF_NULL_CHECK(TAG, arg_desc, {return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_NULL_CHECK(TAG, buf, {return ESP_GMF_ERR_INVALID_ARG;});
uint8_t src_idx = (uint8_t)(*buf);
return esp_gmf_mixer_set_mode(handle, src_idx, *((esp_ae_mixer_mode_t *)(buf + arg_desc->next->offset)));
}
static esp_gmf_err_t __mixer_set_audio_info(esp_gmf_element_handle_t handle, esp_gmf_args_desc_t *arg_desc,
uint8_t *buf, int buf_len)
{
ESP_GMF_NULL_CHECK(TAG, handle, {return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_NULL_CHECK(TAG, arg_desc, {return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_NULL_CHECK(TAG, buf, {return ESP_GMF_ERR_INVALID_ARG;});
esp_gmf_args_desc_t *mix_desc = arg_desc;
uint32_t rate = *((uint32_t *)buf);
mix_desc = mix_desc->next;
uint8_t ch = (uint8_t)(*(buf + mix_desc->offset));
mix_desc = mix_desc->next;
uint8_t bits = (uint8_t)(*(buf + mix_desc->offset));
return esp_gmf_mixer_set_audio_info(handle, rate, bits, ch);
}
static esp_gmf_err_t esp_gmf_mixer_new(void *cfg, esp_gmf_obj_handle_t *handle)
{
return esp_gmf_mixer_init(cfg, (esp_gmf_element_handle_t *)handle);
}
static esp_gmf_job_err_t esp_gmf_mixer_open(esp_gmf_element_handle_t self, void *para)
{
esp_gmf_mixer_t *mixer = (esp_gmf_mixer_t *)self;
esp_ae_mixer_cfg_t *mixer_info = (esp_ae_mixer_cfg_t *)OBJ_GET_CFG(self);
ESP_GMF_NULL_CHECK(TAG, mixer_info, {return ESP_GMF_JOB_ERR_FAIL;})
mixer->bytes_per_sample = (mixer_info->bits_per_sample >> 3) * mixer_info->channel;
esp_ae_mixer_open(mixer_info, &mixer->mixer_hd);
ESP_GMF_CHECK(TAG, mixer->mixer_hd, {return ESP_GMF_JOB_ERR_FAIL;}, "Failed to create mixer handle");
for (int i = 0; i < mixer_info->src_num; i++) {
esp_ae_mixer_set_mode(mixer->mixer_hd, i, mixer->mode[i]);
}
uint32_t process_num = MIXER_GET_FRAME_BYTE_SIZE(mixer_info);
mixer->process_num = (ESP_GMF_ELEMENT_GET(mixer)->in_attr.data_size == 0) ? (process_num) : (ESP_GMF_ELEMENT_GET(mixer)->in_attr.data_size);
mixer->frame_time = GMF_AUDIO_CALC_PTS(mixer->process_num, mixer_info->sample_rate, mixer_info->channel, mixer_info->bits_per_sample);
mixer->src_num = mixer_info->src_num;
mixer->in_load = esp_gmf_oal_calloc(1, sizeof(esp_gmf_payload_t *) * mixer_info->src_num);
ESP_GMF_MEM_VERIFY(TAG, mixer->in_load, {return ESP_GMF_JOB_ERR_FAIL;},
"in load", sizeof(esp_gmf_payload_t *) * mixer_info->src_num);
mixer->in_arr = esp_gmf_oal_calloc(1, sizeof(int *) * mixer_info->src_num);
ESP_GMF_MEM_VERIFY(TAG, mixer->in_arr, {return ESP_GMF_JOB_ERR_FAIL;},
"in buffer array", sizeof(int *) * mixer_info->src_num);
GMF_AUDIO_UPDATE_SND_INFO(self, mixer_info->sample_rate, mixer_info->bits_per_sample, mixer_info->channel);
mixer->need_reopen = false;
ESP_LOGD(TAG, "Open, %p", self);
return ESP_GMF_JOB_ERR_OK;
}
static esp_gmf_job_err_t esp_gmf_mixer_close(esp_gmf_element_handle_t self, void *para)
{
esp_gmf_mixer_t *mixer = (esp_gmf_mixer_t *)self;
ESP_LOGD(TAG, "Closed, %p", self);
if (mixer->mixer_hd != NULL) {
esp_ae_mixer_close(mixer->mixer_hd);
mixer->mixer_hd = NULL;
}
if (mixer->in_arr != NULL) {
esp_gmf_oal_free(mixer->in_arr);
mixer->in_arr = NULL;
}
if (mixer->in_load != NULL) {
esp_gmf_oal_free(mixer->in_load);
mixer->in_load = NULL;
}
return ESP_GMF_ERR_OK;
}
static esp_gmf_job_err_t esp_gmf_mixer_process(esp_gmf_element_handle_t self, void *para)
{
esp_gmf_mixer_t *mixer = (esp_gmf_mixer_t *)self;
esp_gmf_job_err_t out_len = ESP_GMF_JOB_ERR_OK;
if (mixer->need_reopen) {
esp_gmf_mixer_close(self, NULL);
out_len = esp_gmf_mixer_open(self, NULL);
if (out_len != ESP_GMF_JOB_ERR_OK) {
ESP_LOGE(TAG, "Mixer reopen failed");
return out_len;
}
}
int read_len = 0;
esp_gmf_err_io_t ret = ESP_GMF_IO_OK;
int status_end = 0;
ESP_GMF_RET_ON_NOT_OK(TAG, ret, { return ESP_GMF_JOB_ERR_FAIL;}, "Failed to apply mixer setting");
esp_gmf_port_handle_t in = ESP_GMF_ELEMENT_GET(self)->in;
esp_gmf_port_handle_t in_port = in;
esp_gmf_port_handle_t out_port = ESP_GMF_ELEMENT_GET(self)->out;
int index = mixer->src_num;
memset(mixer->in_load, 0, sizeof(esp_gmf_payload_t *) * index);
mixer->out_load = NULL;
int i = 0;
#ifdef MIXER_DEBUG
uint64_t start = 0;
uint64_t end = 0;
uint64_t acquire_in_time = 0;
uint32_t frame_time = mixer->frame_time;
#endif
while (in_port != NULL) {
#ifdef MIXER_DEBUG
start = esp_timer_get_time();
#endif
int wait_ticks = ((in_port->wait_ticks < (mixer->frame_time / index)) ? (mixer->frame_time / index) : (in_port->wait_ticks));
ret = esp_gmf_port_acquire_in(in_port, &(mixer->in_load[i]), mixer->process_num, wait_ticks);
#ifdef MIXER_DEBUG
end = esp_timer_get_time();
acquire_in_time += ((end - start) / 1000);
ESP_LOGD(TAG, "Port %d acquire in time: %lld ms, wait ticks: %d", i, (end - start) / 1000, wait_ticks);
#endif
if (ret == ESP_GMF_IO_FAIL || mixer->in_load[i]->buf == NULL) {
ESP_LOGE(TAG, "Acquire in failed, idx:%d, ret: %d", i, ret);
out_len = ESP_GMF_JOB_ERR_FAIL;
goto __mixer_release;
}
if (ret == ESP_GMF_IO_TIMEOUT || ret == ESP_GMF_IO_ABORT || mixer->in_load[i]->is_done) {
status_end++;
}
read_len = mixer->in_load[i]->valid_size;
mixer->in_arr[i] = mixer->in_load[i]->buf;
if (read_len < mixer->process_num) {
memset(mixer->in_arr[i] + read_len, 0, mixer->process_num - read_len);
}
in_port = in_port->next;
ESP_LOGV(TAG, "IN: idx: %d load: %p, buf: %p, valid size: %d, buf length: %d, done: %d",
i, mixer->in_load[i], mixer->in_load[i]->buf, mixer->in_load[i]->valid_size,
mixer->in_load[i]->buf_length, mixer->in_load[i]->is_done);
i++;
}
#ifdef MIXER_DEBUG
if (acquire_in_time > frame_time) {
ESP_LOGW(TAG, "Total acquire in time: %lld ms, frame time: %ld ms", acquire_in_time, frame_time);
}
#endif
ret = esp_gmf_port_acquire_out(out_port, &mixer->out_load, mixer->process_num, ESP_GMF_MAX_DELAY);
ESP_GMF_PORT_ACQUIRE_OUT_CHECK(TAG, ret, out_len, { goto __mixer_release;});
// Down-mixer never stop in gmf, only user can set to stop
if (status_end == index) {
memset(mixer->out_load->buf, 0, mixer->process_num);
out_len = ESP_GMF_JOB_ERR_OK;
goto __mixer_release;
}
esp_gmf_oal_mutex_lock(((esp_gmf_audio_element_t *)self)->lock);
esp_ae_err_t porc_ret = esp_ae_mixer_process(mixer->mixer_hd, mixer->process_num / mixer->bytes_per_sample,
(void *)mixer->in_arr, mixer->out_load->buf);
esp_gmf_oal_mutex_unlock(((esp_gmf_audio_element_t *)self)->lock);
if (porc_ret != ESP_AE_ERR_OK) {
ESP_LOGE(TAG, "Mix process error %d.", porc_ret);
return ESP_GMF_JOB_ERR_FAIL;
}
ESP_LOGV(TAG, "OUT: load: %p, buf: %p, valid size: %d, buf length: %d",
mixer->out_load, mixer->out_load->buf, mixer->out_load->valid_size, mixer->out_load->buf_length);
mixer->out_load->valid_size = mixer->process_num;
if (mixer->out_load->valid_size > 0) {
esp_gmf_audio_el_update_file_pos((esp_gmf_element_handle_t)self, mixer->out_load->valid_size);
}
__mixer_release:
// Release in and out port
if (mixer->out_load != NULL) {
ret = esp_gmf_port_release_out(out_port, mixer->out_load, ESP_GMF_MAX_DELAY);
if ((ret < ESP_GMF_IO_OK) && (ret != ESP_GMF_IO_ABORT)) {
ESP_LOGE(TAG, "OUT port release error, ret:%d", ret);
out_len = ESP_GMF_JOB_ERR_FAIL;
}
}
in_port = in;
i = 0;
while (in_port != NULL && mixer->in_load[i] != NULL) {
ret = esp_gmf_port_release_in(in_port, mixer->in_load[i], ESP_GMF_MAX_DELAY);
if ((ret < ESP_GMF_IO_OK) && (ret != ESP_GMF_IO_ABORT)) {
ESP_LOGE(TAG, "IN port release error, ret:%d", ret);
out_len = ESP_GMF_JOB_ERR_FAIL;
}
in_port = in_port->next;
i++;
}
return out_len;
}
static esp_gmf_err_t mixer_received_event_handler(esp_gmf_event_pkt_t *evt, void *ctx)
{
ESP_GMF_NULL_CHECK(TAG, ctx, {return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_NULL_CHECK(TAG, evt, {return ESP_GMF_ERR_INVALID_ARG;});
if ((evt->type != ESP_GMF_EVT_TYPE_REPORT_INFO)
|| (evt->sub != ESP_GMF_INFO_SOUND)
|| (evt->payload == NULL)) {
return ESP_GMF_ERR_OK;
}
esp_gmf_element_handle_t self = (esp_gmf_element_handle_t)ctx;
esp_gmf_element_handle_t el = evt->from;
esp_gmf_event_state_t state = ESP_GMF_EVENT_STATE_NONE;
esp_gmf_element_get_state(self, &state);
esp_gmf_info_sound_t *info = (esp_gmf_info_sound_t *)evt->payload;
esp_ae_mixer_cfg_t *config = (esp_ae_mixer_cfg_t *)OBJ_GET_CFG(self);
ESP_GMF_NULL_CHECK(TAG, config, return ESP_GMF_ERR_FAIL);
esp_gmf_mixer_t *mixer = (esp_gmf_mixer_t *)self;
mixer->need_reopen = (config->sample_rate != info->sample_rates) || (info->channels != config->channel) || (config->bits_per_sample != info->bits);
config->sample_rate = info->sample_rates;
config->channel = info->channels;
config->bits_per_sample = info->bits;
ESP_LOGD(TAG, "RECV element info, from: %s-%p, next: %p, self: %s-%p, type: %x, state: %s, rate: %d, ch: %d, bits: %d",
OBJ_GET_TAG(el), el, esp_gmf_node_for_next((esp_gmf_node_t *)el), OBJ_GET_TAG(self), self, evt->type,
esp_gmf_event_get_state_str(state), info->sample_rates, info->channels, info->bits);
if (state == ESP_GMF_EVENT_STATE_NONE) {
esp_gmf_element_set_state(self, ESP_GMF_EVENT_STATE_INITIALIZED);
}
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t esp_gmf_mixer_destroy(esp_gmf_element_handle_t self)
{
esp_gmf_mixer_t *mixer = (esp_gmf_mixer_t *)self;
ESP_LOGD(TAG, "Destroyed, %p", self);
free_esp_ae_mixer_cfg(OBJ_GET_CFG(self));
if (mixer->mode) {
esp_gmf_oal_free(mixer->mode);
}
esp_gmf_audio_el_deinit(self);
esp_gmf_oal_free(mixer);
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t _load_mixer_caps_func(esp_gmf_element_handle_t handle)
{
esp_gmf_cap_t *caps = NULL;
esp_gmf_cap_t dec_caps = {0};
dec_caps.cap_eightcc = ESP_GMF_CAPS_AUDIO_MIXER;
dec_caps.attr_fun = NULL;
int ret = esp_gmf_cap_append(&caps, &dec_caps);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to create capability");
esp_gmf_element_t *el = (esp_gmf_element_t *)handle;
el->caps = caps;
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t _load_mixer_methods_func(esp_gmf_element_handle_t handle)
{
esp_gmf_method_t *method = NULL;
esp_gmf_args_desc_t *set_args = NULL;
esp_gmf_err_t ret = esp_gmf_args_desc_append(&set_args, AMETHOD_ARG(MIXER, SET_INFO, RATE), ESP_GMF_ARGS_TYPE_UINT32, sizeof(uint32_t), 0);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to append RATE argument");
ret = esp_gmf_args_desc_append(&set_args, AMETHOD_ARG(MIXER, SET_INFO, CH), ESP_GMF_ARGS_TYPE_UINT8,
sizeof(uint8_t), sizeof(uint32_t));
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to append CHANNEL argument");
ret = esp_gmf_args_desc_append(&set_args, AMETHOD_ARG(MIXER, SET_INFO, BITS), ESP_GMF_ARGS_TYPE_UINT8,
sizeof(uint8_t), sizeof(uint8_t) + sizeof(uint32_t));
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to append BITS argument");
ret = esp_gmf_method_append(&method, AMETHOD(MIXER, SET_INFO), __mixer_set_audio_info, set_args);
ESP_GMF_RET_ON_ERROR(TAG, ret, {return ret;}, "Failed to register %s method", AMETHOD(MIXER, SET_INFO));
set_args = NULL;
ret = esp_gmf_args_desc_append(&set_args, AMETHOD_ARG(MIXER, SET_MODE, IDX), ESP_GMF_ARGS_TYPE_UINT8, sizeof(uint8_t), 0);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to append INDEX argument");
ret = esp_gmf_args_desc_append(&set_args, AMETHOD_ARG(MIXER, SET_MODE, MODE), ESP_GMF_ARGS_TYPE_INT32,
sizeof(int32_t), sizeof(uint8_t));
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to append MODE argument");
ret = esp_gmf_method_append(&method, AMETHOD(MIXER, SET_MODE), __mixer_set_mode, set_args);
ESP_GMF_RET_ON_ERROR(TAG, ret, {return ret;}, "Failed to register %s method", AMETHOD(MIXER, SET_MODE));
esp_gmf_element_t *el = (esp_gmf_element_t *)handle;
el->method = method;
return ESP_GMF_ERR_OK;
}
esp_gmf_err_t esp_gmf_mixer_set_mode(esp_gmf_element_handle_t handle, uint8_t src_idx, esp_ae_mixer_mode_t mode)
{
ESP_GMF_NULL_CHECK(TAG, handle, { return ESP_GMF_ERR_INVALID_ARG;});
esp_gmf_mixer_t *mixer = (esp_gmf_mixer_t *)handle;
esp_ae_mixer_cfg_t *cfg = (esp_ae_mixer_cfg_t *)OBJ_GET_CFG(handle);
ESP_GMF_NULL_CHECK(TAG, cfg, return ESP_GMF_ERR_FAIL);
if (src_idx >= cfg->src_num) {
ESP_LOGE(TAG, "Source index %d overlimit %d hd:%p", src_idx, cfg->src_num, mixer);
return ESP_GMF_ERR_INVALID_ARG;
}
if (mixer->mixer_hd) {
esp_gmf_oal_mutex_lock(((esp_gmf_audio_element_t *)handle)->lock);
esp_ae_err_t ret = esp_ae_mixer_set_mode(mixer->mixer_hd, src_idx, mode);
esp_gmf_oal_mutex_unlock(((esp_gmf_audio_element_t *)handle)->lock);
ESP_GMF_RET_ON_ERROR(TAG, ret, return ESP_GMF_JOB_ERR_FAIL;, "mixerualize set error %d", ret);
}
mixer->mode[src_idx] = mode;
return ESP_GMF_ERR_OK;
}
esp_gmf_err_t esp_gmf_mixer_set_audio_info(esp_gmf_element_handle_t handle, uint32_t sample_rate,
uint8_t bits, uint8_t channel)
{
ESP_GMF_NULL_CHECK(TAG, handle, { return ESP_GMF_ERR_INVALID_ARG;});
esp_ae_mixer_cfg_t *cfg = (esp_ae_mixer_cfg_t *)OBJ_GET_CFG(handle);
ESP_GMF_NULL_CHECK(TAG, cfg, return ESP_GMF_ERR_FAIL);
if (cfg->sample_rate == sample_rate && cfg->bits_per_sample == bits && cfg->channel == channel) {
return ESP_GMF_ERR_OK;
}
cfg->sample_rate = sample_rate;
cfg->bits_per_sample = bits;
cfg->channel = channel;
esp_gmf_mixer_t *mixer = (esp_gmf_mixer_t *)handle;
mixer->need_reopen = true;
return ESP_GMF_ERR_OK;
}
esp_gmf_err_t esp_gmf_mixer_init(esp_ae_mixer_cfg_t *config, esp_gmf_element_handle_t *handle)
{
ESP_GMF_NULL_CHECK(TAG, handle, {return ESP_GMF_ERR_INVALID_ARG;});
*handle = NULL;
esp_gmf_err_t ret = ESP_GMF_ERR_OK;
esp_gmf_mixer_t *mixer = esp_gmf_oal_calloc(1, sizeof(esp_gmf_mixer_t));
ESP_GMF_MEM_VERIFY(TAG, mixer, {return ESP_GMF_ERR_MEMORY_LACK;}, "mixer", sizeof(esp_gmf_mixer_t));
esp_gmf_obj_t *obj = (esp_gmf_obj_t *)mixer;
obj->new_obj = esp_gmf_mixer_new;
obj->del_obj = esp_gmf_mixer_destroy;
esp_ae_mixer_cfg_t *cfg = NULL;
if (config) {
if (config->src_info == NULL) {
config->src_info = (esp_ae_mixer_info_t *)esp_gmf_default_mixer_src_info;
config->src_num = sizeof(esp_gmf_default_mixer_src_info) / sizeof(esp_ae_mixer_info_t);
}
dupl_esp_ae_mixer_cfg(config, &cfg);
} else {
esp_ae_mixer_cfg_t dcfg = DEFAULT_ESP_GMF_MIXER_CONFIG();
dcfg.src_info = (esp_ae_mixer_info_t *)esp_gmf_default_mixer_src_info;
dcfg.src_num = sizeof(esp_gmf_default_mixer_src_info) / sizeof(esp_ae_mixer_info_t);
dupl_esp_ae_mixer_cfg(&dcfg, &cfg);
}
ESP_GMF_CHECK(TAG, cfg, ret = ESP_GMF_ERR_MEMORY_LACK; goto MIXER_INIT_FAIL;, "Failed to allocate mixer configuration");
esp_gmf_obj_set_config(obj, cfg, sizeof(esp_ae_mixer_cfg_t));
mixer->mode = esp_gmf_oal_calloc(cfg->src_num, sizeof(esp_ae_mixer_mode_t));
ESP_GMF_MEM_VERIFY(TAG, mixer->mode, {ret = ESP_GMF_ERR_MEMORY_LACK; goto MIXER_INIT_FAIL;}, "Allocate(%d) failed", cfg->src_num * sizeof(esp_ae_mixer_mode_t));
for (int i = 0; i < cfg->src_num; i++) {
mixer->mode[i] = ESP_AE_MIXER_MODE_FADE_UPWARD;
}
ret = esp_gmf_obj_set_tag(obj, "aud_mixer");
ESP_GMF_RET_ON_NOT_OK(TAG, ret, goto MIXER_INIT_FAIL, "Failed to set obj tag");
esp_gmf_element_cfg_t el_cfg = {0};
mixer->process_num = MIXER_GET_FRAME_BYTE_SIZE(cfg);
ESP_GMF_ELEMENT_IN_PORT_ATTR_SET(el_cfg.in_attr, ESP_GMF_EL_PORT_CAP_MULTI, 0, 0,
ESP_GMF_PORT_TYPE_BLOCK | ESP_GMF_PORT_TYPE_BYTE, mixer->process_num);
ESP_GMF_ELEMENT_IN_PORT_ATTR_SET(el_cfg.out_attr, ESP_GMF_EL_PORT_CAP_SINGLE, 0, 0,
ESP_GMF_PORT_TYPE_BLOCK | ESP_GMF_PORT_TYPE_BYTE, mixer->process_num);
el_cfg.dependency = true;
ret = esp_gmf_audio_el_init(mixer, &el_cfg);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, goto MIXER_INIT_FAIL, "Failed to initialize mixer element");
ESP_GMF_ELEMENT_GET(mixer)->ops.open = esp_gmf_mixer_open;
ESP_GMF_ELEMENT_GET(mixer)->ops.process = esp_gmf_mixer_process;
ESP_GMF_ELEMENT_GET(mixer)->ops.close = esp_gmf_mixer_close;
ESP_GMF_ELEMENT_GET(mixer)->ops.event_receiver = mixer_received_event_handler;
ESP_GMF_ELEMENT_GET(mixer)->ops.load_caps = _load_mixer_caps_func;
ESP_GMF_ELEMENT_GET(mixer)->ops.load_methods = _load_mixer_methods_func;
*handle = obj;
ESP_LOGD(TAG, "Initialization, %s-%p", OBJ_GET_TAG(obj), obj);
return ESP_GMF_ERR_OK;
MIXER_INIT_FAIL:
esp_gmf_mixer_destroy(obj);
return ret;
}
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_audio/esp_gmf_rate_cvt.c
+276
View File
@@ -0,0 +1,276 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#include <string.h>
#include "esp_log.h"
#include "esp_gmf_oal_mem.h"
#include "esp_gmf_oal_mutex.h"
#include "esp_gmf_node.h"
#include "esp_gmf_rate_cvt.h"
#include "gmf_audio_common.h"
#include "esp_gmf_audio_methods_def.h"
#include "esp_gmf_cap.h"
#include "esp_gmf_caps_def.h"
#include "esp_gmf_audio_element.h"
/**
* @brief Audio rate conversion context in GMF
*/
typedef struct {
esp_gmf_audio_element_t parent; /*!< The GMF rate cvt handle */
esp_ae_rate_cvt_handle_t rate_hd; /*!< The audio effects rate cvt handle */
uint8_t bytes_per_sample; /*!< Bytes number of per sampling point */
bool need_reopen : 1; /*!< Whether need to reopen.
True: Execute the close function first, then execute the open function
False: Do nothing */
bool bypass : 1; /*!< Whether bypass. True: need bypass. False: needn't bypass */
} esp_gmf_rate_cvt_t;
static const char *TAG = "ESP_GMF_RATE_CVT";
static esp_gmf_err_t __rate_cvt_set_dest_rate(esp_gmf_element_handle_t handle, esp_gmf_args_desc_t *arg_desc,
uint8_t *buf, int buf_len)
{
ESP_GMF_NULL_CHECK(TAG, buf, {return ESP_GMF_ERR_INVALID_ARG;});
uint32_t dest_rate = *((uint32_t *)buf);
return esp_gmf_rate_cvt_set_dest_rate(handle, dest_rate);
}
static esp_gmf_err_t esp_gmf_rate_cvt_new(void *cfg, esp_gmf_obj_handle_t *handle)
{
return esp_gmf_rate_cvt_init(cfg, (esp_gmf_element_handle_t *)handle);
}
static esp_gmf_job_err_t esp_gmf_rate_cvt_open(esp_gmf_element_handle_t self, void *para)
{
esp_gmf_rate_cvt_t *rate_cvt = (esp_gmf_rate_cvt_t *)self;
esp_ae_rate_cvt_cfg_t *rate_info = (esp_ae_rate_cvt_cfg_t *)OBJ_GET_CFG(self);
ESP_GMF_NULL_CHECK(TAG, rate_info, {return ESP_GMF_JOB_ERR_FAIL;});
rate_cvt->bytes_per_sample = (rate_info->bits_per_sample >> 3) * rate_info->channel;
esp_ae_rate_cvt_open(rate_info, &rate_cvt->rate_hd);
ESP_GMF_CHECK(TAG, rate_cvt->rate_hd, {return ESP_GMF_JOB_ERR_FAIL;}, "Failed to create rate conversion handle");
GMF_AUDIO_UPDATE_SND_INFO(self, rate_info->dest_rate, rate_info->bits_per_sample, rate_info->channel);
ESP_LOGD(TAG, "Open, src: %"PRIu32", dest: %"PRIu32", ch: %d, bits: %d",
rate_info->src_rate, rate_info->dest_rate, rate_info->channel, rate_info->bits_per_sample);
rate_cvt->need_reopen = false;
rate_cvt->bypass = rate_info->src_rate == rate_info->dest_rate;
return ESP_GMF_JOB_ERR_OK;
}
static esp_gmf_job_err_t esp_gmf_rate_cvt_close(esp_gmf_element_handle_t self, void *para)
{
esp_gmf_rate_cvt_t *rate_cvt = (esp_gmf_rate_cvt_t *)self;
ESP_LOGD(TAG, "Closed, %p", self);
if (rate_cvt->rate_hd != NULL) {
esp_ae_rate_cvt_close(rate_cvt->rate_hd);
rate_cvt->rate_hd = NULL;
}
return ESP_GMF_JOB_ERR_OK;
}
static esp_gmf_job_err_t esp_gmf_rate_cvt_process(esp_gmf_element_handle_t self, void *para)
{
esp_gmf_rate_cvt_t *rate_cvt = (esp_gmf_rate_cvt_t *)self;
esp_gmf_job_err_t out_len = ESP_GMF_JOB_ERR_OK;
if (rate_cvt->need_reopen) {
esp_gmf_rate_cvt_close(self, NULL);
out_len = esp_gmf_rate_cvt_open(self, NULL);
if (out_len != ESP_GMF_JOB_ERR_OK) {
ESP_LOGE(TAG, "Rate conversion reopen failed");
return out_len;
}
}
esp_ae_err_t ret = ESP_AE_ERR_OK;
esp_gmf_port_handle_t in_port = ESP_GMF_ELEMENT_GET(self)->in;
esp_gmf_port_handle_t out_port = ESP_GMF_ELEMENT_GET(self)->out;
esp_gmf_payload_t *in_load = NULL;
esp_gmf_payload_t *out_load = NULL;
int samples_num = ESP_GMF_ELEMENT_GET(rate_cvt)->in_attr.data_size / (rate_cvt->bytes_per_sample);
int bytes = samples_num * rate_cvt->bytes_per_sample;
esp_gmf_err_io_t load_ret = esp_gmf_port_acquire_in(in_port, &in_load, bytes, ESP_GMF_MAX_DELAY);
samples_num = in_load->valid_size / (rate_cvt->bytes_per_sample);
bytes = samples_num * rate_cvt->bytes_per_sample;
if((bytes != in_load->valid_size) || (load_ret < ESP_GMF_IO_OK)) {
ESP_LOGE(TAG, "Invalid in load size %d, ret %d", in_load->valid_size, load_ret);
out_len = ESP_GMF_JOB_ERR_FAIL;
goto __rate_release;
}
uint32_t out_samples_num = 0;
if (samples_num) {
ret = esp_ae_rate_cvt_get_max_out_sample_num(rate_cvt->rate_hd, samples_num, &out_samples_num);
ESP_GMF_RET_ON_ERROR(TAG, ret, {out_len = ESP_GMF_JOB_ERR_FAIL; goto __rate_release;}, "Failed to get resample out size, ret: %d", ret);
}
int acq_out_size = out_samples_num == 0 ? in_load->buf_length : out_samples_num * rate_cvt->bytes_per_sample;
if (rate_cvt->bypass && (in_port->is_shared == true)) {
// This case rate conversion is do bypass
out_load = in_load;
}
load_ret = esp_gmf_port_acquire_out(out_port, &out_load, acq_out_size, ESP_GMF_MAX_DELAY);
ESP_GMF_PORT_ACQUIRE_OUT_CHECK(TAG, load_ret, out_len, {goto __rate_release;});
if (samples_num) {
ret = esp_ae_rate_cvt_process(rate_cvt->rate_hd, (unsigned char *)in_load->buf, samples_num,
(unsigned char *)out_load->buf, &out_samples_num);
ESP_GMF_RET_ON_ERROR(TAG, ret, {out_len = ESP_GMF_JOB_ERR_FAIL; goto __rate_release;}, "Rate conversion process error, ret: %d", ret);
}
out_load->valid_size = out_samples_num * rate_cvt->bytes_per_sample;
out_load->pts = in_load->pts;
out_load->is_done = in_load->is_done;
ESP_LOGV(TAG, "Out Samples: %ld, IN-PLD: %p-%p-%d-%d-%d, OUT-PLD: %p-%p-%d-%d-%d", out_samples_num, in_load, in_load->buf,
in_load->valid_size, in_load->buf_length, in_load->is_done, out_load,
out_load->buf, out_load->valid_size, out_load->buf_length, out_load->is_done);
esp_gmf_audio_el_update_file_pos((esp_gmf_element_handle_t)self, out_load->valid_size);
if (in_load->is_done) {
out_len = ESP_GMF_JOB_ERR_DONE;
ESP_LOGD(TAG, "Rate convert done, out len: %d", out_load->valid_size);
}
__rate_release:
// Release in and out port
if (out_load != NULL) {
load_ret = esp_gmf_port_release_out(out_port, out_load, ESP_GMF_MAX_DELAY);
if ((load_ret < ESP_GMF_IO_OK) && (load_ret != ESP_GMF_IO_ABORT)) {
ESP_LOGE(TAG, "OUT port release error, ret:%d", load_ret);
out_len = ESP_GMF_JOB_ERR_FAIL;
}
}
if (in_load != NULL) {
load_ret = esp_gmf_port_release_in(in_port, in_load, ESP_GMF_MAX_DELAY);
if ((load_ret < ESP_GMF_IO_OK) && (load_ret != ESP_GMF_IO_ABORT)) {
ESP_LOGE(TAG, "IN port release error, ret:%d", load_ret);
out_len = ESP_GMF_JOB_ERR_FAIL;
}
}
return out_len;
}
static esp_gmf_err_t rate_cvt_received_event_handler(esp_gmf_event_pkt_t *evt, void *ctx)
{
ESP_GMF_NULL_CHECK(TAG, ctx, {return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_NULL_CHECK(TAG, evt, {return ESP_GMF_ERR_INVALID_ARG;});
if ((evt->type != ESP_GMF_EVT_TYPE_REPORT_INFO)
|| (evt->sub != ESP_GMF_INFO_SOUND)
|| (evt->payload == NULL)) {
return ESP_GMF_ERR_OK;
}
esp_gmf_element_handle_t self = (esp_gmf_element_handle_t)ctx;
esp_gmf_element_handle_t el = evt->from;
esp_gmf_event_state_t state = ESP_GMF_EVENT_STATE_NONE;
esp_gmf_element_get_state(self, &state);
esp_gmf_info_sound_t *info = (esp_gmf_info_sound_t *)evt->payload;
esp_ae_rate_cvt_cfg_t *config = (esp_ae_rate_cvt_cfg_t *)OBJ_GET_CFG(self);
ESP_GMF_NULL_CHECK(TAG, config, return ESP_GMF_ERR_FAIL);
esp_gmf_rate_cvt_t *rate_cvt = (esp_gmf_rate_cvt_t *)self;
rate_cvt->need_reopen = (config->src_rate != info->sample_rates) || (info->channels != config->channel) || (config->bits_per_sample != info->bits);
config->src_rate = info->sample_rates;
config->channel = info->channels;
config->bits_per_sample = info->bits;
ESP_LOGD(TAG, "RECV element info, from: %s-%p, next: %p, self: %s-%p, type: %x, state: %s, rate: %d, ch: %d, bits: %d",
OBJ_GET_TAG(el), el, esp_gmf_node_for_next((esp_gmf_node_t *)el), OBJ_GET_TAG(self), self, evt->type,
esp_gmf_event_get_state_str(state), info->sample_rates, info->channels, info->bits);
if (state == ESP_GMF_EVENT_STATE_NONE) {
esp_gmf_element_set_state(self, ESP_GMF_EVENT_STATE_INITIALIZED);
}
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t esp_gmf_rate_cvt_destroy(esp_gmf_element_handle_t self)
{
esp_gmf_rate_cvt_t *rate_cvt = (esp_gmf_rate_cvt_t *)self;
ESP_LOGD(TAG, "Destroyed, %p", self);
void *cfg = OBJ_GET_CFG(self);
if (cfg) {
esp_gmf_oal_free(cfg);
}
esp_gmf_audio_el_deinit(self);
esp_gmf_oal_free(rate_cvt);
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t _load_rate_cvt_caps_func(esp_gmf_element_handle_t handle)
{
esp_gmf_cap_t *caps = NULL;
esp_gmf_cap_t dec_caps = {0};
dec_caps.cap_eightcc = ESP_GMF_CAPS_AUDIO_RATE_CONVERT;
dec_caps.attr_fun = NULL;
int ret = esp_gmf_cap_append(&caps, &dec_caps);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to create capability");
esp_gmf_element_t *el = (esp_gmf_element_t *)handle;
el->caps = caps;
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t _load_rate_cvt_methods_func(esp_gmf_element_handle_t handle)
{
esp_gmf_method_t *method = NULL;
esp_gmf_args_desc_t *set_args = NULL;
esp_gmf_err_t ret = esp_gmf_args_desc_append(&set_args, AMETHOD_ARG(RATE_CVT, SET_DEST_RATE, RATE),
ESP_GMF_ARGS_TYPE_UINT32, sizeof(uint32_t), 0);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to append RATE argument");
ret = esp_gmf_method_append(&method, AMETHOD(RATE_CVT, SET_DEST_RATE), __rate_cvt_set_dest_rate, set_args);
ESP_GMF_RET_ON_ERROR(TAG, ret, {return ret;}, "Failed to register %s method", AMETHOD(RATE_CVT, SET_DEST_RATE));
esp_gmf_element_t *el = (esp_gmf_element_t *)handle;
el->method = method;
return ESP_GMF_ERR_OK;
}
esp_gmf_err_t esp_gmf_rate_cvt_set_dest_rate(esp_gmf_element_handle_t handle, uint32_t dest_rate)
{
ESP_GMF_NULL_CHECK(TAG, handle, { return ESP_GMF_ERR_INVALID_ARG;});
esp_ae_rate_cvt_cfg_t *cfg = (esp_ae_rate_cvt_cfg_t *)OBJ_GET_CFG(handle);
ESP_GMF_NULL_CHECK(TAG, cfg, return ESP_GMF_ERR_FAIL);
if (cfg->dest_rate == dest_rate) {
return ESP_GMF_ERR_OK;
}
cfg->dest_rate = dest_rate;
esp_gmf_rate_cvt_t *rate_cvt = (esp_gmf_rate_cvt_t *)handle;
rate_cvt->need_reopen = true;
return ESP_GMF_ERR_OK;
}
esp_gmf_err_t esp_gmf_rate_cvt_init(esp_ae_rate_cvt_cfg_t *config, esp_gmf_element_handle_t *handle)
{
ESP_GMF_NULL_CHECK(TAG, handle, {return ESP_GMF_ERR_INVALID_ARG;});
*handle = NULL;
esp_gmf_err_t ret = ESP_GMF_ERR_OK;
esp_gmf_rate_cvt_t *rate_cvt = esp_gmf_oal_calloc(1, sizeof(esp_gmf_rate_cvt_t));
ESP_GMF_MEM_VERIFY(TAG, rate_cvt, {return ESP_GMF_ERR_MEMORY_LACK;}, "rate conversion", sizeof(esp_gmf_rate_cvt_t));
esp_gmf_obj_t *obj = (esp_gmf_obj_t *)rate_cvt;
obj->new_obj = esp_gmf_rate_cvt_new;
obj->del_obj = esp_gmf_rate_cvt_destroy;
esp_ae_rate_cvt_cfg_t *cfg = esp_gmf_oal_calloc(1, sizeof(esp_ae_rate_cvt_cfg_t));
ESP_GMF_MEM_VERIFY(TAG, cfg, {ret = ESP_GMF_ERR_MEMORY_LACK; goto RATE_CVT_INIT_FAIL;}, "Rate conversion configuration", sizeof(esp_ae_rate_cvt_cfg_t));
esp_gmf_obj_set_config(obj, cfg, sizeof(esp_ae_rate_cvt_cfg_t));
if (config) {
memcpy(cfg, config, sizeof(esp_ae_rate_cvt_cfg_t));
} else {
esp_ae_rate_cvt_cfg_t dcfg = DEFAULT_ESP_GMF_RATE_CVT_CONFIG();
memcpy(cfg, &dcfg, sizeof(esp_ae_rate_cvt_cfg_t));
}
ret = esp_gmf_obj_set_tag(obj, "aud_rate_cvt");
ESP_GMF_RET_ON_NOT_OK(TAG, ret, goto RATE_CVT_INIT_FAIL, "Failed to set obj tag");
esp_gmf_element_cfg_t el_cfg = {0};
ESP_GMF_ELEMENT_IN_PORT_ATTR_SET(el_cfg.in_attr, ESP_GMF_EL_PORT_CAP_SINGLE, 0, 0,
ESP_GMF_PORT_TYPE_BLOCK | ESP_GMF_PORT_TYPE_BYTE, ESP_GMF_ELEMENT_PORT_DATA_SIZE_DEFAULT);
ESP_GMF_ELEMENT_IN_PORT_ATTR_SET(el_cfg.out_attr, ESP_GMF_EL_PORT_CAP_SINGLE, 0, 0,
ESP_GMF_PORT_TYPE_BLOCK | ESP_GMF_PORT_TYPE_BYTE, ESP_GMF_ELEMENT_PORT_DATA_SIZE_DEFAULT);
el_cfg.dependency = true;
ret = esp_gmf_audio_el_init(rate_cvt, &el_cfg);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, goto RATE_CVT_INIT_FAIL, "Failed to initialize rate conversion element");
ESP_GMF_ELEMENT_GET(rate_cvt)->ops.open = esp_gmf_rate_cvt_open;
ESP_GMF_ELEMENT_GET(rate_cvt)->ops.process = esp_gmf_rate_cvt_process;
ESP_GMF_ELEMENT_GET(rate_cvt)->ops.close = esp_gmf_rate_cvt_close;
ESP_GMF_ELEMENT_GET(rate_cvt)->ops.event_receiver = rate_cvt_received_event_handler;
ESP_GMF_ELEMENT_GET(rate_cvt)->ops.load_caps = _load_rate_cvt_caps_func;
ESP_GMF_ELEMENT_GET(rate_cvt)->ops.load_methods = _load_rate_cvt_methods_func;
*handle = obj;
ESP_LOGD(TAG, "Initialization, %s-%p", OBJ_GET_TAG(obj), obj);
return ESP_GMF_ERR_OK;
RATE_CVT_INIT_FAIL:
esp_gmf_rate_cvt_destroy(obj);
return ret;
}
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_audio/esp_gmf_sonic.c
+435
View File
@@ -0,0 +1,435 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#include <string.h>
#include "esp_log.h"
#include "esp_gmf_oal_mem.h"
#include "esp_gmf_oal_mutex.h"
#include "esp_gmf_node.h"
#include "esp_gmf_sonic.h"
#include "gmf_audio_common.h"
#include "esp_gmf_audio_methods_def.h"
#include "esp_gmf_cap.h"
#include "esp_gmf_caps_def.h"
#include "esp_gmf_audio_element.h"
#define SONIC_DEFAULT_OUTPUT_TIME_MS (10)
/**
* @brief Audio sonic context in GMF
*/
typedef struct {
esp_gmf_audio_element_t parent; /*!< The GMF sonic handle */
esp_ae_sonic_handle_t sonic_hd; /*!< The audio effects sonic handle */
uint8_t bytes_per_sample; /*!< Bytes number of per sampling point */
uint32_t sample_rate; /*!< The audio sample rate */
uint8_t bits_per_sample; /*!< Bits number of per sampling point */
uint8_t channel; /*!< The audio channel */
esp_ae_sonic_in_data_t in_data_hd; /*!< The sonic input data handle */
esp_ae_sonic_out_data_t out_data_hd; /*!< The sonic output data handle */
float speed; /*!< The audio speed */
float pitch; /*!< The audio pitch */
int32_t out_size; /*!< The acquired out size */
int64_t cur_pts; /*!< The audio Presentation Time Stamp(pts) */
bool need_reopen : 1; /*!< Whether need to reopen.
True: Execute the close function first, then execute the open function
False: Do nothing */
bool is_done : 1; /*!< Whether the input data is done */
} esp_gmf_sonic_t;
static const char *TAG = "ESP_GMF_SONIC";
static esp_gmf_err_t __sonic_set_speed(esp_gmf_element_handle_t handle, esp_gmf_args_desc_t *arg_desc,
uint8_t *buf, int buf_len)
{
ESP_GMF_NULL_CHECK(TAG, arg_desc, {return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_NULL_CHECK(TAG, buf, {return ESP_GMF_ERR_INVALID_ARG;});
float *speed = (float *)buf;
return esp_gmf_sonic_set_speed(handle, *speed);
}
static esp_gmf_err_t __sonic_get_speed(esp_gmf_element_handle_t handle, esp_gmf_args_desc_t *arg_desc,
uint8_t *buf, int buf_len)
{
ESP_GMF_NULL_CHECK(TAG, buf, {return ESP_GMF_ERR_INVALID_ARG;});
float *speed = (float *)buf;
return esp_gmf_sonic_get_speed(handle, speed);
}
static esp_gmf_err_t __sonic_set_pitch(esp_gmf_element_handle_t handle, esp_gmf_args_desc_t *arg_desc,
uint8_t *buf, int buf_len)
{
ESP_GMF_NULL_CHECK(TAG, arg_desc, {return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_NULL_CHECK(TAG, buf, {return ESP_GMF_ERR_INVALID_ARG;});
float *pitch = (float *)buf;
return esp_gmf_sonic_set_pitch(handle, *pitch);
}
static esp_gmf_err_t __sonic_get_pitch(esp_gmf_element_handle_t handle, esp_gmf_args_desc_t *arg_desc,
uint8_t *buf, int buf_len)
{
ESP_GMF_NULL_CHECK(TAG, buf, {return ESP_GMF_ERR_INVALID_ARG;});
float *pitch = (float *)buf;
return esp_gmf_sonic_get_pitch(handle, pitch);
}
static esp_gmf_err_t esp_gmf_sonic_new(void *cfg, esp_gmf_obj_handle_t *handle)
{
return esp_gmf_sonic_init(cfg, (esp_gmf_element_handle_t *)handle);
}
static esp_gmf_job_err_t esp_gmf_sonic_open(esp_gmf_element_handle_t self, void *para)
{
esp_gmf_sonic_t *sonic = (esp_gmf_sonic_t *)self;
esp_ae_sonic_cfg_t *sonic_info = (esp_ae_sonic_cfg_t *)OBJ_GET_CFG(self);
ESP_GMF_NULL_CHECK(TAG, sonic_info, {return ESP_GMF_JOB_ERR_FAIL;});
sonic->sample_rate = sonic_info->sample_rate;
sonic->channel = sonic_info->channel;
sonic->bits_per_sample = sonic_info->bits_per_sample;
sonic->bytes_per_sample = (sonic_info->bits_per_sample >> 3) * sonic_info->channel;
GMF_AUDIO_UPDATE_SND_INFO(self, sonic_info->sample_rate, sonic_info->bits_per_sample, sonic_info->channel);
sonic->out_size = SONIC_DEFAULT_OUTPUT_TIME_MS * sonic->sample_rate * sonic->bytes_per_sample / 1000;
esp_ae_sonic_open(sonic_info, &sonic->sonic_hd);
ESP_GMF_CHECK(TAG, sonic->sonic_hd, {return ESP_GMF_JOB_ERR_FAIL;}, "Failed to create sonic handle");
esp_ae_sonic_set_speed(sonic->sonic_hd, sonic->speed);
esp_ae_sonic_set_pitch(sonic->sonic_hd, sonic->pitch);
sonic->need_reopen = false;
ESP_LOGD(TAG, "Open, %p", self);
return ESP_GMF_JOB_ERR_OK;
}
static esp_gmf_job_err_t esp_gmf_sonic_close(esp_gmf_element_handle_t self, void *para)
{
esp_gmf_sonic_t *sonic = (esp_gmf_sonic_t *)self;
ESP_LOGD(TAG, "Closed, %p", self);
if (sonic->sonic_hd != NULL) {
esp_ae_sonic_close(sonic->sonic_hd);
sonic->sonic_hd = NULL;
}
sonic->cur_pts = 0;
return ESP_GMF_ERR_OK;
}
static esp_gmf_job_err_t gmf_sonic_bypass_process(esp_gmf_sonic_t *sonic, esp_gmf_port_handle_t in_port, esp_gmf_port_handle_t out_port,
esp_gmf_payload_t **in_load, esp_gmf_payload_t **out_load)
{
esp_gmf_job_err_t out_len = ESP_GMF_JOB_ERR_OK;
int load_ret = esp_gmf_port_acquire_in(in_port, in_load, ESP_GMF_ELEMENT_GET(sonic)->in_attr.data_size, ESP_GMF_MAX_DELAY);
ESP_GMF_PORT_ACQUIRE_IN_CHECK(TAG, load_ret, out_len, return out_len);
sonic->is_done = (*in_load)->is_done;
if (!sonic->is_done && (*in_load)->valid_size == 0) {
return ESP_GMF_JOB_ERR_CONTINUE;
}
if (in_port->is_shared == 1) {
*out_load = *in_load;
}
size_t out_size = (*in_load)->valid_size ? (*in_load)->valid_size : ESP_GMF_ELEMENT_GET(sonic)->in_attr.data_size;
load_ret = esp_gmf_port_acquire_out(out_port, out_load, out_size, ESP_GMF_MAX_DELAY);
ESP_GMF_PORT_ACQUIRE_OUT_CHECK(TAG, load_ret, out_len, return out_len);
if (in_port->is_shared != 1) {
memcpy((*out_load)->buf, (*in_load)->buf, (*in_load)->valid_size);
(*out_load)->valid_size = (*in_load)->valid_size;
(*out_load)->is_done = (*in_load)->is_done;
(*out_load)->pts = (*in_load)->pts;
}
return sonic->is_done ? ESP_GMF_JOB_ERR_DONE : out_len;
}
static esp_gmf_job_err_t esp_gmf_sonic_process(esp_gmf_element_handle_t self, void *para)
{
esp_gmf_sonic_t *sonic = (esp_gmf_sonic_t *)self;
esp_gmf_job_err_t out_len = ESP_GMF_JOB_ERR_OK;
esp_gmf_err_io_t load_ret = ESP_GMF_IO_OK;
if (sonic->need_reopen) {
esp_gmf_sonic_close(self, NULL);
out_len = esp_gmf_sonic_open(self, NULL);
if (out_len != ESP_GMF_JOB_ERR_OK) {
ESP_LOGE(TAG, "Sonic reopen failed");
return out_len;
}
}
esp_gmf_port_handle_t in_port = ESP_GMF_ELEMENT_GET(self)->in;
esp_gmf_port_handle_t out_port = ESP_GMF_ELEMENT_GET(self)->out;
esp_gmf_payload_t *in_load = NULL;
esp_gmf_payload_t *out_load = NULL;
bool is_done = false;
sonic->out_data_hd.needed_num = sonic->out_size / sonic->bytes_per_sample;
if (sonic->speed == 1.0f && sonic->pitch == 1.0f && (sonic->out_data_hd.out_num < sonic->out_data_hd.needed_num)) {
out_len = gmf_sonic_bypass_process(sonic, in_port, out_port, &in_load, &out_load);
goto __sonic_release;
} else {
if ((sonic->out_data_hd.out_num < sonic->out_data_hd.needed_num) && sonic->is_done == false) {
int samples_num = ESP_GMF_ELEMENT_GET(sonic)->in_attr.data_size / (sonic->bytes_per_sample);
int bytes = samples_num * sonic->bytes_per_sample;
load_ret = esp_gmf_port_acquire_in(in_port, &in_load, bytes, ESP_GMF_MAX_DELAY);
samples_num = in_load->valid_size / (sonic->bytes_per_sample);
bytes = samples_num * sonic->bytes_per_sample;
sonic->is_done = in_load->is_done;
if ((bytes != in_load->valid_size) || (load_ret < ESP_GMF_IO_OK)) {
ESP_LOGE(TAG, "Invalid in load size %d, ret %d", in_load->valid_size, load_ret);
out_len = ESP_GMF_JOB_ERR_FAIL;
goto __sonic_release;
}
sonic->in_data_hd.samples = in_load->buf;
sonic->in_data_hd.num = samples_num;
sonic->cur_pts = in_load->pts;
if (sonic->in_data_hd.num == 0) {
if (sonic->is_done == true) {
out_len = ESP_GMF_JOB_ERR_DONE;
is_done = true;
} else {
out_len = ESP_GMF_JOB_ERR_CONTINUE;
goto __sonic_release;
}
}
}
load_ret = esp_gmf_port_acquire_out(out_port, &out_load, sonic->out_size, ESP_GMF_MAX_DELAY);
ESP_GMF_PORT_ACQUIRE_OUT_CHECK(TAG, load_ret, out_len, goto __sonic_release);
sonic->out_data_hd.samples = out_load->buf;
out_load->valid_size = 0;
out_load->pts = sonic->cur_pts;
if (!is_done) {
esp_gmf_oal_mutex_lock(((esp_gmf_audio_element_t *)self)->lock);
esp_ae_err_t ret = esp_ae_sonic_process(sonic->sonic_hd, &sonic->in_data_hd, &sonic->out_data_hd);
esp_gmf_oal_mutex_unlock(((esp_gmf_audio_element_t *)self)->lock);
ESP_GMF_RET_ON_ERROR(TAG, ret, {out_len = ESP_GMF_JOB_ERR_FAIL; goto __sonic_release;}, "Sonic process error %d", ret);
sonic->in_data_hd.samples = ((uint8_t *)sonic->in_data_hd.samples) + sonic->in_data_hd.consume_num * sonic->bytes_per_sample;
sonic->in_data_hd.num -= sonic->in_data_hd.consume_num;
sonic->in_data_hd.consume_num = 0;
out_load->valid_size = sonic->out_data_hd.out_num * sonic->bytes_per_sample;
if (out_load->valid_size > 0) {
esp_gmf_audio_el_update_file_pos((esp_gmf_element_handle_t)self, out_load->valid_size);
}
out_load->is_done = false;
out_load->pts = sonic->cur_pts;
sonic->cur_pts += (uint64_t)((float)(GMF_AUDIO_CALC_PTS(out_load->valid_size, sonic->sample_rate, sonic->channel, sonic->bits_per_sample)) * sonic->speed);
ESP_LOGV(TAG, "%s, I: %p-buf: %p-sz: %d, O: %p-buf: %p-sz: %d, ret: %d", __func__, in_port,
in_load->buf, in_load->valid_size, out_port,
out_load->buf, out_load->buf_length, ret);
if (sonic->out_data_hd.out_num == sonic->out_data_hd.needed_num) {
out_len = ESP_GMF_JOB_ERR_TRUNCATE;
goto __sonic_release;
}
}
if (sonic->is_done == true) {
out_load->is_done = true;
out_len = ESP_GMF_JOB_ERR_DONE;
goto __sonic_release;
}
}
__sonic_release:
if (out_load != NULL) {
load_ret = esp_gmf_port_release_out(out_port, out_load, ESP_GMF_MAX_DELAY);
if ((load_ret < ESP_GMF_IO_OK) && (load_ret != ESP_GMF_IO_ABORT)) {
ESP_LOGE(TAG, "OUT port release error, ret:%d", load_ret);
out_len = ESP_GMF_JOB_ERR_FAIL;
}
}
if (in_load != NULL) {
load_ret = esp_gmf_port_release_in(in_port, in_load, ESP_GMF_MAX_DELAY);
if ((load_ret < ESP_GMF_IO_OK) && (load_ret != ESP_GMF_IO_ABORT)) {
ESP_LOGE(TAG, "IN port release error, ret:%d", load_ret);
out_len = ESP_GMF_JOB_ERR_FAIL;
}
}
return out_len;
}
static esp_gmf_err_t sonic_received_event_handler(esp_gmf_event_pkt_t *evt, void *ctx)
{
ESP_GMF_NULL_CHECK(TAG, ctx, {return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_NULL_CHECK(TAG, evt, {return ESP_GMF_ERR_INVALID_ARG;});
if ((evt->type != ESP_GMF_EVT_TYPE_REPORT_INFO)
|| (evt->sub != ESP_GMF_INFO_SOUND)
|| (evt->payload == NULL)) {
return ESP_GMF_ERR_OK;
}
esp_gmf_element_handle_t self = (esp_gmf_element_handle_t)ctx;
esp_gmf_element_handle_t el = evt->from;
esp_gmf_event_state_t state = ESP_GMF_EVENT_STATE_NONE;
esp_gmf_element_get_state(self, &state);
esp_gmf_info_sound_t *info = (esp_gmf_info_sound_t *)evt->payload;
esp_ae_sonic_cfg_t *config = (esp_ae_sonic_cfg_t *)OBJ_GET_CFG(self);
ESP_GMF_NULL_CHECK(TAG, config, return ESP_GMF_ERR_FAIL);
esp_gmf_sonic_t *sonic = (esp_gmf_sonic_t *)self;
sonic->need_reopen = (config->sample_rate != info->sample_rates) || (info->channels != config->channel) || (config->bits_per_sample != info->bits);
config->sample_rate = info->sample_rates;
config->channel = info->channels;
config->bits_per_sample = info->bits;
ESP_LOGD(TAG, "RECV element info, from: %s-%p, next: %p, self: %s-%p, type: %x, state: %s, rate: %d, ch: %d, bits: %d",
OBJ_GET_TAG(el), el, esp_gmf_node_for_next((esp_gmf_node_t *)el), OBJ_GET_TAG(self), self, evt->type,
esp_gmf_event_get_state_str(state), info->sample_rates, info->channels, info->bits);
if (state == ESP_GMF_EVENT_STATE_NONE) {
esp_gmf_element_set_state(self, ESP_GMF_EVENT_STATE_INITIALIZED);
}
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t esp_gmf_sonic_destroy(esp_gmf_element_handle_t self)
{
esp_gmf_sonic_t *sonic = (esp_gmf_sonic_t *)self;
ESP_LOGD(TAG, "Destroyed, %p", self);
void *cfg = OBJ_GET_CFG(self);
if (cfg) {
esp_gmf_oal_free(cfg);
}
esp_gmf_audio_el_deinit(self);
esp_gmf_oal_free(sonic);
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t _load_sonic_caps_func(esp_gmf_element_handle_t handle)
{
esp_gmf_cap_t *caps = NULL;
esp_gmf_cap_t dec_caps = {0};
dec_caps.cap_eightcc = ESP_GMF_CAPS_AUDIO_SONIC;
dec_caps.attr_fun = NULL;
int ret = esp_gmf_cap_append(&caps, &dec_caps);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to create capability");
esp_gmf_element_t *el = (esp_gmf_element_t *)handle;
el->caps = caps;
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t _load_sonic_methods_func(esp_gmf_element_handle_t handle)
{
esp_gmf_method_t *method = NULL;
esp_gmf_args_desc_t *set_args = NULL;
esp_gmf_args_desc_t *get_args = NULL;
esp_gmf_err_t ret = esp_gmf_args_desc_append(&set_args, AMETHOD_ARG(SONIC, SET_SPEED, SPEED),
ESP_GMF_ARGS_TYPE_FLOAT, sizeof(float), 0);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to append argument");
ret = esp_gmf_method_append(&method, AMETHOD(SONIC, SET_SPEED), __sonic_set_speed, set_args);
ESP_GMF_RET_ON_ERROR(TAG, ret, {return ret;}, "Failed to register %s method", AMETHOD(SONIC, SET_SPEED));
ret = esp_gmf_args_desc_copy(set_args, &get_args);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to copy argument");
ret = esp_gmf_method_append(&method, AMETHOD(SONIC, GET_SPEED), __sonic_get_speed, get_args);
ESP_GMF_RET_ON_ERROR(TAG, ret, {return ret;}, "Failed to register %s method", AMETHOD(SONIC, GET_SPEED));
set_args = NULL;
get_args = NULL;
ret = esp_gmf_args_desc_append(&set_args, AMETHOD_ARG(SONIC, SET_PITCH, PITCH), ESP_GMF_ARGS_TYPE_FLOAT, sizeof(float), 0);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to append argument");
ret = esp_gmf_method_append(&method, AMETHOD(SONIC, SET_PITCH), __sonic_set_pitch, set_args);
ESP_GMF_RET_ON_ERROR(TAG, ret, {return ret;}, "Failed to register %s method", AMETHOD(SONIC, SET_PITCH));
ret = esp_gmf_args_desc_copy(set_args, &get_args);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, {return ret;}, "Failed to copy argument");
ret = esp_gmf_method_append(&method, AMETHOD(SONIC, GET_PITCH), __sonic_get_pitch, get_args);
ESP_GMF_RET_ON_ERROR(TAG, ret, {return ret;}, "Failed to register %s method", AMETHOD(SONIC, GET_PITCH));
esp_gmf_element_t *el = (esp_gmf_element_t *)handle;
el->method = method;
return ESP_GMF_ERR_OK;
}
esp_gmf_err_t esp_gmf_sonic_set_speed(esp_gmf_element_handle_t handle, float speed)
{
ESP_GMF_NULL_CHECK(TAG, handle, { return ESP_GMF_ERR_INVALID_ARG;});
esp_gmf_sonic_t *sonic = (esp_gmf_sonic_t *)handle;
if (sonic->sonic_hd) {
esp_gmf_oal_mutex_lock(((esp_gmf_audio_element_t *)handle)->lock);
esp_ae_err_t ret = esp_ae_sonic_set_speed(sonic->sonic_hd, speed);
esp_gmf_oal_mutex_unlock(((esp_gmf_audio_element_t *)handle)->lock);
ESP_GMF_RET_ON_ERROR(TAG, ret, return ESP_GMF_JOB_ERR_FAIL;, "sonicualize set error %d", ret);
}
sonic->speed = speed;
return ESP_GMF_ERR_OK;
}
esp_gmf_err_t esp_gmf_sonic_get_speed(esp_gmf_element_handle_t handle, float *speed)
{
ESP_GMF_NULL_CHECK(TAG, handle, { return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_NULL_CHECK(TAG, speed, { return ESP_GMF_ERR_INVALID_ARG;});
esp_gmf_sonic_t *sonic = (esp_gmf_sonic_t *)handle;
if (sonic->sonic_hd) {
esp_gmf_oal_mutex_lock(((esp_gmf_audio_element_t *)handle)->lock);
esp_ae_err_t ret = esp_ae_sonic_get_speed(sonic->sonic_hd, speed);
esp_gmf_oal_mutex_unlock(((esp_gmf_audio_element_t *)handle)->lock);
ESP_GMF_RET_ON_ERROR(TAG, ret, return ESP_GMF_JOB_ERR_FAIL;, "sonicualize set error %d", ret);
} else {
*speed = sonic->speed;
}
return ESP_GMF_ERR_OK;
}
esp_gmf_err_t esp_gmf_sonic_set_pitch(esp_gmf_element_handle_t handle, float pitch)
{
ESP_GMF_NULL_CHECK(TAG, handle, { return ESP_GMF_ERR_INVALID_ARG;});
esp_gmf_sonic_t *sonic = (esp_gmf_sonic_t *)handle;
if (sonic->sonic_hd) {
esp_gmf_oal_mutex_lock(((esp_gmf_audio_element_t *)handle)->lock);
esp_ae_err_t ret = esp_ae_sonic_set_pitch(sonic->sonic_hd, pitch);
esp_gmf_oal_mutex_unlock(((esp_gmf_audio_element_t *)handle)->lock);
ESP_GMF_RET_ON_ERROR(TAG, ret, return ESP_GMF_JOB_ERR_FAIL;, "sonicualize set error %d", ret);
}
sonic->pitch = pitch;
return ESP_GMF_ERR_OK;
}
esp_gmf_err_t esp_gmf_sonic_get_pitch(esp_gmf_element_handle_t handle, float *pitch)
{
ESP_GMF_NULL_CHECK(TAG, handle, { return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_NULL_CHECK(TAG, pitch, { return ESP_GMF_ERR_INVALID_ARG;});
esp_gmf_sonic_t *sonic = (esp_gmf_sonic_t *)handle;
if (sonic->sonic_hd) {
esp_gmf_oal_mutex_lock(((esp_gmf_audio_element_t *)handle)->lock);
esp_ae_err_t ret = esp_ae_sonic_get_pitch(sonic->sonic_hd, pitch);
esp_gmf_oal_mutex_unlock(((esp_gmf_audio_element_t *)handle)->lock);
ESP_GMF_RET_ON_ERROR(TAG, ret, return ESP_GMF_JOB_ERR_FAIL;, "sonicualize set error %d", ret);
} else {
*pitch = sonic->pitch;
}
return ESP_GMF_ERR_OK;
}
esp_gmf_err_t esp_gmf_sonic_init(esp_ae_sonic_cfg_t *config, esp_gmf_element_handle_t *handle)
{
ESP_GMF_NULL_CHECK(TAG, handle, {return ESP_GMF_ERR_INVALID_ARG;});
*handle = NULL;
esp_gmf_err_t ret = ESP_GMF_ERR_OK;
esp_gmf_sonic_t *sonic = esp_gmf_oal_calloc(1, sizeof(esp_gmf_sonic_t));
ESP_GMF_MEM_VERIFY(TAG, sonic, {return ESP_GMF_ERR_MEMORY_LACK;}, "sonic", sizeof(esp_gmf_sonic_t));
esp_gmf_obj_t *obj = (esp_gmf_obj_t *)sonic;
obj->new_obj = esp_gmf_sonic_new;
obj->del_obj = esp_gmf_sonic_destroy;
esp_ae_sonic_cfg_t *cfg = esp_gmf_oal_calloc(1, sizeof(esp_ae_sonic_cfg_t));
ESP_GMF_MEM_VERIFY(TAG, cfg, ret = ESP_GMF_ERR_MEMORY_LACK; goto SONIC_INIT_FAIL;, "sonic configuration", sizeof(esp_ae_sonic_cfg_t));
esp_gmf_obj_set_config(obj, cfg, sizeof(esp_ae_sonic_cfg_t));
if (config) {
memcpy(cfg, config, sizeof(esp_ae_sonic_cfg_t));
} else {
esp_ae_sonic_cfg_t dcfg = DEFAULT_ESP_GMF_SONIC_CONFIG();
memcpy(cfg, &dcfg, sizeof(esp_ae_sonic_cfg_t));
}
ret = esp_gmf_obj_set_tag(obj, "aud_sonic");
ESP_GMF_RET_ON_NOT_OK(TAG, ret, goto SONIC_INIT_FAIL, "Failed to set obj tag");
sonic->speed = 1.0f;
sonic->pitch = 1.0f;
esp_gmf_element_cfg_t el_cfg = {0};
ESP_GMF_ELEMENT_IN_PORT_ATTR_SET(el_cfg.in_attr, ESP_GMF_EL_PORT_CAP_SINGLE, 0, 0,
ESP_GMF_PORT_TYPE_BLOCK | ESP_GMF_PORT_TYPE_BYTE, ESP_GMF_ELEMENT_PORT_DATA_SIZE_DEFAULT);
ESP_GMF_ELEMENT_IN_PORT_ATTR_SET(el_cfg.out_attr, ESP_GMF_EL_PORT_CAP_SINGLE, 0, 0,
ESP_GMF_PORT_TYPE_BLOCK | ESP_GMF_PORT_TYPE_BYTE, ESP_GMF_ELEMENT_PORT_DATA_SIZE_DEFAULT);
el_cfg.dependency = true;
ret = esp_gmf_audio_el_init(sonic, &el_cfg);
ESP_GMF_RET_ON_NOT_OK(TAG, ret, goto SONIC_INIT_FAIL, "Failed to initialize sonic element");
ESP_GMF_ELEMENT_GET(sonic)->ops.open = esp_gmf_sonic_open;
ESP_GMF_ELEMENT_GET(sonic)->ops.process = esp_gmf_sonic_process;
ESP_GMF_ELEMENT_GET(sonic)->ops.close = esp_gmf_sonic_close;
ESP_GMF_ELEMENT_GET(sonic)->ops.event_receiver = sonic_received_event_handler;
ESP_GMF_ELEMENT_GET(sonic)->ops.load_caps = _load_sonic_caps_func;
ESP_GMF_ELEMENT_GET(sonic)->ops.load_methods = _load_sonic_methods_func;
*handle = obj;
ESP_LOGD(TAG, "Initialization, %s-%p", OBJ_GET_TAG(obj), obj);
return ESP_GMF_ERR_OK;
SONIC_INIT_FAIL:
esp_gmf_sonic_destroy(obj);
return ret;
}
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_audio/idf_component.yml
+19
View File
@@ -0,0 +1,19 @@
version: "0.7.2"
description: Espressif GMF Audio Elements Module
url: https://github.com/espressif/esp-gmf/tree/main/elements/gmf_audio
documentation: "https://github.com/espressif/esp-gmf/blob/main/elements/gmf_audio/README.md"
issues: "https://github.com/espressif/esp-gmf/issues"
tags:
- "gmf"
dependencies:
espressif/esp_audio_codec:
require: public
version: "~2.3"
espressif/esp_audio_effects:
require: public
version: "~1.1"
espressif/gmf_core:
require: public
version: "^0.7"
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_audio/include/esp_gmf_alc.h
+72
View File
@@ -0,0 +1,72 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#pragma once
#include "esp_gmf_err.h"
#include "esp_gmf_element.h"
#include "esp_ae_alc.h"
#ifdef __cplusplus
extern "C" {
#endif /* __cplusplus */
#define DEFAULT_ESP_GMF_ALC_CONFIG() { \
.sample_rate = 48000, \
.channel = 2, \
.bits_per_sample = 16, \
}
/**
* @brief Initializes the GMF ALC with the provided configuration
*
* @param[in] config Pointer to the ALC configuration
* @param[out] handle Pointer to the ALC handle to be initialized
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_INVALID_ARG Invalid configuration provided
* - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory
*/
esp_gmf_err_t esp_gmf_alc_init(esp_ae_alc_cfg_t *config, esp_gmf_element_handle_t *handle);
/**
* @brief Set specific channel gain of the ALC handle.
* Positive gain indicates an increase in volume,
* negative gain indicates a decrease in volume.
* 0 gain indicates the volume level remains unchanged.
*
* @param[in] handle The ALC handle
* @param[in] idx The channel index of the gain to retrieve. eg: 0 refers to the first channel
* @param[in] gain The gain value needs to conform to the following conditions:
* - Supported range [-64, 63]
* - Below -64 will set to mute
* - Higher than 63 not supported
* Unit: dB
*
* @return
* - ESP_GMF_ERR_OK Operation succeeded
* - ESP_GMF_ERR_INVALID_ARG Invalid input parameter
*/
esp_gmf_err_t esp_gmf_alc_set_gain(esp_gmf_element_handle_t handle, uint8_t idx, int8_t gain);
/**
* @brief Get the gain for a specific channel from the ALC handle
*
* @param[in] handle The ALC handle
* @param[in] ch_idx The channel index of the gain to retrieve. eg: 0 refers to the first channel
* @param[out] gain Pointer to store the retrieved gain. Unit: dB
*
* @return
* - ESP_GMF_ERR_OK Operation succeeded
* - ESP_GMF_ERR_INVALID_ARG Invalid input parameter
*/
esp_gmf_err_t esp_gmf_alc_get_gain(esp_gmf_element_handle_t handle, uint8_t idx, int8_t *gain);
#ifdef __cplusplus
}
#endif /* __cplusplus */
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_audio/include/esp_gmf_audio_dec.h
+75
View File
@@ -0,0 +1,75 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#pragma once
#include "esp_audio_simple_dec.h"
#include "esp_gmf_element.h"
#ifdef __cplusplus
extern "C" {
#endif /* __cplusplus */
#define DEFAULT_ESP_GMF_AUDIO_DEC_CONFIG() { \
.dec_type = ESP_AUDIO_SIMPLE_DEC_TYPE_NONE, \
.dec_cfg = NULL, \
.cfg_size = 0, \
}
/**
* @brief Initializes the GMF audio decoder with the provided configuration
*
* @param[in] config Pointer to the audio decoder configuration
* @param[out] handle Pointer to the audio decoder handle to be initialized
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_INVALID_ARG Invalid configuration provided
* - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory
*/
esp_gmf_err_t esp_gmf_audio_dec_init(esp_audio_simple_dec_cfg_t *config, esp_gmf_element_handle_t *handle);
/**
* @brief Reconfigure the GMF audio decoder and filled into provided configuration
*
* @note Only allowed when not running, i.e., in the `ESP_GMF_EVENT_STATE_NONE` or `ESP_GMF_EVENT_STATE_INITIALIZED` state
*
* @param[in] handle Audio decoder handle to be reconfigured
* @param[in] config Pointer to the new simple decoder configuration
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_INVALID_ARG Invalid handle or configuration provided
* - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory
*/
esp_gmf_err_t esp_gmf_audio_dec_reconfig(esp_gmf_element_handle_t handle, esp_audio_simple_dec_cfg_t *config);
/**
* @brief Reconfigures the GMF audio decoder with default configuration based on provided sound information
*
* @note 1. Only allowed when not running, i.e., in the `ESP_GMF_EVENT_STATE_NONE` or `ESP_GMF_EVENT_STATE_INITIALIZED` state
* 2. The actual reconfiguration behavior depends on the relationship between the provided `sound info` and
* the current decoder configuration (`cfg`):
* - If the `format_id` in `sound info` matches the decoder's current `type`, and the decoder's sub-config (`sub cfg`) is not NULL,
* only the basic fields in the sub-config will be updated with information from `sound info` (e.g., sample rate, channel count).
* - If the `format_id` differs from the current `type`, or if the sub-config is NULL,
* the decoder will be reconfigured using the **default configuration** for the specified `format_id`,
* and the current decoder `type` will be updated accordingly.
*
* @param[in] handle Audio decoder handle to be reconfigured
* @param[in] info Sound information to be configured
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_INVALID_ARG Invalid handle or decoder type provided
* - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory
*/
esp_gmf_err_t esp_gmf_audio_dec_reconfig_by_sound_info(esp_gmf_element_handle_t handle, esp_gmf_info_sound_t *info);
#ifdef __cplusplus
}
#endif /* __cplusplus */
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_audio/include/esp_gmf_audio_enc.h
+113
View File
@@ -0,0 +1,113 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#pragma once
#include "esp_gmf_err.h"
#include "esp_gmf_element.h"
#include "encoder/esp_audio_enc.h"
#ifdef __cplusplus
extern "C" {
#endif /* __cplusplus */
#define DEFAULT_ESP_GMF_AUDIO_ENC_CONFIG() { \
.type = ESP_AUDIO_TYPE_UNSUPPORT, \
.cfg = NULL, \
.cfg_sz = 0, \
}
/**
* @brief Initializes the GMF audio encoder with the provided configuration
*
* @param[in] config Pointer to the audio encoder configuration
* @param[out] handle Pointer to the audio encoder handle to be initialized
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_INVALID_ARG Invalid configuration provided
* - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory
*/
esp_gmf_err_t esp_gmf_audio_enc_init(esp_audio_enc_config_t *config, esp_gmf_element_handle_t *handle);
/**
* @brief Get the input and output frame size information of the GMF audio encoder
*
* @param[in] handle Audio encoder handle
* @param[out] in_size Pointer to store input frame size
* @param[out] out_size Pointer to store suggested output frame size
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_INVALID_ARG Invalid handle or size pointers provided
*/
esp_gmf_err_t esp_gmf_audio_enc_get_frame_size(esp_gmf_element_handle_t handle, uint32_t *in_size, uint32_t *out_size);
/**
* @brief Set the bitrate for the GMF audio encoder
*
* @param[in] handle Audio encoder handle
* @param[in] bitrate The bitrate of the encoder
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_INVALID_ARG Invalid handle or bitrate value provided
*/
esp_gmf_err_t esp_gmf_audio_enc_set_bitrate(esp_gmf_element_handle_t handle, uint32_t bitrate);
/**
* @brief Get the current bitrate of the GMF audio encoder
*
* @param[in] handle Audio encoder handle
* @param[out] bitrate Pointer to store the current bitrate
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_INVALID_ARG Invalid handle or bitrate pointer provided
*/
esp_gmf_err_t esp_gmf_audio_enc_get_bitrate(esp_gmf_element_handle_t handle, uint32_t *bitrate);
/**
* @brief Reconfigure the GMF audio encoder and filled into provided configuration
*
* @note Only allowed when not running, i.e., in the `ESP_GMF_EVENT_STATE_NONE` or `ESP_GMF_EVENT_STATE_INITIALIZED` state
*
* @param[in] handle Audio encoder handle to be reconfigured
* @param[in] config Pointer to the new audio encoder configuration
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_INVALID_ARG Invalid handle or configuration provided
* - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory
*/
esp_gmf_err_t esp_gmf_audio_enc_reconfig(esp_gmf_element_handle_t handle, esp_audio_enc_config_t *config);
/**
* @brief Reconfigures the GMF audio encoder with default configuration for the sound information
*
* @note 1. Only allowed when not running, i.e., in the `ESP_GMF_EVENT_STATE_NONE` or `ESP_GMF_EVENT_STATE_INITIALIZED` state
* 2. The actual reconfiguration behavior depends on the relationship between the provided `sound info` and
* the current encoder configuration (`cfg`):
* - If the `format_id` in `sound info` matches the encoder's current `type`, and the encoder's sub-config (`sub cfg`) is not NULL,
* only the basic fields in the sub-config will be updated with information from `sound info` (e.g., sample rate, channel count).
* - If the `format_id` differs from the current `type`, or if the sub-config is NULL,
* the encoder will be reconfigured using the **default configuration** for the specified `format_id`,
* and the current encoder `type` will be updated accordingly.
*
* @param[in] handle Audio encoder handle to be reconfigured
* @param[in] info Sound information to be configured
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_INVALID_ARG Invalid handle or encoder type provided
* - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory
*/
esp_gmf_err_t esp_gmf_audio_enc_reconfig_by_sound_info(esp_gmf_element_handle_t handle, esp_gmf_info_sound_t *info);
#ifdef __cplusplus
}
#endif /* __cplusplus */
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_audio/include/esp_gmf_audio_helper.h
+30
View File
@@ -0,0 +1,30 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#pragma once
#include "esp_gmf_err.h"
#ifdef __cplusplus
extern "C" {
#endif /* __cplusplus */
/**
* @brief Get audio codec type by uri
*
* @param[in] uri URI of audio codec
* @param[out] format_id Type of audio codec(ESP_FourCC type)
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_NOT_SUPPORT Not supported audio codec type
*/
esp_gmf_err_t esp_gmf_audio_helper_get_audio_type_by_uri(const char *uri, uint32_t *format_id);
#ifdef __cplusplus
}
#endif /* __cplusplus */
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_audio/include/esp_gmf_audio_methods_def.h
+217
View File
@@ -0,0 +1,217 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#pragma once
#include "esp_gmf_method_helper.h"
#ifdef __cplusplus
extern "C" {
#endif /* __cplusplus */
/**
* @brief Audio general method argument definition
*/
#define ESP_GMF_AUDIO_INDEX_ARG "index"
#define ESP_GMF_AUDIO_GAIN_ARG "gain"
#define ESP_GMF_AUDIO_BITS_ARG "bits"
#define ESP_GMF_AUDIO_CHANNEL_ARG "channel"
#define ESP_GMF_AUDIO_SAMPLE_RATE_ARG "sample_rate"
#define ESP_GMF_AUDIO_BITRATE_ARG "bitrate"
#define ESP_GMF_AUDIO_ENABLE_ARG "enable"
#define ESP_GMF_AUDIO_TYPE_ARG "type"
#define ESP_GMF_AUDIO_PARA_ARG "para"
#define ESP_GMF_AUDIO_MODE_ARG "mode"
#define ESP_GMF_AUDIO_SPEED_ARG "speed"
#define ESP_GMF_AUDIO_PITCH_ARG "pitch"
#define ESP_GMF_AUDIO_INSIZE_ARG "in_size"
#define ESP_GMF_AUDIO_OUTSIZE_ARG "out_size"
#define ESP_GMF_AUDIO_CFG_ARG "cfg"
#define ESP_GMF_AUDIO_INFO_ARG "info"
/**
* @brief Audio EQ parameter argument definition
*/
#define ESP_GMF_AUDIO_EQ_PARA_ARG ESP_GMF_AUDIO_PARA_ARG
#define ESP_GMF_AUDIO_EQ_PARA_FT_ARG ESP_GMF_AUDIO_TYPE_ARG
#define ESP_GMF_AUDIO_EQ_PARA_FC_ARG "fc"
#define ESP_GMF_AUDIO_EQ_PARA_Q_ARG "q"
#define ESP_GMF_AUDIO_EQ_PARA_GAIN_ARG ESP_GMF_AUDIO_GAIN_ARG
/**
* @brief Audio general config parameter argument definition
*/
#define ESP_GMF_AUDIO_CFG_TYPE_ARG ESP_GMF_AUDIO_TYPE_ARG
#define ESP_GMF_AUDIO_CFG_SUBCFGPTR_ARG "sub_cfg"
#define ESP_GMF_AUDIO_CFG_SUBCFGSZ_ARG "sub_cfg_sz"
/**
* @brief Audio sound information parameter argument definition
*/
#define ESP_GMF_AUDIO_SND_INFO_ARG ESP_GMF_AUDIO_INFO_ARG
#define ESP_GMF_AUDIO_SND_INFO_TYPE_ARG ESP_GMF_AUDIO_TYPE_ARG
#define ESP_GMF_AUDIO_SND_INFO_SAMPLERATE_ARG ESP_GMF_AUDIO_SAMPLE_RATE_ARG
#define ESP_GMF_AUDIO_SND_INFO_BITRATE_ARG ESP_GMF_AUDIO_BITRATE_ARG
#define ESP_GMF_AUDIO_SND_INFO_CHANNEL_ARG ESP_GMF_AUDIO_CHANNEL_ARG
#define ESP_GMF_AUDIO_SND_INFO_BITS_ARG ESP_GMF_AUDIO_BITS_ARG
/**
* @brief GMF audio method definition
*/
#define AMETHOD_DEF(module, method, str) ESP_GMF_METHOD_DEF(AUDIO, module, method, str)
/**
* @brief Get GMF audio method string
*/
#define AMETHOD(module, method) ESP_GMF_METHOD_STR(AUDIO, module, method)
/**
* @brief GMF audio method argument definition
*/
#define AMETHOD_ARG_DEF(module, method, arg, str) ESP_GMF_METHOD_ARG_DEF(AUDIO, module, method, arg, str)
/**
* @brief Get GMF audio method argument string
*/
#define AMETHOD_ARG(module, method, arg) ESP_GMF_METHOD_ARG_STR(AUDIO, module, method, arg)
/**
* @brief ALC method
*/
AMETHOD_DEF(ALC, SET_GAIN, "set_gain");
AMETHOD_ARG_DEF(ALC, SET_GAIN, IDX, ESP_GMF_AUDIO_INDEX_ARG);
AMETHOD_ARG_DEF(ALC, SET_GAIN, GAIN, ESP_GMF_AUDIO_GAIN_ARG);
AMETHOD_DEF(ALC, GET_GAIN, "get_gain");
AMETHOD_ARG_DEF(ALC, GET_GAIN, IDX, ESP_GMF_AUDIO_INDEX_ARG);
AMETHOD_ARG_DEF(ALC, GET_GAIN, GAIN, ESP_GMF_AUDIO_GAIN_ARG);
/**
* @brief BIT CVT method
*/
AMETHOD_DEF(BIT_CVT, SET_DEST_BITS, "set_dest_bits");
AMETHOD_ARG_DEF(BIT_CVT, SET_DEST_BITS, BITS, ESP_GMF_AUDIO_BITS_ARG);
/**
* @brief CH CVT method
*/
AMETHOD_DEF(CH_CVT, SET_DEST_CH, "set_dest_ch");
AMETHOD_ARG_DEF(CH_CVT, SET_DEST_CH, CH, ESP_GMF_AUDIO_CHANNEL_ARG);
/**
* @brief RATE CVT method
*/
AMETHOD_DEF(RATE_CVT, SET_DEST_RATE, "set_dest_rate");
AMETHOD_ARG_DEF(RATE_CVT, SET_DEST_RATE, RATE, ESP_GMF_AUDIO_SAMPLE_RATE_ARG);
/**
* @brief EQ method
*/
AMETHOD_DEF(EQ, SET_PARA, "set_para");
AMETHOD_ARG_DEF(EQ, SET_PARA, IDX, ESP_GMF_AUDIO_INDEX_ARG);
AMETHOD_ARG_DEF(EQ, SET_PARA, PARA, ESP_GMF_AUDIO_EQ_PARA_ARG);
AMETHOD_ARG_DEF(EQ, SET_PARA, PARA_FT, ESP_GMF_AUDIO_EQ_PARA_FT_ARG);
AMETHOD_ARG_DEF(EQ, SET_PARA, PARA_FC, ESP_GMF_AUDIO_EQ_PARA_FC_ARG);
AMETHOD_ARG_DEF(EQ, SET_PARA, PARA_Q, ESP_GMF_AUDIO_EQ_PARA_Q_ARG);
AMETHOD_ARG_DEF(EQ, SET_PARA, PARA_GAIN, ESP_GMF_AUDIO_EQ_PARA_GAIN_ARG);
AMETHOD_DEF(EQ, GET_PARA, "get_para");
AMETHOD_ARG_DEF(EQ, GET_PARA, IDX, ESP_GMF_AUDIO_INDEX_ARG);
AMETHOD_ARG_DEF(EQ, GET_PARA, PARA, ESP_GMF_AUDIO_EQ_PARA_ARG);
AMETHOD_ARG_DEF(EQ, GET_PARA, PARA_FT, ESP_GMF_AUDIO_EQ_PARA_FT_ARG);
AMETHOD_ARG_DEF(EQ, GET_PARA, PARA_FC, ESP_GMF_AUDIO_EQ_PARA_FC_ARG);
AMETHOD_ARG_DEF(EQ, GET_PARA, PARA_Q, ESP_GMF_AUDIO_EQ_PARA_Q_ARG);
AMETHOD_ARG_DEF(EQ, GET_PARA, PARA_GAIN, ESP_GMF_AUDIO_EQ_PARA_GAIN_ARG);
AMETHOD_DEF(EQ, ENABLE_FILTER, "enable_filter");
AMETHOD_ARG_DEF(EQ, ENABLE_FILTER, IDX, ESP_GMF_AUDIO_INDEX_ARG);
AMETHOD_ARG_DEF(EQ, ENABLE_FILTER, ENABLE, ESP_GMF_AUDIO_ENABLE_ARG);
/**
* @brief FADE method
*/
AMETHOD_DEF(FADE, SET_MODE, "set_mode");
AMETHOD_ARG_DEF(FADE, SET_MODE, MODE, ESP_GMF_AUDIO_MODE_ARG);
AMETHOD_DEF(FADE, GET_MODE, "get_mode");
AMETHOD_ARG_DEF(FADE, GET_MODE, MODE, ESP_GMF_AUDIO_MODE_ARG);
AMETHOD_DEF(FADE, RESET_WEIGHT, "reset_weight");
/**
* @brief MIXER method
*/
AMETHOD_DEF(MIXER, SET_MODE, "set_mode");
AMETHOD_ARG_DEF(MIXER, SET_MODE, IDX, ESP_GMF_AUDIO_INDEX_ARG);
AMETHOD_ARG_DEF(MIXER, SET_MODE, MODE, ESP_GMF_AUDIO_MODE_ARG);
AMETHOD_DEF(MIXER, SET_INFO, "set_info");
AMETHOD_ARG_DEF(MIXER, SET_INFO, RATE, ESP_GMF_AUDIO_SAMPLE_RATE_ARG);
AMETHOD_ARG_DEF(MIXER, SET_INFO, CH, ESP_GMF_AUDIO_CHANNEL_ARG);
AMETHOD_ARG_DEF(MIXER, SET_INFO, BITS, ESP_GMF_AUDIO_BITS_ARG);
/**
* @brief SONIC method
*/
AMETHOD_DEF(SONIC, SET_SPEED, "set_speed");
AMETHOD_ARG_DEF(SONIC, SET_SPEED, SPEED, ESP_GMF_AUDIO_SPEED_ARG);
AMETHOD_DEF(SONIC, GET_SPEED, "get_speed");
AMETHOD_ARG_DEF(SONIC, GET_SPEED, SPEED, ESP_GMF_AUDIO_SPEED_ARG);
AMETHOD_DEF(SONIC, SET_PITCH, "set_pitch");
AMETHOD_ARG_DEF(SONIC, SET_PITCH, PITCH, ESP_GMF_AUDIO_PITCH_ARG);
AMETHOD_DEF(SONIC, GET_PITCH, "get_pitch");
AMETHOD_ARG_DEF(SONIC, GET_PITCH, PITCH, ESP_GMF_AUDIO_PITCH_ARG);
/**
* @brief ENCODER method
*/
AMETHOD_DEF(ENCODER, GET_FRAME_SZ, "get_frame_sz");
AMETHOD_ARG_DEF(ENCODER, GET_FRAME_SZ, INSIZE, ESP_GMF_AUDIO_INSIZE_ARG);
AMETHOD_ARG_DEF(ENCODER, GET_FRAME_SZ, OUTSIZE, ESP_GMF_AUDIO_OUTSIZE_ARG);
AMETHOD_DEF(ENCODER, SET_BITRATE, "set_bitrate");
AMETHOD_ARG_DEF(ENCODER, SET_BITRATE, BITRATE, ESP_GMF_AUDIO_BITRATE_ARG);
AMETHOD_DEF(ENCODER, GET_BITRATE, "get_bitrate");
AMETHOD_ARG_DEF(ENCODER, GET_BITRATE, BITRATE, ESP_GMF_AUDIO_BITRATE_ARG);
AMETHOD_DEF(ENCODER, RECONFIG, "recfg");
AMETHOD_ARG_DEF(ENCODER, RECONFIG, CFG, ESP_GMF_AUDIO_CFG_ARG);
AMETHOD_ARG_DEF(ENCODER, RECONFIG, CFG_TYPE, ESP_GMF_AUDIO_CFG_TYPE_ARG);
AMETHOD_ARG_DEF(ENCODER, RECONFIG, CFG_SUBCFGPTR, ESP_GMF_AUDIO_CFG_SUBCFGPTR_ARG);
AMETHOD_ARG_DEF(ENCODER, RECONFIG, CFG_SUBCFGSZ, ESP_GMF_AUDIO_CFG_SUBCFGSZ_ARG);
AMETHOD_DEF(ENCODER, RECONFIG_BY_SND_INFO, "recfg_by_sndinfo");
AMETHOD_ARG_DEF(ENCODER, RECONFIG_BY_SND_INFO, INFO, ESP_GMF_AUDIO_SND_INFO_ARG);
AMETHOD_ARG_DEF(ENCODER, RECONFIG_BY_SND_INFO, INFO_TYPE, ESP_GMF_AUDIO_SND_INFO_TYPE_ARG);
AMETHOD_ARG_DEF(ENCODER, RECONFIG_BY_SND_INFO, INFO_SAMPLERATE, ESP_GMF_AUDIO_SND_INFO_SAMPLERATE_ARG);
AMETHOD_ARG_DEF(ENCODER, RECONFIG_BY_SND_INFO, INFO_BITRATE, ESP_GMF_AUDIO_SND_INFO_BITRATE_ARG);
AMETHOD_ARG_DEF(ENCODER, RECONFIG_BY_SND_INFO, INFO_CHANNEL, ESP_GMF_AUDIO_SND_INFO_CHANNEL_ARG);
AMETHOD_ARG_DEF(ENCODER, RECONFIG_BY_SND_INFO, INFO_BITS, ESP_GMF_AUDIO_SND_INFO_BITS_ARG);
/**
* @brief DECODER method
*/
AMETHOD_DEF(DECODER, RECONFIG, "recfg");
AMETHOD_ARG_DEF(DECODER, RECONFIG, CFG, ESP_GMF_AUDIO_CFG_ARG);
AMETHOD_ARG_DEF(DECODER, RECONFIG, CFG_TYPE, ESP_GMF_AUDIO_CFG_TYPE_ARG);
AMETHOD_ARG_DEF(DECODER, RECONFIG, CFG_SUBCFGPTR, ESP_GMF_AUDIO_CFG_SUBCFGPTR_ARG);
AMETHOD_ARG_DEF(DECODER, RECONFIG, CFG_SUBCFGSZ, ESP_GMF_AUDIO_CFG_SUBCFGSZ_ARG);
AMETHOD_DEF(DECODER, RECONFIG_BY_SND_INFO, "recfg_by_sndinfo");
AMETHOD_ARG_DEF(DECODER, RECONFIG_BY_SND_INFO, INFO, ESP_GMF_AUDIO_SND_INFO_ARG);
AMETHOD_ARG_DEF(DECODER, RECONFIG_BY_SND_INFO, INFO_TYPE, ESP_GMF_AUDIO_SND_INFO_TYPE_ARG);
AMETHOD_ARG_DEF(DECODER, RECONFIG_BY_SND_INFO, INFO_SAMPLERATE, ESP_GMF_AUDIO_SND_INFO_SAMPLERATE_ARG);
AMETHOD_ARG_DEF(DECODER, RECONFIG_BY_SND_INFO, INFO_CHANNEL, ESP_GMF_AUDIO_SND_INFO_CHANNEL_ARG);
AMETHOD_ARG_DEF(DECODER, RECONFIG_BY_SND_INFO, INFO_BITS, ESP_GMF_AUDIO_SND_INFO_BITS_ARG);
#ifdef __cplusplus
}
#endif /* __cplusplus */
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_audio/include/esp_gmf_audio_param.h
+128
View File
@@ -0,0 +1,128 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#pragma once
#include "esp_gmf_element.h"
#ifdef __cplusplus
extern "C" {
#endif
/**
* @brief Set frame destination sample rate for audio element
*
* @param[in] handle Audio element handle
* @param[in] dest_rate Destination sample rate
*
* @return
* - ESP_GMF_ERR_OK On success
* - ESP_GMF_ERR_NOT_FOUND Not found the method
* - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory
* - Others Failed to apply method
*/
esp_gmf_err_t esp_gmf_audio_param_set_dest_rate(esp_gmf_element_handle_t self, uint32_t dest_rate);
/**
* @brief Set frame destination bits for audio element
*
* @param[in] handle Audio element handle
* @param[in] dest_bits Destination bits
*
* @return
* - ESP_GMF_ERR_OK On success
* - ESP_GMF_ERR_NOT_FOUND Not found the method
* - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory
* - Others Failed to apply method
*/
esp_gmf_err_t esp_gmf_audio_param_set_dest_bits(esp_gmf_element_handle_t self, uint8_t dest_bits);
/**
* @brief Set frame destination channel for audio element
*
* @param[in] handle Audio element handle
* @param[in] dest_ch Destination channel
*
* @return
* - ESP_GMF_ERR_OK On success
* - ESP_GMF_ERR_NOT_FOUND Not found the method
* - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory
* - Others Failed to apply method
*/
esp_gmf_err_t esp_gmf_audio_param_set_dest_ch(esp_gmf_element_handle_t self, uint8_t dest_ch);
/**
* @brief Set speed for audio element
*
* @param[in] handle Audio element handle
* @param[in] speed Audio playback speed
*
* @return
* - ESP_GMF_ERR_OK On success
* - ESP_GMF_ERR_NOT_FOUND Not found the method
* - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory
* - Others Failed to apply method
*/
esp_gmf_err_t esp_gmf_audio_param_set_speed(esp_gmf_element_handle_t self, float speed);
/**
* @brief Set pitch for audio element
*
* @param[in] handle Audio element handle
* @param[in] pitch Audio pitch to set
*
* @return
* - ESP_GMF_ERR_OK On success
* - ESP_GMF_ERR_NOT_FOUND Not found the method
* - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory
* - Others Failed to apply method
*/
esp_gmf_err_t esp_gmf_audio_param_set_pitch(esp_gmf_element_handle_t self, float pitch);
/**
* @brief Set ALC certain channel gain for audio element
*
* @param[in] handle Audio element handle
* @param[in] ch_idx Channel index (0: Left channel 1: Right Channel 0xFF: All channels)
* @param[in] gain_db ALC gain (unit decibel)
*
* @return
* - ESP_GMF_ERR_OK On success
* - ESP_GMF_ERR_NOT_FOUND Not found the method
* - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory
* - Others Failed to apply method
*/
esp_gmf_err_t esp_gmf_audio_param_set_alc_channel_gain(esp_gmf_element_handle_t self, uint8_t ch_idx, float gain_db);
/**
* @brief Set ALC gain for all channels
*/
#define esp_gmf_audio_param_set_alc_gain(self, gain_db) esp_gmf_audio_param_set_alc_channel_gain(self, 0xFF, gain_db)
/**
* @brief Set fade in/out direction for audio element
*
* @param[in] handle Audio element handle
* @param[in] is_fade_in Whether fade in or out
*
* @return
* - ESP_GMF_ERR_OK On success
* - ESP_GMF_ERR_NOT_FOUND Not found the method
* - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory
* - Others Failed to apply method
*/
esp_gmf_err_t esp_gmf_audio_param_set_fade(esp_gmf_element_handle_t self, bool is_fade_in);
/**
* @brief Helper wrapper for fade in and fade out setting
*/
#define esp_gmf_audio_param_set_fade_in(self) esp_gmf_audio_param_set_fade(self, true);
#define esp_gmf_audio_param_set_fade_out(self) esp_gmf_audio_param_set_fade(self, false);
#ifdef __cplusplus
}
#endif
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_audio/include/esp_gmf_bit_cvt.h
+53
View File
@@ -0,0 +1,53 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#pragma once
#include "esp_gmf_err.h"
#include "esp_ae_bit_cvt.h"
#include "esp_gmf_element.h"
#ifdef __cplusplus
extern "C" {
#endif /* __cplusplus */
#define DEFAULT_ESP_GMF_BIT_CVT_CONFIG() { \
.sample_rate = 48000, \
.channel = 2, \
.src_bits = 16, \
.dest_bits = 16, \
}
/**
* @brief Initializes the GMF bit conversion with the provided configuration
*
* @param[in] config Pointer to the bit conversion configuration
* @param[out] handle Pointer to the bit conversion handle to be initialized
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_INVALID_ARG Invalid configuration provided
* - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory
*/
esp_gmf_err_t esp_gmf_bit_cvt_init(esp_ae_bit_cvt_cfg_t *config, esp_gmf_element_handle_t *handle);
/**
* @brief Set destination bits for the bit conversion handle
*
* @param[in] handle The channel bit handle
* @param[in] dest_bits The destination bits
*
* @return
* - ESP_GMF_ERR_OK Operation succeeded
* - ESP_GMF_ERR_INVALID_ARG Invalid input parameter
* - ESP_GMF_ERR_FAIL Failed to set configuration
*/
esp_gmf_err_t esp_gmf_bit_cvt_set_dest_bits(esp_gmf_element_handle_t handle, uint8_t dest_bits);
#ifdef __cplusplus
}
#endif /* __cplusplus */
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_audio/include/esp_gmf_ch_cvt.h
+55
View File
@@ -0,0 +1,55 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#pragma once
#include "esp_gmf_err.h"
#include "esp_ae_ch_cvt.h"
#include "esp_gmf_element.h"
#ifdef __cplusplus
extern "C" {
#endif /* __cplusplus */
#define DEFAULT_ESP_GMF_CH_CVT_CONFIG() { \
.sample_rate = 48000, \
.bits_per_sample = 16, \
.src_ch = 2, \
.dest_ch = 2, \
.weight = NULL, \
.weight_len = 0, \
}
/**
* @brief Initializes the GMF channel conversion with the provided configuration
*
* @param[in] config Pointer to the channel conversion configuration
* @param[out] handle Pointer to the channel conversion handle to be initialized
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_INVALID_ARG Invalid configuration provided
* - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory
*/
esp_gmf_err_t esp_gmf_ch_cvt_init(esp_ae_ch_cvt_cfg_t *config, esp_gmf_element_handle_t *handle);
/**
* @brief Set dest channel in the channel conversion handle
*
* @param[in] handle The channel conversion handle
* @param[in] dest_ch The dest channel
*
* @return
* - ESP_GMF_ERR_OK Operation succeeded
* - ESP_GMF_ERR_INVALID_ARG Invalid input parameter
* - ESP_GMF_ERR_FAIL Failed to set configuration
*/
esp_gmf_err_t esp_gmf_ch_cvt_set_dest_channel(esp_gmf_element_handle_t handle, uint8_t dest_ch);
#ifdef __cplusplus
}
#endif /* __cplusplus */
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_audio/include/esp_gmf_deinterleave.h
+47
View File
@@ -0,0 +1,47 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#pragma once
#include "esp_gmf_err.h"
#include "esp_gmf_element.h"
#ifdef __cplusplus
extern "C" {
#endif /* __cplusplus */
#define DEFAULT_ESP_GMF_DEINTERLEAVE_CONFIG() { \
.sample_rate = 48000, \
.bits_per_sample = 16, \
.channel = 2, \
}
/**
* @brief Configuration structure for deinterleave
*/
typedef struct {
uint32_t sample_rate; /*!< The audio sample rate */
uint8_t bits_per_sample; /*!< The audio bits per sample, supports 16, 24, 32 bits */
uint8_t channel; /*!< The audio channel */
} esp_gmf_deinterleave_cfg;
/**
* @brief Initializes the GMF deinterleave with the provided configuration
*
* @param[in] config Pointer to the deinterleave configuration
* @param[out] handle Pointer to the deinterleave handle to be initialized
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_INVALID_ARG Invalid configuration provided
* - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory
*/
esp_gmf_err_t esp_gmf_deinterleave_init(esp_gmf_deinterleave_cfg *config, esp_gmf_element_handle_t *handle);
#ifdef __cplusplus
}
#endif /* __cplusplus */
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_audio/include/esp_gmf_eq.h
+87
View File
@@ -0,0 +1,87 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#pragma once
#include "esp_gmf_err.h"
#include "esp_ae_eq.h"
#include "esp_gmf_element.h"
#ifdef __cplusplus
extern "C" {
#endif /* __cplusplus */
/**
* @brief Default EQ parameters
* If `para` is NULL in the `esp_ae_eq_cfg_t` , the EQ will use `esp_gmf_default_eq_paras` as default
* And the `filter_num` will be set to sizeof(esp_gmf_default_eq_paras) / sizeof(esp_ae_eq_filter_para_t)
*/
#define DEFAULT_ESP_GMF_EQ_CONFIG() { \
.sample_rate = 48000, \
.channel = 2, \
.bits_per_sample = 16, \
.filter_num = 0, \
.para = NULL, \
}
/**
* @brief Initializes the GMF EQ with the provided configuration
*
* @param[in] config Pointer to the EQ configuration
* @param[out] handle Pointer to the EQ handle to be initialized
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_INVALID_ARG Invalid configuration provided
* - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory
*/
esp_gmf_err_t esp_gmf_eq_init(esp_ae_eq_cfg_t *config, esp_gmf_element_handle_t *handle);
/**
* @brief Set the filter parameters for a specific filter identified by 'idx'
*
* @param[in] handle The EQ handle
* @param[in] idx The index of a specific filter for which the parameters are to be set.
* eg: 0 refers to the first filter
* @param[in] para The filter setup parameter
*
* @return
* - ESP_GMF_ERR_OK Operation succeeded
* - ESP_GMF_ERR_INVALID_ARG Invalid input parameter
*/
esp_gmf_err_t esp_gmf_eq_set_para(esp_gmf_element_handle_t handle, uint8_t idx, esp_ae_eq_filter_para_t *para);
/**
* @brief Get the filter parameters for a specific filter identified by 'idx'
*
* @param[in] handle The EQ handle
* @param[in] idx The index of a specific filter for which the parameters are to be retrieved.
* eg: 0 refers to first filter
* @param[out] para The filter setup parameter
*
* @return
* - ESP_GMF_ERR_OK Operation succeeded
* - ESP_GMF_ERR_INVALID_ARG Invalid input parameter
*/
esp_gmf_err_t esp_gmf_eq_get_para(esp_gmf_element_handle_t handle, uint8_t idx, esp_ae_eq_filter_para_t *para);
/**
* @brief Choose to enable or disable filter processing for a specific filter identified by 'idx' in the equalizer
*
* @param[in] handle The EQ handle
* @param[in] idx The index of a specific filter to be enabled
* @param[in] is_enable The flag of whether to enable band filter processing
*
* @return
* - ESP_GMF_ERR_OK Operation succeeded
* - ESP_GMF_ERR_INVALID_ARG Invalid input parameter
*/
esp_gmf_err_t esp_gmf_eq_enable_filter(esp_gmf_element_handle_t handle, uint8_t idx, bool is_enable);
#ifdef __cplusplus
}
#endif /* __cplusplus */
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_audio/include/esp_gmf_fade.h
+81
View File
@@ -0,0 +1,81 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#pragma once
#include "esp_gmf_err.h"
#include "esp_ae_fade.h"
#include "esp_gmf_element.h"
#ifdef __cplusplus
extern "C" {
#endif /* __cplusplus */
#define DEFAULT_ESP_GMF_FADE_CONFIG() { \
.mode = ESP_AE_FADE_MODE_FADE_IN, \
.curve = ESP_AE_FADE_CURVE_LINE, \
.transit_time = 500, \
.sample_rate = 48000, \
.channel = 2, \
.bits_per_sample = 16, \
}
/**
* @brief Initializes the GMF fade with the provided configuration
*
* @param[in] config Pointer to the fade configuration
* @param[out] handle Pointer to the fade handle to be initialized
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_INVALID_ARG Invalid configuration provided
* - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory
*/
esp_gmf_err_t esp_gmf_fade_init(esp_ae_fade_cfg_t *config, esp_gmf_element_handle_t *handle);
/**
* @brief Set the fade process mode
*
* @param[in] handle The fade handle
* @param[in] mode The mode of fade
*
* @return
* - ESP_GMF_ERR_OK Operation succeeded
* - ESP_GMF_ERR_INVALID_ARG Invalid input parameter
*/
esp_gmf_err_t esp_gmf_fade_set_mode(esp_gmf_element_handle_t handle, esp_ae_fade_mode_t mode);
/**
* @brief Get the fade process mode
*
* @param[in] handle The fade handle
* @param[out] mode The mode of fade
*
* @return
* - ESP_GMF_ERR_OK Operation succeeded
* - ESP_GMF_ERR_INVALID_ARG Invalid input parameter
*/
esp_gmf_err_t esp_gmf_fade_get_mode(esp_gmf_element_handle_t handle, esp_ae_fade_mode_t *mode);
/**
* @brief Reset the fade process to the initial configuration state.
* If the initial configuration mode is fade in, the current weight is set to 0.
* If the initial configuration mode is fade out, the current weight is set to 1.
* This function helps the user to restart fade in or fade out without
* reopening and closing.
*
* @param[in] handle The fade handle
*
* @return
* - ESP_GMF_ERR_OK Operation succeeded
* - ESP_GMF_ERR_INVALID_ARG Invalid input parameter
*/
esp_gmf_err_t esp_gmf_fade_reset_weight(esp_gmf_element_handle_t handle);
#ifdef __cplusplus
}
#endif /* __cplusplus */
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_audio/include/esp_gmf_interleave.h
+47
View File
@@ -0,0 +1,47 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#pragma once
#include "esp_gmf_err.h"
#include "esp_gmf_element.h"
#ifdef __cplusplus
extern "C" {
#endif /* __cplusplus */
#define DEFAULT_ESP_GMF_INTERLEAVE_CONFIG() { \
.sample_rate = 48000, \
.bits_per_sample = 16, \
.src_num = 2, \
}
/**
* @brief Configuration structure for interleave
*/
typedef struct {
uint32_t sample_rate; /*!< The audio sample rate */
uint8_t bits_per_sample; /*!< The audio bits per sample, supports 16, 24, 32 bits */
uint8_t src_num; /*!< The number of input source num */
} esp_gmf_interleave_cfg;
/**
* @brief Initializes the GMF interleave with the provided configuration
*
* @param[in] config Pointer to the interleave configuration
* @param[out] handle Pointer to the interleave handle to be initialized
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_INVALID_ARG Invalid configuration provided
* - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory
*/
esp_gmf_err_t esp_gmf_interleave_init(esp_gmf_interleave_cfg *config, esp_gmf_element_handle_t *handle);
#ifdef __cplusplus
}
#endif /* __cplusplus */
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_audio/include/esp_gmf_mixer.h
+76
View File
@@ -0,0 +1,76 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#pragma once
#include "esp_gmf_err.h"
#include "esp_ae_mixer.h"
#include "esp_gmf_element.h"
#ifdef __cplusplus
extern "C" {
#endif /* __cplusplus */
/**
* @brief Default mixer source info
* If `src_info` is NULL in `esp_ae_mixer_cfg_t`, the mixer will use esp_gmf_default_mixer_src_info as default
* And `src_num` will set to `sizeof(esp_gmf_default_mixer_src_info) / sizeof(esp_ae_mixer_info_t)`;
*/
#define DEFAULT_ESP_GMF_MIXER_CONFIG() { \
.sample_rate = 48000, \
.channel = 2, \
.bits_per_sample = 16, \
.src_num = 0, \
.src_info = NULL, \
}
/**
* @brief Initializes the GMF mixer with the provided configuration
*
* @param[in] config Pointer to the mixer configuration
* @param[out] handle Pointer to the mixer handle to be initialized
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_INVALID_ARG Invalid configuration provided
* - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory
*/
esp_gmf_err_t esp_gmf_mixer_init(esp_ae_mixer_cfg_t *config, esp_gmf_element_handle_t *handle);
/**
* @brief Set the transit mode of a certain stream according to src_idx
*
* @param[in] handle The mixer handle
* @param[in] src_idx The index of a certain source stream which want to set transit mode.
* eg: 0 refer to first source stream
* @param[in] mode The transit mode of source stream
*
* @return
* - ESP_GMF_ERR_OK Operation succeeded
* - ESP_GMF_ERR_INVALID_ARG Invalid input parameter
*/
esp_gmf_err_t esp_gmf_mixer_set_mode(esp_gmf_element_handle_t handle, uint8_t src_idx, esp_ae_mixer_mode_t mode);
/**
* @brief Set audio information to the mixer handle
*
* @param[in] handle The mixer handle
* @param[in] sample_rate The audio sample rate
* @param[in] bits The audio bits per sample
* @param[in] channel The audio channel
*
* @return
* - ESP_GMF_ERR_OK Operation succeeded
* - ESP_GMF_ERR_INVALID_ARG Invalid input parameter
* - ESP_GMF_ERR_FAIL Failed to set configuration
*/
esp_gmf_err_t esp_gmf_mixer_set_audio_info(esp_gmf_element_handle_t handle, uint32_t sample_rate,
uint8_t bits, uint8_t channel);
#ifdef __cplusplus
}
#endif /* __cplusplus */
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_audio/include/esp_gmf_rate_cvt.h
+55
View File
@@ -0,0 +1,55 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#pragma once
#include "esp_gmf_err.h"
#include "esp_ae_rate_cvt.h"
#include "esp_gmf_element.h"
#ifdef __cplusplus
extern "C" {
#endif /* __cplusplus */
#define DEFAULT_ESP_GMF_RATE_CVT_CONFIG() { \
.src_rate = 44100, \
.dest_rate = 48000, \
.channel = 2, \
.bits_per_sample = 16, \
.complexity = 2, \
.perf_type = ESP_AE_RATE_CVT_PERF_TYPE_SPEED, \
}
/**
* @brief Initializes the GMF rate conversion with the provided configuration
*
* @param[in] config Pointer to the rate conversion configuration
* @param[out] handle Pointer to the rate conversion handle to be initialized
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_INVALID_ARG Invalid configuration provided
* - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory
*/
esp_gmf_err_t esp_gmf_rate_cvt_init(esp_ae_rate_cvt_cfg_t *config, esp_gmf_element_handle_t *handle);
/**
* @brief Set dest rate in the rate conversion handle
*
* @param[in] handle The rate conversion handle
* @param[in] dest_ch The dest rate
*
* @return
* - ESP_GMF_ERR_OK Operation succeeded
* - ESP_GMF_ERR_INVALID_ARG Invalid input parameter
* - ESP_GMF_ERR_FAIL Failed to set configuration
*/
esp_gmf_err_t esp_gmf_rate_cvt_set_dest_rate(esp_gmf_element_handle_t handle, uint32_t dest_rate);
#ifdef __cplusplus
}
#endif /* __cplusplus */
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_audio/include/esp_gmf_sonic.h
+92
View File
@@ -0,0 +1,92 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#pragma once
#include "esp_gmf_err.h"
#include "esp_ae_sonic.h"
#include "esp_gmf_element.h"
#ifdef __cplusplus
extern "C" {
#endif /* __cplusplus */
#define DEFAULT_ESP_GMF_SONIC_CONFIG() { \
.sample_rate = 48000, \
.channel = 2, \
.bits_per_sample = 16, \
}
/**
* @brief Initializes the GMF sonic with the provided configuration
*
* @param[in] config Pointer to the sonic configuration
* @param[out] handle Pointer to the sonic handle to be initialized
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_ERR_INVALID_ARG Invalid configuration provided
* - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory
*/
esp_gmf_err_t esp_gmf_sonic_init(esp_ae_sonic_cfg_t *config, esp_gmf_element_handle_t *handle);
/**
* @brief Set the audio speed
*
* @param[in] handle The handle of the sonic
* @param[in] speed The scaling factor of audio speed.
* The range of speed is [0.5, 2.0]
*
* @return
* - ESP_GMF_ERR_OK Operation succeeded
* - ESP_GMF_ERR_INVALID_ARG Invalid input parameter
*/
esp_gmf_err_t esp_gmf_sonic_set_speed(esp_gmf_element_handle_t handle, float speed);
/**
* @brief Get the audio speed
*
* @param[in] handle The handle of the sonic
* @param[out] speed The scaling factor of audio speed
*
* @return
* - ESP_GMF_ERR_OK Operation succeeded
* - ESP_GMF_ERR_INVALID_ARG Invalid input parameter
*/
esp_gmf_err_t esp_gmf_sonic_get_speed(esp_gmf_element_handle_t handle, float *speed);
/**
* @brief Set the audio pitch
*
* @param[in] handle The handle of the sonic
* @param[in] pitch The scaling factor of audio pitch.
* The range of pitch is [0.5, 2.0].
* If the pitch value is smaller than 1.0, the sound is deep voice;
* if the pitch value is equal to 1.0, the sound is no change;
* if the pitch value is gather than 1.0, the sound is sharp voice;
*
* @return
* - ESP_GMF_ERR_OK Operation succeeded
* - ESP_GMF_ERR_INVALID_ARG Invalid input parameter
*/
esp_gmf_err_t esp_gmf_sonic_set_pitch(esp_gmf_element_handle_t handle, float pitch);
/**
* @brief Get the audio pitch
*
* @param[in] handle The handle of the sonic
* @param[out] pitch The scaling factor of audio pitch
*
* @return
* - ESP_GMF_ERR_OK Operation succeeded
* - ESP_GMF_ERR_INVALID_ARG Invalid input parameter
*/
esp_gmf_err_t esp_gmf_sonic_get_pitch(esp_gmf_element_handle_t handle, float *pitch);
#ifdef __cplusplus
}
#endif /* __cplusplus */
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_audio/private_include/gmf_audio_common.h
+35
View File
@@ -0,0 +1,35 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#pragma once
#include "esp_err.h"
#include "esp_gmf_info.h"
#include "esp_gmf_audio_element.h"
#ifdef __cplusplus
extern "C" {
#endif /* __cplusplus */
#define GMF_AUDIO_UPDATE_SND_INFO(self, sample_rate, bits, channel) gmf_audio_update_snd_info(self, sample_rate, bits, channel)
#define GMF_AUDIO_INPUT_SAMPLE_NUM (256)
#define GMF_AUDIO_CALC_PTS(out_len, sample_rate, ch, bits) ((out_len) * 8000 / ((sample_rate) * (ch) * (bits)))
static inline void gmf_audio_update_snd_info(void *self, uint32_t sample_rate, uint8_t bits, uint8_t channel)
{
esp_gmf_info_sound_t snd_info = {0};
esp_gmf_audio_el_get_snd_info(self, &snd_info);
snd_info.sample_rates = sample_rate;
snd_info.channels = channel;
snd_info.bits = bits;
esp_gmf_audio_el_set_snd_info(self, &snd_info);
esp_gmf_element_notify_snd_info(self, &snd_info);
}
#ifdef __cplusplus
}
#endif /* __cplusplus */
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_io/CHANGELOG.md
+97
View File
@@ -0,0 +1,97 @@
# Changelog
## v0.7.6
### Features
- Support C++ build
## v0.7.5
### Bug Fixes
- Fixed the issue where idf_build_set_property is not a valid function for component
## v0.7.4
### Bug Fixes
- Fixed data bus init when direction is write in `io_http`
## v0.7.3
### Bug Fixes
- Fixed cache caps issues for esp32 in `io_file`
## v0.7.2
### Features
- Added io_file cache size configuration to enhance read and write performance
### Bug Fixes
- Delay task creation to open stage in `io_http` to avoid task resource waste if no used
## v0.7.1
- Updated `esp-codec-dev` dependency to v1.4.0
## v0.7.0
### Features
- Added `esp_gmf_io_codec_dev_set_dev` API to set the audio codec device handle
- Added `esp_gmf_io_http_set_event_callback` API to set an HTTP event callback
- Added basic reset functionality to each IO to properly clean up resources and reset state
### Bug Fixes
- Fixed an issue where HTTP IO did not reset the thread and databus properly
- Fixed file open check for treat return error as valid fd
- Corrected return value validation for *acq_write/read and *acq_release_write/read callback function implementations
- Fixed HTTP connection timeout handling during _http_close operations
- Standardize TAG identifier format across all I/O elements with `io` prefix
## v0.6.3
### Bug Fixes
- Changed the tag for embedded flash IO from `embed_flash` to `embed`
- Refactor the script to make the generated files follow our coding style
## v0.6.2
### Features
- Limit the version of `esp_codec_dev`
## v0.6.1
- Fixed memory leaks in I/O flash destroy API
- Corrected default values for I/O file and I/O I2S elements to ensure stable initialization
- Enforced esp_gmf_err_t return type for all element initialization functions (I/O flash, HTTP)
## v0.6.0
### Features
- Added support for NULL object configuration
- Renamed component to `gmf_io`
- Updated the License
### Bug Fixes
- Standardized return values of port and data bus acquire/release APIs to esp_gmf_err_io_t only
## v0.5.1
### Bug Fixes
- Fixed the component requirements
## v0.5.0
### Features
- Initial version of `esp-gmf-io`
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_io/CMakeLists.txt
+23
View File
@@ -0,0 +1,23 @@
idf_build_get_property(build_components BUILD_COMPONENTS)
set(search_pattern "esp_codec_dev")
set(index -1)
foreach(item IN LISTS build_components)
if("${item}" MATCHES "${search_pattern}.*")
set(index ${item})
break()
endif()
endforeach()
list(APPEND io_srcs "esp_gmf_io_file.c" "esp_gmf_io_embed_flash.c" "esp_gmf_io_i2s_pdm.c" "esp_gmf_io_http.c" "http_lib/gzip/gzip_miniz.c")
if(index EQUAL -1)
set(io_inc "")
else()
message(STATUS "The esp_codec_dev is installed")
list(APPEND io_srcs "codec_dev_io/esp_gmf_io_codec_dev.c")
set(io_inc "codec_dev_io")
endif()
idf_component_register(SRCS ${io_srcs}
INCLUDE_DIRS ./include ${io_inc} "http_lib/gzip/" "http_lib/gzip/include"
PRIV_INCLUDE_DIRS ""
REQUIRES "driver" "esp_http_client")
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_io/LICENSE
+22
View File
@@ -0,0 +1,22 @@
Espressif Modified MIT License
Copyright (c) 2025 Espressif Systems (Shanghai) CO., LTD
Permission is hereby granted for use EXCLUSIVELY with Espressif Systems products.
This includes the right to use, copy, modify, merge, publish, distribute, and sublicense
the Software, subject to the following conditions:
1. This Software MUST BE USED IN CONJUNCTION WITH ESPRESSIF SYSTEMS PRODUCTS.
2. The above copyright notice and this permission notice shall be included in all copies
or substantial portions of the Software.
3. Redistribution of the Software in source or binary form FOR USE WITH NON-ESPRESSIF PRODUCTS
is strictly prohibited.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR
PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE
FOR ANY CLAIM, DAMAGES, OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.
SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_io/README.md
+36
View File
@@ -0,0 +1,36 @@
# ESP-GMF-IO
- [![Component Registry](https://components.espressif.com/components/espressif/gmf_io/badge.svg)](https://components.espressif.com/components/espressif/gmf_io)
- [中文版](./README_CN.md)
ESP GMF IO is a collection of GMF input and output streams. IO streams have two data flow directions: reading and writing. The input type is readable, and the output type is writable. IO streams classify data transmission types into byte access and block access types. In general, byte access involves copying the source data before transferring, while block access only transfers the source data address without copy overhead. Additionally, IO streams provide optional thread configuration, allowing some IO streams to establish their own independent threads. The currently supported IO streams are listed in the table below:
| Name | TAG | Data flow direction | Thread | Data Type| Dependent Components | Notes |
| :----: | :----: | :----: | :----: | :----: | :----: |:----: |
| File | io_file | RW | NO | Byte |NA | NA |
| HTTP | io_http | RW | YES | Block | NA | Not support HTTP Live Stream |
| Codec Dev IO | io_codec_dev | RW | NO | Byte | [ESP codec dev](https://components.espressif.com/components/espressif/esp_codec_dev) | NA |
| Embed Flash | io_embed_flash | R | NO | Byte | NA | NA |
| I2S PDM | io_i2s_pdm | RW | NO | Byte | NA | NA |
## Usage
ESP GMF IO is typically used in combination with GMF Elements to form a pipeline, but it can also be used independently. In a pipeline, the IO stream is connected to the GMF Element's Port, and when used independently, the IO stream is accessed through the ESP_GMF_IO interface. For example code, please refer to [test_app](../test_apps/main/elements/gmf_audio_play_el_test.c)。
You can also create and compile a project using the following commands, taking the `pipeline_play_embed_music` project as an example. Before starting, make sure you have a working [ESP-IDF](https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/get-started/index.html) environment.
### 1. Create the Example Project
Create the `pipeline_play_embed_music` example project based on the `gmf_examples` component (using version v0.7.0 as an example; update the version as needed):
```shell
idf.py create-project-from-example "espressif/gmf_examples=0.7.0:pipeline_play_embed_music"
```
### 2. Build and Flash the Project Using an ESP32-S3 Board
```shell
cd pipeline_play_embed_music
idf.py set-target esp32s3
idf.py -p YOUR_PORT flash monitor
```
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_io/README_CN.md
+36
View File
@@ -0,0 +1,36 @@
# ESP-GMF-IO
- [![Component Registry](https://components.espressif.com/components/espressif/gmf_io/badge.svg)](https://components.espressif.com/components/espressif/gmf_io)
- [English](./README.md)
ESP GMF IO 是 GMF 输入和输出流的集合。IO 流有读写两个数据流方向，输入类型为可读，输出类型为可写。IO 流对数据传输类型进行了分类，分别是字节访问和块访问类型，一般来说，字节访问对源数据进行拷贝后传递，块访问仅传递源数据地址无拷贝开销。另外，IO 流还提供了线程的配置可选项，使一些 IO Stream 可以建立自己独立的线程。目前已支持的 IO 流参见下表：
| 名称 | TAG | 数据流方向 | 作为线程 | 数据类型| 依赖的组件 | 备注 |
| :----: | :----: | :----: | :----: | :----: | :----: |:----: |
| File | io_file | RW | NO | Byte |NA | NA |
| HTTP | io_http | RW | YES | Block | NA | Not support HTTP Live Stream |
| Codec Dev IO | io_codec_dev | RW | NO | Byte | [ESP codec dev](https://components.espressif.com/components/espressif/esp_codec_dev) | NA |
| Embed Flash | io_embed_flash | R | NO | Byte | NA | NA |
| I2S PDM | io_i2s_pdm | RW | NO | Byte | NA | NA |
## 示例
ESP GMF IO 一般与 GMF 元素组成管道进行使用，也可以独立使用。在管道中，IO stream 与 GMF 元素的端口连接，独立使用时调用 IO stream 使用 ESP_GMF_IO 接口。 示例代码请参考 [test_app](../test_apps/main/elements/gmf_audio_play_el_test.c)。
此外，也可通过以下命令创建并编译工程，以 `pipeline_play_embed_music` 示例工程为例。开始之前需要有可运行的 [ESP-IDF](https://docs.espressif.com/projects/esp-idf/en/latest/esp32s3/get-started/index.html) 环境。
### 1. 创建示例工程
基于 `gmf_examples` 组件创建 `pipeline_play_embed_music` 的示例（以 v0.7.0 版本为例， 请根据实际使用更新版本参数）
``` shell
idf.py create-project-from-example "espressif/gmf_examples=0.7.0:pipeline_play_embed_music"
```
### 2. 基于 ESP32S3 编译和下载
```shell
cd pipeline_play_embed_music
idf.py set-target esp32s3`
idf.py -p YOUR_PORT flash monitor
```
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_io/codec_dev_io/esp_gmf_io_codec_dev.c
+174
View File
@@ -0,0 +1,174 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#include <stdio.h>
#include <string.h>
#include "esp_gmf_io_codec_dev.h"
#include "esp_gmf_oal_mem.h"
#include "esp_log.h"
/**
* @brief Codec device io context in GMF
*/
typedef struct {
esp_gmf_io_t base; /*!< The GMF codec dev io handle */
bool is_open; /*!< The flag of whether opened */
} codec_dev_io_stream_t;
static const char *TAG = "ESP_GMF_CODEC_DEV";
static esp_gmf_err_t esp_gmf_io_codec_dev_new(void *cfg, esp_gmf_obj_handle_t *io)
{
return esp_gmf_io_codec_dev_init(cfg, io);
}
static esp_gmf_err_t _codec_dev_open(esp_gmf_io_handle_t io)
{
codec_dev_io_stream_t *codec_dev_io = (codec_dev_io_stream_t *)io;
codec_dev_io_cfg_t *cfg = (codec_dev_io_cfg_t *)OBJ_GET_CFG(codec_dev_io);
ESP_GMF_NULL_CHECK(TAG, cfg, return ESP_GMF_ERR_FAIL;);
ESP_GMF_CHECK(TAG, cfg->dev, {return ESP_GMF_ERR_FAIL;}, "There is no activated I2S driver handle");
codec_dev_io->is_open = true;
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_io_t _codec_dev_acquire_read(esp_gmf_io_handle_t handle, void *payload, uint32_t wanted_size, int block_ticks)
{
codec_dev_io_stream_t *codec_dev_io = (codec_dev_io_stream_t *)handle;
esp_gmf_payload_t *pload = (esp_gmf_payload_t *)payload;
codec_dev_io_cfg_t *cfg = (codec_dev_io_cfg_t *)OBJ_GET_CFG(codec_dev_io);
ESP_GMF_NULL_CHECK(TAG, cfg, return ESP_GMF_IO_FAIL;);
if (esp_codec_dev_read(cfg->dev, pload->buf, wanted_size) != ESP_GMF_IO_OK) {
ESP_LOGE(TAG, "Read failed, wanted: %ld", wanted_size);
return ESP_GMF_IO_FAIL;
}
pload->valid_size = wanted_size;
return ESP_GMF_IO_OK;
}
static esp_gmf_err_io_t _codec_dev_release_read(esp_gmf_io_handle_t handle, void *payload, int block_ticks)
{
codec_dev_io_stream_t *codec_dev_io = (codec_dev_io_stream_t *)handle;
esp_gmf_payload_t *pload = (esp_gmf_payload_t *)payload;
esp_gmf_info_file_t info = {0};
esp_gmf_io_get_info((esp_gmf_io_handle_t)codec_dev_io, &info);
ESP_LOGD(TAG, "Update len = %d, pos = %d/%d", pload->valid_size, (int)info.pos, (int)info.size);
esp_gmf_io_update_pos((esp_gmf_io_handle_t)handle, pload->valid_size);
return ESP_GMF_IO_OK;
}
static esp_gmf_err_io_t _codec_dev_acquire_write(esp_gmf_io_handle_t handle, void *payload, uint32_t wanted_size, int block_ticks)
{
return ESP_GMF_IO_OK;
}
static esp_gmf_err_io_t _codec_dev_release_write(esp_gmf_io_handle_t handle, void *payload, int block_ticks)
{
codec_dev_io_stream_t *codec_dev_io = (codec_dev_io_stream_t *)handle;
esp_gmf_payload_t *pload = (esp_gmf_payload_t *)payload;
codec_dev_io_cfg_t *cfg = (codec_dev_io_cfg_t *)OBJ_GET_CFG(codec_dev_io);
ESP_GMF_NULL_CHECK(TAG, cfg, return ESP_GMF_IO_FAIL;);
size_t wlen = pload->valid_size;
if (esp_codec_dev_write(cfg->dev, pload->buf, pload->valid_size) != ESP_GMF_ERR_OK) {
ESP_LOGE(TAG, "Write failed, valid: %d", pload->valid_size);
return ESP_GMF_IO_FAIL;
}
esp_gmf_info_file_t info = {0};
if (wlen > 0) {
esp_gmf_io_update_pos((esp_gmf_io_handle_t)handle, wlen);
}
esp_gmf_io_get_info((esp_gmf_io_handle_t)codec_dev_io, &info);
ESP_LOGD(TAG, "Write len: %d, pos: %d/%d", pload->valid_size, (int)info.pos, (int)info.size);
return ESP_GMF_IO_OK;
}
static esp_gmf_err_t _codec_dev_seek(esp_gmf_io_handle_t io, uint64_t seek_byte_pos)
{
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t _codec_dev_close(esp_gmf_io_handle_t io)
{
codec_dev_io_stream_t *codec_dev_io = (codec_dev_io_stream_t *)io;
esp_gmf_info_file_t info = {0};
esp_gmf_io_get_info((esp_gmf_io_handle_t)codec_dev_io, &info);
ESP_LOGI(TAG, "CLose, %p, pos = %d/%d", codec_dev_io, (int)info.pos, (int)info.size);
if (codec_dev_io->is_open) {
codec_dev_io->is_open = false;
}
esp_gmf_io_set_pos((esp_gmf_io_handle_t)io, 0);
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t _codec_dev_delete(esp_gmf_io_handle_t io)
{
if (io != NULL) {
codec_dev_io_stream_t *codec_dev_io = (codec_dev_io_stream_t *)io;
ESP_LOGD(TAG, "Delete, %s-%p", OBJ_GET_TAG(codec_dev_io), codec_dev_io);
esp_gmf_oal_free(OBJ_GET_CFG(codec_dev_io));
esp_gmf_io_deinit(io);
esp_gmf_oal_free(codec_dev_io);
}
return ESP_GMF_ERR_OK;
}
esp_gmf_err_t esp_gmf_io_codec_dev_init(codec_dev_io_cfg_t *config, esp_gmf_io_handle_t *io)
{
ESP_GMF_NULL_CHECK(TAG, config, {return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_NULL_CHECK(TAG, io, {return ESP_GMF_ERR_INVALID_ARG;});
*io = NULL;
esp_gmf_err_t ret = ESP_GMF_ERR_OK;
codec_dev_io_stream_t *codec_dev_io = esp_gmf_oal_calloc(1, sizeof(codec_dev_io_stream_t));
ESP_GMF_MEM_VERIFY(TAG, codec_dev_io, return ESP_GMF_ERR_MEMORY_LACK,
"codec device", sizeof(codec_dev_io_stream_t));
codec_dev_io->base.dir = config->dir;
codec_dev_io->base.type = ESP_GMF_IO_TYPE_BYTE;
esp_gmf_obj_t *obj = (esp_gmf_obj_t *)codec_dev_io;
obj->new_obj = esp_gmf_io_codec_dev_new;
obj->del_obj = _codec_dev_delete;
codec_dev_io_cfg_t *cfg = esp_gmf_oal_calloc(1, sizeof(*config));
ESP_GMF_MEM_VERIFY(TAG, cfg, {ret = ESP_GMF_ERR_MEMORY_LACK; goto _codec_dev_fail;},
"codec device configuration", sizeof(*config));
memcpy(cfg, config, sizeof(*config));
esp_gmf_obj_set_config(obj, cfg, sizeof(*config));
ret = esp_gmf_obj_set_tag(obj, (config->name == NULL ? "io_codec_dev" : config->name));
ESP_GMF_RET_ON_NOT_OK(TAG, ret, goto _codec_dev_fail, "Failed to set obj tag");
codec_dev_io->base.close = _codec_dev_close;
codec_dev_io->base.open = _codec_dev_open;
codec_dev_io->base.seek = _codec_dev_seek;
codec_dev_io->base.reset = NULL;
esp_gmf_io_init(obj, NULL);
if (codec_dev_io->base.dir == ESP_GMF_IO_DIR_WRITER) {
codec_dev_io->base.acquire_write = _codec_dev_acquire_write;
codec_dev_io->base.release_write = _codec_dev_release_write;
} else if (codec_dev_io->base.dir == ESP_GMF_IO_DIR_READER) {
codec_dev_io->base.acquire_read = _codec_dev_acquire_read;
codec_dev_io->base.release_read = _codec_dev_release_read;
} else {
ESP_LOGE(TAG, "Does not set read or write function");
ret = ESP_GMF_ERR_NOT_SUPPORT;
goto _codec_dev_fail;
}
*io = obj;
ESP_LOGD(TAG, "Initialization, %s-%p", OBJ_GET_TAG(obj), codec_dev_io);
return ESP_GMF_ERR_OK;
_codec_dev_fail:
esp_gmf_obj_delete(obj);
return ret;
}
esp_gmf_err_t esp_gmf_io_codec_dev_set_dev(esp_gmf_io_handle_t io, esp_codec_dev_handle_t dev)
{
codec_dev_io_stream_t *codec_dev_io = (codec_dev_io_stream_t *)io;
ESP_GMF_NULL_CHECK(TAG, codec_dev_io, return ESP_GMF_ERR_INVALID_ARG;);
codec_dev_io_cfg_t *cfg = (codec_dev_io_cfg_t *)OBJ_GET_CFG(codec_dev_io);
ESP_GMF_NULL_CHECK(TAG, cfg, return ESP_GMF_ERR_INVALID_STATE;);
cfg->dev = dev;
return ESP_GMF_ERR_OK;
}
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_io/codec_dev_io/esp_gmf_io_codec_dev.h
+60
View File
@@ -0,0 +1,60 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#pragma once
#include "esp_codec_dev.h"
#include "esp_gmf_io.h"
#ifdef __cplusplus
extern "C" {
#endif /* __cplusplus */
/**
* @brief Audio Codec Device IO configurations, if any entry is zero then the configuration will be set to default values
*/
typedef struct {
esp_codec_dev_handle_t dev; /*!< Audio Codec Device handler */
esp_gmf_io_dir_t dir; /*!< IO direction, reader or writer */
const char *name; /*!< Name for this instance */
} codec_dev_io_cfg_t;
#define ESP_GMF_IO_CODEC_DEV_CFG_DEFAULT() { \
.dev = NULL, \
.dir = ESP_GMF_IO_DIR_NONE, \
.name = NULL, \
}
/**
* @brief Initializes the Audio Codec Device I/O with the provided configuration
*
* @param[in] config Audio codec device io configuration
* @param[out] io Audio codec device io handle
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_INVALID_ARG Invalid configuration provided
* - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory
*/
esp_gmf_err_t esp_gmf_io_codec_dev_init(codec_dev_io_cfg_t *config, esp_gmf_io_handle_t *io);
/**
* @brief Set or update the audio codec device handle for an existing I/O instance
*
* @param[in] io Audio codec device I/O handle
* @param[in] dev New audio codec device handle to set
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_INVALID_ARG Invalid arguments provided
* - ESP_GMF_ERR_INVALID_STATE Invalid state
*/
esp_gmf_err_t esp_gmf_io_codec_dev_set_dev(esp_gmf_io_handle_t io, esp_codec_dev_handle_t dev);
#ifdef __cplusplus
}
#endif /* __cplusplus */
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_io/esp_gmf_io_embed_flash.c
+182
View File
@@ -0,0 +1,182 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#include "esp_gmf_io_embed_flash.h"
#include "esp_gmf_oal_mem.h"
#include "esp_log.h"
#define MAX_FILES_INDEX 1000
/**
* @brief Embed flash io context in GMF
*/
typedef struct {
esp_gmf_io_t base; /*!< The GMF embed flash io handle */
int cur; /*!< The current stream pos */
int max_files; /*!< The max file number */
embed_item_info_t *items; /*!< The embed flash stream item */
} embed_flash_io_t;
static const char *const TAG = "ESP_GMF_EMBED_FLASH";
static esp_gmf_err_t _embed_flash_new(void *cfg, esp_gmf_obj_handle_t *io)
{
return esp_gmf_io_embed_flash_init(cfg, io);
}
static esp_gmf_err_t _embed_flash_open(esp_gmf_io_handle_t io)
{
embed_flash_io_t *embed_flash = (embed_flash_io_t *)io;
if (embed_flash->items == NULL) {
ESP_LOGE(TAG, "There is no embedded items, please call embed_flash_io_set_context first");
return ESP_GMF_ERR_FAIL;
}
char *uri = NULL;
esp_gmf_io_get_uri((esp_gmf_io_handle_t)embed_flash, &uri);
if (uri == NULL) {
ESP_LOGE(TAG, "The file URI is NULL!");
return ESP_GMF_ERR_FAIL;
}
// The embedded file path like "embed://tone/0_alarm.mp3"
char *temp = strchr(uri, '_');
if (temp == NULL) {
ESP_LOGE(TAG, "No _ in file name, %s", uri);
return ESP_GMF_ERR_FAIL;
}
*temp = 0;
temp = strrchr(uri, '/');
if (temp == NULL) {
ESP_LOGE(TAG, "The file name is incorrect!, %s", uri);
return ESP_GMF_ERR_FAIL;
}
temp++;
int file_index = strtoul(temp, 0, 10);
ESP_LOGI(TAG, "The read item is %d, %s", file_index, uri);
if (embed_flash->max_files < file_index) {
ESP_LOGE(TAG, "The file index is out of range, %d", file_index);
return ESP_GMF_ERR_FAIL;
}
embed_flash->cur = file_index;
uint64_t total_bytes = embed_flash->items[embed_flash->cur].size;
esp_gmf_io_set_size(io, total_bytes);
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_io_t _embed_flash_acquire_read(esp_gmf_io_handle_t io, void *payload, uint32_t wanted_size, int block_ticks)
{
embed_flash_io_t *embed_flash = (embed_flash_io_t *)io;
esp_gmf_payload_t *pload = (esp_gmf_payload_t *)payload;
esp_gmf_info_file_t info = {0};
esp_gmf_io_get_info((esp_gmf_io_handle_t)embed_flash, &info);
if ((info.pos + wanted_size) > info.size) {
wanted_size = info.size - info.pos;
}
ESP_LOGD(TAG, "Embed read data, ret:%ld, pos: %llu/%llu", wanted_size, info.pos, info.size);
if (wanted_size == 0) {
ESP_LOGW(TAG, "No more data, ret:%ld, pos: %llu/%llu", wanted_size, info.pos, info.size);
pload->is_done = true;
}
memcpy(pload->buf, embed_flash->items[embed_flash->cur].address + info.pos, wanted_size);
pload->valid_size = wanted_size;
return ESP_GMF_IO_OK;
}
static esp_gmf_err_io_t _embed_flash_release_read(esp_gmf_io_handle_t io, void *payload, int block_ticks)
{
embed_flash_io_t *embed_flash = (embed_flash_io_t *)io;
esp_gmf_payload_t *pload = (esp_gmf_payload_t *)payload;
esp_gmf_info_file_t info = {0};
esp_gmf_io_get_info((esp_gmf_io_handle_t)embed_flash, &info);
ESP_LOGD(TAG, "Update len = %d, pos = %d/%d", pload->valid_size, (int)info.pos, (int)info.size);
esp_gmf_io_update_pos((esp_gmf_io_handle_t)embed_flash, pload->valid_size);
return ESP_GMF_IO_OK;
}
static esp_gmf_err_t _embed_flash_close(esp_gmf_io_handle_t io)
{
esp_gmf_info_file_t info = {0};
esp_gmf_io_get_info((esp_gmf_io_handle_t)io, &info);
ESP_LOGI(TAG, "Closed, pos: %llu/%llu", info.pos, info.size);
esp_gmf_io_set_pos(io, 0);
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t _embed_flash_destroy(esp_gmf_io_handle_t io)
{
embed_flash_io_t *embed_flash = (embed_flash_io_t *)io;
ESP_LOGD(TAG, "Delete, %s-%p", OBJ_GET_TAG(embed_flash), embed_flash);
void *cfg = OBJ_GET_CFG(io);
if (cfg) {
esp_gmf_oal_free(cfg);
}
if (embed_flash->items) {
esp_gmf_oal_free(embed_flash->items);
}
esp_gmf_io_deinit(io);
esp_gmf_oal_free(embed_flash);
return ESP_GMF_ERR_OK;
}
esp_gmf_err_t esp_gmf_io_embed_flash_init(embed_flash_io_cfg_t *config, esp_gmf_io_handle_t *io)
{
ESP_GMF_NULL_CHECK(TAG, config, return ESP_GMF_ERR_INVALID_ARG;);
ESP_GMF_NULL_CHECK(TAG, io, return ESP_GMF_ERR_INVALID_ARG;);
*io = NULL;
esp_gmf_err_t ret = ESP_GMF_ERR_OK;
embed_flash_io_t *embed_flash = esp_gmf_oal_calloc(1, sizeof(embed_flash_io_t));
ESP_GMF_MEM_VERIFY(TAG, embed_flash, return ESP_GMF_ERR_MEMORY_LACK,
"embed flash stream", sizeof(embed_flash_io_t));
embed_flash->base.dir = ESP_GMF_IO_DIR_READER;
embed_flash->base.type = ESP_GMF_IO_TYPE_BYTE;
embed_flash->max_files = config->max_files;
esp_gmf_obj_t *obj = (esp_gmf_obj_t *)embed_flash;
obj->new_obj = _embed_flash_new;
obj->del_obj = _embed_flash_destroy;
embed_flash_io_cfg_t *cfg = esp_gmf_oal_calloc(1, sizeof(*config));
ESP_GMF_MEM_VERIFY(TAG, cfg, {ret = ESP_GMF_ERR_MEMORY_LACK; goto _embed_fail;},
"embed flash stream configuration", sizeof(*config));
memcpy(cfg, config, sizeof(*config));
esp_gmf_obj_set_config(obj, cfg, sizeof(*config));
ret = esp_gmf_obj_set_tag(obj, (config->name == NULL ? "io_embed_flash" : config->name));
ESP_GMF_RET_ON_NOT_OK(TAG, ret, goto _embed_fail, "Failed to set obj tag");
embed_flash->base.open = _embed_flash_open;
embed_flash->base.close = _embed_flash_close;
embed_flash->base.seek = NULL;
embed_flash->base.reset = NULL;
esp_gmf_io_init(obj, NULL);
if (embed_flash->base.dir == ESP_GMF_IO_DIR_READER) {
embed_flash->base.acquire_read = _embed_flash_acquire_read;
embed_flash->base.release_read = _embed_flash_release_read;
} else {
ESP_LOGE(TAG, "Does not support this operation, %d", embed_flash->base.dir);
ret = ESP_GMF_ERR_NOT_SUPPORT;
goto _embed_fail;
}
*io = obj;
ESP_LOGD(TAG, "Initialization, %s-%p", OBJ_GET_TAG(obj), embed_flash);
return ESP_GMF_ERR_OK;
_embed_fail:
esp_gmf_obj_delete(obj);
return ret;
}
esp_gmf_err_t esp_gmf_io_embed_flash_set_context(esp_gmf_io_handle_t io, const embed_item_info_t *context, int max_num)
{
ESP_GMF_NULL_CHECK(TAG, io, return ESP_GMF_ERR_INVALID_ARG);
ESP_GMF_NULL_CHECK(TAG, context, return ESP_GMF_ERR_INVALID_ARG);
embed_flash_io_t *embed_flash = (embed_flash_io_t *)io;
if (embed_flash->items) {
esp_gmf_oal_free(embed_flash->items);
}
embed_flash->items = esp_gmf_oal_calloc(max_num, sizeof(embed_item_info_t));
ESP_GMF_MEM_VERIFY(TAG, embed_flash->items, return ESP_GMF_ERR_MEMORY_LACK,
"embed item information", sizeof(embed_item_info_t));
embed_flash->max_files = max_num;
memcpy(embed_flash->items, context, sizeof(embed_item_info_t) * max_num);
return ESP_GMF_ERR_OK;
}
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_io/esp_gmf_io_file.c
+298
View File
@@ -0,0 +1,298 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#include <sys/unistd.h>
#include <sys/stat.h>
#include <stdio.h>
#include <string.h>
#include "errno.h"
#include "esp_heap_caps.h"
#include "esp_gmf_io_file.h"
#include "esp_gmf_oal_mem.h"
#include "fcntl.h"
#include "esp_log.h"
#define IO_FILE_ALIGN_UP(num, align) (((num) + ((align) - 1)) & ~((align) - 1))
#define MIN(a, b) ((a) < (b) ? (a) : (b))
#define IO_FILE_DEFAULT_CACHE_ALIGN (512)
/**
* @brief File io context in GMF
*/
typedef struct {
esp_gmf_io_t base; /*!< The GMF file io handle */
bool is_open; /*!< The flag of whether opened */
FILE *file; /*!< The handle of file stream */
uint8_t *cache; /*!< The cache buffer */
int cache_size; /*!< The cache size */
int cache_caps; /*!< The cache memory capabilities */
} file_io_stream_t;
static const char *TAG = "ESP_GMF_FILE";
static char *get_mount_path(char *uri)
{
/* support format: /sdcard, /spiffs, /storage etc ... */
if (uri[0] == '/') {
return uri;
}
/* support format: scheme://basepath... */
char *skip_scheme = strstr(uri, "://");
if (skip_scheme == NULL) {
return NULL;
}
skip_scheme += 2;
/* support format: scheme:///basepath... */
if (skip_scheme[1] == '/') {
skip_scheme++;
}
return skip_scheme;
}
static esp_gmf_err_t _file_new(void *cfg, esp_gmf_obj_handle_t *io)
{
return esp_gmf_io_file_init(cfg, io);
}
static esp_gmf_err_t _file_open(esp_gmf_io_handle_t io)
{
file_io_stream_t *file_io = (file_io_stream_t *)io;
char *uri = NULL;
esp_gmf_io_get_uri((esp_gmf_io_handle_t)file_io, &uri);
if (uri == NULL) {
ESP_LOGE(TAG, "Error, uri is not set, handle: %p", io);
return ESP_GMF_ERR_FAIL;
}
ESP_LOGI(TAG, "Open, dir:%d, uri:%s", ((file_io_cfg_t *)file_io->base.parent.cfg)->dir, uri);
char *path = get_mount_path(uri);
if (path == NULL) {
ESP_LOGE(TAG, "Invalid URI (%s).", uri);
return ESP_GMF_ERR_FAIL;
}
if (file_io->is_open) {
ESP_LOGE(TAG, "Already opened, p: %p, path: %s", file_io, path);
return ESP_GMF_ERR_FAIL;
}
if (((file_io_cfg_t *)file_io->base.parent.cfg)->dir == ESP_GMF_IO_DIR_READER) {
file_io->file = fopen(path, "rb");
if (file_io->file == NULL) {
ESP_LOGE(TAG, "Failed to open on read, path: %s, err: %s", path, strerror(errno));
return ESP_GMF_ERR_FAIL;
}
struct stat sz = {0};
stat(path, &sz);
esp_gmf_io_set_size((esp_gmf_io_handle_t)file_io, sz.st_size);
esp_gmf_info_file_t info = {0};
esp_gmf_io_get_info((esp_gmf_io_handle_t)file_io, &info);
ESP_LOGI(TAG, "File size: %d byte, file position: %lld", (int)sz.st_size, info.pos);
if (info.pos > 0) {
if (fseek(file_io->file, info.pos, SEEK_SET) != 0) {
ESP_LOGE(TAG, "Seek to %lld failed, err: %s", info.pos, strerror(errno));
return ESP_GMF_ERR_FAIL;
}
}
} else if (((file_io_cfg_t *)file_io->base.parent.cfg)->dir == ESP_GMF_IO_DIR_WRITER) {
file_io->file = fopen(path, "wb");
if (file_io->file == NULL) {
ESP_LOGE(TAG, "Failed to open on write, path: %s, err: %s", path, strerror(errno));
return ESP_GMF_ERR_FAIL;
}
} else {
ESP_LOGE(TAG, "The type must be reader or writer");
return ESP_GMF_ERR_FAIL;
}
if (file_io->cache_size > 0) {
file_io->cache = heap_caps_malloc(file_io->cache_size, file_io->cache_caps);
ESP_GMF_MEM_VERIFY(TAG, file_io->cache, { fclose(file_io->file); return ESP_GMF_ERR_MEMORY_LACK;},
"file stream cache", file_io->cache_size);
setvbuf(file_io->file, (char *)file_io->cache, _IOFBF, file_io->cache_size);
ESP_LOGD(TAG, "File_io cache: %p, size: %d, caps: 0x%x", file_io->cache, file_io->cache_size, file_io->cache_caps);
}
file_io->is_open = true;
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_io_t _file_acquire_read(esp_gmf_io_handle_t handle, void *payload, uint32_t wanted_size, int block_ticks)
{
file_io_stream_t *file_io = (file_io_stream_t *)handle;
esp_gmf_payload_t *pload = (esp_gmf_payload_t *)payload;
size_t rlen = fread(pload->buf, 1, wanted_size, file_io->file);
ESP_LOGD(TAG, "Read len: %zu-%ld", rlen, wanted_size);
if (rlen == 0 && ferror(file_io->file)) {
ESP_LOGE(TAG, "The error is happened in reading data, error msg: %s", strerror(errno));
return ESP_GMF_IO_FAIL;
}
pload->valid_size = rlen;
if (rlen < wanted_size) {
rlen = fread(pload->buf + rlen, 1, (wanted_size - rlen), file_io->file);
if (rlen == 0) {
pload->is_done = true;
ESP_LOGI(TAG, "No more data, ret: %zu", rlen);
return ESP_GMF_IO_OK;
}
pload->valid_size += rlen;
}
return ESP_GMF_IO_OK;
}
static esp_gmf_err_io_t _file_release_read(esp_gmf_io_handle_t handle, void *payload, int block_ticks)
{
file_io_stream_t *file_io = (file_io_stream_t *)handle;
esp_gmf_payload_t *pload = (esp_gmf_payload_t *)payload;
esp_gmf_info_file_t info = {0};
esp_gmf_io_get_info((esp_gmf_io_handle_t)file_io, &info);
ESP_LOGD(TAG, "Update len = %d, pos = %d/%d", pload->valid_size, (int)info.pos, (int)info.size);
esp_gmf_io_update_pos((esp_gmf_io_handle_t)handle, pload->valid_size);
return ESP_GMF_IO_OK;
}
static esp_gmf_err_io_t _file_acquire_write(esp_gmf_io_handle_t handle, void *payload, uint32_t wanted_size, int block_ticks)
{
return ESP_GMF_IO_OK;
}
static esp_gmf_err_io_t _file_release_write(esp_gmf_io_handle_t handle, void *payload, int block_ticks)
{
file_io_stream_t *file_io = (file_io_stream_t *)handle;
esp_gmf_payload_t *pload = (esp_gmf_payload_t *)payload;
size_t wlen_total = 0;
uint8_t *pbuf = pload->buf;
if (file_io->cache_size > 0) {
while (pload->valid_size > wlen_total) {
int size = MIN(pload->valid_size - wlen_total, file_io->cache_size - 1);
size_t wlen = fwrite(pbuf, 1, size, file_io->file);
if (wlen == 0) {
break;
}
pbuf += wlen;
wlen_total += wlen;
}
} else {
wlen_total = fwrite(pload->buf, 1, pload->valid_size, file_io->file);
}
if (wlen_total != pload->valid_size) {
ESP_LOGE(TAG, "The error is happened in writing data, error msg:%s", strerror(errno));
return ESP_GMF_IO_FAIL;
}
esp_gmf_info_file_t info = {0};
esp_gmf_io_get_info((esp_gmf_io_handle_t)file_io, &info);
ESP_LOGD(TAG, "Write len = %zu, pos = %d/%d", wlen_total, (int)info.pos, (int)info.size);
esp_gmf_io_update_pos((esp_gmf_io_handle_t)handle, wlen_total);
return ESP_GMF_IO_OK;
}
static esp_gmf_err_t _file_seek(esp_gmf_io_handle_t io, uint64_t seek_byte_pos)
{
file_io_stream_t *file_io = (file_io_stream_t *)io;
esp_gmf_info_file_t info = {0};
esp_gmf_io_get_info((esp_gmf_io_handle_t)file_io, &info);
ESP_LOGI(TAG, "Seek position, total_bytes: %lld, seek: %lld",
info.size, seek_byte_pos);
if (seek_byte_pos > info.size) {
ESP_LOGE(TAG, "Seek position is out of range, total_bytes: %lld, seek: %lld",
info.size, seek_byte_pos);
return ESP_GMF_ERR_OUT_OF_RANGE;
}
if (fseek(file_io->file, seek_byte_pos, SEEK_SET) != 0) {
ESP_LOGE(TAG, "Error seek file, error message: %s, line: %d", strerror(errno), __LINE__);
return ESP_GMF_ERR_FAIL;
}
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t _file_reset(esp_gmf_io_handle_t io)
{
file_io_stream_t *file_io = (file_io_stream_t *)io;
if (file_io->file != NULL) {
fseek(file_io->file, 0, SEEK_SET);
}
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t _file_close(esp_gmf_io_handle_t io)
{
file_io_stream_t *file_io = (file_io_stream_t *)io;
esp_gmf_info_file_t info = {0};
esp_gmf_io_get_info((esp_gmf_io_handle_t)file_io, &info);
ESP_LOGI(TAG, "CLose, %p, pos = %d/%d", file_io, (int)info.pos, (int)info.size);
if (file_io->is_open) {
fclose(file_io->file);
file_io->file = NULL;
file_io->is_open = false;
}
esp_gmf_io_set_pos((esp_gmf_io_handle_t)io, 0);
if (file_io->cache) {
esp_gmf_oal_free(file_io->cache);
file_io->cache = NULL;
}
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t _file_delete(esp_gmf_io_handle_t io)
{
file_io_stream_t *file_io = (file_io_stream_t *)io;
ESP_LOGD(TAG, "Delete, %s-%p", OBJ_GET_TAG(file_io), file_io);
void *cfg = OBJ_GET_CFG(io);
if (cfg) {
esp_gmf_oal_free(cfg);
}
esp_gmf_io_deinit(io);
esp_gmf_oal_free(file_io);
return ESP_GMF_ERR_OK;
}
esp_gmf_err_t esp_gmf_io_file_init(file_io_cfg_t *config, esp_gmf_io_handle_t *io)
{
ESP_GMF_NULL_CHECK(TAG, config, return ESP_GMF_ERR_INVALID_ARG);
ESP_GMF_NULL_CHECK(TAG, io, return ESP_GMF_ERR_INVALID_ARG);
*io = NULL;
esp_gmf_err_t ret = ESP_GMF_ERR_OK;
file_io_stream_t *file_io = esp_gmf_oal_calloc(1, sizeof(file_io_stream_t));
ESP_GMF_MEM_VERIFY(TAG, file_io, return ESP_GMF_ERR_MEMORY_LACK,
"file stream", sizeof(file_io_stream_t));
file_io->base.dir = config->dir;
file_io->base.type = ESP_GMF_IO_TYPE_BYTE;
esp_gmf_obj_t *obj = (esp_gmf_obj_t *)file_io;
obj->new_obj = _file_new;
obj->del_obj = _file_delete;
file_io_cfg_t *cfg = esp_gmf_oal_calloc(1, sizeof(*config));
ESP_GMF_MEM_VERIFY(TAG, cfg, {ret = ESP_GMF_ERR_MEMORY_LACK; goto _file_fail;}, "file_io cfg", sizeof(*config));
memcpy(cfg, config, sizeof(*config));
esp_gmf_obj_set_config(obj, cfg, sizeof(*config));
ret = esp_gmf_obj_set_tag(obj, (config->name == NULL ? "io_file" : config->name));
ESP_GMF_RET_ON_NOT_OK(TAG, ret, goto _file_fail, "Failed to set obj tag");
file_io->base.close = _file_close;
file_io->base.open = _file_open;
file_io->base.seek = _file_seek;
file_io->base.reset = _file_reset;
file_io_cfg_t *fat_cfg = (file_io_cfg_t *)config;
esp_gmf_io_init(obj, NULL);
if (fat_cfg->dir == ESP_GMF_IO_DIR_WRITER) {
file_io->base.acquire_write = _file_acquire_write;
file_io->base.release_write = _file_release_write;
} else if (fat_cfg->dir == ESP_GMF_IO_DIR_READER) {
file_io->base.acquire_read = _file_acquire_read;
file_io->base.release_read = _file_release_read;
} else {
ESP_LOGW(TAG, "Does not set read or write function");
ret = ESP_GMF_ERR_NOT_SUPPORT;
goto _file_fail;
}
*io = obj;
if (cfg->cache_size <= IO_FILE_DEFAULT_CACHE_ALIGN) {
file_io->cache_size = 0;
} else {
file_io->cache_caps = (cfg->cache_caps == 0) ? (MALLOC_CAP_DMA) : cfg->cache_caps;
file_io->cache_size = IO_FILE_ALIGN_UP(cfg->cache_size, IO_FILE_DEFAULT_CACHE_ALIGN);
}
return ESP_GMF_ERR_OK;
_file_fail:
esp_gmf_obj_delete(obj);
return ret;
}
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_io/esp_gmf_io_http.c
+534
View File
@@ -0,0 +1,534 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#include <stdbool.h>
#include <errno.h>
#include "esp_log.h"
#include "esp_http_client.h"
#include "gzip_miniz.h"
#include "esp_gmf_new_databus.h"
#include "esp_gmf_io_http.h"
#include "esp_gmf_oal_mem.h"
#define HTTP_STREAM_BUFFER_SIZE (3 * 1024)
#define HTTP_MAX_CONNECT_TIMES (5)
/**
* @brief Http io context in GMF
*/
typedef struct http_io {
esp_gmf_io_t base; /*!< The GMF http io handle */
bool is_open; /*!< The flag of whether opened */
esp_http_client_handle_t client; /*!< The http client handle */
int _errno; /*!< Errno code for http */
int connect_times; /*!< Max reconnect times */
bool gzip_encoding; /*!< Content is encoded */
gzip_miniz_handle_t gzip; /*!< GZIP instance */
esp_gmf_db_handle_t data_bus; /*!< The data bus handle */
} http_stream_t;
static const char *TAG = "ESP_GMF_HTTP";
static esp_gmf_err_t _http_destroy(esp_gmf_io_handle_t self);
static int _gzip_read_data(uint8_t *data, int size, void *ctx)
{
http_stream_t *http = (http_stream_t *)ctx;
return esp_http_client_read(http->client, (char *)data, size);
}
static esp_gmf_err_t _http_event_handle(esp_http_client_event_t *evt)
{
http_stream_t *http = (http_stream_t *)evt->user_data;
if (evt->event_id != HTTP_EVENT_ON_HEADER) {
return ESP_GMF_ERR_OK;
}
if (strcasecmp(evt->header_key, "Content-Encoding") == 0) {
http->gzip_encoding = true;
if (strcasecmp(evt->header_value, "gzip") == 0) {
gzip_miniz_cfg_t cfg = {
.chunk_size = 1024,
.ctx = http,
.read_cb = _gzip_read_data,
};
http->gzip = gzip_miniz_init(&cfg);
}
if (http->gzip == NULL) {
ESP_LOGE(TAG, "Content-Encoding %s not supported", evt->header_value);
return ESP_GMF_ERR_FAIL;
}
}
return ESP_GMF_ERR_OK;
}
static int dispatch_hook(esp_gmf_io_handle_t self, http_stream_event_id_t type, void *buffer, int buffer_len)
{
http_stream_t *http_stream = (http_stream_t *)self;
http_io_cfg_t *http_io_cfg = (http_io_cfg_t *)OBJ_GET_CFG(http_stream);
http_stream_event_msg_t msg;
msg.event_id = type;
msg.http_client = http_stream->client;
msg.user_data = http_io_cfg->user_data;
msg.buffer = buffer;
msg.buffer_len = buffer_len;
if (http_io_cfg->event_handle) {
return http_io_cfg->event_handle(&msg);
}
return ESP_GMF_ERR_OK;
}
static int _http_read_data(http_stream_t *http, char *buffer, int len)
{
if (http->gzip_encoding == false) {
return esp_http_client_read(http->client, buffer, len);
}
// use gzip to uncompress data
return gzip_miniz_read(http->gzip, (uint8_t *)buffer, len);
}
static esp_gmf_err_t _http_new(void *cfg, esp_gmf_obj_handle_t *io)
{
return esp_gmf_io_http_init(cfg, io);
}
static esp_gmf_err_t _http_open(esp_gmf_io_handle_t self)
{
http_stream_t *http = (http_stream_t *)self;
esp_gmf_err_t err;
if (http->is_open) {
ESP_LOGW(TAG, "The HTTP already opened, %p", http);
return ESP_GMF_ERR_OK;
}
http->_errno = 0;
esp_gmf_info_file_t info = {0};
esp_gmf_io_get_info((esp_gmf_io_handle_t)http, &info);
char *uri = (char *)info.uri;
if (uri == NULL) {
ESP_LOGE(TAG, "Error open connection, uri = NULL");
return ESP_GMF_ERR_FAIL;
}
char *hls_type = strrchr(uri, '/');
if (hls_type && strstr(hls_type, ".m3u")) {
ESP_LOGE(TAG, "The HTTP stream does not support HTTP Live Streaming. URI:%s", uri);
return ESP_GMF_ERR_FAIL;
}
esp_gmf_io_get_info((esp_gmf_io_handle_t)http, &info);
http_io_cfg_t *http_io_cfg = (http_io_cfg_t *)OBJ_GET_CFG(http);
ESP_LOGI(TAG, "HTTP Open, URI = %s", uri);
if (http->client == NULL) {
esp_http_client_config_t http_cfg = {
.url = uri,
.event_handler = _http_event_handle,
.user_data = self,
.timeout_ms = 30 * 1000,
.buffer_size = HTTP_STREAM_BUFFER_SIZE,
.buffer_size_tx = 1024,
.cert_pem = http_io_cfg->cert_pem,
#ifdef CONFIG_MBEDTLS_CERTIFICATE_BUNDLE
.crt_bundle_attach = http_io_cfg->crt_bundle_attach,
#endif /* CONFIG_MBEDTLS_CERTIFICATE_BUNDLE */
};
http->client = esp_http_client_init(&http_cfg);
ESP_GMF_CHECK(TAG, http->client, return ESP_GMF_ERR_MEMORY_LACK, "Failed to initialize http client");
} else {
esp_http_client_set_url(http->client, uri);
}
if (info.pos) {
char rang_header[32];
snprintf(rang_header, 32, "bytes=%d-", (int)info.pos);
esp_http_client_set_header(http->client, "Range", rang_header);
} else {
esp_http_client_delete_header(http->client, "Range");
}
if (dispatch_hook(self, HTTP_STREAM_PRE_REQUEST, NULL, 0) != ESP_GMF_ERR_OK) {
ESP_LOGE(TAG, "Failed to process user callback:%d", __LINE__);
return ESP_GMF_ERR_FAIL;
}
if (http->data_bus == NULL) {
err = esp_gmf_db_new_block(1, http_io_cfg->out_buf_size, &http->data_bus);
if (err != ESP_GMF_ERR_OK) {
ESP_LOGE(TAG, "Failed to create the buffer for %d, sz: %d, %s-%p", http_io_cfg->dir, http_io_cfg->out_buf_size, OBJ_GET_TAG(http), http);
return err;
}
esp_gmf_data_bus_type_t db_type = 0;
esp_gmf_db_get_type(http->data_bus, &db_type);
http->base.type = db_type;
}
if (http_io_cfg->dir == ESP_GMF_IO_DIR_WRITER) {
err = esp_http_client_open(http->client, -1);
if (err == ESP_GMF_ERR_OK) {
http->is_open = true;
}
return err;
}
char *buffer = NULL;
int post_len = esp_http_client_get_post_field(http->client, &buffer);
_stream_redirect:
if (http->gzip_encoding) {
gzip_miniz_deinit(http->gzip);
http->gzip = NULL;
http->gzip_encoding = false;
}
if ((err = esp_http_client_open(http->client, post_len)) != ESP_GMF_ERR_OK) {
ESP_LOGE(TAG, "Failed to open http stream");
return err;
}
int wrlen = dispatch_hook(self, HTTP_STREAM_ON_REQUEST, buffer, post_len);
if (wrlen < 0) {
ESP_LOGE(TAG, "Failed to process user callback:%d", __LINE__);
return ESP_GMF_ERR_FAIL;
}
if (post_len && buffer && wrlen == 0) {
if (esp_http_client_write(http->client, buffer, post_len) <= 0) {
ESP_LOGE(TAG, "Failed to write data to http stream");
return ESP_GMF_ERR_FAIL;
}
ESP_LOGD(TAG, "len=%d, data=%s", post_len, buffer);
}
if (dispatch_hook(self, HTTP_STREAM_POST_REQUEST, NULL, 0) < 0) {
esp_http_client_close(http->client);
return ESP_GMF_ERR_FAIL;
}
/**
* Due to the total byte of content has been changed after seek, set info.total_bytes at beginning only.
*/
int64_t cur_pos = esp_http_client_fetch_headers(http->client);
esp_gmf_io_get_info((esp_gmf_io_handle_t)http, &info);
if (info.pos <= 0) {
info.size = cur_pos;
}
ESP_LOGI(TAG, "The total size is %d bytes", (int)info.size);
int status_code = esp_http_client_get_status_code(http->client);
if (status_code == 301 || status_code == 302) {
esp_http_client_set_redirection(http->client);
goto _stream_redirect;
}
if (status_code != 200
&& (esp_http_client_get_status_code(http->client) != 206)) {
ESP_LOGE(TAG, "Invalid HTTP stream, status code = %d", status_code);
return ESP_GMF_ERR_FAIL;
}
esp_gmf_io_set_size(self, info.size);
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t _http_prev_close(esp_gmf_io_handle_t self)
{
http_stream_t *http = (http_stream_t *)self;
esp_gmf_db_abort(http->data_bus);
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t _http_close(esp_gmf_io_handle_t self)
{
http_stream_t *http = (http_stream_t *)self;
ESP_LOGD(TAG, "_http_close, %p", http);
http_io_cfg_t *http_io_cfg = (http_io_cfg_t *)OBJ_GET_CFG(http);
if (http->is_open && (http_io_cfg->dir == ESP_GMF_IO_DIR_WRITER)) {
do {
if (dispatch_hook(self, HTTP_STREAM_POST_REQUEST, NULL, 0) < 0) {
break;
}
if (esp_http_client_fetch_headers(http->client) < 0) {
break;
}
if (dispatch_hook(self, HTTP_STREAM_FINISH_REQUEST, NULL, 0) < 0) {
break;
}
} while (0);
}
http->is_open = false;
if (http->gzip) {
gzip_miniz_deinit(http->gzip);
http->gzip = NULL;
}
if (http->client) {
esp_http_client_close(http->client);
esp_http_client_cleanup(http->client);
http->client = NULL;
}
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t _http_reconnect(esp_gmf_io_handle_t self)
{
esp_gmf_err_t err = ESP_GMF_ERR_OK;
ESP_GMF_NULL_CHECK(TAG, self, return ESP_GMF_ERR_FAIL);
esp_gmf_info_file_t info = {0};
err |= esp_gmf_io_get_info((esp_gmf_io_handle_t)self, &info);
err |= _http_close(self);
err |= esp_gmf_io_update_pos(self, info.pos);
err |= _http_open(self);
return err;
}
static int _http_read(esp_gmf_io_handle_t self, char *buffer, int len, TickType_t ticks_to_wait, void *context)
{
http_stream_t *http = (http_stream_t *)self;
esp_gmf_info_file_t info = {0};
esp_gmf_io_get_info((esp_gmf_io_handle_t)http, &info);
int wrlen = dispatch_hook(self, HTTP_STREAM_ON_RESPONSE, buffer, len);
int rlen = wrlen;
if (rlen == 0) {
rlen = _http_read_data(http, buffer, len);
}
if (rlen <= 0) {
http->_errno = esp_http_client_get_errno(http->client);
ESP_LOGW(TAG, "No more data, errno: %d, read bytes: %llu, rlen = %d", http->_errno, info.pos, rlen);
if (http->_errno != 0) { // Error occuered, reset connection
ESP_LOGW(TAG, "Got %d errno(%s)", http->_errno, strerror(http->_errno));
return http->_errno;
}
return ESP_GMF_ERR_OK;
} else {
esp_gmf_io_update_pos(self, rlen);
}
ESP_LOGD(TAG, "req length = %d, read = %d, pos = %d/%d", len, rlen, (int)info.pos, (int)info.size);
return rlen;
}
static int _http_write(esp_gmf_io_handle_t self, char *buffer, int len, TickType_t ticks_to_wait, void *context)
{
http_stream_t *http = (http_stream_t *)self;
int wrlen = dispatch_hook(self, HTTP_STREAM_ON_REQUEST, buffer, len);
if (wrlen < 0) {
ESP_LOGE(TAG, "Failed to process user callback:%d", __LINE__);
return ESP_GMF_ERR_FAIL;
}
if (wrlen > 0) {
return wrlen;
}
if ((wrlen = esp_http_client_write(http->client, buffer, len)) <= 0) {
http->_errno = esp_http_client_get_errno(http->client);
ESP_LOGE(TAG, "Failed to write data to http stream, wrlen = %d, errno = %d(%s)", wrlen, http->_errno, strerror(http->_errno));
}
return wrlen;
}
static esp_gmf_job_err_t _http_process(esp_gmf_io_handle_t self, void *params)
{
int r_size = 0;
http_stream_t *http = (http_stream_t *)self;
esp_gmf_data_bus_block_t blk = {0};
esp_gmf_job_err_t job_err = ESP_GMF_JOB_ERR_OK;
http->is_open = true;
http_io_cfg_t *http_io_cfg = (http_io_cfg_t *)OBJ_GET_CFG(http);
if (http_io_cfg->dir == ESP_GMF_IO_DIR_READER) {
esp_gmf_db_acquire_write(http->data_bus, &blk, HTTP_STREAM_BUFFER_SIZE, portMAX_DELAY);
r_size = _http_read(self, (char *)blk.buf, blk.buf_length, portMAX_DELAY, NULL);
blk.valid_size = r_size;
ESP_LOGD(TAG, "Read: %d, len: %d", r_size, blk.buf_length);
if (r_size > 0) {
if (http->_errno != 0) {
esp_gmf_err_t ret = ESP_GMF_ERR_OK;
if (http->connect_times > HTTP_MAX_CONNECT_TIMES) {
ESP_LOGE(TAG, "Reconnect times more than %d, disconnect http stream", HTTP_MAX_CONNECT_TIMES);
return ESP_GMF_ERR_FAIL;
};
http->connect_times++;
ret = _http_reconnect(self);
if (ret != ESP_GMF_ERR_OK) {
ESP_LOGE(TAG, "Failed to reset connection");
return ret;
}
ESP_LOGW(TAG, "Reconnect to peer successful");
return ESP_GMF_ERR_INVALID_STATE;
} else {
http->connect_times = 0;
if (http_io_cfg->dir == ESP_GMF_IO_DIR_READER) {
esp_gmf_db_release_write(http->data_bus, &blk, portMAX_DELAY);
}
}
} else if (r_size == 0) {
esp_gmf_db_done_write(http->data_bus);
esp_gmf_db_release_write(http->data_bus, &blk, portMAX_DELAY);
job_err = ESP_GMF_JOB_ERR_DONE;
} else {
job_err = r_size;
esp_gmf_db_abort(http->data_bus);
}
} else {
r_size = esp_gmf_db_acquire_read(http->data_bus, &blk, HTTP_STREAM_BUFFER_SIZE, portMAX_DELAY);
ESP_LOGD(TAG, "ACQ, read: %d, vld: %d, buf_len: %d", r_size, blk.valid_size, blk.buf_length);
if (blk.valid_size > 0) {
int w_size = _http_write(self, (char *)blk.buf, blk.valid_size, portMAX_DELAY, NULL);
if (w_size <= 0) {
job_err = ESP_GMF_JOB_ERR_FAIL;
}
} else if (r_size == ESP_GMF_IO_OK || r_size == ESP_GMF_IO_ABORT) {
job_err = ESP_GMF_JOB_ERR_DONE;
} else {
job_err = r_size;
}
esp_gmf_db_release_read(http->data_bus, &blk, portMAX_DELAY);
}
return job_err;
}
static esp_gmf_err_t _http_destroy(esp_gmf_io_handle_t self)
{
http_stream_t *http = (http_stream_t *)self;
ESP_LOGD(TAG, "%s-%p", __FUNCTION__, http);
if (http->data_bus) {
esp_gmf_db_deinit(http->data_bus);
http->data_bus = NULL;
}
void *cfg = OBJ_GET_CFG(self);
if (cfg) {
esp_gmf_oal_free(cfg);
}
esp_gmf_io_deinit(http);
esp_gmf_oal_free(http);
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t _http_seek(esp_gmf_io_handle_t handle, uint64_t pos)
{
http_stream_t *http = (http_stream_t *)handle;
esp_gmf_info_file_t info = {0};
esp_gmf_io_get_info((esp_gmf_io_handle_t)handle, &info);
if (pos > info.size) {
ESP_LOGE(TAG, "The seek position is out of range, pos %llu > %llu, http: %p", pos, info.size, http);
return ESP_GMF_ERR_OUT_OF_RANGE;
}
ESP_LOGD(TAG, "HTTP Seek to: %lld, %p", pos, http);
_http_close(handle);
esp_gmf_io_set_pos(http, pos);
esp_gmf_db_reset(http->data_bus);
_http_open(handle);
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_io_t _http_acquire_read(esp_gmf_io_handle_t handle, void *payload, uint32_t wanted_size, int block_ticks)
{
http_stream_t *http = (http_stream_t *)handle;
esp_gmf_data_bus_block_t *blk = (esp_gmf_data_bus_block_t *)payload;
esp_gmf_err_io_t ret = esp_gmf_db_acquire_read(http->data_bus, payload, wanted_size, block_ticks);
ESP_LOGD(TAG, "acq_rd: %ld, vld: %d, done: %d, %p, %d", wanted_size, blk->valid_size, blk->is_last, blk->buf, blk->buf_length);
return ret;
}
static esp_gmf_err_io_t _http_release_read(esp_gmf_io_handle_t handle, void *payload, int block_ticks)
{
http_stream_t *http = (http_stream_t *)handle;
esp_gmf_data_bus_block_t *blk = (esp_gmf_data_bus_block_t *)payload;
ESP_LOGD(TAG, "rel_rd: %p, vld: %d, len: %d done: %d", blk->buf, blk->valid_size, blk->buf_length, blk->is_last);
return esp_gmf_db_release_read(http->data_bus, payload, block_ticks);
}
static esp_gmf_err_io_t _http_acquire_write(esp_gmf_io_handle_t handle, void *payload, uint32_t wanted_size, int block_ticks)
{
http_stream_t *http = (http_stream_t *)handle;
esp_gmf_data_bus_block_t *blk = (esp_gmf_data_bus_block_t *)payload;
ESP_LOGD(TAG, "acq_wr: %ld, vld: %d, done: %d, %p, %d", wanted_size, blk->valid_size, blk->is_last, blk->buf, blk->buf_length);
return esp_gmf_db_acquire_write(http->data_bus, payload, wanted_size, block_ticks);
}
static esp_gmf_err_io_t _http_release_write(esp_gmf_io_handle_t handle, void *payload, int block_ticks)
{
http_stream_t *http = (http_stream_t *)handle;
esp_gmf_data_bus_block_t *blk = (esp_gmf_data_bus_block_t *)payload;
ESP_LOGD(TAG, "rel_wr: %p, vld: %d, len: %d, done: %d", blk->buf, blk->valid_size, blk->buf_length, blk->is_last);
return esp_gmf_db_release_write(http->data_bus, payload, block_ticks);
}
esp_gmf_err_t esp_gmf_io_http_reset(esp_gmf_io_handle_t handle)
{
ESP_GMF_NULL_CHECK(TAG, handle, return ESP_GMF_ERR_INVALID_ARG;);
http_stream_t *http = (http_stream_t *)handle;
if (http->data_bus) {
esp_gmf_db_reset(http->data_bus);
}
ESP_LOGD(TAG, "Reset, %p", http);
return ESP_GMF_ERR_OK;
}
esp_gmf_err_t esp_gmf_io_http_set_server_cert(esp_gmf_io_handle_t handle, const char *cert)
{
ESP_GMF_NULL_CHECK(TAG, handle, return ESP_GMF_ERR_INVALID_ARG;);
ESP_GMF_NULL_CHECK(TAG, cert, return ESP_GMF_ERR_INVALID_ARG);
http_io_cfg_t *http_io_cfg = (http_io_cfg_t *)OBJ_GET_CFG(handle);
http_io_cfg->cert_pem = cert;
return ESP_GMF_ERR_OK;
}
esp_gmf_err_t esp_gmf_io_http_set_event_callback(esp_gmf_io_handle_t handle, http_io_event_handle_t event_callback)
{
ESP_GMF_NULL_CHECK(TAG, handle, return ESP_GMF_ERR_INVALID_ARG;);
http_io_cfg_t *http_io_cfg = (http_io_cfg_t *)OBJ_GET_CFG(handle);
http_io_cfg->event_handle = event_callback;
return ESP_GMF_ERR_OK;
}
esp_gmf_err_t esp_gmf_io_http_init(http_io_cfg_t *config, esp_gmf_io_handle_t *io)
{
ESP_GMF_NULL_CHECK(TAG, config, return ESP_GMF_ERR_INVALID_ARG);
ESP_GMF_NULL_CHECK(TAG, io, return ESP_GMF_ERR_INVALID_ARG);
*io = NULL;
esp_gmf_err_t ret = ESP_GMF_ERR_OK;
http_stream_t *http = esp_gmf_oal_calloc(1, sizeof(http_stream_t));
ESP_GMF_MEM_VERIFY(TAG, http, return ESP_GMF_ERR_MEMORY_LACK,
"http stream", sizeof(http_stream_t));
esp_gmf_obj_t *obj = (esp_gmf_obj_t *)http;
obj->new_obj = _http_new;
obj->del_obj = _http_destroy;
http_io_cfg_t *cfg = esp_gmf_oal_calloc(1, sizeof(*config));
ESP_GMF_MEM_VERIFY(TAG, cfg, {ret = ESP_GMF_ERR_MEMORY_LACK; goto _http_init_fail;},
"http stream configuration", sizeof(*config));
memcpy(cfg, config, sizeof(*config));
esp_gmf_obj_set_config(obj, cfg, sizeof(*config));
ret = esp_gmf_obj_set_tag(obj, "io_http");
ESP_GMF_RET_ON_NOT_OK(TAG, ret, goto _http_init_fail, "Failed to set obj tag");
http->base.dir = config->dir;
http->base.type = ESP_GMF_IO_TYPE_BLOCK;
http->base.open = _http_open;
http->base.process = _http_process;
http->base.seek = _http_seek;
http->base.prev_close = _http_prev_close;
http->base.close = _http_close;
http->base.reset = esp_gmf_io_http_reset;
if (config->dir == ESP_GMF_IO_DIR_WRITER) {
http->base.acquire_write = _http_acquire_write;
http->base.release_write = _http_release_write;
} else if (config->dir == ESP_GMF_IO_DIR_READER) {
http->base.acquire_read = _http_acquire_read;
http->base.release_read = _http_release_read;
} else {
ESP_LOGE(TAG, "Does not set read or write function");
ret = ESP_GMF_ERR_NOT_SUPPORT;
goto _http_init_fail;
}
esp_gmf_io_cfg_t io_cfg = {
.thread.stack = config->task_stack,
.thread.prio = config->task_prio,
.thread.core = config->task_core,
.thread.stack_in_ext = config->stack_in_ext,
};
ret = esp_gmf_io_init(&http->base, &io_cfg);
if(ret != ESP_GMF_ERR_OK) {
goto _http_init_fail;
}
*io = obj;
ESP_LOGD(TAG, "Initialization, %s-%p", OBJ_GET_TAG(http), http);
return ESP_GMF_ERR_OK;
_http_init_fail:
esp_gmf_obj_delete(obj);
return ret;
}
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_io/esp_gmf_io_i2s_pdm.c
+212
View File
@@ -0,0 +1,212 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#include <stdio.h>
#include <string.h>
#include "esp_gmf_io_i2s_pdm.h"
#include "esp_gmf_oal_mem.h"
#include "esp_log.h"
#include "freertos/FreeRTOS.h"
#include "freertos/event_groups.h"
#define PDM_TX_DONE_BIT BIT(0)
/**
* @brief I2S pdm io context in GMF
*/
typedef struct {
esp_gmf_io_t base; /*!< The GMF i2s pdm io handle */
bool is_open; /*!< The flag of whether opened */
EventGroupHandle_t pdm_event; /*!< The event group handle of i2s pdm */
} i2s_pdm_io_stream_t;
static const char *TAG = "ESP_GMF_IIS_PDM";
bool _i2s_pdm_tx_done_callback(i2s_chan_handle_t handle, i2s_event_data_t *event, void *user_ctx)
{
EventGroupHandle_t evt = (EventGroupHandle_t)user_ctx;
BaseType_t xHigherPriorityTaskWoken = pdFALSE;
BaseType_t xResult = xEventGroupSetBitsFromISR(evt, PDM_TX_DONE_BIT, &xHigherPriorityTaskWoken);
return xResult;
}
static esp_gmf_err_t _i2s_pdm_new(void *cfg, esp_gmf_obj_handle_t *io)
{
return esp_gmf_io_i2s_pdm_init(cfg, io);
}
static esp_gmf_err_t _i2s_pdm_open(esp_gmf_io_handle_t io)
{
i2s_pdm_io_stream_t *i2s_pdm_io = (i2s_pdm_io_stream_t *)io;
i2s_pdm_io_cfg_t *cfg = (i2s_pdm_io_cfg_t *)OBJ_GET_CFG(i2s_pdm_io);
ESP_GMF_NULL_CHECK(TAG, cfg, return ESP_GMF_ERR_FAIL;);
if (cfg->pdm_chan == NULL) {
ESP_LOGE(TAG, "There is no activated I2S driver handle");
return ESP_GMF_ERR_FAIL;
}
if (cfg->dir == ESP_GMF_IO_DIR_WRITER) {
i2s_pdm_io->pdm_event = xEventGroupCreate();
ESP_GMF_CHECK(TAG, i2s_pdm_io->pdm_event, return ESP_GMF_ERR_MEMORY_LACK, "Failed to create i2s pdm event");
i2s_event_callbacks_t cbs = {
.on_recv = NULL,
.on_recv_q_ovf = NULL,
.on_sent = NULL,
.on_send_q_ovf = _i2s_pdm_tx_done_callback,
};
i2s_channel_register_event_callback(cfg->pdm_chan, &cbs, i2s_pdm_io->pdm_event);
}
i2s_channel_enable(cfg->pdm_chan);
i2s_pdm_io->is_open = true;
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_io_t _i2s_pdm_acquire_read(esp_gmf_io_handle_t handle, void *payload, uint32_t wanted_size, int block_ticks)
{
i2s_pdm_io_stream_t *i2s_pdm_io = (i2s_pdm_io_stream_t *)handle;
esp_gmf_payload_t *pload = (esp_gmf_payload_t *)payload;
size_t rlen = 0;
i2s_pdm_io_cfg_t *cfg = (i2s_pdm_io_cfg_t *)OBJ_GET_CFG(i2s_pdm_io);
ESP_GMF_NULL_CHECK(TAG, cfg, return ESP_GMF_IO_FAIL;);
if (i2s_channel_read(cfg->pdm_chan, pload->buf, wanted_size, &rlen, ESP_GMF_MAX_DELAY) != ESP_GMF_ERR_OK) {
ESP_LOGE(TAG, "Read I2S data error, wanted: %ld", wanted_size);
return ESP_GMF_IO_FAIL;
}
pload->valid_size = rlen;
ESP_LOGD(TAG, "Read len: %d", rlen);
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_io_t _i2s_pdm_release_read(esp_gmf_io_handle_t handle, void *payload, int block_ticks)
{
i2s_pdm_io_stream_t *i2s_pdm_io = (i2s_pdm_io_stream_t *)handle;
esp_gmf_payload_t *pload = (esp_gmf_payload_t *)payload;
esp_gmf_info_file_t info = {0};
esp_gmf_io_get_info((esp_gmf_io_handle_t)i2s_pdm_io, &info);
ESP_LOGD(TAG, "Update len = %d, pos = %d/%d", pload->valid_size, (int)info.pos, (int)info.size);
esp_gmf_io_update_pos((esp_gmf_io_handle_t)handle, pload->valid_size);
return ESP_GMF_IO_OK;
}
static esp_gmf_err_io_t _i2s_pdm_acquire_write(esp_gmf_io_handle_t handle, void *payload, uint32_t wanted_size, int block_ticks)
{
return ESP_GMF_IO_OK;
}
static esp_gmf_err_io_t _i2s_pdm_release_write(esp_gmf_io_handle_t handle, void *payload, int block_ticks)
{
i2s_pdm_io_stream_t *i2s_pdm_io = (i2s_pdm_io_stream_t *)handle;
esp_gmf_payload_t *pload = (esp_gmf_payload_t *)payload;
size_t wlen = 0;
i2s_pdm_io_cfg_t *cfg = (i2s_pdm_io_cfg_t *)OBJ_GET_CFG(i2s_pdm_io);
ESP_GMF_NULL_CHECK(TAG, cfg, return ESP_GMF_IO_FAIL;);
if (i2s_channel_write(cfg->pdm_chan, pload->buf, pload->valid_size, &wlen, ESP_GMF_MAX_DELAY) != ESP_GMF_ERR_OK) {
ESP_LOGE(TAG, "I2S write failed, valid: %d", pload->valid_size);
return ESP_GMF_IO_FAIL;
}
esp_gmf_info_file_t info = {0};
if (pload->is_done && i2s_pdm_io->pdm_event) {
ESP_LOGE(TAG, "Clear the PDM_TX_DONE_BIT, len = %d", pload->valid_size);
xEventGroupClearBits(i2s_pdm_io->pdm_event, PDM_TX_DONE_BIT);
}
if (wlen > 0) {
esp_gmf_io_update_pos((esp_gmf_io_handle_t)handle, wlen);
}
esp_gmf_io_get_info((esp_gmf_io_handle_t)i2s_pdm_io, &info);
ESP_LOGD(TAG, "Write len = %d, pos = %d/%d", pload->valid_size, (int)info.pos, (int)info.size);
return ESP_GMF_IO_OK;
}
static esp_gmf_err_t _i2s_pdm_seek(esp_gmf_io_handle_t io, uint64_t seek_byte_pos)
{
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t _i2s_pdm_reset(esp_gmf_io_handle_t io)
{
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t _i2s_pdm_close(esp_gmf_io_handle_t io)
{
i2s_pdm_io_stream_t *i2s_pdm_io = (i2s_pdm_io_stream_t *)io;
esp_gmf_info_file_t info = {0};
i2s_pdm_io_cfg_t *cfg = (i2s_pdm_io_cfg_t *)OBJ_GET_CFG(i2s_pdm_io);
ESP_GMF_NULL_CHECK(TAG, cfg, return ESP_GMF_IO_FAIL;);
esp_gmf_io_get_info((esp_gmf_io_handle_t)i2s_pdm_io, &info);
ESP_LOGI(TAG, "Try to close, %p, pos = %d/%d, pdm_event: %p", i2s_pdm_io, (int)info.pos, (int)info.size, i2s_pdm_io->pdm_event);
if (i2s_pdm_io->pdm_event) {
xEventGroupWaitBits(i2s_pdm_io->pdm_event, PDM_TX_DONE_BIT, pdTRUE, pdFALSE, ESP_GMF_MAX_DELAY);
}
ESP_LOGI(TAG, "CLose, %p, pos = %d/%d", i2s_pdm_io, (int)info.pos, (int)info.size);
if (i2s_pdm_io->is_open) {
i2s_pdm_io->is_open = false;
}
esp_gmf_io_set_pos((esp_gmf_io_handle_t)io, 0);
i2s_channel_disable(cfg->pdm_chan);
return ESP_GMF_ERR_OK;
}
static esp_gmf_err_t _i2s_pdm_delete(esp_gmf_io_handle_t io)
{
i2s_pdm_io_stream_t *i2s_pdm_io = (i2s_pdm_io_stream_t *)io;
ESP_LOGD(TAG, "Delete, %s-%p", OBJ_GET_TAG(i2s_pdm_io), i2s_pdm_io);
if (i2s_pdm_io->pdm_event) {
vEventGroupDelete(i2s_pdm_io->pdm_event);
}
void *cfg = OBJ_GET_CFG(io);
if (cfg) {
esp_gmf_oal_free(cfg);
}
esp_gmf_io_deinit(io);
esp_gmf_oal_free(i2s_pdm_io);
return ESP_GMF_ERR_OK;
}
esp_gmf_err_t esp_gmf_io_i2s_pdm_init(i2s_pdm_io_cfg_t *config, esp_gmf_io_handle_t *io)
{
ESP_GMF_NULL_CHECK(TAG, config, {return ESP_GMF_ERR_INVALID_ARG;});
ESP_GMF_NULL_CHECK(TAG, io, {return ESP_GMF_ERR_INVALID_ARG;});
*io = NULL;
esp_gmf_err_t ret = ESP_GMF_ERR_OK;
i2s_pdm_io_stream_t *i2s_pdm_io = esp_gmf_oal_calloc(1, sizeof(i2s_pdm_io_stream_t));
ESP_GMF_MEM_VERIFY(TAG, i2s_pdm_io, {return ESP_GMF_ERR_MEMORY_LACK;},
"I2s pdm stream", sizeof(i2s_pdm_io_stream_t));
i2s_pdm_io->base.dir = config->dir;
i2s_pdm_io->base.type = ESP_GMF_IO_TYPE_BYTE;
esp_gmf_obj_t *obj = (esp_gmf_obj_t *)i2s_pdm_io;
obj->new_obj = _i2s_pdm_new;
obj->del_obj = _i2s_pdm_delete;
i2s_pdm_io_cfg_t *cfg = esp_gmf_oal_calloc(1, sizeof(*config));
ESP_GMF_MEM_VERIFY(TAG, cfg, {ret = ESP_GMF_ERR_MEMORY_LACK; goto _i2s_pdm_fail;},
"I2s pdm stream configuration", sizeof(*config));
memcpy(cfg, config, sizeof(*config));
esp_gmf_obj_set_config(obj, cfg, sizeof(*config));
ret = esp_gmf_obj_set_tag(obj, (config->name == NULL ? "io_i2s_pdm" : config->name));
ESP_GMF_RET_ON_NOT_OK(TAG, ret, goto _i2s_pdm_fail, "Failed to set obj tag");
i2s_pdm_io->base.close = _i2s_pdm_close;
i2s_pdm_io->base.open = _i2s_pdm_open;
i2s_pdm_io->base.seek = _i2s_pdm_seek;
i2s_pdm_io->base.reset = _i2s_pdm_reset;
esp_gmf_io_init(obj, NULL);
if (cfg->dir == ESP_GMF_IO_DIR_WRITER) {
i2s_pdm_io->base.acquire_write = _i2s_pdm_acquire_write;
i2s_pdm_io->base.release_write = _i2s_pdm_release_write;
} else if (cfg->dir == ESP_GMF_IO_DIR_READER) {
i2s_pdm_io->base.acquire_read = _i2s_pdm_acquire_read;
i2s_pdm_io->base.release_read = _i2s_pdm_release_read;
} else {
ESP_LOGW(TAG, "Does not set read or write function");
ret = ESP_GMF_ERR_NOT_SUPPORT;
goto _i2s_pdm_fail;
}
*io = obj;
ESP_LOGD(TAG, "Initialization, %s-%p", OBJ_GET_TAG(obj), i2s_pdm_io);
return ESP_GMF_ERR_OK;
_i2s_pdm_fail:
esp_gmf_obj_delete(obj);
return ret;
}
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_io/http_lib/gzip/gzip_miniz.c
+310
View File
@@ -0,0 +1,310 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#include <stdint.h>
#include <stdlib.h>
#include <string.h>
#include <stdbool.h>
#include "esp_log.h"
#include "gzip_miniz.h"
#include "miniz_inflate.h"
#define TAG "GZIP_MINIZ"
#define GZIP_HEADER_SIZE (10)
typedef struct {
gzip_miniz_cfg_t cfg;
uint8_t *chunk_ptr;
int chunk_filled;
int chunk_consumed;
bool unzip_error;
bool first_data;
bool final_data;
int head_flag;
int extra_len;
int head_filled;
mz_stream s;
} gzip_miniz_t;
static bool verify_gzip_header(gzip_miniz_t *zip, uint8_t *data, int size)
{
if (size < GZIP_HEADER_SIZE) {
return false;
}
if (data[0] != 0x1F || data[1] != 0x8B || data[2] != 0x8) {
return false;
}
zip->head_flag = data[3];
return true;
}
static int gzip_miniz_skip_head(gzip_miniz_t *zip, uint8_t *data, int size)
{
if (zip->head_flag == 0) {
return 0;
}
int org_size = size;
if (zip->head_flag & 4) {
// 2 bytes extra len
if (zip->head_filled > 2) {
return -1;
}
if (zip->head_filled + size >= 2) {
int used = 2 - zip->head_filled;
if (used) {
size -= used;
while (used--) {
zip->extra_len = zip->extra_len + (*data << (zip->head_filled * 8));
zip->head_filled++;
data++;
}
}
if (size >= zip->extra_len) {
size -= zip->extra_len;
data += zip->extra_len;
zip->extra_len = 0;
zip->head_flag &= ~4;
zip->head_filled = 0;
} else {
zip->extra_len -= size;
return org_size;
}
} else {
for (int i = 0; i < size; i++) {
zip->extra_len = zip->extra_len + (*data << (zip->head_filled * 8));
zip->head_filled++;
data++;
}
zip->head_filled += size;
return org_size;
}
}
if (zip->head_flag & 8) {
// name
while (size) {
size--;
data++;
if (*(data - 1) == '\0') {
zip->head_flag &= ~8;
break;
}
}
if (size == 0) {
return org_size;
}
}
if (zip->head_flag & 0x10) {
// comment
while (size) {
size--;
data++;
if (*(data - 1) == '\0') {
zip->head_flag &= ~0x10;
break;
}
}
if (size == 0) {
return org_size;
}
}
if (zip->head_flag & 0x2) {
// CRC16
if (zip->head_filled > 2) {
return -1;
}
if (zip->head_filled + size >= 2) {
int used = 2 - zip->head_filled;
size -= used;
data += used;
zip->head_flag &= ~0x2;
} else {
zip->head_filled += size;
return org_size;
}
}
return org_size - size;
}
gzip_miniz_handle_t gzip_miniz_init(gzip_miniz_cfg_t *cfg)
{
if (cfg->read_cb == NULL) {
ESP_LOGE(TAG, "Read callback must be provided");
return NULL;
}
gzip_miniz_t *zip = (gzip_miniz_t *) calloc(1, sizeof(gzip_miniz_t));
if (zip == NULL) {
ESP_LOGE(TAG, "No memory for instance");
return NULL;
}
zip->cfg = *cfg;
int chunk_size = cfg->chunk_size ? cfg->chunk_size : 32;
zip->chunk_ptr = (uint8_t *) malloc(chunk_size);
if (zip->chunk_ptr == NULL) {
free(zip);
ESP_LOGE(TAG, "No memory for chunk");
return NULL;
}
zip->cfg.chunk_size = chunk_size;
zip->first_data = true;
mz_inflateInit2(&zip->s, -MZ_DEFAULT_WINDOW_BITS);
return (gzip_miniz_handle_t)zip;
}
int gzip_miniz_read(gzip_miniz_handle_t h, uint8_t *out, int out_size)
{
gzip_miniz_t *zip = (gzip_miniz_t *) h;
if (zip == NULL) {
return -1;
}
if (zip->unzip_error) {
return -2;
}
if (zip->final_data) {
return 0;
}
int size = 0;
if (zip->first_data == true) {
zip->first_data = false;
int size = zip->cfg.read_cb(zip->chunk_ptr, zip->cfg.chunk_size, zip->cfg.ctx);
if (size < 0) {
zip->unzip_error = true;
return -1;
}
zip->chunk_filled = size;
if (verify_gzip_header(zip, zip->chunk_ptr, zip->chunk_filled) == false) {
zip->unzip_error = true;
ESP_LOGE(TAG, "Wrong data not match gzip header");
return -1;
}
zip->chunk_consumed = GZIP_HEADER_SIZE;
}
while (1) {
uint8_t *data = zip->chunk_ptr + zip->chunk_consumed;
size = zip->chunk_filled - zip->chunk_consumed;
if (size) {
int skip = gzip_miniz_skip_head(zip, data, size);
if (skip == 0) {
break;
}
zip->chunk_consumed += skip;
if (size > skip) {
break;
}
}
size = zip->cfg.read_cb(zip->chunk_ptr, zip->cfg.chunk_size, zip->cfg.ctx);
if (size < 0) {
zip->unzip_error = true;
ESP_LOGE(TAG, "Fail to read data");
return -1;
}
if (size == 0) {
zip->final_data = true;
return 0;
}
zip->chunk_filled = size;
zip->chunk_consumed = 0;
}
int org_size = out_size;
// set input buffer
mz_stream *s = &zip->s;
s->next_in = zip->chunk_ptr + zip->chunk_consumed;
s->avail_in = zip->chunk_filled - zip->chunk_consumed;
s->next_out = out;
s->avail_out = out_size;
while (1) {
int ret = mz_inflate(s, MZ_SYNC_FLUSH);
if (ret < 0) {
zip->unzip_error = true;
ESP_LOGE(TAG, "Fail to inflate ret %d", ret);
break;
}
int out_consume = out_size - s->avail_out;
out += out_consume;
out_size -= out_consume;
if (ret == MZ_STREAM_END) {
zip->final_data = true;
break;
}
if (s->avail_out == 0) {
// All output consumed
int org_size = zip->chunk_filled - zip->chunk_consumed;
zip->chunk_consumed += (org_size - s->avail_in);
break;
}
if (s->avail_in == 0) {
size = zip->cfg.read_cb(zip->chunk_ptr, zip->cfg.chunk_size, zip->cfg.ctx);
if (size < 0) {
zip->unzip_error = true;
ESP_LOGE(TAG, "Fail to read data");
return -1;
}
zip->chunk_filled = size;
zip->chunk_consumed = 0;
// Update pointer
s->next_in = zip->chunk_ptr;
s->avail_in = zip->chunk_filled - zip->chunk_consumed;
s->next_out = out;
s->avail_out = out_size;
} else {
// Impossible case
break;
}
}
return org_size - out_size;
}
int gzip_miniz_deinit(gzip_miniz_handle_t h)
{
gzip_miniz_t *zip = (gzip_miniz_t *) h;
if (zip == NULL) {
return -1;
}
mz_inflateEnd(&zip->s);
free(zip->chunk_ptr);
free(zip);
return 0;
}
int gzip_miniz_zip(const uint8_t *input, size_t input_size, uint8_t *out, int out_size)
{
int pos = 0;
const uint8_t header[10] = {0x1F, 0x8B, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0x03};
memcpy(out, header, 10);
pos += 10;
mz_stream stream = {0};
if (mz_deflateInit2(&stream, MZ_BEST_COMPRESSION, MZ_DEFLATED, -MZ_DEFAULT_WINDOW_BITS, 1, MZ_DEFAULT_STRATEGY) != MZ_OK) {
ESP_LOGE(TAG, "Failed to init deflate");
return -1;
}
stream.next_in = (const unsigned char *)input;
stream.avail_in = input_size;
int ret;
do {
stream.next_out = out + pos;
stream.avail_out = out_size - pos;
ret = mz_deflate(&stream, MZ_FINISH);
if (ret != MZ_OK && ret != MZ_STREAM_END) {
ESP_LOGE(TAG, "Failed to deflate ret %d", ret);
mz_deflateEnd(&stream);
return -2;
}
pos += (out_size - pos) - stream.avail_out;
} while (ret != MZ_STREAM_END);
// Clean up compression stream
mz_deflateEnd(&stream);
uint32_t crc_value = mz_crc32(0, input, input_size);
memcpy(out + pos, &crc_value, 4);
pos += 4;
memcpy(out + pos, &input_size, 4);
pos += 4;
return pos;
}
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_io/http_lib/gzip/include/gzip_miniz.h
+71
View File
@@ -0,0 +1,71 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#ifndef _GZIP_MINIZ_H_
#define _GZIP_MINIZ_H_
#ifdef __cplusplus
extern "C" {
#endif
/**
* @brief Configuration for gzip using miniz library
*/
typedef struct {
int (*read_cb)(uint8_t *data, int size, void *ctx); /*!< Read callback return size being read */
int chunk_size; /*!< Chunk size default 32 if set to 0 */
void *ctx; /*!< Read context */
} gzip_miniz_cfg_t;
/**
* @brief Handle for gzip
*/
typedef void *gzip_miniz_handle_t;
/**
* @brief Initialize for gzip using miniz
* @param cfg: Configuration for gzip using miniz
* @return NULL: Input parameter wrong or no memory
* Others: Handle for gzip inflate operation
*/
gzip_miniz_handle_t gzip_miniz_init(gzip_miniz_cfg_t *cfg);
/**
* @brief Inflate and read data
* @param out: Data to read after inflated
* @param out_size: Data size to read
* @return >= 0: Data size being read
* -1: Wrong input parameter or wrong data
* -2: Inflate error by miniz
*/
int gzip_miniz_read(gzip_miniz_handle_t zip, uint8_t *out, int out_size);
/**
* @brief Deinitialize gzip using miniz
* @param zip: Handle for gzip
* @return 0: On success
* -1: Input parameter wrong
*/
int gzip_miniz_deinit(gzip_miniz_handle_t zip);
/**
* @brief Zip data into gzip format
* @param in: Data to be zipped
* @param in_size: Data size to be zipped
* @param in_size: Data size to be zipped
* @param out: Store output gzip format data
* @param out_size: Zipped data size
* @return >= 0: On success
* -1: Fail to init gzip deflate
* -2: Fail to deflate
*/
int gzip_miniz_zip(const uint8_t* in, size_t in_size, uint8_t *out, int out_size);
#ifdef __cplusplus
}
#endif
#endif
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_io/http_lib/gzip/miniz_inflate.h
+365
View File
@@ -0,0 +1,365 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#ifndef _MINIZ_INFLATE_H
#define _MINIZ_INFLATE_H
#include <string.h>
#include "esp_idf_version.h"
#if (ESP_IDF_VERSION_MAJOR == 4) && (ESP_IDF_VERSION_MINOR < 3)
#include "esp32/rom/miniz.h"
#elif (ESP_IDF_VERSION >= ESP_IDF_VERSION_VAL(4, 0, 0) && ESP_IDF_VERSION < ESP_IDF_VERSION_VAL(5, 3, 0))
#include "rom/miniz.h"
#else
#include "miniz.h"
#endif
// Add the API missing in miniz of ROM code
// Original code can be gotten from: https://github.com/richgel999/miniz/blob/master/miniz.c
#ifdef MINIZ_NO_ZLIB_APIS
// Flush values. For typical usage you only need MZ_NO_FLUSH and MZ_FINISH. The other values are for advanced use (refer
// to the zlib docs).
enum { MZ_NO_FLUSH = 0, MZ_PARTIAL_FLUSH = 1, MZ_SYNC_FLUSH = 2, MZ_FULL_FLUSH = 3, MZ_FINISH = 4, MZ_BLOCK = 5 };
// Return status codes. MZ_PARAM_ERROR is non-standard.
enum {
MZ_OK = 0,
MZ_STREAM_END = 1,
MZ_NEED_DICT = 2,
MZ_ERRNO = -1,
MZ_STREAM_ERROR = -2,
MZ_DATA_ERROR = -3,
MZ_MEM_ERROR = -4,
MZ_BUF_ERROR = -5,
MZ_VERSION_ERROR = -6,
MZ_PARAM_ERROR = -10000
};
// Compression levels: 0-9 are the standard zlib-style levels, 10 is best possible compression (not zlib compatible, and
// may be very slow), MZ_DEFAULT_COMPRESSION=MZ_DEFAULT_LEVEL.
enum {
MZ_NO_COMPRESSION = 0,
MZ_BEST_SPEED = 1,
MZ_BEST_COMPRESSION = 9,
MZ_UBER_COMPRESSION = 10,
MZ_DEFAULT_LEVEL = 6,
MZ_DEFAULT_COMPRESSION = -1
};
// Window bits
#define MZ_DEFAULT_WINDOW_BITS 15
#define MZ_MIN(a, b) (a > b ? b : a)
struct mz_internal_state;
// Compression/decompression stream struct.
typedef struct mz_stream_s {
const unsigned char *next_in; // pointer to next byte to read
unsigned int avail_in; // number of bytes available at next_in
mz_ulong total_in; // total number of bytes consumed so far
unsigned char *next_out; // pointer to next byte to write
unsigned int avail_out; // number of bytes that can be written to next_out
mz_ulong total_out; // total number of bytes produced so far
char *msg; // error msg (unused)
struct mz_internal_state *state; // internal state, allocated by zalloc/zfree
int data_type; // data_type (unused)
mz_ulong adler; // adler32 of the source or uncompressed data
mz_ulong crc32; // crc32 of the source or uncompressed data
} mz_stream;
typedef mz_stream *mz_streamp;
typedef struct {
tinfl_decompressor m_decomp;
mz_uint m_dict_ofs, m_dict_avail, m_first_call, m_has_flushed;
int m_window_bits;
mz_uint8 m_dict[TINFL_LZ_DICT_SIZE];
tinfl_status m_last_status;
} inflate_state;
static int mz_inflateInit2(mz_streamp pStream, int window_bits)
{
inflate_state *pDecomp;
if (!pStream)
return MZ_STREAM_ERROR;
if ((window_bits != MZ_DEFAULT_WINDOW_BITS) && (-window_bits != MZ_DEFAULT_WINDOW_BITS))
return MZ_PARAM_ERROR;
pStream->data_type = 0;
pStream->adler = 0;
pStream->crc32 = 0;
pStream->msg = NULL;
pStream->total_in = 0;
pStream->total_out = 0;
pDecomp = (inflate_state *) calloc(1, sizeof(inflate_state));
if (!pDecomp)
return MZ_MEM_ERROR;
pStream->state = (struct mz_internal_state *) pDecomp;
tinfl_init(&pDecomp->m_decomp);
pDecomp->m_dict_ofs = 0;
pDecomp->m_dict_avail = 0;
pDecomp->m_last_status = TINFL_STATUS_NEEDS_MORE_INPUT;
pDecomp->m_first_call = 1;
pDecomp->m_has_flushed = 0;
pDecomp->m_window_bits = window_bits;
return MZ_OK;
}
static int mz_inflate(mz_streamp pStream, int flush)
{
inflate_state *pState;
mz_uint n, first_call, decomp_flags = TINFL_FLAG_COMPUTE_ADLER32;
size_t in_bytes, out_bytes, orig_avail_in;
tinfl_status status;
if ((!pStream) || (!pStream->state))
return MZ_STREAM_ERROR;
if (flush == MZ_PARTIAL_FLUSH)
flush = MZ_SYNC_FLUSH;
if ((flush) && (flush != MZ_SYNC_FLUSH) && (flush != MZ_FINISH))
return MZ_STREAM_ERROR;
pState = (inflate_state *) pStream->state;
if (pState->m_window_bits > 0)
decomp_flags |= TINFL_FLAG_PARSE_ZLIB_HEADER;
orig_avail_in = pStream->avail_in;
first_call = pState->m_first_call;
pState->m_first_call = 0;
if (pState->m_last_status < 0)
return MZ_DATA_ERROR;
if (pState->m_has_flushed && (flush != MZ_FINISH))
return MZ_STREAM_ERROR;
pState->m_has_flushed |= (flush == MZ_FINISH);
if ((flush == MZ_FINISH) && (first_call)) {
// MZ_FINISH on the first call implies that the input and output buffers are large enough to hold the entire
// compressed/decompressed file.
decomp_flags |= TINFL_FLAG_USING_NON_WRAPPING_OUTPUT_BUF;
in_bytes = pStream->avail_in;
out_bytes = pStream->avail_out;
status = tinfl_decompress(&pState->m_decomp, pStream->next_in, &in_bytes, pStream->next_out, pStream->next_out,
&out_bytes, decomp_flags);
pState->m_last_status = status;
pStream->next_in += (mz_uint) in_bytes;
pStream->avail_in -= (mz_uint) in_bytes;
pStream->total_in += (mz_uint) in_bytes;
pStream->adler = tinfl_get_adler32(&pState->m_decomp);
pStream->next_out += (mz_uint) out_bytes;
pStream->avail_out -= (mz_uint) out_bytes;
pStream->total_out += (mz_uint) out_bytes;
if (status < 0)
return MZ_DATA_ERROR;
else if (status != TINFL_STATUS_DONE) {
pState->m_last_status = TINFL_STATUS_FAILED;
return MZ_BUF_ERROR;
}
return MZ_STREAM_END;
}
// flush != MZ_FINISH then we must assume there's more input.
if (flush != MZ_FINISH)
decomp_flags |= TINFL_FLAG_HAS_MORE_INPUT;
if (pState->m_dict_avail) {
n = MZ_MIN(pState->m_dict_avail, pStream->avail_out);
memcpy(pStream->next_out, pState->m_dict + pState->m_dict_ofs, n);
pStream->next_out += n;
pStream->avail_out -= n;
pStream->total_out += n;
pState->m_dict_avail -= n;
pState->m_dict_ofs = (pState->m_dict_ofs + n) & (TINFL_LZ_DICT_SIZE - 1);
return ((pState->m_last_status == TINFL_STATUS_DONE) && (!pState->m_dict_avail)) ? MZ_STREAM_END : MZ_OK;
}
for (;;) {
in_bytes = pStream->avail_in;
out_bytes = TINFL_LZ_DICT_SIZE - pState->m_dict_ofs;
status = tinfl_decompress(&pState->m_decomp, pStream->next_in, &in_bytes, pState->m_dict,
pState->m_dict + pState->m_dict_ofs, &out_bytes, decomp_flags);
pState->m_last_status = status;
pStream->next_in += (mz_uint) in_bytes;
pStream->avail_in -= (mz_uint) in_bytes;
pStream->total_in += (mz_uint) in_bytes;
pStream->adler = tinfl_get_adler32(&pState->m_decomp);
pState->m_dict_avail = (mz_uint) out_bytes;
n = MZ_MIN(pState->m_dict_avail, pStream->avail_out);
memcpy(pStream->next_out, pState->m_dict + pState->m_dict_ofs, n);
pStream->next_out += n;
pStream->avail_out -= n;
pStream->total_out += n;
pState->m_dict_avail -= n;
pState->m_dict_ofs = (pState->m_dict_ofs + n) & (TINFL_LZ_DICT_SIZE - 1);
if (status < 0)
return MZ_DATA_ERROR; // Stream is corrupted (there could be some uncompressed data left in the output
// dictionary - oh well).
else if ((status == TINFL_STATUS_NEEDS_MORE_INPUT) && (!orig_avail_in))
return MZ_BUF_ERROR; // Signal caller that we can't make forward progress without supplying more input or by
// setting flush to MZ_FINISH.
else if (flush == MZ_FINISH) {
// The output buffer MUST be large to hold the remaining uncompressed data when flush==MZ_FINISH.
if (status == TINFL_STATUS_DONE)
return pState->m_dict_avail ? MZ_BUF_ERROR : MZ_STREAM_END;
// status here must be TINFL_STATUS_HAS_MORE_OUTPUT, which means there's at least 1 more byte on the way. If
// there's no more room left in the output buffer then something is wrong.
else if (!pStream->avail_out)
return MZ_BUF_ERROR;
} else if ((status == TINFL_STATUS_DONE) || (!pStream->avail_in) || (!pStream->avail_out) ||
(pState->m_dict_avail))
break;
}
return ((status == TINFL_STATUS_DONE) && (!pState->m_dict_avail)) ? MZ_STREAM_END : MZ_OK;
}
static int mz_inflateEnd(mz_streamp pStream)
{
if (!pStream)
return MZ_STREAM_ERROR;
if (pStream->state) {
free(pStream->state);
pStream->state = NULL;
}
return MZ_OK;
}
static int mz_deflateEnd(mz_streamp pStream)
{
if (!pStream)
return MZ_STREAM_ERROR;
if (pStream->state)
{
free(pStream->state);
pStream->state = NULL;
}
return MZ_OK;
}
static int tdefl_create_comp_flags_from_zip_params(int level, int window_bits, int strategy)
{
static const mz_uint s_tdefl_num_probes[11] = { 0, 1, 6, 32, 16, 32, 128, 256, 512, 768, 1500 };
mz_uint comp_flags = s_tdefl_num_probes[(level >= 0) ? MZ_MIN(10, level) : MZ_DEFAULT_LEVEL] | ((level <= 3) ? TDEFL_GREEDY_PARSING_FLAG : 0);
if (window_bits > 0)
comp_flags |= TDEFL_WRITE_ZLIB_HEADER;
if (!level)
comp_flags |= TDEFL_FORCE_ALL_RAW_BLOCKS;
else if (strategy == MZ_FILTERED)
comp_flags |= TDEFL_FILTER_MATCHES;
else if (strategy == MZ_HUFFMAN_ONLY)
comp_flags &= ~TDEFL_MAX_PROBES_MASK;
else if (strategy == MZ_FIXED)
comp_flags |= TDEFL_FORCE_ALL_STATIC_BLOCKS;
else if (strategy == MZ_RLE)
comp_flags |= TDEFL_RLE_MATCHES;
return comp_flags;
}
static int mz_deflateInit2(mz_streamp pStream, int level, int method, int window_bits, int mem_level, int strategy)
{
tdefl_compressor *pComp;
mz_uint comp_flags = TDEFL_COMPUTE_ADLER32 | tdefl_create_comp_flags_from_zip_params(level, window_bits, strategy);
if (!pStream)
return MZ_STREAM_ERROR;
if ((method != MZ_DEFLATED) || ((mem_level < 1) || (mem_level > 9)) || ((window_bits != MZ_DEFAULT_WINDOW_BITS) && (-window_bits != MZ_DEFAULT_WINDOW_BITS)))
return MZ_PARAM_ERROR;
pStream->data_type = 0;
pStream->adler = MZ_ADLER32_INIT;
pStream->msg = NULL;
pStream->total_in = 0;
pStream->total_out = 0;
pComp = (tdefl_compressor *)calloc(1, sizeof(tdefl_compressor));
if (!pComp)
return MZ_MEM_ERROR;
pStream->state = (struct mz_internal_state *)pComp;
if (tdefl_init(pComp, NULL, NULL, comp_flags) != TDEFL_STATUS_OKAY)
{
mz_deflateEnd(pStream);
return MZ_PARAM_ERROR;
}
return MZ_OK;
}
static int mz_deflate(mz_streamp pStream, int flush)
{
size_t in_bytes, out_bytes;
mz_ulong orig_total_in, orig_total_out;
int mz_status = MZ_OK;
if ((!pStream) || (!pStream->state) || (flush < 0) || (flush > MZ_FINISH) || (!pStream->next_out))
return MZ_STREAM_ERROR;
if (!pStream->avail_out)
return MZ_BUF_ERROR;
if (flush == MZ_PARTIAL_FLUSH)
flush = MZ_SYNC_FLUSH;
if (((tdefl_compressor *)pStream->state)->m_prev_return_status == TDEFL_STATUS_DONE)
return (flush == MZ_FINISH) ? MZ_STREAM_END : MZ_BUF_ERROR;
orig_total_in = pStream->total_in;
orig_total_out = pStream->total_out;
for (;;)
{
tdefl_status defl_status;
in_bytes = pStream->avail_in;
out_bytes = pStream->avail_out;
defl_status = tdefl_compress((tdefl_compressor *)pStream->state, pStream->next_in, &in_bytes, pStream->next_out, &out_bytes, (tdefl_flush)flush);
pStream->next_in += (mz_uint)in_bytes;
pStream->avail_in -= (mz_uint)in_bytes;
pStream->total_in += (mz_uint)in_bytes;
pStream->adler = tdefl_get_adler32((tdefl_compressor *)pStream->state);
pStream->next_out += (mz_uint)out_bytes;
pStream->avail_out -= (mz_uint)out_bytes;
pStream->total_out += (mz_uint)out_bytes;
if (defl_status < 0)
{
mz_status = MZ_STREAM_ERROR;
break;
}
else if (defl_status == TDEFL_STATUS_DONE)
{
mz_status = MZ_STREAM_END;
break;
}
else if (!pStream->avail_out)
break;
else if ((!pStream->avail_in) && (flush != MZ_FINISH))
{
if ((flush) || (pStream->total_in != orig_total_in) || (pStream->total_out != orig_total_out))
break;
return MZ_BUF_ERROR; /* Can't make forward progress without some input.
*/
}
}
return mz_status;
}
#endif
#endif
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_io/idf_component.yml
+16
View File
@@ -0,0 +1,16 @@
version: "0.7.6"
description: Espressif GMF IO Module
url: https://github.com/espressif/esp-gmf/tree/main/elements/gmf_io
documentation: "https://github.com/espressif/esp-gmf/blob/main/elements/gmf_io/README.md"
issues: "https://github.com/espressif/esp-gmf/issues"
tags:
- "gmf"
dependencies:
espressif/esp_codec_dev:
require: "public"
version: "^1.4"
espressif/gmf_core:
require: public
version: "^0.7"
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_io/include/esp_gmf_io_embed_flash.h
+66
View File
@@ -0,0 +1,66 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#pragma once
#include "esp_gmf_io.h"
#ifdef __cplusplus
extern "C" {
#endif /* __cplusplus */
/**
* @brief Espressif embed flash configurations, if any entry is zero then the configuration will be set to default values
*/
typedef struct {
int max_files; /*!< IO direction, reader or writer */
const char *name; /*!< Name for this instance */
} embed_flash_io_cfg_t;
#define EMBED_FLASH_CFG_DEFAULT() { \
.max_files = 200, \
.name = NULL, \
}
/**
* @brief Embed information in flash
*/
typedef struct embed_item_info {
const uint8_t *address; /*!< The corresponding address in flash */
int size; /*!< Size of corresponding data */
} embed_item_info_t;
/**
* @brief Initializes the embed flash stream I/O with the provided configuration
*
* @param[in] config Pointer to the embed flash IO configuration
* @param[out] io Pointer to the embed flash IO handle to be initialized
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_INVALID_ARG Invalid configuration provided
* - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory
*/
esp_gmf_err_t esp_gmf_io_embed_flash_init(embed_flash_io_cfg_t *config, esp_gmf_io_handle_t *io);
/**
* @brief Set the embed flash context
* This function mainly provides information about embed flash data
*
* @param[in] io The embed flash element handle
* @param[in] context The embed flash context
* @param[in] max_num The number of embed flash context
*
* @return
* - ESP_GMF_ERR_OK
* - ESP_GMF_ERR_FAIL
*/
esp_gmf_err_t esp_gmf_io_embed_flash_set_context(esp_gmf_io_handle_t io, const embed_item_info_t *context, int max_num);
#ifdef __cplusplus
}
#endif /* __cplusplus */
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_io/include/esp_gmf_io_file.h
+52
View File
@@ -0,0 +1,52 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#pragma once
#include "esp_gmf_io.h"
#ifdef __cplusplus
extern "C" {
#endif /* __cplusplus */
/**
* @brief File IO configurations, if any entry is zero then the configuration will be set to default values
*/
typedef struct {
int dir; /*!< IO direction, reader or writer */
const char *name; /*!< Name for this instance */
int cache_size; /*!< Cache size for file IO operations in bytes. If size <= 512, it will be set to 0.
Note: Larger cache size will improve read and write performance but consume more memory */
int cache_caps; /*!< Cache memory capabilities, if zero then it will be set to MALLOC_CAP_DMA.
Note:
1. If chips have SOC_SDMMC_PSRAM_DMA_CAPABLE capability(such as ESP32P4),
then you can set (MALLOC_CAP_SPIRAM | MALLOC_CAP_DMA) to save SRAM
2. For ESP32, should use (MALLOC_CAP_INTERNAL | MALLOC_CAP_8BIT) or MALLOC_CAP_DMA caps to malloc cache
3. For ESP32Sxx and ESP32Cxx, can also use MALLOC_CAP_INTERNAL caps to malloc cache */
} file_io_cfg_t;
#define FILE_IO_CFG_DEFAULT() { \
.dir = ESP_GMF_IO_DIR_READER, \
.name = NULL, \
}
/**
* @brief Initializes the file stream I/O with the provided configuration
*
* @param[in] config Pointer to the file IO configuration
* @param[out] io Pointer to the file IO handle to be initialized
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_INVALID_ARG Invalid configuration provided
* - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory
*/
esp_gmf_err_t esp_gmf_io_file_init(file_io_cfg_t *config, esp_gmf_io_handle_t *io);
#ifdef __cplusplus
}
#endif /* __cplusplus */
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_io/include/esp_gmf_io_http.h
+141
View File
@@ -0,0 +1,141 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#pragma once
#include "esp_gmf_io.h"
#ifdef __cplusplus
extern "C" {
#endif /* __cplusplus */
#define HTTP_STREAM_TASK_STACK (6 * 1024)
#define HTTP_STREAM_TASK_CORE (0)
#define HTTP_STREAM_TASK_PRIO (10)
#define HTTP_STREAM_RINGBUFFER_SIZE (20 * 1024)
/**
* @brief HTTP Stream hook type
*/
typedef enum {
HTTP_STREAM_PRE_REQUEST = 0x01, /*!< The event handler will be called before HTTP Client making the connection to the server */
HTTP_STREAM_ON_REQUEST, /*!< The event handler will be called when HTTP Client is requesting data,
* If the function return the value (-1: ESP_GMF_ERR_FAIL), HTTP Client will be stopped
* If the function return the value > 0, HTTP Stream will ignore the post_field
* If the function return the value = 0, HTTP Stream continue send data from post_field (if any)
*/
HTTP_STREAM_ON_RESPONSE, /*!< The event handler will be called when HTTP Client is receiving data
* If the function return the value (-1: ESP_GMF_ERR_FAIL), HTTP Client will be stopped
* If the function return the value > 0, HTTP Stream will ignore the read function
* If the function return the value = 0, HTTP Stream continue read data from HTTP Server
*/
HTTP_STREAM_POST_REQUEST, /*!< The event handler will be called after HTTP Client send header and body to the server, before fetching the headers */
HTTP_STREAM_FINISH_REQUEST, /*!< The event handler will be called after HTTP Client fetch the header and ready to read HTTP body */
} http_stream_event_id_t;
/**
* @brief Stream event message
*/
typedef struct {
http_stream_event_id_t event_id; /*!< Event ID */
void *http_client; /*!< Reference to HTTP Client using by this HTTP Stream */
void *buffer; /*!< Reference to Buffer using by the IO */
int buffer_len; /*!< Length of buffer */
void *user_data; /*!< User data context, from `http_io_cfg_t` */
} http_stream_event_msg_t;
typedef int (*http_io_event_handle_t)(http_stream_event_msg_t *msg);
/**
* @brief HTTP Stream configurations
* Default value will be used if any entry is zero
*/
typedef struct {
int task_stack; /*!< Task stack size */
int task_core; /*!< Task running in core (0 or 1) */
int task_prio; /*!< Task priority (based on freeRTOS priority) */
bool stack_in_ext; /*!< Try to allocate stack in external memory */
int dir; /*!< Type of stream */
int out_buf_size; /*!< Size of output buffer */
http_io_event_handle_t event_handle; /*!< The hook function for HTTP Stream */
void *user_data; /*!< User data context */
const char *cert_pem; /*!< SSL server certification, PEM format as string, if the client requires to verify server */
esp_err_t (*crt_bundle_attach)(void *conf); /*!< Function pointer to esp_crt_bundle_attach. Enables the use of certification
bundle for server verification, must be enabled in menuconfig */
} http_io_cfg_t;
#define HTTP_STREAM_CFG_DEFAULT() { \
.task_stack = HTTP_STREAM_TASK_STACK, \
.task_core = HTTP_STREAM_TASK_CORE, \
.task_prio = HTTP_STREAM_TASK_PRIO, \
.stack_in_ext = true, \
.dir = ESP_GMF_IO_DIR_READER, \
.out_buf_size = HTTP_STREAM_RINGBUFFER_SIZE, \
.event_handle = NULL, \
.user_data = NULL, \
.cert_pem = NULL, \
.crt_bundle_attach = NULL, \
}
/**
* @brief Initialize the HTTP stream I/O with the specified configuration
*
* @param[in] config Pointer to an `http_io_cfg_t` structure containing the configuration
* settings for the HTTP I/O
* @param[out] io Pointer to a `esp_gmf_io_handle_t` where the initialized I/O object
* handle will be stored
*
* @return
* - ESP_GMF_ERR_OK Initialization successful
* - ESP_GMF_ERR_INVALID_ARG Invalid argument(s)
* - ESP_GMF_ERR_MEMORY_LACK Insufficient memory for initialization
*/
esp_gmf_err_t esp_gmf_io_http_init(http_io_cfg_t *config, esp_gmf_io_handle_t *io);
/**
* @brief Reset http information.
*
* This function can be used in event_handler of http IO.
* User can call this function to connect to next track in playlist when he/she gets `HTTP_STREAM_FINISH_TRACK` event
*
* @param handle The http IO handle
*
* @return
* - ESP_GMF_ERR_OK On success
* - ESP_GMF_ERR_INVALID_ARG Invalid argument
*/
esp_gmf_err_t esp_gmf_io_http_reset(esp_gmf_io_handle_t handle);
/**
* @brief Set SSL server certification
*
* @note PEM format as string, if the client requires to verify server
*
* @param handle The http IO handle
* @param cert Server certification
*
* @return
* - ESP_GMF_ERR_OK On success
* - ESP_GMF_ERR_INVALID_ARG Invalid argument
*/
esp_gmf_err_t esp_gmf_io_http_set_server_cert(esp_gmf_io_handle_t handle, const char *cert);
/**
* @brief Set the event handle for the HTTP IO
*
* @param handle The http IO handle
* @param event_callback The event callback handle
*
* @return
* - ESP_GMF_ERR_OK on success
* - ESP_GMF_ERR_INVALID_ARG Invalid argument
*/
esp_gmf_err_t esp_gmf_io_http_set_event_callback(esp_gmf_io_handle_t handle, http_io_event_handle_t event_callback);
#ifdef __cplusplus
}
#endif /* __cplusplus */
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_io/include/esp_gmf_io_i2s_pdm.h
+47
View File
@@ -0,0 +1,47 @@
/*
* SPDX-FileCopyrightText: 2025 Espressif Systems (Shanghai) CO., LTD
* SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
*
* See LICENSE file for details.
*/
#pragma once
#include "driver/i2s_pdm.h"
#include "esp_gmf_io.h"
#ifdef __cplusplus
extern "C" {
#endif /* __cplusplus */
/**
* @brief I2S PDM IO configurations, if any entry is zero then the configuration will be set to default values
*/
typedef struct {
i2s_chan_handle_t pdm_chan; /*!< I2S tx channel handler */
int dir; /*!< IO direction, reader or writer */
const char *name; /*!< Name for this instance */
} i2s_pdm_io_cfg_t;
#define ESP_GMF_IO_I2S_PDM_CFG_DEFAULT() { \
.pdm_chan = NULL, \
.dir = ESP_GMF_IO_DIR_READER, \
.name = NULL, \
}
/**
* @brief Initializes the I2S PDM I/O with the provided configuration
*
* @param[in] config Pointer to the file IO configuration
* @param[out] io Pointer to the file IO handle to be initialized
*
* @return
* - ESP_GMF_ERR_OK Success
* - ESP_GMF_ERR_INVALID_ARG Invalid configuration provided
* - ESP_GMF_ERR_MEMORY_LACK Failed to allocate memory
*/
esp_gmf_err_t esp_gmf_io_i2s_pdm_init(i2s_pdm_io_cfg_t *config, esp_gmf_io_handle_t *io);
#ifdef __cplusplus
}
#endif /* __cplusplus */
firmware/echoear-platform-agent/components/esp-gmf/elements/gmf_io/mk_flash_embed_tone.py
+215
View File
@@ -0,0 +1,215 @@
#!/user/bin/env python3
# Espressif Modified MIT License
#
# Copyright (c) 2025 Espressif Systems (Shanghai) CO., LTD
#
# Permission is hereby granted for use EXCLUSIVELY with Espressif Systems products.
# This includes the right to use, copy, modify, merge, publish, distribute, and sublicense
# the Software, subject to the following conditions:
#
# 1. This Software MUST BE USED IN CONJUNCTION WITH ESPRESSIF SYSTEMS PRODUCTS.
# 2. The above copyright notice and this permission notice shall be included in all copies
# or substantial portions of the Software.
# 3. Redistribution of the Software in source or binary form FOR USE WITH NON-ESPRESSIF PRODUCTS
# is strictly prohibited.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
# INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR
# PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE
# FOR ANY CLAIM, DAMAGES, OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
# OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
# DEALINGS IN THE SOFTWARE.
#
# SPDX-License-Identifier: LicenseRef-Espressif-Modified-MIT
import sys
import os
import time
import argparse
import logging
from typing import Dict, Tuple, List
from datetime import datetime
# Configure logging
logging.basicConfig(
level=logging.INFO,
format='%(asctime)s - %(levelname)s - %(message)s'
)
logger = logging.getLogger(__name__)
# Constants
LICENSE_STR = f'''/*
* SPDX-FileCopyrightText: {datetime.now().year} Espressif Systems (Shanghai) CO LTD
*
* SPDX-License-Identifier: Unlicense OR CC0-1.0
*/
'''
GEN_HEAD_FILE_NAME = 'esp_embed_tone.h'
GEN_CMAKE_FILE_NAME = 'esp_embed_tone.cmake'
SUPPORTED_EXTENSIONS = ('.wav', '.mp3', '.aac', '.m4a')
def sanitize_filename(filename: str) -> str:
# Replace invalid characters with underscore
sanitized = filename.replace('-', '_').replace('.', '_').replace(' ', '_')
# Ensure it starts with a letter
if sanitized[0].isdigit():
sanitized = 'tone_' + sanitized
return sanitized
def get_file_info(path: str) -> Dict[str, int]:
try:
file_list = [x for x in os.listdir(path) if x.lower().endswith(SUPPORTED_EXTENSIONS)]
file_list.sort()
if not file_list:
logger.warning(f'No supported audio files found in {path}')
return {}
file_info = {}
for filename in file_list:
try:
size = os.path.getsize(os.path.join(path, filename))
file_info[filename] = size
logger.info(f'Found audio file: {filename} ({size} bytes)')
except OSError as e:
logger.error(f'Error getting size for {filename}: {e}')
continue
return file_info
except OSError as e:
logger.error(f'Error scanning directory {path}: {e}')
return {}
def gen_h_file(file_info: Dict[str, int]) -> Tuple[str, str]:
if not file_info:
logger.error('No file information provided')
return '', ''
# Header file content
h_file = [
'#pragma once',
'',
'/**',
' * @brief Structure for embedding tone information',
' */',
'typedef struct {',
' const uint8_t *address; /*!< Pointer to the embedded tone data */',
' int size; /*!< Size of the tone data in bytes */',
'} esp_embed_tone_t;',
''
]
# CMake file content
cmake_file = ['set(COMPONENT_EMBED_TXTFILES']
# Generate content for each file
enum_entries = []
url_entries = []
tone_entries = []
for idx, (filename, size) in enumerate(file_info.items()):
sanitized = sanitize_filename(filename)
# Add external reference
h_file.extend([
'/**',
f' * @brief External reference to embedded tone data: {filename}',
' */',
f'extern const uint8_t {sanitized}[] asm("_binary_{sanitized}_start");',
''
])
# Add to CMake file
cmake_file.append(f' {filename}')
# Add tone entry
tone_entries.append(
f' [{idx}] = {{\n'
f' .address = {sanitized},\n'
f' .size = {size},\n'
f' }},'
)
# Add enum entry
enum_entries.append(
f' ESP_EMBED_TONE_{sanitized.upper()} = {idx},'
)
# Add URL entry with hyphens replaced by underscores
url_filename = filename.replace('-', '_')
url_entries.append(
f' "embed://tone/{idx}_{url_filename}",'
)
# Complete header file
h_file.extend([
'/**',
' * @brief Array of embedded tone information',
' */',
'esp_embed_tone_t g_esp_embed_tone[] = {',
*tone_entries,
'};',
'',
'/**',
' * @brief Enumeration for tone URLs',
' */',
'enum esp_embed_tone_index {',
*enum_entries,
f' ESP_EMBED_TONE_URL_MAX = {len(file_info)}',
'};',
'',
'/**',
' * @brief Array of tone URLs',
' */',
'const char * esp_embed_tone_url[] = {',
*url_entries,
'};',
''
])
# Complete CMake file
cmake_file.append(')')
return '\n'.join(h_file), ''.join(cmake_file)
def write_file(content: str, filepath: str, is_header: bool = False) -> bool:
try:
if os.path.exists(filepath):
os.remove(filepath)
with open(filepath, 'w+') as f:
if is_header:
f.write(LICENSE_STR)
f.write(content)
logger.info(f'Successfully wrote {filepath}')
return True
except OSError as e:
logger.error(f'Error writing to {filepath}: {e}')
return False
def main():
parser = argparse.ArgumentParser(
description='Generate embedded audio tone header and CMake files',
formatter_class=argparse.ArgumentDefaultsHelpFormatter
)
parser.add_argument(
'-p', '--path',
type=str,
required=True,
help='Base folder containing audio files'
)
parser.add_argument(
'-v', '--verbose',
action='store_true',
help='Enable verbose output'
)
args = parser.parse_args()
if args.verbose:
logger.setLevel(logging.DEBUG)
# Get file information
file_info = get_file_info(args.path)
if not file_info:
logger.error('No valid audio files found')
sys.exit(1)
# Generate files
h_content, cmake_content = gen_h_file(file_info)
# Write files
h_path = os.path.join(args.path, GEN_HEAD_FILE_NAME)
cmake_path = os.path.join(args.path, GEN_CMAKE_FILE_NAME)
if not write_file(h_content, h_path, is_header=True):
sys.exit(1)
if not write_file(cmake_content, cmake_path):
sys.exit(1)
logger.info('Successfully generated embedded tone files')
if __name__ == '__main__':
main()

Some files were not shown because too many files have changed in this diff Show More