|
@@ -0,0 +1,1008 @@
|
|
|
|
|
+# 音频真实时长 + DetailedReport 重播 实施计划
|
|
|
|
|
+
|
|
|
|
|
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
|
|
|
+
|
|
|
|
|
+**Goal:** 让学生语音消息(实时 / 历史 / 报告卡片)显示后端权威的真实时长,并让 DetailedReport 学生卡片可重播 audioUrl;AI 卡片彻底移除时长 + 重播按钮。
|
|
|
|
|
+
|
|
|
|
|
+**Architecture:** 后端 `dialogue_message` 表新增 `audio_duration` 列,HTTP `/speak` 与 WS `/speak-stream` 收到学生 WAV 时同步解析时长写库;该值通过 SSE/WS 的 `transcript` 事件、`/sessions/latest`、`/report` 三个端点透传给前端。前端类型链路 + `useAudioPlayer` URL 模式 + 模板替换写死兜底,DetailedReport 使用同一个 player 实例承载学生卡片重播。
|
|
|
|
|
+
|
|
|
|
|
+**Tech Stack:**
|
|
|
|
|
+- 后端:FastAPI / SQLAlchemy 2.0 async / Python `wave`+`io` / pytest
|
|
|
|
|
+- 前端:Vue 3 `<script setup>` / TypeScript / Vite / 现有 SSE+WS 协议
|
|
|
|
|
+
|
|
|
|
|
+**Spec:** `docs/superpowers/specs/2026-05-07-audio-duration-and-replay-design.md`
|
|
|
|
|
+
|
|
|
|
|
+**Repos:**
|
|
|
|
|
+- 后端 `cococlass-english-speaking-api`:`/Users/buoy/Development/gitrepo/cococlass-english-speaking-api`
|
|
|
|
|
+- 前端 `PPT`:`/Users/buoy/Development/gitrepo/PPT`(当前 working dir)
|
|
|
|
|
+
|
|
|
|
|
+> **跨仓注意**:Task 1–3 在后端 repo 操作;Task 4–9 在前端 repo。每个 commit 都在各自 repo 内完成,不要跨仓 stage。
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## Task 1: 后端 — DB Migration
|
|
|
|
|
+
|
|
|
|
|
+**Files:**
|
|
|
|
|
+- Create: `cococlass-english-speaking-api/migrations/2026-05-07-add-dialogue-message-audio-duration.sql`
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 1.1: 创建 migration SQL**
|
|
|
|
|
+
|
|
|
|
|
+写入完整内容:
|
|
|
|
|
+```sql
|
|
|
|
|
+-- 2026-05-07 add audio_duration to dialogue_message
|
|
|
|
|
+-- 学生语音真实时长(秒),由 /speak 与 /speak-stream 在解析 WAV 时写入。
|
|
|
|
|
+-- Nullable 兼容老数据;老会话保持 NULL,前端显示 --:--。
|
|
|
|
|
+
|
|
|
|
|
+USE speaking;
|
|
|
|
|
+
|
|
|
|
|
+ALTER TABLE dialogue_message
|
|
|
|
|
+ADD COLUMN audio_duration FLOAT NULL AFTER audio_url;
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 1.2: 在本地 dev DB 应用 migration**
|
|
|
|
|
+
|
|
|
|
|
+Run(在后端 repo 根目录):`mysql -u <user> -p speaking < migrations/2026-05-07-add-dialogue-message-audio-duration.sql`
|
|
|
|
|
+
|
|
|
|
|
+Expected: command 返回 0(无错误),`DESCRIBE dialogue_message;` 能看到新增 `audio_duration float YES NULL`。
|
|
|
|
|
+
|
|
|
|
|
+> 如果 dev DB 用的是 SQLite,先 skip 此步——SQLAlchemy `MetaData.create_all` 会按 model 创建。但生产 MySQL 必须手动跑此脚本。
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 1.3: Commit**
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+cd /Users/buoy/Development/gitrepo/cococlass-english-speaking-api
|
|
|
|
|
+git add migrations/2026-05-07-add-dialogue-message-audio-duration.sql
|
|
|
|
|
+git commit -m "$(cat <<'EOF'
|
|
|
|
|
+feat(speaking): migration add dialogue_message.audio_duration
|
|
|
|
|
+
|
|
|
|
|
+Adds nullable FLOAT column to store WAV-parsed audio duration in seconds.
|
|
|
|
|
+Backward-compatible: existing rows remain NULL; surfaces fall back to
|
|
|
|
|
+'--:--' on the frontend.
|
|
|
|
|
+
|
|
|
|
|
+Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|
|
|
|
+EOF
|
|
|
|
|
+)"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## Task 2: 后端 — `audio_utils.parse_wav_duration` (TDD)
|
|
|
|
|
+
|
|
|
|
|
+**Files:**
|
|
|
|
|
+- Create: `cococlass-english-speaking-api/app/service/speaking/audio_utils.py`
|
|
|
|
|
+- Test: `cococlass-english-speaking-api/tests/service/speaking/test_audio_utils.py`
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 2.1: 写失败的测试(5 case)**
|
|
|
|
|
+
|
|
|
|
|
+新建测试文件 `tests/service/speaking/test_audio_utils.py`,写入:
|
|
|
|
|
+
|
|
|
|
|
+```python
|
|
|
|
|
+import io
|
|
|
|
|
+import wave
|
|
|
|
|
+
|
|
|
|
|
+from app.service.speaking.audio_utils import parse_wav_duration
|
|
|
|
|
+
|
|
|
|
|
+
|
|
|
|
|
+def _make_silent_wav(duration_seconds: float, sample_rate: int = 16000) -> bytes:
|
|
|
|
|
+ """Synthesize a silent mono 16-bit WAV blob of the given duration."""
|
|
|
|
|
+ n_frames = int(duration_seconds * sample_rate)
|
|
|
|
|
+ buf = io.BytesIO()
|
|
|
|
|
+ with wave.open(buf, "wb") as wf:
|
|
|
|
|
+ wf.setnchannels(1)
|
|
|
|
|
+ wf.setsampwidth(2)
|
|
|
|
|
+ wf.setframerate(sample_rate)
|
|
|
|
|
+ wf.writeframes(b"\x00\x00" * n_frames)
|
|
|
|
|
+ return buf.getvalue()
|
|
|
|
|
+
|
|
|
|
|
+
|
|
|
|
|
+def test_parse_valid_one_second_wav():
|
|
|
|
|
+ audio_bytes = _make_silent_wav(1.0)
|
|
|
|
|
+ duration = parse_wav_duration(audio_bytes)
|
|
|
|
|
+ assert duration is not None
|
|
|
|
|
+ assert abs(duration - 1.0) < 0.01
|
|
|
|
|
+
|
|
|
|
|
+
|
|
|
|
|
+def test_parse_empty_bytes_returns_none():
|
|
|
|
|
+ assert parse_wav_duration(b"") is None
|
|
|
|
|
+
|
|
|
|
|
+
|
|
|
|
|
+def test_parse_non_riff_magic_returns_none():
|
|
|
|
|
+ # webm-ish header (EBML magic) — must not crash, must return None
|
|
|
|
|
+ fake = b"\x1A\x45\xDF\xA3" + b"\x00" * 100
|
|
|
|
|
+ assert parse_wav_duration(fake) is None
|
|
|
|
|
+
|
|
|
|
|
+
|
|
|
|
|
+def test_parse_truncated_wav_returns_none():
|
|
|
|
|
+ full = _make_silent_wav(1.0)
|
|
|
|
|
+ truncated = full[:30] # less than 44-byte header
|
|
|
|
|
+ assert parse_wav_duration(truncated) is None
|
|
|
|
|
+
|
|
|
|
|
+
|
|
|
|
|
+def test_parse_wav_with_zero_sample_rate_returns_none():
|
|
|
|
|
+ """Construct a valid-magic WAV-like blob with sampleRate=0 in header."""
|
|
|
|
|
+ # RIFF + size + WAVE + fmt + ... patched samplerate
|
|
|
|
|
+ base = _make_silent_wav(1.0)
|
|
|
|
|
+ # In WAV header, samplerate is bytes [24:28] little-endian uint32.
|
|
|
|
|
+ corrupted = bytearray(base)
|
|
|
|
|
+ corrupted[24:28] = b"\x00\x00\x00\x00"
|
|
|
|
|
+ assert parse_wav_duration(bytes(corrupted)) is None
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 2.2: 运行测试看到 fail**
|
|
|
|
|
+
|
|
|
|
|
+Run: `cd /Users/buoy/Development/gitrepo/cococlass-english-speaking-api && pytest tests/service/speaking/test_audio_utils.py -v`
|
|
|
|
|
+Expected: 5 个 test 全 FAIL,错误是 `ModuleNotFoundError: No module named 'app.service.speaking.audio_utils'`。
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 2.3: 实现 audio_utils.py**
|
|
|
|
|
+
|
|
|
|
|
+新建 `app/service/speaking/audio_utils.py`:
|
|
|
|
|
+
|
|
|
|
|
+```python
|
|
|
|
|
+"""WAV duration parsing helper.
|
|
|
|
|
+
|
|
|
|
|
+Defensive utility: NEVER raises. The duration field is non-essential UX
|
|
|
|
|
+metadata; a parse failure must not block audio_url persistence. Callers
|
|
|
|
|
+treat None as 'unknown duration' and pass it through to the frontend.
|
|
|
|
|
+"""
|
|
|
|
|
+import io
|
|
|
|
|
+import logging
|
|
|
|
|
+import struct
|
|
|
|
|
+import wave
|
|
|
|
|
+
|
|
|
|
|
+logger = logging.getLogger(__name__)
|
|
|
|
|
+
|
|
|
|
|
+
|
|
|
|
|
+def parse_wav_duration(audio_bytes: bytes) -> float | None:
|
|
|
|
|
+ """Parse a WAV blob and return duration in seconds, or None on any failure.
|
|
|
|
|
+
|
|
|
|
|
+ Returns None on:
|
|
|
|
|
+ - empty / too-short bytes (< 44 = WAV header length)
|
|
|
|
|
+ - non-RIFF / non-WAVE magic (e.g. webm/opus / corrupt header)
|
|
|
|
|
+ - parse exception inside the wave module
|
|
|
|
|
+ - sampleRate <= 0 (corrupt header that wave module accepts)
|
|
|
|
|
+ """
|
|
|
|
|
+ if len(audio_bytes) < 44:
|
|
|
|
|
+ return None
|
|
|
|
|
+ if audio_bytes[:4] != b"RIFF" or audio_bytes[8:12] != b"WAVE":
|
|
|
|
|
+ return None
|
|
|
|
|
+ try:
|
|
|
|
|
+ with wave.open(io.BytesIO(audio_bytes), "rb") as wf:
|
|
|
|
|
+ frames = wf.getnframes()
|
|
|
|
|
+ rate = wf.getframerate()
|
|
|
|
|
+ if rate <= 0 or frames < 0:
|
|
|
|
|
+ return None
|
|
|
|
|
+ return frames / rate
|
|
|
|
|
+ except (wave.Error, EOFError, struct.error, ValueError) as e:
|
|
|
|
|
+ logger.warning(f"parse_wav_duration failed: {e}")
|
|
|
|
|
+ return None
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 2.4: 运行测试看到 pass**
|
|
|
|
|
+
|
|
|
|
|
+Run: `pytest tests/service/speaking/test_audio_utils.py -v`
|
|
|
|
|
+Expected: 5/5 PASS。
|
|
|
|
|
+
|
|
|
|
|
+> 不在此 task commit。task 3 一起 commit(spec §11 step 2 是单 commit)。
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## Task 3: 后端 — Model + 写入点 + 输出点
|
|
|
|
|
+
|
|
|
|
|
+**Files:**
|
|
|
|
|
+- Modify: `cococlass-english-speaking-api/app/models/dialogue.py:48-62` (DialogueMessage)
|
|
|
|
|
+- Modify: `cococlass-english-speaking-api/app/service/speaking/dialogue_service.py`
|
|
|
|
|
+ - 三个 `transcript` yield:`:206`、`:499`、`:525`
|
|
|
|
|
+ - 一个 student_msg 写入:`:527-535`
|
|
|
|
|
+ - `/sessions/latest` 输出:`:684-693` 附近(rounds dict)
|
|
|
|
|
+ - `/report` 输出:`:921-928` 附近(rounds entry)
|
|
|
|
|
+- Modify: `cococlass-english-speaking-api/app/api/dialogue.py`
|
|
|
|
|
+ - WS transcript yield:`:476-479`
|
|
|
|
|
+ - WS student_msg 写入:`:485-494`
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 3.1: 给 model 加列**
|
|
|
|
|
+
|
|
|
|
|
+编辑 `app/models/dialogue.py`,在 `DialogueMessage` 类的 `audio_url` 字段后插入一行:
|
|
|
|
|
+
|
|
|
|
|
+```python
|
|
|
|
|
+audio_url: Mapped[Optional[str]] = mapped_column(String(512), nullable=True)
|
|
|
|
|
+audio_duration: Mapped[Optional[float]] = mapped_column(Float, nullable=True) # NEW: 秒,由 parse_wav_duration 在 /speak 与 /speak-stream 写入
|
|
|
|
|
+client_turn_id: Mapped[Optional[str]] = mapped_column(String(36), nullable=True)
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+`Float` 已在 `from sqlalchemy import (...)` 顶部 import,无需添加。
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 3.2: HTTP /speak 全 miss 路径写入 audio_duration**
|
|
|
|
|
+
|
|
|
|
|
+编辑 `app/service/speaking/dialogue_service.py`。文件顶部 import 区追加:
|
|
|
|
|
+
|
|
|
|
|
+```python
|
|
|
|
|
+from app.service.speaking.audio_utils import parse_wav_duration
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+找到 `:525-535`,把 transcript yield 与 student_msg 写入按下面顺序改写——**duration 必须在 yield 之前算好**,因为 yield 与 INSERT 都要用它:
|
|
|
|
|
+
|
|
|
|
|
+```python
|
|
|
|
|
+# 改前(line 525 起):
|
|
|
|
|
+yield ("transcript", {"text": transcript, "round": current_round})
|
|
|
|
|
+
|
|
|
|
|
+student_msg = DialogueMessage(
|
|
|
|
|
+ session_id=session.id,
|
|
|
|
|
+ round=current_round,
|
|
|
|
|
+ role="student",
|
|
|
|
|
+ content=transcript,
|
|
|
|
|
+ audio_url=audio_url,
|
|
|
|
|
+ client_turn_id=client_turn_id,
|
|
|
|
|
+)
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+```python
|
|
|
|
|
+# 改后:
|
|
|
|
|
+audio_duration = parse_wav_duration(audio_bytes)
|
|
|
|
|
+yield ("transcript", {
|
|
|
|
|
+ "text": transcript,
|
|
|
|
|
+ "round": current_round,
|
|
|
|
|
+ "audioDuration": audio_duration,
|
|
|
|
|
+})
|
|
|
|
|
+
|
|
|
|
|
+student_msg = DialogueMessage(
|
|
|
|
|
+ session_id=session.id,
|
|
|
|
|
+ round=current_round,
|
|
|
|
|
+ role="student",
|
|
|
|
|
+ content=transcript,
|
|
|
|
|
+ audio_url=audio_url,
|
|
|
|
|
+ audio_duration=audio_duration,
|
|
|
|
|
+ client_turn_id=client_turn_id,
|
|
|
|
|
+)
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 3.3: HTTP /speak partial-hit 路径透传 audio_duration**
|
|
|
|
|
+
|
|
|
|
|
+同文件 `:494-499` 附近的 partial-hit 分支:
|
|
|
|
|
+
|
|
|
|
|
+```python
|
|
|
|
|
+if turn_state == "partial":
|
|
|
|
|
+ student_row = next(r for r in existing_rows if r.role == "student")
|
|
|
|
|
+ transcript = student_row.content
|
|
|
|
|
+ student_msg = student_row
|
|
|
|
|
+ yield ("transcript", {
|
|
|
|
|
+ "text": transcript,
|
|
|
|
|
+ "round": student_row.round,
|
|
|
|
|
+ "audioDuration": student_row.audio_duration,
|
|
|
|
|
+ })
|
|
|
|
|
+ evaluation = None
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 3.4: HTTP /speak 全 hit replay 路径透传 audio_duration**
|
|
|
|
|
+
|
|
|
|
|
+同文件 `:206` 附近的 `_replay_full_hit`:
|
|
|
|
|
+
|
|
|
|
|
+```python
|
|
|
|
|
+async def _replay_full_hit(...):
|
|
|
|
|
+ ...
|
|
|
|
|
+ student = next(r for r in rows if r.role == "student")
|
|
|
|
|
+ ai = next(r for r in rows if r.role == "ai")
|
|
|
|
|
+ yield ("transcript", {
|
|
|
|
|
+ "text": student.content,
|
|
|
|
|
+ "round": student.round,
|
|
|
|
|
+ "audioDuration": student.audio_duration,
|
|
|
|
|
+ })
|
|
|
|
|
+ yield ("token", {"content": ai.content})
|
|
|
|
|
+ ...
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 3.5: WS /speak-stream 写入 + transcript 透传**
|
|
|
|
|
+
|
|
|
|
|
+编辑 `app/api/dialogue.py`:
|
|
|
|
|
+
|
|
|
|
|
+(a) 文件顶部 import:
|
|
|
|
|
+```python
|
|
|
|
|
+from app.service.speaking.audio_utils import parse_wav_duration
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+(b) `:476-479` 把 audioDuration 放进 transcript payload。改之前要先把 `wav_bytes`(`:482`)的产生提前到 transcript 之前——当前顺序是先 `yield transcript` 再 `_pcm_to_wav`,要改为先 pack WAV 再算 duration 再 yield:
|
|
|
|
|
+
|
|
|
|
|
+改前:
|
|
|
|
|
+```python
|
|
|
|
|
+# 7. yield transcript 事件
|
|
|
|
|
+await websocket.send_json(
|
|
|
|
|
+ {"type": "transcript", "text": transcript, "round": current_round}
|
|
|
|
|
+)
|
|
|
|
|
+
|
|
|
|
|
+# 8. 打包 PCM → WAV(用于 S3 和发音评估)
|
|
|
|
|
+wav_bytes = _pcm_to_wav(bytes(audio_buffer), sample_rate, bits, channels)
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+改后:
|
|
|
|
|
+```python
|
|
|
|
|
+# 7. 打包 PCM → WAV(提前到 transcript 之前以便算时长)
|
|
|
|
|
+wav_bytes = _pcm_to_wav(bytes(audio_buffer), sample_rate, bits, channels)
|
|
|
|
|
+audio_duration = parse_wav_duration(wav_bytes)
|
|
|
|
|
+
|
|
|
|
|
+# 8. yield transcript 事件(带 audioDuration 与 HTTP 协议对齐)
|
|
|
|
|
+await websocket.send_json({
|
|
|
|
|
+ "type": "transcript",
|
|
|
|
|
+ "text": transcript,
|
|
|
|
|
+ "round": current_round,
|
|
|
|
|
+ "audioDuration": audio_duration,
|
|
|
|
|
+})
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+(c) `:485-494` student_msg 写入加 audio_duration:
|
|
|
|
|
+```python
|
|
|
|
|
+student_msg = DialogueMessage(
|
|
|
|
|
+ session_id=session.id,
|
|
|
|
|
+ round=current_round,
|
|
|
|
|
+ role="student",
|
|
|
|
|
+ content=transcript,
|
|
|
|
|
+ audio_url=None,
|
|
|
|
|
+ audio_duration=audio_duration,
|
|
|
|
|
+ client_turn_id=turn_id,
|
|
|
|
|
+)
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+> 注意:原代码注释“# 9. 保存学生消息 + 评估占位”和编号会因为步骤合并而漂移。如果有时间,把段落注释序号改顺;如果不改不影响功能。
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 3.6: `/sessions/latest` 输出 audioDuration**
|
|
|
|
|
+
|
|
|
|
|
+编辑 `app/service/speaking/dialogue_service.py:684-693` 附近的 messages 列表 dict:
|
|
|
|
|
+
|
|
|
|
|
+```python
|
|
|
|
|
+"messages": [
|
|
|
|
|
+ {
|
|
|
|
|
+ "id": msg.uuid,
|
|
|
|
|
+ "round": msg.round,
|
|
|
|
|
+ "role": msg.role,
|
|
|
|
|
+ "content": msg.content,
|
|
|
|
|
+ "audioUrl": msg.audio_url,
|
|
|
|
|
+ "audioDuration": msg.audio_duration, # NEW
|
|
|
|
|
+ "clientTurnId": msg.client_turn_id,
|
|
|
|
|
+ }
|
|
|
|
|
+ for msg in messages
|
|
|
|
|
+],
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 3.7: `/report` 输出 audioDuration**
|
|
|
|
|
+
|
|
|
|
|
+同文件 `:921-928` 附近 rounds entry:
|
|
|
|
|
+
|
|
|
|
|
+```python
|
|
|
|
|
+entry = {
|
|
|
|
|
+ "round": msg.round,
|
|
|
|
|
+ "role": msg.role,
|
|
|
|
|
+ "content": msg.content,
|
|
|
|
|
+ "audioUrl": msg.audio_url,
|
|
|
|
|
+ "audioDuration": msg.audio_duration, # NEW
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 3.8: 跑后端测试套件**
|
|
|
|
|
+
|
|
|
|
|
+Run: `pytest tests/ -v`
|
|
|
|
|
+Expected: `test_audio_utils.py` 5/5 PASS;其余测试不受 schema 变更影响(新增 nullable 列不破坏既有 query)。如有任何 fail,修复后再继续。
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 3.9: Commit(合并 task 2 + task 3)**
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+cd /Users/buoy/Development/gitrepo/cococlass-english-speaking-api
|
|
|
|
|
+git add app/models/dialogue.py \
|
|
|
|
|
+ app/service/speaking/audio_utils.py \
|
|
|
|
|
+ app/service/speaking/dialogue_service.py \
|
|
|
|
|
+ app/api/dialogue.py \
|
|
|
|
|
+ tests/service/speaking/test_audio_utils.py
|
|
|
|
|
+git commit -m "$(cat <<'EOF'
|
|
|
|
|
+feat(speaking): persist + surface dialogue_message.audio_duration
|
|
|
|
|
+
|
|
|
|
|
+Adds defensive parse_wav_duration helper, writes audio_duration on both
|
|
|
|
|
+HTTP /speak and WS /speak-stream student-message inserts, and surfaces
|
|
|
|
|
+the value through transcript SSE/WS events, /sessions/latest, and /report
|
|
|
|
|
+so all three frontend surfaces (live bubble, refreshed history, detailed
|
|
|
|
|
+report card) read from a single backend source.
|
|
|
|
|
+
|
|
|
|
|
+Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|
|
|
|
+EOF
|
|
|
|
|
+)"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## Task 4: 前端 — 类型扩展
|
|
|
|
|
+
|
|
|
|
|
+**Files:**
|
|
|
|
|
+- Modify: `PPT/src/types/englishSpeaking.ts:197-226` (PreviewChatMessage), `:231-235` (SSEEvent), `:264-271` (HistoricalDialogueMessage)
|
|
|
|
|
+- Modify: `PPT/src/views/Editor/EnglishSpeaking/services/llmService.ts:51-52` (parser), `:88-94` (BackendRound), `:127-149` (adaptReport)
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 4.1: PreviewChatMessage 加 audioDuration**
|
|
|
|
|
+
|
|
|
|
|
+编辑 `src/types/englishSpeaking.ts:197-226`,在 `audioUrl?: string` 后面添加:
|
|
|
|
|
+
|
|
|
|
|
+```ts
|
|
|
|
|
+export interface PreviewChatMessage {
|
|
|
|
|
+ id: string
|
|
|
|
|
+ role: 'ai' | 'student'
|
|
|
|
|
+ content: string
|
|
|
|
|
+ timestamp: Date
|
|
|
|
|
+ status?: MessageStatus
|
|
|
|
|
+ error?: string
|
|
|
|
|
+ turnId?: string
|
|
|
|
|
+ recovery?: 'retry' | 'rerecord' | 'restart'
|
|
|
|
|
+ audioBlob?: Blob
|
|
|
|
|
+ audioUrl?: string
|
|
|
|
|
+ audioDuration?: number // NEW: 秒,由后端 transcript 事件 / sessions-latest 写入
|
|
|
|
|
+ evaluation?: { ... } // 不变
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 4.2: SSEEvent transcript 分支加 audioDuration**
|
|
|
|
|
+
|
|
|
|
|
+`src/types/englishSpeaking.ts:231-235`:
|
|
|
|
|
+
|
|
|
|
|
+```ts
|
|
|
|
|
+export type SSEEvent =
|
|
|
|
|
+ | { type: 'transcript'; text: string; audioDuration?: number | null } // 加字段
|
|
|
|
|
+ | { type: 'token'; text: string }
|
|
|
|
|
+ | { type: 'done'; isComplete: boolean }
|
|
|
|
|
+ | { type: 'error'; message: string }
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 4.3: HistoricalDialogueMessage 加 audioDuration**
|
|
|
|
|
+
|
|
|
|
|
+`src/types/englishSpeaking.ts:264-271`:
|
|
|
|
|
+
|
|
|
|
|
+```ts
|
|
|
|
|
+export interface HistoricalDialogueMessage {
|
|
|
|
|
+ id: string
|
|
|
|
|
+ round: number
|
|
|
|
|
+ role: 'ai' | 'student'
|
|
|
|
|
+ content: string
|
|
|
|
|
+ audioUrl?: string | null
|
|
|
|
|
+ audioDuration?: number | null // NEW
|
|
|
|
|
+ clientTurnId?: string | null
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 4.4: parseSSEStream 透传 audioDuration**
|
|
|
|
|
+
|
|
|
|
|
+编辑 `src/views/Editor/EnglishSpeaking/services/llmService.ts:51-52`:
|
|
|
|
|
+
|
|
|
|
|
+```ts
|
|
|
|
|
+if (eventType === 'transcript') {
|
|
|
|
|
+ yield {
|
|
|
|
|
+ type: 'transcript',
|
|
|
|
|
+ text: parsed.text,
|
|
|
|
|
+ audioDuration: parsed.audioDuration ?? null,
|
|
|
|
|
+ }
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 4.5: BackendRound 加 audioDuration**
|
|
|
|
|
+
|
|
|
|
|
+同文件 `:88-94`:
|
|
|
|
|
+
|
|
|
|
|
+```ts
|
|
|
|
|
+interface BackendRound {
|
|
|
|
|
+ round: number
|
|
|
|
|
+ role: 'ai' | 'student'
|
|
|
|
|
+ content: string
|
|
|
|
|
+ audioUrl: string | null
|
|
|
|
|
+ audioDuration: number | null // NEW
|
|
|
|
|
+ evaluation?: BackendEvaluation
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 4.6: adaptReport 把 audioDuration 写进 SentenceEvaluation**
|
|
|
|
|
+
|
|
|
|
|
+同文件 `:127-149` 的 map 内 return:
|
|
|
|
|
+
|
|
|
|
|
+```ts
|
|
|
|
|
+return {
|
|
|
|
|
+ id: `${raw.sessionId}-${idx}`,
|
|
|
|
|
+ round: r.round,
|
|
|
|
|
+ role: r.role,
|
|
|
|
|
+ content: r.content,
|
|
|
|
|
+ audioUrl: r.audioUrl ?? undefined,
|
|
|
|
|
+ audioDuration: r.audioDuration ?? undefined, // NEW
|
|
|
|
|
+ score: pronunciation
|
|
|
|
|
+ ? Math.round((pronunciation.accuracy + pronunciation.fluency + pronunciation.intonation + pronunciation.stress) / 4)
|
|
|
|
|
+ : undefined,
|
|
|
|
|
+ pronunciation,
|
|
|
|
|
+ feedback: r.evaluation?.contentFeedback ?? undefined,
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+> `SentenceEvaluation.audioDuration` 已在 `englishSpeaking.ts:147` 存在(`audioDuration?: number`),这里只是开始真的填值。
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 4.7: 跑 typecheck**
|
|
|
|
|
+
|
|
|
|
|
+Run(在 PPT repo):`npm run type-check`(如果命令名是 `npx vue-tsc --noEmit` 也可,按 package.json scripts 走)。
|
|
|
|
|
+Expected: 0 errors。如果出现 `audioDuration` 相关红线,是消费侧的 task 5–8 还没填齐——容许;type-check 在最后一个前端 task 必须全过。
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 4.8: Commit**
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+cd /Users/buoy/Development/gitrepo/PPT
|
|
|
|
|
+git add src/types/englishSpeaking.ts src/views/Editor/EnglishSpeaking/services/llmService.ts
|
|
|
|
|
+git commit -m "$(cat <<'EOF'
|
|
|
|
|
+feat(speaking): wire audioDuration through type chain
|
|
|
|
|
+
|
|
|
|
|
+Extends HistoricalDialogueMessage / PreviewChatMessage / SSEEvent /
|
|
|
|
|
+BackendRound to carry the new audio_duration field. parseSSEStream
|
|
|
|
|
+forwards it on the transcript event; adaptReport plumbs it into
|
|
|
|
|
+SentenceEvaluation.audioDuration which already existed on the type.
|
|
|
|
|
+
|
|
|
|
|
+Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|
|
|
|
+EOF
|
|
|
|
|
+)"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## Task 5: 前端 — `useAudioPlayer` 加 URL 模式 + `useDialogueEngine` 写 audioDuration
|
|
|
|
|
+
|
|
|
|
|
+**Files:**
|
|
|
|
|
+- Modify: `PPT/src/views/Editor/EnglishSpeaking/composables/useAudioPlayer.ts:4-6` (PlaySource), `:69-86` (play branches)
|
|
|
|
|
+- Modify: `PPT/src/views/Editor/EnglishSpeaking/composables/useDialogueEngine.ts:33-43` (toPreviewMessage), `:144-149` (HTTP transcript handler), `:425-432` (WS transcript handler)
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 5.1: PlaySource 增 url 分支**
|
|
|
|
|
+
|
|
|
|
|
+编辑 `src/views/Editor/EnglishSpeaking/composables/useAudioPlayer.ts:4-6`:
|
|
|
|
|
+
|
|
|
|
|
+```ts
|
|
|
|
|
+export type PlaySource =
|
|
|
|
|
+ | { kind: 'tts'; text: string }
|
|
|
|
|
+ | { kind: 'blob'; blob: Blob }
|
|
|
|
|
+ | { kind: 'url'; url: string } // NEW: DetailedReport 学生卡片重播 audioUrl 用
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 5.2: play() 加 url 分支**
|
|
|
|
|
+
|
|
|
|
|
+同文件 `:69-86`,在 `if (source.kind === 'blob')` 之后、`else { TTS path }` 之前插入 url 分支。改完整段:
|
|
|
|
|
+
|
|
|
|
|
+```ts
|
|
|
|
|
+let blob: Blob
|
|
|
|
|
+if (source.kind === 'blob') {
|
|
|
|
|
+ blob = source.blob
|
|
|
|
|
+}
|
|
|
|
|
+else if (source.kind === 'url') {
|
|
|
|
|
+ synthAbort = new AbortController()
|
|
|
|
|
+ const res = await fetch(source.url, { signal: synthAbort.signal })
|
|
|
|
|
+ synthAbort = null
|
|
|
|
|
+ if (loadingId.value !== id) return // 期间被打断
|
|
|
|
|
+ if (!res.ok) throw new Error(`fetch audio failed: ${res.status}`)
|
|
|
|
|
+ blob = await res.blob()
|
|
|
|
|
+}
|
|
|
|
|
+else {
|
|
|
|
|
+ const cached = ttsCache.get(id)
|
|
|
|
|
+ if (cached) {
|
|
|
|
|
+ blob = cached
|
|
|
|
|
+ }
|
|
|
|
|
+ else {
|
|
|
|
|
+ synthAbort = new AbortController()
|
|
|
|
|
+ blob = await synthesize(source.text, synthAbort.signal)
|
|
|
|
|
+ synthAbort = null
|
|
|
|
|
+ if (loadingId.value !== id) return
|
|
|
|
|
+ ttsCache.set(id, blob)
|
|
|
|
|
+ }
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+> 复用已有的 `synthAbort` 名做 fetch 中断(`synthAbort` 命名虽然偏 TTS,但是统一的 in-flight 取消 controller,复用合理;不新加 controller 字段以免破坏 stop() 单点收口)。
|
|
|
|
|
+> 不缓存 url blob:CDN 端会自带浏览器缓存。
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 5.3: useDialogueEngine.toPreviewMessage 透传 audioDuration**
|
|
|
|
|
+
|
|
|
|
|
+编辑 `src/views/Editor/EnglishSpeaking/composables/useDialogueEngine.ts:33-43`:
|
|
|
|
|
+
|
|
|
|
|
+```ts
|
|
|
|
|
+function toPreviewMessage(message: HistoricalDialogueMessage): PreviewChatMessage {
|
|
|
|
|
+ return {
|
|
|
|
|
+ id: message.id,
|
|
|
|
|
+ role: message.role,
|
|
|
|
|
+ content: message.content,
|
|
|
|
|
+ timestamp: new Date(),
|
|
|
|
|
+ status: 'done',
|
|
|
|
|
+ turnId: message.clientTurnId ?? undefined,
|
|
|
|
|
+ audioUrl: message.audioUrl ?? undefined,
|
|
|
|
|
+ audioDuration: message.audioDuration ?? undefined, // NEW
|
|
|
|
|
+ }
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 5.4: sendStudentMessage 的 HTTP transcript handler 写 audioDuration**
|
|
|
|
|
+
|
|
|
|
|
+同文件 `:144-149`:
|
|
|
|
|
+
|
|
|
|
|
+```ts
|
|
|
|
|
+if (event.type === 'transcript') {
|
|
|
|
|
+ studentMsg.content = event.text
|
|
|
|
|
+ studentMsg.status = 'done'
|
|
|
|
|
+ if (event.audioDuration != null) {
|
|
|
|
|
+ studentMsg.audioDuration = event.audioDuration
|
|
|
|
|
+ }
|
|
|
|
|
+ if (!isFinalRound.value) {
|
|
|
|
|
+ messages.value.push(aiMsg)
|
|
|
|
|
+ }
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 5.5: beginStudentStream 的 WS transcript handler 写 audioDuration**
|
|
|
|
|
+
|
|
|
|
|
+同文件 `:425-432`:
|
|
|
|
|
+
|
|
|
|
|
+```ts
|
|
|
|
|
+if (data.type === 'transcript' && studentMsg) {
|
|
|
|
|
+ studentMsg.content = data.text
|
|
|
|
|
+ studentMsg.status = 'done'
|
|
|
|
|
+ if (data.audioDuration != null) {
|
|
|
|
|
+ studentMsg.audioDuration = data.audioDuration
|
|
|
|
|
+ }
|
|
|
|
|
+ if (aiMsg && !messages.value.includes(aiMsg)) {
|
|
|
|
|
+ messages.value.push(aiMsg)
|
|
|
|
|
+ }
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+> regenerate 路径**故意不动**——它复用 prevStudent 不重新 push student。spec §4 / §7.5 已确认。
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 5.6: 跑 typecheck**
|
|
|
|
|
+
|
|
|
|
|
+Run: `npm run type-check`
|
|
|
|
|
+Expected: 0 errors。
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 5.7: Commit**
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+git add src/views/Editor/EnglishSpeaking/composables/useAudioPlayer.ts \
|
|
|
|
|
+ src/views/Editor/EnglishSpeaking/composables/useDialogueEngine.ts
|
|
|
|
|
+git commit -m "$(cat <<'EOF'
|
|
|
|
|
+feat(speaking): audio player URL mode + record audioDuration in engine
|
|
|
|
|
+
|
|
|
|
|
+useAudioPlayer gains a url PlaySource so DetailedReport can replay the
|
|
|
|
|
+S3 audioUrl. useDialogueEngine writes audioDuration into the student
|
|
|
|
|
+PreviewChatMessage from both HTTP SSE and WS transcript events, and
|
|
|
|
|
+toPreviewMessage forwards it from history on refresh.
|
|
|
|
|
+
|
|
|
|
|
+Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|
|
|
|
+EOF
|
|
|
|
|
+)"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## Task 6: 前端 — `DialogueChatView.vue` 模板替换写死时长
|
|
|
|
|
+
|
|
|
|
|
+**Files:**
|
|
|
|
|
+- Modify: `PPT/src/views/Editor/EnglishSpeaking/preview/DialogueChatView.vue:98-101` (AI duration), `:139-140` (student duration), `<script setup>`(新增 formatDuration)
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 6.1: 在 `<script setup>` 内新增 formatDuration**
|
|
|
|
|
+
|
|
|
|
|
+打开 `src/views/Editor/EnglishSpeaking/preview/DialogueChatView.vue`,在 `<script setup>` 中合适位置(其他 helper 旁,例如靠近 `hasRetryButton`)插入:
|
|
|
|
|
+
|
|
|
|
|
+```ts
|
|
|
|
|
+function formatDuration(seconds: number | null | undefined): string {
|
|
|
|
|
+ if (seconds == null || !Number.isFinite(seconds)) return '--:--'
|
|
|
|
|
+ const total = Math.round(seconds)
|
|
|
|
|
+ const mins = Math.floor(total / 60)
|
|
|
|
|
+ const secs = total % 60
|
|
|
|
|
+ return `${mins}:${String(secs).padStart(2, '0')}`
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 6.2: 学生泡泡 duration 用真值**
|
|
|
|
|
+
|
|
|
|
|
+同文件 `:137-140`:
|
|
|
|
|
+
|
|
|
|
|
+```vue
|
|
|
|
|
+<span
|
|
|
|
|
+ v-else
|
|
|
|
|
+ class="voice-duration voice-duration-student"
|
|
|
|
|
+>{{ formatDuration(message.audioDuration) }}</span>
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 6.3: AI 泡泡删掉 duration span**
|
|
|
|
|
+
|
|
|
|
|
+同文件 `:94-101`,把 v-else 那段整段删除。改前:
|
|
|
|
|
+
|
|
|
|
|
+```vue
|
|
|
|
|
+<span
|
|
|
|
|
+ v-if="player.errorId.value === message.id"
|
|
|
|
|
+ class="play-error-hint"
|
|
|
|
|
+>点击重试</span>
|
|
|
|
|
+<span
|
|
|
|
|
+ v-else
|
|
|
|
|
+ class="voice-duration voice-duration-ai"
|
|
|
|
|
+>0:04</span>
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+改后:
|
|
|
|
|
+
|
|
|
|
|
+```vue
|
|
|
|
|
+<span
|
|
|
|
|
+ v-if="player.errorId.value === message.id"
|
|
|
|
|
+ class="play-error-hint"
|
|
|
|
|
+>点击重试</span>
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+> 如果 `.voice-duration-ai` CSS 在 `<style>` 下变成无人引用的死规则,本任务**不删 SCSS**——保持 patch 范围最小,后续清理 phase 处理。
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 6.4: typecheck + 启动 dev server 手动 smoke**
|
|
|
|
|
+
|
|
|
|
|
+Run(终端 1):`npm run dev`(或仓库标准的 dev script)
|
|
|
|
|
+Run(终端 2):`npm run type-check`
|
|
|
|
|
+Expected: type-check 0 errors;dev server 起来后浏览器跑一段对话,观察学生泡泡时长不再是 `0:04`。
|
|
|
|
|
+
|
|
|
|
|
+> 这一步仅做"跑通 + 不再写死"的烟测;正式 8 条验收在 task 8。
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 6.5: Commit**
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+git add src/views/Editor/EnglishSpeaking/preview/DialogueChatView.vue
|
|
|
|
|
+git commit -m "$(cat <<'EOF'
|
|
|
|
|
+feat(speaking): show real audioDuration on student voice bubble
|
|
|
|
|
+
|
|
|
|
|
+Replaces hardcoded '0:04' on the student bubble with a formatted
|
|
|
|
|
+audioDuration value (falls back to '--:--' when null). Removes the AI
|
|
|
|
|
+bubble duration span entirely — TTS audio has no persisted duration
|
|
|
|
|
+and the visual was misleading.
|
|
|
|
|
+
|
|
|
|
|
+Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|
|
|
|
+EOF
|
|
|
|
|
+)"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## Task 7: 前端 — `DetailedReport.vue` 时长 + 学生卡片重播
|
|
|
|
|
+
|
|
|
|
|
+**Files:**
|
|
|
|
|
+- Modify: `PPT/src/views/Editor/EnglishSpeaking/preview/DetailedReport.vue` 全文(template + script setup)
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 7.1: 引入 useAudioPlayer 实例**
|
|
|
|
|
+
|
|
|
|
|
+编辑 `src/views/Editor/EnglishSpeaking/preview/DetailedReport.vue`,在 `<script lang="ts" setup>` 顶部追加 import 与实例化:
|
|
|
|
|
+
|
|
|
|
|
+```ts
|
|
|
|
|
+import { ref, computed } from 'vue'
|
|
|
|
|
+import type { SentenceEvaluation } from '@/types/englishSpeaking'
|
|
|
|
|
+import { useAudioPlayer } from '../composables/useAudioPlayer'
|
|
|
|
|
+
|
|
|
|
|
+interface Props {
|
|
|
|
|
+ sentenceEvaluations: SentenceEvaluation[]
|
|
|
|
|
+}
|
|
|
|
|
+
|
|
|
|
|
+const props = defineProps<Props>()
|
|
|
|
|
+
|
|
|
|
|
+const player = useAudioPlayer()
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 7.2: 替换 formatDuration 为 null-safe 版**
|
|
|
|
|
+
|
|
|
|
|
+同文件,把现有 `formatDuration(seconds: number)` 替换为:
|
|
|
|
|
+
|
|
|
|
|
+```ts
|
|
|
|
|
+function formatDuration(seconds: number | null | undefined): string {
|
|
|
|
|
+ if (seconds == null || !Number.isFinite(seconds)) return '--:--'
|
|
|
|
|
+ const total = Math.round(seconds)
|
|
|
|
|
+ const mins = Math.floor(total / 60)
|
|
|
|
|
+ const secs = total % 60
|
|
|
|
|
+ return `${mins}:${String(secs).padStart(2, '0')}`
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 7.3: 加 handleSentencePlay**
|
|
|
|
|
+
|
|
|
|
|
+同文件 `<script setup>`,在 `getScoreClass` 旁追加:
|
|
|
|
|
+
|
|
|
|
|
+```ts
|
|
|
|
|
+function handleSentencePlay(s: SentenceEvaluation) {
|
|
|
|
|
+ if (!s.audioUrl) return
|
|
|
|
|
+ // 同 id 二次点击:toggle 停止;否则播放新源
|
|
|
|
|
+ if (player.playingId.value === s.id || player.loadingId.value === s.id) {
|
|
|
|
|
+ player.stop()
|
|
|
|
|
+ return
|
|
|
|
|
+ }
|
|
|
|
|
+ player.play(s.id, { kind: 'url', url: s.audioUrl })
|
|
|
|
|
+}
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 7.4: 学生卡片 play button 接 handler**
|
|
|
|
|
+
|
|
|
|
|
+template `:32-38`,把学生分支独立出来。改前是 AI/student 共用一个 button:
|
|
|
|
|
+
|
|
|
|
|
+```vue
|
|
|
|
|
+<div class="card-voice-bar" :class="sentence.role === 'ai' ? 'bar-ai' : 'bar-student'">
|
|
|
|
|
+ <button class="card-play-btn" :class="sentence.role === 'ai' ? 'play-ai' : 'play-student'" @click.stop>
|
|
|
|
|
+ <svg ...><polygon ... /><path ... /></svg>
|
|
|
|
|
+ </button>
|
|
|
|
|
+ ...
|
|
|
|
|
+</div>
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+改后(学生有 button,AI 没 button):
|
|
|
|
|
+
|
|
|
|
|
+```vue
|
|
|
|
|
+<div class="card-voice-bar" :class="sentence.role === 'ai' ? 'bar-ai' : 'bar-student'">
|
|
|
|
|
+ <button
|
|
|
|
|
+ v-if="sentence.role === 'student'"
|
|
|
|
|
+ class="card-play-btn play-student"
|
|
|
|
|
+ :disabled="!sentence.audioUrl"
|
|
|
|
|
+ @click.stop="handleSentencePlay(sentence)"
|
|
|
|
|
+ >
|
|
|
|
|
+ <!-- loading -->
|
|
|
|
|
+ <svg
|
|
|
|
|
+ v-if="player.loadingId.value === sentence.id"
|
|
|
|
|
+ class="play-spinner"
|
|
|
|
|
+ width="10" height="10" viewBox="0 0 24 24" fill="none" stroke="currentColor"
|
|
|
|
|
+ stroke-width="2" stroke-linecap="round"
|
|
|
|
|
+ >
|
|
|
|
|
+ <path d="M21 12a9 9 0 1 1-6.219-8.56" />
|
|
|
|
|
+ </svg>
|
|
|
|
|
+ <!-- playing -->
|
|
|
|
|
+ <svg
|
|
|
|
|
+ v-else-if="player.playingId.value === sentence.id"
|
|
|
|
|
+ width="10" height="10" viewBox="0 0 24 24" fill="currentColor"
|
|
|
|
|
+ >
|
|
|
|
|
+ <rect x="6" y="4" width="4" height="16" /><rect x="14" y="4" width="4" height="16" />
|
|
|
|
|
+ </svg>
|
|
|
|
|
+ <!-- error -->
|
|
|
|
|
+ <svg
|
|
|
|
|
+ v-else-if="player.errorId.value === sentence.id"
|
|
|
|
|
+ width="10" height="10" viewBox="0 0 24 24" fill="none" stroke="currentColor"
|
|
|
|
|
+ stroke-width="2" stroke-linecap="round" stroke-linejoin="round"
|
|
|
|
|
+ >
|
|
|
|
|
+ <path d="M12 9v4" /><path d="M12 17h.01" />
|
|
|
|
|
+ <path d="M10.29 3.86 1.82 18a2 2 0 0 0 1.71 3h16.94a2 2 0 0 0 1.71-3L13.71 3.86a2 2 0 0 0-3.42 0z" />
|
|
|
|
|
+ </svg>
|
|
|
|
|
+ <!-- idle -->
|
|
|
|
|
+ <svg
|
|
|
|
|
+ v-else
|
|
|
|
|
+ width="10" height="10" viewBox="0 0 24 24" fill="none" stroke="currentColor"
|
|
|
|
|
+ stroke-width="2" stroke-linecap="round" stroke-linejoin="round"
|
|
|
|
|
+ >
|
|
|
|
|
+ <polygon points="11 5 6 9 2 9 2 15 6 15 11 19 11 5" />
|
|
|
|
|
+ <path d="M15.54 8.46a5 5 0 0 1 0 7.07" />
|
|
|
|
|
+ </svg>
|
|
|
|
|
+ </button>
|
|
|
|
|
+ ...
|
|
|
|
|
+</div>
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+> AI 卡片完全无 button——保留波形 + 文字即可。
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 7.5: 时长 span:学生显示 audioDuration,AI 直接删除**
|
|
|
|
|
+
|
|
|
|
|
+template `:48-50`,改:
|
|
|
|
|
+
|
|
|
|
|
+```vue
|
|
|
|
|
+<span class="card-duration" :class="sentence.role === 'ai' ? 'cd-ai' : 'cd-student'">
|
|
|
|
|
+ {{ formatDuration(sentence.audioDuration || 3) }}
|
|
|
|
|
+</span>
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+为:
|
|
|
|
|
+
|
|
|
|
|
+```vue
|
|
|
|
|
+<span
|
|
|
|
|
+ v-if="sentence.role === 'student'"
|
|
|
|
|
+ class="card-duration cd-student"
|
|
|
|
|
+>
|
|
|
|
|
+ {{ formatDuration(sentence.audioDuration) }}
|
|
|
|
|
+</span>
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+> AI 分支删掉,与 DialogueChatView 一致。
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 7.6: typecheck + dev smoke**
|
|
|
|
|
+
|
|
|
|
|
+Run: `npm run type-check`
|
|
|
|
|
+Expected: 0 errors。
|
|
|
|
|
+
|
|
|
|
|
+Dev server 中跑一次完整对话进入报告:
|
|
|
|
|
+- 学生卡片可点 play,听到自己的录音;
|
|
|
|
|
+- 时长显示真实秒数(不是 `0:03`);
|
|
|
|
|
+- AI 卡片只有波形 + 文字,无 button、无时长。
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 7.7: Commit**
|
|
|
|
|
+
|
|
|
|
|
+```bash
|
|
|
|
|
+git add src/views/Editor/EnglishSpeaking/preview/DetailedReport.vue
|
|
|
|
|
+git commit -m "$(cat <<'EOF'
|
|
|
|
|
+feat(speaking): DetailedReport student replay + real duration
|
|
|
|
|
+
|
|
|
|
|
+Wires DetailedReport student cards to useAudioPlayer (url mode) so the
|
|
|
|
|
+play button actually plays the persisted S3 audio. Replaces the
|
|
|
|
|
+audioDuration || 3 fallback with a null-safe formatter that shows
|
|
|
|
|
+'--:--' on legacy data. AI cards drop the play button and duration span
|
|
|
|
|
+entirely — TTS audio has no persisted url/duration to surface.
|
|
|
|
|
+
|
|
|
|
|
+Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
|
|
|
|
+EOF
|
|
|
|
|
+)"
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## Task 8: 手动验收(spec §9 全部 8 条)
|
|
|
|
|
+
|
|
|
|
|
+**No file changes.** 单独 task 用来强制不漏验收项。
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 8.1: 录新消息 → 真实时长**
|
|
|
|
|
+
|
|
|
|
|
+dev 环境登录学生账号,新建一段对话录一条消息。
|
|
|
|
|
+Expected: 学生泡泡时长不是 `0:04`,跟实际录音时长接近(±1s)。
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 8.2: 连续录两条 → 各自时长**
|
|
|
|
|
+
|
|
|
|
|
+同会话录第二条不刷新页面。
|
|
|
|
|
+Expected: 两条都显示各自真实时长。
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 8.3: 刷新页面 → 历史消息时长保留**
|
|
|
|
|
+
|
|
|
|
|
+不结束会话直接 F5。
|
|
|
|
|
+Expected: 历史消息回填后,所有学生泡泡时长仍正确显示,不出现 `--:--` 或 `0:04`。
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 8.4: 完整对话 → 报告页学生卡片**
|
|
|
|
|
+
|
|
|
|
|
+跑完所有轮次进 DetailedReport。
|
|
|
|
|
+Expected:
|
|
|
|
|
+- 学生卡片显示真实时长(非 `0:03`);
|
|
|
|
|
+- 学生卡片 play 按钮可点,能听到自己的录音;
|
|
|
|
|
+- AI 卡片只有波形 + 文字,无 button、无时长。
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 8.5: 升级前的旧会话 → `--:--`**
|
|
|
|
|
+
|
|
|
|
|
+打开数据库找一条 `audio_duration IS NULL` 的旧 session uuid,从历史入口打开。
|
|
|
|
|
+Expected: 所有学生消息时长显示 `--:--`,不报错;play 按钮在 audioUrl 存在时仍能点(旧会话可能 audioUrl 也无,则 disabled)。
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 8.6: 后端单测 5 case**
|
|
|
|
|
+
|
|
|
|
|
+Run: `cd /Users/buoy/Development/gitrepo/cococlass-english-speaking-api && pytest tests/service/speaking/test_audio_utils.py -v`
|
|
|
|
|
+Expected: 5/5 PASS。
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 8.7: HTTP partial-hit 路径**
|
|
|
|
|
+
|
|
|
|
|
+发同一条录音两次(模拟客户端 retry),观察第二次响应。
|
|
|
|
|
+Expected: transcript 事件 audioDuration 仍是上一次入库的值;前端泡泡时长正确。
|
|
|
|
|
+
|
|
|
|
|
+> 如果难以触发 partial-hit,spec §6.5.1 已说明 partial-hit 从 `student_row.audio_duration` 取,code review 阶段 step 3.3 已检查。可降级为查 server log 确认 `partial` 分支被走过即可。
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 8.8: WS /speak-stream 路径**
|
|
|
|
|
+
|
|
|
|
|
+确认浏览器 transport 走 WS(默认即是;可在 DevTools Network 看 `/speak-stream` 升级握手)。录一条。
|
|
|
|
|
+Expected: 学生泡泡时长正确显示。
|
|
|
|
|
+
|
|
|
|
|
+- [ ] **Step 8.9: 验收完成后 push**
|
|
|
|
|
+
|
|
|
|
|
+8 条全过后:
|
|
|
|
|
+```bash
|
|
|
|
|
+cd /Users/buoy/Development/gitrepo/cococlass-english-speaking-api && git push
|
|
|
|
|
+cd /Users/buoy/Development/gitrepo/PPT && git push
|
|
|
|
|
+```
|
|
|
|
|
+
|
|
|
|
|
+任一条不过:原地修,不再开新 task;修完重跑该条 + 后续条。
|
|
|
|
|
+
|
|
|
|
|
+---
|
|
|
|
|
+
|
|
|
|
|
+## 自检:Spec 覆盖核对
|
|
|
|
|
+
|
|
|
|
|
+| Spec 条 | 实现位置 |
|
|
|
|
|
+|---|---|
|
|
|
|
|
+| §6.1 migration | Task 1 |
|
|
|
|
|
+| §6.2 model | Task 3.1 |
|
|
|
|
|
+| §6.3 audio_utils | Task 2 |
|
|
|
|
|
+| §6.4 写入点 HTTP | Task 3.2 |
|
|
|
|
|
+| §6.4 写入点 WS | Task 3.5 |
|
|
|
|
|
+| §6.5.1 SSE transcript(full miss/partial/full hit)| Task 3.2 / 3.3 / 3.4 |
|
|
|
|
|
+| §6.5.2 sessions/latest | Task 3.6 |
|
|
|
|
|
+| §6.5.3 /report | Task 3.7 |
|
|
|
|
|
+| §6.5.4 WS transcript | Task 3.5 |
|
|
|
|
|
+| §6.6 unit test | Task 2 |
|
|
|
|
|
+| §7.1 类型补字段 | Task 4.1–4.5 |
|
|
|
|
|
+| §7.2 SSEEvent | Task 4.2 |
|
|
|
|
|
+| §7.4 useAudioPlayer URL 模式 | Task 5.1–5.2 |
|
|
|
|
|
+| §7.5 useDialogueEngine 三处(toPreviewMessage / HTTP / WS) | Task 5.3–5.5 |
|
|
|
|
|
+| §7.6 adaptReport | Task 4.6 |
|
|
|
|
|
+| §7.7 DialogueChatView 模板 | Task 6 |
|
|
|
|
|
+| §7.8 DetailedReport | Task 7 |
|
|
|
|
|
+| §9 验收 1–8 | Task 8 |
|
|
|
|
|
+
|
|
|
|
|
+> §10 Out-of-scope(AI 时长、AI 重播、回填脚本、E2E、blob 缓存)—— **不实现**,prompt 不构造任何相关代码或 commit。
|