PPT/docs/superpowers/specs/2026-04-26-dialogue-recording-resilience-design.md

Dialogue Recording Resilience & Idempotency Design

Date: 2026-04-26 Scope:

• Frontend: src/views/Editor/EnglishSpeaking/preview/DialogueChatView.vue and its composables (useDialogueEngine, useAudioRecorder).
• Backend: cococlass-english-speaking-api — app/service/speaking/dialogue_service.py, app/api/dialogue.py, schema migration. Status: Pending user review

Context

The English-speaking dialogue feature has five interlocking UX / correctness gaps in the recording flow:

1. No feedback after clicking 开始录音. handleStartRecording awaits getUserMedia + AudioContext setup synchronously. First-time mic permission can take 1–3 seconds; users see no UI change and assume the button is broken.
2. Cancel is destructive in the wrong way. handleCancelRecording calls streamCtl.abortStream(), which marks the just-pushed student/AI placeholders as error with text "已取消". The user sees what looks like a failure when they themselves chose to abort.
3. Placeholder push happens too early. beginStudentStream pushes studentMsg and aiMsg (both loading) the moment the WebSocket opens — before the user has even started speaking. Two consequences: an empty student row (the voice-bar is hidden while status==='loading' with no content) and a misleading AI typing-bubble that animates while the student is mid-recording.
4. Failure handling is one-size-fits-all. Every error gets a generic "重试" button that re-sends the same audio. For "No speech detected" this is futile; for session-expiry it is wrong; for ambiguous network errors there is no escape from the failed turn except restarting the entire session.
5. Network-flake double-billing. If the WebSocket / HTTP path completes server-side but the done event never reaches the client, the user clicks 重试 and the server processes a second turn — running STT + LLM again, incrementing current_round again. There is no idempotency key on any write endpoint.

We additionally observed a sixth issue while diagnosing the above:

1. Last round runs an unnecessary LLM round-trip. On the final student turn, the server still calls LLM + TTS for a closing AI reply that the user barely hears before the report transition. This wastes ~¥0.07–0.25 per session (LLM + Azure TTS), adds 3–5 s of latency, and creates a visual mismatch between the round indicator (3/3) and the actual number of AI utterances (4 — one greeting + three replies).

This spec addresses all six in one pass because they share the same files (engine + service + dialogue API + schema) and the fixes for 1–4 depend on the placeholder-timing fix in 3, which depends on the engine API split for 2.

Goals

1. Show a starting state with cancel affordance during mic acquisition.
2. Make 取消 a no-op on chat history (not a fake error).
3. Push student / AI placeholders only after the user clicks 完成 录音, eliminating the premature typing-bubble.
4. Classify failures into retry | rerecord | restart and render the correct button(s) per error kind.
5. Add weak idempotency via client_turn_id so retries cannot double-bill or double-advance the round counter.
6. Skip the LLM call on the final student turn and replace it with a "finalizing" transition card that masks the report-fetch latency.

Non-goals

• Cancellation during STT / ai_thinking (after 完成 has been clicked). Once the user commits the turn, they wait. Adding a "cancel server-side processing" affordance is out of scope; if needed later it can be a separate spec (and requires server cooperation to release locks / abort the LLM stream).
• Generic / cross-endpoint idempotency middleware (the "textbook A" approach with a separate cache table + TTL + cron cleanup). We use the pragmatic A.1 variant scoped to dialogue_message; see D5.
• Mic permission UX deep-dive (better permission-denied recovery, OS-level Settings deep links). The existing permission-banner is acceptable.
• Streaming TTS for closing remarks. With D6 we skip the closing reply entirely; no closing TTS is needed.
• Refactoring useDialogueEngine beyond what these problems require. No migration to Pinia, no event-bus rework.

Non-trivial decisions

D1. Push placeholders at commit(), not at begin()

beginStudentStream currently pushes both placeholders the moment the WebSocket is being opened. The original justification ("立即占位 — 录音按完成的那一刻 UI 就已经显示学生泡泡 + AI placeholder") is hollow because the student voice-bar template is gated on message.content || message.status !== 'loading', so during recording the placeholder renders as an empty row, while the AI placeholder renders an active typing-bubble — exactly the wrong visual ("AI is thinking" while the student is still talking).

Pushing at commit() (i.e., when the user clicks 完成) gives the right semantics: the bubble appears at the moment the user "sends" their turn, the typing-bubble corresponds to actual server processing, and 取消 needs no cleanup (nothing was pushed). This change requires splitting the engine API.

D2. Engine API split: begin → commit | abort

beginStudentStream is restructured (name kept; only its return shape and side-effect contract change):

type StreamSession = {
  turnId: string                 // generated here, exposed for retry/UI
  pushChunk: (buf: ArrayBuffer) => void
  commit: (blob: Blob) => void   // push placeholders, attach blob, send 'stop'
  abort: () => void              // close WS; no message-list mutation
}
function beginStudentStream(opts): StreamSession | null

attachStudentBlob is removed (folded into commit). The only externally-observable side effect of begin is opening the WebSocket and starting to stream chunks; if abort is called before commit, the message list is untouched.

Two adjacent helpers ship with this:

discardCurrentTurn(messageId: string): void   // splice failed student+ai pair, reset currentTurnId

Used by 重录 to cleanly drop a failed turn before the user starts over.

D3. Error classification: 3 kinds, encoded on the message

PreviewChatMessage gains two fields:

turnId?: string                                   // set when the message is created via begin/commit
recovery?: 'retry' | 'rerecord' | 'restart'      // set on error; drives the button matrix

turnId is the same value across the student / ai pair of one turn (so regenerateAiMessage can read it from the AI message and retryMessage from the student message). The pre-existing unrecoverable?: boolean is removed (subsumed by recovery === 'restart'). A pure function classifyError(rawText: string, status?: number, msgRole: 'student' | 'ai'): Recovery lives in the engine and is called by every error path:

function classifyError(raw, status, role) {
  if (status === 404 || status === 409) return 'restart'
  if (raw === 'Session not found' || raw === 'Session is not active') return 'restart'
  if (raw === 'No speech detected' && role === 'student') return 'rerecord'
  return 'retry'
}

role === 'ai' cannot legitimately be rerecord (transcript was received → audio is fine; rerecording would discard a known-good utterance). The classifier enforces this.

D4. Button matrix in the error card

Rendered solely from recovery and role:

role	recovery	Buttons	On click
student	retry	重试 + 重录	retry: retryMessage(id) (HTTP, same turnId)
			rerecord: drop turn, reset currentTurnId, idle
student	rerecord	重录	drop turn, reset currentTurnId, idle
student	restart	返回重开	emit 'restart' to parent
ai	retry	重试	regenerateAiMessage(id) (HTTP, same turnId)
ai	restart	返回重开	emit 'restart'

The student / retry row is the only case with two buttons. Reasoning: when classification is ambiguous (generic 5xx, network blip), the user is the best judge of whether to wait/retry the network or to re-record. For the unambiguous cases, a single button gives the unambiguously correct affordance.

D5. Idempotency: pragmatic A.1, scoped to dialogue_message

Rejected: textbook idempotency-key middleware + dedicated cache table + TTL cleanup. Reasoning: the project has three write endpoints (/greeting, /speak, /regenerate-ai) all in the same domain; building generic infrastructure carries no payoff and introduces Redis or a cleanup job we currently don't need.

Adopted: a single column client_turn_id VARCHAR(36) NULL on dialogue_message, indexed by (session_id, client_turn_id, role) UNIQUE. The handler lookup distinguishes three states:

Existing rows	Action
0 (full miss)	Run STT → INSERT student → run LLM → INSERT ai → current_round++
1 (student only — partial hit)	Skip STT, replay stored transcript, run LLM, INSERT ai, round++
2 (full hit)	Replay transcript + tokens + done from existing rows; no side effects

The "1 row" branch is what makes A.1 strictly better than A.2 (atomic-only): a partial server failure (STT succeeded, LLM crashed) does not force the client to pay for STT a second time on retry. For Azure STT pricing this is a real win and also reduces retry latency by ~1–2 s.

TTL: none (intentional). The client_turn_id column is business data — it records which retry attempt produced this turn. The data lives and dies with the session itself; session expiry (Session is not active) blocks future retries at the validation layer before the idempotency lookup runs.

Concurrency (defense in depth): in normal use — single user, single tab, frontend disabling the retry button while a request is in flight — two concurrent requests with the same turnId cannot occur. The UNIQUE index on (session_id, client_turn_id, role) exists as a zero-cost backstop in case the frontend disable is ever bypassed (multi-tab, browser bug, manual API call). If that pathological race ever happens, the second INSERT raises IntegrityError; the handler catches it and falls through to the SELECT-based hit branch. We do not add a server-side mutex; the index alone is sufficient.

D6. Skip LLM on the final student turn

The closing AI reply on the last round is removed. The student speaks their final sentence, the server runs STT + INSERTs the student row, then immediately yields done with isComplete: true — no LLM call, no AI row.

For the engine to know "this commit is the last turn", attachSession is extended to accept totalRounds:

attachSession(info: { sessionId; expiresAt?; totalRounds })

The engine stores totalRounds and computes isFinalRound = currentRound.value >= totalRounds. commit() reads this flag:

• If isFinalRound: push only studentMsg (no aiMsg placeholder).
• Otherwise: push both, as today.

state adds a finalizing value, displayed as a "正在生成你的本次对话报告..." card layered into state-stack. The existing watch(isComplete) continues to call fetchReportSafe and emit complete. The state === 'finalizing' derivation: engine.isComplete.value === true && reportFetchInflight.value === true — see D7.

This makes the round indicator semantically clean (3/3 = three student turns, three AI replies, plus one greeting) and saves one LLM + one TTS call per session.

D7. State derivation for new states

Both starting and finalizing need flags the existing state computed doesn't have. Adding one ref each in DialogueChatView.vue:

const isStarting = ref(false)            // true between handleStartRecording entry and recorder.startRecording resolution
const reportFetchInflight = ref(false)   // true while fetchReportSafe is awaiting

const state = computed(() => {
  if (isStarting.value) return 'starting'
  if (recorder.isRecording.value) return 'recording'
  if (engine.isComplete.value) return reportFetchInflight.value ? 'finalizing' : 'done'
  // ...rest unchanged
})

isStarting is local view state because cancellation is also local (the AbortController is held in the view); putting it in the recorder composable would create cross-layer coupling without benefit.

Architecture

State machine

              ┌──────────────────── 取消 ─────────────────────────────┐
              ▼                                                       │
  idle ─→ starting ─→ recording ─→ (commit) ─→ stt ─→ ai_thinking ─→ done
              │           │            │         │         │
              │ cancel    │ cancel     │         │         │
              ▼           ▼            ▼         ▼         ▼
            idle        idle         error   error      error
                                     ↘          ↘          ↘
                                    {retry, rerecord, restart}

  done ──(if isComplete)──▶ finalizing ──▶ (parent emits, transitions away)

stt and ai_thinking are unchanged from current code (derived from engine.messages state). The new states are starting (between mic-tap and recorder ready) and finalizing (between session-complete-flag-flip and parent transition).

Component / module map

DialogueChatView.vue
├── computes  state ∈ {idle, starting, recording, stt, ai_thinking, finalizing, error, done}
├── handlers  handleStartRecording / handleCancelRecording / handleFinishRecording
│             handleRetry (per recovery kind), handleRerecord, handleRestart
├── renders   state-stack with new layers: state-starting, state-finalizing
└── consumes  useDialogueEngine, useAudioRecorder, useAudioPlayer

useDialogueEngine.ts (engine)
├── state    messages, currentTurnId, currentRound, totalRounds, isComplete
├── exports  attachSession({ sessionId, expiresAt?, totalRounds })
│             beginStudentStream() → { turnId, pushChunk, commit(blob), abort() }
│             retryMessage(id), regenerateAiMessage(id)  // both pass turnId from message
│             discardCurrentTurn(id)                       // 重录 helper
│             classifyError(raw, status, role)
└── api      sends turnId in WS 'start' frame and HTTP /speak body

useAudioRecorder.ts (recorder)
└── new     startRecording accepts AbortSignal so handleCancelRecording can
            abort getUserMedia in the starting state

llmService.ts
└── speak(sessionId, blob, signal, turnId)        // body adds turnId
    generateGreeting(sessionId, signal, turnId)   // body adds turnId

Backend

app/api/dialogue.py
├── POST /greeting                body: { sessionId, turnId }
├── POST /speak                    body: { sessionId, turnId, audio }
└── WS   /speak-stream             start frame: { ..., turnId }

app/service/speaking/dialogue_service.py
└── speak()
    ├── lookup_existing(session_id, turn_id) → 0 | student-only | full
    ├── full hit       → replay (transcript, tokens, done)
    ├── partial hit    → reuse transcript, run LLM, insert ai, round++
    └── full miss      → original flow, write turn_id on every INSERT
    
    On final round (current_round >= total_rounds):
    └── after student INSERT, yield done(isComplete=True), do NOT call LLM

migrations/004_add_client_turn_id.sql
├── ALTER TABLE dialogue_message ADD COLUMN client_turn_id VARCHAR(36) NULL
├── ADD UNIQUE INDEX uk_session_turn_role (session_id, client_turn_id, role)
└── (no backfill — existing rows leave it NULL; SELECT WHERE client_turn_id=?
     never returns NULL rows because we only query with non-null keys)

Detailed flows

Flow 1: Happy path with WebSocket

1. user clicks 开始录音
2. state → starting
3. recorder.startRecording() acquires mic         (~0.5–3 s, cancellable)
4. engine.beginStudentStream({ sampleRate, ... })
   - generates turnId (UUID v4)
   - opens WS, sends 'start' { sessionId, turnId, sampleRate, bits, channels }
   - returns { turnId, pushChunk, commit, abort }
5. state → recording
6. recorder.onChunk = streamSession.pushChunk
7. user clicks 完成
8. blob = recorder.stopRecording()
9. streamSession.commit(blob)
   - pushes studentMsg (loading, audioBlob=blob, turnId) + aiMsg (loading, turnId)
   - sends WS 'stop' frame
10. server STT → emits transcript → studentMsg.content/.status
11. state derives stt → ai_thinking from message states
12. server LLM → emits tokens → aiMsg.content (incremental)
13. server emits done { isComplete: false } → aiMsg.status='done', round++
    or                  { isComplete: true  } → state → finalizing → fetchReport

Flow 2: Cancel during starting

1. user clicks 开始录音
2. state → starting; recorder.startRecording(signal) in flight
3. user clicks 取消
4. signal.abort() → getUserMedia rejects → recorder cleanup
5. (engine.beginStudentStream was never called → no WS, no messages pushed)
6. state → idle

Flow 3: Cancel during recording

1. ... (flow 1 step 1–6 complete; user is recording)
2. user clicks 取消
3. recorder.onChunk = null
4. streamSession.abort() → ws.close()
5. recorder.stopRecording().catch(() => {}) (we discard the blob)
6. recorder.cleanup()
7. (no messages were pushed during recording — nothing to splice)
8. state → idle

Flow 4: Network drop after server commit (idempotency in action)

1. ... (flow 1 through step 12 — server has emitted all tokens)
2. server INSERTs ai row, round++, but ws.close fires before 'done' frame arrives
3. client sees ws.onclose with aiMsg.status === 'loading'
4. finalizeError → classifyError('Connection closed') → 'retry' → aiMsg.status='error'
5. user clicks 重试
6. handleRetry → engine.regenerateAiMessage(aiMsg.id)
   - HTTP POST /speak with prevStudent.audioBlob and turnId = prevStudent.turnId
7. server SELECT (session_id, turnId) → 2 rows (full hit)
8. server replays transcript / tokens / done from DB
9. client receives the same content; round NOT incremented again
10. Azure: 0 calls. Cost saved. Round counter accurate.

Flow 5: STT succeeds, LLM dies, retry (partial-hit branch)

1. server STT done → INSERT student row (turn=T1, content="...")
2. LLM throws → exception bubbles, no ai INSERT, no round++
3. client sees aiMsg.status='error' (or studentMsg if exception happened before transcript yield)
4. user clicks 重试
5. server SELECT → 1 row (student only)
6. server reads stored transcript, runs LLM, INSERTs ai, round++, yields done
7. Azure STT: 0 calls (saved). Azure LLM: 1 call (necessary).

Flow 6: User chooses 重录 instead of 重试

1. (any error state)
2. user clicks 重录
3. engine.discardCurrentTurn(messageId)
   - splices the failed student + ai pair from messages
   - resets currentTurnId = null
4. state → idle
5. user clicks 开始录音 again
6. begin generates a NEW turnId — the failed turn's row in DB becomes an orphan
   (SELECT by new turnId returns 0 rows; existing turnId is never queried again
   because the client has discarded it)
7. flow 1 from step 2

DB state after step 6: [student, T1] (orphan, never queried) + the new flow's rows. The orphan row is acceptable: it represents an utterance the user chose to discard, and report aggregation groups by round, not by client_turn_id, so it does not pollute reports. (See "Out-of-scope follow-ups" → orphan row cleanup if storage growth becomes a concern.)

Flow 7: Final round with isComplete=true

1. user is on round N, total_rounds = N (last round)
2. user records and clicks 完成
3. commit pushes ONLY studentMsg (no aiMsg — totalRounds reached)
4. server STT → INSERT student (turnId, round=N)
5. server detects current_round >= total_rounds:
   - skip LLM
   - session.current_round = N + 1
   - yield ('transcript', ...)
   - yield ('done', { isComplete: true })
6. client: studentMsg.content/.status = 'done', isComplete = true
7. state → finalizing; "正在生成你的本次对话报告..." card displayed
8. existing watcher fetches report → emit('complete')
9. parent transitions to report view

Schema & API contract

Schema migration: migrations/004_add_client_turn_id.sql

ALTER TABLE dialogue_message
  ADD COLUMN client_turn_id VARCHAR(36) NULL;

CREATE UNIQUE INDEX uk_session_turn_role
  ON dialogue_message (session_id, client_turn_id, role);

The unique index includes role because student + ai rows for the same turn share client_turn_id; we want both rows insertable but not duplicates of either. NULL values in unique indexes do not collide in MySQL, so legacy rows (pre-migration) are unaffected.

HTTP contract changes

• POST /api/speaking/dialogue/greeting — body adds turnId: string (UUID).
• POST /api/speaking/dialogue/speak — body adds turnId: string.
• (Existing 404 / 409 / error response shapes unchanged; clients already handle them.)

WebSocket contract change

• /api/speaking/dialogue/speak-stream — start frame schema:

{
  "type": "start",
  "sessionId": "...",
  "turnId": "...",      // NEW
  "sampleRate": 48000,
  "bits": 16,
  "channels": 1
}

Backend validation: turnId is required and must be a non-empty string. If missing, reject with {type: 'error', message: 'turnId required'}. Backwards compatibility is not required (no production deployments yet); the migration coordinates frontend + backend ship together.

Error handling

Every error finalization point in the engine (HTTP catch, WS onerror, WS onclose while loading, WS error frame) goes through one helper:

function finalizeError(
  msg: PreviewChatMessage,
  raw: string,
  status?: number,
) {
  msg.status = 'error'
  msg.error = friendlyErrorMessage(raw)
  msg.recovery = classifyError(raw, status, msg.role)
}

unrecoverable is removed; retryGreeting becomes recovery === 'restart' ? emit('restart') : retryGreeting() at the call site (already handled by the existing retryAiMessage shape — adapt to the new field name).

UI

state-starting layer

A new state-layer in state-stack, between idle and recording in source order. Reuses the record-capsule shell so the visual transition from starting to recording is a fade-in of the wave + counter rather than a layout swap:

<div class="state-layer state-starting" :style="stateStyle('starting')">
  <div class="record-capsule record-capsule-starting">
    <button class="cancel-btn" @click="handleCancelStarting">取消</button>
    <div class="record-meter">
      <svg class="spinner" .../>
      <span class="record-time">准备录音中...</span>
    </div>
    <button class="finish-btn" disabled>完成</button>
  </div>
</div>

Visually: same capsule, same buttons, but the wave is replaced by a small spinner and the counter reads "准备录音中..." instead of 0:00. finish-btn is disabled (no audio yet).

state-finalizing layer

<div class="state-layer state-finalizing" :style="stateStyle('finalizing')">
  <svg class="spinner" .../>
  <span class="center-text">正在生成你的本次对话报告...</span>
</div>

Same shape as state-center (used for stt / ai_thinking). Live for ~0.5–3 s during fetchReportSafe.

Error card with recovery-aware buttons

<div v-if="message.status === 'error'" class="error-card">
  <span class="error-text">{{ message.error }}</span>
  <button v-if="hasRetryButton(message)" class="retry-btn" @click="handleRetry(message)">
    {{ message.recovery === 'restart' ? '返回重开' : '重试' }}
  </button>
  <button v-if="hasRerecordButton(message)" class="rerecord-btn" @click="handleRerecord(message)">
    重录
  </button>
</div>

Helpers (in <script setup>):

function hasRetryButton(m) {
  return m.recovery === 'retry' || m.recovery === 'restart'
}
function hasRerecordButton(m) {
  return m.role === 'student' && (m.recovery === 'retry' || m.recovery === 'rerecord')
}

The 重录 button has secondary styling (outlined or text-link) when alongside 重试; primary styling when standalone (rerecord-only case).

Testing strategy

Frontend unit tests (Vitest)

• classifyError — table-driven across each known raw string + status + role combo.
• useDialogueEngine.beginStudentStream — abort before commit pushes nothing; commit pushes both placeholders; error finalization sets recovery correctly.
• useDialogueEngine.retryMessage / regenerateAiMessage — both pass turnId (mock fetch, assert body shape).

Backend unit tests (pytest)

• dialogue_service.speak happy path — full-miss branch unchanged from current behavior.
• dialogue_service.speak partial-hit branch — pre-insert a student row with client_turn_id=T, call speak with same T, assert no STT call, assert ai row inserted, assert round incremented exactly once.
• dialogue_service.speak full-hit branch — pre-insert student + ai with T, call speak, assert no STT/LLM calls, assert events replayed, assert current_round unchanged.
• Concurrency — two parallel coroutines with same (session, turnId); assert one wins, the other returns the same content via IntegrityError recovery.
• Final-round skip — current_round = total_rounds, call speak, assert no LLM call, assert isComplete: true in done frame, assert no ai row inserted.

Integration tests

• WS happy path with turnId in start frame; client receives transcript + tokens + done.
• WS network drop after final ai INSERT → client retry via HTTP /speak with same turnId → full-hit replay.
• Recording cancellation during starting (mock getUserMedia to delay) — UI returns to idle, no messages pushed, no WS opened.

Manual QA checklist

• Click 开始录音 with mic permission already granted → starting flicker (~50 ms) → recording.
• Click 开始录音 first time (permission prompt) → starting visible for ~1–2 s with cancel affordance → cancel works.
• Cancel during recording → no error bubbles in chat, return to idle.
• Cancel during starting → no error bubbles in chat, return to idle, mic released.
• Throttle network in DevTools, complete a turn, kill connection just after done would have arrived → 重试 yields cached content, round counter stays correct.
• Throw "No speech detected" by recording silence → only 重录 button.
• Throw 404 by manually deleting session in DB → only 返回重开 button.
• Reach last round, click 完成 → no AI bubble, finalizing card → report.

Out-of-scope follow-ups

1. Cancel during stt / ai_thinking. Need server cooperation (cancel WS / abort LLM). Track separately if user research surfaces demand.
2. Replay-fidelity guarantee on full hits. Currently the tokens event is replayed as a single chunk during full-hit; original streaming is lost. If consumers depend on streaming behavior we can store token boundaries; not urgent.
3. Orphan row cleanup. A 重录 leaves a student row in DB with no matching ai row. Reports already group by round, but if storage growth becomes a concern a periodic DELETE FROM dialogue_message WHERE ai_row_missing can be added.
4. Generic idempotency middleware. When a fourth or fifth write endpoint surfaces, revisit textbook A.

Ship plan

Single coordinated PR pair (frontend + backend) since the WS contract changes lockstep. Order of commits within the backend PR:

1. Migration 004_add_client_turn_id.sql.
2. Service-layer lookup helper + three-branch routing (full miss / partial hit / full hit).
3. API-layer turnId validation in HTTP body and WS start frame.
4. Final-round skip-LLM branch.

Frontend ships in the same release window as the backend. No feature flag — the migration is additive (NULL column, no backfill) and the API change is mandatory once the backend ships.