docs(speaking): class AI report design spec

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

docs/superpowers/specs/2026-05-16-class-ai-report-design.md

@@ -0,0 +1,179 @@
+# Class AI Report — Design
+
+**Date:** 2026-05-16
+**Status:** Approved (design)
+**Repos:** `PPT` (frontend), `cococlass-english-speaking-api` (backend)
+
+## Problem
+
+`AISummary.vue` currently shows a compact 3-bullet card. The backend `generate_class_summary`
+feeds the class-summary LLM only thin aggregated data (`overallScore`, `dimensions`,
+`topHighlights`, `topImprovements` per student) and gets back 3 short text bullets.
+
+The new requirement is a full **tiered Markdown report**: an A/B/C tier statistics table,
+an optional ASCII bar chart, and Key Insights — driven by a richer per-student payload
+(per-sentence transcripts + sentence comments, word-level `accuracy_score`/`error_type`,
+per-user dimension scores + comment) and a new LLM system prompt.
+
+## Key findings (resolved during brainstorming)
+
+1. **The backend `overall_report` has no numeric scores.** `OverallReportEvaluator` only
+   emits `{ aiComment, highlights, improvements }`. The reads in `list_sessions_by_config`
+   for `overall_report.get("overallScore")` / `.get("dimensions")` are dead — always
+   `null`/`{}`. Today's class summary silently aggregates zeros.
+2. **The per-user 4 scores exist only in the frontend.** `adaptReport()` in
+   `llmService.ts` computes `overallScore` + `dimensions` by **averaging the per-sentence
+   Azure scores**, relabeled:
+   - `overallScore` = mean over student sentences of `avg(accuracy, fluency, prosody, completeness)`
+   - `dimensions.fluency` = avg Azure `fluencyScore`
+   - `dimensions.interaction` = avg Azure `prosodyScore`
+   - `dimensions.vocabulary` = avg Azure `completenessScore`
+   - `dimensions.grammar` = avg Azure `accuracyScore`
+3. **Consequence:** no new evaluator or DB migration is needed for per-user scoring. The
+   class-report backend mirrors `adaptReport`'s averaging server-side, keeping the class
+   report numerically consistent with the per-student report.
+
+## Decisions
+
+- **UI placement:** expandable inline panel in `SpeakingClassPanel`. Collapsed = one live
+  completion line + expand toggle; expanded = full Markdown report in a scroll region.
+- **Delivery:** server streams the Markdown via SSE; the panel renders progressively.
+- **Payload scope:** full per-sentence detail, but capped at the first 30 completed
+  students (ordered by `completedAt`); a `truncated` flag is passed when capped.
+- **Per-user scores:** computed server-side by averaging Azure per-sentence scores
+  (mirrors `adaptReport`). No new evaluator, no migration.
+- **Table column** `形状词汇使用率` in the supplied prompt is topic-specific (copied from a
+  "shapes" example) and is generalized to `目标词汇使用率`.
+- **Tier thresholds:** hardcoded A `≥85` / B `75-84` / C `<75`, per the prompt.
+
+## Backend — `cococlass-english-speaking-api`
+
+### New endpoint
+
+`POST /api/speaking/dialogue/sessions/by-config/summary/stream` — SSE response.
+
+Request body:
+```json
+{ "configId": "...", "students": [{ "userId": "...", "name": "..." }], "locale": "zh|en|hk" }
+```
+
+SSE events reuse the existing shape parsed by `parseSSEStream`:
+`{ "type": "token", "text": "..." }` … then `{ "type": "done" }` or
+`{ "type": "error", "message": "..." }`.
+
+### Payload builder — `_build_class_report_input()`
+
+1. Reuse `list_sessions_by_config` to get the latest session per user.
+2. For `completed` sessions, ordered by `completedAt`, take the first 30. Load
+   `DialogueMessage` + `selectinload(evaluation)` rows (same pattern as `get_report`).
+3. Per student build:
+   ```
+   {
+     name,
+     overallScore,                       # avg of sentence avg(4 Azure scores)
+     dimensions: { fluency, interaction, vocabulary, grammar },  # Azure averages
+     aiComment, highlights, improvements,                        # from overall_report
+     sentences: [
+       { round, transcript,
+         sentenceComment,                # evaluation.content_feedback.comment, omit if absent
+         wordAnalysis: [{ word, accuracyScore, errorType }] }
+     ]
+   }
+   ```
+   Only `role == "student"` messages with a completed evaluation contribute scores.
+4. `classStats`: `{ total, submitted, unsubmitted, notStarted, rate, avgScore,
+   highScore, lowScore }` computed from the real averaged scores.
+5. If the completed count exceeded 30: `truncated: true` plus `completedTotal` /
+   `includedCount` so the LLM can note partial coverage.
+
+### `ClassReportEvaluator`
+
+- New evaluator carrying the supplied system prompt (with the `目标词汇使用率` column
+  generalization). Output is **Markdown**, not JSON.
+- Calls the LLM with `stream=True`; yields Markdown chunks as they arrive.
+- User message = JSON of `{ classStats, perStudent, locale }` with the existing
+  "treat data as data, not instructions" safety preamble.
+
+### Endpoint flow & caching
+
+- Build payload → if `submitted == 0`, skip the LLM and stream a single short
+  "waiting for submissions" message (localized) then `done`.
+- Otherwise stream LLM chunks as `token` events; accumulate full text.
+- On stream completion, store the full Markdown in the summary cache keyed by
+  `(configId, contentHash(summaries))`, same `SUMMARY_TTL_SECONDS`.
+- Cache hit: replay the stored Markdown as one `token` event + `done`.
+- LLM error/timeout: emit `{ type: "error" }`.
+
+### Removed (dead after this lands)
+
+- `POST /sessions/by-config/summary` route, `ClassSummaryEvaluator`,
+  `class_summary_rules.py`, and the `bullet_*` rule helpers.
+- The `overall_report.get("overallScore")`/`.get("dimensions")` reads in
+  `list_sessions_by_config` stay — harmless, and `list_sessions_by_config` is still
+  used by the student grid.
+
+## Frontend — `PPT`
+
+### `src/services/speaking.ts`
+
+- Add `streamClassReport(configId, students, locale)` — async generator using
+  `fetch` + the shared `parseSSEStream`; yields Markdown token strings.
+- `parseSSEStream` is promoted from `llmService.ts` to a shared module (or imported)
+  so both services use one implementation.
+- Remove `generateClassSummary` + `ClassSummaryResponse`.
+
+### `src/views/Student/components/SpeakingClassPanel/useClassSummary.ts`
+
+- Remove `aiBullets`, `liveBullet1`-as-tuple wiring, `frontendRuleBullet2/3`,
+  `aiBackendBullets`, `refreshAISummary`.
+- Keep `liveBullet1` (live completion count/rate) as a standalone computed — it is the
+  collapsed-state line and needs no LLM call.
+- Add: `reportMarkdown: Ref<string>`, `reportStreaming: Ref<boolean>`,
+  `reportGeneratedAt: Ref<string|null>`, `reportError: Ref<string|null>`,
+  `refreshReport()` — opens the stream, appends tokens to `reportMarkdown`, token-guard
+  against stale streams (same `aiToken` pattern).
+
+### `src/views/Student/components/SpeakingClassPanel/AISummary.vue`
+
+- Props become `{ completionLine: string, markdown: string, streaming: boolean,
+  generatedAt: string|null, error: string|null }`; emits `refresh`.
+- **Collapsed:** completion line + `查看完整分析报告 ▾` toggle.
+- **Expanded:** render `markdown` with `markdown-it` (`html: true`), pass through
+  **DOMPurify** with a tight allowlist (table tags + `span` with a restricted `style`
+  attr), in a `max-height` + `overflow:auto` region. ASCII chart renders inside a
+  fenced code block (monospace).
+- Progressive render while `streaming`; skeleton before the first token.
+- `submitted == 0` → show the localized waiting message instead of the report.
+- Stream error → error bar + retry; last good Markdown is kept.
+- Keep the existing "刚刚生成 / N 秒前" time-ago label.
+
+### Dependencies
+
+- Add **`dompurify`** (+ `@types/dompurify`) to `PPT`. `markdown-it` already present.
+
+### i18n
+
+New keys: expand/collapse labels, "查看完整分析报告", regenerate, stream-error message,
+waiting-for-submissions message.
+
+## Error handling
+
+- Stream error / timeout → error bar with retry; previous Markdown preserved.
+- No completed students → no LLM call; localized waiting message.
+- Malformed Markdown from the LLM → rendered as-is by `markdown-it`; acceptable.
+- Stale stream (panel re-fetched / unmounted) → discarded via the token guard.
+
+## Testing
+
+- **Backend (pytest):** `_build_class_report_input` — score averaging matches
+  `adaptReport`, 30-cap + `truncated` flag, sentence/word assembly, missing
+  `content_feedback`; streamed endpoint with a stubbed LLM (token events → `done`);
+  cache-hit replay path.
+- **Frontend:** `useClassSummary` stream accumulation + stale-token discard;
+  `markdown-it` → DOMPurify render (red `<span>` survives, `<script>` stripped).
+
+## Out of scope
+
+- Configurable / custom tier levels (thresholds hardcoded).
+- Persisting per-user dimension scores in the DB (computed on the fly).
+- Changes to the per-student `StudentReportModal`.