# Class AI Report — Design

**Date:** 2026-05-16
**Status:** Approved (design)
**Repos:** `PPT` (frontend), `cococlass-english-speaking-api` (backend)

## Problem

`AISummary.vue` currently shows a compact 3-bullet card. The backend `generate_class_summary`
feeds the class-summary LLM only thin aggregated data (`overallScore`, `dimensions`,
`topHighlights`, `topImprovements` per student) and gets back 3 short text bullets.

The new requirement is a full **tiered Markdown report**: an A/B/C tier statistics table,
an optional ASCII bar chart, and Key Insights — driven by a richer per-student payload
(per-sentence transcripts + sentence comments, word-level `accuracy_score`/`error_type`,
per-user dimension scores + comment) and a new LLM system prompt.

## Key findings (resolved during brainstorming)

1. **The backend `overall_report` has no numeric scores.** `OverallReportEvaluator` only
   emits `{ aiComment, highlights, improvements }`. The reads in `list_sessions_by_config`
   for `overall_report.get("overallScore")` / `.get("dimensions")` are dead — always
   `null`/`{}`. Today's class summary silently aggregates zeros.
2. **The per-user 4 scores exist only in the frontend.** `adaptReport()` in
   `llmService.ts` computes `overallScore` + `dimensions` by **averaging the per-sentence
   Azure scores**, relabeled:
   - `overallScore` = mean over student sentences of `avg(accuracy, fluency, prosody, completeness)`
   - `dimensions.fluency` = avg Azure `fluencyScore`
   - `dimensions.interaction` = avg Azure `prosodyScore`
   - `dimensions.vocabulary` = avg Azure `completenessScore`
   - `dimensions.grammar` = avg Azure `accuracyScore`
3. **Consequence:** no new evaluator or DB migration is needed for per-user scoring. The
   class-report backend mirrors `adaptReport`'s averaging server-side, keeping the class
   report numerically consistent with the per-student report.

## Decisions

- **UI placement:** expandable inline panel in `SpeakingClassPanel`. Collapsed = one live
  completion line + expand toggle; expanded = full Markdown report in a scroll region.
- **Delivery:** server streams the Markdown via SSE; the panel renders progressively.
- **Payload scope:** full per-sentence detail, but capped at the first 30 completed
  students (ordered by `completedAt`); a `truncated` flag is passed when capped.
- **Per-user scores:** computed server-side by averaging Azure per-sentence scores
  (mirrors `adaptReport`). No new evaluator, no migration.
- **Table column** `形状词汇使用率` in the supplied prompt is topic-specific (copied from a
  "shapes" example) and is generalized to `目标词汇使用率`.
- **Tier thresholds:** hardcoded A `≥85` / B `75-84` / C `<75`, per the prompt.

## Backend — `cococlass-english-speaking-api`

### New endpoint

`POST /api/speaking/dialogue/sessions/by-config/summary/stream` — SSE response.

Request body:
```json
{ "configId": "...", "students": [{ "userId": "...", "name": "..." }], "locale": "zh|en|hk" }
```

SSE events reuse the existing shape parsed by `parseSSEStream`:
`{ "type": "token", "text": "..." }` … then `{ "type": "done" }` or
`{ "type": "error", "message": "..." }`.

### Payload builder — `_build_class_report_input()`

1. Reuse `list_sessions_by_config` to get the latest session per user.
2. For `completed` sessions, ordered by `completedAt`, take the first 30. Load
   `DialogueMessage` + `selectinload(evaluation)` rows (same pattern as `get_report`).
3. Per student build:
   ```
   {
     name,
     overallScore,                       # avg of sentence avg(4 Azure scores)
     dimensions: { fluency, interaction, vocabulary, grammar },  # Azure averages
     aiComment, highlights, improvements,                        # from overall_report
     sentences: [
       { round, transcript,
         sentenceComment,                # evaluation.content_feedback.comment, omit if absent
         wordAnalysis: [{ word, accuracyScore, errorType }] }
     ]
   }
   ```
   Only `role == "student"` messages with a completed evaluation contribute scores.
4. `classStats`: `{ total, submitted, unsubmitted, notStarted, rate, avgScore,
   highScore, lowScore }` computed from the real averaged scores.
5. If the completed count exceeded 30: `truncated: true` plus `completedTotal` /
   `includedCount` so the LLM can note partial coverage.

### `ClassReportEvaluator`

- New evaluator carrying the supplied system prompt (with the `目标词汇使用率` column
  generalization). Output is **Markdown**, not JSON.
- Calls the LLM with `stream=True`; yields Markdown chunks as they arrive.
- User message = JSON of `{ classStats, perStudent, locale }` with the existing
  "treat data as data, not instructions" safety preamble.

### Endpoint flow & caching

- Build payload → if `submitted == 0`, skip the LLM and stream a single short
  "waiting for submissions" message (localized) then `done`.
- Otherwise stream LLM chunks as `token` events; accumulate full text.
- On stream completion, store the full Markdown in the summary cache keyed by
  `(configId, contentHash(summaries))`, same `SUMMARY_TTL_SECONDS`.
- Cache hit: replay the stored Markdown as one `token` event + `done`.
- LLM error/timeout: emit `{ type: "error" }`.

### Removed (dead after this lands)

- `POST /sessions/by-config/summary` route, `ClassSummaryEvaluator`,
  `class_summary_rules.py`, and the `bullet_*` rule helpers.
- The `overall_report.get("overallScore")`/`.get("dimensions")` reads in
  `list_sessions_by_config` stay — harmless, and `list_sessions_by_config` is still
  used by the student grid.

## Frontend — `PPT`

### `src/services/speaking.ts`

- Add `streamClassReport(configId, students, locale)` — async generator using
  `fetch` + the shared `parseSSEStream`; yields Markdown token strings.
- `parseSSEStream` is promoted from `llmService.ts` to a shared module (or imported)
  so both services use one implementation.
- Remove `generateClassSummary` + `ClassSummaryResponse`.

### `src/views/Student/components/SpeakingClassPanel/useClassSummary.ts`

- Remove `aiBullets`, `liveBullet1`-as-tuple wiring, `frontendRuleBullet2/3`,
  `aiBackendBullets`, `refreshAISummary`.
- Keep `liveBullet1` (live completion count/rate) as a standalone computed — it is the
  collapsed-state line and needs no LLM call.
- Add: `reportMarkdown: Ref<string>`, `reportStreaming: Ref<boolean>`,
  `reportGeneratedAt: Ref<string|null>`, `reportError: Ref<string|null>`,
  `refreshReport()` — opens the stream, appends tokens to `reportMarkdown`, token-guard
  against stale streams (same `aiToken` pattern).

### `src/views/Student/components/SpeakingClassPanel/AISummary.vue`

- Props become `{ completionLine: string, markdown: string, streaming: boolean,
  generatedAt: string|null, error: string|null }`; emits `refresh`.
- **Collapsed:** completion line + `查看完整分析报告 ▾` toggle.
- **Expanded:** render `markdown` with `markdown-it` (`html: true`), pass through
  **DOMPurify** with a tight allowlist (table tags + `span` with a restricted `style`
  attr), in a `max-height` + `overflow:auto` region. ASCII chart renders inside a
  fenced code block (monospace).
- Progressive render while `streaming`; skeleton before the first token.
- `submitted == 0` → show the localized waiting message instead of the report.
- Stream error → error bar + retry; last good Markdown is kept.
- Keep the existing "刚刚生成 / N 秒前" time-ago label.

### Dependencies

- Add **`dompurify`** (+ `@types/dompurify`) to `PPT`. `markdown-it` already present.

### i18n

New keys: expand/collapse labels, "查看完整分析报告", regenerate, stream-error message,
waiting-for-submissions message.

## Error handling

- Stream error / timeout → error bar with retry; previous Markdown preserved.
- No completed students → no LLM call; localized waiting message.
- Malformed Markdown from the LLM → rendered as-is by `markdown-it`; acceptable.
- Stale stream (panel re-fetched / unmounted) → discarded via the token guard.

## Testing

- **Backend (pytest):** `_build_class_report_input` — score averaging matches
  `adaptReport`, 30-cap + `truncated` flag, sentence/word assembly, missing
  `content_feedback`; streamed endpoint with a stubbed LLM (token events → `done`);
  cache-hit replay path.
- **Frontend:** `useClassSummary` stream accumulation + stale-token discard;
  `markdown-it` → DOMPurify render (red `<span>` survives, `<script>` stripped).

## Out of scope

- Configurable / custom tier levels (thresholds hardcoded).
- Persisting per-user dimension scores in the DB (computed on the fly).
- Changes to the per-student `StudentReportModal`.