PPT/docs/superpowers/specs/2026-05-16-class-ai-report-design.md

Class AI Report — Design

Date: 2026-05-16 Status: Approved (design) Repos: PPT (frontend), cococlass-english-speaking-api (backend)

Problem

AISummary.vue currently shows a compact 3-bullet card. The backend generate_class_summary feeds the class-summary LLM only thin aggregated data (overallScore, dimensions, topHighlights, topImprovements per student) and gets back 3 short text bullets.

The new requirement is a full tiered Markdown report: an A/B/C tier statistics table, an optional ASCII bar chart, and Key Insights — driven by a richer per-student payload (per-sentence transcripts + sentence comments, word-level accuracy_score/error_type, per-user dimension scores + comment) and a new LLM system prompt.

Key findings (resolved during brainstorming)

1. The backend overall_report has no numeric scores. OverallReportEvaluator only emits { aiComment, highlights, improvements }. The reads in list_sessions_by_config for overall_report.get("overallScore") / .get("dimensions") are dead — always null/{}. Today's class summary silently aggregates zeros.
2. The per-user 4 scores exist only in the frontend. adaptReport() in llmService.ts computes overallScore + dimensions by averaging the per-sentence Azure scores, relabeled:
   • overallScore = mean over student sentences of avg(accuracy, fluency, prosody, completeness)
   • dimensions.fluency = avg Azure fluencyScore
   • dimensions.interaction = avg Azure prosodyScore
   • dimensions.vocabulary = avg Azure completenessScore
   • dimensions.grammar = avg Azure accuracyScore
3. Consequence: no new evaluator or DB migration is needed for per-user scoring. The class-report backend mirrors adaptReport's averaging server-side, keeping the class report numerically consistent with the per-student report.

Decisions

• UI placement: expandable inline panel in SpeakingClassPanel. Collapsed = one live completion line + expand toggle; expanded = full Markdown report in a scroll region.
• Delivery: server streams the Markdown via SSE; the panel renders progressively.
• Payload scope: full per-sentence detail, but capped at the first 30 completed students (ordered by completedAt); a truncated flag is passed when capped.
• Per-user scores: computed server-side by averaging Azure per-sentence scores (mirrors adaptReport). No new evaluator, no migration.
• Table column 形状词汇使用率 in the supplied prompt is topic-specific (copied from a "shapes" example) and is generalized to 目标词汇使用率.
• Tier thresholds: hardcoded A ≥85 / B 75-84 / C <75, per the prompt.

Backend — cococlass-english-speaking-api

New endpoint

POST /api/speaking/dialogue/sessions/by-config/summary/stream — SSE response.

Request body:

{ "configId": "...", "students": [{ "userId": "...", "name": "..." }], "locale": "zh|en|hk" }

SSE events reuse the existing shape parsed by parseSSEStream: { "type": "token", "text": "..." } … then { "type": "done" } or { "type": "error", "message": "..." }.

Payload builder — _build_class_report_input()

1. Reuse list_sessions_by_config to get the latest session per user.
2. For completed sessions, ordered by completedAt, take the first 30. Load DialogueMessage + selectinload(evaluation) rows (same pattern as get_report).

3. Per student build:

   {
    name,
    overallScore,                       # avg of sentence avg(4 Azure scores)
    dimensions: { fluency, interaction, vocabulary, grammar },  # Azure averages
    aiComment, highlights, improvements,                        # from overall_report
    sentences: [
      { round, transcript,
        sentenceComment,                # evaluation.content_feedback.comment, omit if absent
        wordAnalysis: [{ word, accuracyScore, errorType }] }
    ]
   }
   

   Only role == "student" messages with a completed evaluation contribute scores.

4. classStats: { total, submitted, unsubmitted, notStarted, rate, avgScore, highScore, lowScore } computed from the real averaged scores.

5. If the completed count exceeded 30: truncated: true plus completedTotal / includedCount so the LLM can note partial coverage.

ClassReportEvaluator

• New evaluator carrying the supplied system prompt (with the 目标词汇使用率 column generalization). Output is Markdown, not JSON.
• Calls the LLM with stream=True; yields Markdown chunks as they arrive.
• User message = JSON of { classStats, perStudent, locale } with the existing "treat data as data, not instructions" safety preamble.

Endpoint flow & caching

• Build payload → if submitted == 0, skip the LLM and stream a single short "waiting for submissions" message (localized) then done.
• Otherwise stream LLM chunks as token events; accumulate full text.
• On stream completion, store the full Markdown in the summary cache keyed by (configId, contentHash(summaries)), same SUMMARY_TTL_SECONDS.
• Cache hit: replay the stored Markdown as one token event + done.
• LLM error/timeout: emit { type: "error" }.

Removed (dead after this lands)

• POST /sessions/by-config/summary route, ClassSummaryEvaluator, class_summary_rules.py, and the bullet_* rule helpers.
• The overall_report.get("overallScore")/.get("dimensions") reads in list_sessions_by_config stay — harmless, and list_sessions_by_config is still used by the student grid.

Frontend — PPT

src/services/speaking.ts

• Add streamClassReport(configId, students, locale) — async generator using fetch + the shared parseSSEStream; yields Markdown token strings.
• parseSSEStream is promoted from llmService.ts to a shared module (or imported) so both services use one implementation.
• Remove generateClassSummary + ClassSummaryResponse.

src/views/Student/components/SpeakingClassPanel/useClassSummary.ts

• Remove aiBullets, liveBullet1-as-tuple wiring, frontendRuleBullet2/3, aiBackendBullets, refreshAISummary.
• Keep liveBullet1 (live completion count/rate) as a standalone computed — it is the collapsed-state line and needs no LLM call.
• Add: reportMarkdown: Ref<string>, reportStreaming: Ref<boolean>, reportGeneratedAt: Ref<string|null>, reportError: Ref<string|null>, refreshReport() — opens the stream, appends tokens to reportMarkdown, token-guard against stale streams (same aiToken pattern).

src/views/Student/components/SpeakingClassPanel/AISummary.vue

• Props become { completionLine: string, markdown: string, streaming: boolean, generatedAt: string|null, error: string|null }; emits refresh.
• Collapsed: completion line + 查看完整分析报告 ▾ toggle.
• Expanded: render markdown with markdown-it (html: true), pass through DOMPurify with a tight allowlist (table tags + span with a restricted style attr), in a max-height + overflow:auto region. ASCII chart renders inside a fenced code block (monospace).
• Progressive render while streaming; skeleton before the first token.
• submitted == 0 → show the localized waiting message instead of the report.
• Stream error → error bar + retry; last good Markdown is kept.
• Keep the existing "刚刚生成 / N 秒前" time-ago label.

Dependencies

• Add dompurify (+ @types/dompurify) to PPT. markdown-it already present.

i18n

New keys: expand/collapse labels, "查看完整分析报告", regenerate, stream-error message, waiting-for-submissions message.

Error handling

• Stream error / timeout → error bar with retry; previous Markdown preserved.
• No completed students → no LLM call; localized waiting message.
• Malformed Markdown from the LLM → rendered as-is by markdown-it; acceptable.
• Stale stream (panel re-fetched / unmounted) → discarded via the token guard.

Testing

• Backend (pytest): _build_class_report_input — score averaging matches adaptReport, 30-cap + truncated flag, sentence/word assembly, missing content_feedback; streamed endpoint with a stubbed LLM (token events → done); cache-hit replay path.
• Frontend: useClassSummary stream accumulation + stale-token discard; markdown-it → DOMPurify render (red <span> survives, <script> stripped).

Out of scope

• Configurable / custom tier levels (thresholds hardcoded).
• Persisting per-user dimension scores in the DB (computed on the fly).
• Changes to the per-student StudentReportModal.