semantic-live-caption
Chinese real-time captioning app where Jev annotates each spoken sentence for key points, emotion and intent as it streams.
Overview
Semantic live caption (听写纸) is a real-time transcription tool, documented in Chinese, for meetings, interviews, podcasts and live streams. Streaming speech recognition from NetEase Youdao's Confucius4-R2T2 produces append-only captions with around 100 ms latency, and TypeSafe Jev annotates each sentence: key points are bolded and underlined, emotions shown as a note and background colour, and intent stamped at the end (question, action, agree, objection). DeepSeek can optionally listen at low frequency and propose new labels.
Annotation runs at two levels: a live pass mid-sentence and a final pass when the sentence closes. One Jev call fans out an is_key Noul, an emotion Choice and an intent Choice plus speculative Noul re-checks per visible intent, all in parallel. Intent must pass four gates: the choice hit, confidence at least 0.55, a margin of at least 0.12, and a Noul re-check (0.70 for high-risk categories like objection). The README notes Jev's calibration on Chinese is weaker than English and that adding English label descriptions helps. ASR needs an NVIDIA GPU with 16 GB or more.
What the decision model decides
| Type | Question | Answers |
|---|---|---|
| Yes / no | is_key: Is this sentence a key point? | — |
| Choice | emotion: Which emotion does this sentence express? | — |
| Choice | intent: What is the intent (question, action, agree, objection)? | question · action · agree · objection |
| Yes / no | Re-check: does this sentence really express the chosen intent? | — |
How it works
- Stream microphone audio to the ASR service
- On each partial and closed sentence, call Jev with parallel questions
- Apply key-point threshold (0.60) and the four intent gates
- Render bold, colour and stamp annotations; optionally propose new labels via DeepSeek
Strengths
- Annotations appear nearly in sync with text
- Prefers no label over a wrong one
- Fully customisable labels without code changes
Caveats
- Jev calibration weaker for Chinese per the README
- ASR needs a 16 GB+ NVIDIA GPU
- Commercial use subject to upstream model licences
Getting started
Deploy Confucius4-R2T2 ASR on a GPU server, run server/server.py, open web/index.html and enter a TypeSafe key.At a glance
- Type
- Project
- Status
- prototype
- Language
- Python
- Runtime
- ASR service + stdlib proxy + single-file web page
- Has code
- yes
- License
- MIT
Use cases and integrations
How curated lists describe it
听写纸 · Real-time speech captions with live semantic annotation (key points / emotion / intent) — Confucius4-R2T2 + TypeSafe Jev + DeepSeek.
hellogumbo/awesome-jev
Frequently asked questions
What does Jev decide?
Whether each sentence is a key point, its emotion, and its intent, with a re-check before stamping intent.
Does it work well in Chinese?
The README says Jev's calibration on Chinese is weaker and low-confidence results are left unlabelled.
Where is the API key stored?
In browser localStorage, passed through a same-origin proxy and not stored server-side.
Related projects
Android WeChat assistant showing Jev emotion probabilities and intent readings beside messages; never sends replies.
Appbeta- jev-chat-windows★ 802
Windows WeChat reply assistant: local offline OCR reads the chat, Jev judges intent and tension and ranks three drafted replies to fill in.
ProjectproductionPython DeepSeek Harness plugin that classifies each user message's emotion, urgency and department with Jev and injects it as context.
PluginbetaTypeScriptAndroid chat co-pilot that reads the active chat via Accessibility and returns typed verdicts on intent, emotion and risk; never replies.
AppbetaKotlinZero-dependency Android app that judges the visible WeChat message's intent, emotion, urgency and reply posture with Jev.
AppbetaJavaAndroid WeChat assistant where Jev judges the chat and scores the relationship, and a chat model drafts three replies.
AppbetaKotlin