01
<obs>Observe
Keep watching without disturbing the stream when no intervention is needed.
The problem, at a glance
From answering queries to discovering needs
Instead of waiting for a prompt, LiveAssistant continuously reads the social stream and chooses whether to observe, remember, or assist the right participant.
Live qualitative cases
Watch the policy think in time
Each frame is one step in a continuous trajectory. The model can wait, preserve context, or produce a role-aware response only when the stream calls for it.
LIVE TRAJECTORY
10-minute benchmark overview
One stream. Sixty chunks. One selective policy.
This complete 10-minute example visualizes all 60 causal decisions, from state selection to recipient routing, and explains why useful assistance sometimes means not emitting anything at all.
Core idea
Silence is not a failure. It is a decision.
Most streaming models are optimized to keep talking. LiveAssistant reframes livestream understanding as selective participation in a shared, multi-party environment.
02
<mem>Remember
Write a private clue to memory so future decisions retain causal context.
03
<ans>Assist
Speak to viewers, hosts, or moderators with a specific, grounded task.
VideoAudioCommentsGiftsViewer dynamicsRoom metadata
Why LiveAssistant
Built for the social dynamics of live video
A native livestream task requires more than frame understanding. It must reason causally over time, route actions across roles, and learn restraint.
A new problem, not another score
Causal, mixed-initiative decision-making over heterogeneous, long-horizon signals.
Multi-party routing
Viewer, host, and moderator are first-class policy decisions.
Omni-native input
Video, audio, comments, gifts, dynamics, and metadata are fused directly.
Trajectory data engine
Aligned 10-second decisions with automatic checks and human review.
Two-stage training
MA-MSFT learns the grammar; SM-GSPO optimizes the streaming policy.
Method
One continuous path from signals to actions
The same decision trajectory connects annotation, supervised fine-tuning, reinforcement learning, and bounded-cache inference.
Serialize live-native signals
Align video, audio, interaction, gifts, dynamics, and metadata in causal order.
Learn the action grammar
MA-MSFT protects rare structural decisions with marker-aware objectives.
Optimize trajectories
SM-GSPO combines structure, content, turn-level, and trajectory-level rewards.
Reason over long streams
A dense recent window and compressed long-term memory preserve useful context.
Benchmark & results
Strictly causal. Human verified. Room-disjoint.
Evaluation uses native audio, video, comments, and gifts. The non-trivial state distribution penalizes both “always talk” and “always silent” policies.
38.17hbenchmark video
13,812unique chunks
625Kcomments
9content categories
State distribution
OBS 48.2%MEM 25.9%ANS 25.9%
Controlled ablation Progressive training strategy, %
| Training strategy | State overall | OBS recall | MEM recall | ANS recall | Recipient overall | Viewer accuracy | Host accuracy | Task accuracy | Overall average |
|---|---|---|---|---|---|---|---|---|---|
| Ordinary multiturn SFT | 69.36 | 87.71 | 35.99 | 68.64 | 63.54 | 65.77 | 49.70 | 51.12 | 61.34 |
| MA-MSFT | 70.05 | 87.06 | 36.50 | 72.00 | 66.42 | 69.09 | 49.90 | 52.55 | 63.01 |
| MA-MSFT + turn-level GSPO | 69.98 | 89.79 | 28.52 | 74.64 | 68.81 | 71.76 | 50.50 | 54.46 | 64.42 |
| MA-MSFT + SM-GSPO ★ | 71.14 | 78.84 | 48.20 | 79.26 | 72.67 | 75.57 | 56.71 | 58.41 | 67.48 |
What changes with SM-GSPO?
Compared with MA-MSFT, trajectory-level optimization raises the overall average by 4.47 points, with clear gains in memory recall (+11.70), answer recall (+7.26), recipient routing (+6.25), and task accuracy (+5.86).
One assistant, three audiences
Useful to the entire live room
Explain what matters
Real-time narration, newcomer recaps, highlights, and direct answers.
Surface what is emerging
Question clusters, pacing alerts, key moments, and audience feedback.
Flag what needs review
Grounded risk cues, anomalies, and precise segments for follow-up.
Not talking more — knowing when to talk, what to retain, and whom to help.
Citation
Cite LiveAssistant
Read the paper on arXiv:2609.27303 ↗, and explore the official artifacts in our Hugging Face collection ↗.
@article{gao2026liveassistant,
title = {Live Assistant: Learning Whether, When, and Whom to
Assist in Real-World Live Social Streams},
author = {Gao, Shujian and Yan, Jiamei and Yang, Yuchen and
Zhou, Penghao and Wang, Qinglei and Fan, Tiehan and
Wang, Yuan and Wu, Zuxuan and Jiang, Yu-Gang},
journal = {arXiv preprint arXiv:2609.27303},
year = {2026},
doi = {10.48550/arXiv.2609.27303}
}