Experience · Software component
Stream Connection Manager
Software componentExperienceExperience & Human Oversightarc:StreamConnectionManager
A session component that tracks open streaming connections and their users, detects client disconnects and idle timeouts, and releases the associated generation resources.
Responsibility. Manages the lifecycle of streaming connections so abandoned streams do not consume server resources.
Also known as: Connection manager, Disconnection detector
Relationships
is invoked by dependency
reads dependency
- Conversation State Store abstract Ch2.9
writes dependency
- Conversation State Store abstract Ch2.9
monitors assurance
- Streaming Transport abstract Ch2.9
Design guidance
- SHOULD detect client disconnection and stop generation to prevent zombie streams.
- SHOULD define idle-connection timeouts and resume behaviour for interrupted conversations.
Classification
- Patterns
- HeartbeatIdle timeoutReconnection and resume
- Quality attributes
- Performance efficiency (ISO/IEC 25010)Reliability (ISO/IEC 25010 | NIST AI RMF: valid and reliable)
- Risks mitigated
- Zombie streams consuming server resources after client disconnectLost conversation state on dropped connections
Sources
- Ch2.9: T. Nguyen, "Streaming and Real-Time Responses," in Mastering Agentic AI Systems: Guide for the NVIDIA NCP-AAI Exam, 1st ed. 2026, ch. 2.9. ISBN: 9798244538229.