# Streaming Real-time streaming responses --- Enable streaming by setting `stream: true` in your request. ## Request ```json { "model": "myapp-123", "messages": [{"role": "user", "content": "Hello!"}], "stream": true } ``` ## Response Format Server-sent events with JSON chunks. Every chunk carries `chatSessionId`, the session this turn belongs to: ``` data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1699451234,"model":"myapp-123","chatSessionId":"550e8400-e29b-41d4-a716-446655440000","choices":[{"index":0,"delta":{"role":"assistant"},"finish_reason":null}]} data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1699451234,"model":"myapp-123","chatSessionId":"550e8400-e29b-41d4-a716-446655440000","choices":[{"index":0,"delta":{"content":"Hello"},"finish_reason":null}]} data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1699451234,"model":"myapp-123","chatSessionId":"550e8400-e29b-41d4-a716-446655440000","choices":[{"index":0,"delta":{"content":" there!"},"finish_reason":null}]} data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1699451234,"model":"myapp-123","chatSessionId":"550e8400-e29b-41d4-a716-446655440000","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]} data: [DONE] ``` ## JavaScript Example ```javascript async function streamChat(model, messages) { const response = await fetch('https://build.chipp.ai/api/v1/chat/completions', { method: 'POST', headers: { 'Content-Type': 'application/json', 'Authorization': `Bearer ${process.env.CHIPP_API_KEY}` }, body: JSON.stringify({ model, messages, stream: true }) }); const reader = response.body.getReader(); const decoder = new TextDecoder(); let chatSessionId; while (true) { const { done, value } = await reader.read(); if (done) break; const chunk = decoder.decode(value); const lines = chunk.split('\n'); for (const line of lines) { if (line.startsWith('data: ')) { const data = line.slice(6); if (data === '[DONE]') return chatSessionId; let parsed; try { parsed = JSON.parse(data); } catch (e) { continue; // A partial line; a production client should buffer it. } if (parsed.error) { throw new Error(`Chipp stream error (${parsed.error.code}): ${parsed.error.message}`); } chatSessionId = parsed.chatSessionId; // Send it on your next call. process.stdout.write(parsed.choices[0]?.delta?.content || ''); } } } } ``` ## Python Example ```python import requests import json response = requests.post( 'https://build.chipp.ai/api/v1/chat/completions', headers={ 'Content-Type': 'application/json', 'Authorization': f'Bearer {os.environ["CHIPP_API_KEY"]}' }, json={ 'model': 'myapp-123', 'messages': [{'role': 'user', 'content': 'Hello!'}], 'stream': True }, stream=True ) chat_session_id = None for line in response.iter_lines(): if line: line = line.decode('utf-8') if line.startswith('data: '): data = line[6:] if data == '[DONE]': break chunk = json.loads(data) if 'error' in chunk: raise RuntimeError(f"Chipp stream error ({chunk['error']['code']}): {chunk['error']['message']}") chat_session_id = chunk['chatSessionId'] # send it on your next call if chunk['choices']: print(chunk['choices'][0]['delta'].get('content', ''), end='', flush=True) ``` ## OpenAI SDK ```javascript import OpenAI from 'openai'; const openai = new OpenAI({ apiKey: process.env.CHIPP_API_KEY, baseURL: 'https://build.chipp.ai/api/v1' }); const stream = await openai.chat.completions.create({ model: 'myapp-123', messages: [{ role: 'user', content: 'Hello!' }], stream: true }); let chatSessionId; for await (const chunk of stream) { // A Chipp field: not in the OpenAI types, but present on every chunk. chatSessionId = chunk.chatSessionId; process.stdout.write(chunk.choices[0]?.delta?.content || ''); } ``` The OpenAI SDKs raise an `APIError` when the stream carries an error event (see [Errors During a Stream](#errors-during-a-stream)), so wrap the loop in `try`/`catch`. ## Chunk Structure Each chunk contains: - `id`: Same for all chunks in a response - `object`: Always `"chat.completion.chunk"` - `created`: Unix timestamp - `model`: Your app name - `choices[0].delta`: Content increment (the first chunk carries `{"role":"assistant"}` and no content) - `choices[0].finish_reason`: `null` until the last chunk - `chatSessionId`: The chat session ID. The same on every chunk of a response. This is a Chipp field, not part of the OpenAI chunk format, and OpenAI clients ignore it ## Continuing a Conversation Read `chatSessionId` from any chunk (the first one is enough) and send it on your next request, exactly as with a non-streaming response: ```json { "model": "myapp-123", "messages": [{"role": "user", "content": "And what about Pro?"}], "stream": true, "chatSessionId": "550e8400-e29b-41d4-a716-446655440000" } ``` You can also read the whole conversation back later with [`GET /api/v1/chat/transcript/{sessionId}`](/docs/api/chat-transcript). ## Errors During a Stream Errors found before the stream starts (bad API key, unknown model, insufficient credits, rate limit) return a normal HTTP error status with a JSON body. See the [API Reference](/docs/api/reference#status-codes). An error after the stream has started cannot change the HTTP status, which is already `200`. Instead, the stream sends one error event, then `[DONE]`. There is no `finish_reason: "stop"` chunk, so a failed turn never looks like a complete reply: ``` data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1699451234,"model":"myapp-123","chatSessionId":"550e8400-e29b-41d4-a716-446655440000","choices":[{"index":0,"delta":{"content":"Hello"},"finish_reason":null}]} data: {"id":"chatcmpl-abc123","object":"chat.completion.chunk","created":1699451234,"model":"myapp-123","chatSessionId":"550e8400-e29b-41d4-a716-446655440000","choices":[],"error":{"message":"The AI model is temporarily unavailable. Please try again in a moment.","type":"server_error","param":null,"code":"service_unavailable"}} data: [DONE] ``` | Field | Description | |-------|-------------| | `error.message` | A human-readable message, safe to show to an end user | | `error.type` | OpenAI error type: `server_error`, `rate_limit_error`, `insufficient_quota`, or `invalid_request_error` | | `error.code` | A Chipp error code, for example `service_unavailable`, `rate_limited`, `billing_exhausted`, `timeout`, `input_too_large`, `content_filter`, `generic` | | `error.param` | Always `null` | | `choices` | Always an empty array | Text that arrived before the error event is a partial reply. Chipp saves your user message but not the partial reply. To retry, send the message again with the same `chatSessionId` (the transcript then shows your message twice).