Client Events

Overview

These events are accepted by the Realtime servers via WebRTC datachannel from the client. Each event has a specific structure and purpose for managing realtime interactions.

`session.update`

Send this event to update the session’s default configuration. The client may send this event at any time to update any field, except for voice. Note that once a session has been initialized with a particular model, it can’t be changed to another model using session.update. When the server receives a session.update, it will respond with a session.updated event showing the full, effective configuration. Only the fields that are present are updated. To clear a field like instructions, pass an empty string.

Properties

string

Client-generated ID used to identify this event.

string

required

The event type, must be session.update.

object

required

Realtime session object configuration.

{
    "event_id": "event_123",
    "type": "session.update",
    "session": {
        "modalities": ["text", "audio"],
        "instructions": "You are a helpful assistant.",
        "voice": "sage",
        "input_audio_format": "pcm16",
        "output_audio_format": "pcm16",
        "input_audio_transcription": {
            "model": "whisper-1"
        },
        "turn_detection": {
            "type": "server_vad",
            "threshold": 0.5,
            "prefix_padding_ms": 300,
            "silence_duration_ms": 500,
            "create_response": true
        },
        "tools": [
            {
                "type": "function",
                "name": "get_weather",
                "description": "Get the current weather...",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "location": { "type": "string" }
                    },
                    "required": ["location"]
                }
            }
        ],
        "tool_choice": "auto",
        "temperature": 0.8,
        "max_response_output_tokens": "inf"
    }
}

`conversation.item.create`

Add a new Item to the Conversation’s context, including messages, function calls, and function call responses. This event can be used both to populate a “history” of the conversation and to add new items mid-stream, but has the current limitation that it cannot populate assistant audio messages.

Properties

string

Client-generated ID used to identify this event.

string

required

The event type, must be conversation.item.create.

string

The ID of the preceding item after which the new item will be inserted. If not set, the new item will be appended to the end of the conversation.

object

required

The item to add to the conversation.

{
    "event_id": "event_345",
    "type": "conversation.item.create",
    "previous_item_id": null,
    "item": {
        "id": "msg_001",
        "type": "message",
        "role": "user",
        "content": [
            {
                "type": "input_text",
                "text": "Hello, how are you?"
            }
        ]
    }
}

`response.create`

This event instructs the server to create a Response, which means triggering model inference. When in Server VAD mode, the server will create Responses automatically.

Properties

string

Client-generated ID used to identify this event.

string

required

The event type, must be response.create.

object

required

Create a new Realtime response with these parameters.

{
    "event_id": "event_234",
    "type": "response.create",
    "response": {
        "modalities": ["text", "audio"],
        "instructions": "Please assist the user.",
        "voice": "sage",
        "output_audio_format": "pcm16",
        "tools": [
            {
                "type": "function",
                "name": "calculate_sum",
                "description": "Calculates the sum of two numbers.",
                "parameters": {
                    "type": "object",
                    "properties": {
                        "a": { "type": "number" },
                        "b": { "type": "number" }
                    },
                    "required": ["a", "b"]
                }
            }
        ],
        "tool_choice": "auto",
        "temperature": 0.8,
        "max_output_tokens": 1024
    }
}

API Reference

Compatibility

Overview

`session.update`

Properties

`conversation.item.create`

Properties

`response.create`

Properties

​Overview

​session.update

​Properties

​conversation.item.create

​Properties

​response.create

​Properties

Overview

`session.update`

Properties

`conversation.item.create`

Properties

`response.create`

Properties