Make sure you have gone over QuickStart before trying this example.

In this example, we will build a realtime voice bot that uses OpenAI Realtime API. We accomplish this by using microphone audio to interact with the bot.

​
Setup Application with Realtime API in 4 steps

This app will process input from your microphone, send it to an LLM, and convert the response back to voice.

  • The full code is available here.
1

Install Dependencies

2

Create VoiceBot Application (using Realtime API)

Go over to QuickStart to understand the structure of an Outspeed application

Create a file voice_bot_rt.py and add the following code:

import outspeed as sp

"""
The @outspeed.App() decorator is used to wrap the VoiceBot class.
This tells the outspeed server which functions to run.
"""
@sp.App()
class VoiceBot:
    async def setup(self) -> None:
        # Initialize the AI services
        self.rt_node = sp.OpenAIRealtime()

    @sp.streaming_endpoint()
    async def run(self, audio_input_queue: sp.AudioStream, text_input_queue: sp.TextStream):
        # Set up the AI service pipeline
        audio_output_stream: sp.AudioStream
        audio_output_stream, text_output_stream = self.rt_node.run(text_input_queue, audio_input_queue)

        return audio_output_stream, text_output_stream

    async def teardown(self) -> None:
        # Close the OpenAI Realtime node when the application is shutting down
        await self.rt_node.close()


if __name__ == "__main__":
    # Start the VoiceBot when the script is run directly
    VoiceBot().start()
3

Setup API Keys and Run

You will need the following environment variable:

  1. OPENAI_API_KEY - You can get this by visiting the OpenAI Realtime API documentation and navigating to the API keys section to generate a new key.

Once you have your key, create a .env file in the same directory as voice_bot_with_rt.py and add the following:

OPENAI_API_KEY=<your_openai_api_key>

Finally, run the following command to start the server:

python voice_bot_rt.py

The console will output the URL you can use to connect to the server (default is http://localhost:8080).

4

Run Demo

You can use our playground to interact with the voice bot.

  1. Navigate to playground and select “Voice Bot”.
  2. Paste the link your received from the previous step into the URL field.
  3. Select Audio device. Leave Video device blank. Click Run to begin.

The playground is built using our our React SDK. You can use it to build your own frontends, or integrate with an existing one!

​
Understanding the Process

Review the code in voice_bot_with_rt.py.

​
Setup

The VoiceBot class initializes with the setup method, which is automatically called when the application starts. This method is responsible for setting up the necessary services and loading models. Here’s a breakdown of the services initialized:

  • OpenAIRealtime: Processes audio and text inputs to generate insightful commentary using OpenAI’s Realtime API, guided by a detailed system prompt. It operates with a response temperature of 0.9 and maintains a chat history for context.

​
Streaming Endpoint

The run method in the VoiceBot class is marked as a streaming endpoint, handling real-time audio and text streams.

The method outputs two streams: the audio stream of the commentary and the chat history text stream.

​
Next Step

Now, we’ll setup a search tool for the LLM in our application.

​
Support

For any assistance or questions, feel free to join our Discord community. We’re excited to see what you build!

Was this page helpful?

Adding LanguagesUsing Tools (Online Search)