Overview

​

What is a Tool in Outspeed?

The Tool class in the Outspeed package is a foundational component designed to facilitate function calling within the Outspeed ecosystem.

It provides a structured way to define and execute functions with specific input and output types, leveraging Pydantic models for validation. This ensures that the data flowing through your tools adheres to the expected formats, enhancing reliability and maintainability.

​

Key Features of the Tool Class

• Structured Definition: Clearly define the name, description, input parameters, and response types.
• Pydantic Integration: Utilize Pydantic models for robust input validation and response formatting.
• Asynchronous Execution: Implement asynchronous logic within the run method for efficient performance.
• Lifecycle Management: Manage the entire lifecycle of a function call, from validation to execution and response.

​

Creating a Tool

Creating a tool involves subclassing the Tool class and implementing the necessary components. Follow these steps to create your own tool:

​

Step 1: Define Input and Response Models

First, define the Pydantic models that represent the input parameters and the response structure of your tool.

from pydantic import BaseModel

class Query(BaseModel):
    query_for_neural_search: str


class RAGResult(BaseModel):
    result: str

​

Step 2: Implement the Tool Class

Subclass the Tool class and implement the required attributes and the run method.

import outspeed as sp

class RAGTool(sp.Tool):
    name = "rag"
    description = "Search the knowledge base for information"
    parameters_type = Query
    response_type = RAGResult

    def __init__(self):
        super().__init__()
        documents = SimpleDirectoryReader(f"{PARENT_DIR}/data/").load_data()
        node_parser = SimpleNodeParser.from_defaults(chunk_size=512)
        nodes = node_parser.get_nodes_from_documents(documents=documents)
        vector_index = VectorStoreIndex(nodes)
        self.query_engine = vector_index.as_query_engine(similarity_top_k=2)

    async def run(self, query: Query) -> RAGResult:
        logging.info(f"Searching for: {query.query_for_neural_search}")
        response = self.query_engine.query(query.query_for_neural_search)
        logging.info(f"RAG Response: {response}")
        return RAGResult(result=str(response))

​

Breakdown of the RAGTool Class

• Initialization (__init__ Method):

  • Loads documents from the specified directory.
  • Parses documents into nodes using SimpleNodeParser.
  • Creates a VectorStoreIndex for efficient similarity searches.
  • Initializes the query_engine for conducting neural searches.

• Execution (run Method):

  • Receives a Query object containing the search term.
  • Logs the search query for debugging purposes.
  • Executes the query against the vector index to retrieve relevant information.
  • Logs the response and returns it encapsulated within a RAGResult object.

​

Step 3: Integrate the Tool into Your Application

Once your tool is defined, integrate it into your application to handle function calls seamlessly.

@sp.App()
class VoiceBot:
    async def setup(self) -> None:
        # Initialize the AI services
        self.deepgram_node = sp.DeepgramSTT()
        self.llm_node = sp.OpenAILLM(
            tool_choice="required",
            tools=[
                RAGTool(),
            ],
        )
        self.token_aggregator_node = sp.TokenAggregator()
        self.tts_node = sp.CartesiaTTS(
            voice_id="95856005-0332-41b0-935f-352e296aa0df",
        )
        self.vad_node = sp.SileroVAD(min_volume=0)

    @sp.streaming_endpoint()
    async def run(self, audio_input_queue: sp.AudioStream, text_input_queue: sp.TextStream):
        # Set up the AI service pipeline
        deepgram_stream: sp.TextStream = self.deepgram_node.run(audio_input_queue)

        vad_stream: sp.VADStream = self.vad_node.run(audio_input_queue.clone())

        text_input_queue = sp.map(text_input_queue, lambda x: json.loads(x).get("content"))

        llm_input_queue: sp.TextStream = sp.merge(
            [deepgram_stream, text_input_queue],
        )

        llm_token_stream: sp.TextStream
        chat_history_stream: sp.TextStream
        llm_token_stream, chat_history_stream = self.llm_node.run(llm_input_queue)

        token_aggregator_stream: sp.TextStream = self.token_aggregator_node.run(llm_token_stream)
        tts_stream: sp.AudioStream = self.tts_node.run(token_aggregator_stream)

        self.llm_node.set_interrupt_stream(vad_stream)
        self.token_aggregator_node.set_interrupt_stream(vad_stream.clone())
        self.tts_node.set_interrupt_stream(vad_stream.clone())

        return tts_stream, chat_history_stream

    async def teardown(self) -> None:
        """
        Clean up resources when the VoiceBot is shutting down.

        This method is called when the app stops or is shut down unexpectedly.
        It should be used to release resources and perform any necessary cleanup.
        """
        await self.deepgram_node.close()
        await self.llm_node.close()
        await self.token_aggregator_node.close()
        await self.tts_node.close()


if __name__ == "__main__":
    # Start the VoiceBot when the script is run directly
    VoiceBot().start()

​

Steps for Integration

1. Initialize Tools: In the setup method, instantiate your tools and integrate them with the language model nodes.
2. Define Endpoints: Use decorators like @sp.streaming_endpoint() to define processing pipelines that utilize your tools.
3. Manage Resources: Ensure that all tools and services are properly closed in the teardown method to free up resources.

By following these guidelines and leveraging the Tool class, you can create sophisticated and reliable function-calling capabilities within your Outspeed applications.

​

Tool Lifecycle

The Tool class manages the entire lifecycle of a function call. Understanding this lifecycle is crucial for developing effective tools.

1. Validation of Input Parameters: The parameters_type model validates the incoming input parameters to ensure they meet the expected schema.
2. Execution of the run Method: The validated input is passed to the run method, where the main logic of the tool is executed asynchronously.
3. Validation of the Response: The output from the run method is validated against the response_type model to ensure consistency and correctness.
4. Serialization and Response: The validated response is serialized into JSON format for seamless integration with other components.