Tools

Tools allow LLMs to perform actions. They are defined in the "Tool" section of the dashboard or via the createToolDefinition API. Tools are defined by a snake_case name, a description, a list of parameters and their descriptions, and a destination URL.

How Tool Calling Works

Tools can be configured when using the /chat/completions endpoint or when starting realtime sessions.

/chat/completions

Chat completion tools are used via the typical OpenAI-compatible API. In the completions request, a list of tools is supplied. One small difference is that Gabber introduces a gabber_tool type. Tools with this type are invoked via Gabber. When the gabber_tool type is used, the gabber tool's id should be used in place of the tool name.

More details can be found in Using Gabber Tools

Realtime Sessions

Tool calls can be configured when starting a realtime session. For phone call sessions, tool calls can be configured when creating the phone number attachment.

Gabber Tool Calling System

Gabber uses a dedicated LLM for tool calls. If tools are defined in the request, the normal LLM will be invoked without tools supplied so that it produces an assistant response. Gabber's internal tool LLM will be invoked in parallel with tools supplied.

If Gabber's tool LLM decides tools need to be called, the assistant response from the normal LLM will be discarded. At the same time, the tool call web requests will be called asynchronously and an assistant message with tool_calls and an empty content will be returned, similar to OpenAI responses.

You can wait for the tool_call responses by the id field returned in the each of the calls in the tool_calls section using the getToolCallResult API.

You can retrieve results for each tool call using the id field in tool_calls with the getToolCallResult API. Whenever a tool is called, the LLM sends a POST request to the configured tool URL, including the tool parameters as key-value pairs in the request body.

When an LLM decides to use a tool, it makes a POST request to the configured web request URL in the tool definition. This POST request will contain the tool call parameter values as key-values in the request body.

The reponse will be stored as a string, retrievable from the above getTooLCallResult call.

Local Development

Tool calls be developed locally using a tunneling service, similar to webhooks. Ngrok is a very popular free way to do this.

The Local Development Webhooks section provides some more detail.

Testing Tool Calls

The best way to test and iterate on tool calls is in the chat playground on the dashboard.

Security

Web request bodies will be signed using the selected service key using HMAC-SHA256. The signature will be sent as a header: X-Gabber-Body-Signature. This signature can be used to verify that the request is coming from Gabber and not a malicious actor.

// ...existing code...
import express from "express";
import crypto from "crypto";

const router = express.Router();

router.post("/get_weather", (req, res) => {
  const chunks: Buffer[] = [];
  req.on("data", (chunk) => chunks.push(chunk));
  req.on("end", () => {
    try {
      const bodyBuffer = Buffer.concat(chunks);
      const webhookSignature = req.headers["X-Gabber-Body-Signature"];
      const key = process.env.GABBER_SECRET_KEY || "";
      const computedSignature = crypto
        .createHmac("sha256", key)
        .update(bodyBuffer)
        .digest("hex");

      if (computedSignature !== webhookSignature) {
        console.error("Signature mismatch");
        return res.status(403).send("Invalid signature");
      }

      // ... handle verified request ...
      return res.status(200).send("Signature verified");
    } catch (err) {
      console.error(err);
      return res.status(500).send("Server error");
    }
  });
});

export default router;
// ...existing code...