Chat Completions Tool Calling

Gabber tools can be called via the /chat/completions endpoint.

This uses the typical tool-calling OpenAI-compatible strategy but a new gabber_tool tool type has been introduced.

It even works well with LLMs that don't natively support tool calls or have poor tool call support. See Tools for more information.

Making a Request with a Tools

To use your Gabber tools in a completion request, set the type to gabber_tool and in place of the function name, use the Gabber tool definition id.

Gabber will automatically fill in the parameters associated with the tool definition.

const tools = [
    {
        type: "gabber_tool",
        function: {
            name: "gabber-tool-definition-id"
        },
    },
]

const body = {
  model: selectedLlm.id,
  messages: [
    { role: "user", content: "Hello" },
  ],
  stream: true,
  gabber: {
    context: gabberMemoryContext
  },
  tools,
};

const responseStream = await openAI.chat.completions.create(
  { stream: true, model: "", messages: [] }, // To satisfy the OpenAI SDK typings
  { body },
);

Tool Calls

When a tool is called, the LLM makes a POST request to the configured web request URL in the tool definition. This POST request will contain the tool call parameter values as key-values in the request body.

The chat/completion response will not wait for the tool call to complete. It will return an assistant message with tool_calls and an empty content. The tool calls will have a gabber_tool type.

The id of each tool call can be used to wait for tool call results using the getToolCallResult API.

import axios from "axios";

for await (const message of responseStream) {
    if (message.tool_calls) {
        for (const toolCall of message.tool_calls) {
            if (toolCall.type !== "gabber_tool") continue;
            // This will wait for the tool call to complete
            const res = await axios.get(`https://api.gabber.dev/tool/call/${toolCall.id}/result`);
            console.log("Tool Call Result: ", res.data);
        }
    }
}