Skip to main content

Building a Sample AI Companion App From Scratch With No Infrastructure Required

In this guide, we'll walk through creating a full-featured AI chat application using Gabber's SDK, Stripe, and Google OAuth. This is a pared down version of the full app, Rizz.ai. We'll be using Next.js as our framework along with TypeScript and Tailwind CSS for styling.

To follow along, we suggest forking the Rizz.ai repo: https://github.com/gabber-dev/example-app-rizz-ai

The above app, Rizz.ai was built entirely using Gabber's React SDK and APIs, with no infrastructure required (just vercel for hosting and Stripe for payments).

Rizz.ai

The general structure of this tutorial will be:

  1. Setting up the project
  2. Setting up Stripe and Google OAuth
  3. Implementing the layout.tsx file
  4. How to build an app with state
  5. Using Gabber's SDK to create realtime Voice and Text sessions
  6. Scoring
  7. Personas and Scenarios
  8. Historical Sessions
  9. Using Gabber's credit system to track usage
  10. Deployment to Vercel
  11. Bonus APIs like voice snippet generation, text to text, voice cloning, and more

Initial Setup​

First, let's create a new Next.js project and install the necessary dependencies. To install Node.js and Next.js, head to https://nodejs.org/en and download the latest version.

Once done, you can use the following commands to create a new Next.js project and install the necessary dependencies:

npx create-next-app@latest rizz-ai
cd rizz-ai-app
npm install

Setting Up Gabber​

Next, we'll need to install the Gabber SDK and configure it in our project:

npm install gabber-client-react react-hot-toast react-modal @mui/icons-material stripe google-auth-library

Throughout this set up process, we will be using a .env file for our secret API keys. .env.local is a good place to keep these keys for you development environment. When we go to production, we'll move them to .env.production

Head into the Gabber dashboard https://app.gabber.dev and create a new API key. You'll need to add it to your .env.local file.

Create API Key

Project Structure​

After installation, add a .env.local file to your project. Your project structure should look similar to this:

rizz-ai/
├── app/
│   ├── layout.tsx
│   └── page.tsx
├── public/
├── styles/
│   └── globals.css
├── package.json
└── tsconfig.json
└── .env.local

Stripe Integration​

1. Create a Stripe Account​

2. Obtain API Keys​

  • Navigate to Developers > API keys
  • Copy the Publishable key and Secret key

3. Set Up Products and Pricing​

  • Go to Products in the Stripe Dashboard
  • Create products and pricing plans
  • Important: One each product, set a metadata key called credit_amount with an integer value for each product to represent the number of credits the user will receive.

4. Environment Configuration​

Add to your .env.local file:

STRIPE_SECRET_KEY=<your-stripe-secret-key>
STRIPE_REDIRECT_HOST=http://localhost:3000

Google OAuth Integration​

1. Create Google Cloud Project​

  1. Go to the Google Cloud Console
  2. Create a new project or select an existing one
  1. Navigate to APIs & Services > OAuth consent screen
  2. Select "external" for non-organization apps
  3. Configure the consent screen
  4. Publish the app

3. Create OAuth Client ID​

  1. Go to APIs & Services > Credentials
  2. Create OAuth 2.0 client ID
  3. Configure authorized origins and redirect URIs:
    • Origin: http://localhost:3000
    • Redirect URI: http://localhost:3000/auth/google/callback

4. Environment Configuration​

Add to your .env.local file:

GOOGLE_CLIENT_ID=<your-google-client-id>
GOOGLE_CLIENT_SECRET=<your-google-client-secret>
GOOGLE_REDIRECT_URI=http://localhost:3000/auth/google/callback

Implementing Google OAuth Routes and Components​

Let's set up the necessary routes and components for Google OAuth authentication.

Google Login Route​

Create the file src/auth/google/login/route.tsx:

import { GenerateAuthUrlOpts, OAuth2Client } from "google-auth-library";
import { redirect } from "next/navigation";

export const runtime = "nodejs";
export const dynamic = "force-dynamic";

export async function GET() {
  const client = new OAuth2Client(
    process.env.GOOGLE_CLIENT_ID,
    process.env.GOOGLE_CLIENT_SECRET,
    process.env.GOOGLE_REDIRECT_URI,
  );

  const opts: GenerateAuthUrlOpts = {
    access_type: "offline",
    scope: [
      "https://www.googleapis.com/auth/userinfo.email",
      "https://www.googleapis.com/auth/userinfo.profile",
    ],
  };

  const url = client.generateAuthUrl(opts);

  redirect(url);
}

Google Callback Route​

Create the file src/auth/google/callback/route.tsx:

import { UserController } from "@/lib/server/controller/user";
import { OAuth2Client } from "google-auth-library";
import { cookies } from "next/headers";
import { redirect } from "next/navigation";
import { NextRequest } from "next/server";

export const runtime = "nodejs";
export const dynamic = "force-dynamic";

export async function GET(req: NextRequest) {
  const client = new OAuth2Client(
    process.env.GOOGLE_CLIENT_ID,
    process.env.GOOGLE_CLIENT_SECRET,
    process.env.GOOGLE_REDIRECT_URI,
  );

  const searchParams = req.nextUrl.searchParams;
  const code = searchParams.get("code");
  if (!code) {
    return Response.json({ message: "Missing code" }, { status: 400 });
  }

  const { tokens } = await client.getToken(code);
  const { access_token } = tokens;
  if (!access_token) {
    return Response.json({ message: "Missing access token" }, { status: 500 });
  }

  const { authToken } = await UserController.loginGoogleUser({
    access_token,
  });
  const cooks = await cookies();
  cooks.set("auth_token", authToken, {
    expires: new Date(Date.now() + 1000 * 60 * 60 * 24 * 180),
  });
  redirect("/");
}

Google User Type Definition​

Create the file lib/server/model/google_user.ts:

export type GoogleUser = {
  id: string;
  email: string;
  given_name?: string;
  family_name?: string;
  image_url?: string;
};

User Controller​

Add this method to your lib/server/controller/user.ts:

static async loginGoogleUser({
  access_token,
}: {
  access_token: string;
}): Promise<{ authToken: string }> {
  const userInfoReponse = await fetch(
    "https://www.googleapis.com/oauth2/v2/userinfo",
    {
      headers: {
        Authorization: `Bearer ${access_token}`,
      },
    },
  );

  const userInfo: GoogleUser = await userInfoReponse.json();
  userInfo.id = userInfo.id.toString();
  let stripeCustomer = await CreditsController.getCustomerByEmail(
    userInfo.email,
  );
  if (!stripeCustomer) {
    stripeCustomer = await CreditsController.createCustomer({
      email: userInfo.email,
    });
  }
  const authToken = UserController.generateJWT({
    email: userInfo.email,
    stripe_customer: stripeCustomer?.id,
  });
  return { authToken };
}

Login Button Component​

Create a login button component that triggers the Google OAuth flow:

const LoginButton = () => {
  const router = useRouter();

  return (
    <Button
      onClick={() => {
        router.push("/auth/google/login");
      }}
    >
      <div className="text-primary-content font-bold">Login or Sign-Up</div>
    </Button>
  );
};

Flow Explanation​

  1. When a user clicks the login button, they are redirected to /auth/google/login
  2. The login route generates a Google OAuth URL and redirects the user to Google's consent screen
  3. After the user approves, Google redirects back to our callback URL
  4. The callback route:
    • Exchanges the code for tokens
    • Fetches the user's information
    • Creates or retrieves a Stripe customer
    • Generates an auth token
    • Sets a cookie
    • Redirects to the home page

Important Notes​

  • Make sure all environment variables are properly set in the .env.local file
  • The callback URL in Google Cloud Console must exactly match your GOOGLE_REDIRECT_URI
  • The auth token cookie is set to expire in 180 days
  • A Stripe customer is automatically created for new users

Metadata and Layout.tsx Configuration​

The layout.tsx file is where we'll initialize the Gabber session and fetch the user's information. This is also where we'll fetch the personas and scenarios, as well as the user's credits, products, and usage. This info will be used to initialize the AppStateProvider. This is also where tracking tags like Google Analytics will be added.

Before we get into the more technical details of the layout.tsx file, let's add some metadata to the app as well as some optional Google Analytics tags.


export const metadata: Metadata = {
  title: "Rizz.AI - The Gym For Your Social Skills",
  description:
    "Practice dating, socializing, and speaking with confidence by having conversation with AI. Get tailored feedback to improve your skills.",
  openGraph: {
    title: "Rizz.AI - The Gym For Your Social Skills",
    description:
      "Practice dating, socializing, and speaking with confidence by having conversation with AI. Get tailored feedback to improve your skills.",
    images: [
      {
        url: "https://rizz.ai/og-image.png",
        width: 1200,
        height: 630,
        alt: "Rizz.AI - Practice your dating skills with AI",
      },
    ],
  },
  twitter: {
    card: "summary_large_image",
    title: "Rizz.AI - The Gym For Your Social Skills",
    description:
      "Practice dating, socializing, and speaking with confidence by having conversation with AI. Get tailored feedback to improve your skills.",
    images: ["https://rizz.ai/og-image.png"],
  },
};

export default async function RootLayout({
  children,
}: {
  children: React.ReactNode;
}) {
  const user = await UserController.getUserFromCookies();

  const config = new Configuration({ apiKey: process.env.GABBER_API_KEY });
  const personaApi = new PersonaApi(config);
  const scenarioApi = new ScenarioApi(config);

  const [personas, scenarios] = await Promise.all([
    personaApi.listPersonas(),
    scenarioApi.listScenarios(),
  ]);

  let sessions: RealtimeSession[] = [];
  console.log("user", user);
  if (user) {
    const realtimeApi = new RealtimeApi(config);
    try {
      const response = await realtimeApi.listRealtimeSessions({
        headers: { "x-human-id": user.stripe_customer },
      });
      sessions = response.data.values;
    } catch (e) {
    }
  }

  const [credits, hasPaid, products, usageToken] = await Promise.all([
    user?.stripe_customer
      ? CreditsController.getCreditBalance(user.stripe_customer)
      : 0,
    user?.stripe_customer
      ? CreditsController.checkHasPaid(user.stripe_customer)
      : false,
    CreditsController.getProducts(),
    UserController.createUsageToken(),
  ]);

  return (
    <html
      lang="en"
      data-theme="rizz"
      className={`${sfProRounded.className} ${sfProRounded.className}`}
    >
      <Head>
        <link rel="icon" href="/favicon.ico" />
      </Head>
      <Script
        async
        src="https://www.googletagmanager.com/gtag/js?id="
        strategy="afterInteractive"
      ></Script>
      <body className="relative h-screen bg-base-100">
        <AppStateProvider
          userInfo={user}
          usageToken={usageToken}
          initialCredits={credits}
          initialHasPaid={hasPaid}
          initialProducts={products}
          initialPersonas={personas.data.values}
          initialScenarios={scenarios.data.values}
          initialSessions={sessions}
        >
          <ClientLayout>{children}</ClientLayout>
        </AppStateProvider>
      </body>
    </html>
  );
}

How does this all work?

We have a provider setup that initializes the Gabber session using a token. Read more about the token here: https://docs.gabber.dev/intro

The token is tied to a user and is typically something like a user id (in our case Stripe Customer ID). The client (the frontend app) then uses the token to initialize the session, add and deduct credits, and more. You can think of the token as your Gabber API key wrapped in auser-specific identifier that makes it easy to track usage, while still keeping the user and their usage, tied to your account.

Once you initialize the session, you can use the session with a token and the persona and scenario you want to use, the session will handle the rest.

Application State Management​

Application state is the "state" that is shared across the app. The things stored in the state are things that every piece of your app could benefit from knowing about. Examples include things like user information, session information, various API clients, and other data that is shared across the app.

AppState Types​

First, create the types for our application state in src/components/AppStateProvider.tsx:

Here's a basic example of what the types might look like, keeping track of the user's info, how many credits they have, past sessions, and the various personas and scenarios.

type AppStateContextType = {
  // User related
  userInfo: UserInfo | null;
  usageToken: string;
  user: UserInfo | null;

  // Credits system
  credits: number;
  creditsLoading: boolean;
  refreshCredits: () => void;

  // Sessions management
  sessions: RealtimeSession[];
  sessionsLoading: boolean;
  refreshSessions: () => void;

  // Persona management
  personas: Persona[];
  personasLoading: boolean;
  selectedPersona: Persona | null;
  setSelectedPersona: (p: Persona | null) => void;
  refreshPersonas: () => void;
  shufflePersonas: () => void;

  // Additional state types...
};

AppStateProvider Implementation​

Here's the core implementation of our state provider. Again, this is in the src/components/AppStateProvider.tsx file. You can see this is a two way street: the state provider is passed down to the components as props, and the components can also update the state.

export function AppStateProvider({
  children,
  userInfo,
  usageToken,
  initialSessions,
  initialPersonas,
  initialScenarios,
  initialCredits,
  initialHasPaid,
  initialProducts,
}: AppStateProviderProps) {
  // State declarations
  const [gender, setGender] = useState<"men" | "women" | "all">("all");
  const [credits, setCredits] = useState<number>(initialCredits);
  const [creditsLoading, setCreditsLoading] = useState<boolean>(true);

  // API instances using useMemo
  const realtimeApi = useMemo(() => {
    return new RealtimeApi(new Configuration({ accessToken: usageToken }));
  }, [usageToken]);

  // Refresh functions using useCallback
  const refreshPersonas = useCallback(async () => {
    if (personasLock.current) return;
    personasLock.current = true;
    setPersonasLoading(true);
    try {
      const { data } = await personaApi.listPersonas();
      setPersonas(data.values || []);
    } catch (error) {
      toast.error("Failed to load personas");
    } finally {
      setPersonasLoading(false);
      personasLock.current = false;
    }
  }, [personaApi]);

  // Additional implementation...
}

Using the AppState​

Here's how to use the AppState in components. It's as simple as using the useAppState hook and pulling the parts of the state you need.

Header Component Example​

export function AuthenticatedHeader() {
  const { credits, setShowPaywall, userInfo } = useAppState();
  const router = useRouter();

  return (
    <div className="p-2 h-full w-full flex items-center bg-base-300">
      <a className="h-3/5 ml-2" href="/">
        <Logo className="text-primary" />
      </a>
      <div className="grow" />
      {userInfo ? (
        <ShinyButton
          onClick={() => setShowPaywall({ session: null })}
        >
          <div className="text-primary-content font-bold">
            Credits: {credits}
          </div>
        </ShinyButton>
      ) : (
        <ShinyButton
          onClick={() => router.push("/auth/google/login")}
        >
          <div className="text-primary-content font-bold">
            Login or Sign-Up
          </div>
        </ShinyButton>
      )}
    </div>
  );
}

Setting Up the Provider​

More generally, this is how you'd initialize the AppStateProvider. This is typically done in the root layout or app component, like the layout.tsx file from above.

function YourApp({ children }) {
  return (
    <AppStateProvider
      userInfo={user}
      usageToken={usageToken}
      initialCredits={credits}
      initialHasPaid={hasPaid}
      initialProducts={products}
      initialPersonas={personas}
      initialScenarios={scenarios}
      initialSessions={sessions}
    >
      {children}
    </AppStateProvider>
  );
}

Using AppState in Components​

A second example of how you'd use the AppState in a general component. Again, anytime you'd want to use from the AppState, you'd use the useAppState hook.

function YourComponent() {
  const { 
    userInfo, 
    credits, 
    refreshCredits,
    selectedPersona,
    setSelectedPersona 
  } = useAppState();

  return (
    <div>
      {userInfo ? (
        <div>Credits: {credits}</div>
      ) : (
        <div>Please log in</div>
      )}
    </div>
  );
}

Key Features of the AppState​

  1. Centralized State Management: All application state is managed in one place
  2. Type Safety: Full TypeScript support for state and actions
  3. Memoized API Instances: API clients are memoized for performance
  4. Locked Operations: Prevents concurrent operations with ref locks
  5. Error Handling: Built-in error handling with toast notifications
  6. Filtered Data: Supports filtered views of data (e.g., filtered personas)

Best Practices​

  1. Always use useCallback for functions passed through context
  2. Implement loading states for async operations
  3. Use useMemo for expensive computations
  4. Handle errors gracefully with user feedback
  5. Use TypeScript for better type safety and developer experience

Implementing Live Chat with Gabber​

Now for the main event.Let's break down how to implement a real-time chat interface using Gabber's SDK.

Main Client Page Component​

First, create the main client page that initializes the Gabber session. In other words, we'll make the page that the user interacts with, and we'll also initialize the Gabber session here. We'll place the SessionProvider around the components that need to use the session.

"use client";

import React, { useState, useEffect } from "react";
import { SessionProvider, useSession } from "gabber-client-react";
import { Persona, Scenario } from "@/generated";

export function ClientPage({ persona, scenario }: Props) {
  const { credits, setShowPaywall, usageToken } = useAppState();
  const [connectionOpts, setConnectionOpts] = useState<null | {
    token: string;
    sessionConnectOptions: { persona: string; scenario: string };
  }>(null);

  useEffect(() => {
    if (credits >= 0) {
      setConnectionOpts({
        token: usageToken,
        sessionConnectOptions: { persona: persona.id, scenario: scenario.id },
      });
    }
  }, [credits, persona.id, scenario.id, setShowPaywall, usageToken]);

  return (
    <SessionProvider connectionOpts={connectionOpts}>
      <ClientSessionPageInner persona={persona} scenario={scenario} />
    </SessionProvider>
  );
}

Chat Interface Component​

The inner component handles the chat interface and user interactions:

export function ClientSessionPageInner({ persona, scenario }: Props) {
  const { messages, id } = useSession();
  const { credits, setShowPaywall, refreshCredits } = useAppState();
  const router = useRouter();

  // Credit refresh loop
  useEffect(() => {
    const interval = setInterval(refreshCredits, 5000);
    return () => clearInterval(interval);
  }, [refreshCredits]);

  return (
    <div className="relative w-full h-full pt-4">
      {/* Persona Image and Info */}
      <div className="relative h-[200px] w-[200px] rounded-[8px] overflow-hidden">
        <Image
          fill={true}
          className="w-full h-full object-cover"
          src={persona.image_url || ""}
          alt={persona.name}
        />
        <AgentAudioVisualizer />
      </div>

      {/* Chat Messages and Input */}
      <div className="relative w-full grow max-w-[600px]">
        <Messages />
      </div>
      <div className="absolute bottom-0 w-full">
        <InputBar />
      </div>
    </div>
  );
}

Messages Component​

The component that displays chat messages:

export function Messages() {
  const { messages } = useSession();
  const [isAtBottom, setIsAtBottom] = useState(true);
  const containerRef = useRef<HTMLDivElement>(null);

  const scrollToBottom = () => {
    if (containerRef.current) {
      containerRef.current.scrollTo({
        top: containerRef.current.scrollHeight,
        behavior: "smooth",
      });
    }
  };

  useEffect(() => {
    if (isAtBottom) {
      scrollToBottom();
    }
  }, [isAtBottom, messages]);

  return (
    <div className="relative w-full h-full">
      <div
        ref={containerRef}
        className="w-full h-full overflow-y-scroll"
      >
        {messages.map((message) => (
          <div
            key={`${message.id}_${message.agent}`}
            className={`text-${message.agent ? "primary" : "accent"}`}
          >
            {message.text}
          </div>
        ))}
      </div>
      {/* Scroll to bottom button */}
    </div>
  );
}

Input Bar Component​

The component for sending messages:

export function InputBar() {
  const [text, setText] = useState("");
  const {
    sendChatMessage,
    microphoneEnabled,
    setMicrophoneEnabled,
    userVolume,
  } = useSession();

  return (
    <div className="w-full h-full px-2 max-w-[600px] mx-auto">
      <form
        className="flex gap-2 w-full h-full items-center"
        onSubmit={(e) => {
          e.preventDefault();
          if (text) {
            sendChatMessage({ text });
            setText("");
          }
        }}
      >
        {/* Microphone toggle button */}
        <input
          type="text"
          value={text}
          onChange={(e) => setText(e.target.value)}
          className="input input-bordered grow min-w-0 p-2"
          placeholder="Type a message..."
        />
        {/* Send button */}
      </form>
    </div>
  );
}

Key Features​

  1. Real-time Chat: Messages are sent and received in real-time
  2. Voice Input: Support for microphone input
  3. Auto-scrolling: Automatically scrolls to new messages
  4. Credit System: Integrates with a credit-based system
  5. Session Management: Handles chat session initialization and state

How It Works​

  1. Session Initialization:

    • User provides a token
    • Session is created with specific persona and scenario
    • Connection is established

  2. Message Flow:

    • User sends message via text or voice
    • Message is processed by Gabber
    • AI response is received and displayed

  3. State Management:

    • Messages are managed by Gabber session
    • Credits are tracked and updated
    • UI state (scroll position, input text) is managed locally

Best Practices​

  1. Regular credit checks
  2. Proper error handling
  3. Smooth scrolling behavior
  4. Responsive design
  5. Voice input toggle
  6. Message persistence

Implementing Scoring System with Gabber​

Let's break down how to implement a scoring system that evaluates chat conversations using LLMs.

Score

Score Controller​

First, let's create the scoring controller that processes chat transcripts. This is a lite version and I'd encourage you to use the full scoring system from the forked repo.

// src/lib/server/controller/score.ts

export class ScoreController {
  static async calculateScore(session: string): Promise<Score> {
    // Initialize APIs
    const openai = new OpenAI({
      apiKey: "",
      baseURL: "https://api.gabber.dev/v1",
      defaultHeaders: { "x-api-key": process.env.GABBER_API_KEY },
    });

    // Initialize Gabber APIs
    const config = new Configuration({
      apiKey: process.env.GABBER_API_KEY,
      basePath: "https://app.gabber.dev",
    });
    
    // Get session data and context
    const sessionObj = await realtimeApi.getRealtimeSession(session);
    const messages = await llmAPI.listContextMessages(
      sessionObj.config.generative.context.id
    );

    // Generate and process score
    const result = await openai.chat.completions.create({
      model: llm,
      messages: generateScoringMessages(personaObj, scenarioObj, history),
    });

    return processScore(result);
  }
}

Scoring Prompts​

The system uses carefully crafted prompts to generate consistent scoring. See the full prompt in the /src/lib/server/controller/score.ts file.

const generateSystemMessage = (persona: Persona, scenario: Scenario) => {
  return `Your name is ${persona?.name}. ${persona?.description}.
         I want you to pretend ${scenario?.prompt}.
         At the end of this exchange I will ask you to score me and return the score in JSON format.`;
};

const generateScoreMessage = () => {
  return `Please rate my performance based on:
    - wit
    - humor
    - confidence
    - seductiveness
    - flow
    - kindness

    Each with scores: poor, fair, good and return the score in JSON format. Also include a general summary of the score. Example:
    {
      "wit": "poor",
      "humor": "fair",
      "seductiveness": "poor",
      "summary": "You were not very good at this conversation."
    }
    .`;
};

Score Display Component​

The component to display the scoring results. Again, this is a lite version and I'd encourage you to use the full scoring system from the forked repo.

export function Score({ score }: Props) {
  const [isSummaryModalOpen, setSummaryModalOpen] = useState(false);

  const attributes: AttributeRating[] = [
    { name: "Wit", score: score.wit },
    { name: "Humor", score: score.humor },
    // ... other attributes
  ];

  return (
    <motion.div className="flex flex-col gap-2 items-center">
      {/* Score Ring */}
      <Ring percentage={score.rizz_score / 100} />
      
      {/* Attribute Grid */}
      <motion.div className="grid grid-cols-2 gap-4">
        {attributes.map((attr) => (
          <AttributeCard key={attr.name} attr={attr} />
        ))}
      </motion.div>

      {/* Summary Modal */}
      {isSummaryModalOpen && (
        <SummaryModal 
          summary={score.summary}
          onClose={() => setSummaryModalOpen(false)}
        />
      )}
    </motion.div>
  );
}

Key Features​

  1. Multi-attribute Scoring:

    • Wit
    • Humor
    • Confidence
    • Seductiveness
    • Flow
    • Kindness

  2. Visual Feedback:

    • Score ring visualization
    • Individual attribute cards
    • Animated transitions
    • Modal summary view

  3. Scoring Logic:

    const calculateRizzScore = (scoreObj: Record<string, string>): number => {
      const pointValues = {
        poor: 0,
        fair: 1,
        good: 2,
      };

      const attributes = [
        "wit", "humor", "confidence",
        "seductiveness", "flow", "kindness"
      ];

      let totalPoints = 0;
      const maxPoints = attributes.length * 2;

      for (const attr of attributes) {
        totalPoints += pointValues[scoreObj[attr]] || 0;
      }

      return Math.round((totalPoints / maxPoints) * 100);
    };

Implementation Steps​

  1. Setup Scoring Controller:

    • Initialize APIs
    • Configure LLM settings
    • Set up scoring prompts

  2. Process Chat Transcript:

    • Retrieve session data
    • Format conversation history
    • Generate scoring prompt

  3. Generate Score:

    • Send to LLM
    • Parse JSON response
    • Calculate final score

  4. Display Results:

    • Show overall score
    • Display attribute breakdowns
    • Provide detailed feedback

Best Practices​

  1. Error handling for LLM responses
  2. JSON validation and repair
  3. Smooth animations for better UX
  4. Responsive design for all screen sizes
  5. Clear feedback presentation
  6. Consistent scoring criteria

Creating Personas and Scenarios in Gabber​

Let's walk through how to create personas and scenarios in the Gabber dashboard.

Accessing Gabber Dashboard​

  1. Navigate to app.gabber.dev
  2. Sign up for an account
  3. Create a new project

Project

Voice Cloning​

Gabber has 60 default voices, and you can clone your own if you have an mp3 file. Gabber will use the audio to create a voice model. You can then use this voice model in your personas.

Voice Cloning

Creating Personas​

Create Persona

Personas are the characters that the AI will embody during conversations. You can add an image and voice to your persona. Here's how to create effective ones:

Best Practices for Persona Creation:​

  1. Use Direct Address

    "You are a charismatic startup founder who..."

    Instead of:

    "The persona is a startup founder who..."

  2. Include Personality Traits

    • Confidence level
    • Communication style
    • Personality quirks
    • Background story

Example Persona:​

You are a tech-savvy startup founder in your early 30s. You're passionate about 
artificial intelligence and its potential to change the world. You have a 
confident but approachable demeanor, often using tech industry jargon but able 
to explain complex concepts simply. You have a dry sense of humor and tend to 
make witty observations about the tech industry.

Creating Scenarios​

Scenarios set the context for interactions between the AI persona and the user. You are talking to the AI, so the scenario is the context of the conversation, written from the perspective of the user.

Best Practices for Scenario Creation:​

  1. Clear Context Setting

    "You and I are meeting at a tech conference where..."

  2. Specific Situation Details

    "We're at your company's booth, and I've expressed interest in your AI product..."

Create Scenario

Example Scenario:​

You and I are at a tech startup networking event. I'm a potential investor 
interested in AI companies. We've just been introduced by a mutual connection, 
and I'm curious about your startup. You're excited to pitch your company but 
want to build rapport before diving into the business details.

Key Components​

For Personas:​

  • Background
  • Personality traits
  • Communication style
  • Knowledge areas
  • Quirks or unique characteristics

For Scenarios:​

  • Setting
  • Context
  • Relationship between participants
  • Initial situation
  • Objectives or goals

Tips for Effective Creation​

  1. Be Specific

    • Include detailed characteristics
    • Define clear behavioral patterns
    • Set specific knowledge boundaries

  2. Maintain Consistency

    • Keep personality traits aligned
    • Ensure scenario matches persona capabilities
    • Maintain realistic interactions

  3. Consider User Experience

    • Make interactions natural
    • Allow for various conversation paths
    • Include appropriate response styles

Example Combinations​

  1. Sales Context
Persona: You are an experienced SaaS sales representative with a 
consultative approach. You're knowledgeable about enterprise software 
but avoid technical jargon unless asked.

Scenario: You and I are having a discovery call. I'm a potential client 
looking for a new CRM solution for my growing business.
  1. Dating Context
Persona: You are a charming barista who loves discussing art and music. 
You're witty, slightly flirtatious, and genuinely interested in learning 
about others.

Scenario: You and I are meeting for the first time at your coffee shop. 
I'm a regular customer who's finally worked up the courage to have a real 
conversation with you.

Testing Your Creations​

  1. Run test conversations in the Gabber dashboard simulate tab
  2. Verify personality consistency
  3. Check scenario flow
  4. Adjust based on interactions

Simulate

Managing Historical Sessions and Analytics​

Let's break down how to implement session history and analytics using Gabber's API.

Transcript

Session Detail Modal Component​

// src/components/SessionDetail.tsx

type SessionMessage = {
  agent: boolean;
  content?: string;
  created_at: string;
};

type TimelineItem = {
  type: "user" | "agent" | "silence";
  seconds: number;
};

export function SessionDetailModal({ sessionId, onClose }: Props) {
  const [messages, setMessages] = useState<SessionMessage[]>([]);
  const [timeline, setTimeline] = useState<TimelineItem[]>([]);
  const { sessionApi, realtimeApi } = useAppState();

  // Fetch session details
  useEffect(() => {
    const fetchSessionDetails = async () => {
      if (!sessionId) return;
      
      const [messagesRes, timelineRes] = await Promise.all([
        sessionApi.apiV1SessionSessionIdMessagesGet(sessionId),
        sessionApi.apiV1SessionSessionIdTimelineGet(sessionId),
      ]);

      setMessages(messagesRes.data.values);
      setTimeline(timelineRes.data.values);
    };

    fetchSessionDetails();
  }, [sessionId]);

  // Calculate statistics
  const stats = calculateSessionStats(timeline);

  return (
    <div className="fixed inset-0 bg-black bg-opacity-50 flex items-center justify-center">
      <div className="bg-base-300 rounded-lg w-full max-w-2xl">
        {/* Stats Cards */}
        <div className="grid grid-cols-3 gap-4">
          <StatCard
            label="Time Speaking"
            value={`${Math.round(stats.userTime)}s`}
            percentage={stats.userPercentage}
          />
          {/* Additional stat cards */}
        </div>

        {/* Transcript */}
        <div className="space-y-3">
          {messages.map((message, i) => (
            <MessageBubble key={i} message={message} />
          ))}
        </div>
      </div>
    </div>
  );
}

Session Controller​

The session controller is used to fetch the session data. The two endpoints we'll use are apiV1SessionSessionIdMessagesGet and apiV1SessionSessionIdTimelineGet. The messages endpoint returns the transcript of the conversation, and the timeline endpoint returns the timeline of the conversation. This is a lite version of the session controller. I'd encourage you to use the full session controller from the forked repo.

The timeline endpoint returns a list of timeline items, each with a type and seconds. The type is either "user", "agent", or "silence". The seconds is the duration of the item in seconds.

// src/lib/server/controller/session.ts

export class SessionController {
  static async getSessionDetails(sessionId: string) {
    const config = new Configuration({
      apiKey: process.env.GABBER_API_KEY,
      basePath: "https://app.gabber.dev",
    });

    const sessionApi = SessionApiFactory(config);

    // Fetch all session data in parallel
    const [sessionData, messages, timeline] = await Promise.all([
      sessionApi.apiV1SessionSessionIdGet(sessionId),
      sessionApi.apiV1SessionSessionIdMessagesGet(sessionId),
      sessionApi.apiV1SessionSessionIdTimelineGet(sessionId),
    ]);

    const stats = this.calculateSessionStats(timeline.data.values);

    return {
      session: sessionData.data,
      messages: messages.data.values,
      timeline: timeline.data.values,
      stats,
      duration: this.calculateTotalDuration(timeline.data.values),
    };
  }

  private static calculateSessionStats(timeline: SessionTimelineItem[]) {
    // Calculate various time metrics
    const totalDuration = timeline.reduce(
      (acc, item) => acc + (item.seconds ?? 0),
      0
    );

    // Calculate individual times
    const userTime = this.calculateTimeByType(timeline, "user");
    const silenceTime = this.calculateTimeByType(timeline, "silence");
    const agentTime = this.calculateTimeByType(timeline, "agent");

    return {
      totalDuration,
      userTime,
      silenceTime,
      agentTime,
      userPercentage: (userTime / totalDuration) * 100,
      silencePercentage: (silenceTime / totalDuration) * 100,
      agentPercentage: (agentTime / totalDuration) * 100,
    };
  }
}

Key Features​

  1. Timeline Analysis

    • User speaking time
    • AI response time
    • Silence periods
    • Total duration

  2. Message Transcript

    • Chronological view
    • Speaker identification
    • Timestamp display

  3. Statistical Analysis

    • Time distribution
    • Participation percentages
    • Response patterns

Analytics Components​

// Stat Card Component
function StatCard({ label, value, percentage }: StatCardProps) {
  return (
    <div className="bg-base-200 p-4 rounded-lg">
      <div className="text-sm opacity-70">{label}</div>
      <div className="text-2xl font-bold">{value}</div>
      <div className="text-sm">{percentage.toFixed(1)}%</div>
    </div>
  );
}

// Message Bubble Component
function MessageBubble({ message }: { message: SessionMessage }) {
  return (
    <div
      className={`p-4 rounded-lg ${
        message.agent ? "bg-base-200" : "bg-primary text-primary-content ml-8"
      }`}
    >
      <div className="text-xs opacity-70">
        {formatDistanceToNow(new Date(message.created_at))} ago
      </div>
      <div>{message.content}</div>
    </div>
  );
}

Implementation Steps​

  1. Setup Session Fetching

    • Configure API client
    • Handle authentication
    • Implement error handling

  2. Process Timeline Data

    • Calculate durations
    • Compute percentages
    • Generate statistics

  3. Display Messages

    • Format timestamps
    • Style message bubbles
    • Handle different message types

  4. Visualize Analytics

    • Create stat cards
    • Implement progress bars
    • Show time distributions

Implementing Credit System with Gabber​

Let's break down how to implement a credit-based system without a database using Gabber's built-in credit tracking. This is going to come across as extar steps since Gabber could in theory just report to itself without a webhook. However, this is a unique situation where Gabber is both the thing generating the conversation and the credit ledger, so we need to report usage back to the app to deduct credits from the user's balance.

The gist here is:

  1. Purchase a product with an associated amount of credits from Stripe
  2. Stripe will tell Gabber's credit system about those purchased credits, updating the user's balance in the Gabber credit system
  3. As users interact with the Gabber session, Gabber's backend will ping the webhook you've set up in the app
  4. You app will recieve this usage report and deduct credits from the user's balance in the Gabber credit system

Credit Controller Setup​

First, create a credit controller to manage all credit-related operations. This is a very pared down version, see the full version at /src/lib/server/controller/credit.ts:

// src/lib/server/controller/credit.ts

export class CreditsController {
  private static getStripeClient() {
    if (!process.env.STRIPE_SECRET_KEY) {
      throw new Error("Stripe secret key not found");
    }
    return new Stripe(process.env.STRIPE_SECRET_KEY);
  }

  // Customer Management
  static async getCustomer(customer_id: string) {
    const client = await CreditsController.getStripeClient();
    const customer = await client.customers.retrieve(customer_id);
    return customer.id;
  }

  // Credit Balance Management
  static async getCreditBalance(customer: string) {
    const configuration = new Configuration({
      apiKey: process.env.GABBER_API_KEY,
    });
    const creditApi = CreditApiFactory(configuration);
    
    // Get latest credit entry and calculate balance
    const latestEntry = await creditApi.getLatestCreditLedgerEntry(
      process.env.GABBER_CREDIT_ID!,
      { headers: { "x-human-id": customer } }
    );
    
    return latestEntry.data.balance;
  }

   static async reportCreditUsage(customer: string, amount: number) {
    if (!process.env.GABBER_CREDIT_ID) {
      throw new Error("Server misconfigured - missing credit id");
    }
    const configuration = new Configuration({
      apiKey: process.env.GABBER_API_KEY,
    });
    const creditApi = CreditApiFactory(configuration);
    try {
      await creditApi.createCreditLedgerEntry(
        process.env.GABBER_CREDIT_ID,
        {
          amount,
          idempotency_key: v4(),
        },
        { headers: { "x-human-id": customer } },
      );
    } catch (e: any) {
      console.error("Failed to report credit usage:", e.message);
      throw e;
    }
  }
}

Credit Usage Tracking​

Now we'll create the webhook. You'll need to add a link to this to your Gabber dashboard.

Create Webhook

This is how Gabber will report usage back to your app so you can deduct credits from the user's balance in real-time. This is the route that Gabber will call to report usage back to your app. We'll use this to deduct credits from the user's balance in real-time.

// src/app/api/credits/route.ts

export async function POST(req: Request) {
  const user = await UserController.getUserFromCookies();
  if (!user) {
    return new Response("Unauthorized", { status: 401 });
  }

  const { amount } = await req.json();
  
  try {
    await CreditsController.reportCreditUsage(user.stripe_customer, -amount);
    return new Response(null, { status: 200 });
  } catch (e) {
    return new Response("Failed to report usage", { status: 500 });
  }
}

Credit Display Component​

// src/components/credits.tsx

export function Credits() {
  const { credits, creditsLoading, setShowPaywall } = useAppState();

  if (creditsLoading) {
    return <div className="animate-pulse bg-base-200 h-8 w-20 rounded" />;
  }

  return (
    <button
      onClick={() => setShowPaywall({ session: null })}
      className="bg-primary text-primary-content px-4 py-2 rounded-lg font-bold"
    >
      Credits: {credits}
    </button>
  );
}

Paywall Component​

// src/components/PaywallPopup.tsx

export function PaywallPopup() {
  const { products, showPaywall, hasPaid } = useAppState();
  const [activeTab, setActiveTab] = useState(hasPaid ? "oneTime" : "recurring");

  const getCheckoutSession = async (priceId: string) => {
    const response = await fetch("/api/checkout", {
      method: "POST",
      body: JSON.stringify({
        price_id: priceId,
        gabber_session: showPaywall?.session,
      }),
    });
    
    const { url } = await response.json();
    if (url) router.push(url);
  };

  return (
    <div className="flex flex-col items-center w-full h-full p-6">
      {/* Tab Navigation */}
      <div className="flex justify-center w-full mb-4">
        <TabButtons activeTab={activeTab} setActiveTab={setActiveTab} />
      </div>

      {/* Product Display */}
      <div className="grid grid-cols-1 md:grid-cols-2 gap-4">
        {activeTab === "recurring" ? (
          <RecurringProducts products={recurringProducts} onSelect={getCheckoutSession} />
        ) : (
          <OneTimeProducts products={oneTimeProducts} onSelect={getCheckoutSession} />
        )}
      </div>
    </div>
  );
}

Paywall

Key Features​

  1. Credit Management

    • Balance tracking
    • Usage reporting
    • Automatic deduction

  2. Payment Integration

    • Stripe checkout
    • Subscription handling
    • One-time purchases

  3. User Experience

    • Credit display
    • Purchase options
    • Loading states

Implementation Steps​

  1. Setup Environment

    STRIPE_SECRET_KEY=sk_test_...
    GABBER_API_KEY=...
    GABBER_CREDIT_ID=...
    STRIPE_WEBHOOK_SECRET=...

  2. Create Stripe Products

    • Set up recurring plans
    • Configure one-time purchases
    • Add credit_amount metadata

  3. Configure Webhooks

    • Set up Stripe webhook endpoint
    • Configure event handling
    • Test webhook functionality

  4. Implement Credit Tracking

    • Monitor usage
    • Update balances
    • Handle errors

Deployment Guide for Gabber Application​

Let's walk through the deployment process and configuration requirements.

Next.js Configuration​

// next.config.js
module.exports = {
  output: "standalone",
  images: {
    domains: ["imagedelivery.net"],
  },
};

Environment Variables​

This is the list of environment variables you'll need to set in your Vercel dashboard. It should match what you have in your .env file.

# Gabber Configuration
GABBER_API_KEY=<api-key-from-gabber-dashboard>
GABBER_CREDIT_ID=<credit-id-in-gabber-dashboard>

# Google OAuth
GOOGLE_CLIENT_ID=<your-google-client-id>
GOOGLE_CLIENT_SECRET=<your-google-client-secret>
GOOGLE_REDIRECT_URI=<your-vercel-domain>/auth/google/callback

# Stripe Configuration
STRIPE_SECRET_KEY=<your-stripe-secret-key>
STRIPE_REDIRECT_HOST=<your-vercel-domain>

Package Configuration​

{
  "scripts": {
    "dev": "next dev",
    "build": "next build",
    "start": "next start",
    "lint": "next lint --fix",
    "tsc": "tsc",
    "generate": "openapi-generator-cli generate -i https://api.gabber.dev/openapi.yaml -g typescript-axios -o ./src/generated"
  }
}

TypeScript Configuration​

{
  "compilerOptions": {
    "target": "es5",
    "lib": ["dom", "dom.iterable", "esnext"],
    "allowJs": true,
    "skipLibCheck": true,
    "strict": true,
    "noEmit": true,
    "esModuleInterop": true,
    "module": "esnext",
    "moduleResolution": "bundler",
    "resolveJsonModule": true,
    "isolatedModules": true,
    "jsx": "preserve",
    "incremental": true,
    "plugins": [
      {
        "name": "next"
      }
    ],
    "paths": {
      "@/*": ["./src/*"]
    }
  },
  "include": ["next-env.d.ts", "**/*.ts", "**/*.tsx", ".next/types/**/*.ts"],
  "exclude": ["node_modules"]
}

Webhook Configuration​

// src/app/api/webhook/route.ts
export async function POST(req: NextRequest) {
  const textBody = await req.text();
  const webhookSignature = req.headers.get("X-Webhook-Signature");
  
  // Verify signature
  const computedSignature = crypto
    .createHmac("sha256", process.env.GABBER_API_KEY!)
    .update(textBody, "utf8")
    .digest("hex");

  if (computedSignature !== webhookSignature) {
    return new Response("Invalid signature", { status: 403 });
  }

  // Handle usage tracking
  const { type, payload } = JSON.parse(textBody);
  if (type === "usage.tracked") {
    const { human_id, type, value } = payload;
    if (type === "conversational_seconds") {
      await CreditsController.reportCreditUsage(human_id, value * -1);
    }
  }

  return new Response(null, { status: 200 });
}

Vercel Deployment​

  1. Using Vercel CLI:
npm i -g vercel
vercel
  1. Using Vercel Dashboard:
  • Connect Git repository
  • Configure environment variables
  • Deploy

Vercel Configuration​

// vercel.json
{
  "git": {
    "deploymentEnabled": {
      "main": true,
      "dev": false
    }
  },
  "headers": [
    {
      "source": "/api/(.*)",
      "headers": [
        {
          "key": "Cache-Control",
          "value": "no-store, max-age=0"
        }
      ]
    }
  ]
}

There were a number of other things we could have done in the app, but this is a good start. Shoot us an email if you have any questions @ [email protected]

If you are curious, check out the rest of the docs—here's some of the noteworthy functionality to consider: