Building an AI feature into a Next.js app

Clients are asking for AI features now. Not "make this look smart", but actual AI: a chatbot that knows the product catalogue, a tool that generates a draft, a form that gives instant feedback. Here's how I wire these up cleanly in a Next.js app.

Keep the AI call server-side

The API key never goes to the browser. Full stop. An API route (or a Server Action) handles the call:

// app/api/chat/route.ts
import OpenAI from "openai";
 
const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });
 
export async function POST(request: Request) {
  const { message } = await request.json();
 
  const stream = await openai.chat.completions.create({
    model: "gpt-4o",
    stream: true,
    messages: [
      { role: "system", content: "You are a helpful assistant for CreaTech." },
      { role: "user", content: message },
    ],
  });
 
  return new Response(stream.toReadableStream());
}

The client calls /api/chat. It never touches the OpenAI SDK or the key.

Stream the response

Nobody wants to wait six seconds for a block of text to appear at once. Streaming makes the AI feel fast even when it isn't:

"use client";
async function sendMessage(message: string) {
  const res = await fetch("/api/chat", {
    method: "POST",
    body: JSON.stringify({ message }),
  });
 
  const reader = res.body!.getReader();
  const decoder = new TextDecoder();
 
  while (true) {
    const { done, value } = await reader.read();
    if (done) break;
    setOutput(prev => prev + decoder.decode(value));
  }
}

Each chunk arrives as it's generated. The UI updates token by token. It feels responsive.

Constrain what the model can do

An open-ended chat is hard to control and easy to abuse. Narrow the model's scope with a system prompt that describes exactly what it's for:

You are a product assistant for AstraLuna. Answer questions about our tea
range, brewing guides, and delivery. If asked about anything else, politely
redirect to the topic at hand. Do not mention competitors.

A tight system prompt dramatically reduces unexpected outputs and keeps the feature focused.

Rate-limit the endpoint

Without rate limiting, a publicly accessible AI endpoint is expensive. Even basic limiting at the middleware level (five requests per IP per minute) prevents the obvious abuse cases:

// Vercel KV or a simple in-memory map for development
const requests = new Map<string, number>();
 
export function rateLimit(ip: string, limit = 5): boolean {
  const count = (requests.get(ip) ?? 0) + 1;
  requests.set(ip, count);
  setTimeout(() => requests.delete(ip), 60_000);
  return count <= limit;
}

For production, use a proper store like Vercel KV or Upstash Redis.

The pattern is the same across most AI features: keep the model call server-side, stream the output, constrain the prompt, and protect the endpoint. Everything else is product work.