LLM Integration Guide: Adding AI to Your Application in 2026

Integrating large language models into production applications is now a core engineering skill. The patterns have matured, the costs have dropped significantly, and the tooling is excellent. This guide covers everything from model selection to production deployment.

Choosing the Right Model

// Model selection decision framework
const models = {
  "gpt-4o": {
    strengths: "Best reasoning, code, complex analysis",
    inputCost: "$2.50/1M tokens",
    outputCost: "$10.00/1M tokens",
    contextWindow: "128K tokens",
    bestFor: "Complex tasks, agentic workflows, nuanced generation"
  },
  "gpt-4o-mini": {
    strengths: "Fast, cheap, good quality",
    inputCost: "$0.15/1M tokens",
    outputCost: "$0.60/1M tokens",
    contextWindow: "128K tokens",
    bestFor: "High-volume tasks, classification, extraction, simple generation"
  },
  "claude-3-5-sonnet": {
    strengths: "Long context, coding, nuanced writing",
    inputCost: "$3.00/1M tokens",
    outputCost: "$15.00/1M tokens",
    contextWindow: "200K tokens",
    bestFor: "Document analysis, complex writing, code review"
  },
  "llama-3.1-70b": {
    strengths: "Self-hosted, no data leaves your infra",
    inputCost: "Infrastructure cost only",
    outputCost: "Infrastructure cost only",
    contextWindow: "128K tokens",
    bestFor: "Privacy-sensitive applications, high volume"
  }
}

Rule of thumb: start with GPT-4o-mini for most tasks (80% of quality, 17x cheaper than GPT-4o). Switch to GPT-4o only for tasks where quality measurably matters.

Production API Integration

import OpenAI from "openai"
import { z } from "zod"
import { zodResponseFormat } from "openai/helpers/zod"

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY })

// Structured output — always prefer over parsing free text
const AnalysisSchema = z.object({
  sentiment: z.enum(["positive", "neutral", "negative"]),
  confidence: z.number().min(0).max(1),
  keyPoints: z.array(z.string()).max(5),
  actionRequired: z.boolean(),
})

async function analyseCustomerFeedback(text: string) {
  const response = await openai.beta.chat.completions.parse({
    model: "gpt-4o-mini",
    messages: [
      {
        role: "system",
        content: `You are a customer feedback analyst for a B2B SaaS company.
Analyse the feedback and extract sentiment, confidence score, and key points.
Be concise. Key points should be specific and actionable.`
      },
      { role: "user", content: text }
    ],
    response_format: zodResponseFormat(AnalysisSchema, "analysis"),
    temperature: 0.1,  // Low temperature for consistent analysis
    max_tokens: 500,
  })

  return response.choices[0].message.parsed
}

// Streaming for long-form generation
async function* generateBlogPost(topic: string) {
  const stream = await openai.chat.completions.stream({
    model: "gpt-4o",
    messages: [
      { role: "system", content: "You are an expert technical writer." },
      { role: "user", content: `Write a 1000-word post about: ${topic}` }
    ],
  })

  for await (const chunk of stream) {
    const delta = chunk.choices[0]?.delta?.content
    if (delta) yield delta
  }
}

RAG (Retrieval-Augmented Generation)

RAG lets your LLM answer questions about your proprietary data without fine-tuning:

import { OpenAIEmbeddings } from "@langchain/openai"
import { PineconeStore } from "@langchain/pinecone"
import { Pinecone } from "@pinecone-database/pinecone"

// 1. Ingest documents
async function ingestDocument(text: string, metadata: object) {
  const embeddings = new OpenAIEmbeddings({ model: "text-embedding-3-small" })
  const pinecone = new Pinecone()
  const index = pinecone.Index("knowledge-base")
  const store = await PineconeStore.fromExistingIndex(embeddings, { pineconeIndex: index })
  
  await store.addDocuments([{ pageContent: text, metadata }])
}

// 2. Query with context
async function queryWithContext(question: string): Promise<string> {
  const embeddings = new OpenAIEmbeddings({ model: "text-embedding-3-small" })
  const store = await PineconeStore.fromExistingIndex(embeddings, {
    pineconeIndex: pinecone.Index("knowledge-base")
  })
  
  // Retrieve relevant documents
  const docs = await store.similaritySearch(question, 5)
  const context = docs.map(d => d.pageContent).join("

---

")
  
  // Generate answer grounded in context
  const response = await openai.chat.completions.create({
    model: "gpt-4o-mini",
    messages: [
      {
        role: "system",
        content: `Answer questions using ONLY the provided context. 
If the answer is not in the context, say "I don't have information about that."
Never make up information.

Context:
${context}`
      },
      { role: "user", content: question }
    ],
    temperature: 0.1,
  })
  
  return response.choices[0].message.content!
}

Cost Management

At scale, LLM API costs can become significant. Strategies:

Prompt caching — OpenAI caches repeated prompt prefixes. Structure prompts so the static system prompt is at the beginning.

Model routing — classify task complexity first, route simple tasks to cheaper models.

Token optimisation — shorter prompts cost less. Remove unnecessary instructions and examples.

Caching responses — cache identical queries with Redis. LLM output is deterministic at temperature=0.

Budget alerts — set OpenAI usage limits and configure alerting before costs exceed expectations.

---

Need LLM integrated into your product? Viprasol builds production AI systems with RAG, agents, and structured outputs. Contact us.

See also: Generative AI Development Company Guide · AI Chatbot Development Company Guide