AI Integration Services: Adding Intelligence to Existing Software

By Viprasol Tech Team

---

The majority of AI projects in 2026 are not greenfield AI products — they're AI features added to existing software. A SaaS product adds an AI writing assistant. An ERP adds intelligent invoice processing. A trading platform adds signal generation. A customer support system adds an AI triage bot.

Adding AI to existing software is different from building AI from scratch. The integration work — connecting LLM APIs to your data model, managing context and prompts, handling errors gracefully, controlling cost, and building UIs that make AI output useful — is the actual engineering challenge. The underlying AI capability is a commodity API call.

This guide covers the patterns for adding AI capabilities to existing software and what integration work actually costs.

---

The AI Integration Stack

When a product team decides to "add AI," what they're actually building is a stack of components that didn't exist before:

User Action (trigger)
    ↓
Context Assembly (gather relevant data from your database)
    ↓
Prompt Construction (system prompt + context + user input)
    ↓
LLM API Call (OpenAI / Anthropic / Google / local model)
    ↓
Response Processing (parse, validate, format)
    ↓
Output Delivery (stream to UI / store result / trigger action)
    ↓
Logging + Cost Tracking

Each component needs to be built, tested, and maintained. The LLM API call itself is the smallest part.

---

The LLM Abstraction Layer

Building directly against OpenAI's API creates tight coupling. When Anthropic releases a better model, or when your cost model shifts, you want to swap providers without rewriting your application:

// Provider-agnostic LLM abstraction
interface LLMMessage {
  role:    'system' | 'user' | 'assistant';
  content: string;
}

interface LLMConfig {
  model:       string;
  temperature: number;
  maxTokens:   number;
  stream:      boolean;
}

interface LLMProvider {
  complete(messages: LLMMessage[], config: LLMConfig): Promise<string>;
  stream(messages: LLMMessage[], config: LLMConfig): AsyncGenerator<string>;
}

// OpenAI implementation
class OpenAIProvider implements LLMProvider {
  async complete(messages: LLMMessage[], config: LLMConfig): Promise<string> {
    const response = await openai.chat.completions.create({
      model:       config.model,
      messages,
      temperature: config.temperature,
      max_tokens:  config.maxTokens,
    });
    return response.choices[0].message.content!;
  }

  async *stream(messages: LLMMessage[], config: LLMConfig): AsyncGenerator<string> {
    const stream = await openai.chat.completions.create({
      model: config.model, messages, temperature: config.temperature,
      max_tokens: config.maxTokens, stream: true,
    });
    for await (const chunk of stream) {
      const delta = chunk.choices[0]?.delta?.content;
      if (delta) yield delta;
    }
  }
}

// Anthropic Claude implementation
class AnthropicProvider implements LLMProvider {
  async complete(messages: LLMMessage[], config: LLMConfig): Promise<string> {
    const systemMsg = messages.find(m => m.role === 'system')?.content ?? '';
    const userMsgs  = messages.filter(m => m.role !== 'system');
    const response  = await anthropic.messages.create({
      model:      config.model,
      max_tokens: config.maxTokens,
      system:     systemMsg,
      messages:   userMsgs as any,
    });
    return (response.content[0] as any).text;
  }

  async *stream(messages: LLMMessage[], config: LLMConfig): AsyncGenerator<string> {
    // ...similar pattern
  }
}

// Factory: choose provider based on use case / cost / capability
function createLLMProvider(useCase: 'document-qa' | 'code-gen' | 'summarization'): LLMProvider {
  const providerConfig = {
    'document-qa':   { provider: 'anthropic', model: 'claude-3-5-haiku-20241022' },
    'code-gen':      { provider: 'openai',    model: 'gpt-4o' },
    'summarization': { provider: 'openai',    model: 'gpt-4o-mini' },  // cheaper for simple tasks
  }[useCase];

  return providerConfig.provider === 'openai'
    ? new OpenAIProvider()
    : new AnthropicProvider();
}

---

Streaming Responses to the UI

Users abandon AI features that make them wait 10–30 seconds for a full response. Streaming — sending tokens as they're generated — is non-negotiable for text generation:

// Server: Server-Sent Events for streaming
app.post('/api/ai/generate', async (req, res) => {
  res.setHeader('Content-Type',  'text/event-stream');
  res.setHeader('Cache-Control', 'no-cache');
  res.setHeader('Connection',    'keep-alive');

  const { prompt, documentContext } = req.body;
  const messages = buildMessages(prompt, documentContext);
  const provider = createLLMProvider('document-qa');

  let totalContent = '';
  try {
    for await (const chunk of provider.stream(messages, { model: 'claude-3-5-haiku-20241022', temperature: 0.3, maxTokens: 1000, stream: true })) {
      totalContent += chunk;
      res.write(`data: ${JSON.stringify({ chunk })}\n\n`);
    }
    res.write(`data: ${JSON.stringify({ done: true })}\n\n`);
  } catch (error) {
    res.write(`data: ${JSON.stringify({ error: 'Generation failed' })}\n\n`);
  } finally {
    // Log usage for cost tracking
    await logLLMUsage({ tokens: estimateTokens(totalContent), model: 'claude-3-5-haiku-20241022', userId: req.user.id });
    res.end();
  }
});

// Client: React hook for streaming
function useAIStream() {
  const [content, setContent] = useState('');
  const [isStreaming, setIsStreaming] = useState(false);

  const generate = async (prompt: string, context: string) => {
    setContent('');
    setIsStreaming(true);
    const response = await fetch('/api/ai/generate', {
      method: 'POST',
      headers: { 'Content-Type': 'application/json' },
      body: JSON.stringify({ prompt, documentContext: context }),
    });
    const reader = response.body!.getReader();
    const decoder = new TextDecoder();
    while (true) {
      const { done, value } = await reader.read();
      if (done) break;
      const lines = decoder.decode(value).split('\n').filter(l => l.startsWith('data: '));
      for (const line of lines) {
        const data = JSON.parse(line.slice(6));
        if (data.chunk) setContent(prev => prev + data.chunk);
        if (data.done) setIsStreaming(false);
      }
    }
  };

  return { content, isStreaming, generate };
}

---

Prompt Management and Caching

Prompt versioning — prompts change. Version them like code, track which version produced which output, run evaluation on prompt changes before deploying:

const PROMPTS: Record<string, { version: string; template: string }> = {
  'document-summary': {
    version: '2.1',
    template: `You are an expert at summarizing business documents concisely.

Summarize the following document in 3-5 bullet points. Each bullet should be a complete sentence.
Focus on: key decisions made, action items, and important numbers or dates.

Document:
{document_content}

Summary:`,
  },
};

function buildPrompt(name: string, vars: Record<string, string>): string {
  const prompt = PROMPTS[name];
  if (!prompt) throw new Error(`Unknown prompt: ${name}`);

  let text = prompt.template;
  for (const [key, value] of Object.entries(vars)) {
    text = text.replace(`{${key}}`, value);
  }
  return text;
}

Semantic caching — identical or semantically similar requests return cached responses instead of calling the LLM. Reduces cost by 20–60% for applications with repetitive queries:

import { createHash } from 'crypto';

async function cachedLLMCall(messages: LLMMessage[], config: LLMConfig): Promise<string> {
  // Exact cache key for identical requests
  const cacheKey = createHash('sha256')
    .update(JSON.stringify({ messages, model: config.model }))
    .digest('hex');

  const cached = await redis.get(`llm:${cacheKey}`);
  if (cached) return cached;

  const result = await provider.complete(messages, config);

  // Cache for 1 hour — adjust based on how dynamic your content is
  await redis.setex(`llm:${cacheKey}`, 3600, result);
  return result;
}

---

Cost Management

LLM costs are measured in tokens (roughly 0.75 words per token). At scale, costs add up fast:

Cost control strategies:

1. Route simple tasks to cheap models (GPT-4o-mini, Haiku) and complex tasks to expensive ones
2. Cache repeated requests
3. Compress context — send only relevant chunks, not entire documents
4. Set per-user monthly token budgets
5. Track cost per feature, not just total — identify expensive features early

---

Common AI Integration Patterns

---

Cost Ranges for AI Integration

Ongoing AI infrastructure cost (LLM API fees): $0.10–$5.00 per active user per month depending on feature usage.

---

Working With Viprasol

Our AI development services cover LLM API integration, RAG system development, document processing, and AI feature development for existing SaaS products. We build the full integration stack — abstraction layer, streaming, prompt management, semantic caching, and cost monitoring.

Adding AI to your product? Viprasol Tech integrates LLM and ML capabilities into existing software. Contact us.

---

See also: LLM Integration Guide · Generative AI Consulting · Generative AI Development Company

Sources: OpenAI API Reference · Anthropic API Documentation · LangChain Cost Tracking