The LLM Revolution

Large Language Models have transformed what's possible in software applications. From customer support to code generation, LLMs are becoming essential enterprise tools.

Integration Patterns

Direct API Integration

import OpenAI from 'openai';

const openai = new OpenAI({ apiKey: process.env.OPENAI_API_KEY });

async function generateResponse(userMessage) {
  const response = await openai.chat.completions.create({
    model: 'gpt-4',
    messages: [
      { role: 'system', content: 'You are a helpful assistant.' },
      { role: 'user', content: userMessage }
    ],
    temperature: 0.7,
    max_tokens: 500
  });
  
  return response.choices[0].message.content;
}

RAG (Retrieval Augmented Generation)

async function answerWithContext(question) {
  // 1. Convert question to embedding
  const embedding = await openai.embeddings.create({
    model: 'text-embedding-3-small',
    input: question
  });
  
  // 2. Search vector database for relevant documents
  const relevantDocs = await vectorDb.search(embedding.data[0].embedding, 5);
  
  // 3. Generate answer with context
  const context = relevantDocs.map(d => d.content).join('\n\n');
  
  return await openai.chat.completions.create({
    model: 'gpt-4',
    messages: [
      { role: 'system', content: `Answer based on this context:\n${context}` },
      { role: 'user', content: question }
    ]
  });
}

Prompt Engineering

System Prompts

const systemPrompt = `
You are a customer support agent for TechCorp.

Rules:
- Be helpful and professional
- Only answer questions about our products
- If you don't know, say so - don't make up information
- For billing issues, direct to billing@techcorp.com

Product catalog: [product details here]
`;

Few-Shot Examples

const messages = [
  { role: 'system', content: 'Extract order information from customer messages.' },
  { role: 'user', content: 'I ordered 5 blue widgets yesterday' },
  { role: 'assistant', content: '{"product": "widget", "color": "blue", "quantity": 5}' },
  { role: 'user', content: 'Need 10 red gadgets shipped to NYC' },
  { role: 'assistant', content: '{"product": "gadget", "color": "red", "quantity": 10, "destination": "NYC"}' },
  { role: 'user', content: actualUserMessage }
];

Production Considerations

Rate Limiting

const rateLimiter = new RateLimiter({
  tokensPerInterval: 60,
  interval: 'minute'
});

async function limitedLLMCall(prompt) {
  await rateLimiter.removeTokens(1);
  return await openai.chat.completions.create(...);
}

Caching

async function cachedLLMCall(prompt) {
  const cacheKey = hash(prompt);
  const cached = await redis.get(cacheKey);
  if (cached) return JSON.parse(cached);
  
  const response = await openai.chat.completions.create(...);
  await redis.setEx(cacheKey, 3600, JSON.stringify(response));
  return response;
}

Fallback Strategy

async function resilientLLMCall(prompt) {
  try {
    return await openai.chat.completions.create({ model: 'gpt-4', ... });
  } catch (error) {
    console.warn('GPT-4 failed, falling back to GPT-3.5');
    return await openai.chat.completions.create({ model: 'gpt-3.5-turbo', ... });
  }
}

Cost Optimization

• Use smaller models for simple tasks
• Implement caching for repeated queries
• Set appropriate max_tokens limits
• Batch requests where possible
• Monitor usage with alerts

Conclusion

LLMs are powerful but require careful integration. Focus on reliability, cost management, and appropriate guardrails for production deployments.