Monitoring

Gain deep insights into your prompts' performance, costs, and behavior in production environments with Basalt's prompt monitoring system. Our monitoring tools help you track, analyze, and optimize your AI applications in real-time.

Before using the monitoring features, make sure you have your API key ready. You can find it in your Basalt workspace settings.

Overview

Basalt's Monitoring provides real-time visibility into your prompt engineering lifecycle, helping you optimize costs, improve response quality, and maintain reliability at scale. Our monitoring system is specifically designed for AI applications, offering insights that matter for prompt engineering teams.

Creating Traces

Traces allow you to group related monitoring events together. Start by initializing the SDK with your API key and creating a trace.

import { Basalt } from '@basalt-ai/sdk'

// Initialize with API key from environment
const basalt = new Basalt({
  apiKey: process.env.BASALT_API_KEY,
  logLevel: 'warning' // Optional, defaults to 'warning'
})

// Create a trace to group related monitoring events
const trace = basalt.monitor.createTrace('chain-slug', {
  name: 'user-conversation',
  metadata: {
    sessionId: 'session-456',
  }
})

Advanced Monitoring with Spans

Spans allow you to track specific operations within a trace, creating a hierarchical view of your application's execution:

// Create a main trace for the entire user request
const mainTrace = basalt.monitor.createTrace('content-processing', {
  metadata: {
    userId: 'user-123',
    contentType: 'article',
    requestId: 'req-789'
  }
})

// Create a span for a specific operation
const generationSpan = mainTrace.createLog({
  name: 'content-generation',
  type: 'span',
  input: 'Create an article about AI monitoring',
  metadata: {
    operation: 'text-generation',
    priority: 'high'
  }
})

try {
  // Perform your operation
  const result = await someAsyncOperation()
  
  // End the span with the result
  generationSpan.end(result)
} catch (error) {
  // Update span with error information
  generationSpan.update({ 
    metadata: {
      error: {
        name: 'OperationError',
        message: error.message
      }
    }
  })
  generationSpan.end()
} finally {
  // Always end the trace when done
  mainTrace.end()
}

Tracking LLM Generations

Track detailed information about your LLM generations, including prompts, completions, and performance metrics:

// There are 2 ways to create a generation:
// 1. Using the prompt stored in Basalt

// Get the production version of a prompt
const { value, generation } = await basalt.prompt.get('welcome-message')
// {value} contains the prompt text located in value.text and the systemText located in value.systemText
// These are the values you should use to feed into the LLM provider SDK

// Here you have 2 options, use the instanciated generation or create yours

// If you need to create your own generation you can reuse the prompt retrieved from Basalt
const customGeneration = generationSpan.createGeneration({
  name: 'text-generation',
  // If you provide the prompt object Basalt will retrieve the prompt text and system text and use them as the input
  prompt: {
    slug: 'prompt-slug'
  },
  // If you provide the prompt object you need to pass the variables used for the inference (you can also use generation.update with the variables later on)
  variables: { 'query': userQuery },
  metadata: {
    model: 'gpt-4',
    temperature: 0.7,
    requestTime: new Date().toISOString()
  }
})

// Make the LLM call
const completion = await openai.chat.completions.create({
  model: "gpt-4",
  messages: [
    { role: "system", content: value.systemText },
    { role: "user", content: value.text }
  ],
  temperature: 0.7,
  max_tokens: 500,
})

// Using the generation from the prompt retrieval step
// You don't need to provide the variables as they were already setup in the prompt.get method above
generation.end(completion.choices[0].message.content)

// Update the generation with results and metrics
customGeneration.update({ 
  output: completion.choices[0].message.content,
  metadata: {
    ...generation.metadata,
    promptTokens: completion.usage.prompt_tokens,
    completionTokens: completion.usage.completion_tokens,
    totalTokens: completion.usage.total_tokens,
    latencyMs: Date.now() - startTime
  }
})

customGeneration.end()

Real-world Example: Multi-step AI Pipeline

Here's a comprehensive example of monitoring a complex AI pipeline with multiple steps:

async function processUserContent(user, content) {
  // Create a main trace for the entire user request
  const mainTrace = basalt.monitor.createTrace('content-processing', {
    metadata: {
      userId: user.id,
      userName: user.name,
      contentType: 'article'
    }
  })
  
  try {
    // Step 1: Content generation
    const generationSpan = mainTrace.createLog({
      name: 'content-generation',
      type: 'span',
      input: content
    })
    
    try {
      // Get prompt from Basalt
      const basaltPrompt = await basalt.prompt.get('article-generator', {
        variables: { 'article_subject': content },
        version: '1.2'
      })
      
      // Create generation record
      const generationLog = generationSpan.createGeneration({
        name: 'text-generation',
        prompt: basaltPrompt.generation.prompt,
        variables: { 'article_subject': content }
      })
      
      // Generate content with LLM
      const generatedText = await generateText(basaltPrompt.value.text)
      
      // Update with results
      generationLog.update({ 
        output: generatedText,
        metadata: {
          contentLength: generatedText.length,
          processingTime: 1200 // ms
        }
      })
      
      generationSpan.end(generatedText)
      
      // Step 2: Parallel processing - Classification and Translation
      const parallelSpan = mainTrace.createLog({
        name: 'parallel-processing', 
        type: 'span',
        input: generatedText
      })
      
      // Run operations in parallel and track each one
      const [categories, translatedText] = await Promise.all([
        classifyContent(generatedText, parallelSpan),
        translateContent(generatedText, parallelSpan)
      ])
      
      parallelSpan.end()
      
      // Step 3: Summarization
      const summarySpan = mainTrace.createGeneration({
        name: 'summarization',
        input: generatedText
      })
      
      const summary = await summarizeContent(generatedText, summarySpan)
      summarySpan.end(summary)
      
      // Complete the main trace with final output
      mainTrace.update({
        output: summary,
        metadata: {
          processingSteps: 3,
          totalContentLength: generatedText.length,
          status: 'completed'
        }
      })
      
      return summary
    } catch (error) {
      // Handle and record errors
      mainTrace.update({
        metadata: {
          error: {
            name: 'ProcessingError',
            message: error.message
          },
          status: 'failed'
        }
      })
      throw error
    } finally {
      // Always end the trace
      mainTrace.end()
    }
  }
}

Parameters Reference

Trace Parameters

interface TraceParams {
  metadata?: Record<string, any>;  // Optional metadata for the trace
  input?: string;                  // Input data for the trace
  output?: string;                 // Output data for the trace
  name?: string;                   // Optional name for the trace
}

Span Parameters

interface SpanParams {
  name: string;                    // Name of the span
  input?: string;                  // Input data for the span
  output?: string;                 // Output data for the span
  metadata?: Record<string, any>;  // Optional metadata for the span
}

Generation Parameters

interface GenerationParams {
  name?: string;                   // Optional name for the generation
  traceId?: string;                // Optional trace ID to associate with
  input?: string;                  // Input data for the generation
  output?: string;                 // Output from the LLM
  prompt?: string;                 // The prompt template sent to the LLM
  variables?: Record<string, any>; // Variables used in the prompt
  metadata?: Record<string, any>;  // Additional contextual information
}

Error Handling

The SDKs handle errors differently based on the language conventions:

try {
  const trace = basalt.monitor.createTrace('user-conversation')
  
  // Use the trace
  console.log('Trace created with ID:', trace.id)
} catch (error) {
  console.error('Error creating trace:', error.message)
}

Dashboard Features

The Basalt monitoring dashboard provides:

Real-time prompt performance visualization
Token usage and cost breakdowns
Response quality trends
Anomaly detection and alerts
Prompt version comparison tools
Custom metric tracking capabilities

For additional support or to discuss specific monitoring needs, please contact our team.