Potential Improvements with langchaingo v0.1.14 and mcp-go v0.42.0

Last Updated: 2025-10-29 Context: Post-upgrade analysis for langchaingo v0.1.14 and mcp-go v0.42.0

This document outlines potential improvements enabled by new features and bug fixes in the latest versions of our core dependencies.

Recent Updates:

Reprioritized Session Management from P0 to P1 due to high complexity and risk
Adjusted effort estimates upward by 1.5-2.5x for complex tasks
Added testing strategies for all improvements
Reordered roadmap to prioritize observability before session management
Enhanced risk assessment with upstream dependencies and performance degradation risks

Priority Matrix

Priority

Impact

Complexity

Timeline

High

Low-Medium

Immediate

High

Medium-High

Next Sprint

Medium

Low-Medium

Future

Low

Any

Nice to have

High Priority Improvements (P0-P1)

P0-1: Enhanced Token Usage Monitoring

Enabled By: langchaingo v0.1.14 - Exposed token usage details including reasoning tokens

Current State:

Basic token metrics in internal/monitoring/metrics.go
Limited visibility into reasoning vs completion tokens
No per-model or per-provider breakdown

Improvement:

// Add detailed token tracking
type TokenMetrics struct {
    PromptTokens      int
    CompletionTokens  int
    ReasoningTokens   int  // New in v0.1.14
    CachedTokens      int  // Prompt caching support
    Model             string
    Provider          string
    Timestamp         time.Time
}

// Expose via Prometheus
- llm_prompt_tokens_total{model, provider}
- llm_completion_tokens_total{model, provider}
- llm_reasoning_tokens_total{model, provider}  // NEW
- llm_cached_tokens_total{model, provider}     // NEW

Files to Modify:

internal/monitoring/metrics.go - Add new metrics
internal/llm/*_factory.go - Capture token details from responses
internal/observability/langfuse.go - Track reasoning tokens

Benefit:

Better cost tracking and optimization
Identify expensive reasoning operations
Monitor prompt caching effectiveness

Complexity: Medium

P1-2: Session-Specific Resource Management

Enabled By: mcp-go v0.42.0 - Session-specific resources support

Current State:

All MCP resources are global/shared
No per-user or per-thread resource isolation
Potential security/privacy concerns

Improvement:

// Implement session-based resource isolation
type SessionManager struct {
    sessions map[string]*Session  // key: Slack thread ID
}

type Session struct {
    ThreadID    string
    UserID      string
    Resources   []Resource
    MCPClient   *mcp.Client
    CreatedAt   time.Time
    LastAccess  time.Time
}

// Usage: Resources scoped to Slack threads
- Thread A can have private resources not visible to Thread B
- User-specific credentials per session
- Automatic cleanup of stale sessions

Files to Create:

internal/session/manager.go - Session lifecycle management
internal/session/resource.go - Resource isolation

Files to Modify:

internal/mcp/client.go - Add session support
internal/slack/client.go - Associate threads with sessions

Benefit:

Enhanced privacy (thread-isolated resources)
Better multi-tenancy support
Cleaner resource lifecycle

Testing Strategy:

Unit tests for SessionManager concurrency safety
Integration tests for session isolation verification
Security tests to ensure no cross-session data leaks
Load tests for session cleanup and memory management

Complexity: High

P1-1: Improved Streaming Reliability

Enabled By: langchaingo v0.1.14 - Fixed memory and goroutine leaks in streaming

Current State:

Streaming disabled in some scenarios due to reliability concerns
No streaming response updates in Slack
Potential for long waits without feedback

Improvement:

// Enable safe streaming with real-time Slack updates
type StreamingResponse struct {
    MessageTS    string  // Slack message timestamp
    Accumulator  strings.Builder
    UpdateTicker *time.Ticker  // Update Slack every N seconds
}

// Features:
- Progressive message updates in Slack (edit message as tokens arrive)
- Typing indicators during streaming
- Cancellation support via Slack button
- Graceful error handling with partial results

Files to Create:

internal/slack/streaming.go - Streaming message updates

Files to Modify:

internal/slack/client.go - Add streaming message editing
internal/handlers/llm_mcp_bridge.go - Enable streaming mode
internal/llm/langchain.go - Wire up streaming callbacks

Benefit:

Better UX with real-time feedback
Lower perceived latency
Users can see progress on long-running operations

Testing Strategy:

Unit tests for streaming message accumulation and updates
Integration tests with mock LLM streaming responses
End-to-end tests for Slack message editing
Graceful degradation tests when streaming fails

Complexity: Medium-High

P1-3: Resource Middleware for Observability

Enabled By: mcp-go v0.42.0 - Resource middleware extensions

Current State:

Limited visibility into MCP resource access
No caching layer for frequently accessed resources
No audit trail for resource operations

Improvement:

// Implement middleware chain for resources
type ResourceMiddleware interface {
    Before(ctx context.Context, req ResourceRequest) error
    After(ctx context.Context, req ResourceRequest, resp ResourceResponse) error
}

// Built-in middlewares:
1. LoggingMiddleware - Audit all resource access
2. CachingMiddleware - Cache immutable resources
3. MetricsMiddleware - Track access patterns
4. RateLimitMiddleware - Prevent abuse
5. AuthzMiddleware - Per-resource authorization

Files to Create:

internal/middleware/resource_logging.go
internal/middleware/resource_caching.go
internal/middleware/resource_metrics.go

Files to Modify:

internal/mcp/client.go - Apply middleware chain

Benefit:

Better observability and debugging
Performance optimization via caching
Security auditing
Resource access analytics

Testing Strategy:

Unit tests for each middleware component
Integration tests for middleware chain execution
Performance tests for caching effectiveness
Audit log verification tests

Complexity: Medium

Medium Priority Improvements (P2)

P2-1: Enhanced Error Context with Sanitized Messages

Enabled By: langchaingo v0.1.14 - API key sanitization in error messages

Current State:

Generic error messages to users
Risk of exposing sensitive data in logs
Limited context for debugging

Improvement:

// Safe error reporting with rich context
type SafeError struct {
    UserMessage    string           // Sanitized, user-friendly
    InternalError  error            // Full error for logs
    Context        map[string]any   // Safe context data
    RequestID      string
    Timestamp      time.Time
}

// Features:
- Automatic API key redaction
- User-friendly error messages in Slack
- Detailed internal logs for debugging
- Error categorization (transient, permanent, user-error)

Files to Create:

internal/errors/safe_error.go

Files to Modify:

internal/common/errors/errors.go - Add sanitization
internal/slack/formatter/formatter.go - Format user-facing errors

Benefit:

Better security (no accidental leaks)
Improved user experience
Easier debugging

Testing Strategy:

Unit tests for sanitization functions
Security tests to verify no sensitive data in user-facing messages
Integration tests for error propagation through stack

Complexity: Low-Medium

P2-2: Flexible Tool Properties with WithAny

Enabled By: mcp-go v0.42.0 - WithAny for adaptable tool properties

Current State:

Static tool configurations
Hard to add dynamic tool metadata
Limited extensibility

Improvement:

// Dynamic tool configuration
type DynamicTool struct {
    BaseConfig   ToolConfig
    Extensions   map[string]any  // Using WithAny
}

// Use cases:
- Runtime tool configuration updates
- Per-user tool customization
- Feature flags for experimental tools
- A/B testing tool variations

Files to Modify:

internal/mcp/mcpTool.go - Support dynamic properties
internal/config/config.go - Load dynamic configurations

Benefit:

More flexible tool management
Easier experimentation
Per-tenant customization

Testing Strategy:

Unit tests for dynamic property loading
Integration tests for runtime tool reconfiguration
Validation tests for tool property schemas

Complexity: Medium

P2-3: HTTP Sampling for Debugging

Enabled By: mcp-go v0.42.0 - HTTP sampling improvements

Current State:

Limited HTTP request/response visibility
Hard to debug MCP transport issues
No sampling for production debugging

Improvement:

// HTTP request/response sampling
type HTTPSampler struct {
    SampleRate   float64  // 0.0 to 1.0
    MaxBodySize  int
    Destinations []SampleSink
}

// Features:
- Configurable sampling rate (e.g., 1% in production)
- Request/response body capture
- Export to Langfuse, file, or telemetry system
- Performance impact monitoring

Files to Create:

internal/observability/http_sampler.go

Files to Modify:

internal/mcp/sseClient.go - Add sampling hooks
internal/mcp/client.go - Configure sampling

Benefit:

Production debugging capability
Better issue reproduction
Performance analysis

Testing Strategy:

Unit tests for sampling rate logic
Integration tests for sample capture and export
Performance tests to measure sampling overhead
Privacy tests to ensure sensitive data handling

Complexity: Medium

P2-4: Improved Agent Multi-Tool Orchestration

Enabled By: langchaingo v0.1.14 - Improved multi-tool support and parsing

Current State:

Agents process tools sequentially
Limited parallel tool execution
No dependency graph for tools

Improvement:

// Parallel and dependency-aware tool execution
type ToolOrchestrator struct {
    DependencyGraph map[string][]string
    Executor        *ParallelExecutor
}

// Features:
- Identify independent tools and run in parallel
- Build dependency graphs from tool schemas
- Automatic retry with backoff
- Circuit breaker for failing tools

Files to Create:

internal/agents/orchestrator.go
internal/agents/dependency_graph.go

Files to Modify:

internal/llm/langchain.go - Use orchestrator

Benefit:

Faster multi-tool workflows
Better resource utilization
More robust agent behavior

Testing Strategy:

Unit tests for dependency graph construction
Integration tests for parallel execution scenarios
Failure recovery and retry mechanism tests
Performance benchmarks for parallel vs sequential execution

Complexity: High (requires design spike first)

Low Priority Improvements (P3)

P3-1: Streaming Control with WithDisableStreaming

Enabled By: mcp-go v0.42.0 - WithDisableStreaming option

Current State:

Streaming always enabled or disabled globally
No per-request streaming control
Can't optimize based on request type

Improvement:

// Dynamic streaming control
func (c *Client) CallTool(ctx context.Context, req ToolRequest) {
    // Disable streaming for small/fast operations
    if req.ExpectedDuration < 5*time.Second {
        client = client.WithDisableStreaming(true)
    }
    // Enable for long-running operations
}

Benefit: Optimized performance for different request types Complexity: Low

P3-2: Enhanced Reconnection Strategy

Enabled By: mcp-go v0.42.0 - Idempotent Start() method

Current State:

Fixed backoff strategy
Limited reconnection intelligence
No adaptive retry logic

Improvement:

// Intelligent reconnection with adaptive backoff
type AdaptiveReconnection struct {
    SuccessRate      float64
    HealthScore      int
    BackoffStrategy  BackoffFunc  // Adaptive based on failure patterns
}

// Features:
- Track success patterns and adjust accordingly
- Different strategies for different failure types
- Circuit breaker after repeated failures
- Automatic recovery testing

Benefit: More reliable connections, faster recovery Complexity: Medium

P3-3: Tool Result Annotations

Enabled By: mcp-go v0.41.0 - Call tool result annotations support

Current State:

Plain text tool results
No structured metadata
Limited result interpretation

Improvement:

// Rich tool results with annotations
type AnnotatedResult struct {
    Content      string
    Annotations  map[string]Annotation
    Confidence   float64
    Sources      []Source
    Metadata     map[string]any
}

// Use in Slack:
- Show confidence scores
- Link to sources
- Highlight important parts
- Structured data rendering

Benefit: Better result presentation, more context Complexity: Medium

P3-4: Advanced Callback Handlers

Enabled By: langchaingo v0.1.14 - Improved callback handling

Current State:

Basic callbacks in agentCallbackHandler.go
Limited insight into agent reasoning
No callback composition

Improvement:

// Composable callback handlers
type CallbackChain struct {
    Handlers []callbacks.Handler
}

// Built-in handlers:
- DebugCallbackHandler - Detailed logging
- MetricsCallbackHandler - Track agent performance
- SlackCallbackHandler - Real-time Slack updates
- AuditCallbackHandler - Compliance logging
- CostCallbackHandler - Token usage per step

Benefit: Better observability, flexible monitoring Complexity: Low-Medium

Implementation Roadmap

Phase 1: Foundation (Sprint 1)

✅ Upgrade langchaingo to v0.1.14 (DONE)
✅ Upgrade mcp-go to v0.42.0 (DONE)
P0-1: Enhanced Token Usage Monitoring
P2-1: Enhanced Error Context

Phase 2: UX & Observability (Sprint 2)

P1-1: Improved Streaming Reliability
P1-3: Resource Middleware for Observability (start)

Phase 3: Core Architecture (Sprint 3)

P1-3: Resource Middleware for Observability (complete)
P1-2: Session-Specific Resource Management (start)
P2-3: HTTP Sampling for Debugging

Phase 4: Core Architecture Completion (Sprint 4)

P1-2: Session-Specific Resource Management (complete and test)

Phase 5: Optimization (Sprint 5)

P2-2: Flexible Tool Properties
P2-4: Improved Agent Multi-Tool Orchestration (requires design spike)

Phase 6: Polish (Sprint 6)

P3 items as capacity allows
Documentation updates
Performance testing

Metrics for Success

Token Monitoring (P0-1)

Target: 100% token visibility across all providers
KPI: Cost reduction by 15% through optimization insights

Streaming (P1-1)

Target: 90% of responses use streaming
KPI: 50% reduction in perceived latency

Session Management (P1-2)

Target: Thread-isolated resources for 100% of conversations
KPI: Zero cross-thread resource leaks

Observability (P1-3, P2-3)

Target: 95% of issues debuggable from metrics alone
KPI: 30% reduction in MTTR (Mean Time To Resolution)

Dependencies and Prerequisites

Required Before Implementation

Testing Infrastructure: Integration tests for streaming, sessions
Monitoring Stack: Prometheus + Grafana for new metrics
Documentation: Update architecture docs with new patterns
Configuration Management: Strategy for managing new feature flags, middleware toggles, sampling rates, and session timeouts
Performance Baseline: Establish current performance metrics before adding new features

Nice to Have

Staging environment for feature validation
Load testing capabilities
Automated performance benchmarks
Centralized feature flagging system for gradual rollouts

Risk Assessment

Improvement

Risk Level

Mitigation

P0-1: Token Monitoring

Low

Additive only, no behavior changes

P1-1: Streaming

Medium

Fallback to non-streaming, graceful degradation

P1-2: Session Management

Critical

Feature flag, gradual rollout, extensive security testing to prevent cross-session data leaks

P1-3: Resource Middleware

Medium

Disable individual middlewares, performance monitoring

P2-1: Error Context

Low

Extensive testing for leaks

P2-4: Multi-Tool Orchestration

High

Design spike first, start with opt-in agent mode

Upstream Dependencies

Medium

Monitor for bugs in new langchaingo/mcp-go features, maintain rollback capability

Performance Degradation

Medium

Establish performance baselines, continuous load testing, add performance regression tests

Cost-Benefit Analysis

High ROI Improvements

P0-1 (Token Monitoring): Low cost, high benefit for cost optimization
P1-1 (Streaming): Medium cost, high UX improvement - delivers immediate user value
P2-1 (Error Context): Low cost, immediate security benefit

Strategic Investments

P1-3 (Resource Middleware): Medium cost, foundational for observability - enables safer rollout of complex features
P1-2 (Session Management): High cost, enables multi-tenancy and privacy - requires extensive testing and careful rollout

Future Considerations

P3 items provide incremental improvements
Implement based on user feedback and telemetry

References

Feedback and Iteration

This document should be reviewed and updated:

After each sprint/milestone
When new library versions are released
Based on user feedback and production metrics
Quarterly for priority re-evaluation

Next Review Date: 2026-01-29

PreviousImplementation Notes: Slack MCP Client NextSimpleProvider RAG Implementation

Last updated 21 days ago

Was this helpful?

Good afternoon

Priority Matrix

High Priority Improvements (P0-P1)

P0-1: Enhanced Token Usage Monitoring

P1-2: Session-Specific Resource Management

P1-1: Improved Streaming Reliability

P1-3: Resource Middleware for Observability

Medium Priority Improvements (P2)

P2-1: Enhanced Error Context with Sanitized Messages

P2-2: Flexible Tool Properties with WithAny

P2-3: HTTP Sampling for Debugging

P2-4: Improved Agent Multi-Tool Orchestration

Low Priority Improvements (P3)

P3-1: Streaming Control with WithDisableStreaming

P3-2: Enhanced Reconnection Strategy

P3-3: Tool Result Annotations

P3-4: Advanced Callback Handlers

Implementation Roadmap

Phase 1: Foundation (Sprint 1)

Phase 2: UX & Observability (Sprint 2)

Phase 3: Core Architecture (Sprint 3)

Phase 4: Core Architecture Completion (Sprint 4)

Phase 5: Optimization (Sprint 5)

Phase 6: Polish (Sprint 6)

Metrics for Success

Token Monitoring (P0-1)

Streaming (P1-1)

Session Management (P1-2)

Observability (P1-3, P2-3)

Dependencies and Prerequisites

Required Before Implementation

Nice to Have

Risk Assessment

Cost-Benefit Analysis

High ROI Improvements

Strategic Investments

Future Considerations

References

Feedback and Iteration