LegalFab AI & LLM

Version: 1.1 Last Updated: January 2026

Component Overview

The AI & LLM Architecture provides the intelligence layer for the LegalFab platform. Built on LightLLM, the platform is LLM-provider-agnostic, enabling flexible model selection while maintaining consistent security controls. The architecture ensures reproducible, auditable AI operations through schema validation, ontology-based execution, and comprehensive provenance tracking.

Key Security Characteristics:

• Prompt injection defense
• Data isolation prevents cross-tenant information leakage
• Secure API integration with LLM providers
• Comprehensive content filtering and safety guardrails
• Privacy-preserving retrieval
• Human-in-the-loop controls for high-risk operations
• Output consistency through schema validation and deterministic workflows
• Complete provenance tracking for audit and reproducibility

LightLLM Router Architecture

LegalFab uses LightLLM as a unified routing layer for all LLM provider access:

┌─────────────────────────────────────────────────────────────────────┐
│                    LIGHTLLM ROUTER                                  │
├─────────────────────────────────────────────────────────────────────┤
│  ┌─────────────┐  ┌─────────────┐  ┌─────────────────────┐          │
│  │  Unified    │  │  Load       │  │  Fallback           │          │
│  │  API        │  │  Balancing  │  │  Routing            │          │
│  └─────────────┘  └─────────────┘  └─────────────────────┘          │
│                              │                                      │
│         ┌────────────────────┼────────────────────┐                 │
│         ▼                    ▼                    ▼                 │
│  ┌─────────────┐      ┌─────────────┐      ┌─────────────┐          │
│  │  Provider A │      │  Provider B │      │  Self-      │          │
│  │             │      │             │      │  Hosted     │          │
│  └─────────────┘      └─────────────┘      └─────────────┘          │
└─────────────────────────────────────────────────────────────────────┘

LightLLM Security Benefits

LLM Output Consistency

Traditional LLM applications can produce different outputs for identical inputs. LegalFab addresses this through a multi-layered architectural approach that constrains and validates LLM outputs at every step.

Schema-Validated Extraction

Ontology-Based Execution

The ontology defines relationships between entities, execution sequences, and dependencies among analytical steps. This ensures consistent reasoning pathways regardless of when or how often an analysis is performed.

Reasoning Chain Transparency

Each analytical step documents its reasoning for complete traceability:

Deterministic vs. Probabilistic Components

LLM Variability Controls:

LLM Monitoring and Observability

Phoenix Monitoring Integration

LegalFab integrates with LLM observability tools for comprehensive monitoring:

Quality Metrics

Model Version Management

Model Update Process

When LLM models are updated, organizations need assurance that investigative conclusions remain credible and traceable.

Pre-Deployment Testing:

Controlled Rollout:

Rollback Capability:

LLM Provenance

Complete Invocation Logging

Every LLM invocation captures:

Provenance Security

Provider Security

Provider Assessment Requirements

Provider Isolation

Prompt Security

Prompt Injection Defense

┌─────────────────────────────────────────────────────────────────────┐
│                    PROMPT INJECTION DEFENSE                          │
├─────────────────────────────────────────────────────────────────────┤
│  LAYER 1: INPUT PREPROCESSING                                       │
│  • Character filtering    • Length limits    • Encoding validation  │
├─────────────────────────────────────────────────────────────────────┤
│  LAYER 2: INJECTION DETECTION                                       │
│  • ML-based classifier    • Pattern matching    • Semantic analysis │
├─────────────────────────────────────────────────────────────────────┤
│  LAYER 3: PROMPT ARCHITECTURE                                       │
│  • Instruction hierarchy    • Delimiter separation    • Anchoring   │
├─────────────────────────────────────────────────────────────────────┤
│  LAYER 4: OUTPUT VALIDATION                                         │
│  • Format validation    • Action authorization    • Anomaly check   │
└─────────────────────────────────────────────────────────────────────┘

Input Sanitization

Instruction Hierarchy

Data Protection in AI Pipelines

PII Handling

Pre-Processing (Before LLM):

Post-Processing (After LLM):

Tenant Data Isolation

Content Safety

Input Content Filtering

Output Content Filtering

Hallucination Mitigation

Vector and Embedding Security

Embedding Security

Retrieval Security

Audit Logging

AI-Specific Events

Security Events

Responsible AI

Ethical Principles

Explainability

Quality Assurance and Feedback

Output Tracking Mechanisms

Model outputs are tracked through three complementary mechanisms:

Quality Adjustment Mechanisms

When quality falls short, adjustments occur without traditional ML retraining:

Continuous Improvement

Model Lifecycle Management

Model Selection and Configuration

Model-Agnostic Architecture

The Knowledge Fabric and agent infrastructure consume structured metadata outputs regardless of which LLM generated them:

Model Deprecation