Skip to main content

Gateway Pattern: Zero-Inference Routing

SuperModel’s MCP gateway pattern provides intelligent routing and orchestration of multiple UI generation tools while maintaining zero server inference. This is similar to existing MCP gateway tools, with a critical difference: all routing decisions happen via MCP sampling.

Traditional vs SuperModel Gateway

class TraditionalGateway {
  async route(request: string): Promise<Tool> {
    // Server pays for LLM inference
    const decision = await this.llm.analyze({
      prompt: "Which tool should handle this request?",
      context: request
    });
    
    return this.tools[decision.toolName];
  }
}

// Cost: $0.02-0.10 per routing decision
Problems:
  • Server incurs LLM costs for every request
  • Need to maintain LLM infrastructure
  • Scaling costs increase linearly

Gateway Architecture

Core Components

1. Tool Registry

The gateway maintains a registry of available UI generation tools:
interface ToolDefinition {
  id: string;
  name: string;
  description: string;
  capabilities: string[];
  examples: string[];
  contextRequirements?: string[];
  outputFormats: string[];
}

class ToolRegistry {
  private tools: Map<string, ToolDefinition> = new Map();
  
  register(tool: ToolDefinition): void {
    this.tools.set(tool.id, tool);
  }
  
  getAll(): ToolDefinition[] {
    return Array.from(this.tools.values());
  }
  
  findByCapability(capability: string): ToolDefinition[] {
    return this.getAll().filter(tool => 
      tool.capabilities.includes(capability)
    );
  }
}

// Example tool registration
const registry = new ToolRegistry();

registry.register({
  id: "product-search-ui",
  name: "Product Search Interface",
  description: "E-commerce product search with filters, sorting, and shopping cart functionality",
  capabilities: ["search", "filter", "sort", "cart", "e-commerce"],
  examples: [
    "Find wireless headphones under $200",
    "Search for office furniture with reviews",
    "Browse laptops with technical specifications"
  ],
  contextRequirements: ["product_catalog"],
  outputFormats: ["ag-ui"]
});

registry.register({
  id: "data-viz-ui", 
  name: "Data Visualization",
  description: "Interactive charts, graphs, and dashboard components",
  capabilities: ["charts", "graphs", "analytics", "dashboard"],
  examples: [
    "Show sales trends over time",
    "Create a revenue dashboard",
    "Visualize user engagement metrics"
  ],
  contextRequirements: ["data_source"],
  outputFormats: ["ag-ui", "react"]
});

2. Sampling Router

The router uses MCP sampling to make intelligent tool selection decisions:
class SamplingRouter {
  constructor(
    private gateway: SuperModelGateway,
    private registry: ToolRegistry
  ) {}

  async route(
    request: string, 
    context: UserContext
  ): Promise<RoutingDecision> {
    const availableTools = this.registry.getAll();
    const routingPrompt = this.buildRoutingPrompt(request, context, availableTools);

    const samplingResponse = await this.gateway.sample({
      messages: [{
        role: "user",
        content: { type: "text", text: routingPrompt }
      }],
      systemPrompt: this.getRoutingSystemPrompt(),
      maxTokens: 200
    });

    const decision = JSON.parse(samplingResponse.content.text);
    
    return {
      tool: decision.tool,
      params: decision.params,
      confidence: decision.confidence,
      reasoning: decision.reasoning
    };
  }

  private buildRoutingPrompt(
    request: string,
    context: UserContext,
    tools: ToolDefinition[]
  ): string {
    return `
User Request: "${request}"

User Context:
${JSON.stringify({
  journey: context.journey,
  preferences: context.preferences,
  data: context.data
}, null, 2)}

Available Tools:
${tools.map(tool => `
- ${tool.id}: ${tool.description}
  Capabilities: ${tool.capabilities.join(', ')}
  Examples: ${tool.examples.slice(0, 2).join('; ')}
`).join('\n')}

Which tool should handle this request? Consider:
1. User's current journey stage
2. Request intent and context
3. Tool capabilities and examples
4. User's demonstrated preferences

Return JSON:
{
  "tool": "tool-id",
  "params": {
    // Parameters to pass to the tool
  },
  "confidence": 0.0-1.0,
  "reasoning": "Why this tool is the best choice"
}`;
  }

  private getRoutingSystemPrompt(): string {
    return `You are an intelligent routing assistant for a UI generation system. 

Your job is to analyze user requests and context to select the most appropriate tool for generating a UI response.

Rules:
1. Always return valid JSON with the specified structure
2. Consider user context and journey stage
3. Match request intent to tool capabilities
4. Provide confidence score based on certainty of match
5. Give clear reasoning for the selection

Be decisive - users expect a specific tool selection, not multiple options.`;
  }
}

3. Gateway Orchestrator

The main gateway coordinates the entire flow:
class SuperModelGateway {
  private router: SamplingRouter;
  private registry: ToolRegistry;
  private tools: Map<string, UIGenerationTool>;

  constructor(config: GatewayConfig) {
    this.registry = new ToolRegistry();
    this.router = new SamplingRouter(this, this.registry);
    this.tools = new Map();
    
    // Load tools from config
    this.loadTools(config.tools);
  }

  async execute(
    request: string,
    context: UserContext = {}
  ): Promise<UIResponse> {
    // Step 1: Route request to appropriate tool
    const routingDecision = await this.router.route(request, context);
    
    // Step 2: Get and execute selected tool
    const tool = this.tools.get(routingDecision.tool);
    if (!tool) {
      throw new Error(`Tool not found: ${routingDecision.tool}`);
    }

    // Step 3: Execute tool with routing parameters
    const toolResponse = await tool.execute({
      request,
      context,
      params: routingDecision.params
    });

    // Step 4: Package response
    return {
      content: [
        {
          type: "text",
          text: this.generateResponseText(routingDecision, toolResponse)
        },
        toolResponse.uiResource
      ],
      context: toolResponse.enrichedContext,
      metadata: {
        tool_used: routingDecision.tool,
        routing_confidence: routingDecision.confidence,
        routing_reasoning: routingDecision.reasoning
      }
    };
  }

  // MCP sampling method
  async sample(request: SamplingRequest): Promise<SamplingResponse> {
    // This would be implemented by the MCP client
    // For now, this is the interface the gateway expects
    throw new Error("sample() must be implemented by MCP client");
  }

  registerTool(id: string, tool: UIGenerationTool): void {
    this.tools.set(id, tool);
  }
}

Configuration and Setup

Gateway Configuration

supermodel.config.json
{
  "gateway": {
    "name": "SuperModel Gateway",
    "version": "1.0.0",
    "sampling": {
      "default_max_tokens": 1500,
      "default_temperature": 0.7,
      "timeout_ms": 30000
    }
  },
  "tools": {
    "product-search-ui": {
      "class": "ProductSearchTool",
      "description": "E-commerce product search with filters and shopping cart",
      "capabilities": ["search", "filter", "cart", "e-commerce"],
      "config": {
        "max_results": 50,
        "supported_filters": ["price", "brand", "rating", "category"]
      }
    },
    "bundle-builder-ui": {
      "class": "BundleBuilderTool",
      "description": "Create product bundles with complementary items",
      "capabilities": ["bundle", "recommendations", "pricing"],
      "config": {
        "max_bundle_size": 10,
        "discount_tiers": [0.05, 0.10, 0.15]
      }
    },
    "data-viz-ui": {
      "class": "DataVisualizationTool",
      "description": "Interactive charts and analytics dashboards",
      "capabilities": ["charts", "graphs", "analytics", "dashboard"],
      "config": {
        "chart_types": ["line", "bar", "pie", "scatter", "heatmap"],
        "max_data_points": 10000
      }
    }
  }
}

Tool Registration

// Gateway initialization
const gateway = new SuperModelGateway({
  configPath: './supermodel.config.json'
});

// Register tools
gateway.registerTool('product-search-ui', new ProductSearchTool(gateway));
gateway.registerTool('bundle-builder-ui', new BundleBuilderTool(gateway));
gateway.registerTool('data-viz-ui', new DataVisualizationTool(gateway));

// Start serving
gateway.listen(3000, () => {
  console.log('SuperModel Gateway running on port 3000');
});

Advanced Features

class WorkflowOrchestrator {
  async executeWorkflow(
    steps: WorkflowStep[],
    context: UserContext
  ): Promise<WorkflowResult> {
    let currentContext = context;
    const results = [];

    for (const step of steps) {
      // Each step uses sampling for tool selection
      const routingDecision = await this.router.route(
        step.request,
        currentContext
      );

      const result = await this.gateway.execute(
        step.request,
        currentContext
      );

      results.push(result);
      currentContext = result.context;
    }

    return { steps: results, finalContext: currentContext };
  }
}
class CapabilityMatcher {
  findBestMatch(
    request: string,
    requiredCapabilities: string[]
  ): ToolDefinition[] {
    return this.registry.getAll()
      .filter(tool => 
        requiredCapabilities.every(cap => 
          tool.capabilities.includes(cap)
        )
      )
      .sort((a, b) => 
        this.calculateCapabilityScore(b, requiredCapabilities) -
        this.calculateCapabilityScore(a, requiredCapabilities)
      );
  }
}
class ContextAwareRouter extends SamplingRouter {
  protected buildRoutingPrompt(
    request: string,
    context: UserContext,
    tools: ToolDefinition[]
  ): string {
    // Add context-specific routing logic
    const journeyStage = this.inferJourneyStage(context);
    const userPreferences = context.preferences;
    
    return super.buildRoutingPrompt(request, context, tools) + `

Journey Stage: ${journeyStage}
User demonstrates: ${Object.entries(userPreferences.values || {})
.filter(([_, value]) => value)
.map(([key]) => key)
.join(', ')}

Consider the user's journey progression and demonstrated values when routing.`;
  }
}

Performance Optimizations

Routing Cache

Cache common routing decisions to reduce sampling requests for repeated patterns.

Parallel Tool Loading

Load and prepare multiple tools in parallel for faster response times.

Context Compression

Compress context data to reduce payload size in sampling requests.

Tool Preloading

Preload likely next tools based on user journey patterns.

Monitoring and Analytics

class GatewayAnalytics {
  trackRouting(decision: RoutingDecision, context: UserContext): void {
    this.metrics.increment('routing.decisions', {
      tool: decision.tool,
      confidence: this.bucketConfidence(decision.confidence),
      journey_stage: this.inferJourneyStage(context)
    });
  }

  trackToolPerformance(tool: string, responseTime: number): void {
    this.metrics.histogram('tool.response_time', responseTime, {
      tool: tool
    });
  }
}

Next Steps