UI Affordance Registration Framework

A flexible system for dynamically registering and controlling UI components through client-side tools. This framework allows the assistant to create rich, interactive UI experiences by registering components in dedicated containers and interacting with them through a standardized API.

The Affordance Framework provides:

• Dynamic Component Registration: Load and mount components from file keys
• Multiple Container Types: Overlay, header, footer, sidebar, and inline containers
• Standardized Interface: Consistent API for component interaction
• Lifecycle Management: Proper mounting, unmounting, and cleanup
• Method Invocation: Safe method calls on registered components

1. AffordanceManager (/frontend/utils/affordanceManager.ts)

   • Central registry for all affordances
   • Handles component lifecycle
   • Manages container creation and destruction

2. Container Managers (/frontend/utils/containers/)

   • OverlayContainerManager - Modal/popup overlays
   • SidebarContainerManager - Collapsible side panels
   • HeaderContainerManager - Fixed header extensions
   • FooterContainerManager - Footer extensions
   • InlineContainerManager - Chat-embedded components

3. Client Tools (integrated in /frontend/utils/clientTools.ts)

   • register_affordance - Register new components
   • call_affordance_method - Invoke component methods
   • unregister_affordance - Remove components
   • list_affordances - List active components

Modal/popup containers with backdrop support.

Features:

• Modal backdrop (closable)
• Flexible positioning (center, top, bottom, left, right)
• Custom sizing
• Close button
• Animation support

Use Cases:

• Forms and dialogs
• Image viewers
• Settings panels
• Confirmation dialogs

Collapsible side panels with toggle functionality.

Features:

• Left/right positioning
• Collapsible with toggle button
• Custom width
• Header with title and controls
• Persistent state

Use Cases:

• Navigation menus
• Tool palettes
• Status dashboards
• File browsers

Fixed header extensions that integrate with existing layout.

Features:

• Priority-based ordering
• Left/right positioning
• Automatic layout adjustment
• Close functionality

Use Cases:

• Status indicators
• Quick actions
• Breadcrumbs
• Search bars

Footer extensions that work alongside existing controls.

Features:

• Priority-based ordering
• Integration with existing footer
• Compact design
• Close functionality

Use Cases:

• Status indicators
• Progress bars
• Quick stats
• Action buttons

Components embedded directly in the chat flow.

Features:

• Automatic insertion point detection
• Flexible sizing
• Animation support
• Close functionality

Use Cases:

• Interactive widgets
• Data visualizations
• Embedded forms
• Rich content displays

All affordance components must implement the AffordanceComponent interface:

interface AffordanceComponent {
  mount(container: HTMLElement, config: AffordanceConfig): Promise<void>;
  unmount(): Promise<void>;
  getPublicMethods(): Record<string, Function>;
  [customMethod: string]: any;
}

• mount(container, config): Initialize and render the component
• unmount(): Clean up resources and remove from DOM
• getPublicMethods(): Return object with callable methods

interface AffordanceConfig {
  // Common
  title?: string;
  className?: string;
  zIndex?: number;
  persistent?: boolean;
  
  // Positioning
  position?: 'left' | 'right' | 'top' | 'bottom' | 'center';
  width?: string;
  height?: string;
  maxWidth?: string;
  maxHeight?: string;
  
  // Overlay specific
  modal?: boolean;
  backdrop?: boolean;
  closable?: boolean;
  
  // Header/Footer specific
  priority?: number;
  
  // Sidebar specific
  collapsible?: boolean;
  defaultCollapsed?: boolean;
  
  // Custom properties
  [key: string]: any;
}

// Register counter in sidebar
const counterId = await register_affordance('sidebar', 
  '/frontend/components/affordances/CounterAffordance.tsx', 
  {
    title: 'Counter Widget',
    position: 'right',
    width: '250px',
    initialValue: 10,
    step: 5
  }
);

// Interact with counter
await call_affordance_method(counterId, 'increment', []);
const value = await call_affordance_method(counterId, 'getValue', []);

// Register status dashboard
const dashboardId = await register_affordance('sidebar',
  '/frontend/components/affordances/StatusDashboard.tsx',
  {
    title: 'System Status',
    position: 'left',
    width: '300px',
    refreshInterval: 5000
  }
);

// Add status items
await call_affordance_method(dashboardId, 'addItem', [{
  id: 'cpu',
  label: 'CPU Usage',
  value: '45%',
  status: 'success'
}]);

await call_affordance_method(dashboardId, 'addItem', [{
  id: 'memory',
  label: 'Memory',
  value: '2.1GB',
  status: 'warning'
}]);

// Register notification center
const notificationId = await register_affordance('overlay',
  '/frontend/components/affordances/NotificationCenter.tsx',
  {
    title: 'Notifications',
    position: 'top-right',
    maxWidth: '400px',
    maxNotifications: 10
  }
);

// Add notifications
await call_affordance_method(notificationId, 'addNotification', [{
  title: 'Task Complete',
  message: 'Your data processing task has finished successfully.',
  type: 'success',
  autoClose: true,
  duration: 3000
}]);

// Register inline chart
const chartId = await register_affordance('inline',
  '/frontend/components/affordances/ChartWidget.tsx',
  {
    title: 'Sales Data',
    width: '100%',
    height: '300px',
    chartType: 'line'
  }
);

// Update chart data
await call_affordance_method(chartId, 'updateData', [
  [
    { x: 'Jan', y: 100 },
    { x: 'Feb', y: 150 },
    { x: 'Mar', y: 120 }
  ]
]);

/** @jsxImportSource https://esm.sh/react@18.2.0 */
import React from "https://esm.sh/react@18.2.0";
import { createRoot } from "https://esm.sh/react-dom@18.2.0/client";
import { AffordanceComponent, AffordanceConfig } from "../../../shared/affordance-types.ts";

// Your React component
const MyComponent: React.FC<Props> = (props) => {
  return <div>My Custom Component</div>;
};

// Affordance wrapper class
export default class MyAffordance implements AffordanceComponent {
  private root: any = null;
  private container: HTMLElement | null = null;

  async mount(container: HTMLElement, config: AffordanceConfig): Promise<void> {
    this.container = container;
    this.root = createRoot(container);
    this.root.render(<MyComponent {...config} />);
  }

  async unmount(): Promise<void> {
    if (this.root) {
      this.root.unmount();
      this.root = null;
    }
    this.container = null;
  }

  getPublicMethods(): Record<string, Function> {
    return {
      myMethod: this.myMethod.bind(this)
    };
  }

  // Custom methods
  myMethod(arg: any): any {
    // Implementation
    return result;
  }
}

1. Error Handling: Always wrap async operations in try-catch
2. Resource Cleanup: Properly clean up timers, listeners, and subscriptions in unmount()
3. State Management: Use React state for UI state, class properties for persistent data
4. Method Binding: Bind methods in getPublicMethods() to maintain this context
5. Configuration: Use the config parameter for initial setup and customization
6. Responsive Design: Consider different container sizes and positions

The framework includes several example components:

1. CounterAffordance (/frontend/components/affordances/CounterAffordance.tsx)

   • Simple counter with increment/decrement
   • Demonstrates basic state management and method exposure

2. StatusDashboard (/frontend/components/affordances/StatusDashboard.tsx)

   • System status monitoring
   • Shows complex state management and refresh functionality

3. NotificationCenter (/frontend/components/affordances/NotificationCenter.tsx)

   • Notification management system
   • Demonstrates auto-close timers and list management

The framework integrates seamlessly with the existing chat application:

• Client Tools: New affordance tools are added to the existing client tools system
• Styling: Uses existing CSS custom properties and design tokens
• Layout: Containers respect existing layout and don't interfere with chat functionality
• Performance: Lazy loading and efficient lifecycle management

All affordance operations are logged to the console with the [AffordanceManager] prefix.

• Component loading errors are caught and reported
• Method call errors return structured error responses
• Container creation failures are handled gracefully

• Use list_affordances to see all registered components
• Check browser console for detailed logging
• Use React DevTools for component debugging

Potential future improvements:

• State Persistence: Save/restore component state across sessions
• Component Communication: Message passing between affordances
• Theme Integration: Automatic theme switching support
• Performance Monitoring: Component performance metrics
• Hot Reloading: Development-time component reloading
• Component Library: Curated library of common components

• Components are loaded from trusted file sources only
• Method calls are validated before execution
• DOM manipulation is contained within assigned containers
• No direct access to sensitive application state