Execution & Activity
Learn how to monitor, debug, and optimize tool execution in Prisme.ai agents using Activity tracking and event management
Understanding how tools execute and monitoring their performance is critical for developing reliable tool-using agents. Prisme.ai provides powerful Activity tracking capabilities that give you visibility into every aspect of tool execution, from request processing to result delivery.
The Value of Activity Tracking
Activity tracking in Prisme.ai provides:
Visibility
Complete transparency into how tools execute
Diagnostics
Powerful debugging for error resolution
Analytics
Performance metrics and usage patterns
Logging
Comprehensive audit trails of all operations
Monitoring
Real-time observation of system behavior
Optimization
Data-driven insights for performance improvements
Understanding the Event System
At the core of Prisme.ai’s Activity tracking is its event-driven architecture:
Key Components
Events
Structured messages representing actions, status changes, or notifications
Key attributes:
- Event type
- Timestamp
- Source identifier
- Payload data
- Request context
Publishers
Components that emit events when actions occur
Examples:
- Tool execution requests
- Automation processes
- User interactions
- System status changes
Subscribers
Components that listen for and process specific events
Examples:
- Automation workflows
- Activity monitors
- Logging systems
- Analytics processors
Event Bus
The central message broker that routes events
Responsibilities:
- Reliable message delivery
- Event persistence
- Publisher/subscriber coordination
- Load management
The Execution Lifecycle
Tool execution follows a well-defined lifecycle that generates events at each stage:
Request Initiation
The process begins when an agent decides to use a tool.
Events generated:
tool.request.initiated
tool.parameter.validation.started
Key information:
- Tool name and parameters
- Request source (conversation ID, user ID)
- Timestamp and request ID
- Context information
Parameter Validation
The system validates that all required parameters are present and properly formatted.
Events generated:
tool.parameter.validation.completed
tool.parameter.validation.failed
(if issues occur)
Key information:
- Validation status
- Error details (if any)
- Parameter transformations
- Schema conformance
Execution
The actual tool code runs, performing its designated function.
Events generated:
tool.execution.started
tool.execution.step.completed
(for multi-step tools)tool.execution.completed
Key information:
- Execution duration
- Resource utilization
- Intermediate state
- External system interactions
Result Processing
The system processes the tool’s output for agent consumption.
Events generated:
tool.result.processing.started
tool.result.processing.completed
Key information:
- Raw output data
- Processed results
- Formatting operations
- Output size and structure
Response Delivery
The formatted results are returned to the agent.
Events generated:
tool.response.delivery.started
tool.response.delivery.completed
Key information:
- Final response format
- Delivery time
- End-to-end latency
- Response size
Accessing Activity in Prisme.ai
You can monitor and analyze tool execution through several interfaces:
The primary interface for monitoring and debugging tool execution.
Key features:
- Chronological event listing
- Filtering by event type, status, or date range
- Detailed event inspection
- Real-time monitoring
- Search capabilities
Access:
- Navigate to the “Activity” tab in your AI Builder workspace
- Use filtering to focus on relevant events
- Click on events to see detailed information
- Toggle real-time updates for live monitoring
The primary interface for monitoring and debugging tool execution.
Key features:
- Chronological event listing
- Filtering by event type, status, or date range
- Detailed event inspection
- Real-time monitoring
- Search capabilities
Access:
- Navigate to the “Activity” tab in your AI Builder workspace
- Use filtering to focus on relevant events
- Click on events to see detailed information
- Toggle real-time updates for live monitoring
Programmatic access to event data for custom monitoring and analysis.
Key capabilities:
- Query historical events
- Filter by type, source, or timestamp
- Subscribe to real-time event streams
- Integrate with external monitoring tools
Example usage:
All activity is stored in your own OpenSearch/Elasticsearch. Build dashboards for performance monitoring and trend analysis.
Key metrics:
- Tool usage frequency
- Execution latency
- Error rates
- Resource utilization
- Success/failure trends
Features:
- Customizable dashboards
- Real-time updates
- Historical comparisons
- Threshold alerting
- Export capabilities
Common Event Types
Prisme.ai’s event system uses a hierarchical naming convention for clarity and filtering:
Debugging with Activity
Activity tracking provides powerful capabilities for diagnosing and resolving issues:
Identify the Issue
Use filtering to locate relevant events.
Effective filtering strategies:
- Search for specific request IDs
- Filter by error event types
- Focus on a specific time range
- Filter by tool name or source
- Search for specific error codes
Trace Execution Flow
Follow the sequence of events to understand the execution path.
Key analysis techniques:
- Map the complete event sequence
- Identify missing or unexpected events
- Analyze timing between events
- Examine state changes across events
- Look for execution branches
Examine Event Details
Inspect detailed event data for diagnostic information.
Focus areas:
- Parameter values and transformations
- Error details and context
- Performance metrics
- External system interactions
- Resource utilization
Compare with Successful Executions
Analyze differences between failed and successful runs.
Comparison strategies:
- Identify parameter differences
- Compare timing patterns
- Analyze resource usage
- Examine external factors
- Look for environmental changes
Implement and Verify Fixes
Make changes and confirm resolution through activity monitoring.
Verification approach:
- Test the same parameters
- Monitor in real-time
- Verify complete event sequence
- Check performance metrics
- Test edge cases
Monitoring Performance with Activity
Activity data is invaluable for performance optimization:
Optimizing Tool Execution
Use Activity data to drive execution optimizations:
Parameter Optimization
Refine parameter handling based on usage patterns
Strategies:
- Add intelligent defaults for common values
- Implement parameter validation improvements
- Optimize parameter extraction from user inputs
- Add preprocessing for frequently used parameter formats
Execution Efficiency
Improve tool execution performance
Strategies:
- Identify and optimize slow execution steps
- Implement caching for frequent operations
- Reduce external API latency
- Optimize resource utilization
Error Reduction
Minimize tool execution failures
Strategies:
- Add robust error handling for common failures
- Implement automatic retries for transient issues
- Improve validation to catch problems early
- Create fallback mechanisms for critical functions
Result Processing
Enhance result formatting and delivery
Strategies:
- Optimize result formatting for LLM consumption
- Reduce result size for better context utilization
- Implement progressive result delivery for long operations
- Add enrichment for more valuable outputs
Custom Event Emission
You can emit custom events to enhance visibility into your tools:
Real-Time Monitoring
For critical tools, implement real-time monitoring using Activity:
Set Up Real-Time Dashboards
Create specialized dashboards for key metrics.
Include dashboards for:
- Error rates and types
- Execution latency
- Usage volume
- Resource utilization
- Business-specific metrics
Configure Alerts
Define thresholds and conditions for notifications.
Alert types to consider:
- Error rate spikes
- Latency threshold violations
- Abnormal usage patterns
- Resource exhaustion warnings
- Custom business metric anomalies
Implement Event Subscribers
Create subscribers for real-time processing.
Example implementation:
Monitor Service Health
Track overall system performance and health.
Focus areas:
- API response times
- Service availability
- Resource utilization
- Queue lengths
- Error distributions
Activity for Compliance and Audit
Beyond operational uses, Activity provides essential compliance capabilities:
Audit Trails
Maintain comprehensive records of all tool interactions
Key aspects:
- Complete chronological record of events
- User and system actions
- Parameter and result tracking
- Timing information
- Access evidence
Compliance Reporting
Generate reports for regulatory requirements
Capabilities:
- Data access audit reports
- Processing evidence
- Security monitoring
- Policy enforcement verification
- Chain of custody documentation
Forensic Analysis
Detailed investigation of specific incidents
Use cases:
- Security incident investigation
- Error root cause analysis
- User behavior auditing
- System interaction reconstruction
- Performance issue diagnostics
Records Retention
Maintain activity data according to policies
Considerations:
- Retention period configuration
- Data archiving strategies
- Privacy compliance
- Data minimization
- Access controls
Best Practices for Activity Utilization
Activity in AI Knowledge
For AI Knowledge users, Activity information is accessible through built-in interfaces:
Access the Activity Log
Find the Activity section in AI Knowledge.
Key features:
- Chronological event listing
- Filtering by event type and date
- Search capabilities
- Drill-down for details
Monitor Tool Usage
Track how agents are using tools.
Available information:
- Tool usage frequency
- Success and error rates
- Performance metrics
- Parameter patterns
Analyze Conversation Flows
Examine how tools integrate into conversations.
Analysis capabilities:
- Tool usage in conversation context
- User reactions to tool results
- Multi-turn tool interactions
- Conversation paths and patterns
Export Activity Data
Extract data for external analysis.
Export options:
- CSV for spreadsheet analysis
- JSON for programmatic processing
- Filtered or complete data sets
- Custom date ranges
Next Steps
Ready to implement effective execution monitoring and activity tracking? Continue with these resources:
Was this page helpful?