Human-in-the-Loop
Human-in-the-Loop (HITL) enables agents to pause execution and request human approval for high-stakes decisions.
Why HITL Matters
Fully autonomous AI can:
- Make costly mistakes at scale
- Act on incomplete information
- Miss context that humans would catch
- Create compliance and liability issues
HITL provides the right balance: automation for routine tasks, human oversight for important decisions.
How It Works
┌─────────────────────────────────────────────────────────────────────┐
│ Agent Execution Flow │
│ │
│ ┌──────────┐ │
│ │ Start │ │
│ └────┬─────┘ │
│ │ │
│ ▼ │
│ ┌──────────┐ Low Risk ┌──────────┐ │
│ │ TDAO │─────────────────▶│ Execute │──┐ │
│ │ Loop │ └──────────┘ │ │
│ └────┬─────┘ │ │
│ │ │ │
│ │ High Risk │ │
│ ▼ │ │
│ ┌──────────────────────────────┐ │ │
│ │ DECISION POINT CREATED │ │ │
│ │ │ │ │
│ │ • Pause execution │ │ │
│ │ • Notify approvers │ │ │
│ │ • Wait for response │ │ │
│ └────────────┬─────────────────┘ │ │
│ │ │ │
│ ┌───────┼───────┐ │ │
│ ▼ ▼ ▼ │ │
│ Approve Deny Escalate │ │
│ │ │ │ │ │
│ ▼ ▼ ▼ │ │
│ Execute Abort Notify │ │
│ │ │ Manager │ │
│ │ │ │ │ │
│ └───────┴───────┴───────────────────────┘ │
│ │ │
│ ▼ │
│ ┌──────────┐ │
│ │ Continue │ │
│ │ TDAO │ │
│ └──────────┘ │
│ │
└─────────────────────────────────────────────────────────────────────┘Decision Point Types
Approval Requests
Binary yes/no decisions for specific actions:
┌─────────────────────────────────────────────────────────────────────┐
│ 🛑 APPROVAL REQUIRED │
│ │
│ Action: Revoke AWS Production admin access │
│ Target: marcus.chen@company.com │
│ │
│ Risk Level: HIGH (85/100) │
│ │
│ Risk Factors: │
│ • Production system access │
│ • Competitor departure │
│ • Admin-level privileges │
│ │
│ Impact: │
│ • 3 production databases │
│ • 12 EC2 instances │
│ • S3 buckets with customer data │
│ │
│ ───────────────────────────────────────────────────────────────── │
│ │
│ [✓ Approve] [✗ Deny] [⚡ Escalate] │
│ │
└─────────────────────────────────────────────────────────────────────┘Choice Selections
Select from multiple options:
┌─────────────────────────────────────────────────────────────────────┐
│ 📋 CHOICE REQUIRED │
│ │
│ Question: How should we handle the customer's open support tickets? │
│ │
│ Context: Marcus has 7 open tickets assigned to him. │
│ │
│ Options: │
│ ○ Reassign all to Sarah (backup support lead) │
│ ○ Reassign based on ticket category │
│ ○ Keep assigned and notify customers of transition │
│ ○ Close with automated response │
│ │
│ ───────────────────────────────────────────────────────────────── │
│ │
│ [Confirm Selection] │
│ │
└─────────────────────────────────────────────────────────────────────┘Information Requests
Gather additional information:
┌─────────────────────────────────────────────────────────────────────┐
│ ❓ INFORMATION NEEDED │
│ │
│ Question: Who should receive ownership of Marcus's documents? │
│ │
│ Context: Marcus owns 23 Google Docs including: │
│ • Architecture diagrams (8) │
│ • Process documentation (12) │
│ • Customer-facing materials (3) │
│ │
│ Suggested: Sarah Chen (most collaboration history) │
│ │
│ New Owner: [_______________________] │
│ │
│ ───────────────────────────────────────────────────────────────── │
│ │
│ [Submit] [Use Suggestion] │
│ │
└─────────────────────────────────────────────────────────────────────┘Configuring HITL
Agent Settings
interactivity:
mode: key_decisions
# Risk threshold for requiring approval
risk_threshold: 50
# Decision timeout settings
decision_timeout_seconds: 14400 # 4 hours
timeout_action: escalate
# Specific actions that always require approval
always_require_approval:
- delete_*
- revoke_admin_*
- transfer_ownership
# Actions that never require approval
never_require_approval:
- read_*
- list_*Notification Settings
notifications:
channels:
- type: slack
webhook: "https://hooks.slack.com/..."
channel: "#approvals"
- type: email
recipients:
- "managers@company.com"
on_decision_required: true
on_timeout_warning: true # 30 min before timeout
on_escalation: trueApproval Workflow
1. Decision Point Created
When an agent encounters a high-risk action:
- Execution pauses
- Decision point record created
- Notifications sent
2. Approver Reviews
Approvers can:
- View full context
- See agent reasoning
- Check risk factors
- Review impact
3. Response Recorded
All responses are traced:
{
"decision_point_id": "dp_123",
"response": "approved",
"responded_by": "john@company.com",
"responded_at": "2026-01-15T14:35:00Z",
"notes": "Verified with IT Security team"
}4. Execution Resumes
Agent continues with the approved (or alternative) action.
Timeout Handling
If no response within the timeout period:
| Action | Behavior |
|---|---|
| escalate | Notify manager, extend timeout |
| continue | Proceed with default/safe action |
| fail | Abort the run |
| skip | Skip the action, continue workflow |
Escalation Chain
escalation:
levels:
- timeout: 4h
notify: direct_manager
- timeout: 8h
notify: department_head
- timeout: 24h
action: fail
notify: adminAPI Integration
Creating Decision Points
Agents create decision points during DECIDE phase:
mutation CreateDecisionPoint($input: CreateDecisionPointInput!) {
createDecisionPoint(input: {
runId: "run_123"
type: APPROVAL
title: "Approve access revocation"
context: { ... }
riskLevel: HIGH
riskScore: 85
timeoutSeconds: 14400
}) {
id
status
}
}Resolving Decision Points
mutation ResolveDecisionPoint($id: ID!, $input: ResolveDecisionPointInput!) {
resolveDecisionPoint(id: $id, input: {
response: APPROVED
notes: "Verified with security team"
}) {
id
status
resolvedAt
}
}Subscribing to Decisions
Real-time updates:
subscription OnDecisionRequired($orgId: ID!) {
decisionRequired(orgId: $orgId) {
id
runId
agentName
title
riskLevel
createdAt
}
}Best Practices
Design Clear Decision Points
- Provide sufficient context for informed decisions
- Explain the risk factors clearly
- Show the impact of each option
- Include agent reasoning
Set Appropriate Thresholds
- Too low: Decision fatigue, delays
- Too high: Risky actions go unchecked
- Start conservative, adjust based on data
Train Approvers
- Ensure approvers understand the context
- Document approval criteria
- Review approval patterns regularly
Monitor Approval Metrics
Track:
- Average response time
- Approval vs denial rate
- Timeout frequency
- Escalation frequency