Skip to main content
When AI and human judgments disagree, Artanis uses AI to classify the root cause into error types. This helps identify systemic issues in your AI system or processes.

​
Error Type Reference

Error TypeDisplay NameCategoryDescription
ambiguous_instructionsAmbiguous instructionsUser ErrorThe instructions or guidelines are unclear or open to interpretation
inconsistent_instructionsInconsistent instructionsUser ErrorThe instructions contradict themselves or vary across contexts
label_contradicts_instructionsLabel contradicts instructionsHuman ErrorThe human label doesn’t follow the stated guidelines
labelling_errorLabelling errorHuman ErrorThe human reviewer made a mistake
document_retrieval_failureDocument retrieval failureAI ErrorThe AI retrieved incorrect or irrelevant information

​
Categories

​
User Error

Issues stemming from unclear or inconsistent documentation, guidelines, or instructions. These require updating your knowledge base or guidelines.

​
Human Error

Mistakes made by human reviewers during the labeling process. These may indicate a need for reviewer training or guideline clarification.

​
AI Error

Failures in the AI system itself, such as retrieval or generation issues. These require technical investigation and system improvements.

​
Using Error Types

Error types are returned by the Attribution endpoint after analyzing a trace with feedback: 
{
  "error_type": "ambiguous_instructions",
  "error_type_name": "Ambiguous instructions",
  "error_type_category": "User Error",
  "reasoning": "The guidelines don't specify whether..."
}
Use these to:
  • Track patterns in disagreements over time
  • Prioritize improvements (fix common error types first)
  • Distinguish between process issues vs. AI issues