Chapter 6: Evaluation

กระบวนการวัดผลและทดสอบระบบ AI อย่างเป็นระบบในไปป์ไลน์ MLOps/LLMOps

Evaluation Frameworks

LLM-as-a-Judge

• Using LLMs to evaluate LLM outputs
• Prompt design for evaluation
• Consistency and reliability

RAG Evaluation Metrics

• Retrieval accuracy
• Context relevance
• Answer quality
• Faithfulness scoring

Testing Strategies

Red Teaming

• Adversarial testing
• Safety evaluation
• Bias detection
• Edge case discovery

Performance Metrics

• Response time
• Token efficiency
• Cost per query
• Success rate

Production Monitoring

• Continuous evaluation pipelines
• Drift detection
• User feedback integration
• A/B testing frameworks

Building Evaluation CI/CD

• Automated testing in pipelines
• Gate criteria for deployment
• Regression prevention
• Performance baselines