AI World 3 · Agents and Tools · Stage 6

Read the Trace

Six tickets about a fleet of agents. Read the trace before you touch the model, then measure the agents you fixed.

AgentCore Observability
Traces and spans
AgentCore Evaluations

Domain: AI and machine learning
World: AI World 3: Agents and Tools (Associate)
Game type: Evidence desk: Each ticket is the reporter's words: find its cause in the evidence.
Time: About 30 minutes on a first attempt (estimated)
Exam guides: MLA-C02Skill 4.1.1: Monitor model performance in production by using Amazon CloudWatch generative AI observability, Amazon Bedrock Model Evaluation, and drift detection pipelines; Skill 4.1.5: Monitor and automate the management of agent performance and coordination (for example, coordination failure detection, truncated streaming, tool failures); Skill 4.2.2: Configure and use tools to troubleshoot and analyze resources (for example, Amazon CloudWatch, Amazon Bedrock AgentCore Observability, AWS X-Ray)
AIP-C01Skill 3.4.1: Develop transparent AI systems in FM outputs (for example, by using reasoning displays to provide user-facing explanations, CloudWatch to collect confidence metrics and quantify uncertainty, evidence presentation for source attribution); Skill 4.3.4: Create tool performance frameworks to ensure optimal tool operation and utilization for FMs (for example, by using call pattern tracking, performance metric collection, tool calling observability and multi-agent coordination tracking, usage baselines for anomaly detection); Skill 5.1.7: Develop agent performance frameworks to ensure that agents perform tasks correctly and efficiently (for example, by using task completion rate measurements, tool usage effectiveness evaluations, Amazon Bedrock Agent evaluations, reasoning quality assessment in multi-step workflows)