{"revision":"c10276e953456835cdbb1f3a60d58ae1980be353","document":{"id":23294,"path":"intelligent-experiences/eval-parser-tool.md","title":"Agentic evaluation parser tool","bundle":null,"doc_type":null,"release":"australia","body":"\n# Agentic evaluation parser tool\n\nUse the outputs of the agentic evaluation parser tool in your scripts for custom metrics to customize the criteria for effective AI agents and agentic workflows.\n\n## Parser tool overview\n\nThe agentic evaluation parser tool extracts structured execution data from the execution logs of an agentic workflow or AI agent. You can use the information gathered by the tool to create custom metrics that use scripts to evaluate agentic workflows.\n\nThe parser tool returns structured AI agent or agentic workflow execution data within the **output.payload** object, which contains the following:\n\n-   **executionInputs**: A JSON object that contains initial workflow setup information, such as the names and instructions of agents and tools and the initial user utterance\n-   **executionOutputs**: A JSON object with AI agent actions and tool execution results\n-   **executionMessages**: A JSON object array of user-facing conversation flow and system responses\n-   **executionPlanDetails**: A JSON object of execution metadata, such as status, sys\\_ids, and configuration values\n\n```\n{\n     \"output\": {\n \"payload\": {\n  \"executionInputs\": { ... },\n  \"executionOutputs\": { ... },\n  \"executionMessages\": [ ... ],\n  \"executionPlanDetails\": { ... }\n }\n}\n\n```\n\n## Accessing the parser tool's output\n\nTo view the complete parser tool output for testing and development, perform the following steps.\n\n-   Navigate to the Script Editor view in your custom metric guided setup.\n-   Select **Run Test**.\n-   Wait for the test to complete.\n-   View the complete JSON output in the Tools section of the test results.\n\nReviewing the parser tool's output before designing your custom metric enables you to inspect the data structure before implementing any specific logic.\n\n## executionInputs data structure\n\nThe executionInputs attribute contains a JSON object with the following structure:\n\n```\n\"executionInputs\": {\n \"agenticWorkflow\": \"(name of agentic workflow)\",\n \"description\": \"(descriptions for agentic workflow)\",\n \"instructions\": \"(list of steps for agentic workflow)\", \n \"utterance\": \"(initial user utterance)\", \n \"agents\": [ \n  { \n   \"name\": \"(AI agent name)\",  \n   \"instructions\": \"(list of steps for AI agent)\", \n   \"tools\": [ \n    { \n     \"name\": \"(tool name)\", \n     \"description\": \"(tool description)\", \n     \"executionMode\": \"(execution mode, either Autonomous or Supervised)\", \n     \"inputs\": { ... } \n    }, \n    { ... }, ... \n   ]\n  }, \n  { ... }, ... \n ]\n}\n```\n\n## executionOutputs data structure\n\nThe executionOutputs attribute contains a JSON object with the following structure:\n\n```\n\"executionOutputs\": {\n \"agents\": [ \n  { \n   \"name\": \"(AI agent name)\", \n   \"subTask\": { ... }, \n   \"tools\": [ \n    { \n     \"name\": \"(tool name)\", \n     \"inputs\": { ... }, \n     \"output\": { ... }\n    }\n   ]\n  }, \n  { ... }, ... \n ]\n}\n```\n\n## executionMessages data structure\n\nThe executionOutputs attribute contains an array of JSON objects with the following structure:\n\n```\n\"executionMessages\": [ \n { \n  \"role\": \"(Message sender, either 'agent' or 'user')\", \n   \"message\": \"(Content of message)\", \n   \"order\": \"(Sequence number indicating order of message in the conversation)\"\n  }, \n  { ... }, ... \n]\n   \n```\n\n## executionPlanDetails data structure\n\nThe executionPlanDetails attribute contains a JSON object with the following structure:\n\n```\n\"executionPlanDetails\": { \n    \"state\": \"(Current execution status)\", \n    \"runType\": \"(Type of execution)\", \n    \"conversationId\": \"(sys_id of conversation)\", \n    \"relatedTask\": \"(sys_id of the associated task or record)\", \n    \"relatedTaskTable\": \"(Table name where the related task is stored)\", \n    \"context\": { ... } (May be null)\n    \"builtInTools\": [ { ... } ]\n    }\n   \n```\n\nThis section provides execution metadata for tracking workflow performance, debugging issues, and correlating executions with specific tasks or conversations.\n\nThe possibilities for runType include the following:\n\n-   API\n-   Chat\n-   Evaluation\n-   Testing\n-   Trigger\n\n## Using parser tool output in metric scripts\n\nThe parser tool data is available in your metric script through the **context** parameter. Access the structured data using the following code:\n\n```\n// Access the parser tool output from context\nvar parserToolOutput = context['AgenticExecutionParserTool.output'];\nif (typeof parserToolOutput == \"string\") {\n parserToolOutput = JSON.parse(parserToolOutput);\n}\nvar parserToolPayload = parserToolOutput.payload;\nvar parserToolStatus = parserToolOutput.status;\n    \n// Extract individual sections from payload\nvar inputs = parserToolPayload.executionInputs;\nvar outputs = parserToolPayload.executionOutputs;\nvar messages = parserToolPayload.executionMessages;\nvar planDetails = parserToolPayload.executionPlanDetails;\n   \n```\n\n"}}