{"revision":"c10276e953456835cdbb1f3a60d58ae1980be353","document":{"id":23114,"path":"intelligent-experiences/create-custom-metric.md","title":"Create a custom metric for evaluating agentic workflows","bundle":null,"doc_type":null,"release":"australia","body":"\n# Create a custom metric for evaluating agentic workflows\n\nCreate a custom metric for evaluating AI agents and agentic workflows to test the outputs against expected responses.\n\n## Before you begin\n\nRole required: sn\\_aia.admin\n\n## About this task\n\nCustom metrics enable you to choose you measure your AI agents and agentic workflow for effectiveness. You can create custom metrics that apply to one agentic AI asset or multiple ones. Once created, you can select custom metrics in the guided setup for agentic evaluations. You can select both or either custom metrics and metrics installed with AI applications. Examples of custom metrics include:\n\n-   Testing that a certain phrase is in the response\n-   Measuring the length of a response to test verbosity\n\nWhen writing the script, you have access to the output of a parser tool. See [agentic evaluation parser tool](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/intelligent-experiences/eval-parser-tool.md) for more details on how it works.\n\n## Procedure\n\n1.  Navigate to **All** > **AI Skill Kit** > **Agentic Evaluations**.\n\n2.  Go to the Evaluation metrics tab and select **Create metric**.\n\n3.  In the **General information** step, enter a name and short description for your custom metric.\n\n    You can navigate between steps in the guided setup with the **Continue** and **Back** buttons. After entering a name for your custom metric, you can skip directly to writing your script by selecting **Skip to Script Editor**. You can fill out the details later from the Metric settings tab in the Script Editor.\n\n4.  In the **Metric details** step, add information about the evaluation metric, how it works, and what the output format of the metric is.\n\n    If you select **Use this content**, then the details you provide are displayed along with the option in the guided setup for an agentic evaluation run.\n\n    You do not have to fill in details, but you have the option to provide useful information and context to aid the custom metric development process. You can also add metric details from the **Metric settings** tab of the Script Editor.\n\n5.  After reviewing your metric information in the **Summary** step of the guided setup, select **Finish setup** to be redirected to the Script Editor.\n\n6.  Add metric inputs.\n\n    By default, the execution plan record sys\\_id is included as a metric input.\n\n    The parser tool lets you access content from the execution plan log without having to parse through the XML or JSON content yourself. You can access the parser tool's outputs with `tool.output`.\n\n    1.  In the Metric inputs section, select the plus icon to open the Add metric input modal.\n\n    2.  Complete the form.\n\n        |Field|Description|\n        |-----|-----------|\n        |Input datatype|Drop-down selector to choose what data type and format the input is|\n        |Name|Short, user-friendly name for the input|\n        |Description|Description of the input. You can view the description of any input by selecting it in the Metric inputs section of the Script Editor.|\n        |Test values|Default input values used for testing your script in the Script Editor|\n        |Mandatory|Determines whether providing the input is necessary for the evaluation to run|\n\n    3.  Select **Add**.\n\n7.  Write your script-based metric.\n\n    The default code block in the Script Editor contains commented code that can guide you through the script-writing process.\n\n    At any time, you can select **Metric output template** to see the code necessary for the output to display correctly when looking at the evaluation run results in Skill Kit. This content is also included in the default code block when you first open the Script Editor. If necessary, you can copy the template and paste it into the script. Be sure to change the values to the results of your custom metric.\n\n    You can review the details of the custom metric from the guided setup from the Metric settings tab.\n\n    When you have made changes, you can select the **Save** button to save your changes.\n\n    **Important:** Even if you save, the metric won’t be available to use in agentic evaluation runs until you publish it.\n\n8.  Select **Run test** to open the Run test modal, enter your metric test inputs, and run the test.\n\n    After you have run at least one test, you can review the response and request of both the overall execution and the parser tool.\n\n    After you have run at least one test, you can select the **Run test** drop-down button and select **Run test history** to open a new sidebar with past test results. You can filter test runs by the user who executed the test.\n\n9.  Select **Publish metric** to save your script and activate the metric.\n\n    Before publishing, ensure that your output matches the metric output template. If not, your metric may not work as expected or display correctly on the evaluation run results page.\n\n\n## Result\n\nYou have a new custom metric that can be used for evaluating agentic AI assets.\n\n## What to do next\n\nExecute an evaluation run and select your custom metric in the guided setup. See [Evaluate agentic AI](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/intelligent-experiences/execute-aia-eval.md) for more information on executing an evaluation run.\n\nIf you do not see your custom metric in the guided setup for an evaluation run, make sure that you have published it.\n\n"}}