| \n\nScheduled job\n\n | \n\nDescription\n\n |
|---|---|
| \n\nCE Populate Value Aggregates Chats – Daily\n\n | \n\nThis scheduled script runs daily and randomly selects 1000 conversations from yesterday's conversations. After that, for each conversation, this job extracts the chat duration and classifies them as small, medium, or large. It also classifies the chats in which a Knowledge article or catalog item was invoked. For the evaluated chat, it also classifies the conversations based on the chat performance and populates that data into the Evaluation Value Aggregates table.\n\n |
| \n\nEvaluation Value Calcuation - Runs Only once after install\n\n | \n\nDeletes all the records on the Evaluation Value Aggregates tables, runs the calculations again and stores the aggregated value in the Evaluation Value Aggregates table. The data is from the first evaluation date.\n\n |
| \n\nLabel\n\n | \n\nName\n\n |
|---|---|
| \n\nEvaluation\n\n | \n\n\\[sn\\_na\\_conv\\_eval\\_evaluation\\]\n\n |
| \n\nEvaluation configurations\n\n | \n\n\\[sn\\_na\\_conv\\_eval\\_evaluation\\_configurations\\]\n\n |
| \n\nEvaluation Metrics\n\n | \n\n\\[sn\\_na\\_conv\\_eval\\_evaluation\\_metrics\\]\n\n |
| \n\nEvaluation Set\n\n | \n\n\\[sn\\_na\\_conv\\_eval\\_evaluation\\_set\\]\n\n |
| \n\nEvaluation Value Aggregates\n\n | \n\n\\[sn\\_na\\_conv\\_eval\\_evaluation\\_value\\_aggregates\\]\n\n |
| \n\nTable\n\n | \n\nDescription\n\n |
|---|---|
| \n\nConversation Evaluator Value Calculations\n\n \\[sn\\_na\\_conv\\_eval\\_st\\_value\\_calcs\\]\n\n | \n\nFor the given query, the definition for this remote table calculates the time savings and efficiency percentage for small, medium, and large chats. Also, it returns the time savings and efficiency when a Knowledge article or catalog item was invoked.\n\n |
| \n\nConversation weekly calculations\\[sn\\_na\\_conv\\_eval\\_weekly\\_cals\\]\n\n | \n\nFor the given query, the definition for this remote table calculates the time savings and efficiency percentage for small, medium, and large chats for different weeks of the selected date range. Also, it returns the time savings and efficiency when aKnowledge article or catalog item was invoked for all the different weeks of the selected date range.\n\n |
| \n\nProperty\n\n | \n\nDescription\n\n |
|---|---|
| \n\nsn\\_na\\_conv\\_eval.errorBandMinRecords\n\n | \n\nMinimum number of records required to calculate the error band for upper and lower deviation. By default, the value is 30.\n\n |
| \n\nsn\\_na\\_conv\\_eval.evalWeights\n\n | \n\nContains weights to each evaluation metric for chat evaluation. This property is used to compute total or composite scores for evaluation records.\n\n |
| \n\nsn\\_na\\_conv\\_eval.maxEvaluateCount\n\n | \n\nMaximum number of records to evaluate in a day. By default, the value is 200.\n\n |
| \n\nsn\\_na\\_conv\\_eval.total\\_sampled\\_conv\\_count\n\n | \n\nEdit this property to control the total number of conversations that can be sampled for value calculations. By default, the value is 1000.\n\n |
| \n\nsn\\_na\\_conv\\_eval.value\\_chat\\_classifier\n\n | \n\nEdit this property to change the definition of small, medium, and large conversations. By default, the value it stores is 4, 10.\n\n Here, 4 and 10 signify the total number of inbound messages. Fewer than or equal to 4 inbound messages in the sys\\_cs\\_message table for a conversation means that it’s a small conversation. More than 4 inbound messages and fewer than or equal to 10 inbound messages means that it’s a medium conversation, and more than 10 inbound messages means that it’s a large conversation.\n\n |
| \n\nsn\\_na\\_conv\\_eval.ce\\_value\\_calculation\\_weights\n\n | \n\nValue calculation weight values for each type of evaluated chat.\n\n |
| \n\nsn\\_na\\_conv\\_eval.eval\\_value\\_rerun\\_status\n\n | \n\nReruns the value calculations once after the installation. This property will check the status of the Conversation Evaluator Value Rerun status. If it has run, then the script will change the value of this system property to false.\n\n |
| \n\nFlow\n\n | \n\nDescription\n\n |
|---|---|
| \n\nExecute Evaluation\n\n | \n\nPerforms evaluations when conversations are completed.\n\n By default, the Execute Evaluation flow is deactivated. You can use the nightly scheduled job Execute Evaluations to evaluate the chats. If you want to evaluate the chats on chat completion, activate the Execute Evaluation flow.\n\n |
| \n\nExecute Batch Evaluation\n\n | \n\nPerforms batch evaluations, evaluating up to 100 completed virtual agent conversations. Flow is triggered when the Evaluation set is created or updated and the Evaluation Type is Conversation.\n\n |
| \n\nFlow action\n\n | \n\nDescription\n\n |
|---|---|
| \n\nRandomize conversations\n\n | \n\nPerforms randomization of conversations and returns 100 conversations randomly from a given query.\n\n |
| \n\ninvokeApiDefinition\n\n | \n\nInvokes OneExtend Capability in the large language model \\(LLM\\).\n\n |
| \n\nChat Classifier Eval\n\n | \n\nGives the title, category, and whether the evaluation should be executed.\n\n |
| \n\nbuildTranscript\n\n | \n\nBuilds the transcript from a conversation.\n\n |
| \n\nevalExecuteCondition\n\n | \n\nChecks if the transcript is good enough to be evaluated.\n\n |
| \n\nScript includes\n\n | \n\nDescription\n\n |
|---|---|
| \n\nevalExecuteCondition\n\n | \n\nUse this script include to update the evaluation condition.\n\n |
| \n\nevalUtils\n\n | \n\nPrimary Utility function for the Evaluator.\n\n |