{"revision":"c10276e953456835cdbb1f3a60d58ae1980be353","document":{"id":36385,"path":"now-intelligence/usage-insights/consuming-data-export-results.md","title":"Consuming data export results","bundle":null,"doc_type":null,"release":"australia","body":"\n# Consuming data export results\n\nConsume the results of usage insights data exported via REST API, from the Kafka topic.\n\n## Before you begin\n\nRole required: admin\n\n## Procedure\n\n1.  Get the topic name from the 202 response.\n\n2.  Configure two Kafka consumer processes \\(one per Hermes cluster\\) with the bootstrap servers, the topic name, and your SSL keystore and truststore.\n\n3.  Use the same Consumer Group ID for both processes.\n\n    This confirms that messages are distributed across the two processes without duplication.\n\n4.  Subscribe to the topic and start consuming.\n\n    Results arrive as JSON batches. Each batch includes the `job_id`, `batch_num`, `total` \\(total batches\\), column names, and row data.\n\n    |Field|Description|\n    |-----|-----------|\n    |job\\_id|Matches the job\\_id returned when you submitted the request.|\n    |batch\\_num|The number of this batch.|\n    |total\\_batches|The total number of batches for the job. When batch\\_num equals total, all batches have been produced.|\n    |columns|The column names, in the same order as the values in each row.|\n    |rows|An array of rows. Each row is an array of values matching the column order.|\n\n5.  Use the `hermes_topic_endpoint` value from your data export API response as the topic name.\n\n    For example:\n\n    ```\n    \n    snc.<instance>.uxa.sn_uxa_data_export.data_export_results\n              \n    ```\n\n    ```\n    \n    snc.<instance>.uxa.sn_uxa_data_export.data_export_results\n              \n    ```\n\n    Start both consumer processes, and they will begin receiving batches as the export job produces results.\n\n\n## Result\n\nYour Kafka consumer environment is now securely connected to the managed Hermes cluster. You can submit data export requests and consume results from the Kafka topic using your configured consumers. Use a dedicated integration user for automated exports and grant it only the `sn_uxa_data_export.user` role and de-duplicate using the Kafka offset.\n\n**Note:** Schedule recurring exports \\(e.g., daily\\) so each run stays within the 60-request-per-hour and 100 GB-per-month limits.\n\n**Tip:** Follow these tips, as batch delivery order is not guaranteed and duplicate batches may occasionally be delivered:\n\n-   Reassemble: Sort batches by `batch_num` to reconstruct the full result set.\n-   Track completion: When `batch_num` equals `total`, all batches for the job have been produced.\n-   Consume within retention window: Results are retained on the topic for 36 hours. Consume all batches before they expire.\n-   Each instance has its own dedicated Kafka topic for result delivery. The SSL certificate used to access the topic is scoped to your instance's topics only. Exports are bound to your customer account: data from one account is never accessible from another account's instance.\n\n**Parent Topic:**[Bulk export of Usage Insights data via REST API](https://raw.githubusercontent.com/ServiceNow/ServiceNowDocs/australia/markdown/australia/now-intelligence/usage-insights/data-export-restapi.md)\n\n"}}