The Customer Events Stream to EventHubs process is part of the eShare Analyze data pipeline. Its main goal is to combine audit event data from multiple systems, clean and standardize it, and send it to an EventHubs for the Customer’s consumption. Once the Audit events are sent to the EventHub, the Customer can create a connection using the standard Azure EventHub connector in the SIEM of their choosing (Splunk, Sentinel, Exabeam, etc.). At a high level, the steps involved are:
EventHub Stream Explained
This flow continuously listens to several event hubs that contain customer activity data — for example, file shares, user actions, or policy events — and merges them into a single event stream.
Here’s what happens step-by-step:
Read data from several input topics (streams)
Web portal events
Secure mail gateway events
Govern for outlook and sharepoint events
(Optional) Microsoft 365 DLP-enriched events
Filter and standardize event fields so all event data follows a single consistent schema.
Optionally enrich events with DLP (Data Loss Prevention) metadata when enabled.
Combine all event sources into one unified stream.
Write the unified event stream to a Kafka topic connected to Azure EventHubs, where customers can consume it in real time.
Step 1: Input Streams
The job reads from up to three input data sources:
Input Source | Required | Refresh Period |
Collaborate Web Events | ✅ | 12 hours |
Secure Collaboration Gateway Events | ✅ | Live |
Govern for Outlook and SharePoint events | ✅ | Live |
M365 DLP Events (Purview) | Optional | Live |
Step 2: Transformation and Combination
The ETL job then processes and merges these inputs:
Removes internal system metadata
Filters data by customer organization key
Renames columns to a standard naming format
Combines events from all sources into one consistent DataFrame
Optionally includes M365 DLP enrichment if enabled
This ensures that no matter the source, every event looks and behaves the same downstream.
Step 3: Output Stream
After transformation, the job writes to a destination Azure EventHub.
Data Schema
Dimension | Field | Description |
Event | event_id | A unique identifier assigned to the event |
event_type_name | The type of the audit event, which indicates the action performed | |
event_server_timestamp | The timestamp when the event was logged by the server | |
event_timestamp | The timestamp when the event was logged by the client | |
event_category | The category of the event (e.g. Sharing Event, Admin Event) | |
event_duration_millis | The duration of the event in milliseconds | |
event_source_app_id | The source system that generated this event | |
Actor | actor_user_email | The email of the user who performed the action |
Location | location_continent | The continent where the event took place |
location_country | The country where the event occurred | |
location_country_code | The country code where the event occurred | |
location_region | The region or state where the event occurred | |
location_city | The city where the event occurred | |
location_longitude | The longitude coordinate of the event’s location | |
location_latitude | The latitude coordinate of the event’s location | |
IP address | ip_ip_address | The IP address from which the event originated |
ip_is_private | Indicates where the IP address is a private address | |
ip_registry | The IP registry associated with the IP address | |
ip_asn_id | The autonomous system number (ASN) associated with the IP address | |
ip_asn_description | The description of the autonomous system number (ASN) | |
ip_asn_country_code | The country code associated with the autonomous system number (ASN) | |
ip_ip_network_cidr | The Classless Inter-Domain Routing (CIDR) block of the IP address. | |
Trusted Share | trusted_share_id | A unique identifier for the Trusted Share |
trusted_share_name | The name of the Trusted Share | |
trusted_share_owner_eshare_identity_id | A unique identifier for the identity of the share owner | |
trusted_share_owner_fullname | The full name of the trusted share owner | |
trusted_share_owner_email | The email of the trusted share owner | |
trusted_share_recipient_email | The email of the recipient of the Trusted Share | |
trusted_share_recipient_expiry | The expiration date of the Trusted Share for this recipient | |
trusted_share_recipient_permissions_watermark | Indicates whether the Sharing Policy enforces the use of a watermark when files are downloaded | |
trusted_share_recipient_permissions_can_create | Indicates whether the Sharing Policy allows content creation. | |
trusted_share_recipient_permissions_can_delete | Indicates whether the Sharing Policy allows content deletion. | |
trusted_share_recipient_permissions_can_download | Indicates whether the Sharing Policy allows content downloading. | |
trusted_share_recipient_permissions_can_edit | Indicates whether the Sharing Policy allows content editing. | |
trusted_share_recipient_permissions_can_read | Indicates whether the Sharing Policy allows content viewing. | |
trusted_share_recipient_permissions_can_share | Indicates whether the Sharing Policy allows content sharing. | |
trusted_share_recipient_permissions_convert_to_pdf | Indicates where the Sharing Policy enforces conversion of the content to be downloaded to PDF format and addition of Terms of Use as cover page, if applicable | |
trusted_share_recipient_permissions_login_required | Indicates whether the Sharing Policy requires login to access the Trusted Share | |
trusted_share_recipient_permissions_pin_protected | Indicates whether the Sharing Policy requires a PIN to access the Trusted Share | |
Sharing Policy | sharing_policy_id | A unique identifier of the Sharing Policy used |
sharing_policy_name | The name of the Sharing Policy used | |
sharing_policy_sensitivity_label_id | A unique identifier of the sensitivity label of the Sharing Policy | |
sharing_policy_sensitivity_label_name | The sensitivity label name of the Sharing Policy | |
File | file_name | The name of the file involved in the event |
file_is_folder | Indicates whether the item is a folder | |
file_path | The path of the file in this event | |
file_extension | The file extension (e.g., .pdf, .docx) | |
file_size | The size of the file in bytes | |
file_sensitivity_label_id | The unique identifier for the sensitivity label of the file | |
file_sensitivity_label_name | The name of the sensitivity label of the file | |
file_dlp_tag_id | The unique identifier for the Data Loss Prevention (DLP) tag of the file | |
file_dlp_tag_name | The name for the Data Loss Prevention (DLP) tag of the file | |
file_page | The page of the file associated with the event, if applicable | |
file_pageview_id | The unique identifier for the page of the file associated with the event, if applicable | |
file_url | The url of the location of the file, if applicable | |
sharepoint_site_name | The name of the SharePoint Site the file belongs to, if applicable | |
sharepoint_site_url | The url of the SharePoint Site the file belongs to, if applicable | |
cloud_storage_provider | The cloud storage provider in this event (e.g., SharePoint, OneDrive, blob, Dropbox) | |
cloud_storage_provider_organization_name | The organization name of the cloud storage accout provider | |
M365 DLP | m365_dlp_event_id | The M365 DLP audit event id of the DlpRuleMatch event |
m365_dlp_event_timestamp | The timestamp of the DlpRuleMatch M365 DLP event | |
m365_dlp_policy | The M365 DLP policy that was matched through one of its rules | |
m365_dlp_rule | The M365 DLP rule that was matched | |
m365_dlp_rule_severity | The severity of the M365 DLP rule match | |
m365_dlp_sit | The M365 DLP SIT (sensitive information type) that was detected | |
m365_dlp_sit_count | The number of times the M365 DLP SIT was detected | |
m365_dlp_sit_confidence | The confidence with which the M365 DLP SIT was detected |
EventHub Configuration Steps
NOTE:
The following configuration assumes the customer would prefer to host the EventHub. If there is concern with hosting the EventHub in your tenant, please contact your eShare Customer Success Manager to discuss.
eShare requires the following items be provided to your Customer Success Manager so eShare audit events can be forwarded to the EventHub accordingly:
The EventHub connection string
The EventHub must have Kafka surface enabled
A Private Link from the EventHub to the eShare Cloud Service must be created
Below are instructions and screenshots that showcase how to get the EventHub's required information.
Create/Retrieve EventHub's Connection String
In the EventHub's namespace, create a new EventHub where audit events will be forwarded to. Under ‘Shared Access Policies‘ of the EventHub (not the EventHub namespace), create a new policy with ‘Send‘ permission. Two connection strings will be generated, provide either the ‘Primary connection string‘ or the ‘Secondary connection string’.
Verify Kafka Surface is Enabled
In your EventHub's namespace, in ‘Overview’, verify that Kafka surface is enabled. This is enabled at ‘Standard’ pricing and above.
Private Endpoint
In your EventHub's namespace, in ‘Overview’, select ‘JSON View‘ and copy the resource ID from the bar and provide this to your eShare CSM. eShare will then initiate a Private Endpoint service which your team will accept at the appropriate time.
Ingesting Events into a SIEM Tool
Every SIEM Tool has a different way of ingesting content sent via an EventHub. You will be able to get the required connection information from your Azure EventHub, or your eShare CSM will provide the details of the eShare hosted EventHub. Here are resources on how to connect the EventHub stream to your SIEM tool: