Help Center

Menu

Logs streaming

This documentation provides technical guidance on using the Logs Streaming feature within the Squiz DXP Logging Service.

Logs Streaming streams logs directly from the Squiz DXP to your external observability platforms.

This feature is particularly useful for centralized tracking across your DXP and other internal systems.

It enables deeper analysis and monitoring of your DXP logs.

It maintains a separate, potentially longer-lasting, store for all your logs, especially for compliance requirements.

For example:

  • Australian record retention laws require that most company records be kept for at least seven years.

  • New Zealand record retention laws require that company records be kept for at least seven years.

  • United Kingdom record retention laws require company records to be kept for at least six years, sometimes longer.

  • United States federal and state company law requires many company records to be kept for at least seven years. And some must be kept indefinitely.

Accessing logs streaming configuration

Access the configuration for Logs Streaming from the ⋯ menu on the Logs screen.

Supported destinations

The Logs Streaming feature supports streaming logs to Dynatrace, Datadog, and Microsoft Sentinel.

Setting up logs streaming

The setup process varies depending on your chosen external platform.

Log streaming to Datadog

To configure logs streaming to Datadog:

  1. Obtain an API key from your Datadog account, as described in the vendor’s user documentation.

  2. In the Squiz DXP Logs screen, select Datadog as your destination and enter the required API key.

  3. Enter the correct Datadog site URL. The system includes form validation to guide you on the expected format.

  4. At the bottom of the configuration dialogue, you find a toggle switch labeled Enable streaming.

    1. If you want to start streaming logs immediately, toggle this option on.

    2. If you wish to save the configuration without enabling streaming yet, leave it off and click Save. You can enable it later.

  5. Click Save to apply your settings.

  6. Upon saving, the system attempts to connect and sends a test log to validate the provided details.

    1. If the connection succeeds, a message appears in the Logs UI indicating that the changes were saved.

    2. If the connection fails, an error message displays.

If you encounter connection errors, please consult the platform documentation, support, or technical platform status to resolve this issue.

For detailed troubleshooting, read the Datadog API documentation.

The Squiz DXP merely relays the error message from Datadog.

Log streaming to Dynatrace

To configure logs streaming to Dynatrace:

  1. Obtain an API key and environment ID from your Dynatrace account, as described in the vendor’s user documentation.

  2. In the Squiz DXP Logs screen, select Dynatrace as your destination and enter the required API key and environment ID.

  3. At the bottom of the configuration dialogue, you find a toggle switch labeled Enable streaming.

    1. If you want to start streaming logs immediately, toggle this option on.

    2. If you wish to save the configuration without enabling streaming yet, leave it off and click Save. You can enable it later.

  4. Click Save to apply your settings.

  5. Upon saving, the system attempts to connect and sends a test log to validate the provided details.

    1. If the connection succeeds, a message appears in the Logs UI indicating that the changes were saved.

    2. If the connection fails, an error message displays.

If you encounter connection errors, please consult the platform documentation, support, or technical platform status to resolve this issue.

For detailed troubleshooting, read the Dynatrace Log Monitoring API documentation.

The Squiz DXP merely relays the error message from Dynatrace.

Log streaming to Microsoft Sentinel

Squiz DXP sends logs to Microsoft Sentinel through the Azure Monitor Logs Ingestion API.

Before configuring Logs Streaming in Squiz DXP, prepare these resources in Azure:

  • A Log Analytics workspace with a custom table for the Squiz DXP logs.

  • A data collection rule (DCR) configured for the custom table and a Logs ingestion endpoint.

  • A Microsoft Entra application with a client secret. Assign the application the Monitoring Metrics Publisher role on the DCR.

The DCR stream declaration must accept the fields sent by the DXP:

  • TimeGenerated as a datetime.

  • DxpLogId, DxpService, Severity, Message, Tags, Host, UserId, ApiKeyId, TraceId, and Environment as strings.

  • Details as a dynamic value.

For more information about preparing these resources, read the Microsoft tutorial for setting up the Logs Ingestion API.

To configure Logs Streaming to Microsoft Sentinel:

  1. In the Azure portal, open the DCR’s JSON view and copy the Logs ingestion URI and immutable ID. If you use a DCE, copy its logs ingestion endpoint instead. Copy the client secret value when you create the Microsoft Entra application.

  2. In the Squiz DXP Logs screen, select Microsoft Sentinel as your destination.

  3. Enter these values:

    Endpoint

    Enter the Logs ingestion URI, or the logs ingestion endpoint from a data collection endpoint (DCE). Enter only the base URL, such as https://<endpoint>.<region>-1.ingest.monitor.azure.com.

    DCR immutable ID

    Enter the immutable ID of the DCR. It starts with dcr-.

    Stream name

    Enter the stream name defined in the DCR. Custom streams must start with Custom-.

    Tenant ID

    Enter the Microsoft Entra tenant ID that contains the application.

    Client ID

    Enter the Application (client) ID of the Microsoft Entra application.

    Client secret

    Enter the client secret value for the Microsoft Entra application. Do not enter the secret ID.

  4. At the bottom of the configuration dialogue, you find a toggle switch labeled Enable streaming.

    1. If you want to start streaming logs immediately, toggle this option on.

    2. If you wish to save the configuration without enabling streaming yet, leave it off and click Save. You can enable it later.

  5. Click Save to apply your settings.

  6. Upon saving, the system attempts to connect and sends a test log to validate the provided details.

    1. If the connection succeeds, a message appears in the Logs UI indicating that the changes were saved.

    2. If the connection fails, an error message displays.

If you encounter connection errors, please consult the platform documentation, support, or technical platform status to resolve this issue.

For detailed troubleshooting, read the Azure Monitor Logs Ingestion API documentation.

The Squiz DXP merely relays the error message from Azure Monitor.

Managing logs streaming configuration

Once configured, you can manage your Logs Streaming settings:

Streaming Status

A status indicator appears in the header section of the Logs screen (for example, Streaming disabled or no error text if streaming is active).

Re-accessing Configuration

Clicking on the streaming status indicator relaunches the configuration dialogue, allowing you to modify settings and turn streaming on or off.

Removing Streaming Details

Within the configuration dialogue, click Remove streaming details to clear the configured external platform details.

Monitoring logs streaming status

The Logs screen provides visual indicators of the logs streaming status:

Red status indicator

Indicates that log streaming details are saved, but the feature is not enabled.

No indicator

Displays if no log streaming details have been saved, or if log streaming is working correctly.

If log streaming fails, such as when the external service becomes unreachable, an error message appears at the top of the Logs screen.

This error message is the only notification customers receive regarding log streaming issues.

Error handling and troubleshooting

The system provides comprehensive error handling and diagnostic capabilities:

Connection Validation

Upon initial configuration, the system sends a test log to your external platform to validate the provided details (for example, API key, environment ID). This ensures the connection is valid.

Streaming Failure Notification

If there are issues streaming logs (for example, an invalid API key due to it being deleted in the external platform), an error UI appears on the DXP Console Logs screen, indicating that "logs are not streaming."

Diagnosing Errors

The error message allows filtering logs by "log service" to help diagnose the specific issue (for example, API key is invalid).

Automatic Retries: The system attempts to send logs a few times in the event of transient errors before displaying a persistent error notification.

Resolving Errors

To resolve streaming errors, update the configuration with the correct details (for example, a new, valid API key). The error message disappears once the issue is resolved.

Log delivery and retry mechanism

When log streaming is active, logs are delivered to your external platform as they are received by the DXP logging service.

The system delivers logs to your external platform in near-real time.

If the external service becomes unreachable, the DXP system retries three times with specific delays: 2 seconds, 3 seconds, and 5 seconds.

Each retry attempt may take up to 20 seconds to complete, so the overall retry process can take a few minutes.

If this results in a gap in your logs on the external service, you can manually fill these gaps by using the Log Export to CSV feature to copy logs across. Any missed messages must be resolved manually to have a complete record on your external system.

What is streamed

The Logs Streaming feature streams these types of content:

  • Logs arriving at the DXP logging service are sent to the external platform as they come in.

  • Audit logs and other relevant logs are streamed.

  • For component blocks, developers can configure specific fields to be translatable, ensuring highly targeted content variation.

Technical implementation

The log streaming feature leverages the DXP logging service to forward logs to external platforms.

The system automatically handles log translation and formatting for external platform compatibility.

Permissions for logs streaming

Access and modification rights for Logs Streaming settings are permission-based:

Admins and Owners

Can change log streaming settings.

Developers

Can read log streaming settings.

Admins, Owners, and Developers

Can view the status indicator.

Everyone, including all user roles

Can see the error message if it appears.