Help Center

Menu

Logs streaming

This documentation provides technical guidance on using the Logs Streaming feature within the Squiz DXP Logging Service.

Logs Streaming streams logs directly from the Squiz DXP to your external observability platforms.

This feature is particularly useful for centralized tracking across your DXP and other internal systems.

It enables deeper analysis and monitoring of your DXP logs.

It maintains a separate, potentially longer-lasting, store for all your logs, especially for compliance requirements.

For example:

  • Australian record retention laws require that most company records be kept for at least seven years.

  • New Zealand record retention laws require that company records be kept for at least seven years.

  • United Kingdom record retention laws require company records to be kept for at least six years, sometimes longer.

  • United States federal and state company law requires many company records to be kept for at least seven years. And some are required to be kept indefinitely.

Accessing logs streaming configuration

Access the configuration for Logs Streaming from the ⋯ menu on the Logs UI.

Supported destinations

The Logs Streaming feature supports streaming logs to Dynatrace and Datadog.

Setting up logs streaming

The setup process requires different details depending on your chosen external platform.

Log streaming to Datadog

To configure Logs Streaming to Datadog:

  1. Obtain an API key from your Datadog account.

  2. In the DXP Logs UI, select Datadog as your destination and enter the required API key.

  3. Enter the correct Datadog site URL. The system includes form validation to guide you on the expected format.

  4. At the bottom of the configuration dialogue, you find a toggle switch labeled Enable streaming.

    1. If you want to start streaming logs immediately, toggle this option on.

    2. If you wish to save the configuration without enabling streaming yet, leave it off and click Save. You can enable it later.

  5. Click Save to apply your settings.

  6. Upon saving, the system attempts to connect and sends a test log to validate the provided details.

    1. If the connection succeeds, a message appears in the Logs UI indicating that the changes were saved.

    2. If the connection fails, an error message displays.

If you encounter connection errors, please consult the platform documentation, support, or technical platform status to resolve this issue.

For detailed troubleshooting, read the Datadog API documentation.

The DXP merely relays the error message from Datadog.

Log streaming to Dynatrace

To configure Logs Streaming to Dynatrace:

  1. Obtain an API key and Environment ID from your Dynatrace account.

  2. In the DXP Logs UI, select Dynatrace as your destination and enter the required API key and Environment ID.

  3. At the bottom of the configuration dialogue, you find a toggle switch labeled Enable streaming.

    1. If you want to start streaming logs immediately, toggle this option on.

    2. If you wish to save the configuration without enabling streaming yet, leave it off and click Save. You can enable it later.

  4. Click Save to apply your settings.

  5. Upon saving, the system attempts to connect and sends a test log to validate the provided details.

    1. If the connection succeeds, a message appears in the Logs UI indicating that the changes were saved.

    2. If the connection fails, an error message displays.

If you encounter connection errors, please consult the platform documentation, support, or technical platform status to resolve this issue.

For detailed troubleshooting, read the Dynatrace Log Monitoring API documentation.

The DXP merely relays the error message from Dynatrace.

Managing logs streaming configuration

Once configured, you can manage your Logs Streaming settings:

Streaming Status

A status indicator appears in the header section of the Logs screen (for example, Streaming disabled or no error text if streaming is active).

Re-accessing Configuration

Clicking on the streaming status indicator relaunches the configuration dialogue, allowing you to modify settings and turn streaming on or off.

Removing Streaming Details

Within the configuration dialogue, a Remove streaming details button appears, allowing you to clear the configured external platform details.

Monitoring logs streaming status

The Logs UI provides visual indicators of the Logs Streaming status:

Red status indicator

Indicates that log streaming details are saved, but the feature is not currently enabled.

No indicator

Displays if no log streaming details have been saved, or if log streaming is working correctly.

If log streaming fails, such as when the external service becomes unreachable, an error message appears at the top of the Logs UI.

This error message is the only notification customers receive regarding log streaming issues.

Error handling and troubleshooting

The system provides comprehensive error handling and diagnostic capabilities:

Connection Validation

Upon initial configuration, the system sends a test log to your external platform to validate the provided details (for example, API key, environment ID). This ensures the connection is valid.

Streaming Failure Notification

If there are issues streaming logs (for example, an invalid API key due to it being deleted in the external platform), an error UI appears on the DXP Console Logs screen, indicating that "logs are not streaming."

Diagnosing Errors

The error message allows filtering logs by "log service" to help diagnose the specific issue (for example, API key is invalid).

Automatic Retries

The system attempts to send logs a few times in case of transient errors before displaying a persistent error notification.

Resolving Errors

To resolve streaming errors, update the configuration with the correct details (for example, a new valid API key). The error message disappears once the issue is resolved.

Log delivery and retry mechanism

When log streaming is active, logs are delivered to your external platform as they arrive at the DXP logging service.

The system delivers logs to your external platform in near-real time.

If the external service becomes unreachable, the DXP system retries three times with specific delays: after 2 seconds, then after 3 seconds, then after 5 seconds.

Each retry attempt may take up to 20 seconds to complete, so the total retry process can take up to a few minutes.

If this results in a gap in your logs on the external service, you can manually fill these gaps by using the Log Export to CSV feature to copy logs across. Any missed messages must be resolved manually to have a complete record on your external system.

What is streamed

The Logs Streaming feature streams the following content:

  • Logs arriving at the DXP logging service are sent to the external platform as they come in.

  • Audit logs and other relevant logs are streamed.

  • For component blocks, developers can configure specific fields to be translatable, ensuring highly targeted content variation.

Technical implementation

The log streaming feature leverages the DXP logging service to forward logs to external platforms.

The system automatically handles the translation and formatting of logs for external platform compatibility.

Permissions for logs streaming

Access and modification rights for Logs Streaming settings are permission-based:

Admins and Owners

Can change log streaming settings.

Developers

Can read log streaming settings.

Admins, Owners, and Developers

Can view the status indicator.

Everyone, including all user roles

Can see the error message if it appears.