Batch
The batch component allows Integrations messages to be aggregated into simple collections for processing in batches.
Integrators would need to split an integration logic into two parts.
The first should use the action Add message to batch, which starts to collect messages.
The second should start with the Get ready batches trigger.
It uses the Maester service as a package repository. The legacy component versions support MongoDB (4.0 and higher stable releases) as a batches repository.
| The component does not support external MongoDB as a data store since version 2.0.0 (disabled support for credentials in the component). |
Requirements
Maester microservice, the official Integrations object-storage client, is enabled by default as the Batch component relies on it.
Triggers
Get ready batches
Emits all batches that are ready to be processed.
Input fields description
| Input field | Required | Description | Example |
|---|---|---|---|
Max time in milliseconds before batch is ready. |
No. |
Timeout in milliseconds after batch creation, which makes batch ready for processing. Defaults to 60000 ms. |
|
Max records number in batch. |
No. |
Maximum count of items in batch, which makes batch ready for processing. Defaults to 100. |
|
Max size of batch in bytes. |
No. |
Maximum bytes size of items in batch which makes batch ready for processing 1000000(1 MB). |
|
Correlation ID. |
Yes. |
Correlation ID between action and trigger. Correlation ID helps identify what collection of batches should be used. |
|
Do not delete batch after retrieval. |
No. |
If selected, the batch will not be deleted after emit. |
|
| Use the same configuration in action for correct batch processing. |
Actions
Add message to batch
Stores a message in a batch and emits a created/updated batch with a processed item only.
Input fields description
| Input field | Required | Description | Example |
|---|---|---|---|
Max time in milliseconds before batch is ready. |
No. |
Timeout in milliseconds after batch creation, which makes batch ready for processing. Defaults to 60000 ms. |
|
Max records number in batch. |
No. |
Maximum count of items in batch, which makes batch ready for processing. Defaults to 100. |
|
Max size of batch in bytes. |
No. |
Maximum bytes size of items in batch which makes batch ready for processing 1000000(1 MB). |
|
Correlation ID. |
Yes. |
Correlation ID between action and trigger. Correlation ID helps identify what collection of batches should be used. |
|
| Use the same configuration in trigger for correct batch processing. |
Additional info
-
Objects created in Maester will have the default TTL. Please contact support if you require the value of this property.
-
Please ensure that the value of
Max time in ms before batch is readyis less than Maester’s default TTL. Otherwise, the object in Maester will be deleted before the batch is ready.