Real-time flows
Flow’s step lifecycle
When a flow is started, each step transitions from the Sleeping state into Starting, and the startup is successfully transitioned into Running.
This is illustrated in the following diagram.
The platform’s internal container orchestrator monitors the activity of the flow’s steps and decides to stop them when it believes the data have been processed, and the steps are idling.
In this case, the steps transition to Shutting Down and then to Sleeping, where they wait for the next execution round.
Such a step lifecycle is critical in a cloud environment where millions of Docker containers run on the same hardware. This keeps hardware costs low; however, starting and stopping the containers produces a slight latency, which is acceptable for most of the flows doing background synchronization.
Such latency is not acceptable in real-time use cases, and this is where real-time flows come into play.
Real-time flows
In contrast to an ordinary flow, a real-time flow never sleeps.
All the steps of a real-time flow remain in the Running state until the user explicitly stops the flow. Because the containers run constantly, the data is consumed from the messaging queues and processed in real time without any latency produced by the platform.
When you start a real-time flow, it might take longer to process the first message because the platform needs to start all the containers and make them operational. This initial start-up time is called warm-up time.
All the consecutive messages will process quickly.
The warm-up time should also be considered when you restart your flow.
How to switch to real-time flow
Every integration flow can be switched to a real-time flow:
-
Navigate to the Flows page and open the flow menu.
-
The menu needs to be set to a real-time mode/type.
-
Stop the flow. If the flow is running, the Enable real-time option is unavailable, as can be seen in this screenshot:
-
Open the menu of the inactive flow to see the available Enable real-time option:
-
Select the Enable real-time to switch this flow into real-time mode/type.
The previous screenshot shows the moment when the flow is switched into real-time.
When the process succeeds, a checkmark and a rocket icon appear beside the Enable real-time menu item.
Updating real-time flows
You need to create a new draft to make changes to your real-time flows.
None of the changes made to the draft, whether a mapping adjustment, a component version change, or anything else, will affect the running flow containers.
You need to publish the draft as a new version to apply the changes.
After you publish the draft as a new version, the currently running version of the flow will be stopped by shutting down all the running flow containers.
The new version will be deployed by starting new step containers.
| In this case, the flow goes through the warm-up period again. |