Sequential Loop

Process the items of a list strictly one at a time, in their original order, with the Sequential mode of Each and the Continue Each component.

Introduction

In the default mode the Each component sends all items at once. When the steps in the loop body take a different amount of time for different items, the fast items overtake the slow ones. A delay only spaces the items out; it does not wait for them to finish.

A sequential loop is the recommended way to build a loop in Appmixer. Turn on Sequential on Each and end the loop body with the Continue Each component. Each then sends the next item only after the previous one has been confirmed by Continue Each, so the items are processed one at a time and in their order, and the loop collects a Result from every item.

A sequential loop: Continue Each closes the loop body

How it works

1. Each with Sequential turned on sends the first item.
2. The steps in the loop body process the item.
3. Continue Each confirms the item and adds its entry to the result.
4. Each sends the next item. After the last item has been confirmed, Each sends done with the collected Result.

Every item continues the flow run that started the loop, so the steps in the loop body can use the variables of the components before Each for every item, not only the first one.

Settings

Each

• Sequential: send the next item only after Continue Each has confirmed the previous one.
• Item Timeout (seconds): how long Each waits for the confirmation of one item. Default 300, minimum 60. An item that is not confirmed in time is skipped, and the loop moves on to the next one, so it never hangs. This happens when a step in the loop body fails, or when a Condition filters the item out before it reaches Continue Each. Set the timeout safely above the longest time one item can take.
• Delay: in a sequential loop, an extra pause in milliseconds after an item is confirmed and before the next one is sent.

Each with Sequential turned on

Continue Each

Place Continue Each after the last step that has to finish before the next item starts.

• Correlation ID: the Correlation ID variable of the Each item. It tells Continue Each which loop to continue (see below).
• Index: the Index variable of the Each item. It tells Each which item is done. A repeated or late confirmation, for example for an item that has already timed out, is ignored.
• Add to Result (optional): rows of Name and Value, the values to collect from the item, for example id → the ID of a record created in the loop. See Collecting results.

Continue Each passes Correlation ID and Index on its out port, so the flow can continue after it if needed.

Continue Each: Add to Result with a Name and a Value

The Correlation ID in a sequential loop

The Correlation ID works as described in How the Correlation ID connects the loop: Each creates one per run and puts it in every item. In a sequential loop it also identifies the run that is waiting. While a run is waiting for its item to be confirmed, Each keeps the rest of the list under its Correlation ID. Continue Each looks up the waiting run by that ID and wakes it up.

• Always map the Correlation ID of the item of the Each you want to continue. If Continue Each gets an ID that no sequential loop is waiting for (Sequential is off, or the loop has already finished), it only logs it and passes the message on.
• In nested sequential loops, every Each needs its own Continue Each with its own Correlation ID and Index. The inner Continue Each goes at the end of the inner loop body, the outer one after everything the outer item has to finish, including the inner loop (for example, on the done port of the inner Each).

Collecting results

Every row of Add to Result in Continue Each has a Name and a Value. Each item adds one entry to the Result list that Each sends on its done port once all items are processed: an object with one field per row. The entries are in item order and line up with the items, so you do not need Join Each or Storage in a sequential loop.

For example, with the rows id → ID of the created record and status → Status of the created record:

[
    { "id": "1001", "status": "created" },
    null,
    { "id": "1003", "status": "created" }
]

• A value that is empty for an item is added as null in its field, so every entry has the same fields.
• An item that was skipped after Item Timeout is null as a whole (the second entry above).
• Without rows in Add to Result, every entry is an empty object {}.
• A row with a value but no name stops Continue Each with an error.
• Result is an empty list when Sequential is off.

The done port also contains Items Count and Correlation ID.

Demo flow

The demo flow loops over the list [1500, 100, 800]. The Slow step (a Code Block) takes as many milliseconds as the item says. Without Sequential the items would finish in the order 100, 800, 1500. With Sequential they are processed in the order 1500, 100, 800, and the done port returns:

{
    "count": 3,
    "correlationId": "cid:2407358777",
    "result": [
        { "result": "item 1500 ms" },
        { "result": "item 100 ms" },
        { "result": "item 800 ms" }
    ]
}

The notes in the flow explain each part on the canvas. Create the flow with the POST /flows endpoint, or with the Appmixer CLI:

appmixer flow create demo-each-sequential.json

demo-each-sequential.json

{
    "name": "Demo - Each sequential",
    "flow": {
        "e1a0c001-0000-4000-8000-000000000001": {
            "type": "appmixer.utils.controls.OnStart",
            "x": 64,
            "y": 64,
            "version": "1.0.0",
            "source": {},
            "config": {},
            "errorHandling": {
                "autoRetry": false,
                "onError": "stopFlow"
            }
        },
        "e1a0c001-0000-4000-8000-000000000002": {
            "type": "appmixer.utils.controls.SetVariable",
            "label": "Items",
            "x": 272,
            "y": 64,
            "version": "1.0.0",
            "source": {
                "in": {
                    "e1a0c001-0000-4000-8000-000000000001": [
                        "out"
                    ]
                }
            },
            "config": {
                "transform": {
                    "in": {
                        "e1a0c001-0000-4000-8000-000000000001": {
                            "out": {
                                "type": "json2new",
                                "modifiers": {
                                    "variables": {}
                                },
                                "lambda": {
                                    "variables": {
                                        "ADD": [
                                            {
                                                "type": "text",
                                                "name": "durations",
                                                "text": "[1500, 100, 800]"
                                            }
                                        ]
                                    }
                                }
                            }
                        }
                    }
                }
            },
            "errorHandling": {
                "autoRetry": false,
                "onError": "stopFlow"
            }
        },
        "e1a0c001-0000-4000-8000-000000000003": {
            "type": "appmixer.utils.controls.Each",
            "label": "Each (sequential)",
            "x": 480,
            "y": 64,
            "version": "1.5.0",
            "source": {
                "in": {
                    "e1a0c001-0000-4000-8000-000000000002": [
                        "out"
                    ]
                }
            },
            "config": {
                "transform": {
                    "in": {
                        "e1a0c001-0000-4000-8000-000000000002": {
                            "out": {
                                "type": "json2new",
                                "modifiers": {
                                    "list": {
                                        "a1000000-0000-4000-8000-000000000001": {
                                            "variable": "$.e1a0c001-0000-4000-8000-000000000002.out.durations",
                                            "functions": []
                                        }
                                    },
                                    "sequential": {},
                                    "itemTimeout": {}
                                },
                                "lambda": {
                                    "list": "{{{a1000000-0000-4000-8000-000000000001}}}",
                                    "sequential": true,
                                    "itemTimeout": 120
                                }
                            }
                        }
                    }
                }
            },
            "errorHandling": {
                "autoRetry": false,
                "onError": "stopFlow"
            }
        },
        "e1a0c001-0000-4000-8000-000000000004": {
            "type": "appmixer.utils.controls.CodeBlock",
            "label": "Slow step",
            "x": 688,
            "y": 64,
            "version": "1.0.0",
            "source": {
                "in": {
                    "e1a0c001-0000-4000-8000-000000000003": [
                        "item"
                    ]
                }
            },
            "config": {
                "transform": {
                    "in": {
                        "e1a0c001-0000-4000-8000-000000000003": {
                            "item": {
                                "type": "json2new",
                                "modifiers": {
                                    "code": {},
                                    "variables": {
                                        "a1000000-0000-4000-8000-000000000003": {
                                            "variable": "$.e1a0c001-0000-4000-8000-000000000003.item.value",
                                            "functions": []
                                        }
                                    }
                                },
                                "lambda": {
                                    "code": "(function() { var end = Date.now() + Number($data.ms); while (Date.now() < end) {} return 'item ' + $data.ms + ' ms'; })()",
                                    "variables": {
                                        "ADD": [
                                            {
                                                "type": "text",
                                                "name": "ms",
                                                "text": "{{{a1000000-0000-4000-8000-000000000003}}}"
                                            }
                                        ]
                                    }
                                }
                            }
                        }
                    }
                }
            },
            "errorHandling": {
                "autoRetry": false,
                "onError": "stopFlow"
            }
        },
        "e1a0c001-0000-4000-8000-000000000005": {
            "type": "appmixer.utils.controls.ContinueEach",
            "x": 896,
            "y": 64,
            "version": "1.0.0",
            "source": {
                "in": {
                    "e1a0c001-0000-4000-8000-000000000004": [
                        "out"
                    ]
                }
            },
            "config": {
                "transform": {
                    "in": {
                        "e1a0c001-0000-4000-8000-000000000004": {
                            "out": {
                                "type": "json2new",
                                "modifiers": {
                                    "correlationId": {
                                        "a1000000-0000-4000-8000-000000000004": {
                                            "variable": "$.e1a0c001-0000-4000-8000-000000000003.item.correlationId",
                                            "functions": []
                                        }
                                    },
                                    "index": {
                                        "a1000000-0000-4000-8000-000000000005": {
                                            "variable": "$.e1a0c001-0000-4000-8000-000000000003.item.index",
                                            "functions": []
                                        }
                                    },
                                    "result": {
                                        "a1000000-0000-4000-8000-000000000006": {
                                            "variable": "$.e1a0c001-0000-4000-8000-000000000004.out.result",
                                            "functions": []
                                        }
                                    }
                                },
                                "lambda": {
                                    "correlationId": "{{{a1000000-0000-4000-8000-000000000004}}}",
                                    "index": "{{{a1000000-0000-4000-8000-000000000005}}}",
                                    "result": {
                                        "ADD": [
                                            {
                                                "name": "result",
                                                "value": "{{{a1000000-0000-4000-8000-000000000006}}}"
                                            }
                                        ]
                                    }
                                }
                            }
                        }
                    }
                }
            },
            "errorHandling": {
                "autoRetry": false,
                "onError": "stopFlow"
            }
        },
        "e1a0c001-0000-4000-8000-000000000006": {
            "type": "appmixer.utils.controls.SetVariable",
            "label": "All done",
            "x": 688,
            "y": 256,
            "version": "1.0.0",
            "source": {
                "in": {
                    "e1a0c001-0000-4000-8000-000000000003": [
                        "done"
                    ]
                }
            },
            "config": {
                "transform": {
                    "in": {
                        "e1a0c001-0000-4000-8000-000000000003": {
                            "done": {
                                "type": "json2new",
                                "modifiers": {
                                    "variables": {
                                        "a1000000-0000-4000-8000-000000000007": {
                                            "variable": "$.e1a0c001-0000-4000-8000-000000000003.done.result",
                                            "functions": []
                                        },
                                        "a1000000-0000-4000-8000-000000000008": {
                                            "variable": "$.e1a0c001-0000-4000-8000-000000000003.done.count",
                                            "functions": []
                                        }
                                    }
                                },
                                "lambda": {
                                    "variables": {
                                        "ADD": [
                                            {
                                                "type": "text",
                                                "name": "results",
                                                "text": "{{{a1000000-0000-4000-8000-000000000007}}}"
                                            },
                                            {
                                                "type": "text",
                                                "name": "count",
                                                "text": "{{{a1000000-0000-4000-8000-000000000008}}}"
                                            }
                                        ]
                                    }
                                }
                            }
                        }
                    }
                }
            },
            "errorHandling": {
                "autoRetry": false,
                "onError": "stopFlow"
            }
        }
    },
    "notes": {
        "n0000000-0000-4000-8000-000000000001": {
            "x": 48,
            "y": -176,
            "width": 560,
            "height": 168,
            "content": "## Each - Sequential mode demo\n\nThe list holds three items. **Slow step** takes as many milliseconds as the item says: **1500, 100, 800**.\n\nA plain Each sends all items at once, so the fast items overtake the slow one and finish as 100, 800, 1500.\n\nWith **Sequential** on, Each sends the next item only after **Continue Each** confirms the previous one, so the items are processed strictly as 1500, 100, 800."
        },
        "n0000000-0000-4000-8000-000000000002": {
            "x": 656,
            "y": -224,
            "width": 440,
            "height": 208,
            "content": "## The loop body\n\n**Slow step** stands in for any slow step (an API call, a database write...).\n\n**Continue Each** closes the loop. Map to it:\n- **Correlation ID** and **Index** from Each's item\n- **Add to Result**: `result` = the output of Slow step\n\nIf an item is not confirmed within **Item Timeout** (120 s here), Each skips it and moves on."
        },
        "n0000000-0000-4000-8000-000000000003": {
            "x": 864,
            "y": 224,
            "width": 400,
            "height": 120,
            "content": "## Done\n\nWhen every item is confirmed, Each fires **done**.\n\n**Result** has one entry per item, in item order:\n`[{\"result\": \"item 1500 ms\"}, {\"result\": \"item 100 ms\"}, {\"result\": \"item 800 ms\"}]`"
        }
    }
}