Load File Lines

Overview

The Load File Lines component is a Line-by-Line File Reader Component which reads a file stored in the system using its unique file ID and sends its content to the output port, one line at a time. The order of the lines reaching a destination component is not guaranteed and may not match the exact order in the source file. Each output message contains an index property corresponding to the line number in the source file.

Usage

This component is useful for processing files where line-by-line processing is required, such as log parsing, streaming data, or batch processing workflows.

Properties

Property	Type	Description
fileId	string	The unique identifier of the file to be read.

Input Ports

Port	Description
in	Accepts the file ID to be read.

Output Ports

Port	Description
content	Emits the file’s content line by line.
index	Indicates the original line number from the source file.

Processing Logic

1. Receives Input Data:

   • Accepts a fileId that identifies the stored file.

2. Reads File Content Line by Line:

   • Streams the file content and reads it one line at a time.

   • Each line is assigned an index value corresponding to its position in the file.

3. Sends File Content:

   • Emits each line as a separate output message.

   • The order of the lines in the output is not guaranteed to match the order in the source file.

   • If an error occurs (e.g., file not found), it is logged or raised appropriately.

Output Data Schema

Property	Type	Description
index	integer	The position of the line in the original file.
line	string	The content of the line.

Notes

• Emits Lines Independently: Each line is processed separately, making it useful for parallel processing.

• Order is Not Guaranteed: Due to streaming behavior, the order of lines in the output may not match the original file order.

• Ideal for Log Processing and Streaming Workflows: Useful for scenarios where each line represents an independent unit of data.