Overview

The Remove File Component removes a specified file from the database using its unique file ID. This ensures that unwanted or obsolete files are deleted efficiently from the system.

Usage

This component is used when a file needs to be permanently removed from the database, ensuring storage optimization and compliance with data retention policies.

Properties

Input Ports

Output Ports

Processing Logic

1. Receives Input Data:

   • Accepts a fileId that identifies the file to be deleted.

2. Deletes the File:

   • Locates the file in the database.

   • Removes the file permanently.

3. Confirms Deletion:

   • Emits the fileId of the deleted file as confirmation.

   • If an error occurs (e.g., file not found), it is logged or raised appropriately.

Output Data Schema

Notes

• Permanent Deletion: Once removed, the file cannot be recovered.

• Efficient Storage Management: Helps free up space in the database.

• Error Handling: Ensures errors are logged when deletion fails.

Overview

The Archive File Component allows users to archive a file stored on the system using its unique file ID. This ensures efficient storage management and facilitates long-term data retention.

Usage

This component is used to create an archive of a specific file, making it useful for workflows that require file compression or long-term storage solutions.

Properties

Input Ports

Output Ports

Processing Logic

1. Receives Input Data:

   • Accepts a file ID (fileId) from the input.

2. Archives the File:

   • Calls the system’s file archiving function.

3. Handles Success & Errors:

   • If successful, sends the archived file’s fileId to the out port.

   • If an error occurs, the process fails silently or throws an appropriate error.

Output Data Schema

Notes

• Ensures File Retention: Helps store files in an archived state for long-term preservation.

• Works with File Storage Systems: Compatible with various file storage mechanisms.

• Ideal for Data Management Workflows: Supports automated file compression and archival processes.

This component is essential for workflows that require archiving files for future retrieval or compliance purposes.

Files Connector

Overview

The Load File Component reads a file stored in the system using its unique file ID and sends its content to the output port. It is useful for processing stored files in workflows.

Usage

This component is used to retrieve file content for further processing or validation in workflows.

Properties

Input Ports

Output Ports

Processing Logic

1. Receives Input Data:

   • Accepts a fileId that identifies the stored file.

2. Reads File Content:

   • Opens a stream to read the file contents.

   • Accumulates the content as data chunks arrive.

3. Sends File Content:

   • Emits the full file content to the content output port.

   • If an error occurs (e.g., file not found), it is logged or raised appropriately.

Output Data Schema

Notes

• Retrieves Full File Content: Useful for workflows requiring file content processing.

• Supports Various File Types: Works with text-based and structured data files.

• Ideal for Data Processing Workflows: Enables automated retrieval of stored files for further actions.

This component is essential for workflows that require accessing and processing stored file content dynamically.

Overview

The File Storage Component allows saving a file to the database with its metadata, including filename, MIME type, and encoding. This component enables efficient file storage and retrieval for various applications.

Usage

This component is useful when files need to be uploaded and stored securely in a database for further processing or retrieval.

Properties

Input Ports

Output Ports

Processing Logic

1. Receives Input Data:

   • Accepts filename, mimetype, content, and contentEncoding.

2. Saves the File:

   • Stores the file in the database.

   • Uses the specified encoding format for content.

3. Confirms Storage:

   • Emits metadata of the stored file, including its unique fileId, filename, and contentType.

   • If an error occurs (e.g., invalid content), it is logged or raised appropriately.

Output Data Schema

Notes

• Supports Various Encoding Formats: Content encoding options include utf8, base64, hex, and more.

• Secure and Reliable Storage: Ensures files are stored securely in the database.

• Ideal for File Upload and Management Workflows: Enables seamless file handling for applications requiring document storage and retrieval.

Overview

The Download File Component allows users to download a file from a given link and store it in the system. It supports specifying a custom filename for the downloaded file.

Usage

This component is used to retrieve files from remote URLs and save them for further processing in workflows.

Properties

Input Ports

Output Ports

Processing Logic

1. Receives Input Data:

   • Accepts a link to a file and an optional customFileName.

2. Validates URL:

   • Checks if the provided link is a valid URL.

   • If invalid, throws an error.

3. Downloads the File:

   • Sends an HTTP request to retrieve the file.

   • Extracts the filename from the response headers or URL if no custom filename is provided.

4. Saves the File:

   • Stores the file and generates a unique fileId.

5. Handles Success & Errors:

   • If successful, sends the downloaded file's details to the out port.

   • If an error occurs (e.g., invalid URL or download failure), it is logged or raised appropriately.

Output Data Schema

Notes

• Validates URLs Before Downloading: Ensures the provided link is a valid file URL.

• Supports Custom Filenames: Users can specify a filename or rely on the original filename from the response.

• Ideal for Automated File Retrieval: Useful in workflows that require downloading and storing remote files.

This component is essential for workflows that need to automate file downloads from external sources for further processing.

Overview

The Load File Lines component is a Line-by-Line File Reader Component which reads a file stored in the system using its unique file ID and sends its content to the output port, one line at a time. The order of the lines reaching a destination component is not guaranteed and may not match the exact order in the source file. Each output message contains an index property corresponding to the line number in the source file.

Usage

This component is useful for processing files where line-by-line processing is required, such as log parsing, streaming data, or batch processing workflows.

Properties

Input Ports

Output Ports

Processing Logic

1. Receives Input Data:

   • Accepts a fileId that identifies the stored file.

2. Reads File Content Line by Line:

   • Streams the file content and reads it one line at a time.

   • Each line is assigned an index value corresponding to its position in the file.

3. Sends File Content:

   • Emits each line as a separate output message.

   • The order of the lines in the output is not guaranteed to match the order in the source file.

   • If an error occurs (e.g., file not found), it is logged or raised appropriately.

Output Data Schema

Notes

• Emits Lines Independently: Each line is processed separately, making it useful for parallel processing.

• Order is Not Guaranteed: Due to streaming behavior, the order of lines in the output may not match the original file order.

• Ideal for Log Processing and Streaming Workflows: Useful for scenarios where each line represents an independent unit of data.