File pickup

File pickup

This consumer is used to read files into a flow. Files are read from the input directory. When a backup directory is defined, files can be copied to this location after they have been read. This consumer polls for files itself. Every single file with be a separate ConnectMessage entering the flow.

Through Response Message type, you can determine if files are read as Document, Text, File or Byte Array.

The consumer's output is a ConnectMessage, where the content of the file is copied into the message part defined at the value of “MessagePart”.

 
In the table below, you will find an explanation of these properties. All attributes with a ‘*’ are mandatory.

Non-existing directories will automatically be created.

Attribute

Description

Name*

By default, we fill this out with the technical ‘tag’, followed by a serial number. Changing the name is optional.

Enable

Check this box if you want this consumer to be enabled.

Consumer Autostart

Consumer will be started at startup of the interface.

Pickup Directory*

The directory to read (pick up) the files from. When you use a directory which one of your flows is already using, constructor will give you an indication of the fact that this could potentially be a problem.

File pattern type

There are three options to choose from:
  • LEGACY (default)
  • REGEX
  • ANT

The choosen file pattern type is teh way the file pattern will be interpreted.

Ant-Style Patterns:

  • Format: Ant-style patterns use wildcards and simple placeholders to match filenames or strings.
  • Wildcards: Common wildcards used in Ant-style patterns are * and ?.
    • * matches any number of characters (including none).
    • ? matches any single character.
  • Example: If you want to match all files with a .txt extension, you can use the Ant-style pattern *.txt.

Regex-Style Patterns (Regular Expressions):

  • Format: Regular expressions (regex) are a powerful way to describe complex patterns using a specific syntax.
  • Syntax: Regex patterns can match characters based on precise rules. For instance, . matches any character, ^ matches the start of a line, and $ matches the end of a line.
  • Example: To match all files with a .txt extension using regex, you can use the pattern \.txt$.

Key Differences:

  1. Complexity: Regular expressions are more powerful and flexible than Ant-style patterns. They can handle more complex pattern-matching scenarios, including character classes, repetitions, and more.
  2. Character Matching: Regular expressions are character-based, which means you can specify precise character sequences and rules. In contrast, Ant-style patterns use simpler wildcard characters for matching.
  3. Use Cases: Ant-style patterns are typically used for simple pattern matching, especially in build tools like Apache Ant, while regular expressions are used in various programming languages and text processing tools for more advanced pattern matching.
  4. Performance: Ant-style patterns are generally faster for simple tasks because they are less computationally intensive than regex. Regular expressions can be slower, especially for complex patterns, due to their advanced features.

In the context of your original text, when the "File Pattern" attribute is set to LEGACY, it first tries to interpret the value as a regex. If that interpretation fails, it then attempts to treat it as an Ant-style pattern. This allows for more flexibility and compatibility when dealing with different types of pattern specifications.

File Pattern

Only files that match this pattern will be read (picked up).

Because the File Pickup checks if the pattern is REGEX or ANT Style, some patterns are difficult to recognize.
For example, take the pattern orderinfo*.xml. Internally this will be recognized as a REGEX expression and therefore you will get no result. To resolve this issue add a * before the string like: *orderinfo*.xml.

If you use REGEX patterns, you allways need to describe the whole string, not only a part of the string. This means if you want to pickup files which ends on _0[1-6].jpg, you need to give the pattern: .*_0[1-6]\.jpg$.

DO NOT CREATE THIS OPTION AS DYNAMIC

Backup Directory

Path to the directory where the back-ups are archived. If left empty no back-ups will be made. The complete path of the file after the back-up has completed will be stored in a new property: file_backupFile.

Custom Backup Strategy

Select default strategy for generating backup files. Defaults to [yyyyMMdd-HHmmssSSS]_[filename]. You can create another backup strategy (name format for your files), by creating a Bean or a Script. You can select the script/bean via the selection box.

If you select NONE, the default strategy will be taken.

Delete Original File

Set to TRUE, will delete the files after being handled and backed up. The default value is true.

Messagepart name

Name of the MessagePart in a ConnectMessage where the content of the file is being stored.

Response Message Type

The type of the MessagePart where the file content is being stored and will be passed to the next service.

If the user who logged in has a LargeFileSupport license, you will get an extra LargeFile option. If not this option is greyed out.

Max Messages Per Poll

Number of messages being handled per poll, i.e., if max-messages-per-poll is set to 5 and the timer to every 10 seconds a maximum of 5 messages every 10 seconds is handled.

If set to -1 all messages available are handled per timer interval.
If set to 0 only 1 message is handled per timer interval.

Quartz Cron Expression

Definition of the interval this polling consumer will check for messages. Format (sec) (min) (hour) (day) (month) (year).

Solid File Check Time

When files in the pick up directory are expected being written to, this attribute can be used to allow only files to be passed that have a solid file size. The value of this attribute (in milliseconds) is the interval wherein the Consumer checks if the file size has increased or not.

Keep in mind that this will often not work when using the Windows OS. Windows creates the file with a 0 byte file length until the file is completely written. Therefore this mechanism will often not work.

Legacy Poller
DEPRICATED

THIS SETTING IS DEPRICATED/DO NOT USE IT --> ALWAYS TRUE

If set to true, the legacy synchronous poller will be used. If set to false, the newer asynchronous mechanism will be used.

Encoding

Used to specify desired encoding strategy for inbound files. The default is UTF-8.

See the Oracle encoding documentation for more information.

Rescan Directory

Set to TRUE, the file consumer will rescan the directory with every poll, allowing for concurrent reading of files. The default value is false.

Description

Description of the specific consumer. This is for documentation purposes.

 

If you want to use more than one FilePickup pointing to the same directory, keep the following in mind.

  • Never overlap the time of the pickup. The FilePickup will create an internal file list. If the pickup time is overlapping, errors will occur.
  • Set the rescan directory flag to true. Otherwise only one file list will be created, and a file could be deleted bij another file pickup. If that occurs, an error will be thrown.