= Flatpack Component
:doctitle: Flatpack
:shortname: flatpack
:artifactid: camel-flatpack
:description: Parse fixed width and delimited files using the FlatPack library.
:since: 1.4
:supportlevel: Stable
:component-header: Both producer and consumer are supported
//Manually maintained attributes
:camel-spring-boot-name: flatpack

*Since Camel {since}*

*{component-header}*

The Flatpack component supports fixed width and delimited file parsing
via the http://flatpack.sourceforge.net[FlatPack library]. +
 *Notice:* This component only supports consuming from flatpack files to
Object model. You can not (yet) write from Object model to flatpack
format.


== URI format

---------------------------------------------------------
flatpack:[delim|fixed]:flatPackConfig.pzmap.xml[?options]
---------------------------------------------------------

Or for a delimited file handler with no configuration file just use

---------------------------
flatpack:someName[?options]
---------------------------


// component-configure options: START

// component-configure options: END

// component options: START
include::partial$component-configure-options.adoc[]
include::partial$component-endpoint-options.adoc[]
// component options: END

// endpoint options: START

// endpoint options: END

== Examples

* `flatpack:fixed:foo.pzmap.xml` creates a fixed-width endpoint using
the `foo.pzmap.xml` file configuration.
* `flatpack:delim:bar.pzmap.xml` creates a delimited endpoint using the
`bar.pzmap.xml` file configuration.
* `flatpack:foo` creates a delimited endpoint called `foo` with no file
configuration.

== Message Headers

Camel will store the following headers on the IN message:

[width="100%",cols="50%,50%",options="header",]
|=======================================================================
|Header |Description

|`camelFlatpackCounter` |The current row index. For `splitRows=false` the counter is the total
number of rows.
|=======================================================================

== Message Body

The component delivers the data in the IN message as a
`org.apache.camel.component.flatpack.DataSetList` object that has
converters for `java.util.Map` or `java.util.List`. +
 Usually you want the `Map` if you process one row at a time
(`splitRows=true`). Use `List` for the entire content
(`splitRows=false`), where each element in the list is a `Map`. +
 Each `Map` contains the key for the column name and its corresponding
value.

For example to get the firstname from the sample below:

[source,java]
------------------------------------------------
  Map row = exchange.getIn().getBody(Map.class);
  String firstName = row.get("FIRSTNAME");
------------------------------------------------

However, you can also always get it as a `List` (even for
`splitRows=true`). The same example:

[source,java]
---------------------------------------------------
  List data = exchange.getIn().getBody(List.class);
  Map row = (Map)data.get(0);
  String firstName = row.get("FIRSTNAME");
---------------------------------------------------

== Header and Trailer records

The header and trailer notions in Flatpack are supported. However, you
*must* use fixed record IDs:

* `header` for the header record (must be lowercase)
* `trailer` for the trailer record (must be lowercase)

The example below illustrates this fact that we have a header and a
trailer. You can omit one or both of them if not needed.

[source,xml]
---------------------------------------------------------------------------
    <RECORD id="header" startPosition="1" endPosition="3" indicator="HBT">
        <COLUMN name="INDICATOR" length="3"/>
        <COLUMN name="DATE" length="8"/>
    </RECORD>

    <COLUMN name="FIRSTNAME" length="35" />
    <COLUMN name="LASTNAME" length="35" />
    <COLUMN name="ADDRESS" length="100" />
    <COLUMN name="CITY" length="100" />
    <COLUMN name="STATE" length="2" />
    <COLUMN name="ZIP" length="5" />

    <RECORD id="trailer" startPosition="1" endPosition="3" indicator="FBT">
        <COLUMN name="INDICATOR" length="3"/>
        <COLUMN name="STATUS" length="7"/>
    </RECORD>
---------------------------------------------------------------------------

== Using the endpoint

A common use case is sending a file to this endpoint for further
processing in a separate route. For example:

[source,xml]
-----------------------------------------------------------------------
  <camelContext xmlns="http://activemq.apache.org/camel/schema/spring">
    <route>
      <from uri="file://someDirectory"/>
      <to uri="flatpack:foo"/>
    </route>

    <route>
      <from uri="flatpack:foo"/>
      ...
    </route>
  </camelContext>
-----------------------------------------------------------------------

You can also convert the payload of each message created to a `Map` for
easy Bean Integration

== Flatpack DataFormat

The xref:flatpack-component.adoc[Flatpack] component ships with the Flatpack data
format that can be used to format between fixed width or delimited text
messages to a `List` of rows as `Map`.

* marshal = from `List<Map<String, Object>>` to `OutputStream` (can be
converted to `String`)
* unmarshal = from `java.io.InputStream` (such as a `File` or `String`)
to a `java.util.List` as an
`org.apache.camel.component.flatpack.DataSetList` instance. +
 The result of the operation will contain all the data. If you need to
process each row one by one you can split the exchange, using
Splitter.

*Notice:* The Flatpack library does currently not support header and
trailers for the marshal operation.

== Options

The data format has the following options:

[width="100%",cols="10%,10%,80%",options="header",]
|=======================================================================
|Option |Default |Description

|`definition` |`null` |The flatpack pzmap configuration file. Can be omitted in simpler
situations, but its preferred to use the pzmap.

|`fixed` |`false` |Delimited or fixed.

|`ignoreFirstRecord` |`true` |Whether the first line is ignored for delimited files (for the column
headers).

|`textQualifier` |`"` |If the text is qualified with a char such as `"`.

|`delimiter` |`,` |The delimiter char (could be `;` `,` or similar)

|`parserFactory` |`null` |Uses the default Flatpack parser factory.

|`allowShortLines` |`false` |Allows for lines to be shorter than
expected and ignores the extra characters.

|`ignoreExtraColumns` |`false` |Allows for lines to be longer than
expected and ignores the extra characters.
|=======================================================================

== Usage

To use the data format, simply instantiate an instance and invoke the
marshal or unmarshal operation in the route builder:

[source,java]
---------------------------------------------------------------------------
  FlatpackDataFormat fp = new FlatpackDataFormat();
  fp.setDefinition(new ClassPathResource("INVENTORY-Delimited.pzmap.xml"));
  ...
  from("file:order/in").unmarshal(df).to("seda:queue:neworder");
---------------------------------------------------------------------------

The sample above will read files from the `order/in` folder and
unmarshal the input using the Flatpack configuration file
`INVENTORY-Delimited.pzmap.xml` that configures the structure of the
files. The result is a `DataSetList` object we store on the SEDA queue.

[source,java]
-----------------------------------------------------------------------------------
FlatpackDataFormat df = new FlatpackDataFormat();
df.setDefinition(new ClassPathResource("PEOPLE-FixedLength.pzmap.xml"));
df.setFixed(true);
df.setIgnoreFirstRecord(false);

from("seda:people").marshal(df).convertBodyTo(String.class).to("jms:queue:people");
-----------------------------------------------------------------------------------

In the code above we marshal the data from a Object representation as a
`List` of rows as `Maps`. The rows as `Map` contains the column name as
the key, and the corresponding value. This structure can be created
in Java code from e.g. a processor. We marshal the data according to the
Flatpack format and convert the result as a `String` object and store it
on a JMS queue.

== Dependencies

To use Flatpack in your camel routes you need to add the a dependency on
*camel-flatpack* which implements this data format.

If you use maven you could just add the following to your pom.xml,
substituting the version number for the latest & greatest release (see
the download page for the latest versions).

[source,xml]
-----------------------------------------
<dependency>
  <groupId>org.apache.camel</groupId>
  <artifactId>camel-flatpack</artifactId>
  <version>x.x.x</version>
</dependency>
-----------------------------------------


include::spring-boot:partial$starter.adoc[]
