= FTP Component
:doctitle: FTP
:shortname: ftp
:artifactid: camel-ftp
:description: Upload and download files to/from FTP servers.
:since: 1.1
:supportlevel: Stable
:component-header: Both producer and consumer are supported
//Manually maintained attributes
:camel-spring-boot-name: ftp

*Since Camel {since}*

*{component-header}*

This component provides access to remote file systems over the FTP and
SFTP protocols.

When consuming from remote FTP server make sure you read the section titled _Default when consuming files_
further below for details related to consuming files.

Absolute path is *not* supported. Camel translates absolute path to relative by trimming all
leading slashes from `directoryname`. There'll be WARN message printed
in the logs.

Maven users will need to add the following dependency to their `pom.xml`
for this component:

[source,xml]
----
<dependency>
    <groupId>org.apache.camel</groupId>
    <artifactId>camel-ftp</artifactId>
    <version>x.x.x</version>See the documentation of the Apache Commons
    <!-- use the same version as your Camel core version -->
</dependency>
----


== URI format

----
ftp://[username@]hostname[:port]/directoryname[?options]
sftp://[username@]hostname[:port]/directoryname[?options]
ftps://[username@]hostname[:port]/directoryname[?options]
----

Where *directoryname* represents the underlying directory. The directory
name is a relative path. Absolute path's is *not* supported. The
relative path can contain nested folders, such as /inbox/us.

The `autoCreate` option is supported. When consumer
starts, before polling is scheduled, there's additional FTP operation
performed to create the directory configured for endpoint. The default
value for `autoCreate` is `true`.

If no *username* is provided, then `anonymous` login is attempted using
no password. +
 If no *port* number is provided, Camel will provide default values
according to the protocol (ftp = 21, sftp = 22, ftps = 2222).

You can append query options to the URI in the following format,
`?option=value&option=value&...`

This component uses two different libraries for the actual FTP work. FTP
and FTPS uses http://commons.apache.org/net/[Apache Commons Net] while
SFTP uses http://www.jcraft.com/jsch/[JCraft JSCH].

FTPS (also known as FTP Secure) is an extension to FTP that adds
support for the Transport Layer Security (TLS) and the Secure Sockets
Layer (SSL) cryptographic protocols.



// component-configure options: START

// component-configure options: END

// component options: START
include::partial$component-configure-options.adoc[]
include::partial$component-endpoint-options.adoc[]
// component options: END

// endpoint options: START

// endpoint options: END
// component headers: START
include::partial$component-endpoint-headers.adoc[]
// component headers: END

== FTPS component default trust store

When using the `ftpClient.` properties related to SSL with the FTPS
component, the trust store accept all certificates. If you only want
trust selective certificates, you have to configure the trust store with
the `ftpClient.trustStore.xxx` options or by configuring a custom
`ftpClient`.

When using `sslContextParameters`, the trust store is managed by the
configuration of the provided SSLContextParameters instance.

You can configure additional options on the `ftpClient` and
`ftpClientConfig` from the URI directly by using the `ftpClient.` or
`ftpClientConfig.` prefix.

For example to set the `setDataTimeout` on the `FTPClient` to 30 seconds
you can do:

[source,java]
----
from("ftp://foo@myserver?password=secret&ftpClient.dataTimeout=30000").to("bean:foo");
----

You can mix and match and have use both prefixes, for example to
configure date format or timezones.

[source,java]
----
from("ftp://foo@myserver?password=secret&ftpClient.dataTimeout=30000&ftpClientConfig.serverLanguageCode=fr").to("bean:foo");
----

You can have as many of these options as you like.

See the documentation of the Apache Commons FTP FTPClientConfig for
possible options and more details. And as well for Apache Commons FTP
FTPClient.

If you do not like having many and long configuration in the url you can
refer to the `ftpClient` or `ftpClientConfig` to use by letting Camel
lookup in the Registry for it.

For example:

[source,java]
----
   <bean id="myConfig" class="org.apache.commons.net.ftp.FTPClientConfig">
       <property name="lenientFutureDates" value="true"/>
       <property name="serverLanguageCode" value="fr"/>
   </bean>
----

And then let Camel lookup this bean when you use the # notation in the
url.

[source,java]
----
from("ftp://foo@myserver?password=secret&ftpClientConfig=#myConfig").to("bean:foo");
----


== Examples

----
ftp://someone@someftpserver.com/public/upload/images/holiday2008?password=secret&binary=true

ftp://someoneelse@someotherftpserver.co.uk:12049/reports/2008/password=secret&binary=false

ftp://publicftpserver.com/download
----

== Concurrency

FTP Consumer does not support concurrency

The FTP consumer (with the same endpoint) does not support concurrency
(the backing FTP client is not thread safe). +
 You can use multiple FTP consumers to poll from different endpoints. It
is only a single endpoint that does not support concurrent consumers.

The FTP producer does *not* have this issue, it supports concurrency.

== More information

This component is an extension of the File component.
So there are more samples and details on the File
component page.

== Default when consuming files

The FTP consumer will by default leave the consumed
files untouched on the remote FTP server. You have to configure it
explicitly if you want it to delete the files or move them to another
location. For example you can use `delete=true` to delete the files, or
use `move=.done` to move the files into a hidden done sub directory.

The regular File consumer is different as it will by
default move files to a `.camel` sub directory. The reason Camel does
*not* do this by default for the FTP consumer is that it may lack
permissions by default to be able to move or delete files.

=== limitations

The option *readLock* can be used to force Camel *not* to consume files
that is currently in the progress of being written. However, this option
is turned off by default, as it requires that the user has write access.
See the options table at File2 for more details about
read locks. +
 There are other solutions to avoid consuming files that are currently
being written over FTP; for instance, you can write to a temporary
destination and move the file after it has been written.

When moving files using `move` or `preMove` option the files are
restricted to the FTP_ROOT folder. That prevents you from moving files
outside the FTP area. If you want to move files to another area you can
use soft links and move files into a soft linked folder.

== Exchange Properties

Camel sets the following exchange properties

[width="100%",cols="50%,50%",options="header",]
|=======================================================================
|Header |Description

|`CamelBatchIndex` |Current index out of total number of files being consumed in this batch.

|`CamelBatchSize` |Total number of files being consumed in this batch.

|`CamelBatchComplete` |True if there are no more files in this batch.
|=======================================================================

== About timeouts

The two set of libraries (see top) has different API for setting
timeout. You can use the `connectTimeout` option for both of them to set
a timeout in millis to establish a network connection. An individual
`soTimeout` can also be set on the FTP/FTPS, which corresponds to using
`ftpClient.soTimeout`. Notice SFTP will automatically use
`connectTimeout` as its `soTimeout`. The `timeout` option only applies
for FTP/FTPS as the data timeout, which corresponds to the
`ftpClient.dataTimeout` value. All timeout values are in millis.

== Using Local Work Directory

Camel supports consuming from remote FTP servers and downloading the
files directly into a local work directory. This avoids reading the
entire remote file content into memory as it is streamed directly into
the local file using `FileOutputStream`.

Camel will store to a local file with the same name as the remote file,
though with `.inprogress` as extension while the file is being
downloaded. Afterwards, the file is renamed to remove the `.inprogress`
suffix. And finally, when the Exchange is complete
the local file is deleted.

So if you want to download files from a remote FTP server and store it
as files then you need to route to a file endpoint such as:

[source,java]
----
from("ftp://someone@someserver.com?password=secret&localWorkDirectory=/tmp").to("file://inbox");
----

[TIP]
====
The route above is ultra efficient as it avoids reading the entire file content into memory.
It will download the remote file directly to a local file stream.
The `java.io.File` handle is then used as the Exchange body. The file producer leverages this fact and can work directly on the work file `java.io.File` handle and perform a `java.io.File.rename` to the target filename.
As Camel knows it's a local work file, it can optimize and use a rename instead of a file copy, as the work file is meant to be deleted anyway.
====

== Stepwise changing directories

Camel FTP can operate in two modes in terms of
traversing directories when consuming files (eg downloading) or
producing files (eg uploading)

* stepwise
* not stepwise

You may want to pick either one depending on your situation and security
issues. Some Camel end users can only download files if they use
stepwise, while others can only download if they do not.

You can use the `stepwise` option to control the behavior.

Note that stepwise changing of directory will in most cases only work
when the user is confined to it's home directory and when the home
directory is reported as `"/"`.

The difference between the two of them is best illustrated with an
example. Suppose we have the following directory structure on the remote
FTP server we need to traverse and download files:

----
/
/one
/one/two
/one/two/sub-a
/one/two/sub-b
----

And that we have a file in each of sub-a (a.txt) and sub-b (b.txt)
folder.

== Using stepwise=true (default mode)

----
TYPE A
200 Type set to A
PWD
257 "/" is current directory.
CWD one
250 CWD successful. "/one" is current directory.
CWD two
250 CWD successful. "/one/two" is current directory.
SYST
215 UNIX emulated by FileZilla
PORT 127,0,0,1,17,94
200 Port command successful
LIST
150 Opening data channel for directory list.
226 Transfer OK
CWD sub-a
250 CWD successful. "/one/two/sub-a" is current directory.
PORT 127,0,0,1,17,95
200 Port command successful
LIST
150 Opening data channel for directory list.
226 Transfer OK
CDUP
200 CDUP successful. "/one/two" is current directory.
CWD sub-b
250 CWD successful. "/one/two/sub-b" is current directory.
PORT 127,0,0,1,17,96
200 Port command successful
LIST
150 Opening data channel for directory list.
226 Transfer OK
CDUP
200 CDUP successful. "/one/two" is current directory.
CWD /
250 CWD successful. "/" is current directory.
PWD
257 "/" is current directory.
CWD one
250 CWD successful. "/one" is current directory.
CWD two
250 CWD successful. "/one/two" is current directory.
PORT 127,0,0,1,17,97
200 Port command successful
RETR foo.txt
150 Opening data channel for file transfer.
226 Transfer OK
CWD /
250 CWD successful. "/" is current directory.
PWD
257 "/" is current directory.
CWD one
250 CWD successful. "/one" is current directory.
CWD two
250 CWD successful. "/one/two" is current directory.
CWD sub-a
250 CWD successful. "/one/two/sub-a" is current directory.
PORT 127,0,0,1,17,98
200 Port command successful
RETR a.txt
150 Opening data channel for file transfer.
226 Transfer OK
CWD /
250 CWD successful. "/" is current directory.
PWD
257 "/" is current directory.
CWD one
250 CWD successful. "/one" is current directory.
CWD two
250 CWD successful. "/one/two" is current directory.
CWD sub-b
250 CWD successful. "/one/two/sub-b" is current directory.
PORT 127,0,0,1,17,99
200 Port command successful
RETR b.txt
150 Opening data channel for file transfer.
226 Transfer OK
CWD /
250 CWD successful. "/" is current directory.
QUIT
221 Goodbye
disconnected.
----

As you can see when stepwise is enabled, it will traverse the directory
structure using CD xxx.

== Using stepwise=false

----
230 Logged on
TYPE A
200 Type set to A
SYST
215 UNIX emulated by FileZilla
PORT 127,0,0,1,4,122
200 Port command successful
LIST one/two
150 Opening data channel for directory list
226 Transfer OK
PORT 127,0,0,1,4,123
200 Port command successful
LIST one/two/sub-a
150 Opening data channel for directory list
226 Transfer OK
PORT 127,0,0,1,4,124
200 Port command successful
LIST one/two/sub-b
150 Opening data channel for directory list
226 Transfer OK
PORT 127,0,0,1,4,125
200 Port command successful
RETR one/two/foo.txt
150 Opening data channel for file transfer.
226 Transfer OK
PORT 127,0,0,1,4,126
200 Port command successful
RETR one/two/sub-a/a.txt
150 Opening data channel for file transfer.
226 Transfer OK
PORT 127,0,0,1,4,127
200 Port command successful
RETR one/two/sub-b/b.txt
150 Opening data channel for file transfer.
226 Transfer OK
QUIT
221 Goodbye
disconnected.
----

As you can see when not using stepwise, there are no CD operation
invoked at all.

== Samples

In the sample below we set up Camel to download all the reports from the
FTP server once every hour (60 min) as BINARY content and store it as
files on the local file system.

And the route using XML DSL:

[source,xml]
----
  <route>
     <from uri="ftp://scott@localhost/public/reports?password=tiger&amp;binary=true&amp;delay=60000"/>
     <to uri="file://target/test-reports"/>
  </route>
----

=== Consuming a remote FTPS server (implicit SSL) and client authentication

[source,java]
----
from("ftps://admin@localhost:2222/public/camel?password=admin&securityProtocol=SSL&implicit=true
      &ftpClient.keyStore.file=./src/test/resources/server.jks
      &ftpClient.keyStore.password=password&ftpClient.keyStore.keyPassword=password")
  .to("bean:foo");
----

=== Consuming a remote FTPS server (explicit TLS) and a custom trust store configuration

[source,java]
----
from("ftps://admin@localhost:2222/public/camel?password=admin&ftpClient.trustStore.file=./src/test/resources/server.jks&ftpClient.trustStore.password=password")
  .to("bean:foo");
----

== Custom filtering

Camel supports pluggable filtering strategies. This strategy it to use
the build in `org.apache.camel.component.file.GenericFileFilter` in
Java. You can then configure the endpoint with such a filter to skip
certain filters before being processed.

In the sample we have built our own filter that only accepts files
starting with report in the filename.

And then we can configure our route using the *filter* attribute to
reference our filter (using `#` notation) that we have defined in the
spring XML file:

[source,xml]
----
   <!-- define our sorter as a plain spring bean -->
   <bean id="myFilter" class="com.mycompany.MyFileFilter"/>

  <route>
    <from uri="ftp://someuser@someftpserver.com?password=secret&amp;filter=#myFilter"/>
    <to uri="bean:processInbox"/>
  </route>
----

== Filtering using ANT path matcher

The ANT path matcher is a filter that is shipped out-of-the-box in the
*camel-spring* jar. So you need to depend on *camel-spring* if you are
using Maven. +
 The reason is that we leverage Spring's
http://static.springsource.org/spring/docs/3.0.x/api/org/springframework/util/AntPathMatcher.html[AntPathMatcher]
to do the actual matching.

The file paths are matched with the following rules:

* `?` matches one character
* `*` matches zero or more characters
* `**` matches zero or more directories in a path

The sample below demonstrates how to use it:

== Using a proxy with SFTP

To use an HTTP proxy to connect to your remote host, you can configure
your route in the following way:

[source,xml]
----
<!-- define our sorter as a plain spring bean -->
<bean id="proxy" class="com.jcraft.jsch.ProxyHTTP">
  <constructor-arg value="localhost"/>
  <constructor-arg value="7777"/>
</bean>

<route>
  <from uri="sftp://localhost:9999/root?username=admin&password=admin&proxy=#proxy"/>
  <to uri="bean:processFile"/>
</route>
----

You can also assign a user name and password to the proxy, if necessary.
Please consult the documentation for `com.jcraft.jsch.Proxy` to discover
all options.

== Setting preferred SFTP authentication method

If you want to explicitly specify the list of authentication methods
that should be used by `sftp` component, use `preferredAuthentications`
option. If for example you would like Camel to attempt to authenticate
with private/public SSH key and fallback to user/password authentication
in the case when no public key is available, use the following route
configuration:

[source,java]
----
from("sftp://localhost:9999/root?username=admin&password=admin&preferredAuthentications=publickey,password").
  to("bean:processFile");
----

== Consuming a single file using a fixed name

When you want to download a single file and knows the file name, you can
use `fileName=myFileName.txt` to tell Camel the name of the file to
download. By default the consumer will still do a FTP LIST command to do
a directory listing and then filter these files based on the `fileName`
option. Though in this use-case it may be desirable to turn off the
directory listing by setting `useList=false`. For example the user
account used to login to the FTP server may not have permission to do a
FTP LIST command. So you can turn off this with `useList=false`, and
then provide the fixed name of the file to download with
`fileName=myFileName.txt`, then the FTP consumer can still download the
file. If the file for some reason does not exist, then Camel will by
default throw an exception, you can turn this off and ignore this by
setting `ignoreFileNotFoundOrPermissionError=true`.

For example to have a Camel route that pickup a single file, and delete
it after use you can do

[source,java]
----
from("ftp://admin@localhost:21/nolist/?password=admin&stepwise=false&useList=false&ignoreFileNotFoundOrPermissionError=true&fileName=report.txt&delete=true")
  .to("activemq:queue:report");
----

Notice that we have used all the options we talked above.

You can also use this with `ConsumerTemplate`. For example to download a
single file (if it exists) and grab the file content as a String type:

[source,java]
----
String data = template.retrieveBodyNoWait("ftp://admin@localhost:21/nolist/?password=admin&stepwise=false&useList=false&ignoreFileNotFoundOrPermissionError=true&fileName=report.txt&delete=true", String.class);
----

== Debug logging

This component has log level *TRACE* that can be helpful if you have
problems.



include::spring-boot:partial$starter.adoc[]
