= AWS Managed Streaming for Apache Kafka (MSK) Component
:doctitle: AWS Managed Streaming for Apache Kafka (MSK)
:shortname: aws2-msk
:artifactid: camel-aws2-msk
:description: Manage AWS MSK instances using AWS SDK version 2.x.
:since: 3.1
:supportlevel: Stable
:component-header: Only producer is supported
//Manually maintained attributes
:group: AWS
:camel-spring-boot-name: aws2-msk

*Since Camel {since}*

*{component-header}*

The AWS2 MSK component supports create, run, start, stop and terminate
https://aws.amazon.com/msk/[AWS MSK] instances.

Prerequisites

You must have a valid Amazon Web Services developer account, and be
signed up to use Amazon MSK. More information is available at
https://aws.amazon.com/msk/[Amazon MSK].

== URI Format

-------------------------
aws2-msk://label[?options]
-------------------------

You can append query options to the URI in the following format,
?options=value&option2=value&...

// component-configure options: START

// component-configure options: END

// component options: START
include::partial$component-configure-options.adoc[]
include::partial$component-endpoint-options.adoc[]
// component options: END

// endpoint options: START

// endpoint options: END


Required MSK component options

You have to provide the amazonMskClient in the
Registry or your accessKey and secretKey to access
the https://aws.amazon.com/msk/[Amazon MSK] service.

== Usage

=== Static credentials vs Default Credential Provider

You have the possibility of avoiding the usage of explicit static credentials, by specifying the useDefaultCredentialsProvider option and set it to true.

 - Java system properties - aws.accessKeyId and aws.secretKey
 - Environment variables - AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.
 - Web Identity Token from AWS STS.
 - The shared credentials and config files.
 - Amazon ECS container credentials - loaded from the Amazon ECS if the environment variable AWS_CONTAINER_CREDENTIALS_RELATIVE_URI is set.
 - Amazon EC2 Instance profile credentials. 

For more information about this you can look at https://docs.aws.amazon.com/sdk-for-java/latest/developer-guide/credentials.html[AWS credentials documentation]

// component headers: START
include::partial$component-endpoint-headers.adoc[]
// component headers: END

=== MSK Producer operations

Camel-AWS MSK component provides the following operation on the producer side:

- listClusters
- createCluster
- deleteCluster
- describeCluster

== Examples

=== Producer Examples

- listClusters: this operation will list the available MSK Brokers in AWS

[source,java]
--------------------------------------------------------------------------------
from("direct:listClusters")
    .to("aws2-msk://test?mskClient=#amazonMskClient&operation=listClusters")
--------------------------------------------------------------------------------

- createCluster: this operation will create an MSK Cluster in AWS

[source,java]
--------------------------------------------------------------------------------
from("direct:createCluster")
    .process(new Processor() {
       @Override
       public void process(Exchange exchange) throws Exception {
                exchange.getIn().setHeader(MSK2Constants.CLUSTER_NAME, "test-kafka");
                exchange.getIn().setHeader(MSK2Constants.CLUSTER_KAFKA_VERSION, "2.1.1");
                exchange.getIn().setHeader(MSK2Constants.BROKER_NODES_NUMBER, 2);
                BrokerNodeGroupInfo groupInfo = BrokerNodeGroupInfo.builder().build();
                exchange.getIn().setHeader(MSK2Constants.BROKER_NODES_GROUP_INFO, groupInfo);					
       }
    })
    .to("aws2-msk://test?mskClient=#amazonMskClient&operation=createCluster")
--------------------------------------------------------------------------------

- deleteCluster: this operation will delete an MSK Cluster in AWS

[source,java]
--------------------------------------------------------------------------------
from("direct:deleteCluster")
    .setHeader(MSK2Constants.CLUSTER_ARN, constant("test-kafka"));
    .to("aws2-msk://test?mskClient=#amazonMskClient&operation=deleteCluster")
--------------------------------------------------------------------------------

[source,java]
--------------------------------------------------------------------------------
from("direct:createCluster")
    .process(new Processor() {
       @Override
       public void process(Exchange exchange) throws Exception {
                exchange.getIn().setHeader(MSK2Constants.CLUSTER_NAME, "test-kafka");
                exchange.getIn().setHeader(MSK2Constants.CLUSTER_KAFKA_VERSION, "2.1.1");
                exchange.getIn().setHeader(MSK2Constants.BROKER_NODES_NUMBER, 2);
                BrokerNodeGroupInfo groupInfo = BrokerNodeGroupInfo.builder().build();
                exchange.getIn().setHeader(MSK2Constants.BROKER_NODES_GROUP_INFO, groupInfo);					
       }
    })
    .to("aws2-msk://test?mskClient=#amazonMskClient&operation=deleteCluster")
--------------------------------------------------------------------------------

== Using a POJO as body

Sometimes build an AWS Request can be complex, because of multiple options. We introduce the possibility to use a POJO as body.
In AWS MSK there are multiple operations you can submit, as an example for List clusters request, you can do something like:

[source,java]
------------------------------------------------------------------------------------------------------
from("direct:aws2-msk")
     .setBody(ListClustersRequest.builder().maxResults(10).build())
     .to("aws2-msk://test?mskClient=#amazonMskClient&operation=listClusters&pojoRequest=true")
------------------------------------------------------------------------------------------------------

In this way you'll pass the request directly without the need of passing headers and options specifically related to this operation.

== Dependencies

Maven users will need to add the following dependency to their pom.xml.

*pom.xml*

[source,xml]
---------------------------------------
<dependency>
    <groupId>org.apache.camel</groupId>
    <artifactId>camel-aws2-msk</artifactId>
    <version>${camel-version}</version>
</dependency>
---------------------------------------

where `$\{camel-version}` must be replaced by the actual version of Camel.


include::spring-boot:partial$starter.adoc[]
