Apache Camel Framework and S3 Integration

In this blog, I am going to demonstrate how to integrate AWS S3 in your Java application using Apache Camel Framework.

Introduction about S3:

AWS S3 provides developers and IT teams with secure, durable, highly-scalable object storage. Amazon S3 is easy to use, with a simple web services interface to store and retrieve any amount of data from anywhere on the web. With Amazon S3, you pay only for the storage you actually use. There is no minimum fee and no setup cost.

In this section I will demonstrate how to upload and download to the AWS S3 bucker with the working code.

Refer my previous “Apache Camel Framework and SQS Integration” blog for the code execution environment setup. 

Upload file to S3 bucket:

The camel component for the S3 activity is aws-s3 and in this example I given bucket name as camelblogs3. All you need is Camel producer template object and File object. Camel S3 component will take care of rest. You don’t need to learn the AWS Java API’s to send files to S3 buckets.

AmazonS3 client = new AmazonS3Client( new DefaultAWSCredentialsProviderChain() ); 

client.setRegion( Region.getRegion(Regions.US_WEST_2));

SimpleRegistry registry = new SimpleRegistry();

registry.put("client", client);

CamelContext camelContext = new DefaultCamelContext(registry);

ProducerTemplate template = camelContext.createProducerTemplate();

File file = new File( "<INSERT_FILE_LOCATION>");

template.sendBodyAndHeader("aws-s3://camelblogs3?amazonS3Client=#client", file , S3Constants.KEY, file.getName());

Download file from S3 bucket:

Don’t remember to pass “deleteAfterRead=false” parameter in the Camel endpoint. 

AmazonS3 client = new AmazonS3Client( new DefaultAWSCredentialsProviderChain() ); 

client.setRegion( Region.getRegion(Regions.US_WEST_2));

SimpleRegistry registry = new SimpleRegistry();

registry.put("client", client);

CamelContext camelContext = new DefaultCamelContext(registry);

ConsumerTemplate consumer = camelContext.createConsumerTemplate();

S3ObjectInputStream s3inputStream = (S3ObjectInputStream)consumer.receiveBody("aws-s3://camelblogs3?amazonS3Client=#client&deleteAfterRead=false&fileName=hello.txt");

String outPutFile = "<INSER_FILE_LOCATION>";

FileWriter writer = new FileWriter(outPutFile);

String thisLine = "";

InputStream fileStream = new BufferedInputStream( s3inputStream);

BufferedReader buffered = new BufferedReader(new InputStreamReader(fileStream, "UTF-8"));

while ( (thisLine = buffered.readLine() ) != null ){

writer.write(thisLine+'\n');

}

writer.close();