Spring Boot AWS SQS Listener Example

In this post, we will see Spring Boot AWS SQS listener example from scratch. First, we will see how to set up AWS management Console(Creating AWS account, get access key/secret key and Creating SQS queue) then we will see Spring Boot AWS SQS listener example from scratch. We will also cover how to create two separate spring boot application to send and receive the message using the AWS and SQS.

Note – At the end of tutorial we have GitHub link, from where you can download the source code.

AWS Management Console setup for Spring Boot AWS SQS listener example.

First, we are going to see below points in order to setup AWS management console.

How to create an AWS account. We have separate tutorial here.

How to get access and secret key, and URL in AWS. See an example here.

How to create SQS Queue in AWS. See an example here.

Once we are done with the above steps we should have an AWS access key, AWS secret key, and AWS endpoint/URL. We are going to use these details later for spring boot AWS SQS configuration.

Sample Access Key – AKIAUOHSUZB6KR1BK3UM

Sample Secret key – MWCnSuDS0FJ8ffOcb8sruF2a6bJvuLzvTg0UyNA3

URL – https://sqs.us-east-2.amazonaws.com/123123418618/javatute-first-sqs

Spring Boot AWS SQS Configuration.

Let’s see how to integrate AWS, SQS and Spring Boot.

maven dependency

<!-- https://mvnrepository.com/artifact/org.springframework.cloud/spring-cloud-starter-aws -->
<dependency>
	<groupId>org.springframework.cloud</groupId>
	<artifactId>spring-cloud-starter-aws</artifactId>
	<version>2.2.1.RELEASE</version>
</dependency>

<!-- https://mvnrepository.com/artifact/org.springframework.cloud/spring-cloud-starter-aws-messaging -->
<dependency>
	<groupId>org.springframework.cloud</groupId>
	<artifactId>spring-cloud-starter-aws-messaging</artifactId>
	<version>2.2.1.RELEASE</version>
</dependency>

application.yml

server:
   port: 9091
   
cloud:
   aws:
      region:
         static: us-east-2
         auto: false
      credentials:
         access-key: AKIBJDHSUZB5AR2BK3ULO
         secret-key: AWCnSuWZ0FJ8ffOcb4sruF2a7bVvuLzvTg0UyNZ39
      end-point:
         uri: https://sqs.us-east-2.amazonaws.com/419268184186/javatute-first-sqs

application.properties

server.port = 9091
cloud.aws.credentials.access-key = AKIBJDHSUZB5AR2BK3ULO
cloud.aws.credentials.secret-key = AWCnSuWZ0FJ8ffOcb4sruF2a7bVvuLzvTg0UyNZ39
cloud.aws.region.static = us-east-2
cloud.aws.region.auto = false
cloud.aws.end-point.uri = https://sqs.us-east-2.amazonaws.com/456789124186/javatute-first-sqs

AWS, SQS and Spring Boot Integration – Sample Config class.

package com.javatute.config;

import org.springframework.beans.factory.annotation.Value;
import org.springframework.cloud.aws.messaging.core.QueueMessagingTemplate;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.sqs.AmazonSQSAsync;
import com.amazonaws.services.sqs.AmazonSQSAsyncClientBuilder;

@Configuration
public class SpringCloudSQSConfig {
	@Value("${cloud.aws.region.static}")
	private String region;

	@Value("${cloud.aws.credentials.access-key}")
	private String accessKey;

	@Value("${cloud.aws.credentials.secret-key}")
	private String secretKey;

	@Bean
	public QueueMessagingTemplate queueMessagingTemplate() {
		return new QueueMessagingTemplate(amazonSQSAsync());
	}

	public AmazonSQSAsync amazonSQSAsync() {

		AmazonSQSAsyncClientBuilder amazonSQSAsyncClientBuilder = AmazonSQSAsyncClientBuilder.standard();
		AmazonSQSAsync amazonSQSAsync = null;
		amazonSQSAsyncClientBuilder.withRegion(Regions.US_EAST_2);
		BasicAWSCredentials basicAWSCredentials = new BasicAWSCredentials(accessKey, secretKey);
		amazonSQSAsyncClientBuilder.withCredentials(new AWSStaticCredentialsProvider(basicAWSCredentials));
		amazonSQSAsync = amazonSQSAsyncClientBuilder.build();
		return amazonSQSAsync;

	}

}

Let’s see an aws sqs step by step example.

Spring Boot AWS SQS Listener example from scratch

In this section we will create Spring Boot AWS SQS Listener example using single tomcat. We have a common application where we will define both producer and consumer in same application. See below diagram which tells brief about example.

Open eclipse and create a maven project, Don’t forget to check to ‘create a simple project (skip)’ click on next.  Fill all details(GroupId – spring-boot-aws-sqs-listener-example , ArtifactId – spring-boot-aws-sqs-listener-example , and name – spring-boot-aws-sqs-listener-example) and click on finish. Keep packaging as the jar.

Modify pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<groupId>spring-boot-aws-sqs-listener-example</groupId>
	<artifactId>spring-boot-aws-sqs-listener-example</artifactId>
	<version>0.0.1-SNAPSHOT</version>
	<name>spring-boot-aws-sqs-listener-example</name>
	<description>spring-boot-aws-sqs-listener-example</description>

	<parent>
		<groupId>org.springframework.boot</groupId>
		<artifactId>spring-boot-starter-parent</artifactId>
		<version>2.0.2.RELEASE</version>
	</parent>
	<dependencies>
		<dependency>
			<groupId>org.springframework.boot</groupId>
			<artifactId>spring-boot-starter-web</artifactId>
		</dependency>

		<!-- https://mvnrepository.com/artifact/org.springframework.cloud/spring-cloud-starter-aws -->
		<dependency>
			<groupId>org.springframework.cloud</groupId>
			<artifactId>spring-cloud-starter-aws</artifactId>
			<version>2.2.1.RELEASE</version>
		</dependency>

		<!-- https://mvnrepository.com/artifact/org.springframework.cloud/spring-cloud-starter-aws-messaging -->
		<dependency>
			<groupId>org.springframework.cloud</groupId>
			<artifactId>spring-cloud-starter-aws-messaging</artifactId>
			<version>2.2.1.RELEASE</version>
		</dependency>
	</dependencies>
</project>

Directory structure of Spring Boot AWS SQS Listener Example.

Create a class Student.java.

Note – Since in this example we are sending message as json object, we need POJO class.

package com.javatute.dto;

public class Student {
	private Long id;
	private String name;
	private String rollNumber;

	public Long getId() {
		return id;
	}

	public void setId(Long id) {
		this.id = id;
	}

	public String getName() {
		return name;
	}

	public void setName(String name) {
		this.name = name;
	}

	public String getRollNumber() {
		return rollNumber;
	}

	public void setRollNumber(String rollNumber) {
		this.rollNumber = rollNumber;
	}

}

Create producer.java class.

package com.javatute.producer;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.cloud.aws.messaging.core.QueueMessagingTemplate;
import org.springframework.messaging.support.MessageBuilder;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

import com.fasterxml.jackson.databind.ObjectMapper;
import com.javatute.dto.Student;

@RestController
@RequestMapping("/produce")
public class Producer {

	private static final Logger logger = LoggerFactory.getLogger(Producer.class);

	@Autowired
	private QueueMessagingTemplate queueMessagingTemplate;

	@Value("${cloud.aws.end-point.uri}")
	private String endPoint;

	@PostMapping("/message")
	public Student sendMessage(@RequestBody Student student) {

		try {
			ObjectMapper mapper = new ObjectMapper();
			String jsonString = mapper.writeValueAsString(student);
			queueMessagingTemplate.send(endPoint, MessageBuilder.withPayload(jsonString).build());
			logger.info("Message sent successfully  " + jsonString);
		} catch (Exception e) {
			e.printStackTrace();
		}
		return student;
	}

}

We have defined a Rest API to send the message. We are using the ObjectMapper class to convert student objects to JSON string that we will send later this message to AWS SQS Queue.

To send the message we will use queueMessagingTemplate.send() method.

Create Spring Boot aws sqs listener class.

package com.javatute.consumer;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.cloud.aws.messaging.listener.annotation.SqsListener;
import org.springframework.stereotype.Component;

@Component
public class Consumer {

	private static final Logger logger = LoggerFactory.getLogger(Consumer.class);

	@SqsListener("javatute-first-sqs")
	public void receiveMessage(String stringJson) {

		logger.info("Message Received using SQS Listner " + stringJson);

	}
}

We are using @SqsListener annotation with our listener method receiveMessage(). With @SqsListener annotation we use queue name or queue URL. As soon as the message will get uploaded using producer class, receiveMessage() will receive a message from AWS SQS.

Define SpringCloudSQSConfig.java

package com.javatute.config;

import org.springframework.beans.factory.annotation.Value;
import org.springframework.cloud.aws.messaging.core.QueueMessagingTemplate;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

import com.amazonaws.auth.AWSStaticCredentialsProvider;
import com.amazonaws.auth.BasicAWSCredentials;
import com.amazonaws.regions.Regions;
import com.amazonaws.services.sqs.AmazonSQSAsync;
import com.amazonaws.services.sqs.AmazonSQSAsyncClientBuilder;

@Configuration
public class SpringCloudSQSConfig {
	@Value("${cloud.aws.region.static}")
	private String region;

	@Value("${cloud.aws.credentials.access-key}")
	private String accessKey;

	@Value("${cloud.aws.credentials.secret-key}")
	private String secretKey;

	@Bean
	public QueueMessagingTemplate queueMessagingTemplate() {
		return new QueueMessagingTemplate(amazonSQSAsync());
	}

	public AmazonSQSAsync amazonSQSAsync() {

		AmazonSQSAsyncClientBuilder amazonSQSAsyncClientBuilder = AmazonSQSAsyncClientBuilder.standard();
		AmazonSQSAsync amazonSQSAsync = null;
		amazonSQSAsyncClientBuilder.withRegion(Regions.US_EAST_2);
		BasicAWSCredentials basicAWSCredentials = new BasicAWSCredentials(accessKey, secretKey);
		amazonSQSAsyncClientBuilder.withCredentials(new AWSStaticCredentialsProvider(basicAWSCredentials));
		amazonSQSAsync = amazonSQSAsyncClientBuilder.build();
		return amazonSQSAsync;

	}

}

We have already defined application.yml and application.properties file at the beginning of the tutorial. Let’s see once again here to easy to understand.

Define the application.yml or application.properties file in resources folder.

server:
   port: 9091
   
cloud:
   aws:
      region:
         static: us-east-2
         auto: false
      credentials:
         access-key: AKIAWDHSUZB5KR2BK3UL
         secret-key: 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
      end-point:
         uri: https://sqs.us-east-2.amazonaws.com/419268184186/javatute-first-sqs

Create SpringMain class SpringMain.java

package com.javatute.main;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.cloud.aws.autoconfigure.context.ContextStackAutoConfiguration;
import org.springframework.context.annotation.ComponentScan;

@SpringBootApplication(exclude = { ContextStackAutoConfiguration.class })
@ComponentScan(basePackages = "com.javatute.*")
public class SpringMain {
	public static void main(String[] args) {
		SpringApplication.run(SpringMain.class, args);
	}

}

How to set up AWS Credentials(access key and secret key) environment variable in the windows.

You may get below error if you run above example.

Caused by: com.amazonaws.SdkClientException: Unable to load AWS credentials from any provider in the chain: [com.amazonaws.auth.EC2ContainerCredentialsProviderWrapper@2d9f64c9: Unable to load credentials from service endpoint, com.amazonaws.auth.profile.ProfileCredentialsProvider@21ac5eb4: profile file cannot be null]
at com.amazonaws.auth.AWSCredentialsProviderChain.getCredentials(AWSCredentialsProviderChain.java:136) ~[aws-java-sdk-core-1.11.415.jar:na]
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.getCredentialsFromContext(AmazonHttpClient.java:1186) ~[aws-java-sdk-core-1.11.415.jar:na]

If you running spring boot aws sqs listener application for the first time then you need to set up AWS Credentials(access key and secret key) in local machine. Let’s see how to set up aws credentials in windows.

Step 1 – Create credentials file in below directory.

C:\Users\USERNAME\.aws\credentials

Note – Create .aws folder inside your username folder. For example i have created credentials file in below location.

Step 2 – In credentials file we need to add profile, access key, and secret key as below.

[default]
aws_access_key_id=BKIAWDHSUZA5KR2BK3UL
aws_secret_access_key=NWCnSuWZ0FJ8ffOcb0sruF2a6bVvuLzvTg0UyNZ2

Step 3 – Set access-key and secret-key in environment variable.

Testing of example.

Now we can run out application. Run SpringMain class and deploy the application.

Use below API to send the message.

http://localhost:9091/produce/message

{
    "id": 10,
    "name": "john",
    "rollNumber": "0126CS01"
}

Spring Boot AWS SQS Listener example using two separate instances(Tomcat)

In this section, we will see how to implement producer/sender and consumer/receiver as a separate application. We will create two separate spring boot applications. One is the producer(send the message to SQS queue) and another is consumer(receive a message from SQS queue).

Creating first application – Producer.

In eclipse create a maven project, Don’t forget to check to ‘create a simple project (skip)’ click on next.  Fill all details(GroupId – spring-boot-aws-sqs-producer-example , ArtifactId – spring-boot-aws-sqs-producer-example , and name – spring-boot-aws-sqs-producer-example) and click on finish. Keep packaging as the jar.

pom.xml would be same as above example.

Directory sturcture of spring-boot-aws-sqs-producer-example.

Did you notice we don’t have a consumer folder in the directory structure? We will have a separate application for the consumer. we are going to see soon.

Create Student.java class.

package com.javatute.dto;
public class Student {
	private Long id;
	private String name;
	private String rollNumber;
        //getter & setter
}

SpringCloudSQSConfig.java – Same as above example.

Producer.java – Same as previous example.

apllication.yml – This will also no changes need.

SpringMain.java – Same as previous example.

Let’s run our first application i.e SpringMain.java.

Send message using postman.

http://localhost:9091/produce/message

Obserb the console. We should able to see Message sent successfully in json format.

We have not created a consumer yet. Our message should be in the AWS SQS queue. Once we will create consumer our message will be consumed automatically using consumer listeners. Let’s login to AWS and check the SQS Queue.

Search for Simple Queue Service and click on simple queue service.

We can see our queue name and also we can message available section currently it showing 1. Now Click on our SQS queue.

We can see send and receive message option. Click on that.

Click on poll for messages option.

Once you click on poll for messages option, we should able to see messages(highlighted as below). Click on message.

Click on body section, we should able to see our message.

By defalut, this message will be available for 4 days in Standard SQS Queue.

Let’s create consumer listner (another application) which will consume this message from AWS SQS Queue.

Creating second application – Consumer.

Let’s create consumer.

In eclipse create a maven project, Don’t forget to check to ‘create a simple project (skip)’ click on next.  Fill all details(GroupId – spring-boot-aws-sqs-consumer-example , ArtifactId – spring-boot-aws-sqs-consumer-example , and name – spring-boot-aws-sqs-consumer-example) and click on finish. Keep packaging as the jar.

pom.xml would be same.

Directory sturcture of spring-boot-aws-sqs-consumer-example.

Observe the above directory structure. We have configuration class(i.e SpringCloudSQSConfig.java), Consumer.java class, and SpringMain.java class. Since we are going to receive the message in the form of JSON String, we no need the Student class.

SpringCloudSQSConfig.java would be same.

Consumer.java

package com.javatute.consumer;

import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.cloud.aws.messaging.listener.annotation.SqsListener;
import org.springframework.stereotype.Component;

@Component
public class Consumer {

	private static final Logger logger = LoggerFactory.getLogger(Consumer.class);

	@SqsListener("javatute-first-sqs")
	public void recieveMessage(String stringJson) {

		logger.info("Message Received using SQS Listner " + stringJson);

	}
}

application.yml – same as the above example only change we need to change the port number. Since on 9091, our first application is running. Let’s change it to 9092.

server:
   port: 9092
   
cloud:
   aws:
      region:
         static: us-east-2
         auto: false
      credentials:
         access-key: AKIAWDHSUZB5KR2BK3UL
         secret-key: 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
      end-point:
         uri: https://sqs.us-east-2.amazonaws.com/419268184186/javatute-first-sqs

SpringMain.java same as previous example.

Most of code would be the same. Only we have two separate application one for producer and another for consumer.

Let’s run the SpringMain.java class and deploy out application on embeded tomcat.

Obserb the console, we will immedieatly receive message from AWS SQS queue.

Yes ! Our second apllication that is running on separate tomcat have received message successfuly.

Note – We are using embedded tomcat in this example. We can deploy our application on separate tomcat. See a step by step example here which tells how to deploy spring boot application on external tomcat.

Basic points about AWS SQS.

We have seen how to create AWS SQS Queue in a separate tutorial. We also covered how to see a message in the AWS SQS queue that we sent using our Spring Boot application. Let’s see some more details about the SQS.

SQS(Simple Queue Service) is distributed, reliable and fully managed AWS message queue service.

The default message retention period is 4 days. AWS SQS deletes messages that is in Queue for more that 4 days.

We can configure AWS SQS message retention period from 1 minute to 14 days using setQueueAttributes() method.

SetQueueAttributesRequest setQueueAttributesRequest = new SetQueueAttributesRequest();
Map<String, String> attributes = new HashMap();
setQueueAttributesRequest.setAttributes(attributes);

See more about SetQueueAttributes() docs here.

There are two types of Queue.

1. Standard Queue.
2. FIFO Queue.

Standard Queue – This is default AWS SQS Queue type. Standard Queue supports unlimited number of transactions per second per Action.

In Standard Queue message is delivered at least once. It doesn’t gaurantee that message will sent in same order.

We can have duplicate messages in case of Standard Queue.

FIFO Queue – As name suggest in case of FIFO queue, messages will maintain first in first out order.

We can’t have duplicate message in case of FIFO Queue.

See more about Standard and FIFO AWS SQS Queue here(docs).

That’s all about Spring Boot AWS SQS Listener Example. Feel free to leave comment in case of any doubt.

You may like.

• Spring Boot CRUD Example With MySQL/PostgreSQL.
• Hazelcast Cache Spring Boot Example.
• Spring Data JPA Interview Questions and Answers.
• Hibernate First Level Cache example using Spring Boot.
• JPA EntityManager CRUD example Using Spring Boot.
• Hibernate/JPA EhCache Configuration Example.

Spring Data JPA.

• Spring Data JPA greater than Example
• Spring Data JPA less than Example
• Spring Data JPA IsNull Example Using Spring Boot
• Spring Data findById() Vs getOne()
• Spring Data JPA CrudRepository findById()
• Spring Data JPA JpaRepository getOne()
• Spring Data CrudRepository saveAll() and findAll().
• Spring Data CrudRepository existsById()
• Spring Data JPA delete() vs deleteInBatch()
• Spring Data JPA deleteAll() Vs deleteAllInBatch()
• Spring Data JPA JpaRepository deleteAllInBatch()
• Spring Data JPA deleteInBatch() Example
• Spring Data JPA JpaRepository saveAndFlush() Example
• Spring Data JPA CrudRepository count() Example
• Spring Data JPA CrudRepository delete() and deleteAll()
• Spring Data JPA CrudRepository deleteById() Example
• CrudRepository findAllById() Example Using Spring Boot
• Spring Data CrudRepository save() Method.
• Sorting in Spring Data JPA using Spring Boot.
• Spring Data JPA example using spring boot.
• Spring Data JPA and its benefit.

Spring Boot AWS SQS Listner example Spring docs.

Summary – We have seen about Spring Boot AWS SQS Listener Example. We covered how to create a producer using AWS SQS. We learned how to implement AWS SQS listeners to consume messages. We also covered how to create two separate applications to send and receive the message.

Download the Source Code of Spring Boot AWS SQS Listener Example from GitHub.