Spring Boot Elasticsearch

Filed Under: Spring

Spring Boot Elasticsearch 6

In this post, we will setup up a sample Spring boot Elasticsearch application. We will use latest version of Elasticsearch i.e. 6.1.x. To interact with the Elasticsearch search engine, we will use Elasticsearch Rest client. We are not using Spring Data ElasticSearch because that doesn’t support latest ElasticSearch version i.e 6.x.

Spring Boot ElasticSearch

Setting up the project

Maven Dependencies

We will use Maven build system for this project and here are the dependencies we used:


<parent>
  <groupId>org.springframework.boot</groupId>
  <artifactId>spring-boot-starter-parent</artifactId>
  <version>1.5.9.RELEASE</version>
  <relativePath/>
  <!-- lookup parent from repository -->
</parent>
<properties>
  <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
  <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
  <java.version>1.8</java.version>
</properties>
<dependencies>
  <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter</artifactId>
  </dependency>
  <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
  </dependency>
  <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-test</artifactId>
    <scope>test</scope>
  </dependency>
  <!-- Elasticsearch Dependencies -->
  <dependency>
    <groupId>org.elasticsearch</groupId>
    <artifactId>elasticsearch</artifactId>
    <version>6.1.2</version>
  </dependency>
  <dependency>
    <groupId>org.elasticsearch.client</groupId>
    <artifactId>elasticsearch-rest-high-level-client</artifactId>
    <version>6.1.2</version>
  </dependency>
  <dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-api</artifactId>
    <version>2.8.2</version>
  </dependency>
  <dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-core</artifactId>
    <version>2.8.2</version>
  </dependency>
  <dependency>
    <groupId>org.elasticsearch.client</groupId>
    <artifactId>elasticsearch-rest-client</artifactId>
    <version>6.1.2</version>
  </dependency>
  <dependency>
    <groupId>org.elasticsearch.client</groupId>
    <artifactId>elasticsearch-rest-client-sniffer</artifactId>
    <version>6.1.2</version>
  </dependency>
</dependencies>
<build>
  <plugins>
    <plugin>
      <groupId>org.springframework.boot</groupId>
      <artifactId>spring-boot-maven-plugin</artifactId>
    </plugin>
  </plugins>
</build>

Make sure to use stable version for Spring Boot from the maven central.

Elasticsearch Configuration

Now, we will have to configure ElasticSearch in our application. Let’s d this in two parts. First, we will provide Elasticsearch address in our application.properties file:


spring.data.elasticsearch.cluster-name=elasticsearch
spring.data.elasticsearch.cluster-nodes=elasticsearch

We only provided the Elasticsearch cluster name and node name here and these are actually the default values. Now, it’s time to put these values to use in our Java config class.


package com.journaldev.elasticsearch.config;

import org.apache.http.HttpHost;
import org.elasticsearch.client.RestClient;
import org.elasticsearch.client.RestHighLevelClient;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.beans.factory.config.AbstractFactoryBean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class ElasticSearchConfiguration extends AbstractFactoryBean {

    private static final Logger LOG = LoggerFactory.getLogger(ElasticSearchConfiguration.class);
    @Value("${spring.data.elasticsearch.cluster-nodes}")
    private String clusterNodes;
    @Value("${spring.data.elasticsearch.cluster-name}")
    private String clusterName;
    private RestHighLevelClient restHighLevelClient;

    @Override
    public void destroy() {
        try {
            if (restHighLevelClient != null) {
                restHighLevelClient.close();
            }
        } catch (final Exception e) {
            LOG.error("Error closing ElasticSearch client: ", e);
        }
    }

    @Override
    public Class<RestHighLevelClient> getObjectType() {
        return RestHighLevelClient.class;
    }

    @Override
    public boolean isSingleton() {
        return false;
    }

    @Override
    public RestHighLevelClient createInstance() {
        return buildClient();
    }

    private RestHighLevelClient buildClient() {
        try {
            restHighLevelClient = new RestHighLevelClient(
                    RestClient.builder(
                            new HttpHost("localhost", 9200, "http"),
                            new HttpHost("localhost", 9201, "http")));
        } catch (Exception e) {
            LOG.error(e.getMessage());
        }
        return restHighLevelClient;
    }
}

With this configuration, we ensured that ElasticSearch is able to make a successful connection to its server using the Rest Client API.

Working the app

Let’s start putting the working components of the app now.

Model for the app

We will be just using a simple model as a User:


package com.journaldev.elasticsearch.model;

import com.fasterxml.jackson.annotation.JsonInclude;

@JsonInclude(JsonInclude.Include.NON_EMPTY)
public class Book {

    private String id;
    private String title;
    private String author;
    private float price;

    //standard setters and getters
}

We will be making following functionalities and Database interactions in our app:

  • Get a book with ID
  • Insert a Book
  • Update a Book
  • Delete a Book

Defining the Controller

Let us move to making our Controller:


package com.journaldev.elasticsearch.controller;

import com.journaldev.elasticsearch.model.Book;
import com.journaldev.elasticsearch.dao.BookDao;
import org.springframework.web.bind.annotation.*;

import java.util.Map;

@RestController
@RequestMapping("/books")
public class BookController {

    private BookDao bookDao;

    public BookController(BookDao bookDao) {
        this.bookDao = bookDao;
    }
  ...
}

We just Autowired the DAO dependency and we will use this next.

Defining the APIs

For the functionalities we mentioned, we will now be making APIs and accessing the DAO dependency which will internally use Elasticsearch Rest Client API.

Get a book with ID

Let us get a book with ID:


@GetMapping("/{id}")
public Map<String, Object> getBookById(@PathVariable String id){
  return bookDao.getBookById(id);
}

Insert a Book

Now, let us insert a book now:


@PostMapping
public Book insertBook(@RequestBody Book book) throws Exception {
  return bookDao.insertBook(book);
}

Update a book

We will be updating a book in this snippet:


@PutMapping("/{id}")
public Map<String, Object> updateBookById(@RequestBody Book book, @PathVariable String id) {
  return bookDao.updateBookById(id, book);
}

Deleting a Book

Now that we have added sample data into the DB, let’s try to extract some part of it:


@DeleteMapping("/{id}")
public void deleteBookById(@PathVariable String id) {
  bookDao.deleteBookById(id);
}

This is the ease Spring Data API offers us but it also has some downsides. We will elaborate this when we Have defined the ElasticsearchTemplate version as well. Let’s get started with that too.

Defining the DAO layer

Now, we will actually define the DAL queries which achieves these objectives.

Defining DAO

We will start by mentioning the dependencies we need:


package com.journaldev.elasticsearch.dao;

import com.journaldev.elasticsearch.model.Book;
import com.fasterxml.jackson.core.JsonProcessingException;
import com.fasterxml.jackson.databind.ObjectMapper;
import org.elasticsearch.ElasticsearchException;
import org.elasticsearch.action.delete.DeleteRequest;
import org.elasticsearch.action.delete.DeleteResponse;
import org.elasticsearch.action.get.GetRequest;
import org.elasticsearch.action.get.GetResponse;
import org.elasticsearch.action.index.IndexRequest;
import org.elasticsearch.action.index.IndexResponse;
import org.elasticsearch.action.update.UpdateRequest;
import org.elasticsearch.action.update.UpdateResponse;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.common.xcontent.XContentType;
import org.springframework.stereotype.Repository;

import java.util.HashMap;
import java.util.Map;
import java.util.UUID;

@Repository
public class BookDao {

  private final String INDEX = "bookdata";
  private final String TYPE = "books";  
  private RestHighLevelClient restHighLevelClient;
  private ObjectMapper objectMapper;

  public BookDao( ObjectMapper objectMapper, RestHighLevelClient restHighLevelClient) {
    this.objectMapper = objectMapper;
    this.restHighLevelClient = restHighLevelClient;
  }
  ...
}

Insert Query

We will start by inserting a Book into the ES:


public Book insertBook(Book book){
  book.setId(UUID.randomUUID().toString());
  Map dataMap = objectMapper.convertValue(book, Map.class);
  IndexRequest indexRequest = new IndexRequest(INDEX, TYPE, book.getId())
                .source(dataMap);
  try {
    IndexResponse response = restHighLevelClient.index(indexRequest);
  } catch(ElasticsearchException e) {
    e.getDetailedMessage();
  } catch (java.io.IOException ex){
    ex.getLocalizedMessage();
  }
  return book;
}

Clearly, we need to convert the model data to a Map data structure before we insert it into the ES database.

Search Query

We will now search a book with an ID:


public Map<String, Object> getBookById(String id){
  GetRequest getRequest = new GetRequest(INDEX, TYPE, id);
  GetResponse getResponse = null;
  try {
    getResponse = restHighLevelClient.get(getRequest);
  } catch (java.io.IOException e){
    e.getLocalizedMessage();
  }
  Map<String, Object> sourceAsMap = getResponse.getSourceAsMap();
  return sourceAsMap;
}

Here, this is to be noted that data is searched and got back as a Map data structure as well. This is something we need to handle, converting data to Map and forth to make DB transactions.


public Map<String, Object> updateBookById(String id, Book book){
  UpdateRequest updateRequest = new UpdateRequest(INDEX, TYPE, id)
          .fetchSource(true);    // Fetch Object after its update
  Map<String, Object> error = new HashMap<>();
  error.put("Error", "Unable to update book");
  try {
    String bookJson = objectMapper.writeValueAsString(book);
    updateRequest.doc(bookJson, XContentType.JSON);
    UpdateResponse updateResponse = restHighLevelClient.update(updateRequest);
    Map<String, Object> sourceAsMap = updateResponse.getGetResult().sourceAsMap();
    return sourceAsMap;
  }catch (JsonProcessingException e){
    e.getMessage();
  } catch (java.io.IOException e){
    e.getLocalizedMessage();
  }
  return error;
}

Deleting Data

We will now search a book with an ID:


public void deleteBookById(String id) {
  DeleteRequest deleteRequest = new DeleteRequest(INDEX, TYPE, id);
  try {
    DeleteResponse deleteResponse = restHighLevelClient.delete(deleteRequest);
  } catch (java.io.IOException e){
    e.getLocalizedMessage();
  }
}

I think deleting an object was having the easiest query. Let’s try this application now by running it.

Running the application

We can run this app simply by using a single command:


mvn spring-boot:run

Once the app is running, we can try saving a new book by using this API. Before doing that, just confirm that the ES is running by using this API:

127.0.0.1:9200

We will get this response:
elasticsearch installation
Now let’s inserting the data by using following API:

127.0.0.1:8080/books

We use a POST request with this JSON:


{
  "title" : "Java Always",
  "author" : "JournalDev",
  "price" : 99.1
}

We have the following response:
spring elasticsearch example
Let’s try another request to get the book with above ID. We will use GET request on this API:

127.0.0.1:8080/books/55c200ff-9674-44aa-8779-a0f3ff925e74

This is what we get back:
spring elasticsearch 6
Go on and try more APIs we defined.

Summary

In this lesson, we looked at how Elasticsearch can be queried using a Rest Client API it provides. Download the source code here and modify it.

Comments

  1. victor says:

    Thanks Shubham,

    I was searching for an example with a data type and a nested class like this

    public class Consommation implements Serializable{

    public Consommation() {

    }
    @Id
    private String id;
    private int idcons;
    private int indexactuel;
    private int rabais;
    private Date datePaiement;
    @Field(type = FieldType.Nested)
    private Abonne abonne;
    @Field(type = FieldType.Nested)
    private Tarif tarif;
    @Field(type = FieldType.Nested)
    private Tournee tournee;
    }

  2. javadev says:

    What is the latest version of elasticsearch that spring data elasticsearch support today?

  3. Bhargav says:

    Does elasticsearch @Transactional annotations matter ?

  4. André says:

    Mr Shubham, nice example.
    I have one question. Can you explain why the BookDao ist automatically injected during startup with ObjectMapper and RestHighLevelClient?
    Thanks
    André

  5. xiaoxiao says:

    use this integrated style, how to create complex mapping by itself. such
    @MultiField(
    mainField = @Field(type = FieldType.Text, analyzer = “whitespace”),
    otherFields = {
    @InnerField(suffix = “prefix”, type = FieldType.Text, analyzer = “stop”, searchAnalyzer = “standard”)
    }
    )
    if we have complex mapping, it need to be created by ourself. if use spring data elasticsearch, the complex mapping will be created by itself.

  6. Chinthi says:

    And May I know how to get All data from elastic search

    add – ok
    getById – ok
    update – ok
    delete – ok
    findAll – ?

  7. Chinthi says:

    Hi, That is really work men,
    Thanks a lot 🙂

  8. Yellappa says:

    Hi,

    The application works fine. But when I add date, Bean is failing. Please find the below exception.

    org.springframework.beans.factory.BeanCreationException: Error creating bean with name ‘elasticsearchClient’ defined in class path resource [org/springframework/boot/autoconfigure/data/elasticsearch/ElasticsearchAutoConfiguration.class]: Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.elasticsearch.client.Client]: Factory method ‘elasticsearchClient’ threw exception; nested exception is java.lang.NoClassDefFoundError: org/elasticsearch/common/transport/InetSocketTransportAddress
    at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:599) ~[spring-beans-4.3.13.RELEASE.jar:4.3.13.RELEASE]
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1173) ~[spring-beans-4.3.13.RELEASE.jar:4.3.13.RELEASE]
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1067) ~[spring-beans-4.3.13.RELEASE.jar:4.3.13.RELEASE]
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:513) ~[spring-beans-4.3.13.RELEASE.jar:4.3.13.RELEASE]
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:483) ~[spring-beans-4.3.13.RELEASE.jar:4.3.13.RELEASE]
    at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:306) ~[spring-beans-4.3.13.RELEASE.jar:4.3.13.RELEASE]
    at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230) ~[spring-beans-4.3.13.RELEASE.jar:4.3.13.RELEASE]
    at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:302) ~[spring-beans-4.3.13.RELEASE.jar:4.3.13.RELEASE]

    Thanks,
    Yellappa

  9. Jay says:

    I want to perform GET request based on book title instead of id. How to do it?

  10. Aabid Husain says:

    Good post for beginner where learner can start from. thanks a lot for this valuable post

  11. Karthikeyan says:

    Thanks Shubham,

    Its really really useful post and good to refer the code. Here, we are searching book by its index id rite., how can i search by Author or Title of the book.??

    1. Jose Antonio says:

      I have the same question

  12. Raj says:

    Hi Shubham, thanks for a detailed explanation. i am facing an issue, not getting response for 127.0.0.1:9200
    but i am able to POST, for which i am getting response with ID, if i tried to get using that ID, it throws NPE.

    1. PC says:

      I have the same issue

    2. Manas Kumar Sahoo says:

      I have got the same issue. Did anybody found the solution

  13. Satish says:

    Thats grt help shubham but i m still getting below exception

    Caused by: java.lang.NoClassDefFoundError: org/elasticsearch/common/transport/InetSocketTransportAddress

  14. Muneer says:

    1) what is the use of
    spring.data.elasticsearch.cluster-name=elasticsearch
    spring.data.elasticsearch.cluster-nodes=elasticsearch

    1. Rafael says:

      If I’m right they are mandatory if you use transportClient which will be deprecated soon. HighRestClient has been developed to connect any cluster

  15. Pawan Tiwari says:

    I was searching for an example like this, You guys saved my day.

    It was very easy and explanatory itself to understand. Hat’s off to the writer.

    Thank you so much. !!!

  16. Georges says:

    Mr Shubham, You just saved my day.

    Thank you so much

Leave a Reply

Your email address will not be published. Required fields are marked *

close
Generic selectors
Exact matches only
Search in title
Search in content
Search in posts
Search in pages