Elasticsearch migration from the High Level Rest Client – Java Tips

To prepare for new Elasticsearch version (8.0) I was migrating our app from RestHighLevelClient to new java client – ElasticsearchClient. Here are some information for You to make this transition smoother than it was for me!

I did this migration for 7.16.2 version, which already introduces a new client. Newer versions have further adjustments, for example starting from 7.17.0 You can create mappings from JSON files with .withJSON(loaded_JSON_file); which in 7.16.2 You can’t. But If your indices names are standardized, You can create index templates  and remove mapping creation from Your java app.

Whole migration is done to move from org.elasticsearch to co.elastic dependency.

New elastic dependency requires 2 more dependencies:

• jakarta.json-api
• jackson-databind

ElasticsearchClient creation

Client creation docs it’s straightforward, but if You want to have empty fields still included in documents, You will have to do a custom config JacksonnJsonpMapper because by default empty values are now skipped:

JacksonJsonpMapper jacksonJsonpMapper = new JacksonJsonpMapper();
jacksonJsonpMapper.objectMapper().setSerializationInclusion(JsonInclude.Include.ALWAYS); //This will include every field, beside the value
RestClientTransport restClientTransport = new RestClientTransport(someRestClientConfig, jacksonJsonpMapper);
ElasticsearchClient client = new ElasticsearchClient(restClientTransport);

Moreover, the way to close client also is different now:

elasticsearchClient._transport().close();

All calls to the client doesn’t need right now RequestOptions as a parameter!

Query creation

With new client, we now have two ways of creating Queries:

• We can use provided static builders from QueryBuilders class:

QueryBuilders.bool()

• Or we can use:

BoolQuery.Builder boolQueryBuilder = new BoolQuery.Builder(); //which is the same as QueryBuilders.bool() is doing this under the hood.

Example Before:

private BoolQueryBuilder buildMustNotBoolQueryBuilder() { 
    return QueryBuilder.boolQuery() 
        .mustNot( 
           QueryBuilders.termQuery("Field", "Value")
        )
}

Now:

private BoolQuery.Builder buildMustNotBoolQueryBuilder() {
    return QueryBuilder.bool()
        .mustNot(
           QueryBuilders.term()
             .field("String")
             .value(FieldValue.of("Value")
           .build()._toQuery()
        )
}

// OR

private BoolQuery.Builder buildMustNotBoolQueryBuilder() { 
    return QueryBuilder.bool() 
        .mustNot(queryBuilder ->
                   queryBuilder.term(termQueryBuilder -> 
                     termQueryBuilder.field("String") .value(FieldValue.of("Value")
                   )
                ) 
}

Deserialization of responses

If You’ve done before conversion manually, with extracting a type from a generic converter with monster like this:

(ParameterizedType) getClass().getGenericSuperclass()).getActualTypeArguments()[0];

You can skip that. Object mapping right now is done inside the client library, so for example when we call a simple get call, we should also provide a type for deserialization:

public T get(String id, Class<T> deserializeType) throws IOException {
    return client().get(builder -> builder.index(indexName).id(id), deserializeType).source(); 
}

Due to generic implementation in Java You can’t pass the type from an initialized generic class, so the easiest approach with passing the type to the client would be to create an additional field in the generic class:

private final Class<T> deserializeType;

and initialize it in class constructor.

SearchRequest.scroll(param) right now takes different param type (than String, as before):

import co.elastic.clients.elasticsearch._types.Time;

which You can provide in 2 ways:

SearchRequest.scroll(new Time.Builder().time(scrollTimeoutInString)).build());

or

SearchRequest.scroll(timeBuilder -> timeBuilder.time(scrollTimeoutInString));

This code converts the list of elastic hits of our type to the arrayList of objects with type of the initialized converter:

public List<T> convert(SearchResponse<T> response) {
    List<Hit<T>> hits = response.hits().hits();
    List<T> result = new ArrayList<>(hits.size());
    for (Hit<T> hit : hits) {
        result.add(Objects.requireNonNull(hit.source()));
    }
    return result;
}

Embedded jackson tips

To avoid errors with object mapping in ElasticsearchClient You have to create an empty public constructor, without it some deserialization won’t work.

@SuppressWarnings("unused")
public SomeClass() { }

For some classes, You will have to point embedded Jackson to some values for it to map it:

@JsonProperty("property")
private Map<String, Object> data;

Extracting terms aggregations with HighLevelRestClient:

final List<? extends Terms.Bucket> buckets = ((Terms) searchResponse.getAggregations().get(aggName)).getBuckets();

Now in ElasticsearchClient:

final List<StringTermsBucket> buckets = searchResponse.aggregations().get(aggName).sterms().buckets().array();

For BulkRequest right now we create .operations, not add iterables to request:

BulkRequest bulkRequest = new BulkRequest.Builder().operations(listOfOperations);

To create subAggregation right now, You have to do it this way:

public Aggregation.Builder.ContainerBuilder createSubAggregationBuilder() {
  return new Aggregation.Builder()
    .terms(//some aggregation here)
    .aggregations("aggregationName", builder -> builder //before it was .subAggregation()
      .terms(builder1 -> builder1.field("field").size(1000))
    );
}

Now to configure includes in sourceConfig in search calls, You do it like this:

public SourceConfig.Builder getSearchSourceBuilder() {
return new SourceConfig.Builder().filter(builder -> builder.includes(
  "value1",
  "value2",
  "value3"
));
}

before You passed 2 params to:

SearchSourceBuilde.fetchSource(includesArray, excludesArray);

Script before had a constructor with few parameters, right now, You create it with a builder:

Script script = new Script.Builder().inline(builder -> 
  builder.lang("painless") //checkout languages in documentation
    .source("inline script to be used")
  ).build();

For DeleteIndexRequest there’s now a change as IndicesOptions are converted to flags in fluid builder for this class:

Before:
new DeleteIndexRequest("name").indicesOptions(IndicesOptions.LENIENT_EXPAND_OPEN);

Now:
new DeleteIndexRequest.Builder().index("name").allowNoIndices(true).ignoreUnavailable(true).expandWildcards(ExpandWildcard.Open).build();

Test tips

For tests, it’s almost the same as it was, You just have to change a few classes and calls. Few tips, for example, if You want to mock SearchResponse and do something with the source, cast the mocked response to SearchResponse<type> with Your type. If You intend to mock methods with functional parameters, just use:

mock(java.util.function.Function.class);

Few tips of not so obvious migrations

Please let me know in the comments if You have found some other not so obvious migration tips!