Skip to content

UpdateByQuery fails when max_docs>slices #52786

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
glascaleia opened this issue Feb 25, 2020 · 5 comments
Closed

UpdateByQuery fails when max_docs>slices #52786

glascaleia opened this issue Feb 25, 2020 · 5 comments
Labels
>bug :Distributed Indexing/Reindex Issues relating to reindex that are not caused by issues further down Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination.

Comments

@glascaleia
Copy link

Upgrade elasticsearch from 7.5.2 to 7.6.0 with UpdateByQueryRequest i have the following error
ElasticsearchStatusException[Elasticsearch exception [type=action_request_validation_exception, reason=Validation Failed: 1: maxDocs should be greater than 0 if the request is limited to some number of documents or -1 if it isn't but it was [0];]]
at org.elasticsearch.rest.BytesRestResponse.errorFromXContent(BytesRestResponse.java:177)
at org.elasticsearch.client.RestHighLevelClient.parseEntity(RestHighLevelClient.java:1793)
at org.elasticsearch.client.RestHighLevelClient.parseResponseException(RestHighLevelClient.java:1770)
at org.elasticsearch.client.RestHighLevelClient$1.onFailure(RestHighLevelClient.java:1686)
at org.elasticsearch.client.RestClient$FailureTrackingResponseListener.onDefinitiveFailure(RestClient.java:598)
at org.elasticsearch.client.RestClient$1.completed(RestClient.java:343)
at org.elasticsearch.client.RestClient$1.completed(RestClient.java:327)
at org.apache.http.concurrent.BasicFuture.completed(BasicFuture.java:122)
at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:181)
at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:448)
at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:338)
at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265)
at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81)
at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39)
at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114)
at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591)
at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1$1.run(Executors.java:668)
at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1$1.run(Executors.java:665)
at java.base/java.security.AccessController.doPrivileged(AccessController.java:391)
at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1.run(Executors.java:665)
at java.base/java.lang.Thread.run(Thread.java:830)
Suppressed: org.elasticsearch.client.ResponseException: method [POST], host [http://192.168.3.2:9200], URI [/acta_bidone_index/_update_by_query?slices=2&requests_per_second=-1&ignore_unavailable=false&expand_wildcards=open&allow_no_indices=true&ignore_throttled=true&scroll_size=100&refresh=true&conflicts=proceed&max_docs=1&timeout=2m], status line [HTTP/1.1 400 Bad Request]
{"error":{"root_cause":[{"type":"action_request_validation_exception","reason":"Validation Failed: 1: maxDocs should be greater than 0 if the request is limited to some number of documents or -1 if it isn't but it was [0];"}],"type":"action_request_validation_exception","reason":"Validation Failed: 1: maxDocs should be greater than 0 if the request is limited to some number of documents or -1 if it isn't but it was [0];","suppressed":[{"type":"action_request_validation_exception","reason":"Validation Failed: 1: maxDocs should be greater than 0 if the request is limited to some number of documents or -1 if it isn't but it was [0];"}]},"status":400}
The value of maxDocs is setted to right value that is 1.
With 7.5.2 all rocks after upgrading to 7.6.0 there is this error.
Regards
@mayya-sharipova
Copy link
Contributor

@glascaleia Thanks for submitting the issue. Indeed this looks like a bug. I am surprised that you have not experienced this issue in 7.5.2, because it doesn't look like we have done any changes in the relevant code between 7.5 and 7.6.

I think you can remedy your request by making sure to set max_docs higher than slices.

@mayya-sharipova
Copy link
Contributor

mayya-sharipova commented Feb 26, 2020 •
edited
Loading

The issue seems to be originated from this line in AbstractBulkByScrollRequest:

if (maxDocs != MAX_DOCS_ALL_MATCHES) {
            // maxDocs is split between workers. This means the maxDocs might round
            // down!
            request.setMaxDocs(maxDocs / totalSlices);
}

It looks like when totalSlices<maxDocs, we are incorrectly setting max_docs parameter for a request to 0.

@mayya-sharipova mayya-sharipova added the :Distributed Indexing/Reindex Issues relating to reindex that are not caused by issues further down label Feb 26, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed (:Distributed/Reindex)

@mayya-sharipova mayya-sharipova changed the title Bug UpdateByQuery fails when max_docs>slices Feb 28, 2020
@gaobinlong
Copy link
Contributor

Should we throw an illegal_argument_exception when max_docs is less than slices?

@gaobinlong gaobinlong mentioned this issue Apr 7, 2020
Merged
@rjernst rjernst added the Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. label May 4, 2020
henningandersen pushed a commit that referenced this issue May 28, 2020
@gaobinlong
Reindex and friends fail nicely when max_docs < slices (#54901)

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Verified
Learn about vigilant mode
6e64362 
When the parameter `max_docs` is less than `slices` in update_by_query, delete_by_query or reindex API, `max_docs ` is set to 0 and we throw an action_request_validation_exception with confused error message: "maxDocs should be greater than 0...". 
This change checks that whether `max_docs` is less than `slices` and throw an illegal_argument_exception with clear message.

Relates to #52786.
henningandersen pushed a commit to henningandersen/elasticsearch that referenced this issue May 29, 2020
@gaobinlong @henningandersen
Reindex and friends fail nicely when max_docs < slices (elastic#54901)
8402172 
When the parameter `max_docs` is less than `slices` in update_by_query,
delete_by_query or reindex API, `max_docs ` is set to 0 and we throw an
action_request_validation_exception with confused error message:
"maxDocs should be greater than 0...".
This change checks that whether `max_docs` is less than `slices` and
throw an illegal_argument_exception with clear message.

Relates to elastic#52786.
@henningandersen henningandersen mentioned this issue May 29, 2020
Merged
henningandersen added a commit that referenced this issue May 29, 2020
@henningandersen @gaobinlong
Reindex and friends fail nicely when max_docs < slices (#54901) (#57348)

Verified

This commit was created on GitHub.com and signed with GitHub’s verified signature. The key has expired.
GPG key ID: 4AEE18F83AFDEB23
Expired
Verified
Learn about vigilant mode
8427d67 
When the parameter `max_docs` is less than `slices` in update_by_query,
delete_by_query or reindex API, `max_docs ` is set to 0 and we throw an
action_request_validation_exception with confused error message:
"maxDocs should be greater than 0...".
This change checks that whether `max_docs` is less than `slices` and
throw an illegal_argument_exception with clear message.

Relates to #52786.

Co-authored-by: bellengao <gbl_long@163.com>
@henningandersen
Copy link
Contributor

Solved by @gaobinlong in #54901, so closing this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
>bug :Distributed Indexing/Reindex Issues relating to reindex that are not caused by issues further down Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
6 participants
@rjernst @glascaleia @mayya-sharipova @gaobinlong @elasticmachine @henningandersen