Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support array for all string ingest processors #53343

Merged
merged 11 commits into from Mar 17, 2020

Conversation

gaobinlong
Copy link
Contributor

Relates to #51087.

This work comes from the enhancement that lowercase and uppercase processors can support field with multi values, but lowercase and uppercase processors are AbstractStringProcessor,so maybe we can support array for all the string ingest processors below:

  • Bytes Processors
  • HTML Strip Processors
  • Gsub Processors
  • Lowercase Processors
  • Uppercase Processors
  • Trim Processors
  • URL Decode Processors

@matriv matriv added the :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP label Mar 11, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-features (:Core/Features/Ingest)

@danhermann danhermann self-requested a review March 12, 2020 14:24
@danhermann
Copy link
Contributor

@elasticmachine ok to test

Copy link
Contributor

@danhermann danhermann left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gaobinlong, thanks for making this improvement to all the string ingest processors and for updating the docs and tests as well. I suggested a few changes in wording to the docs and error messages. Once those are in, I will get this merged.

docs/reference/ingest/processors/bytes.asciidoc Outdated Show resolved Hide resolved
docs/reference/ingest/processors/gsub.asciidoc Outdated Show resolved Hide resolved
docs/reference/ingest/processors/html_strip.asciidoc Outdated Show resolved Hide resolved
docs/reference/ingest/processors/lowercase.asciidoc Outdated Show resolved Hide resolved
docs/reference/ingest/processors/trim.asciidoc Outdated Show resolved Hide resolved
docs/reference/ingest/processors/uppercase.asciidoc Outdated Show resolved Hide resolved
docs/reference/ingest/processors/url-decode.asciidoc Outdated Show resolved Hide resolved
gaobinlong and others added 10 commits March 17, 2020 10:26
Co-Authored-By: Dan Hermann <danhermann@users.noreply.github.com>
Co-Authored-By: Dan Hermann <danhermann@users.noreply.github.com>
Co-Authored-By: Dan Hermann <danhermann@users.noreply.github.com>
Co-Authored-By: Dan Hermann <danhermann@users.noreply.github.com>
Co-Authored-By: Dan Hermann <danhermann@users.noreply.github.com>
Co-Authored-By: Dan Hermann <danhermann@users.noreply.github.com>
Co-Authored-By: Dan Hermann <danhermann@users.noreply.github.com>
…ommon/AbstractStringProcessor.java

Co-Authored-By: Dan Hermann <danhermann@users.noreply.github.com>
@danhermann danhermann merged commit 8ffe5d1 into elastic:master Mar 17, 2020
adriansr added a commit to adriansr/beats that referenced this pull request Mar 18, 2020
After elastic/elasticsearch#53343, ingest node's lowercase processor
will lowercase values in arrays. This is causing the integration tests
to fail.
adriansr added a commit to elastic/beats that referenced this pull request Mar 18, 2020
After elastic/elasticsearch#53343, ingest node's lowercase processor
will lowercase values in arrays. This is causing the integration tests
to fail.
adriansr added a commit to adriansr/beats that referenced this pull request Mar 19, 2020
After elastic/elasticsearch#53343, ingest node's lowercase processor
will lowercase values in arrays. This is causing the integration tests
to fail.

(cherry picked from commit 0731f45)
adriansr added a commit to elastic/beats that referenced this pull request Mar 19, 2020
After elastic/elasticsearch#53343, ingest node's lowercase processor
will lowercase values in arrays. This is causing the integration tests
to fail.

(cherry picked from commit 0731f45)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP v7.7.0 v8.0.0-alpha1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants