Skip to content

Possibility use clickhouse with secondary indexes instead of made own "yet another internal storage" #745

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
Slach opened this issue Jul 12, 2019 · 2 comments
Closed

Possibility use clickhouse with secondary indexes instead of made own "yet another internal storage" #745

Slach opened this issue Jul 12, 2019 · 2 comments
Labels
type/question

Comments

@Slach
Copy link

Slach commented Jul 12, 2019

Is your feature request related to a problem? Please describe.
Loki look like great idea and perspective product with great successful usage together Grafana

Describe the solution you'd like
Clickhouse have great performance
and have following features which related to Loki data access pattern
look https://clickhouse.yandex/docs/en/operations/table_engines/mergetree/#data-skipping-indices-experimental
and
https://clickhouse.yandex/docs/en/query_language/create/#column-compression-codecs

could someone discuss about oportunity use clickhouse instead made "yet another own storage"?
@cyriltovena
Copy link
Contributor

cyriltovena commented Jul 15, 2019 •
edited
Loading

We haven't made our own storage, what tell you that ?

Loki has 2 type of storage: index and chunks, index allows to do range queries to fetch multiple chunk ids, chunks storage is a simple chunkid => bytes/blob

We are supporting dynamo/s3, bigtable/gcs and also for users not willing to use any cloud vendor we also offer boltdb/filesystem on disk.

Are you suggesting to add support for clickhouse ? If yes this has to be done in cortex probably at first.

What would you like clickhouse to be used for ? On the website it is explained that it should not be used for blob storage, which Loki and Cortex actually need to store chunks.

I'm happy to discuss with you, I'm interested to know how this benefits Loki.

@Slach
Copy link
Author

Slach commented Jul 15, 2019

thanks for clarify
i misunderstood howto loki store logs on disks ;( sorry for that
i'm close ticket

@Slach Slach closed this as completed Jul 15, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type/question
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Slach @cyriltovena