Possibility use clickhouse with secondary indexes instead of made own "yet another internal storage" #745
Description
Is your feature request related to a problem? Please describe.
Loki look like great idea and perspective product with great successful usage together Grafana
Describe the solution you'd like
Clickhouse have great performance
and have following features which related to Loki data access pattern
look https://clickhouse.yandex/docs/en/operations/table_engines/mergetree/#data-skipping-indices-experimental
and
https://clickhouse.yandex/docs/en/query_language/create/#column-compression-codecs
could someone discuss about oportunity use clickhouse instead made "yet another own storage"?
Activity
cyriltovena commentedon Jul 15, 2019
We haven't made our own storage, what tell you that ?
Loki has 2 type of storage: index and chunks, index allows to do range queries to fetch multiple chunk ids, chunks storage is a simple chunkid => bytes/blob
We are supporting dynamo/s3, bigtable/gcs and also for users not willing to use any cloud vendor we also offer boltdb/filesystem on disk.
Are you suggesting to add support for clickhouse ? If yes this has to be done in cortex probably at first.
What would you like clickhouse to be used for ? On the website it is explained that it should not be used for blob storage, which Loki and Cortex actually need to store chunks.
I'm happy to discuss with you, I'm interested to know how this benefits Loki.
Slach commentedon Jul 15, 2019
thanks for clarify
i misunderstood howto loki store logs on disks ;( sorry for that
i'm close ticket