Skip to content

query-frontend does not support websockets/tailing #2878

@gcotone

Description

@gcotone

Describe the bug
When running loki with query-frontend, live tailing is not possible since the frontend component does not support websockets.
Probably something for the backlog.

To Reproduce
Steps to reproduce the behavior:

  1. Started Loki 2.0.0
  2. Started Promtail 2.0.0-amd64
  3. Query frontend configured as datasource in Grafana
  4. Live tail: {host="HOSTNAME"}

Expected behavior
Live tailing should be supported on the query-frontend, or referenced as unsupported in the documentation. The issue is not seen when configuring directly the queriers as endpoints on the Grafana loki datasource.

Environment:

  • Infrastructure: kubernetes
  • Deployment tool: N/A

Screenshots, Promtail config, or terminal output
The following error messages are seen on the loki-query-frontend component while enabling a live log streaming in grafana Explore:

level=warn ts=2020-11-04T16:19:15.449576459Z caller=logging.go:60 traceID=13b0c358fdeda1be msg="GET /loki/api/v1/tail?query=%7Bhost%3D%22HOSTNAME%22%7D 319.174371ms, error: http: request method or response status code does not allow body ws: true; Accept-Encoding: gzip, deflate, br; Accept-Language: en-US,en;q=0.9,es-AR;q=0.8,es;q=0.7,de-DE;q=0.6,de;q=0.5; Cache-Control: no-cache; Connection: Upgrade; Pragma: no-cache; Sec-Websocket-Extensions: permessage-deflate; client_max_window_bits; Sec-Websocket-Key: q8fiwvRig1GCV0ENXabwRQ==; Sec-Websocket-Version: 13; Upgrade: websocket; User-Agent: Grafana/7.3.1; X-Amzn-Trace-Id: Root=1-5fa2d483-763afc81414231b346f530ac; X-Forwarded-For: 23.116.65.148, 10.2.6.1, 10.2.6.1; "

Query-frontend configuration

server:
  http_listen_port: 8080

schema_config:
  configs:
  - from: 2020-05-15
    store: aws
    object_store: s3
    schema: v11
    index:
      prefix: loki_index_
      period: 24h
      tags:
        application: app
        component: loki
  - from: 2020-10-29
    store: boltdb-shipper
    object_store: s3
    schema: v11
    index:
      prefix: loki_index_
      period: 24h

query_range:
  align_queries_with_step: true
  max_retries: 5
  split_queries_by_interval: 10m
  cache_results: true

  results_cache:
    cache:
      enable_fifocache: true
      fifocache:
        max_size_bytes: 512MB
        validity: 24h

limits_config:
  max_cache_freshness_per_query: '10m'

frontend:
  log_queries_longer_than: 5s
  downstream_url: http://loki-querier-svc:8080
  compress_responses: true

Activity

zhanghjster

zhanghjster commented on Nov 5, 2020

@zhanghjster
Contributor

Try:

frontend:
    tail_proxy_url: http://loki-querier-svc:8080
gcotone

gcotone commented on Nov 5, 2020

@gcotone
Author

it works, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Participants

      @zhanghjster@gcotone

      Issue actions

        query-frontend does not support websockets/tailing · Issue #2878 · grafana/loki