Skip to content

Feature Request: Allow processing of async requests in Knative #4522

Closed
@nimakaviani

Description

@nimakaviani
Contributor

/area API

Describe the feature

As a Knative user, I can request the http workload to be processed asynchronously.

Requirements

  • Same endpoint should allow for both blocking and non-blocking async requests
  • The incoming async requests trigger the running of the async job in Knative
  • An async request does not block. Instead, it returns immediately
  • The response will have an identifier that can be used to track progress for the async job
  • A follow-up request to track progress will need to supply the identifier
    • If the async job is still in progress
      • the response will return an in progress status
        (e.g, a 302 Found status code with an empty body)
    • If the async job is finished
      • the response body will have the result from the app processing the async job
      • the status code will be identical to what is returned from the app after the job is done
  • Any instance of the running Knative app should be able to accept / respond to an async query
  • A failed async job will return a HTTP 5xx code

Usecases

  • Long-running jobs
    • Notifications (e.g. mobile push notification, SMS notification, mass emails, etc)
    • Database migrations
    • Batch processing (e.g. data transformation)
  • Stream processing
  • Highly parallel jobs (document, image, … processing)
  • Fan-out workloads
  • Serveless Operator

Activity

nimakaviani

nimakaviani commented on Jun 25, 2019

@nimakaviani
ContributorAuthor
vagababov

vagababov commented on Jun 25, 2019

@vagababov
Contributor

Question is, why would you want to use knative for this?
Most of what you described is what your app has to do.
I presume, you don't really expect Knative to provide a generic state machine/workflow engine?
Isn't everything you need is minScale=1, to permit async processing, without scaling to zero?

markusthoemmes

markusthoemmes commented on Jun 25, 2019

@markusthoemmes
Contributor

I always thought async workloads are more of a territory of Knative Eventing in conjunction with Knative Serving. After all, the protocol for Knative Serving currently is request/response based and everything revolves around that pretty much.

Any thoughts on why Eventing is not a suitable solution for async workloads like the ones you mentioned?

added this to the Ice Box milestone on Jun 25, 2019
mattmoor

mattmoor commented on Jun 25, 2019

@mattmoor
Member

As Markus says, this is where eventing comes in, perhaps even eventing delivering events to something other than serving.

cc @evankanderson

nimakaviani

nimakaviani commented on Jun 25, 2019

@nimakaviani
ContributorAuthor

@vagababov I think setting minScale=1 is an anti-pattern to support async. Particularly for the case of infrequent async jobs, I dont think the expectation of having resources sitting idle is ideal.

@markusthoemmes good question. I think this is the conjunction of Knative Eventing and Knative Serving that is in fact problematic. Knative's presumption of short-lived requests with persistent connection to the client goes against what is expected from an async job, regardless of whether the request is initiated by an outside client or an internal Cloud Event. This is where I think @mattmoor 's point regarding having "something other than serving" deal with async requests comes into play. But does an app developer really want to have a separate app to send batch emails or perform data transformations? or even worse, to support separate code bases or deployment models?

so to @vagababov's question, I don't really expect Knative to provide a generic state machine or workflow engine, but for it to close the loop on common requirements in a developer's workflow without the developers having to go up and down in the stack to have different pieces of their application deployed.

mattmoor

mattmoor commented on Jun 26, 2019

@mattmoor
Member

This is where I think @mattmoor 's point regarding having "something other than serving" deal with async requests comes into play

To be clear, I mean having something other than serving dealing with the async aspect, but that may result in the Eventing system having a synchonous call to Serving. For sufficiently long-running (a la batch) cases, Serving may be ill-suited for this (or the scaling model may not be right), and so it may make sense to compose eventing with other forms of addressable compute that aren't currently covered by knative.

mikehelmick

mikehelmick commented on Jul 2, 2019

@mikehelmick
Contributor

Another vote for using Eventing to handle this (turning a sync request into an async request).

Also, there is room for serverless products for handling long running job, but currently, I think this it outside the charter of knative/serving.

duglin

duglin commented on Jul 2, 2019

@duglin

I'm not following the relationship to eventing. The connection between a client and the KnService is not an event. It shouldn't be morphed into a CloudEvent, or go thru brokers, triggers, fanned-out, etc. I also wouldn't expect the client to change how it interacts with a KnService based on whether it's doing it via sync or async - meaning I would not expect the URL of the KnService to change, which is what I think using eventing for async would require. People may be interacting in both modes at the same time.

I don't see the internal flow or structure needed by async being that different from sync - with the exception of how responses are handled, so it's hard for me to see why this is out of scope for serving (yes I know there's a bit more to it, but I think the response handling is the biggie). Async is a well established model in the serverless space so it would be odd for us to exclude support for it.

duglin

duglin commented on Jul 2, 2019

@duglin

Thinking more about the tie-in with eventing... if what's meant is that something in the normal flow detects the "run this in async mode" flag, and as a result redirects the request to some eventing component because it knows how to store responses in some kind of persistence, then that's just an impl detail. But converting it to a CloudEvent seems odd.

In the end though, a KnService is called and I think the only connection that we need to ensure stays open the entire time is the one between the user container and the queue proxy - and I'm not sure eventing helps with that since it assumes persistent connections from channels, no? Although, if we combined this support with how eventing manages responses (meaning, instead of the channel waiting for a response, the response is sent via a call-back managed by the queue proxy) then I think those two worlds might be more aligned than I originally considered. But, that all impl details and the user should be unaware of it.

mbehrendt

mbehrendt commented on Jul 2, 2019

@mbehrendt

I think this it outside the charter of knative/serving.

@mikehelmick can you pls point me to that charter?

mbehrendt

mbehrendt commented on Jul 2, 2019

@mbehrendt

re 'handling async via eventing' , adding to what @duglin said above: if we somehow magically handled it via eventing, you still have the issue that somehow under the cover the knservice gets called synchronously. I.e. you're bound to the max execution time associated with synchronous calls, and to the resource consumption implied by potentially keeping 1000's of http connections open.

mbehrendt

mbehrendt commented on Jul 2, 2019

@mbehrendt

other forms of addressable compute that aren't currently covered by knative.

@mattmoor can you pls elaborate on which other forms of addressable compute you're referring to? Are there special semantics behind you emphasizing the currently? E.g. is there sth cooking behind the scenes?

63 remaining items

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

area/APIAPI objects and controllerskind/featureWell-understood/specified features, ready for coding.triage/acceptedIssues which should be fixed (post-triage)

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

    Development

    No branches or pull requests

      Participants

      @dprotaso@greghaynes@mikehelmick@steren@vagababov

      Issue actions

        Feature Request: Allow processing of async requests in Knative · Issue #4522 · knative/serving