Hi @DXist. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

/ok-to-test

@DXist I am also working on this issue and I have discussed the below proposel in Sig weekly meeting, and as per discussion in meeting the @mwielgus will discuss this with networking team, and based on that we can modify the PR and I'll be happy to coordinate with you on this.

https://docs.google.com/document/d/1p_Xlk8e5V32WOBVeRbJqbxhEKcJy_mDu9JgVp5ZNDxs/

@Rajat-0

@DXist I am also working on this issue and I have discussed the below proposel in Sig weekly meeting, and as per discussion in meeting the Sig owner will discuss this with networking team, and based on that we can modify the PR and I'll be happy to coordinate with you on this.

https://docs.google.com/document/d/1p_Xlk8e5V32WOBVeRbJqbxhEKcJy_mDu9JgVp5ZNDxs/

This PR handles more general case and is not specific to HTTP workload.

If an HTTP load balancer/Ingress could buffer requests and export a number of queued requests for given service as a custom metric then HPA can use this metric for scaling. I see that there could be some signaling about metric change or push based model of metric delivery to HPA to speed up the scaling process.

Couple of high-level items:

1. How was it tested?

There are unittests that cover scaling up from zero pods and scaling down to zero for custom object and external metrics.

For end-to-end testing I

• built K8S binaries
• run them using https://github.com/kubernetes-sigs/kubeadm-dind-cluster
• set up RabbitMQ, Prometheus Operator, Prometheus adapter for custom metrics
• set up the service I want to scale and configured Prometheus, Prometheus adapter and HPA.
  HPA used two Object metrics - worker utilization and queue ingress/egress ratio
• run scripts to generate test load - a single message or a series of messages in the rate of ~5X processing throughput of a single pod

Only one metric value (worker utilization) was available when there were no pods.
Metrics used 1m window to calculate averaged value so I delayed start of pod readiness probes via initialDelaySeconds: 60. Due to this delay the next scaling decision was based on a full metric window since previous rescale. The delay worked in the same way as replica calculator code for resource metrics.

2. Why didn't you handle: https://github.com/kubernetes/kubernetes/blob/master/pkg/controller/podautoscaler/horizontal.go#L567

Github collapsed the diff for this module. I've added extra condition to enable scaling when minReplicas==0

3. Did you consider explicitly validating that there is Object/External metric configured?

No, I didn't. It seems to be a good protection from HPA misconfiguration.

4. Maybe we should have a comment update in the API.

A comment update should be near MinReplicas?

Added metrics validation for minReplicas=0 case and a comment to the API.

This PR may require API review.

If so, when the changes are ready, complete the pre-review checklist and request an API review.

Status of requested reviews is tracked in the API Review project.

API LGTM

@mwielgus ping me when LGTM and I will approve

/lgtm
/retest

/retest

@liggitt , @mwielgus What is the next step to make the feature GA?

Just stopping by after six months to see what, if anything, can be done to move this otherwise perfectly reasonable and working feature from Alpha to GA.

This would be great to have on hosted K8S environments like GKE or AKS or EKS or... basically anywhere.

Was just reading the release notes for 1.19,
and noticed the section on avoiding permanent beta, it would be great if there was some clarity on how to foster this work through the process.

@liggitt , @mwielgus What is the next step to make the feature GA?

Would love an answer to this. It's been half a year or so since it was asked.

Very useful feature for shutting down workers processing jobs from queues when the queues have no tasks.

Sounds like a good topic for the SIG Autoscaling meeting :)

@lavalamp were there results from the meeting?

Apologies, that was super unclear of me-- I realize now it reads like I was going to take it to the SIG meeting, but I was actually only trying to suggest that it was the logical next thing to do. I don't personally have time to contribute to this other than the occasional review.

@lavalamp no worries. Is there someone more central to the planning we can poke here to move things along? I think this should be a pretty innocent bump to GA. I'll go ahead and tag others who were in this ticket for Alpha release.

@liggitt @mwielgus

We are also using this feature heavily and I would be happy to contribute time driving this towards GA.

While I contributed some changes to k8s, I've never been part of an API graduation so I would need to some pointers how to do this if help is needed.

I think this should be a pretty innocent bump to GA. I'll go ahead and tag others who were in this ticket for Alpha release.

@liggitt @mwielgus

API changes need an associated doc describing the change and graduation requirements, called a KEP (Kubernetes Enhancement Proposal). The key things we'd be looking for are evidence of sufficient test coverage, any performance implications of the change, and upgrade/downgrade/skew compatibility implications.

It looks like #69687 (comment) might have been the original proposal associated with the alpha change, but before graduating to beta and being enabled by default, that should be ported to a KEP and make sure the test/performance/upgrade questions are answered. A template is available at https://github.com/kubernetes/enhancements/tree/master/keps/NNNN-kep-template and would be placed in https://github.com/kubernetes/enhancements/tree/master/keps/sig-autoscaling

@johanneswuerbach @DXist is this something one of you can lead? It would be my first contribution, and is probably not a great thing to take a bite out of first.

If not, I can attempt the first pass at the proposal.

@jeffreybrowning go ahead. I changed project and work in another context.

I planned to get something drafted on Friday, but I also never worked on a KEP before. If you have more time feel free to pick it @jeffreybrowning :-)

I am embroiled in an update to another package right now -- it would likely be a sizeable delay until I get to this.

I started the official propose here kubernetes/enhancements#2021. Looking forward to any of your comments or feedback on the KEP itself here kubernetes/enhancements#2022.