Skip to content

Kubernetes v1.16.x (default in minikube v1.4) breaks Helm Tiller (v2.14.3) #5429

Closed
@aaronhmiller

Description

@aaronhmiller

The exact command to reproduce the issue:
Install Minikube 1.4.0

helm init
$HELM_HOME has been configured at /Users/amiller/.helm.
Error: error installing: the server could not find the requested resource

The full output of the command that failed:

Note: downgrade to Minikube 1.3.1

helm init
$HELM_HOME has been configured at /Users/amiller/.helm.

Tiller (the Helm server-side component) has been installed into your Kubernetes Cluster.

Please note: by default, Tiller is deployed with an insecure 'allow unauthenticated users' policy.
To prevent this, run helm init with the --tiller-tls-verify flag.
For more information on securing your installation see: https://docs.helm.sh/using_helm/#securing-your-helm-installation

The output of the minikube logs command:


sorry, needed to get this running again. didn't grab logs

The operating system version:

uname -a
Darwin slate.local 18.7.0 Darwin Kernel Version 18.7.0: Tue Aug 20 16:57:14 PDT 2019; root:xnu-4903.271.2~2/RELEASE_X86_64 x86_64
sw_vers
ProductName: Mac OS X
ProductVersion: 10.14.6
BuildVersion: 18G95

Activity

afbjorklund

afbjorklund commented on Sep 22, 2019

@afbjorklund
Collaborator

Helm v2.14.3 (latest) is currently incompatible with Kubernetes version v1.16.0 (latest):

helm/helm#6374

afbjorklund

afbjorklund commented on Sep 22, 2019

@afbjorklund
Collaborator

Workarounds (only need one of them):

  1. Patch the YAML from helm init:
helm init --output yaml | sed 's@apiVersion: extensions/v1beta1@apiVersion: apps/v1@' | sed 's@  replicas: 1@  replicas: 1\n  selector: {"matchLabels": {"app": "helm", "name": "tiller"}}@' | kubectl apply -f -
@@ -1,5 +1,5 @@
 ---
-apiVersion: extensions/v1beta1
+apiVersion: apps/v1
 kind: Deployment
 metadata:
   creationTimestamp: null
@@ -10,6 +10,7 @@
   namespace: kube-system
 spec:
   replicas: 1
+  selector: {"matchLabels": {"app": "helm", "name": "tiller"}}
   strategy: {}
   template:
     metadata:
  1. Use old Kubernetes version v1.15
minikube start --kubernetes-version=v1.15.4
  1. Wait for helm release v2.15.0

"It looks like once we start supporting Kubernetes 1.16.0 we will have to handle that case going forward and migrate to the newer apiVersion."

Either way, it doesn't look like a minikube issue...

$ minikube kubectl version
Client Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.0", GitCommit:"2bd9643cee5b3b3a5ecbd3af49d09018f0773c77", GitTreeState:"clean", BuildDate:"2019-09-18T14:36:53Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"16", GitVersion:"v1.16.0", GitCommit:"2bd9643cee5b3b3a5ecbd3af49d09018f0773c77", GitTreeState:"clean", BuildDate:"2019-09-18T14:27:17Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
$ helm version
Client: &version.Version{SemVer:"v2.14.3", GitCommit:"0e7f3b6637f7af8fcfddb3d2941fcc7cbebb0085", GitTreeState:"clean"}
Server: &version.Version{SemVer:"v2.14.3", GitCommit:"0e7f3b6637f7af8fcfddb3d2941fcc7cbebb0085", GitTreeState:"clean"}

Maybe it should be (include helm), though: #5302

changed the title [-]v.1.4.0 breaks Helm Tiller[/-] [+]Minikube v1.4.0 (Kubernetes v1.16.0) breaks Helm Tiller (v2.14.3)[/+] on Sep 22, 2019
afbjorklund

afbjorklund commented on Sep 22, 2019

@afbjorklund
Collaborator

I think this collaboration could probably be improved somewhat:

https://helm.sh/docs/using_helm/#kubernetes-distribution-guide

MINIKUBE
Helm is tested and known to work with minikube. It requires no additional configuration.

Apparently that statement is only true one-way, new helm releases.

bacongobbler

bacongobbler commented on Sep 22, 2019

@bacongobbler
Contributor

This is the first time in a long time where Helm has been incompatible with the latest release of Kubernetes. That statement has held true from k8s 1.5 up until 1.16.

afbjorklund

afbjorklund commented on Sep 22, 2019

@afbjorklund
Collaborator

@bacongobbler : what I meant was that we should make sure to test new versions of minikube with helm, during our beta phase. We had a similar last-minute experience with the dashboard as well.

I don't think it is possible for downstream projects to test with future releases, but I do think that Kubernetes (and Minikube) could add some extra regression testing for prominent third-parties.

woodcockjosh

woodcockjosh commented on Sep 22, 2019

@woodcockjosh
Contributor

helm/helm#6374
FYI for anyone trying to helm init on k8s version 1.16 this branch works https://github.com/keleustes/helm/tree/kube16. You can build the branch yourself. I also uploaded the binary here for your convenience https://s3-us-west-2.amazonaws.com/bin.cryptexlabs.com/github.com/keleustes/helm/kube16/helm. One caveat is you have to use the canary image flag helm init --canary-image since the build is unreleased

woodcockjosh

woodcockjosh commented on Sep 22, 2019

@woodcockjosh
Contributor

This plugin would also fix the problem :-D #5363

tstromberg

tstromberg commented on Sep 23, 2019

@tstromberg
Contributor

It's worth noting that this also affects older minikube releases which select Kubernetes v1.16.0. In the mean time, you can work around the incompatibility by selecting an older version of Kubernetes:

--kubernetes-version=v1.15.3

https://minikube.sigs.k8s.io/docs/reference/configuration/kubernetes/

changed the title [-]Minikube v1.4.0 (Kubernetes v1.16.0) breaks Helm Tiller (v2.14.3)[/-] [+]Kubernetes v1.16.x (default in minikube v1.4) breaks Helm Tiller (v2.14.3)[/+] on Sep 23, 2019
fejta-bot

fejta-bot commented on Dec 31, 2019

@fejta-bot

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale.
Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle stale

added
lifecycle/staleDenotes an issue or PR has remained open with no activity and has become stale.
on Dec 31, 2019
fejta-bot

fejta-bot commented on Jan 30, 2020

@fejta-bot

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten.
Rotten issues close after an additional 30d of inactivity.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/lifecycle rotten

added
lifecycle/rottenDenotes an issue or PR that has aged beyond stale and will be auto-closed.
and removed
lifecycle/staleDenotes an issue or PR has remained open with no activity and has become stale.
on Jan 30, 2020
fejta-bot

fejta-bot commented on Mar 1, 2020

@fejta-bot

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

k8s-ci-robot

k8s-ci-robot commented on Mar 1, 2020

@k8s-ci-robot
Contributor

@fejta-bot: Closing this issue.

In response to this:

Rotten issues close after 30d of inactivity.
Reopen the issue with /reopen.
Mark the issue as fresh with /remove-lifecycle rotten.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta.
/close

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/kubernetes-versionsImproving support for versions of Kuberneteskind/supportCategorizes issue or PR as a support question.lifecycle/rottenDenotes an issue or PR that has aged beyond stale and will be auto-closed.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

      Development

      No branches or pull requests

        Participants

        @tstromberg@aaronhmiller@bacongobbler@woodcockjosh@afbjorklund

        Issue actions

          Kubernetes v1.16.x (default in minikube v1.4) breaks Helm Tiller (v2.14.3) · Issue #5429 · kubernetes/minikube