Infrastructure Workflow - Import Kubernetes with Hub CLI

Agile Stacks Updated by Agile Stacks

Overview

In this tutorial, you are going to import an existing Kubernetes cluster into SuperHub. The cluster will be integrated with DNS, TLS via Let's Encrypt, and SSO via Okta. After the cluster is imported into SuperHub you'll be able to automate the deployment and management of software stacks - in the cloud and on-prem.

Hub CLI and SuperHub supports importing following cluster types:

  • EKS and OpenShift on AWS
  • GKE on GCP
  • AKS on Azure
  • Bare-metal

Ibn working with examples for EKS or GKE. You should have an existing EKS or GKE cluster which is referred by name cluster-01 in the examples below.

Requirements

You should have an existing Kubernetes cluster which is referred by name cluster-01 in the examples below. After importing the cluster, you will be able to manage it using infrastructure as code approach, deploy overlay stacks, applications, and machine learning pipelines.

Install Hub CLI

First, install Hub CLI binary:

On Mac

curl -O https://controlplane.agilestacks.io/dist/hub-cli/hub.darwin_amd64
mv hub.darwin_amd64 hub
chmod +x hub
sudo mv hub /usr/local/bin

On Linux

curl -O https://controlplane.agilestacks.io/dist/hub-cli/hub.linux_amd64
mv hub.linux_amd64 hub
chmod +x hub
sudo mv hub /usr/local/bin

Login into SuperHub:

$ hub login -u your@email.tld
Password: ****

$ export HUB_TOKEN=sergd......kieud

Add export HUB_TOKEN=... to your ~/.bash_profile / etc.

Create Cloud Account

SuperHub is linked to the AWS or GCP cloud account via credentials supplied by you. When Hub CLI works in the local mode it talks to the cloud directly - by using ~/.aws/credentials or $GOOGLE_APPLICATION_CREDENTIALS. When Hub CLI drives SuperHub via its API, it must setup several essential cloud resources in the account:

  1. An S3 or GCS bucket for Hub CLI and Terraform state, for backups;
  2. DNS zone that will be linked to superhub.io under a subdomain of your choice;
  3. On AWS, a cross-account IAM role that will be used by SuperHub to securely deploy resources in your cloud account. SuperHub does not store your AWS keys.

To onboard a cloud account and setup a my-domain-01.superhub.io subdomain please follow cloud-specific examples.

AWS

# default AWS profile credentials
hub api cloudaccount onboard -w my-domain-01.superhub.io aws us-east-2

GCP

# credentials from $GOOGLE_APPLICATION_CREDENTIALS
hub api cloudaccount onboard -w my-domain-01.superhub.io gcp us-central1

Consult hub api cloudaccount onboard --help for futher examples.

You can list cloud accounts via:

hub api cloudaccount get

Create Environment

The environment is a SuperHub logical entity that organizes deployments, parameters, external facts, and Team's permissions. A good name for the environment is Dev, QA, etc.

Create environment:

hub api environment create Dev01 my-domain-01.superhub.io

You can list environments via:

hub api environment get

A single cloud account can host multiple environments.

Import Cluster

Importing Kubernetes cluster will create two entities in the SuperHub:

  1. Template - a Git repository with infrastructure-as-code automation to perform the import. You can clone and customize this template to suit your needs.
  2. When Template is deployed it creates a Stack Instance. Multiple stack instances can be deployed from the same template. Thus you can reuse the customizations you've made to import more clusters.

EKS

Import cluster:

hub api cluster import eks cluster-01 -e Dev01 --kube-dashboard -w

cluster-01 is EKS cluster name. Add --region if cluster region if different from Cloud Account default region (for example --region us-east-2).

GKE

Import the GKE cluster:

hub api cluster import gke cluster-01 -e Dev01 --kube-dashboard -w

cluster-01 is GKE cluster name. Add --region if cluster region if different from Cloud Account default region (us-central1 from example above).

Validate the Cluster

Importing Kubernetes cluster usually takes several minutes.

Once the cluster is imported you should be able to see output like this:

$ hub api cluster import eks eks05-demo51-superhub-io -e TEST03 --kube-dashboard -w
...
...
2020/06/09 17:18:11 kube-dashboard:component.kubernetes-dashboard.url [Kubernetes Dashboard] => `https://kubernetes.apps.eks05-demo51-superhub-io.demo51.superhub.io`
2020/06/09 17:18:11 Completed deploy on eks-adapter-in-test-03 with components k8s-eks, tiller, cert-manager, traefik, dex, kube-dashboard, automation-tasks-namespace
2020/06/09 00:18:11 Wrote state `.hub/eks05-demo51-superhub-io.demo51.superhub.io.state`
2020/06/09 17:18:11 Wrote state `s3://agilestacks.demo51.superhub.io/eks05-demo51-superhub-io.demo51.superhub.io/hub/eks-adapter/hub.state`
2020/06/09 17:18:11 Syncing Stack Instance state to SuperHub
2020/06/09 17:18:11 All warnings combined:
Stack output `acm:component.tls.kind` not found in outputs:
Stack output `acm:component.acm.certificateArn` not found in outputs:
===> 17:18:16 eks05-demo51-superhub-io [2416] stackInstance update success
===> 17:18:18 eks05-demo51-superhub-io [2416] stackInstance deploy success
$

Now your cluster services are accessible under DNS domain cluster-01.my-domain-01.superhub.io. Kubernetes Dashboard can be accessed at the following URL: https://kubernetes.apps.cluster-01.my-domain-01.superhub.io.

Use the following command to inspect the instance:

hub api instance get cluster-01.my-domain-01.superhub.io

List available stack templates:

hub api template get

Consult hub api cluster import --help for additional parameters.

Importing other cluster types is very similar with a few additional steps necessary to import bare-metal Kubernetes deployments due to datacenter specific configuration. Please contact Agile Stacks Support to learn more about datacenter Kubernetes options with KubeFlex On-Prem.

From here you can continue working with hub api commands. Alternatively, you can use Web based Control Plane https://controlplane.agilestacks.io to view resources created in the UI.

How did we do?

Developer Workflow: Enable Stateful Applications on Kubernetes (201)

Contact