Kubernetes Platform Essentials

Agile Stacks Updated by Agile Stacks

Agile Stacks platform is a Kubernetes cluster with various add-ons to implement essential integrations:

  1. DNS
  2. Ingress controller
  3. TLS
  4. Single Sign-On for web-facing applications
  5. Performance monitoring

When Kubernetes is deployed by Agile Stacks SuperHub and/or from an Infrastructure as Code template, then those integrations are installed automatically.

In case Kubernetes cluster is imported into Agile Stacks SuperHub it may already have some of the integrations implemented.

DNS

Agile Stacks experience is optimized around human-readable domain names - assigned to the cluster (API endpoint), to the applications running on the cluster.

We install External DNS or - as a legacy solution - we create a number of static wildcard records in native cloud DNS with Terraform.

In case DNS is already in place we expect hostnames installed by Ingress objects to become resolvable, either via pre-setup External DNS of by some other means.

Cloud

In the cloud a native implementation is used:

  • AWS Route 53, also for GovCloud
  • GCP CloudDNS
  • Azure DNS

On-prem

Due to the dynamic nature of Kubernetes, the DNS records must be capable of being updated immediately as changes occur in the cluster.
There are 2 options for DNS integration with your on-prem environment:

  1. Custom zone delegation;
  2. Dynamic updates to your existing DNS server via RFC2136.

Custom zone delegation is typically easier and simpler to implement. In this scenario, we install a self-contained DNS installation to support the K8s clusters being managed. Then that custom zone is delegated by your DNS infrastructure.

The "Dynamic updates" approach is more direct, but it requires that your DNS server host the zone that publishes the Kubernetes cluster's records. This approach requires that your DNS server be able to receive RFC2136 dynamic DNS updates.

Ingress controller

We install Traefik ingress controller, both v1 and v2 are supported.

Cloud

In the cloud Traefik install LoadBalancer type of Service, making it available via native cloud LB. The IP / Hostname of the LB is written into DNS by External DNS that receives it via Ingress status block (put back by Traefik), or - in case External DNS is not installed - written by Terraform once at Traefik deploy / redeploy time.

On-prem

On-prem we expect MetalLB.

An alternative is to have Traefik Pod to be bound to a dedicated node with NodePort service exposed via manual out-of-the-cluster configuration.

TLS

We install Cert-manager and TLS Host controller. The later configures Ingress TLS blocks to match Rules Hosts in a non-intrusive way.

Cloud

In the cloud Cert-manager uses HTTP solver to obtain Let's Encrypt certificates. Cert-manager's ingress shim watches for Ingress objects with configured TLS block to issue a certificate request. Where possible we also configure DNS solver (to native cloud DNS) so that requests for wildcard certificates could also be validated.

Istio component installs it's own Certificate custom resource which Cert-manager solves via DNS. The certificate and key is saved as Kubernetes secret that in turn is used by Istio Gateway.

On-prem

On-prem we may have a limitation of ingress not being available on public internet. In this case a DNS validation on public DNS could be configured with static (AWS) cloud credentials.

An alternative is to have a local CA issuer, ie. Vault.

Single Sign-On

We install Dex for SSO that works with Okta and/or static username/password.

Performance monitoring

We install Prometheus.

How did we do?

Importing an upstream Kubernetes cluster

Import of OpenShift Cluster

Contact