Stack Design
Updated
by Michael Delzer
Stack Design using the CLI
The Hub manifest is a flexible yaml based format for defining stacks. Fundamentally the structure follows:
- Metadata
- Component enumeration
- Lifecycle and dependencies
- Component Parameters
Components
- Component's
hub-component.yaml
must not usefromEnv:
to bindkind: user
parameters to the environment variables. The parameters are bound to the environment (if required) on Stack level - either inhub.yaml
orparams*.yaml
. - Explicitly export required parameters via
env:
and set defaults viavalue:
. kind:user
is for high-level input from user for which no sensible decision can be made automatically - like domain of the stack, number of worker nodes.default:
is forkind:user
- if no value is specified, then - if stdin is a TTY - user is asked by Hub CLI to accept the default or provide value. If not a TTY, then the default is used.- Set defaults in Makefile, e.g.
NAME ?= dev
, so that the component is usable out of CLI lifecycle (modulo templates), and also self-documented. - Do not rely on global environment set prior to stack deployment, e.g.
TF_VAR_name
(which should go away anyway). - Component parameters defined in
hub-component.yaml
(implicitkind: tech
) are scoped to the current component only - they are not visible to subsequent components (anymore). Useoutputs:
to provide values explicitly, for example:
outputs:
- name: component.postgresql.namespace
- name: component.postgresql.instance
value: ${component.postgresql.name}-rds
- To bind
hub.yaml
level parameter to a value which is not known in advance (component output) usekind: link
:
outputs:
- name: component.reactor.api
value: ${component.automation-hub.api}
kind: link
Name
<component>
iscomponents/<component>
of components Git repo.<component-name>
is the name of the component in the stack (pg1) - same as${hub.componentName}
:
components:
- name: pg1
source:
dir: components/postgresql-rds
- Finally,
component.*.name
- if required for particular component implementation, is something user may want to set, and may become TF_VAR_name, Kubernetes ingress name, etc. on the component level. We default it to${hub.componentName}
inhub-component.yaml
.
The state is stored as s3://mybucket/<domain>/<component>/<component-name>/terraform.tfstate
, see Terraform state in S3.
Stack
- Bind Instance specific parameters to environment variables via
fromEnv:
so that Template could be deployed multiple times, and parameterized via environment. - Avoid
fromEnv: TF_VAR_*
. - Avoid setting
env:
- the parameters must be exported by each component in component specific way. Ie. every component may have different idea whatTF_VAR_name
is for.
Envrc
- Avoid setting
TF_VAR_*
variable to influence component directly bypassinghub.yaml
/params*.yaml
wiring. Hub CLI will filter these out of children process environment by default (--os-environment). - Set
.envrc
vars forhub.yaml
/params*.yaml
parameters captured viafromEnv
.
Parameters ambiguity
Outputs
When more than a single instance of the same component is deployed in the stack - think two PostgreSQL databases, then there is an ambiguity because both components provides same outputs, ie. endpoint
and password
.
To bind input parameters of the pgweb
to specific PostgreSQL, use depends
:
components:
- name: pg1:
source: components/postgresql
- name: pg2:
source: components/postgresql
- name: pgweb1:
source: components/pgweb
depends:
- pg1
The outputs of depends
are searched in turn, then the global outputs pool.
Inputs
To set parameters for a specific component instance use component
:
parameters:
- name: component.postgresql
component: pg1
parameters:
The component <name>
first looks for parameters that are qualified with component: <name>
, then global parameters space.
Parameters evolution
Elaborate
During elaborate
Hub CLI creates a final manifest assembly that consists of:
- a leading document constructed from
hub.yaml
,params*.yaml
,fromStack
YAML-s, adding parameters markedkind: user
in*/hub-component.yaml
; - all component's
hub-component.yaml
with the component name (meta.name
) changed to matchhub.yaml
specified name.
Let's take AgileStack's infra Dev stack as an example. The stack is built from:
agilestacks/hub.yaml
- core infra;agilestacks/params.yaml
- parameters for the infra;dev/hub.yaml
- usesfromStack: agilestacks
and adds some components: prometheus, efk, pgweb;dev/params.yaml
- parameters for additional components;dev/params-dev.yaml
- Dev stack parameters (Reactor's Cloud Account base DNS domain selection).
Separation between hub.yaml
and params*.yaml
is a convinience.
The files will be parsed and parameters will be arranged by hub elaborate
in following order:
hub-component.yaml
of allagilestacks
components, in order ofcomponents:
hub-component.yaml
of alldev
componentsagilestacks/hub.yaml
agilestacks/params.yaml
dev/hub.yaml
dev/params.yaml
dev/params-dev.yaml
hub-component.yaml
kind: user
parameters (1) are brought to the top level - first document of hub.yaml.elaborate
with a component
qualifier. If there is a fromEnv
and kind
is not user
then a warning is emited.
Parameters from (1), (2) to (7) are merged in the order specified:
kind: user
takes priority overkind: tech
.value
,default
,fromEnv
,env
, andbrief
are overwritten.- In case top-level parameter - ie. from
hub.yaml
orparams*.yaml
has nocomponent
qualifier, thenelaborate
additionally checks for all previosly encountered parameters of the same name with a qualifier, then do merge into those. This the reason for the following inhub.yaml.elaborate
:
- name: component.auth-service.authApiSecret
kind: user
fromEnv: AUTH_API_SECRET
- name: component.auth-service.authApiSecret
component: auth-service
kind: user
fromEnv: AUTH_API_SECRET
Deploy
During deploy
following things happens:
- Top-level parameters are locked:
kind: user
parameters with no values are fetched fromfromEnv
(if any) or asked on the terminal (if stdin is a TTY);default
values are substituted;- expression evaluations are performed;
- cycles become errors;
- empty
kind: user
parameters become errors until supressed withempty: allow
; - all errors and warning are collected and reported.
- If there is a state file, then parameters from state are loaded and merged into global state - but only empty (stack) values are replaced (by state values) and warnings are emited.
- The locked set of top-level parameters are never changed after this step.
Top-level parameters can reference each other via expressions.
On each component deploy:
- Parameters are expanded:
- parameters with no values are searched in the top-level scope taking in account
component
anddepends
qualifiers; - expression evaluations are performed;
- unknown empty parameters become errors until supressed with
empty: allow
.
- Component parameters are never sent to top-level scope, but they are written into state file for informational purposes.
env:
variables are set, then merged on top of OS environ to create process environment for component invocation.- Outputs are collected and put onto global outputs scope - with a qualifier - of the component producing the output. So that when parameters qualifiers are proceed, the
name|qualifier
is searched first and then just plainname
. Outputs and parameters are processed independently although they looks very similar to the user. Outputs have higher precedence.
Component-level parameters cannot reference each other in expressions, only top-level parameters.
Set parameter value: " "
or empty: allow
to make empty parameter on hub-component.yaml
level not an error. During template processing and OS environment setup spaces are trimmed, effectively creating an empty substitution.
Capturing empty parameter from top-level scope does not require empty: allow
on component level. But it might be useful to proceed with deployment when optional upstream component failed thus no outputs were produced.
Conditional deployment
When component declares requires
- a list of capabilities that must be provided by prior components, platform stack, or environment - it became a list of hard requirements that must be satisfied before component (deploy) implementation is invoked. Failure to satisfy any requirement is hard error.
Component provides capabilities by declaring them in component's manifest provides
list. Additionally, dynamic capabilities are collected from deploy
output Provides:
block, similar to Outputs:
.
An aggregation of component's provides became (platform) stack provides.
A list of well known requires
are built into Hub CLI. Such requirements could be sourced from environment, such as aws
(a working AWS CLI). These requirements cannot be tuned.
Optional and Mandatory
A component might be declared optional on stack level
lifecycle:
optional:
- s3-bucket
Failure to deploy optional component is soft error. By default all components are mandatory. Those declared optional
became optional. If mandatory
components are specified, then everything else is optional.
lifecycle:
mandatory:
- kubernetes
- tiller
- traefik
Optional requirements
A requirement could be softened by tuning it via lifecycle
in stack manifest:
lifecycle:
requires:
optional:
- vault
- cdn:control-plane
- component.ingress.ssl.enabled:acm
vault
and cdn
are capabilities, control-plane
and acm
are component names. vault
requirement is relaxed for all components, while cdn
just for control-plane
. Requirement became optional. acm
is deployed only if ingress parameter component.ingress.ssl.enabled
requests TLS-enabled stack (evaluates to truth-y value, ie. everything else besides false
, ""
empty string, 0
, (unknown)
).
When an optional requirement cannot be satisfied for a particular component, the component deployment (or undeployment) implementation is not invoked.
Requirements tuning of Base stack will be merged with tuning of derived stack. The following rules are used:
requirement
supersedes all priorrequirement:component
- making requirement optional for everyone;requirement:component
supersedes priorrequirement
- making requirement optional forcomponent
. There could be multiple declaration for different components;:
erase all prior tuning;requirement:
erase all prior tuning forrequirement
;:component
erase all prior tuning forcomponent
;- otherwise the entry is appended.
Provides
Hub CLI parameter hub.provides
and HUB_PROVIDES
env var exports a space-separated list of currently known provides.