使用Terraform来管理Kubernetes的资源,例如ConfigMap和Secret等
Terraform是一种常用于在AWS等IaaS平台上构建和管理基础架构的工具。
与此同时,它还提供了用于管理Kubernetes资源的官方插件,通过使用这些插件来管理Kubernetes资源非常方便,所以我来介绍一下。
与直接使用kubectl操作资源相比,
terraform planでDryRunできる
その際、もちろん既存のリソースとの差分を見ることができる
リソース同士の参照(Docker RegistryのSecretをPodから見るとか)をコード上で表現でき、参照が壊れてると実行前にエラーにできる
変数や関数、モジュールを使用して定義を共通化できる
terraform destroyで全部なかったことにできる
在附近有什么好处吗?
在本地封闭环境中,使用Mac上安装的minikube从本地的terraform访问,来实际管理Kubernetes资源的过程,我将写下来。
请参考以下链接中推送的terraform项目,该项目在本文中使用:
由于minikube的安装非常简单,我建议您参考以下步骤进行安装。
Kubernetes供应商设置
为了通过Terraform设置Kubernetes集群,通常需要参考以下内容来配置认证信息,如API服务器的URL和用户名等。
然而,Minikube会将认证信息写入.kube/config文件中,而Terraform的Kubernetes Provider默认会读取.kube/config文件,因此如果要从本地访问Minikube环境,只需要进行简单的声明即可。
我们在这里创建一个名为terraform-kubernetes-demo的目录,并创建以下文件。
provider "kubernetes" {}
通过运行terraform init,可以下载Kubernetes Provider并将其配置在kubernetes-provider-demo目录中。
Initializing provider plugins...
- Checking for available provider plugins on https://releases.hashicorp.com...
- Downloading plugin for provider "kubernetes" (1.0.0)...
用Terraform来管理ConfigMap。
首先,让我们来管理ConfigMap。目前的Kubernetes提供商中,我认为这是最方便的ConfigMap和Secret管理方式。
tf文件的内容如下。
resource "kubernetes_config_map" "infra" {
metadata {
name = "infra-env"
}
data {
db_host = "dbhost:5432"
cache_host = "cache:6379"
}
}
现在,我们通过一个合适的文件名创建了上述文件,并在terraform plan命令中进行了Dry Run测试。
然后,我们会看到以下即将创建的ConfigMap的显示结果。
------------------------------------------------------------------------
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
+ kubernetes_config_map.infra
id: <computed>
data.%: "2"
data.cache_host: "cache:6379"
data.db_host: "dbhost:5432"
metadata.#: "1"
metadata.0.generation: <computed>
metadata.0.name: "infra-env"
metadata.0.namespace: "default"
metadata.0.resource_version: <computed>
metadata.0.self_link: <computed>
metadata.0.uid: <computed>
Plan: 1 to add, 0 to change, 0 to destroy.
------------------------------------------------------------------------
在实际创建之前能够确认内容真是太好了。
当执行 `terraform apply` 命令时,会在 Kubernetes 集群中实际创建该 ConfigMap。
terraform apply
# kubernetes_config_map.infra: Creating...
# data.%: "" => "2"
# data.cache_host: "" => "cache:6379"
# data.db_host: "" => "dbhost:5432"
# metadata.#: "" => "1"
# metadata.0.generation: "" => "<computed>"
# metadata.0.name: "" => "infra-env"
# metadata.0.namespace: "" => "default"
# metadata.0.resource_version: "" => "<computed>"
# metadata.0.self_link: "" => "<computed>"
# metadata.0.uid: "" => "<computed>"
# kubernetes_config_map.infra: Creation complete after # 0s (ID: default/infra-env)
# Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
kubectl describe cm infra-env
# Name: infra-env
# Namespace: default
# Labels: <none>
# Annotations: <none>
# Data
# ====
# cache_host:
# ----
# cache:6379
# db_host:
# ----
# dbhost:5432
# Events: <none>
此外,我们尝试按以下方式更改ConfigMap。
resource "kubernetes_config_map" "infra" {
metadata {
name = "infra-env"
}
data {
db_host = "dbhost2:5432" # 変更する
cache_host = "cache:6379"
}
}
在这里执行terraform plan,与其他提供者(例如AWS Provider)一样,会显示出差异。
------------------------------------------------------------------------
An execution plan has been generated and is shown below.
Resource actions are indicated with the following symbols:
~ update in-place
Terraform will perform the following actions:
~ kubernetes_config_map.infra
data.db_host: "dbhost:5432" => "dbhost2:5432"
Plan: 0 to add, 1 to change, 0 to destroy.
------------------------------------------------------------------------
在实际发生变更之前,我们可以通过这种方式来确认ConfigMap将如何被修改。
如果确认差异没有问题,那么我们将使用 terraform apply 来应用这些差异。
terraform apply
# kubernetes_config_map.infra: Refreshing state... (ID: default/infra-env)
# kubernetes_config_map.infra: Modifying... (ID: default/infra-env)
# data.db_host: "dbhost:5432" => "dbhost2:5432"
# kubernetes_config_map.infra: Modifications complete # after 0s (ID: default/infra-env)
使用Terraform来管理Secret.
与 ConfigMap 类似,Secret 也可以在 terraform 中进行管理。相比使用 kubectl 创建,terraform 还提供了直观的 Secret 创建方式,包括 Base64 编码的功能。
首先,创建一个tf文件。
resource "kubernetes_secret" "basic_auth" {
metadata {
name = "basic-auth"
}
data {
username = "admin"
password = "Admin"
}
type = "kubernetes.io/basic-auth"
}
执行terraform apply时,将创建上述内容的Secret。无需进行base64编码。
kubernetes_secret.basic_auth: Creating...
data.%: "" => "2"
data.password: "<sensitive>" => "<sensitive>"
data.username: "<sensitive>" => "<sensitive>"
metadata.#: "" => "1"
metadata.0.generation: "" => "<computed>"
metadata.0.name: "" => "basic-auth"
metadata.0.namespace: "" => "default"
metadata.0.resource_version: "" => "<computed>"
metadata.0.self_link: "" => "<computed>"
metadata.0.uid: "" => "<computed>"
type: "" => "kubernetes.io/basic-auth"
kubernetes_secret.basic_auth: Creation complete after 0s (ID: default/basic-auth)
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
那么,我们稍微修改一下这个tf文件。顺便说一下,我们也可以尝试使用变量。
variable "basicPassword" {
}
resource "kubernetes_secret" "basic_auth" {
metadata {
name = "basic-auth"
}
data {
username = "admin"
password = "${var.basicPassword}"
}
type = "kubernetes.io/basic-auth"
}
TF_VAR_basicPassword=aaa terraform plan
# An execution plan has been generated and is shown below.
# Resource actions are indicated with the following symbols:
# ~ update in-place
# Terraform will perform the following actions:
# ~ kubernetes_secret.basic_auth
# data.password: <sensitive> => <sensitive> (attribute changed)
# Plan: 0 to add, 1 to change, 0 to destroy.
在中国,只需提供一个选项进行本地化:用Terraform,我们可以使用变量、引用和函数,因此共享配置变得轻而易举。
Kubernetes提供商的Secret资源非常方便,因为它可以从文件或模板创建Secret,这对于配置Kubernetes时的Docker Registry身份验证信息非常有用。
整理收拾
您可以使用 `terraform destroy` 命令来删除上述创建的所有资源。(You can use the `terraform destroy` command to delete all the resources created above.)
terraform destroy
# kubernetes_config_map.infra: Refreshing state... (ID: default/infra-env)
# kubernetes_secret.basic_auth: Refreshing state... (ID: default/basic-auth)
# An execution plan has been generated and is shown below.
# Resource actions are indicated with the following symbols:
# - destroy
# Terraform will perform the following actions:
# - kubernetes_config_map.infra
# - kubernetes_secret.basic_auth
# Plan: 0 to add, 0 to change, 2 to destroy.
# Do you really want to destroy?
# Terraform will destroy all your managed infrastructure, as shown above.
# There is no undo. Only 'yes' will be accepted to confirm.
# Enter a value: yes
# kubernetes_config_map.infra: Destroying... (ID: default/infra-env)
# kubernetes_secret.basic_auth: Destroying... (ID: default/basic-auth)
# kubernetes_config_map.infra: Destruction complete after 0s
# kubernetes_secret.basic_auth: Destruction complete after 0s
# Destroy complete! Resources: 2 destroyed.
总结
Kubernetes提供程序可以管理各种资源,包括服务、Pod、命名空间、持久卷声明等(当前存在Pod但却没有部署集是个问题)。
由于terraform具有差异检测的功能,可以通过命令的退出代码来了解是否发生了差异,还可以通过可幂等性的命令管理资源,因此非常适合集成到CI中。如果您正在使用Jenkins等工具管理基础架构,那么建议尝试一下Kubernetes Provider。