使用Image Builder来创建适用于Cluster API的OVA

2 年 ago
文, 翔
6 minutes

“Image Builder是什么？”

Image Builder是一个用于创建预先安装了Kubernetes的虚拟机镜像的工具。它兼容AWS、Azure、vSphere等多个环境。它还支持多种基于不同的客机操作系统，如RHEL和Ubuntu。

我想要为vSphere创建虚拟机镜像，所以这次打算制作一个基于Ubuntu的OVA。

使用 Image Builder 创建 OVA 文件

关于Image Builder的使用方法，详情请参考以下链接。虽然描述可能有些过时，与实际操作略有不同，但整体步骤应该没有变化，我们可以参照此链接进行操作。

    • The Image Builder Book – Quick Start

 

    The Image Builder Book – Building Images for vSphere

顺便提一下，在Ubuntu 22.04上执行了以下命令。

安装所需的软件包。

首先，克隆 Image Builder 的存储库。

$ git clone https://github.com/kubernetes-sigs/image-builder.git

然后，切换到image/capi目录，执行make deps命令，安装所需的软件包。

$ cd image-builder/images/capi/
$ make deps
hack/image-grok-latest-flatcar-version.sh: line 9: jq: command not found
hack/ensure-ansible.sh
Collecting pip
  Downloading pip-22.3.1-py3-none-any.whl (2.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 2.1/2.1 MB 15.5 MB/s eta 0:00:00
Collecting wheel
  Downloading wheel-0.38.4-py3-none-any.whl (36 kB)
Installing collected packages: wheel, pip
  WARNING: The script wheel is installed in '/home/ubuntu/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
  WARNING: The scripts pip, pip3 and pip3.10 are installed in '/home/ubuntu/.local/bin' which is not on PATH.
  Consider adding this directory to PATH or, if you prefer to suppress this warning, use --no-warn-script-location.
Successfully installed pip-22.3.1 wheel-0.38.4
User's Python3 binary directory must be in $PATH
Location of package is:
hack/utils.sh: line 78: pip3: command not found
make: *** [Makefile:56: deps-ami] Error 1

由于找不到jq命令和pip3，我决定安装它们。

$ sudo apt isntall jq python3-pip

重新执行 make deps。

$ make deps
hack/ensure-ansible.sh
Collecting ansible-core==2.11.5
~~ snip ~~
Successfully installed ansible-core-2.11.5 packaging-22.0 resolvelib-0.5.4
User's Python3 binary directory must be in $PATH
Location of package is:
WARNING: Package(s) not found: ansible
make: *** [Makefile:56: deps-ami] Error 1

尽管安装了 Ansible，但仍然出现找不到 Ansible 包的错误。可能是路径设置不正确，因此请尝试找到 Ansible 命令的位置并设置正确的路径。

$ sudo find / -name "ansible"
/home/ubuntu/.local/bin/ansible
/home/ubuntu/.local/lib/python3.10/site-packages/ansible
/home/ubuntu/.local/lib/python3.10/site-packages/ansible_test/_data/injector/ansible
/home/ubuntu/workspace/image-builder/images/capi/ansible

$ export PATH="/home/ubuntu/.local/bin/:$PATH"

重新振作精神，重新执行 make deps，但这次提示没有 unzip 命令，所以需要安装 unzip。

$ make deps
hack/ensure-ansible.sh
Starting galaxy collection install process
Process install dependency map
Starting collection install process
~~ snip ~~
Required-by: 
hack/ensure-packer.sh
packer_1.8.3_linux_amd64.zip: OK
hack/ensure-packer.sh: line 52: unzip: command not found
make: *** [Makefile:58: deps-ami] Error 127

$ sudo apt install unzip -y

我认为这一次会成功，所以我重新运行了make deps，并且顺利完成了。

$ make deps
~~ snip ~~
Nothing to do. All requested collections are already installed. If you want to reinstall them, consider using `--force`.
hack/ensure-packer.sh
hack/ensure-goss.sh
Right version of binary present

使用Packer 在 vSphere 上创建虚拟机

ここからは、vSphere 向けの作成手順になります。 vsphere.json を埋めろ、ということなので、必要な項目を入力しておきます。

$ cat packer/ova/vsphere.json 
{
    "vcenter_server":"FQDN of vcenter",
    "username":"vcenter_username",
    "password":"vcenter_password",
    "datastore":"template_datastore",
    "folder": "template_folder_on_vcenter",
    "cluster": "esxi_cluster_used_for_template_creation",
    "network": "network_attached_to_template",
    "insecure_connection": "false",
    "template": "base_template_used_by_clone_builder",
    "create_snbapshot": "creates a snaphot on base OVA after building",
    "linked_clone": "Uses link cloning in vsphere-clone builder: true, by default"
}

次に、make deps-ova を実行して、OVA 作成に必要なパッケージがインストールされているか確認します。

$ make deps-ova
hack/ensure-ansible.sh
Starting galaxy collection install process
Nothing to do. All requested collections are already installed. If you want to reinstall them, consider using `--force`.
hack/ensure-ansible-windows.sh
hack/ensure-packer.sh
hack/ensure-goss.sh
Right version of binary present
hack/ensure-ovftool.sh

在完成了此前的步骤后，作为可选项，通过使用 ovftool，我们可以创建 OVA 包，因此也要将 ovftool 安装好。

$ sudo ./VMware-ovftool-4.4.3-18663434-lin.x86_64.bundle

另外，我注意到 packer 的路径设置不正确，所以我会将路径设置为 “./image-builder/images/capi.local/bin/packer”。

现在，我们执行以下命令来创建 OVA 文件。

$ IB_OVFTOOL=1 IB_OVFTOOL_ARGS="--allowExtraConfig" make build-node-ova-vsphere-ubuntu-2004

剛剛想到順利進行的時候，突然出現了一個錯誤…

vsphere: output will be in this color.

==> vsphere: File /home/ubuntu/.cache/packer/48e4ec4daa32571605576c5566f486133ecc271f.iso already uploaded; continuing
==> vsphere: File [iscsi01] packer_cache//48e4ec4daa32571605576c5566f486133ecc271f.iso already exists; skipping upload.
==> vsphere: Creating VM...
~~ snip ~~
==> vsphere: Waiting for SSH to become available...
==> vsphere: Connected to SSH!
==> vsphere: Provisioning with shell script: ./packer/files/flatcar/scripts/bootstrap-flatcar.sh
==> vsphere: Provisioning with Ansible...
    vsphere: Setting up proxy adapter for Ansible....
==> vsphere: Executing Ansible: ansible-playbook -e packer_build_name="vsphere" -e packer_*****_type=vsphere-iso -e packer_http_addr=192.168.10.151:8394 --ssh-extra-args '-o IdentitiesOnly=yes' --extra-vars containerd_url=https://github.com/containerd/containerd/releases/download/v1.6.2/cri-containerd-cni-1.6.2-linux-amd64.tar.gz containerd_sha256=91f1087d556ecfb1f148743c8ee78213cd19e07c22787dae07fe6b9314bec121 pause_image=k8s.gcr.io/pause:3.6 containerd_additional_settings= containerd_cri_socket=/var/run/containerd/containerd.sock containerd_version=1.6.2 crictl_url=https://github.com/kubernetes-sigs/cri-tools/releases/download/v1.23.0/crictl-v1.23.0-linux-amd64.tar.gz crictl_sha256=https://github.com/kubernetes-sigs/cri-tools/releases/download/v1.23.0/crictl-v1.23.0-linux-amd64.tar.gz.sha256 crictl_source_type=pkg custom_role_names="" firstboot_custom_roles_pre="" firstboot_custom_roles_post="" node_custom_roles_pre="" node_custom_roles_post="" disable_public_repos=false extra_debs="" extra_repos="" extra_rpms="" http_proxy= https_proxy= kubeadm_template=etc/kubeadm.yml kubernetes_cni_http_source=https://github.com/containernetworking/plugins/releases/download kubernetes_cni_http_checksum=sha256:https://storage.googleapis.com/k8s-artifacts-cni/release/v1.1.1/cni-plugins-linux-amd64-v1.1.1.tgz.sha256 kubernetes_http_source=https://dl.k8s.io/release kubernetes_container_registry=registry.k8s.io kubernetes_rpm_repo=https://packages.cloud.google.com/yum/repos/kubernetes-el7-x86_64 kubernetes_rpm_gpg_key="https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg" kubernetes_rpm_gpg_check=True kubernetes_deb_repo="https://apt.kubernetes.io/ kubernetes-xenial" kubernetes_deb_gpg_key=https://packages.cloud.google.com/apt/doc/apt-key.gpg kubernetes_cni_deb_version=1.1.1-00 kubernetes_cni_rpm_version=1.1.1-0 kubernetes_cni_semver=v1.1.1 kubernetes_cni_source_type=pkg kubernetes_semver=v1.23.10 kubernetes_source_type=pkg kubernetes_load_additional_imgs=false kubernetes_deb_version=1.23.10-00 kubernetes_rpm_version=1.23.10-0 no_proxy= pip_conf_file= python_path= redhat_epel_rpm=https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm epel_rpm_gpg_key= reenable_public_repos=true remove_extra_repos=false systemd_prefix=/usr/lib/systemd sysusr_prefix=/usr sysusrlocal_prefix=/usr/local load_additional_components=false additional_registry_images=false additional_registry_images_list= additional_url_images=false additional_url_images_list= additional_executables=false additional_executables_list= additional_executables_destination_path= build_target=virt amazon_ssm_agent_rpm= --extra-vars guestinfo_datasource_slug=https://raw.githubusercontent.com/vmware/cloud-init-vmware-guestinfo guestinfo_datasource_ref=v1.4.1 guestinfo_datasource_script=https://raw.githubusercontent.com/vmware/cloud-init-vmware-guestinfo/v1.4.1/install.sh --extra-vars  -e ansible_ssh_private_key_file=/tmp/ansible-key2998066837 -i /tmp/packer-provisioner-ansible3438111689 /home/ubuntu/workspace/image-*****/images/capi/ansible/firstboot.yml
    vsphere:
    vsphere: PLAY [all] *********************************************************************
==> vsphere: failed to handshake
    vsphere:
    vsphere: TASK [Gathering Facts] *********************************************************
    vsphere: fatal: [default]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: Unable to negotiate with 127.0.0.1 port 41355: no matching host key type found. Their offer: ssh-rsa", "unreachable": true}
~~ snip ~~

您好，看起来对于连接到本地主机（localhost）的 SSH 出现了故障。当搜索 “no matching host key type found” 时，发现一篇文章提到只需要在 .ssh/config 文件中添加以下3行内容即可解决问题，所以我会进行添加。

Host *
   HostKeyAlgorithms=+ssh-rsa
   PubkeyAcceptedAlgorithms=+ssh-rsa

再次执行创建 OVA 文件的命令时，Ansible 将执行各种处理，大约经过20分钟后，OVA 文件被成功创建。

$ IB_OVFTOOL=1 IB_OVFTOOL_ARGS="--allowExtraConfig" make build-node-ova-vsphere-ubuntu-2004
~~ snip ~~
==> Wait completed after 20 minutes 17 seconds

==> Builds finished. The artifacts of successful builds are:
--> vsphere: ubuntu-2004-kube-v1.23.10
--> vsphere: ubuntu-2004-kube-v1.23.10
--> vsphere: ubuntu-2004-kube-v1.23.10
--> vsphere: ubuntu-2004-kube-v1.23.10

结束

在创建 Cluster API 用的 OVA 过程中，虽然出现了一些意外错误，但比预想的要简单。过去我们一直使用官方提供的映像文件，但以后我想要根据 Kubernetes 版本和宿主操作系统版本，自己创建验证用的 OVA。

✖
广告
将在 10 秒后关闭
bannerAds