关于 Openshift 4.6 的主节点/工作节点资源消耗量
本文是关于Openshift 4.6的Master/Worker节点资源消耗情况的简易汇总，包括实际命令输出。
这是在部署Openshift集群并将Openshift镜像存储库设置为托管的状态下的情况。
在进行大小估算时，可能会忽略CoreOS和系统Pod所预留和消耗的资源量，因此仅供参考。
建构
-
- vSphere UPIデプロイパターン
- Master 3 Node/Worker 2 Node構成
[root@prov ~]# oc get nodes
NAME STATUS ROLES AGE VERSION
master1 Ready master 45d v1.19.14+fcff70a
master2 Ready master 45d v1.19.14+fcff70a
master3 Ready master 45d v1.19.14+fcff70a
worker1 Ready app,worker 45d v1.19.14+fcff70a
worker2 Ready app,worker 45d v1.19.14+fcff70a
[root@prov ~]# oc version
Client Version: 4.7.34
Server Version: 4.6.49
Kubernetes Version: v1.19.14+fcff70a
[root@prov ~]#
各Master Nodeへのリソース搭載量
CPU：8
MEM：8GB
Disk：100GB
各Worker Nodeへのリソース搭載量
CPU：8
MEM：8GB
Disk：100GB
由于管理节点数量、部署运算符和应用程序Pod数量的不同，上述资源量仅供参考，实际资源量可能会有较大变化。因此，建议通过Red Hat的网站来计算适当的资源量。
使用oc描述命令的摘录
- Master Node
[root@prov ~]# oc describe node master1
Name: master1
Roles: master
Capacity:
cpu: 8
ephemeral-storage: 104305644Ki
hugepages-2Mi: 0
memory: 8160732Ki
pods: 250
Allocatable:
cpu: 7500m
ephemeral-storage: 95054339528
hugepages-2Mi: 0
memory: 7009756Ki
pods: 250
ProviderID: vsphere://423d9531-0c8e-1964-5d5d-8f35fc223822
Non-terminated Pods: (31 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits AGE
--------- ---- ------------ ---------- --------------- ------------- ---
openshift-apiserver apiserver-5648856c64-zlx95 110m (1%) 0 (0%) 250Mi (3%) 0 (0%) 45d
openshift-authentication oauth-openshift-7565c7b94d-2kt7n 10m (0%) 0 (0%) 50Mi (0%) 0 (0%) 45d
openshift-cluster-node-tuning-operator tuned-nlfq5 10m (0%) 0 (0%) 50Mi (0%) 0 (0%) 45d
openshift-cluster-samples-operator cluster-samples-operator-7dbc7958f-6ltt8 20m (0%) 0 (0%) 0 (0%) 0 (0%) 45d
openshift-cluster-storage-operator csi-snapshot-controller-5458888996-4zlbd 10m (0%) 0 (0%) 50Mi (0%) 0 (0%) 45d
openshift-console downloads-9776476c4-pd6xm 10m (0%) 0 (0%) 50Mi (0%) 0 (0%) 45d
openshift-controller-manager controller-manager-qdzgz 100m (1%) 0 (0%) 100Mi (1%) 0 (0%) 47h
openshift-dns dns-default-4tbbp 65m (0%) 0 (0%) 110Mi (1%) 512Mi (7%) 45d
openshift-etcd etcd-master1 430m (5%) 0 (0%) 860Mi (12%) 0 (0%) 45d
openshift-etcd etcd-quorum-guard-78d56fdc5b-lnc77 10m (0%) 0 (0%) 5Mi (0%) 0 (0%) 45d
openshift-image-registry node-ca-j6cv5 10m (0%) 0 (0%) 10Mi (0%) 0 (0%) 45d
openshift-kube-apiserver-operator kube-apiserver-operator-6cc8455974-nldsv 10m (0%) 0 (0%) 50Mi (0%) 0 (0%) 45d
openshift-kube-apiserver kube-apiserver-master1 340m (4%) 0 (0%) 1224Mi (17%) 0 (0%) 47h
openshift-kube-controller-manager kube-controller-manager-master1 100m (1%) 0 (0%) 500Mi (7%) 0 (0%) 45d
openshift-kube-scheduler openshift-kube-scheduler-master1 25m (0%) 0 (0%) 150Mi (2%) 0 (0%) 45d
openshift-machine-api cluster-autoscaler-operator-69876758dd-6bhfw 30m (0%) 0 (0%) 70Mi (1%) 0 (0%) 45d
openshift-machine-api machine-api-controllers-654d4df8c9-q8plc 70m (0%) 0 (0%) 140Mi (2%) 0 (0%) 45d
openshift-machine-api machine-api-operator-5ffdfc86c-fk5ct 20m (0%) 0 (0%) 70Mi (1%) 0 (0%) 45d
openshift-machine-config-operator machine-config-controller-656f69c9bb-842xs 20m (0%) 0 (0%) 50Mi (0%) 0 (0%) 45d
openshift-machine-config-operator machine-config-daemon-qh79g 40m (0%) 0 (0%) 100Mi (1%) 0 (0%) 45d
openshift-machine-config-operator machine-config-server-cz6zv 20m (0%) 0 (0%) 50Mi (0%) 0 (0%) 45d
openshift-monitoring node-exporter-hlgvc 9m (0%) 0 (0%) 210Mi (3%) 0 (0%) 45d
openshift-multus multus-5mb6t 10m (0%) 0 (0%) 150Mi (2%) 0 (0%) 45d
openshift-multus multus-admission-controller-kn29z 20m (0%) 0 (0%) 20Mi (0%) 0 (0%) 45d
openshift-multus network-metrics-daemon-c8vgz 20m (0%) 0 (0%) 120Mi (1%) 0 (0%) 45d
openshift-oauth-apiserver apiserver-698fdcc744-rqnh5 150m (2%) 0 (0%) 200Mi (2%) 0 (0%) 45d
openshift-operator-lifecycle-manager packageserver-75b9bddfdd-blt8s 10m (0%) 0 (0%) 50Mi (0%) 0 (0%) 45d
openshift-sdn ovs-fvnpm 100m (1%) 0 (0%) 400Mi (5%) 0 (0%) 45d
openshift-sdn sdn-controller-x2csg 10m (0%) 0 (0%) 50Mi (0%) 0 (0%) 45d
openshift-sdn sdn-d5jvw 110m (1%) 0 (0%) 220Mi (3%) 0 (0%) 45d
openshift-service-ca service-ca-5ff99fbc4c-zs9w9 10m (0%) 0 (0%) 120Mi (1%) 0 (0%) 45d
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 1909m (25%) 0 (0%)
memory 5479Mi (80%) 512Mi (7%)
ephemeral-storage 0 (0%) 0 (0%)
hugepages-2Mi 0 (0%) 0 (0%)
- Worker Node
[root@prov ~]# oc describe node worker1
Name: worker1
Roles: app,worker
Capacity:
cpu: 8
ephemeral-storage: 104305644Ki
hugepages-2Mi: 0
memory: 10225132Ki
pods: 250
Allocatable:
cpu: 7500m
ephemeral-storage: 95054339528
hugepages-2Mi: 0
memory: 9074156Ki
pods: 250
ProviderID: vsphere://423da5a4-0e4e-35f5-07a5-e2bf8e003d88
Non-terminated Pods: (21 in total)
Namespace Name CPU Requests CPU Limits Memory Requests Memory Limits AGE
--------- ---- ------------ ---------- --------------- ------------- ---
openshift-cluster-node-tuning-operator tuned-7b2mg 10m (0%) 0 (0%) 50Mi (0%) 0 (0%) 45d
openshift-dns dns-default-xftxt 65m (0%) 0 (0%) 110Mi (1%) 512Mi (5%) 45d
openshift-image-registry image-registry-549fc8689d-j9cmz 100m (1%) 0 (0%) 256Mi (2%) 0 (0%) 28d
openshift-image-registry node-ca-rfxtk 10m (0%) 0 (0%) 10Mi (0%) 0 (0%) 45d
openshift-ingress router-default-76fd9bbc66-ldqld 100m (1%) 0 (0%) 256Mi (2%) 0 (0%) 28d
openshift-kube-storage-version-migrator migrator-68bc7c89f-9wx22 100m (1%) 0 (0%) 200Mi (2%) 0 (0%) 28d
openshift-machine-config-operator machine-config-daemon-ts248 40m (0%) 0 (0%) 100Mi (1%) 0 (0%) 45d
openshift-monitoring alertmanager-main-0 8m (0%) 0 (0%) 270Mi (3%) 0 (0%) 47h
openshift-monitoring grafana-575fbc5464-n4dz9 5m (0%) 0 (0%) 120Mi (1%) 0 (0%) 28d
openshift-monitoring kube-state-metrics-69b4fbb79-n4s5s 4m (0%) 0 (0%) 120Mi (1%) 0 (0%) 28d
openshift-monitoring node-exporter-rs75q 9m (0%) 0 (0%) 210Mi (2%) 0 (0%) 45d
openshift-monitoring openshift-state-metrics-7fb987f599-4vgxr 3m (0%) 0 (0%) 190Mi (2%) 0 (0%) 28d
openshift-monitoring prometheus-adapter-84dc49f498-8p24m 1m (0%) 0 (0%) 25Mi (0%) 0 (0%) 47h
openshift-monitoring prometheus-k8s-1 75m (1%) 0 (0%) 1194Mi (13%) 0 (0%) 47h
openshift-monitoring telemeter-client-97449bd78-rp7b4 3m (0%) 0 (0%) 20Mi (0%) 0 (0%) 28d
openshift-monitoring thanos-querier-69875575db-fsc9d 9m (0%) 0 (0%) 92Mi (1%) 0 (0%) 28d
openshift-monitoring thanos-querier-69875575db-hmw7g 9m (0%) 0 (0%) 92Mi (1%) 0 (0%) 28d
openshift-multus multus-hmwxt 10m (0%) 0 (0%) 150Mi (1%) 0 (0%) 45d
openshift-multus network-metrics-daemon-z6tsz 20m (0%) 0 (0%) 120Mi (1%) 0 (0%) 45d
openshift-sdn ovs-lbfnl 100m (1%) 0 (0%) 400Mi (4%) 0 (0%) 45d
openshift-sdn sdn-qxvxk 110m (1%) 0 (0%) 220Mi (2%) 0 (0%) 45d
Allocated resources:
(Total limits may be over 100 percent, i.e., overcommitted.)
Resource Requests Limits
-------- -------- ------
cpu 791m (10%) 0 (0%)
memory 4205Mi (47%) 512Mi (5%)
ephemeral-storage 0 (0%) 0 (0%)
hugepages-2Mi 0 (0%) 0 (0%)
[root@prov ~]#
资源考虑因素
-
- 各ノード、500mのCPUリソースが予約されている。（恐らくCoreOS分）
Openshiftは 1Core = 1000m で計算
Allocatable にて 7500m となっている。(8000m – 500m = 7500m)
各ノード、約1GBのMEMリソースが予約されている。（恐らくCoreOS分）
リソースクォータ値について
上記出力結果より、CPUクォータに Master Nodeは1909m、Worker Nodeは791mが設定されている。
上記出力結果より、MEMクォータに Master Nodeは5479Mi、Worker Nodeは4205Miが設定されている。
デプロイするOperatorによりリソース消費量は大きく変わるため余裕を持ったサイジングが必要
Openshift Logging(Elasticsearch/Fluentd/Kibana) Operatorをデプロイすると更にリソース予約・消費量が上昇する。
ログ流量によってリソース消費量が大きく異なるため要注意
Prometheusについては収集対象の数により異なるためアプリケーションコンテナが多くなる環境は要注意
Compliance、File Integrity Operatorなどは全てのNodeでPodが起動するため要注意（インフラノードを用意してもスキャン用Podが起動する）
因为在Openshift中存在各种运算器，且不清楚将消耗多少资源，所以如果考虑在生产环境中使用，最好进行概念验证（PoC）。
以上
