[工作日志] 尝试参加 GCP 亲身实操研讨会 Compute Engine 篇

2 年 ago

雅, 悟

10 minutes

这是什么？

GCEを建てるときのネットワーク構成をどうすれば良いのか知りたかったので触ってみた次第です

表題の章をやってみたので、作業記録を残しておきます。
ついでにサービスアカウント設定どうなってんねんって思ったのでちょっとだけ触ってみました。

实践材料

https://goo.gle/startgcp1

记录任务进度

创建虚拟机实例

gcloud ssh 使うなら基本はGoogle管理の鍵が安全な気がする。どうなんだろ

当你点击 SSH 控制台旁边的三角形▼时，会显示 gcloud 命令，该命令用于显示连接到 VM 的 SSH 命令。

$ gcloud beta compute --project "[YOUR_PROJECT_ID]" ssh --zone "asia-northeast1-a" "www1"

因为多次使用了gcloud命令，所以不会出现类似幻灯片的对话界面。

$ gcloud beta compute --project "[YOUR_PROJECT_ID]" ssh --zone "asia-northeast1-a" "www1"
Warning: Permanently added 'compute.1501268411356974593' (ECDSA) to the list of known hosts.
Linux www1 4.9.0-11-amd64 #1 SMP Debian 4.9.189-3+deb9u2 (2019-11-11) x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
username@www1:~$
username@www1:~$

提供网页

安装Apache。

$ sudo apt update
Get:1 http://security.debian.org stretch/updates InRelease [94.3 kB]
Ign:2 http://deb.debian.org/debian stretch InRelease                                                
Get:3 http://deb.debian.org/debian stretch-updates InRelease [91.0 kB]                              
Get:4 http://deb.debian.org/debian stretch-backports InRelease [91.8 kB]                                                     
Hit:5 http://deb.debian.org/debian stretch Release         
Get:6 http://packages.cloud.google.com/apt cloud-sdk-stretch InRelease [6,377 B]
Get:7 http://packages.cloud.google.com/apt google-compute-engine-stretch-stable InRelease [3,843 B]
Hit:8 http://packages.cloud.google.com/apt google-cloud-packages-archive-keyring-stretch InRelease
Get:9 http://security.debian.org stretch/updates/main Sources [201 kB]
Get:10 http://security.debian.org stretch/updates/main amd64 Packages [508 kB]
Get:11 http://security.debian.org stretch/updates/main Translation-en [224 kB]     
Get:12 http://deb.debian.org/debian stretch-backports/main Sources.diff/Index [27.8 kB]
Get:13 http://deb.debian.org/debian stretch-backports/main amd64 Packages.diff/Index [27.8 kB]
Get:14 http://deb.debian.org/debian stretch-backports/main Sources 2019-11-26-0814.05.pdiff [33 B]
Get:15 http://deb.debian.org/debian stretch-backports/main Sources 2019-11-26-1413.09.pdiff [876 B]
Get:16 http://deb.debian.org/debian stretch-backports/main amd64 Packages 2019-11-26-1413.09.pdiff [544 B]
Get:15 http://deb.debian.org/debian stretch-backports/main Sources 2019-11-26-1413.09.pdiff [876 B]
Get:16 http://deb.debian.org/debian stretch-backports/main amd64 Packages 2019-11-26-1413.09.pdiff [544 B]
Get:18 http://packages.cloud.google.com/apt google-compute-engine-stretch-stable/main amd64 Packages [1,317 B]
Fetched 1,278 kB in 1s (897 kB/s)                  
Reading package lists... Done
Building dependency tree       
Reading state information... Done
All packages are up to date.

$ sudo apt update
Get:1 http://security.debian.org stretch/updates InRelease [94.3 kB]
Ign:2 http://deb.debian.org/debian stretch InRelease                                                
Get:3 http://deb.debian.org/debian stretch-updates InRelease [91.0 kB]                              
Get:4 http://deb.debian.org/debian stretch-backports InRelease [91.8 kB]                                                     
Preparing to unpack .../17-xml-core_0.17_all.deb ...
...

Created symlink /etc/systemd/system/multi-user.target.wants/apache2.service → /lib/systemd/system/apache2.service.
Created symlink /etc/systemd/system/multi-user.target.wants/apache-htcacheclean.service → /lib/systemd/system/apache-htcacheclean.service.
Processing triggers for libc-bin (2.24-11+deb9u4) ...
Processing triggers for sgml-base (1.29) ...
Processing triggers for systemd (232-25+deb9u12) ...

点击控制台的IP地址。由于要连接到HTTPS，因此请在地址栏中将其更改为HTTP并连接。出现了Apache的初始界面。

（练习1）跨区通信

做下列准备。 (Zuò .)

在美国地区创建www2，并连接到默认网络。

既然已经完成了1，那就执行2。按照创建www1时的步骤，在幻灯片中操作控制台。请将地区设置为us-central1，将区域设置为us-central1-a。

请从本地终端发起ping命令，以检查连接到ASIA区域和US区域的虚拟机实例（使用外部IP地址）的延迟。

因为FW没有设置任何东西，所以应该可以通过。

$ ping 35.200.105.40
PING 35.200.105.40 (35.200.105.40): 56 data bytes
64 bytes from 35.200.105.40: icmp_seq=0 ttl=62 time=5.790 ms
64 bytes from 35.200.105.40: icmp_seq=1 ttl=62 time=6.529 ms
64 bytes from 35.200.105.40: icmp_seq=2 ttl=62 time=9.218 ms
64 bytes from 35.200.105.40: icmp_seq=3 ttl=62 time=5.933 ms
64 bytes from 35.200.105.40: icmp_seq=4 ttl=62 time=6.276 ms
64 bytes from 35.200.105.40: icmp_seq=5 ttl=62 time=6.023 ms
64 bytes from 35.200.105.40: icmp_seq=6 ttl=62 time=5.746 ms
64 bytes from 35.200.105.40: icmp_seq=7 ttl=62 time=6.350 ms
^C
--- 35.200.105.40 ping statistics ---
8 packets transmitted, 8 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 5.746/6.483/9.218/1.065 ms

$ ping 35.184.64.37
PING 35.184.64.37 (35.184.64.37): 56 data bytes
64 bytes from 35.184.64.37: icmp_seq=0 ttl=54 time=127.880 ms
64 bytes from 35.184.64.37: icmp_seq=1 ttl=54 time=127.847 ms
64 bytes from 35.184.64.37: icmp_seq=2 ttl=54 time=127.619 ms
64 bytes from 35.184.64.37: icmp_seq=3 ttl=54 time=144.138 ms
64 bytes from 35.184.64.37: icmp_seq=4 ttl=54 time=127.666 ms
64 bytes from 35.184.64.37: icmp_seq=5 ttl=54 time=127.811 ms
64 bytes from 35.184.64.37: icmp_seq=6 ttl=54 time=127.514 ms
64 bytes from 35.184.64.37: icmp_seq=7 ttl=54 time=127.366 ms
^C
--- 35.184.64.37 ping statistics ---
8 packets transmitted, 8 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 127.366/129.730/144.138/5.448 ms

RTT如下（单位：毫秒）。

ゾーンminavgmaxstddevasia-northeast1-a5.7466.4839.2181.065us-central1-a127.366129.730144.1385.448

标准差（stddev）是用来表示数值的离散程度的。可以确认的是，位于不同区域（美国中部1）的往返延迟时间（RTT）较大，这是当然的事情。✅

请通过SSH登录到www1，并执行ping命令以向www2发送请求（执行ping命令[使用www2的内部IP地址]）。

$ ping 10.128.0.2
PING 10.128.0.2 (10.128.0.2) 56(84) bytes of data.
64 bytes from 10.128.0.2: icmp_seq=1 ttl=64 time=126 ms
64 bytes from 10.128.0.2: icmp_seq=2 ttl=64 time=125 ms
64 bytes from 10.128.0.2: icmp_seq=3 ttl=64 time=125 ms
64 bytes from 10.128.0.2: icmp_seq=4 ttl=64 time=125 ms
64 bytes from 10.128.0.2: icmp_seq=5 ttl=64 time=125 ms
64 bytes from 10.128.0.2: icmp_seq=6 ttl=64 time=125 ms
64 bytes from 10.128.0.2: icmp_seq=7 ttl=64 time=125 ms
64 bytes from 10.128.0.2: icmp_seq=8 ttl=64 time=125 ms
^C
--- 10.128.0.2 ping statistics ---
8 packets transmitted, 8 received, 0% packet loss, time 7011ms
rtt min/avg/max/mdev = 125.371/125.565/126.577/0.383 ms

统计数据如下

minavgmaxstddev125.371125.565126.5770.383

ばらつきが少ないってのは低遅延系のアプリにとって良い特性だと思う。

（练习2）创建私人网络。

在我的网络中创建一个名为www3的虚拟机，并在创建www3时，在控制台上设置连接到mynetwork。

可以的男人。

确认通信

确认默认网络和mynetwork网络已经分开。

请登录到www3，并对www1执行ping操作。

$ ping 10.146.0.11
PING 10.146.0.11 (10.146.0.11) 56(84) bytes of data.
^C
--- 10.146.0.11 ping statistics ---
13 packets transmitted, 0 received, 100% packet loss, time 12275ms

无法连上

登录到www3并对www1的公共IP地址执行ping命令。

请进行

$ ping 35.200.105.40
PING 35.200.105.40 (35.200.105.40) 56(84) bytes of data.
64 bytes from 35.200.105.40: icmp_seq=1 ttl=55 time=256 ms
64 bytes from 35.200.105.40: icmp_seq=2 ttl=55 time=255 ms
64 bytes from 35.200.105.40: icmp_seq=3 ttl=55 time=256 ms
64 bytes from 35.200.105.40: icmp_seq=4 ttl=55 time=256 ms
64 bytes from 35.200.105.40: icmp_seq=5 ttl=55 time=256 ms
64 bytes from 35.200.105.40: icmp_seq=6 ttl=55 time=255 ms
^C
--- 35.200.105.40 ping statistics ---
6 packets transmitted, 6 received, 0% packet loss, time 5006ms
rtt min/avg/max/mdev = 255.711/256.056/256.536/0.564 ms

连接

（练习3）添加磁盘

创建一个磁盘并将其挂载到虚拟机上。

添加磁盘

通过控制台操作，在www1上添加了一个硬盘。
连接到实例并通过SSH命令df来确认可用的硬盘。

$ sudo df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            286M     0  286M   0% /dev
tmpfs            60M  2.5M   57M   5% /run
/dev/sda1       9.8G  1.2G  8.1G  13% /
tmpfs           297M     0  297M   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           297M     0  297M   0% /sys/fs/cgroup

创建挂载点

$ sudo mkdir -p /mnt/disk2

请确认磁盘名称。

$ ls -l /dev/disk/by-id/google-*
lrwxrwxrwx 1 root root  9 Nov 28 03:41 /dev/disk/by-id/google-disk-2 -> ../../sdb
lrwxrwxrwx 1 root root  9 Nov 28 02:29 /dev/disk/by-id/google-www1 -> ../../sda
lrwxrwxrwx 1 root root 10 Nov 28 02:29 /dev/disk/by-id/google-www1-part1 -> ../../sda1

/google-disk-2 -> ../../sdb 这就是了。这是/dev/disk/sdb的符号链接。可以通过GCP磁盘名称进行识别。明白了。

将此进行格式化。

$ sudo mkfs.ext4 -F -E lazy_itable_init=0,lazy_journal_init=0,discard /dev/disk/by-id/google-disk-2
mke2fs 1.43.4 (31-Jan-2017)
Discarding device blocks: done               
Creating filesystem with 131072000 4k blocks and 32768000 inodes
Filesystem UUID: a1d2cc1f-d184-4d4c-ba55-086ac7508629
Superblock backups stored on blocks:
        32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
        4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
        102400000

Allocating group tables: done     
Writing inode tables: done     
Creating journal (262144 blocks): done
Writing superblocks and filesystem accounting information: done

挂载硬盘。

$ sudo mount -o discard,defaults /dev/disk/by-id/google-disk-2 /mnt/disk2

确认。

$ sudo df -h
Filesystem      Size  Used Avail Use% Mounted on
udev            286M     0  286M   0% /dev
tmpfs            60M  2.5M   57M   5% /run
/dev/sda1       9.8G  1.2G  8.1G  13% /
tmpfs           297M     0  297M   0% /dev/shm
tmpfs           5.0M     0  5.0M   0% /run/lock
tmpfs           297M     0  297M   0% /sys/fs/cgroup
/dev/sdb        492G   73M  467G   1% /mnt/disk2

$ cd /mnt/disk2
$ ls -al
total 24
drwxr-xr-x 3 root root  4096 Nov 28 03:50 .
drwxr-xr-x 3 root root  4096 Nov 28 03:44 ..
drwx------ 2 root root 16384 Nov 28 03:50 lost+found

/dev/sdb 出现了。

（练习4）Stackdriver

请将代理人放入。

$ curl -sSO https://dl.google.com/cloudagents/install-monitoring-agent.sh
$ sudo bash install-monitoring-agent.sh
==============================================================================
Starting installation of stackdriver-agent
==============================================================================

Installing agent for Debian or Ubuntu.
OK
Hit:1 http://security.debian.org stretch/updates InRelease
Ign:2 http://deb.debian.org/debian stretch InRelease
Hit:3 http://deb.debian.org/debian stretch-updates InRelease
Hit:4 http://deb.debian.org/debian stretch-backports InRelease
Hit:5 http://deb.debian.org/debian stretch Release
Get:7 http://packages.cloud.google.com/apt google-cloud-monitoring-stretch InRelease [3,806 B]
Hit:8 http://packages.cloud.google.com/apt cloud-sdk-stretch InRelease
Hit:9 http://packages.cloud.google.com/apt google-compute-engine-stretch-stable InRelease
Hit:10 http://packages.cloud.google.com/apt google-cloud-packages-archive-keyring-stretch InRelease
Get:11 http://packages.cloud.google.com/apt google-cloud-monitoring-stretch/main amd64 Packages [1,903 B]
Fetched 5,709 B in 0s (5,825 B/s)                    
Reading package lists... Done
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following additional packages will be installed:
  libltdl7 libpython2.7 libyajl2
Suggested packages:
  default-jre libhiredis0.13 libmariadbclient18 libpq5
The following NEW packages will be installed:
  libltdl7 libpython2.7 libyajl2 stackdriver-agent
0 upgraded, 4 newly installed, 0 to remove and 0 not upgraded.
Need to get 3,235 kB of archives.
After this operation, 9,982 kB of additional disk space will be used.
Get:1 http://deb.debian.org/debian stretch/main amd64 libltdl7 amd64 2.4.6-2 [389 kB]
Get:2 http://deb.debian.org/debian stretch/main amd64 libpython2.7 amd64 2.7.13-2+deb9u3 [1,071 kB]
Get:3 http://deb.debian.org/debian stretch/main amd64 libyajl2 amd64 2.1.0-2+b3 [23.2 kB]
Get:4 http://packages.cloud.google.com/apt google-cloud-monitoring-stretch/main amd64 stackdriver-agent amd64 5.5.2-1001.stretch [1,752 kB]
Fetched 3,235 kB in 0s (14.8 MB/s)          
Selecting previously unselected package libltdl7:amd64.
(Reading database ... 40207 files and directories currently installed.)
Preparing to unpack .../libltdl7_2.4.6-2_amd64.deb ...
Unpacking libltdl7:amd64 (2.4.6-2) ...
Selecting previously unselected package libpython2.7:amd64.
Preparing to unpack .../libpython2.7_2.7.13-2+deb9u3_amd64.deb ...
Unpacking libpython2.7:amd64 (2.7.13-2+deb9u3) ...
Selecting previously unselected package libyajl2:amd64.
Preparing to unpack .../libyajl2_2.1.0-2+b3_amd64.deb ...
Unpacking libyajl2:amd64 (2.1.0-2+b3) ...
Selecting previously unselected package stackdriver-agent.
Preparing to unpack .../stackdriver-agent_5.5.2-1001.stretch_amd64.deb ...
Unpacking stackdriver-agent (5.5.2-1001.stretch) ...
Setting up libyajl2:amd64 (2.1.0-2+b3) ...
Processing triggers for libc-bin (2.24-11+deb9u4) ...
Processing triggers for systemd (232-25+deb9u12) ...
Setting up libltdl7:amd64 (2.4.6-2) ...
Setting up libpython2.7:amd64 (2.7.13-2+deb9u3) ...
Setting up stackdriver-agent (5.5.2-1001.stretch) ...
Processing triggers for libc-bin (2.24-11+deb9u4) ...
Processing triggers for systemd (232-25+deb9u12) ...

==============================================================================
Installation of stackdriver-agent-5.5.2-1001 completed successfully.

Please consult the documentation for troubleshooting advice:
  https://cloud.google.com/monitoring/agent

You can monitor the monitoring agent's logfile at:
  /var/log/syslog
==============================================================================

$ curl -sSO https://dl.google.com/cloudagents/install-logging-agent.sh
$ sudo bash install-logging-agent.sh --structured
==============================================================================
Starting installation of google-fluentd
==============================================================================

Installing agent for Debian or Ubuntu.
OK
Get:1 http://packages.cloud.google.com/apt google-cloud-logging-wheezy InRelease [3,795 B]
Get:2 http://packages.cloud.google.com/apt google-cloud-logging-wheezy/main amd64 Packages [690 B]
Fetched 4,485 B in 0s (7,115 B/s)
Reading package lists... Done
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following NEW packages will be installed:
  google-fluentd google-fluentd-catch-all-config-structured
0 upgraded, 2 newly installed, 0 to remove and 0 not upgraded.
Need to get 93.9 MB of archives.
After this operation, 345 MB of additional disk space will be used.
Get:1 http://packages.cloud.google.com/apt google-cloud-logging-wheezy/main amd64 google-fluentd amd64 1.6.25-1 [93.9 MB]
Get:2 http://packages.cloud.google.com/apt google-cloud-logging-wheezy/main amd64 google-fluentd-catch-all-config-structured all 1.0 [5,352 B]
Fetched 93.9 MB in 5s (18.3 MB/s)                                
Selecting previously unselected package google-fluentd.
(Reading database ... 40641 files and directories currently installed.)
Preparing to unpack .../google-fluentd_1.6.25-1_amd64.deb ...
Unpacking google-fluentd (1.6.25-1) ...
Selecting previously unselected package google-fluentd-catch-all-config-structured.
Preparing to unpack .../google-fluentd-catch-all-config-structured_1.0_all.deb ...
Unpacking google-fluentd-catch-all-config-structured (1.0) ...
Setting up google-fluentd (1.6.25-1) ...
Adding system user `google-fluentd' (UID 108) ...
Adding new group `google-fluentd' (GID 113) ...
Adding new user `google-fluentd' (UID 108) with group `google-fluentd' ...
Not creating home directory `/home/google-fluentd'.
Installing default conffile /etc/google-fluentd/google-fluentd.conf ...
Setting up google-fluentd-catch-all-config-structured (1.0) ...

==============================================================================
Installation of google-fluentd complete.

Logs from this machine should be visible in the log viewer at:
  https://console.cloud.google.com/logs/viewer?project=[YOUR_PROJECT_ID]&resource=gce_instance/instance_id/1501268411356974593

A test message has been sent to syslog to help verify proper operation.

Please consult the documentation for troubleshooting advice:
  https://cloud.google.com/logging/docs/agent

You can monitor the logging agent's logfile at:
  /var/log/google-fluentd/google-fluentd.log
==============================================================================

创建工作时间检查

打开控制台并进入 StackDriver 监控。由于工作项目的工作区不存在，于是创建了一个。稍等片刻。。。?

选择了创建稳定性检查，尝试选择 www1，但没有显示出来。稍等片刻后，显示出来了，于是成功创建。

创建警报策略

根据上述的趋势制定警报策略。

可以从“Alerting/Policies”中查看已创建的策略。

制作仪表板和图表

创建仪表板可以通过“仪表板 > 创建仪表板”完成。

可能显示的是只有代理进程的负载平均值。

显示日志

我觉得就像在StackDriver Logging中进行搜索一样。
我认为这也是因为安装了Logging代理，所以可以查看Syslog。

印象

インスタンスとサービスアカウントの設定周りがよくわかってない。でもオーナー権限持ってるから検証がめんどい。

其他

gcloud ssh コマンド関連をIAMで縛れないか調べてた。簡単な方法としては roles/compute.instanceAdmin.v1 or roles/compute.admin 役割を振れば良さそう (ドキュメント)

ちゃんとやるなら OSログイン設定が必要かもしれない。その場合は roles/compute.osAdminLogin or roles/
compute.osLogin 役割が必要になりそう。
さらにドキュメントを調べたら、roles/compute.instanceAdmin.v1 役割でSSH接続できるのは、VMインスタンスがサービスアカウントで実行されるように設定されていない場合のみらしい。サービスアカウントで実行する設定の場合は roles/iam.serviceAccountUser 役割も必要だとか。でも、コンソールで何も考えずにインスタンスを作るとデフォルトのサービスアカウントが設定されるから、これは結局 roles/iam.serviceAccountUser 役割を設定する必要があるってことでは。

以VM实例作为服务账号运行。

由于我对↑的“只有当VM实例未设置为由服务帐号执行时，才可以使用roles/compute.instanceAdmin.v1角色进行SSH连接”这句话有些担心，所以我试着在控制台上设置了一个适当的服务帐号给VM实例。（没有完全验证过）

我們試著使用SSH連接，來確認gcloud的設定。

$ gcloud config list
[core]
account = username-cross-pj-test@[YOUR_PROJECT_ID].iam.gserviceaccount.com
disable_usage_reporting = True
project = [YOUR_PROJECT_ID]
Your active configuration is: [default]

默认情况下可以使用gcloud吗……
似乎已设置了服务帐号。该帐号有BQ的数据编辑者角色，因此可以使用bq命令做很多事情。

$ bq ls
  datasetId
 -----------
  poc_test

甲：“数据集里也有……”可以看到数据集。但是我没有GCE（Google Compute Engine，谷歌计算引擎）相关的权限，所以很可能不能使用那个。

$ gcloud compute instances list
ERROR: (gcloud.compute.instances.list) Some requests did not succeed:
 - Required 'compute.zones.list' permission for 'projects/[YOUR_PROJECT_ID]'

好的，出现了错误。

将其恢复为默认的服务帐户。

$ gcloud config list
[core]
account = 901835264493-compute@developer.gserviceaccount.com
disable_usage_reporting = True
project = [YOUR_PROJECT_ID]

我回来了。我关心的是SSH连接方面，但由于我是这个项目的所有者，所以为了确认，我需要在本地创建服务账号和密钥，并进行项目设置…但这非常麻烦，所以我在这里放弃了。

总结

オーナー権限持ってるとユーザーがサービスアカウント使う設定の検証が面倒臭すぎる。