使用Amazon Elasticsearch Service将在S3上获取的手动快照还原到EC2上的Elasticsearch

简而言之

以下是在将 Amazon Elasticsearch Service 上的手动快照从 S3 恢复到为了某个验证目的而在 EC2 上创建的 Elasticsearch 时的记录。

以下是参考的手动快照操作方法，适用于亚马逊Elasticsearch服务。
链接：https://dev.classmethod.jp/cloud/restore-from-amazon-es-manual-snapshot-ja/

关于环境配置，我们使用了以下设置：
ap-northeast-1（东京区域）
Amazon Elasticsearch Service 5.5
Amazon Linux 2（ami-0d7ed3ddb85b521a6）
Elasticsearch 5.5.2

在本文中，我们将存储快照到名为 es-example-snapshot 的S3存储桶。（如果您要进行实际验证，请使用其他名称代替）

在亚马逊 Elasticsearch 服务中进行手动快照拍摄。

基本上，按照上述参考步骤进行操作，但需要补充一点，即为了S3访问，我将IAM策略设置为了Elasticsearch文档中建议的内容。
https://www.elastic.co/guide/en/elasticsearch/plugins/current/repository-s3-repository.html#repository-s3-permissions
https://docs.aws.amazon.com/ja_jp/AmazonS3/latest/dev/mpuAndPermissions.html

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": [
        "s3:ListBucket",
        "s3:GetBucketLocation",
        "s3:ListBucketMultipartUploads",
        "s3:ListBucketVersions"
      ],
      "Effect": "Allow",
      "Resource": [
        "arn:aws:s3:::es-example-snapshot"
      ]
    },
    {
      "Action": [
        "s3:GetObject",
        "s3:PutObject",
        "s3:DeleteObject",
        "s3:AbortMultipartUpload",
        "s3:ListMultipartUploadParts"
      ],
      "Effect": "Allow",
      "Resource": [
        "arn:aws:s3:::es-example-snapshot/*"
      ]
    }
  ]
}

假设手动截图可以按照以下方式进行。（请参考上述手动截图操作方法链接）

$ curl -sS "https://example.ap-northeast-1.es.amazonaws.com/_snapshot/es-example-snapshot/_all?pretty"
{
  "snapshots" : [ {
    "snapshot" : "my-snapshot-20190206-1",
    "uuid" : "kQAztH78pqRyz9J363Ayz5",
    "version_id" : 6060211,
    "version" : "5.5.2",
    "indices" : [ ".kibana", "sample_20190205" ],
    "state" : "SUCCESS",
    "start_time" : "2019-02-06T05:41:12.401Z",
    "start_time_in_millis" : 1549431672401,
    "end_time" : "2019-02-06T06:04:12.065Z",
    "end_time_in_millis" : 1549433052065,
    "duration_in_millis" : 1379664,
    "failures" : [ ],
    "shards" : {
      "total" : 8,
      "failed" : 0,
      "successful" : 8
    }
  } ]
}

准备EC2的Elasticsearch

我們通常從官方網站下載並安裝相應版本的rpm套件來安裝和啟動Elasticsearch。
https://www.elastic.co/downloads/past-releases

并将该IAM策略分配给EC2上的角色。

$ curl -sS "http://localhost:9200/"
{
  "name" : "4sICf-8",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "WxG9lFAcRpOYrSeVkMhi0Q",
  "version" : {
    "number" : "5.5.2",
    "build_hash" : "b2f0c09",
    "build_date" : "2017-08-14T12:33:14.154Z",
    "build_snapshot" : false,
    "lucene_version" : "6.6.0"
  },
  "tagline" : "You Know, for Search"
}

准备S3存储库插件

根据官方的以下文档，安装用于访问S3快照的插件。
https://www.elastic.co/guide/en/elasticsearch/plugins/current/repository-s3.html

如果在快照的源头中有需要使用的插件，我将一并安装。
https://www.elastic.co/guide/en/elasticsearch/plugins/current/analysis-kuromoji.html

$ sudo /usr/share/elasticsearch/bin/elasticsearch-plugin install repository-s3
$ sudo /usr/share/elasticsearch/bin/elasticsearch-plugin install analysis-kuromoji
# プラグイン読み込ませ
$ sudo systemctl restart elasticsearch.service

与 S3 存储桶进行快照集成注册。

$ curl -sS -H 'Content-Type: application/json' -X PUT "http://localhost:9200/_snapshot/es-example-snapshot"  -d '
{
  "type": "s3",
  "settings": {
    "bucket": "es-example-snapshot"
  }
}'

当检查目标快照时，可以看到已获取的快照在Amazon Elasticsearch服务中可见。

$ curl -sS "http://localhost:9200/_snapshot/es-example-snapshot/_all?pretty"
{
  "snapshots" : [
    {
      "snapshot" : "my-snapshot-20190206-1",
      "uuid" : "kQAztH78pqRyz9J363Ayz5",
      "version_id" : 6060211,
      "version" : "5.5.2",
      "indices" : [
        ".kibana",
        "sample_20190205"
      ],
      "state" : "SUCCESS",
      "start_time" : "2019-02-06T05:41:12.401Z",
      "start_time_in_millis" : 1549431672401,
      "end_time" : "2019-02-06T06:04:12.065Z",
      "end_time_in_millis" : 1549433052065,
      "duration_in_millis" : 1379664,
      "failures" : [ ],
      "shards" : {
        "total" : 8,
        "failed" : 0,
        "successful" : 8
      }
    }
  ]
}

执行恢复

由于EC2与S3之间相关联的IAM角色，您可以直接使用curl发送请求。

$ curl -sS -H 'Content-Type: application/json' -X POST "http://localhost:9200/_snapshot/es-example-snapshot/my-snapshot-20190206-1/_restore" -d '
{
  "indices": "sample_20190205"
}'

如果您想查看进展情况，可以通过以下途径查看碎片信息以了解恢复状态。

$ curl -sS "http://localhost:9200/_cat/shards?v"

取消与S3桶的注册的方法是什么。

我在尝试寻找官方信息时，发现了如下链接，其中解释了如何取消与S3存储桶的协作。
https://www.elastic.co/guide/en/elasticsearch/reference/current/modules-snapshots.html#_snapshot

A repository can be unregistered using the following command:

DELETE /_snapshot/my_backup

When a repository is unregistered, Elasticsearch only removes the reference to
the location where the repository is storing the snapshots.
The snapshots themselves are left untouched and in place.

我认为应该是这样的命令。

$ curl -sS -X DELETE "http://localhost:9200/_snapshot/es-example-snapshot"