2019年AWS re:Invent会议笔记

2 年 ago

新, 韵

8 minutes

参加了2019年的AWS re:Invent活动的总结。

印象

因为在设计中，我注意到了一个关键词”blast radius”，因此个人觉得有必要记住它。

在 AWS 中，有 “train your builders” 这样的关键点，例如个学习型会议 re:invent 本身就是一个例子，还有 builders’ library 提供的最佳实践，以及针对机器学习的开发者学习服务提供（如 DeepRacer）。可以在各个方面感受到 AWS 提供这种机会。

从动向上看，作为新的服务，似乎主要是为了消除与ai/ml和on-prem的障碍而添加了高性能处理和低延迟网络。

关于活动的备忘录

感觉就像是在拉斯维加斯的酒店和赌场群中举行一场大规模的活动，由于我住的酒店（名为Bally’s）的交通不是很方便，所以需要步行大约20分钟到达隔壁的会场，导致我感到很累（如果可能的话，住在活动会场的酒店之一会更轻松，因为可以乘坐穿梭巴士）。

在早晚餐時，食物選擇主要有自助餐和外帶，餐廳空間通常都很寬敞，所以我只在第一天的午餐時選擇了外帶，其他時間都是在自助餐廳用餐。菜肴的種類根據不同的日子而有所變化，有亞洲料理、墨西哥料理等等，但對我的飲食沒有產生特別的影響。

会话

视频列表

https://www.youtube.com/channel/UCdoadna9HFHsxXWhafhNvKw/playlists

重点是引起兴趣的幻灯片

高性能な instances への対応 1

fargate for eks、eks インフラの managed
networking、言及なし

cassandra (mcs) の追加、 wide column database

ai/ml シリーズ

outposts (on-prem workload)

local zones (low latency to local end-users)

wavelength (5G)

一方面吸引了我的注意力的服務

Amazon CodeGuru

Java のみっぽいので色々な言語の対応が待ち遠しい

Amazon Kendra

企業内ドキュメントの検索は皆困っていて需要があるんだろう

Amazon Builders’ Library

読む必要がある

以下是我参加的各种会议的个人备忘录。

午夜疯狂

在晚上8点排队，但等待两个小时到开始时间和等待队伍约一小时都很辛苦。到达拉斯维加斯之前已经耗尽了体力，无法坚持到最后参加比赛。

ARC411-R – [REPEAT] 通过基于细胞的架构减小爆炸半径

regional control plane -> zone (zone control plane, data plane)

well architectured framework
tenets

minimize blast radius (spike, etc.)
serverless (lambda, kinesis – no zone based)
infra as code

analytics cell router -> analytics cells -> KINESIS -> s3 for datastore -> (engineer, asap, ml, etc.)
s3 -> asap -> detections cell router -> detection cells -> findings repos

router
s3 event log -> sns -> (analytics cell router)[sqs -> (lambda -> dynamodb rules)] -> analytics cells
rules

load balancing, data segregation (source recognized source a to cell a)
data replication (prod cells copy to test cells), blue/green deployment
never look s3? spof (よくわからんかった）

cell

apache flink checkpoint and save

key takeways

small vs. large
design for failure
thinnest possible layer

CON421-R -【重复】解读亚马逊弹性Kubernetes服务的核心技术

eks managed vpc | customer managed vpc

zone 間はどうふりわけるのか？ -> 53 っぽい

eks celluar arch (1 cell = 1 aws account = frontend, cluster events, control plane management, etc.)
pipeline -> prow cluster

enhancement

managed node groups (single command, up to date, ha)

simplify worker node management, self managed, k8s ecosystem tooling (autoscaler)

vision (globally available, easy to use, production ready, cost-effective, high performance)

snap service mesh on eks

eks, envoy, switchboard, spinnaker

ARC335-R – 【重复】设计容错：在AWS上构建弹性系统

fault isolation zones: cell-based, multi-az
microservice
distributed best practices: throttling, retry, circuit breaker

strategy: backup & restore (multi-region) high level rto, rpo
strategy: pilot light
strategy: warm standby
strategy: active-active

(one ex. read locally, write globally)

s3 – cross-region replication

replication time control (new! two weeks ago)

ebs – snapshot copy
dynamodb – multi-master, multi-region (no complexity write globally)
rds:- cross-region read replicas
inter-region vpc peering -> white paper
snapchat

99.99% (tier0), 99.95% (tier1), 99% (tier2), 95% (tier3)
legacy

monolithic, single region -> multi-region active-active
repl: dynamodb streams -> stream service -> other regions, etc.

continuous resilience

disaster recovery -> chaos engineering -> continuous resilience

KYN201 – 星期一现场直播

没有新的信息吗？
有人离开座位引人注目。

KYN202 – Andy Jassy的主题演讲

超过65000名与会者
超过3000场会议

! 转变

不要让犹豫阻止你的开始

在本地：97％

serverless

data silos -> data lake

s3 access point https://aws.amazon.com/jp/blogs/aws/easily-manage-shared-data-sets-with-amazon-s3-access-points/

redshift ra3 instances with managed storage https://aws.amazon.com/jp/blogs/aws/amazon-redshift-update-next-generation-compute-instances-and-managed-analytics-optimized-storage/

elasticsearch service (ultrawarm) https://aws.amazon.com/jp/blogs/aws/announcing-ultrawarm-preview-for-amazon-elasticsearch-service/

database

managed cassandra service (wide column) https://aws.amazon.com/jp/blogs/aws/new-amazon-managed-apache-cassandra-service-mcs/

usecase: health care
sage maker studio, notebooks, experiments, debugger, model monitor, autopilot
fraud detector
codeguru (一番盛り上がった）

contact lens for amazon connect
kendra
break on-prem barriers
outposts GA

native aws or vmware cloud on aws

local zones
5G

8 capabilities
wavelength

SVS310-R1 – [重复1次] 保护企业级无服务器应用程序

identity

diy identity store – vulunerable
hashed – attask, salt hashed – …
(ok) ssecure remote password protocol (SRP)
Amazon Cognito (SRP, etc.)

x multiple identities o centralize identity management and privilege management

delegation: OIDC + OAUTH, federation: SAML, no long-term credentials, rbac
delegation

aws security token service -> temporary iam creds

jwt: identity token, access token, refresh token
least privilege, iam condition

access control to api gateway

with cognito
allowed -> jwt + context.identity
with lambda authorizer -> policy
others

basic request vailidations on api gateway: parameters, payload with JSON schema
CORS

access control to dynamodb (iam, cognito condition), s3 (iam, bucket policies)

s3 access point (multi access control)

lambda

shared responsibility model
iam invoke, actions, assume

common vulnerabilities: ddos

aws shield

common: OWASP top risks to web apps – XSS

aws waf filtering rules

common: SQL injection

use prepared

apply security at all layers
secure coding practices

accessing db creds: x hard coded, ! env or config file, o aws secrets manager, o iam authentication for amazon rds
data api for amazon aurora serverless https://docs.aws.amazon.com/ja_jp/AmazonRDS/latest/AuroraUserGuide/data-api.html

monolitic functions -> microservies + event-driven architectures (small functions)
owasp secure coding practices

STP203 – 当代创业公司：探索如今最成功的架构。

remove human interaction, self service tools, abstraction layers allow for modular evolution

enable your developers

preparing hackathon

everything continuous, all the time

how long x takes?

keep your systems simple

gall’s law

democratize your data
develop a culture of openness

legacy: ecs containers

goal: 100x clients, faster, more data

shift

docker -> serverless, tdd, devops, devs, and QA are all one

CON323-R1 – 应用程序网格AWS背后的技术解析

customer want: routing, observability, security, transparency
arch

proxy

virtual node, virtual service, virtual router

design

frontend service, envoy management service, transformer
frontend service

authentication
authorization
validation
persistence

picking

customer configuration/metadata, relationships between resources, serializable cross-key/table -> JournalDB

transformer

mesh id -> buckets (split brain -> allow extra)
event-driven processing

reactive, efficient, creates backlog, requires reconciliation
level-driven processing: bounded workload (bimodal behavior), self-healing, less efficient

envoy

envoy data store

synthesized envoy configuration, relationships node to manifest, eventually consistency, single key -> dynamodb

discovery – listeners – routes – secrets – clusters – endpoints
management

actor system(connection manager, manifest manager, aws cloud map, aws certificate manager)

operations (canary)

deployment: first (container images and cloudformation templates), second, remaining
monitoring: rate, errors, duration (+ cloudwatch log insight)

AIM207-R5 – [重复5次] 开始使用AWS DeepRacer

CON325-R – [全新推出！] [重複] 借助亞馬遜 ECS 能力提供者實現以應用為中心的思維。

terminology: ecs cluster (namespace), ecs task (like pods), service (many tasks)
start ec2 instances first, and then run ecs tasks (infra first)
task placement: available instances -> resources (like resource limit) -> placement constraints (like affinity) -> strategy
infra first: available instances not found, spread placement strategies, scaling (cloudwatch metrics -> alarm auto scaling groups: metrics are based only on existing tasks and resources)

application first

tenets: applications own their requirements, infrastructure responds to application requirements

ECS Capacity Providers

ecs cluster ( ecs cp ( ec2 asg (ec2 instances)))
ecs cp

abstracts capacity
0-10 /cluster
used by tasks/services

run tasks under cp without instances -> instances start -> tasks placed
ec2 spot / fargate spot

price up to 90%/80%
reclaimed by ec2/ reclaimed
instances pools/automatic

ARC349-R1 – [REPEAT 1] 超越五个9：从我们最高可用的数据平面中汲取教训

insist on the highest standards
raise the bar for testing

1000s unit tests, 100s integration tests, pre-prod env, roll-forward/back testing

be technically fearless

mitigate fear with professionalism, testing, and open-@minded scrutiny
aws lindbergh award

modeling
focus on blast radius

shuffle sharding https://aws.amazon.com/jp/blogs/architecture/shuffle-sharding-massive-and-magical-fault-isolation/

modular separation
double down on simplicity

saying no to many features,

unreasonable redundancy

striping, shuffle sharding
teeny cache

static stability
degrade gracefully

KYN204 – 由Werner Vogels博士主讲的主题演讲

DOP210-L – 领导力研讨会：AWS上的开发者工具

delivery: dev/test -> review -> pre-production -> production
dev/test

use cloud desktops at amazon
use cloud9 internally
support aws toolkit for third party IDE

debugging timeline 30min -> 10 min
demo

review

codecommit supports approval rules
codeguru review/profiler

ci/cd

2001: monolith -> 2002: 2-pizza teams (devops, full ownership, full accountability, focused innovation)

delivery pipelines (dev tool team)

delivery: source -> build -> alpha (pre-production, automated tests) -> beta (pre-production, automated integration tests, load/perf tests, browser tests) -> gamma (pre-production, automated integration tests, synthetics tests, api smoke tests) -> one az/fractional (production, synthetic monitoring) … -> one region…
delivery is pessimistic
pipeline blockers: time windows, pipeline policies, ocverage, review, security scans, dependency updates, etc.

modern applications

monolithic -> n-tier -> micoservices & serverless
aws cdk

a look ahead

security

SEC209-R1 – [重复1] 开始使用AWS身份确认

iam role -> resource

short-term creds for iam role
if any policy denies -> access denied
if some policy allows -> allow
otherwise denied

cross account: recommendation – keep it simple

resource based policy
role trust policy

内部部署目前仍占据了97%，所以可以说是在取得内部部署的进展？