Create a High-Availability Kubernetes Cluster on AWS with Kops

https://www.poeticoding.com/create-a-high-availability-kubernetes-cluster-on-aws-with-kops/

AWS上轻松运行 Kubernetes 集群 部署 elixir 项目

使用 kops 部署 Kubernetes Operations (kops) - Production Grade K8s Installation, Upgrades, and Management

AWS 对kops有良好支持

部署此项目 https://github.com/poeticoding/phoenix_chat_example

High-Availability Cluster 高可用集群

结构图

要拥有HA集群,我们至少需要三个master服务器(管理整个Kubernetes集群的服务器)和两个worker服务器

AWS account and IAM role

配置账号

需要用到 aws-cli

$ aws configure

AWS Access Key ID: YOUR_ACCESS_KEY
AWS Secret Access Key: YOUR_SECRET_ACCESS_KEY
Default region name [None]:
Default output format [None]:

iam 用户需要 AdministratorAccess 权限

查看用户

$ aws iam list-users
{
    "Users": [
        {
            "Path": "/",
            "UserName": "alvise",
            "UserId": ...,
            "Arn": ...
        }
    ]
}

kops是在AWS上创建Kubernetes集群的工具 kubectl是我们用来管理集群启动运行

 Real domain in Route53

使用kops 需要配置域名

命令的使用等等

Kubernetes API and Security Group

k8s的api默认是公开的, 需要配置访问权限

Deploy an Nginx server

使用 kubectl get nodes 查看node列表

部署nginx需要 创建 kubernetes deployment

配置

# nginx_deploy.yaml
kind: Deployment
apiVersion: apps/v1
metadata:
  name: nginx
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx

  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1.15
        ports:
        - containerPort: 80

$ kubectl create -f nginx_deploy.yaml
deployment.apps "nginx" created

$ kubectl get pod
NAME                   READY     STATUS    RESTARTS   AGE
nginx-c9bd9bc4-jqvb5   1/1       Running   0          1m

pod 运行起来了
使用 load balancer 访问

# nginx_svc.yaml
kind: Service
apiVersion: v1

metadata:
  name: nginx-elb
  namespace: default
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: "nlb"

spec:
  type: LoadBalancer
  selector:
    app: nginx
  ports:
    - name: http
      port: 80
      targetPort: 80

LoadBalancer 服务 转发流量到nginx

Deploy the Phoenix Chat

配置

kind: Deployment
apiVersion: apps/v1
metadata:
  name: chat
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: chat

  template:
    metadata:
      labels:
        app: chat
    spec:
      containers:
      - name: phoenix-chat
        image: alvises/phoenix-chat-example:1_kops_chat
        ports:
        - containerPort: 4000
        env:
        - name: PORT
          value: "4000"
        - name: PHOENIX_CHAT_HOST
          value: "chat.poeticoding.com"

docker 镜像 https://cloud.docker.com/repository/docker/alvises/phoenix-chat-example

LoadBalancer类似

kind: Service
apiVersion: v1

metadata:
  name: chat-elb
  namespace: default
  annotations:
    service.beta.kubernetes.io/aws-load-balancer-type: "nlb"

spec:
  type: LoadBalancer
  selector:
    app: chat
  ports:
    - name: http
      port: 80
      targetPort: 4000

Multiple Chat replicas

只运行在一个节点 replicas 为 1

多节点需要分布式消息处理

2019/2/4 posted in  阅读笔记

ActiveRecord’s Queries tricks 小记

原文 https://medium.com/rubyinside/active-records-queries-tricks-2546181a98dd

关联表join时使用条件

# User model
scope :activated, ->{
  joins(:profile).where(profiles: { activated: true })
}

更好的做法

# Profile model
scope :activated, ->{ where(activated: true) }
# User model
scope :activated, ->{ joins(:profile).merge(Profile.activated) }

关于 merge
https://apidock.com/rails/ActiveRecord/SpawnMethods/merge
https://api.rubyonrails.org/classes/ActiveRecord/SpawnMethods.html

嵌套join的差异

  • User has_one Profile
  • Profile has_many Skills
User.joins(:profiles).merge(Profile.joins(:skills))
=> SELECT users.* FROM users 
   INNER JOIN profiles    ON profiles.user_id  = users.id
   LEFT OUTER JOIN skills ON skills.profile_id = profiles.id
# So you'd rather use:
User.joins(profiles: :skills)
=> SELECT users.* FROM users 
   INNER JOIN profiles ON profiles.user_id  = users.id
   INNER JOIN skills   ON skills.profile_id = profiles.id
   

内链接和外连接

Exist query

存在和不存在

# Post
scope :famous, ->{ where("view_count > ?", 1_000) }
# User
scope :without_famous_post, ->{
  where(_not_exists(Post.where("posts.user_id = users.id").famous))
}
def self._not_exists(scope)
  "NOT #{_exists(scope)}"
end
def self._exists(scope)
  "EXISTS(#{scope.to_sql})"
end

Subqueries 子查询

比如查询部分用户(user)的帖子(post)

不好的做法

Post.where(user_id: User.created_last_month.pluck(:id))

这里的缺陷是将运行两个SQL查询:一个用于获取用户的ID,另一个用于从这些user_id获取帖子

这样写一个查询就可以了

Post.where(user_id: User.created_last_month)

基础

.to_sql 生成 SQL 语句字符串
.explain 获取查询分析

Booleans

对于User.where.not(tall: true)在pg下会生成
SELECT users.* FROM users WHERE users.tall <> 't'
这返回 tall 是 false 的 记录,不包括是null 的

包括null应该这么写

User.where("users.tall IS NOT TRUE")

or

User.where(tall: [false, nil])
2018/12/22 posted in  阅读笔记

3-2 索引优化SQL的方法 笔记

https://www.imooc.com/video/3998

工具

pt-duplicate-key-checke Percona Toolkit 的一部分

sudo apt-get install percona-toolkit

pt-duplicate-key-checke 使用方法简单

$ pt-duplicate-key-checker -h  host -u root -p  password ########################################################################
# shiji_development.daily_shop_statistics
# ########################################################################

# index_daily_shop_statistics_on_date_id is a left-prefix of index_daily_shop_statistics_on_date_id_and_shop_id
# Key definitions:
#   KEY `index_daily_shop_statistics_on_date_id` (`date_id`),
#   UNIQUE KEY `index_daily_shop_statistics_on_date_id_and_shop_id` (`date_id`,`shop_id`),
# Column types:
#     `date_id` int(11) not null
#     `shop_id` int(11) not null
# To remove this duplicate index, execute:
ALTER TABLE `shiji_development`.`daily_shop_statistics` DROP INDEX `index_daily_shop_statistics_on_date_id`;

# ########################################################################
# shiji_migrate_test.daily_shop_statistics
# ########################################################################

# index_daily_shop_statistics_on_date_id is a left-prefix of index_daily_shop_statistics_on_date_id_and_shop_id
# Key definitions:
#   KEY `index_daily_shop_statistics_on_date_id` (`date_id`),
#   UNIQUE KEY `index_daily_shop_statistics_on_date_id_and_shop_id` (`date_id`,`shop_id`),
# Column types:
#     `date_id` int(11) not null
#     `shop_id` int(11) not null
# To remove this duplicate index, execute:
ALTER TABLE `shiji_migrate_test`.`daily_shop_statistics` DROP INDEX `index_daily_shop_statistics_on_date_id`;

# ########################################################################
# Summary of indexes
# ########################################################################

# Size Duplicate Indexes   65360
# Total Duplicate Indexes  2
# Total Indexes            478

查询表的索引状态 SHOW INDEX FROM orders;

Cardinality 表示离散度

links

2018/9/26 posted in  MySQL

rsyslog debug mode

debug 模式下 启动rsyslog, rsyslogd的日志打印到标准输出
2017/7/6 posted in  rsyslog

rsyslog 输出模块 omprog

把数据通过stdin传给外部程序处理
2017/7/6 posted in  rsyslog

rsyslog 输入模块imfile

rsyslog 读取文件输入
2017/7/5 posted in  rsyslog

UID & EUID

实际用户和有效用户
2017/7/14 posted in  Bash

crontab 可视化编辑

推荐两个crontab 可视化编辑工具
2017/7/10 posted in  Bash

Grep 显示查找内容的上下文

  • grep -C 5 foo file 显示file文件中匹配foo字串那行以及上下5行
  • grep -B 5 foo file 显示foo及前5行
  • grep -A 5 foo file 显示foo及后5行
2017/7/5 posted in  Bash

Linux lsof 命令例子

list open files
2017/6/29 posted in  Bash