zt@dev▊
DevOps · #kubernetes#helm#chart

Helm Chart开发与最佳实践

2024.05.29 8 min 3.0k 字
// 目录 · contents

前言

Helm是Kubernetes的包管理工具，被称为”Kubernetes的apt/yum”。它通过Chart将复杂的应用打包为可复用、可配置的部署单元。本文将深入讲解Helm Chart的开发全流程。

Helm架构

graph TB
    subgraph Client["Helm Client"]
        CLI["helm CLI"]
        SDK["Helm SDK"]
    end

    subgraph ChartSources["Chart来源"]
        Local["本地Chart"]
        Repo["Chart Repository"]
        OCI["OCI Registry"]
    end

    subgraph K8s["Kubernetes Cluster"]
        API["API Server"]
        Secret["Release Secrets"]
        Resources["K8s Resources"]
    end

    CLI --> Local
    CLI --> Repo
    CLI --> OCI
    CLI --> API
    API --> Secret
    API --> Resources

Helm 3的重要变化（相比Helm 2）： - 移除了Tiller服务端组件 - Release信息存储在Kubernetes Secrets中 - 使用三方合并补丁（3-way merge） - 支持OCI Registry存储Chart

Chart结构

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
# 创建新Chart
helm create myapp

# Chart目录结构
myapp/
├── Chart.yaml          # Chart元数据
├── Chart.lock          # 依赖锁定文件
├── values.yaml         # 默认配置值
├── values.schema.json  # values校验Schema
├── .helmignore         # 打包时忽略的文件
├── charts/             # 依赖Chart
├── crds/               # CRD定义
└── templates/          # 模板文件
    ├── NOTES.txt       # 安装后的提示信息
    ├── _helpers.tpl    # 模板辅助函数
    ├── deployment.yaml
    ├── service.yaml
    ├── ingress.yaml
    ├── hpa.yaml
    ├── serviceaccount.yaml
    └── tests/
        └── test-connection.yaml
graph LR
    subgraph Chart["Helm Chart"]
        Meta["Chart.yaml<br>元数据"]
        Values["values.yaml<br>配置参数"]
        Templates["templates/<br>模板文件"]
        Helpers["_helpers.tpl<br>辅助函数"]
    end

    Values --> |"传入参数"| Templates
    Helpers --> |"模板函数"| Templates
    Templates --> |"渲染"| Manifests["K8s Manifests"]
    Manifests --> |"apply"| Cluster["Kubernetes"]

Chart.yaml

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
apiVersion: v2
name: myapp
description: A production-ready web application chart
type: application
version: 1.2.0          # Chart版本
appVersion: "3.5.1"     # 应用版本
kubeVersion: ">=1.25.0"
home: https://github.com/example/myapp
sources:
  - https://github.com/example/myapp
maintainers:
  - name: zt
    email: [email protected]
keywords:
  - web
  - api
annotations:
  artifacthub.io/changes: |
    - kind: added
      description: Support for HPA
    - kind: fixed
      description: Fix service port configuration
dependencies:
  - name: postgresql
    version: "12.x.x"
    repository: https://charts.bitnami.com/bitnami
    condition: postgresql.enabled
  - name: redis
    version: "17.x.x"
    repository: https://charts.bitnami.com/bitnami
    condition: redis.enabled

模板语法

基础语法

Helm模板使用Go template语法，结合Sprig函数库。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
# templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ include "myapp.fullname" . }}
  labels:
    {{- include "myapp.labels" . | nindent 4 }}
  annotations:
    {{- with .Values.deployment.annotations }}
    {{- toYaml . | nindent 4 }}
    {{- end }}
spec:
  {{- if not .Values.autoscaling.enabled }}
  replicas: {{ .Values.replicaCount }}
  {{- end }}
  selector:
    matchLabels:
      {{- include "myapp.selectorLabels" . | nindent 6 }}
  template:
    metadata:
      annotations:
        # 配置变更时自动触发滚动更新
        checksum/config: {{ include (print $.Template.BasePath "/configmap.yaml") . | sha256sum }}
      labels:
        {{- include "myapp.selectorLabels" . | nindent 8 }}
    spec:
      {{- with .Values.imagePullSecrets }}
      imagePullSecrets:
        {{- toYaml . | nindent 8 }}
      {{- end }}
      serviceAccountName: {{ include "myapp.serviceAccountName" . }}
      securityContext:
        {{- toYaml .Values.podSecurityContext | nindent 8 }}
      containers:
        - name: {{ .Chart.Name }}
          image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
          imagePullPolicy: {{ .Values.image.pullPolicy }}
          ports:
            - name: http
              containerPort: {{ .Values.service.targetPort | default 8080 }}
              protocol: TCP
          {{- if .Values.healthCheck.enabled }}
          livenessProbe:
            httpGet:
              path: {{ .Values.healthCheck.livenessPath | default "/healthz" }}
              port: http
            initialDelaySeconds: {{ .Values.healthCheck.initialDelaySeconds | default 15 }}
            periodSeconds: 10
          readinessProbe:
            httpGet:
              path: {{ .Values.healthCheck.readinessPath | default "/ready" }}
              port: http
            initialDelaySeconds: 5
            periodSeconds: 5
          {{- end }}
          env:
            {{- range $key, $value := .Values.env }}
            - name: {{ $key }}
              value: {{ $value | quote }}
            {{- end }}
            {{- range $key, $secret := .Values.envFromSecret }}
            - name: {{ $key }}
              valueFrom:
                secretKeyRef:
                  name: {{ $secret.name }}
                  key: {{ $secret.key }}
            {{- end }}
          resources:
            {{- toYaml .Values.resources | nindent 12 }}
          volumeMounts:
            {{- if .Values.config }}
            - name: config
              mountPath: /etc/app/config.yaml
              subPath: config.yaml
              readOnly: true
            {{- end }}
      volumes:
        {{- if .Values.config }}
        - name: config
          configMap:
            name: {{ include "myapp.fullname" . }}-config
        {{- end }}
      {{- with .Values.nodeSelector }}
      nodeSelector:
        {{- toYaml . | nindent 8 }}
      {{- end }}
      {{- with .Values.tolerations }}
      tolerations:
        {{- toYaml . | nindent 8 }}
      {{- end }}

helpers模板

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
# templates/_helpers.tpl

{{/*
生成应用全名
*/}}
{{- define "myapp.fullname" -}}
{{- if .Values.fullnameOverride }}
{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- $name := default .Chart.Name .Values.nameOverride }}
{{- if contains $name .Release.Name }}
{{- .Release.Name | trunc 63 | trimSuffix "-" }}
{{- else }}
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" }}
{{- end }}
{{- end }}
{{- end }}

{{/*
通用标签
*/}}
{{- define "myapp.labels" -}}
helm.sh/chart: {{ include "myapp.chart" . }}
{{ include "myapp.selectorLabels" . }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end }}

{{/*
选择器标签
*/}}
{{- define "myapp.selectorLabels" -}}
app.kubernetes.io/name: {{ include "myapp.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}

{{/*
Chart名称和版本
*/}}
{{- define "myapp.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
{{- end }}

{{/*
名称
*/}}
{{- define "myapp.name" -}}
{{- default .Chart.Name .Values.nameOverride | trunc 63 | trimSuffix "-" }}
{{- end }}

{{/*
ServiceAccount名称
*/}}
{{- define "myapp.serviceAccountName" -}}
{{- if .Values.serviceAccount.create }}
{{- default (include "myapp.fullname" .) .Values.serviceAccount.name }}
{{- else }}
{{- default "default" .Values.serviceAccount.name }}
{{- end }}
{{- end }}

values.yaml设计

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
# values.yaml - 默认配置
replicaCount: 2

image:
  repository: myregistry.io/myapp
  pullPolicy: IfNotPresent
  tag: ""  # 默认使用Chart.AppVersion

imagePullSecrets: []
nameOverride: ""
fullnameOverride: ""

serviceAccount:
  create: true
  annotations: {}
  name: ""

podSecurityContext:
  fsGroup: 1000
  runAsNonRoot: true

securityContext:
  allowPrivilegeEscalation: false
  readOnlyRootFilesystem: true
  runAsUser: 1000
  capabilities:
    drop:
      - ALL

service:
  type: ClusterIP
  port: 80
  targetPort: 8080

ingress:
  enabled: false
  className: nginx
  annotations: {}
  hosts:
    - host: myapp.example.com
      paths:
        - path: /
          pathType: Prefix
  tls: []

resources:
  requests:
    cpu: 100m
    memory: 128Mi
  limits:
    cpu: 500m
    memory: 512Mi

autoscaling:
  enabled: false
  minReplicas: 2
  maxReplicas: 10
  targetCPUUtilizationPercentage: 70
  targetMemoryUtilizationPercentage: 80

healthCheck:
  enabled: true
  livenessPath: /healthz
  readinessPath: /ready
  initialDelaySeconds: 15

env:
  APP_ENV: production
  LOG_LEVEL: info

envFromSecret: {}
  # DB_PASSWORD:
  #   name: db-secret
  #   key: password

config: {}
  # 应用配置内容，会创建ConfigMap

# 子Chart配置
postgresql:
  enabled: true
  auth:
    database: myapp
    username: myapp

redis:
  enabled: false
  architecture: standalone

deployment:
  annotations: {}

nodeSelector: {}
tolerations: []
affinity: {}

values.schema.json

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
{
  "$schema": "https://json-schema.org/draft-07/schema#",
  "type": "object",
  "required": ["image", "service"],
  "properties": {
    "replicaCount": {
      "type": "integer",
      "minimum": 1
    },
    "image": {
      "type": "object",
      "required": ["repository"],
      "properties": {
        "repository": {
          "type": "string"
        },
        "pullPolicy": {
          "type": "string",
          "enum": ["Always", "IfNotPresent", "Never"]
        },
        "tag": {
          "type": "string"
        }
      }
    },
    "service": {
      "type": "object",
      "properties": {
        "type": {
          "type": "string",
          "enum": ["ClusterIP", "NodePort", "LoadBalancer"]
        },
        "port": {
          "type": "integer",
          "minimum": 1,
          "maximum": 65535
        }
      }
    }
  }
}

Hooks

Helm Hooks允许在Release生命周期的特定时间点执行操作：

graph LR
    A["helm install"] --> B["pre-install"]
    B --> C["安装资源"]
    C --> D["post-install"]

    E["helm upgrade"] --> F["pre-upgrade"]
    F --> G["升级资源"]
    G --> H["post-upgrade"]

    I["helm delete"] --> J["pre-delete"]
    J --> K["删除资源"]
    K --> L["post-delete"]
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
# templates/job-db-migrate.yaml
apiVersion: batch/v1
kind: Job
metadata:
  name: {{ include "myapp.fullname" . }}-db-migrate
  labels:
    {{- include "myapp.labels" . | nindent 4 }}
  annotations:
    "helm.sh/hook": pre-upgrade,pre-install
    "helm.sh/hook-weight": "5"          # 执行顺序（升序）
    "helm.sh/hook-delete-policy": before-hook-creation  # 清理策略
spec:
  backoffLimit: 3
  template:
    spec:
      restartPolicy: Never
      containers:
        - name: migrate
          image: "{{ .Values.image.repository }}:{{ .Values.image.tag | default .Chart.AppVersion }}"
          command: ["./migrate", "up"]
          env:
            - name: DATABASE_URL
              valueFrom:
                secretKeyRef:
                  name: {{ include "myapp.fullname" . }}-db
                  key: url

---
# templates/tests/test-connection.yaml
apiVersion: v1
kind: Pod
metadata:
  name: "{{ include "myapp.fullname" . }}-test"
  annotations:
    "helm.sh/hook": test
    "helm.sh/hook-delete-policy": before-hook-creation
spec:
  restartPolicy: Never
  containers:
    - name: test
      image: busybox:1.36
      command: ['wget', '--timeout=5', '-qO-', 'http://{{ include "myapp.fullname" . }}:{{ .Values.service.port }}/healthz']

依赖管理

1
2
3
4
5
6
7
8
9
10
11
# 添加仓库
helm repo add bitnami https://charts.bitnami.com/bitnami

# 更新依赖
helm dependency update ./myapp

# 构建依赖（下载到charts/目录）
helm dependency build ./myapp

# 列出依赖
helm dependency list ./myapp

条件依赖与标签：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# Chart.yaml中的依赖定义
dependencies:
  - name: postgresql
    version: "12.x.x"
    repository: https://charts.bitnami.com/bitnami
    condition: postgresql.enabled
    tags:
      - database
  - name: redis
    version: "17.x.x"
    repository: https://charts.bitnami.com/bitnami
    condition: redis.enabled
    tags:
      - cache
1
2
# 启用特定标签的依赖
helm install myapp ./myapp --set tags.database=true --set tags.cache=false

Chart测试与调试

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
# 模板渲染（不安装）
helm template myrelease ./myapp -f custom-values.yaml

# 渲染并检查语法
helm template myrelease ./myapp | kubectl apply --dry-run=client -f -

# Lint检查
helm lint ./myapp
helm lint ./myapp -f production-values.yaml

# 调试模式安装
helm install myrelease ./myapp --debug --dry-run

# 运行测试
helm test myrelease

# 查看渲染后的某个模板
helm template myrelease ./myapp -s templates/deployment.yaml

# 对比升级变更
helm diff upgrade myrelease ./myapp -f new-values.yaml

发布与分发

Chart Repository

1
2
3
4
5
6
7
8
9
# 打包Chart
helm package ./myapp
# 输出: myapp-1.2.0.tgz

# 生成index.yaml
helm repo index . --url https://charts.example.com

# 推送到ChartMuseum
curl --data-binary "@myapp-1.2.0.tgz" https://chartmuseum.example.com/api/charts

OCI Registry

1
2
3
4
5
6
7
8
# 登录OCI Registry
helm registry login ghcr.io -u username

# 推送到OCI Registry
helm push myapp-1.2.0.tgz oci://ghcr.io/myorg/charts

# 从OCI Registry安装
helm install myrelease oci://ghcr.io/myorg/charts/myapp --version 1.2.0
graph TB
    Dev["开发者"] --> |"helm push"| Registry["OCI Registry<br>ghcr.io / ECR / ACR"]
    CI["CI Pipeline"] --> |"helm push"| Registry
    Registry --> |"helm install"| Staging["Staging集群"]
    Registry --> |"helm install"| Prod["Production集群"]

最佳实践

1. 版本管理

1
2
3
4
# 使用语义化版本
# Chart版本: 与Chart模板变更对应
# AppVersion: 与应用本身版本对应
# 两者独立管理

2. 安全加固

1
2
3
4
5
6
7
8
9
10
11
12
13
# 默认安全上下文
podSecurityContext:
  runAsNonRoot: true
  fsGroup: 65534
  seccompProfile:
    type: RuntimeDefault

securityContext:
  allowPrivilegeEscalation: false
  readOnlyRootFilesystem: true
  runAsUser: 65534
  capabilities:
    drop: ["ALL"]

3. 多环境管理

1
2
3
4
5
6
7
8
# 使用不同values文件
helm install myapp ./myapp \
  -f values.yaml \
  -f values-production.yaml \
  --set image.tag=v3.5.1

# values文件优先级（从低到高）
# values.yaml < values-production.yaml < --set参数

总结

Helm Chart开发的核心要点：

  1. 结构清晰：遵循标准Chart结构，合理组织模板
  2. 配置灵活：通过values.yaml提供合理的默认值和丰富的配置项
  3. 安全默认：默认启用安全上下文、资源限制
  4. 可测试：编写Chart测试，使用lint和dry-run验证
  5. 文档完善：NOTES.txt提供安装后指引，README说明配置项
  6. 版本规范：Chart版本和AppVersion独立管理，遵循语义化版本
作者 · authorzt
发布 · date2024-05-29
篇幅 · length3.0k 字 · 8 min
许可 · licenseCC BY-SA 4.0
$ echo "comments" · 评论
CI/CD流水线设计：从Jenkins到GitHub Actions Kubernetes资源管理与调度策略