云原生之旅 - 11）基于 Kubernetes 动态伸缩 Jenkins Build Agents _生活百科

前言上一篇文章云原生之旅 - 10）手把手教你安装 Jenkins on Kubernetes 我们介绍了在 Kubernetes 上安装 Jenkins ，本文介绍下如何设置k8s pod作为Jenkins 构建job的 agent 。
Jenkins master 和 agent 均以 pod 的形式运行在 Kubernetes 节点上。Master 运行在其中一个节点上，其配置数据 Jenkins home 使用存储卷挂载，master pod重启不会导致数据丢失。agent 运行在各个节点上，根据需求动态创建并自动释放。这样做的好处很多，比如高可用，高伸缩性，资源利用率高。
关键词：Jenkins on Kubernetes 实践，Jenkins 和 Kubernetes，在Kubernetes上安装Jenkins，Jenkins 高可用安装，Jenkins 动态伸缩构建， Kubernetes Pod as Jenkins build agent
准备

已搭建 Jenkins master on kubernetes 云原生之旅 - 10）手把手教你安装 Jenkins on Kubernetes
准备一个 Service Account ，对目标 cluster 具有k8s admin权限，以便部署。
防火墙已开通 Jenkins 出站到Docker hub，方便 push/pull image
防火墙已开通 Jenkins 到目标 cluster，以便部署。

插件安装

Kubernetes Plugin
Google Kubernetes Engine Plugin (我的例子是部署到 GKE cluster)

Jenkins 配置Manage Nodes and Clouds1. Go to `Manage Jenkins` –> `Manage Nodes and Clouds`2. Click `Configure Clouds`3. Add a new Cloud select `Kubernetes`4. Click `Kubernetes Cloud Detail5. Enter `jenkins` namespace in `Kubernetes Namespace` field6. Click `Test Connection` --> result show `Connected to Kubernetes v1.22.12-gke.2300`7. Click `Save`8. Enter `http://jenkins-service.jenkins.svc.cluster.local:8080` in `Jenkins URL` field9. Enter `jenkins-agent:50000` in `Jenkins tunnel` field

云原生之旅 - 11）基于 Kubernetes 动态伸缩 Jenkins Build Agents

文章插图
10. Click `Add Pod Template` then `Pod Template Details`
11. Input `Name`=`jenkins-agent`, `Namespace`=`jenkins`, `Labels`=`kubeagent`

文章插图
12. (Optional) 如果不添加 container template, the Jenkins Kubernetes plugin will use the default JNLP image from the Docker hub to spin up the agents.如果你要覆盖默认的jnlp image 可以 Click `Add Container` to add Container Template,输入 `Name`=`jnlp`, `Docker Image`=`your_registry/jenkins/inbound-agent:4.11-1-jdk11`

文章插图
Ensure that you remove the sleep and 9999999 default argument from the container template.
Manage Credentials

Add `Usernames with password` for docker hub account/pwd ，比如 wade_test_dockerhub
Add `Google Service Account from private key` 比如 gcp_sa_json_key

Credentials 会在Jenkinsfile里面用到。
### 本文首发于博客园 https://www.cnblogs.com/wade-xu/p/16863955.html
Test a freestyle projectGo to Jenkins home –> New Item and create a freestyle project，命名为 quick-test在 job description 部分, add the label `kubeagent` for `Restrict where this project can be run`.

文章插图
这个label 和我们上面创建 pod template时用的label一致. 这样的话 Jenkins就知道用哪个 pod template 作为 agent container.
随便添加一个shell 作为build steps

文章插图
点Build Now
查看Console OutputAgent jenkins-agent-l7hw9 is provisioned from template jenkins-agent......Building remotely on jenkins-agent-l7hw9 (kubeagent) in workspace /home/jenkins/agent/workspace/quick-test[quick-test] $ /bin/sh -xe /tmp/jenkins17573873264046707236.sh+ echo test pipelinetest pipelineFinished: SUCCESS### 本文首发于博客园 https://www.cnblogs.com/wade-xu/p/16863955.html
JenkinsfileCI接着我们用 Jenkinsfile 写一个 Declarative pipeline - build/push docker image 到docker hub首先需要定义一个 pod.yaml 作为启动 agent 的container

文章插图

文章插图
kind: Podspec:containers:# list of containers that you want present for your build, you can define a default container in the Jenkinsfile- name: mavenimage: maven:3.5.4-jdk-8-slimcommand: ["tail", "-f", "/dev/null"]# this or any command that is bascially a noop is required, this is so that you don't overwrite the entrypoint of the base containerimagePullPolicy: Always # use cache or pull image for agentresources:# request and limit the resources your build contaienrrequests:memory: 4Gicpu: 2limits:memory: 4Gicpu: 2volumeMounts:- mountPath: /root/.m2 # maven .m2 cache directoryname: maven-home- name: gitimage: bitnami/git:2.38.1imagePullPolicy: IfNotPresentcommand: ["tail", "-f", "/dev/null"]resources: # limit the resources your build contaienrlimits:cpu: 100mmemory: 256Mi- name: kubectl-kustomizeimage: line/kubectl-kustomize:1.25.3-4.5.7imagePullPolicy: IfNotPresentcommand: ["tail", "-f", "/dev/null"]resources: # limit the resources your build contaienrlimits:cpu: 100mmemory: 256Mi- name: dockerimage: docker:18.06.1command: ["tail", "-f", "/dev/null"]imagePullPolicy: AlwaysvolumeMounts:- name: dockermountPath: /var/run/docker.sock # We use the k8s host docker enginevolumes:- name: dockerhostPath:path: /var/run/docker.sock- name: maven-homepersistentVolumeClaim:claimName: maven-repo-storage