Skywalking Swck Agent注入实现分析

项目地址:GitHub - apache/skywalking-swck: Apache SkyWalking Cloud on Kubernetes
项目简介:A bridge project between Apache SkyWalking and Kubernetes.SWCK is a platform for the SkyWalking user that provisions, upgrades, maintains SkyWalking relevant components, and makes them work natively on Kubernetes.skywalking-swck是一个在skywalking和kubernetes之间架起一座桥梁性质的项目 。可以给用户提供skywalking相关组件及后期升级、维护 。让他们使用起来更加云原生 。
项目特性

  • Java Agent Injector: Inject the java agent into the application pod natively.
    • Inject the java agent into the application pod.
    • Leverage a global configuration to simplify the agent and injector setup.
    • Use the annotation to customize specific workloads.
    • Synchronize injecting status to JavaAgent CR for monitoring purposes.
  • Operator: Provision and maintain SkyWalking backend components.
  • Custom Metrics Adapter: Provides custom metrics coming from SkyWalking OAP cluster for autoscaling by Kubernetes HPA
  • 注入Java Agent: 以更加云原生的试注入java agent
    • 将java agent注入到应用pod
    • 可以全局化配置,简化agent注入操作
    • 使用annotation自定义一些配置
    • 将注入的状态同步到JavaAgent这个CR对象,便于监控
  • Operator: 提供和维护SkyWalking后端的组件
  • 自定义指标适配: 能够提供来自于SkyWalking OAP的自定义指标给kubernetes HPA,以便自动扩缩容
使用skywalking-swck/java-agent-injector.md at master · apache/skywalking-swck
  1. 下载并安装Operator
  2. 创建ConfigMap/SwAgent等全局配置Annotations > SwAgent > Configmap (Deprecated) > Default Configmap (Deprecated)
  3. 接入应用配置label/annotationsskywalking agent inject on kubernetes
原理概述当kubectl apply 一个 deployment资源后,k8s会创建pod,此时k8s根据mutatingwebhookconfigurations资源配置(配置了监控的资源以及webhook server信息),调用相应的webhook server,webhook server会进行处理,在pod yaml中注入initContainer配置,使业务容器与initContainer容器共享skywalking agent目录 , 并且配置JAVA_TOOL_OPTIONS环境变量值为"-javaagent:/sky/agent/skywalking-agent.jar=agent.service_name=xxxx",这样JVM启动时,会附加上javaagent,以达到目的 。
详述
  1. 首先我们来看一下MutatingWebhookConfiguration和 ValidatingWebhookConfiguration资源查看kubectl explain 对这两个资源的描述
MutatingWebhookConfiguration describes the configuration of and admissionwebhook that accept or reject and may change the object.ValidatingWebhookConfiguration describes the configuration of and admissionwebhook that accept or reject and object without changing it.简而言之,这两种资源都是准入控制器(Admission Controller)的两种实现,都能控制是否接受还是拒绝对资源对象的变化,但不同的是,MutatingWebhookConfiguration可以改变资源对象,而ValidatingWebhookConfiguration不可以,可以参看搞懂 Kubernetes 准入控制(Admission Controller)详细了解 。2. swck就是利用MutatingWebhookConfiguration实现了对pod的修改,我们来看下swck中的定义
apiVersion: admissionregistration.k8s.io/v1kind: MutatingWebhookConfigurationmetadata:annotations:cert-manager.io/inject-ca-from: skywalking-swck-system/skywalking-swck-serving-certname: skywalking-swck-mutating-webhook-configurationwebhooks:- admissionReviewVersions:- v1clientConfig:service:name: skywalking-swck-webhook-servicenamespace: skywalking-swck-systempath: /mutate-v1-podfailurePolicy: Failname: mpod.kb.ionamespaceSelector:matchLabels:swck-injection: enabledrules:- apiGroups:- ""apiVersions:- v1operations:- CREATE- UPDATEresources:- podssideEffects: None从这段定义中可以看出,当带有标签swck-injection: enabled的Namespace下的POD资源有CREATE或者UPDATE操作时,将会调用path: /mutate-v1-pod 。3. 在swck项目中的operator/man.go中找到此URL
// register a webhook to enable the java agent injectorsetupLog.Info("registering /mutate-v1-pod webhook")mgr.GetWebhookServer().Register("/mutate-v1-pod",&webhook.Admission{Handler: &injector.JavaagentInjector{Client: mgr.GetClient()}})setupLog.Info("/mutate-v1-pod webhook is registered")swck向k8s注册了/mutate-v1-pod以及对应的handler 。我们可以想到,当create或update pod时 , k8s将会调用path对应的handler处理 。4. 查看Handler: injector.JavaagentInjector
【Skywalking Swck Agent注入实现分析】// Handle will process every coming pod under the// specified namespace which labeled "swck-injection=enabled"func (r *JavaagentInjector) Handle(ctx context.Context, req admission.Request) admission.Response {pod := &corev1.Pod{}if err := r.decoder.Decode(req, pod); err != nil {return admission.Errored(http.StatusBadRequest, err)}// set Annotations to avoid repeated judgmentsif pod.Annotations == nil {pod.Annotations = map[string]string{}}// 查找所有匹配的swAgentswAgentL := r.findMatchedSwAgentL(ctx, req, pod)//初始化所有annotations(加载所有annotations)anno, err := NewAnnotations()if err != nil {javaagentInjectorLog.Error(err, "get NewAnnotations error")}//创建AnnotationOverlay对象,它是一个map,用于保存被overlaied的annotationao := NewAnnotationOverlay()//创建SidecarInjectField对象s := NewSidecarInjectField()//构建inject链对象ip := NewInjectProcess(ctx, s, anno, ao, swAgentL, pod, req, javaagentInjectorLog, r.Client)//开始injectreturn ip.Run()}

推荐阅读