[실습] 5. Kyverno

Kyverno : K8S Native Policy Mgmt

• [EKS Workshop] Policy management with Kyverno - Link
• Managing Pod Security on Amazon EKS with Kyverno - 링크 & PSS - Link
• Kyverno (Greek for “govern”) is a policy engine designed specifically for Kubernetes.
  • It is a Cloud Native Computing Foundation (CNCF) project allowing teams to collaborate and enforce Policy-as-Code.
  • Kyverno allows for declarative Kubernetes resources written in YAML, with no new policy language to learn, and results are available as Kubernetes resources and as events.
  • Kyverno policies can be used to validate, mutate, and generate resource configurations, and also validate image signatures and attestations, providing all the necessary building blocks for a complete software supply chain security standards enforcement.
  • Kyverno policies can match resources using the resource kind, name, label selectors, and much more.
• 기능 - Link
  • policies as Kubernetes resources (no new language to learn!)
  • validate, mutate, generate, or cleanup (remove) any resource
  • verify container images for software supply chain security
  • inspect image metadata
  • match resources using label selectors and wildcards
  • validate and mutate using overlays (like Kustomize!)
  • synchronize configurations across Namespaces
  • block non-conformant resources using admission controls, or report policy violations
  • self-service reports (no proprietary audit log!)
  • self-service policy exceptions
  • test policies and validate resources using the Kyverno CLI, in your CI/CD pipeline, before applying to your cluster
  • manage policies as code using familiar tools like git and kustomize
• 동작 : Dynamic Admission Control 로 실행, Mutating/Validating admission 에서 동작하여 허용/거부 결과 반환
  • The two major components are the Webhook Server & the Webhook Controller.
    • The Webhook Server handles incoming AdmissionReview requests from the Kubernetes API server and sends them to the Engine for processing.
    • It is dynamically configured by the Webhook Controller which watches the installed policies and modifies the webhooks to request only the resources matched by those policies.
  • The Webhook is the server which handles incoming AdmissionReview requests from the Kubernetes API server and sends them to the Engine for processing.
  • It is dynamically configured by the Webhook Controller which watches the installed policies and modifies the webhooks to request only the resources matched by those policies.
  • The Cert Renewer is responsible for watching and renewing the certificates, stored as Kubernetes Secrets, needed by the webhook.
  • The Background Controller handles all generate and mutate-existing policies by reconciling UpdateRequests, an intermediary resource.
  • And the Report Controllers handle creation and reconciliation of Policy Reports from their intermediary resources, Admission Reports and Background Scan Reports.

설치 - HelmChart

https://kyverno.io/docs/introduction/

https://kyverno.io/docs/installation/

 

# 설치
# EKS 설치 시 참고 <https://kyverno.io/docs/installation/platform-notes/#notes-for-eks-users>
# 모니터링 참고 <https://kyverno.io/docs/monitoring/>
cat << EOF > kyverno-value.yaml
config:
  resourceFiltersExcludeNamespaces: [ kube-system ]

admissionController:
  serviceMonitor:
    enabled: true

backgroundController:
  serviceMonitor:
    enabled: true

cleanupController:
  serviceMonitor:
    enabled: true

reportsController:
  serviceMonitor:
    enabled: true
EOF

 

 

kubectl create ns kyverno
helm repo add kyverno <https://kyverno.github.io/kyverno/>
helm install kyverno kyverno/kyverno --version 3.2.0-rc.3 -f kyverno-value.yaml -n kyverno

 

 

# 확인
kubectl get all -n kyverno
kubectl get crd | grep kyverno
kubectl get pod,svc -n kyverno

 

 

# (참고) 기본 인증서 확인 <https://kyverno.io/docs/installation/customization/#default-certificates>
# step-cli 설치 <https://smallstep.com/docs/step-cli/installation/>
wget <https://dl.smallstep.com/cli/docs-cli-install/latest/step-cli_amd64.rpm>
sudo rpm -i step-cli_amd64.rpm

wget https://dl.smallstep.com/cli/docs-cli-install/latest/step-cli_amd64.rpm

 

kubectl -n kyverno get secret
kubectl -n kyverno get secret kyverno-svc.kyverno.svc.kyverno-tls-ca -o jsonpath='{.data.tls\\.crt}' | base64 -d
kubectl -n kyverno get secret kyverno-svc.kyverno.svc.kyverno-tls-ca -o jsonpath='{.data.tls\\.crt}' | base64 -d | step certificate inspect --short

 

kubectl get validatingwebhookconfiguration kyverno-policy-validating-webhook-cfg -o jsonpath='{.webhooks[0].clientConfig.caBundle}' | base64 -d | step certificate inspect --short

 

프로메테우스

 

그라파나 대시보드 - 15987, 15804

https://kyverno.io/docs/monitoring/bonus-grafana-dashboard/

 

 

Validation - Link

# 모니터링
watch -d kubectl get pod -n kyverno

 

 

# ClusterPolicy 적용
kubectl create -f- << EOF
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: require-labels
spec:
  validationFailureAction: Enforce
  rules:
  - name: check-team
    match:
      any:
      - resources:
          kinds:
          - Pod
    validate:
      message: "label 'team' is required"
      pattern:
        metadata:
          labels:
            team: "?*"
EOF

 

 

# 확인
kubectl get validatingwebhookconfigurations
kubectl get ClusterPolicy

 

# 디플로이먼트 생성 시도
kubectl create deployment nginx --image=nginx

 

 

# 디플로이먼트 생성 시도
kubectl run nginx --image nginx --labels team=backend
kubectl get pod -l team=backend

 

# 확인
kubectl get policyreport -o wide

 

---

 

kubectl get policyreport 776e77e7-4d0a-4011-8a45-273734f3af78 -o yaml | kubectl neat | yh

 

 

# 정책 삭제
kubectl delete clusterpolicy require-labels

 

Policy and Role : Kyverno Policy는 rules 모음 - Link

https://kyverno.io/docs/kyverno-policies/

• 각 규칙은 [match](<https://kyverno.io/docs/writing-policies/match-exclude/>)선언, 선택적 [exclude](<https://kyverno.io/docs/writing-policies/match-exclude/>)선언 및 [validate](<https://kyverno.io/docs/writing-policies/validate/>), [mutate](<https://kyverno.io/docs/writing-policies/mutate/>), [generate](<https://kyverno.io/docs/writing-policies/generate>)또는 [verifyImages](<https://kyverno.io/docs/writing-policies/verify-images>)선언 중 하나로 구성됩니다.
• 각 규칙에는 단일 validate, mutate, generate또는 verifyImages하위 선언만 포함될 수 있습니다.

 

Mutation - Link

kubectl create -f- << EOF
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: add-labels
spec:
  rules:
  - name: add-team
    match:
      any:
      - resources:
          kinds:
          - Pod
    mutate:
      patchStrategicMerge:
        metadata:
          labels:
            +(team): bravo
EOF

 

 

# 확인
kubectl get mutatingwebhookconfigurations
kubectl get ClusterPolicy

 

# 파드 생성 후 label 확인
kubectl run redis --image redis
kubectl get pod redis --show-labels

 

# 파드 생성 후 label 확인 : 바로 위와 차이점은?
kubectl run newredis --image redis -l team=alpha
kubectl get pod newredis --show-labels

 

 

# 삭제
kubectl delete clusterpolicy add-labels

 

Generation

We will use a Kyverno generate policy to generate an image pull secret in a new Namespace. - Link

# First, create this Kubernetes Secret in your cluster which will simulate a real image pull secret.
kubectl -n default create secret docker-registry regcred \\
  --docker-server=myinternalreg.corp.com \\
  --docker-username=john.doe \\
  --docker-password=Passw0rd123! \\
  --docker-email=john.doe@corp.com

 

 

kubectl get secret regcred

 

 

kubectl create -f- << EOF
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata:
  name: sync-secrets
spec:
  rules:
  - name: sync-image-pull-secret
    match:
      any:
      - resources:
          kinds:
          - Namespace
    generate:
      apiVersion: v1
      kind: Secret
      name: regcred
      namespace: "{{request.object.metadata.name}}"
      synchronize: true
      clone:
        namespace: default
        name: regcred
EOF

 

kubectl get ClusterPolicy

 

 

# 신규 네임스페이스 생성 후 확인
kubectl create ns mytestns
kubectl -n mytestns get secret

 

# 삭제
kubectl delete clusterpolicy sync-secrets

 

kyverno 모니터링

프로메테우스 - Link & 그라파나 대시보드 - Link