Add phase 1 of validation tightening.
https://github.com/kubernetes/enhancements/blob/master/keps/sig-storage/177-volume-snapshot/tighten-validation-webhook-crd.md 1. Ratcheting validation webhook server image 2. Controller labels invalid objects 3. Unit tests for webhook 4. Deployment README and example deployment method with certs 5. Update top-level README Racheting validation: 1. webhook is strict on create 2. webhook is strict on updates where the existing object passes strict validation 3. webhook is relaxed on updates where the existing object fails strict validation (allows finalizer removal, status update, deletion, etc) Additionally the validating wehook server will perform immutability checks on scenario 2 above.
This commit is contained in:
Andi Li
81
deploy/kubernetes/webhook-example/README.md
Normal file
81
deploy/kubernetes/webhook-example/README.md
Normal file
@@ -0,0 +1,81 @@
|
||||
# Validating Webhook
|
||||
|
||||
The snapshot validating webhook is an HTTP callback which responds to [admission requests](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/). It is part of a larger [plan](https://github.com/kubernetes/enhancements/blob/master/keps/sig-storage/177-volume-snapshot/tighten-validation-webhook-crd.md) to tighten validation for volume snapshot objects. This webhook introduces the [ratcheting validation](https://github.com/kubernetes/enhancements/blob/master/keps/sig-storage/177-volume-snapshot/tighten-validation-webhook-crd.md#backwards-compatibility) mechanism targeting the tighter validation.
|
||||
|
||||
> :warning: **WARNING**: Choosing not to install the webhook server and participate in the phased release process can cause future problems when upgrading from `v1beta1` to `v1` volumesnapshot API if there are currently persisted objects which fail the new stricter validation. Potential impacts include being unable to delete invalid snapshot objects.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
The following are prerequisites to use this validating webhook:
|
||||
|
||||
- K8s version 1.17+ (v1.9+ to use `admissionregistration.k8s.io/v1beta1`, v1.16+ to use `admissionregistration.k8s.io/v1`, v1.17+ to use `snapshot.storage.k8s.io/v1beta1`)
|
||||
- ValidatingAdmissionWebhook is [enabled](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/#prerequisites). (in v1.18+ it will be enabled by default)
|
||||
- API `admissionregistration.k8s.io/v1beta1` or `admissionregistration.k8s.io/v1` is enabled.
|
||||
|
||||
## How to build the webhook
|
||||
|
||||
Build the binary
|
||||
|
||||
```bash
|
||||
make
|
||||
```
|
||||
|
||||
Build the docker image
|
||||
|
||||
```bash
|
||||
docker build -t validation-webhook:latest -f ./cmd/validation-webhook/Dockerfile .
|
||||
```
|
||||
|
||||
## How to deploy the webhook
|
||||
|
||||
The webhook server is provided as an image which can be built from this repository. It can be deployed anywhere, as long as the api server is able to reach it over HTTPS. It is recommended to deploy the webhook server in the cluster as snapshotting is latency sensitive. A `ValidatingWebhookConfiguration` object is needed to configure the api server to contact the webhook server. Please see the [documentation](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/) for more details. The webhook server code is adapted from the [webhook server](https://github.com/kubernetes/kubernetes/tree/v1.18.6/test/images/agnhost/webhook) used in the kubernetes/kubernetes end to end testing code.
|
||||
|
||||
### Example in-cluster deployment using Kubernetes Secrets
|
||||
|
||||
Please note this is not considered to be a production ready method to deploy the certificates and is only provided for demo purposes. This is only one of many ways to deploy the certificates, it is your responsibility to ensure the security of your cluster. TLS certificates and private keys should be handled with care and you may not want to keep them in plain Kubernetes secrets.
|
||||
|
||||
This method was heavily adapted from [banzai cloud](https://banzaicloud.com/blog/k8s-admission-webhooks/).
|
||||
|
||||
#### Method
|
||||
|
||||
These commands should be run from the top level directory.
|
||||
|
||||
1. Run the `create-cert.sh` script. Note using the default namespace will allow anyone with access to that namespace to read your secret. It is recommended to change the namespace in all the files and the commands given below.
|
||||
|
||||
|
||||
```bash
|
||||
# This script will create a TLS certificate signed by the [cluster](https://kubernetes.io/docs/tasks/tls/managing-tls-in-a-cluster/). It will place the public and private key into a secret on the cluster.
|
||||
./deploy/kubernetes/webhook-example/create-cert.sh --service snapshot-validation-service --secret snapshot-validation-secret --namespace default # Make sure to use a different namespace
|
||||
```
|
||||
|
||||
2. Patch the `ValidatingWebhookConfiguration` file from the template, filling in the CA bundle field.
|
||||
|
||||
```bash
|
||||
cat ./deploy/kubernetes/webhook-example/admission-configuration-template | ./deploy/kubernetes/webhook-example/patch-ca-bundle.sh > ./deploy/kubernetes/webhook-example/admission-configuration.yaml
|
||||
```
|
||||
|
||||
3. Change the namespace in the generated `admission-configuration.yaml` file. Change the namespace in the service and deployment in the `webhook.yaml` file.
|
||||
|
||||
4. Create the deployment, service and admission configuration objects on the cluster.
|
||||
|
||||
```bash
|
||||
kubectl apply -f ./deploy/kubernetes/webhook-example
|
||||
```
|
||||
|
||||
Once all the pods from the deployment are up and running, you should be ready to go.
|
||||
|
||||
#### Verify the webhook works
|
||||
|
||||
Try to create an invalid snapshot object, the snapshot creation should fail.
|
||||
|
||||
```bash
|
||||
kubectl create -f ./examples/kubernetes/invalid-snapshot.yaml
|
||||
```
|
||||
|
||||
### Other methods to deploy the webhook server
|
||||
|
||||
Look into [cert-manager](https://cert-manager.io/) to handle the certificates, and this kube-builder [tutorial](https://book.kubebuilder.io/cronjob-tutorial/cert-manager.html) on how to deploy a webhook.
|
||||
|
||||
#### Important
|
||||
|
||||
Please see the deployment [yaml](./webhook.yaml) for the arguments expected by the webhook server. The snapshot validation webhook is served at the path `/snapshot`.
|
||||
@@ -0,0 +1,23 @@
|
||||
apiVersion: admissionregistration.k8s.io/v1
|
||||
kind: ValidatingWebhookConfiguration
|
||||
metadata:
|
||||
name: "validation-webhook.storage.sigs.k8s.io"
|
||||
namespace: "default"
|
||||
webhooks:
|
||||
- name: "snapshot.validation-webhook.storage.sigs.k8s.io"
|
||||
rules:
|
||||
- apiGroups: ["snapshot.storage.k8s.io"]
|
||||
apiVersions: ["v1beta1"]
|
||||
operations: ["CREATE", "UPDATE"]
|
||||
resources: ["volumesnapshots", "volumesnapshotcontents"]
|
||||
scope: "*"
|
||||
clientConfig:
|
||||
service:
|
||||
namespace: "default"
|
||||
name: "snapshot-validation-service"
|
||||
path: "/volumesnapshot"
|
||||
caBundle: ${CA_BUNDLE}
|
||||
admissionReviewVersions: ["v1", "v1beta1"]
|
||||
sideEffects: None
|
||||
failurePolicy: Fail # We recommend switching to Fail only after successful installation of the server and webhook.
|
||||
timeoutSeconds: 10 # This will affect the latency and performance. Finetune this value based on your application's tolerance.
|
||||
126
deploy/kubernetes/webhook-example/create-cert.sh
Executable file
126
deploy/kubernetes/webhook-example/create-cert.sh
Executable file
@@ -0,0 +1,126 @@
|
||||
#!/bin/bash
|
||||
# File originally from https://github.com/banzaicloud/admission-webhook-example/blob/blog/deployment/webhook-create-signed-cert.sh
|
||||
|
||||
set -e
|
||||
|
||||
usage() {
|
||||
cat <<EOF
|
||||
Generate certificate suitable for use with a webhook service.
|
||||
This script uses k8s' CertificateSigningRequest API to a generate a
|
||||
certificate signed by k8s CA suitable for use with webhook
|
||||
services. This requires permissions to create and approve CSR. See
|
||||
https://kubernetes.io/docs/tasks/tls/managing-tls-in-a-cluster for
|
||||
detailed explantion and additional instructions.
|
||||
The server key/cert k8s CA cert are stored in a k8s secret.
|
||||
usage: ${0} [OPTIONS]
|
||||
The following flags are required.
|
||||
--service Service name of webhook.
|
||||
--namespace Namespace where webhook service and secret reside.
|
||||
--secret Secret name for CA certificate and server certificate/key pair.
|
||||
EOF
|
||||
exit 1
|
||||
}
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case ${1} in
|
||||
--service)
|
||||
service="$2"
|
||||
shift
|
||||
;;
|
||||
--secret)
|
||||
secret="$2"
|
||||
shift
|
||||
;;
|
||||
--namespace)
|
||||
namespace="$2"
|
||||
shift
|
||||
;;
|
||||
*)
|
||||
usage
|
||||
;;
|
||||
esac
|
||||
shift
|
||||
done
|
||||
|
||||
[ -z ${service} ] && service=admission-webhook-example-svc
|
||||
[ -z ${secret} ] && secret=admission-webhook-example-certs
|
||||
[ -z ${namespace} ] && namespace=default
|
||||
|
||||
if [ ! -x "$(command -v openssl)" ]; then
|
||||
echo "openssl not found"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
csrName=${service}.${namespace}
|
||||
tmpdir=$(mktemp -d)
|
||||
echo "creating certs in tmpdir ${tmpdir} "
|
||||
|
||||
cat <<EOF >> ${tmpdir}/csr.conf
|
||||
[req]
|
||||
req_extensions = v3_req
|
||||
distinguished_name = req_distinguished_name
|
||||
[req_distinguished_name]
|
||||
[ v3_req ]
|
||||
basicConstraints = CA:FALSE
|
||||
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
|
||||
extendedKeyUsage = serverAuth
|
||||
subjectAltName = @alt_names
|
||||
[alt_names]
|
||||
DNS.1 = ${service}
|
||||
DNS.2 = ${service}.${namespace}
|
||||
DNS.3 = ${service}.${namespace}.svc
|
||||
EOF
|
||||
|
||||
openssl genrsa -out ${tmpdir}/server-key.pem 2048
|
||||
openssl req -new -key ${tmpdir}/server-key.pem -subj "/CN=${service}.${namespace}.svc" -out ${tmpdir}/server.csr -config ${tmpdir}/csr.conf
|
||||
|
||||
# clean-up any previously created CSR for our service. Ignore errors if not present.
|
||||
kubectl delete csr ${csrName} 2>/dev/null || true
|
||||
|
||||
# create server cert/key CSR and send to k8s API
|
||||
cat <<EOF | kubectl create -f -
|
||||
apiVersion: certificates.k8s.io/v1beta1
|
||||
kind: CertificateSigningRequest
|
||||
metadata:
|
||||
name: ${csrName}
|
||||
spec:
|
||||
groups:
|
||||
- system:authenticated
|
||||
request: $(cat ${tmpdir}/server.csr | base64 | tr -d '\n')
|
||||
usages:
|
||||
- digital signature
|
||||
- key encipherment
|
||||
- server auth
|
||||
EOF
|
||||
|
||||
# verify CSR has been created
|
||||
while true; do
|
||||
kubectl get csr ${csrName}
|
||||
if [ "$?" -eq 0 ]; then
|
||||
break
|
||||
fi
|
||||
done
|
||||
|
||||
# approve and fetch the signed certificate
|
||||
kubectl certificate approve ${csrName}
|
||||
# verify certificate has been signed
|
||||
for x in $(seq 10); do
|
||||
serverCert=$(kubectl get csr ${csrName} -o jsonpath='{.status.certificate}')
|
||||
if [[ ${serverCert} != '' ]]; then
|
||||
break
|
||||
fi
|
||||
sleep 1
|
||||
done
|
||||
if [[ ${serverCert} == '' ]]; then
|
||||
echo "ERROR: After approving csr ${csrName}, the signed certificate did not appear on the resource. Giving up after 10 attempts." >&2
|
||||
exit 1
|
||||
fi
|
||||
echo ${serverCert} | openssl base64 -d -A -out ${tmpdir}/server-cert.pem
|
||||
|
||||
|
||||
# create the secret with CA cert and server cert/key
|
||||
kubectl create secret generic ${secret} \
|
||||
--from-file=key.pem=${tmpdir}/server-key.pem \
|
||||
--from-file=cert.pem=${tmpdir}/server-cert.pem \
|
||||
--dry-run -o yaml |
|
||||
kubectl -n ${namespace} apply -f -
|
||||
16
deploy/kubernetes/webhook-example/patch-ca-bundle.sh
Executable file
16
deploy/kubernetes/webhook-example/patch-ca-bundle.sh
Executable file
@@ -0,0 +1,16 @@
|
||||
#!/bin/bash
|
||||
# File originally from https://github.com/banzaicloud/admission-webhook-example/blob/blog/deployment/webhook-patch-ca-bundle.sh
|
||||
|
||||
ROOT=$(cd $(dirname $0)/../../; pwd)
|
||||
|
||||
set -o errexit
|
||||
set -o nounset
|
||||
set -o pipefail
|
||||
|
||||
export CA_BUNDLE=$(kubectl config view --raw -o json | jq -r '.clusters[0].cluster."certificate-authority-data"' | tr -d '"')
|
||||
|
||||
if command -v envsubst >/dev/null 2>&1; then
|
||||
envsubst
|
||||
else
|
||||
sed -e "s|\${CA_BUNDLE}|${CA_BUNDLE}|g"
|
||||
fi
|
||||
44
deploy/kubernetes/webhook-example/webhook.yaml
Normal file
44
deploy/kubernetes/webhook-example/webhook.yaml
Normal file
@@ -0,0 +1,44 @@
|
||||
apiVersion: apps/v1
|
||||
kind: Deployment
|
||||
metadata:
|
||||
name: snapshot-validation-deployment
|
||||
namespace: default # NOTE: change the namespace
|
||||
labels:
|
||||
app: snapshot-validation
|
||||
spec:
|
||||
replicas: 3
|
||||
selector:
|
||||
matchLabels:
|
||||
app: snapshot-validation
|
||||
template:
|
||||
metadata:
|
||||
labels:
|
||||
app: snapshot-validation
|
||||
spec:
|
||||
containers:
|
||||
- name: snapshot-validation
|
||||
image: k8s.gcr.io/sig-storage/validation-webhook:v2.2.0 # change the image if you wish to use your own custom validation server image
|
||||
args: ['--tls-cert-file=/etc/snapshot-validation-webhook/certs/cert.pem', '--tls-private-key-file=/etc/snapshot-validation-webhook/certs/key.pem']
|
||||
ports:
|
||||
- containerPort: 443 # change the port as needed
|
||||
volumeMounts:
|
||||
- name: snapshot-validation-webhook-certs
|
||||
mountPath: /etc/snapshot-validation-webhook/certs
|
||||
readOnly: true
|
||||
volumes:
|
||||
- name: snapshot-validation-webhook-certs
|
||||
secret:
|
||||
secretName: snapshot-validation-secret
|
||||
---
|
||||
apiVersion: v1
|
||||
kind: Service
|
||||
metadata:
|
||||
name: snapshot-validation-service
|
||||
namespace: default # NOTE: change the namespace
|
||||
spec:
|
||||
selector:
|
||||
app: snapshot-validation
|
||||
ports:
|
||||
- protocol: TCP
|
||||
port: 443 # Change if needed
|
||||
targetPort: 443 # Change if the webserver image expects a different port
|
||||
Reference in New Issue
Block a user