alexlebens/external-snapshotter
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Merge commit '8b2327a85107da8d23afb886bd510749b46e7e50' into prow-update-master

Browse Source
This commit is contained in:
Mauricio Poppe
2021-09-10 18:28:34 +00:00
parent 390a2debe9 80b6445049
commit 8451704d33
6 changed files with 50 additions and 39 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
release-tools/README.md
View File

@@ -42,7 +42,7 @@ images. Building from master creates the main `canary` image.
Sharing and updating Sharing and updating
-------------------- --------------------
[`git subtree`](https://github.com/git/git/blob/master/contrib/subtree/git-subtree.txt) [`git subtree`](https://github.com/git/git/blob/HEAD/contrib/subtree/git-subtree.txt)
is the recommended way of maintaining a copy of the rules inside the is the recommended way of maintaining a copy of the rules inside the
`release-tools` directory of a project. This way, it is possible to make `release-tools` directory of a project. This way, it is possible to make
changes also locally, test them and then push them back to the shared changes also locally, test them and then push them back to the shared
@@ -89,7 +89,7 @@ main
All Kubernetes-CSI repos are expected to switch to Prow. For details All Kubernetes-CSI repos are expected to switch to Prow. For details
on what is enabled in Prow, see on what is enabled in Prow, see
https://github.com/kubernetes/test-infra/tree/master/config/jobs/kubernetes-csi https://github.com/kubernetes/test-infra/tree/HEAD/config/jobs/kubernetes-csi
Test results for periodic jobs are visible in Test results for periodic jobs are visible in
https://testgrid.k8s.io/sig-storage-csi-ci https://testgrid.k8s.io/sig-storage-csi-ci

2
release-tools/SECURITY_CONTACTS
View File

@@ -4,7 +4,7 @@
# to for triaging and handling of incoming issues. # to for triaging and handling of incoming issues.
# #
# The below names agree to abide by the # The below names agree to abide by the
# [Embargo Policy](https://github.com/kubernetes/sig-release/blob/master/security-release-process-documentation/security-release-process.md#embargo-policy) # [Embargo Policy](https://github.com/kubernetes/sig-release/blob/HEAD/security-release-process-documentation/security-release-process.md#embargo-policy)
# and will be removed and replaced if they violate that agreement. # and will be removed and replaced if they violate that agreement.
# #
# DO NOT REPORT SECURITY VULNERABILITIES DIRECTLY TO THESE NAMES, FOLLOW THE # DO NOT REPORT SECURITY VULNERABILITIES DIRECTLY TO THESE NAMES, FOLLOW THE

27
release-tools/SIDECAR_RELEASE_PROCESS.md
View File

@@ -9,13 +9,8 @@ The release manager must:
* Be a member of the kubernetes-csi organization. Open an * Be a member of the kubernetes-csi organization. Open an
[issue](https://github.com/kubernetes/org/issues/new?assignees=&labels=area%2Fgithub-membership&template=membership.md&title=REQUEST%3A+New+membership+for+%3Cyour-GH-handle%3E) in [issue](https://github.com/kubernetes/org/issues/new?assignees=&labels=area%2Fgithub-membership&template=membership.md&title=REQUEST%3A+New+membership+for+%3Cyour-GH-handle%3E) in
kubernetes/org to request membership kubernetes/org to request membership
* Be a top level approver for the repository. To become a top level approver, * Be part of the maintainers group for the repository.
the candidate must demonstrate ownership and deep knowledge of the repository Membership can be requested by submitting a PR to kubernetes/org.
through active maintenance, responding to and fixing issues, reviewing PRs,
test triage.
* Be part of the maintainers or admin group for the repository. admin is a
superset of maintainers, only maintainers level is required for cutting a
release. Membership can be requested by submitting a PR to kubernetes/org.
[Example](https://github.com/kubernetes/org/pull/1467) [Example](https://github.com/kubernetes/org/pull/1467)
## Updating CI Jobs ## Updating CI Jobs
@@ -31,16 +26,16 @@ naming convention `<hostpath-deployment-version>-on-<kubernetes-version>`.
1. "-on-master" jobs are the closest reflection to the new Kubernetes version. 1. "-on-master" jobs are the closest reflection to the new Kubernetes version.
1. Fixes to our prow.sh CI script can be tested in the [CSI hostpath 1. Fixes to our prow.sh CI script can be tested in the [CSI hostpath
repo](https://github.com/kubernetes-csi/csi-driver-host-path) by modifying repo](https://github.com/kubernetes-csi/csi-driver-host-path) by modifying
[prow.sh](https://github.com/kubernetes-csi/csi-driver-host-path/blob/master/release-tools/prow.sh) [prow.sh](https://github.com/kubernetes-csi/csi-driver-host-path/blob/HEAD/release-tools/prow.sh)
along with any overrides in along with any overrides in
[.prow.sh](https://github.com/kubernetes-csi/csi-driver-host-path/blob/master/.prow.sh) [.prow.sh](https://github.com/kubernetes-csi/csi-driver-host-path/blob/HEAD/.prow.sh)
to mirror the failing environment. Once e2e tests are passing (verify-unit tests to mirror the failing environment. Once e2e tests are passing (verify-unit tests
will fail), then the prow.sh changes can be submitted to [csi-release-tools](https://github.com/kubernetes-csi/csi-release-tools). will fail), then the prow.sh changes can be submitted to [csi-release-tools](https://github.com/kubernetes-csi/csi-release-tools).
1. Changes can then be updated in all the sidecar repos and hostpath driver repo 1. Changes can then be updated in all the sidecar repos and hostpath driver repo
by following the [update by following the [update
instructions](https://github.com/kubernetes-csi/csi-release-tools/blob/master/README.md#sharing-and-updating). instructions](https://github.com/kubernetes-csi/csi-release-tools/blob/HEAD/README.md#sharing-and-updating).
1. New pull and CI jobs are configured by adding new K8s versions to the top of 1. New pull and CI jobs are configured by adding new K8s versions to the top of
[gen-jobs.sh](https://github.com/kubernetes/test-infra/blob/master/config/jobs/kubernetes-csi/gen-jobs.sh). [gen-jobs.sh](https://github.com/kubernetes/test-infra/blob/HEAD/config/jobs/kubernetes-csi/gen-jobs.sh).
New pull jobs that have been unverified should be initially made optional by New pull jobs that have been unverified should be initially made optional by
setting the new K8s version as setting the new K8s version as
[experimental](https://github.com/kubernetes/test-infra/blob/a1858f46d6014480b130789df58b230a49203a64/config/jobs/kubernetes-csi/gen-jobs.sh#L40). [experimental](https://github.com/kubernetes/test-infra/blob/a1858f46d6014480b130789df58b230a49203a64/config/jobs/kubernetes-csi/gen-jobs.sh#L40).
@@ -52,7 +47,7 @@ naming convention `<hostpath-deployment-version>-on-<kubernetes-version>`.
1. Identify all issues and ongoing PRs that should go into the release, and 1. Identify all issues and ongoing PRs that should go into the release, and
drive them to resolution. drive them to resolution.
1. Download v2.8+ [K8s release notes 1. Download v2.8+ [K8s release notes
generator](https://github.com/kubernetes/release/tree/master/cmd/release-notes) generator](https://github.com/kubernetes/release/tree/HEAD/cmd/release-notes)
1. Generate release notes for the release. Replace arguments with the relevant 1. Generate release notes for the release. Replace arguments with the relevant
information. information.
* Clean up old cached information (also needed if you are generating release * Clean up old cached information (also needed if you are generating release
@@ -95,15 +90,15 @@ naming convention `<hostpath-deployment-version>-on-<kubernetes-version>`.
1. Check [image build status](https://k8s-testgrid.appspot.com/sig-storage-image-build). 1. Check [image build status](https://k8s-testgrid.appspot.com/sig-storage-image-build).
1. Promote images from k8s-staging-sig-storage to k8s.gcr.io/sig-storage. From 1. Promote images from k8s-staging-sig-storage to k8s.gcr.io/sig-storage. From
the [k8s image the [k8s image
repo](https://github.com/kubernetes/k8s.io/tree/main/k8s.gcr.io/images/k8s-staging-sig-storage), repo](https://github.com/kubernetes/k8s.io/tree/HEAD/k8s.gcr.io/images/k8s-staging-sig-storage),
run `./generate.sh > images.yaml`, and send a PR with the updated images. run `./generate.sh > images.yaml`, and send a PR with the updated images.
Once merged, the image promoter will copy the images from staging to prod. Once merged, the image promoter will copy the images from staging to prod.
1. Update [kubernetes-csi/docs](https://github.com/kubernetes-csi/docs) sidecar 1. Update [kubernetes-csi/docs](https://github.com/kubernetes-csi/docs) sidecar
and feature pages with the new released version. and feature pages with the new released version.
1. After all the sidecars have been released, update 1. After all the sidecars have been released, update
CSI hostpath driver with the new sidecars in the [CSI repo](https://github.com/kubernetes-csi/csi-driver-host-path/tree/master/deploy) CSI hostpath driver with the new sidecars in the [CSI repo](https://github.com/kubernetes-csi/csi-driver-host-path/tree/HEAD/deploy)
and [k/k and [k/k
in-tree](https://github.com/kubernetes/kubernetes/tree/master/test/e2e/testing-manifests/storage-csi/hostpath/hostpath) in-tree](https://github.com/kubernetes/kubernetes/tree/HEAD/test/e2e/testing-manifests/storage-csi/hostpath/hostpath)
## Adding support for a new Kubernetes release ## Adding support for a new Kubernetes release
@@ -134,7 +129,7 @@ naming convention `<hostpath-deployment-version>-on-<kubernetes-version>`.
1. Once all sidecars for the new Kubernetes release are released, 1. Once all sidecars for the new Kubernetes release are released,
either bump the version number of the images in the existing either bump the version number of the images in the existing
[csi-driver-host-path [csi-driver-host-path
deployments](https://github.com/kubernetes-csi/csi-driver-host-path/tree/master/deploy) deployments](https://github.com/kubernetes-csi/csi-driver-host-path/tree/HEAD/deploy)
and/or create a new deployment, depending on what Kubernetes and/or create a new deployment, depending on what Kubernetes
release an updated sidecar is compatible with. If no new deployment release an updated sidecar is compatible with. If no new deployment
is needed, then add a symlink to document that there intentionally is needed, then add a symlink to document that there intentionally

14
release-tools/build.make
View File

@@ -21,6 +21,10 @@ SHELL := /bin/bash
# set in main Makefile of a repository. # set in main Makefile of a repository.
# CMDS= # CMDS=
# Normally, commands are expected in "cmd". That can be changed for a
# repository to something else by setting CMDS_DIR before including build.make.
CMDS_DIR ?= cmd
# This is the default. It can be overridden in the main Makefile after # This is the default. It can be overridden in the main Makefile after
# including build.make. # including build.make.
REGISTRY_NAME?=quay.io/k8scsi REGISTRY_NAME?=quay.io/k8scsi
@@ -90,7 +94,7 @@ $(CMDS:%=build-%): build-%: check-go-version-go
if ! [ $${#os_arch_seen_pre} = $${#os_arch_seen} ]; then \ if ! [ $${#os_arch_seen_pre} = $${#os_arch_seen} ]; then \
continue; \ continue; \
fi; \ fi; \
if ! (set -x; CGO_ENABLED=0 GOOS="$$os" GOARCH="$$arch" go build $(GOFLAGS_VENDOR) -a -ldflags '$(FULL_LDFLAGS)' -o "./bin/$*$$suffix" ./cmd/$*); then \ if ! (set -x; cd ./$(CMDS_DIR)/$* && CGO_ENABLED=0 GOOS="$$os" GOARCH="$$arch" go build $(GOFLAGS_VENDOR) -a -ldflags '$(FULL_LDFLAGS)' -o "$(abspath ./bin)/$*$$suffix" .); then \
echo "Building $* for GOOS=$$os GOARCH=$$arch failed, see error(s) above."; \ echo "Building $* for GOOS=$$os GOARCH=$$arch failed, see error(s) above."; \
exit 1; \ exit 1; \
fi; \ fi; \
@@ -98,7 +102,7 @@ $(CMDS:%=build-%): build-%: check-go-version-go
done done
$(CMDS:%=container-%): container-%: build-% $(CMDS:%=container-%): container-%: build-%
docker build -t $*:latest -f $(shell if [ -e ./cmd/$*/Dockerfile ]; then echo ./cmd/$*/Dockerfile; else echo Dockerfile; fi) --label revision=$(REV) . docker build -t $*:latest -f $(shell if [ -e ./$(CMDS_DIR)/$*/Dockerfile ]; then echo ./$(CMDS_DIR)/$*/Dockerfile; else echo Dockerfile; fi) --label revision=$(REV) .
$(CMDS:%=push-%): push-%: container-% $(CMDS:%=push-%): push-%: container-%
set -ex; \ set -ex; \
@@ -133,7 +137,7 @@ DOCKER_BUILDX_CREATE_ARGS ?=
# This target builds a multiarch image for one command using Moby BuildKit builder toolkit. # This target builds a multiarch image for one command using Moby BuildKit builder toolkit.
# Docker Buildx is included in Docker 19.03. # Docker Buildx is included in Docker 19.03.
# #
# ./cmd/<command>/Dockerfile[.Windows] is used if found, otherwise Dockerfile[.Windows]. # ./$(CMDS_DIR)/<command>/Dockerfile[.Windows] is used if found, otherwise Dockerfile[.Windows].
# It is currently optional: if no such file exists, Windows images are not included, # It is currently optional: if no such file exists, Windows images are not included,
# even when Windows is listed in BUILD_PLATFORMS. That way, projects can test that # even when Windows is listed in BUILD_PLATFORMS. That way, projects can test that
# Windows binaries can be built before adding a Dockerfile for it. # Windows binaries can be built before adding a Dockerfile for it.
@@ -146,8 +150,8 @@ $(CMDS:%=push-multiarch-%): push-multiarch-%: check-pull-base-ref build-%
export DOCKER_CLI_EXPERIMENTAL=enabled; \ export DOCKER_CLI_EXPERIMENTAL=enabled; \
docker buildx create $(DOCKER_BUILDX_CREATE_ARGS) --use --name multiarchimage-buildertest; \ docker buildx create $(DOCKER_BUILDX_CREATE_ARGS) --use --name multiarchimage-buildertest; \
trap "docker buildx rm multiarchimage-buildertest" EXIT; \ trap "docker buildx rm multiarchimage-buildertest" EXIT; \
dockerfile_linux=$$(if [ -e ./cmd/$*/Dockerfile ]; then echo ./cmd/$*/Dockerfile; else echo Dockerfile; fi); \ dockerfile_linux=$$(if [ -e ./$(CMDS_DIR)/$*/Dockerfile ]; then echo ./$(CMDS_DIR)/$*/Dockerfile; else echo Dockerfile; fi); \
dockerfile_windows=$$(if [ -e ./cmd/$*/Dockerfile.Windows ]; then echo ./cmd/$*/Dockerfile.Windows; else echo Dockerfile.Windows; fi); \ dockerfile_windows=$$(if [ -e ./$(CMDS_DIR)/$*/Dockerfile.Windows ]; then echo ./$(CMDS_DIR)/$*/Dockerfile.Windows; else echo Dockerfile.Windows; fi); \
if [ '$(BUILD_PLATFORMS)' ]; then build_platforms='$(BUILD_PLATFORMS)'; else build_platforms="linux amd64"; fi; \ if [ '$(BUILD_PLATFORMS)' ]; then build_platforms='$(BUILD_PLATFORMS)'; else build_platforms="linux amd64"; fi; \
if ! [ -f "$$dockerfile_windows" ]; then \ if ! [ -f "$$dockerfile_windows" ]; then \
build_platforms="$$(echo "$$build_platforms" | sed -e 's/windows *[^ ]* *.exe *[^ ]* *[^ ]*//g' -e 's/; *;/;/g' -e 's/;[ ]*$$//')"; \ build_platforms="$$(echo "$$build_platforms" | sed -e 's/windows *[^ ]* *.exe *[^ ]* *[^ ]*//g' -e 's/; *;/;/g' -e 's/;[ ]*$$//')"; \

4
release-tools/cloudbuild.yaml
View File

@@ -10,10 +10,10 @@
# because binaries will get built for different architectures and then # because binaries will get built for different architectures and then
# get copied from the built host into the container image # get copied from the built host into the container image
# #
# See https://github.com/kubernetes/test-infra/blob/master/config/jobs/image-pushing/README.md # See https://github.com/kubernetes/test-infra/blob/HEAD/config/jobs/image-pushing/README.md
# for more details on image pushing process in Kubernetes. # for more details on image pushing process in Kubernetes.
# #
# To promote release images, see https://github.com/kubernetes/k8s.io/tree/main/k8s.gcr.io/images/k8s-staging-sig-storage. # To promote release images, see https://github.com/kubernetes/k8s.io/tree/HEAD/k8s.gcr.io/images/k8s-staging-sig-storage.
# This must be specified in seconds. If omitted, defaults to 600s (10 mins). # This must be specified in seconds. If omitted, defaults to 600s (10 mins).
# Building three images in external-snapshotter takes roughly half an hour, # Building three images in external-snapshotter takes roughly half an hour,

38
release-tools/prow.sh
View File

@@ -78,7 +78,7 @@ version_to_git () {
# the list of windows versions was matched from: # the list of windows versions was matched from:
# - https://hub.docker.com/_/microsoft-windows-nanoserver # - https://hub.docker.com/_/microsoft-windows-nanoserver
# - https://hub.docker.com/_/microsoft-windows-servercore # - https://hub.docker.com/_/microsoft-windows-servercore
configvar CSI_PROW_BUILD_PLATFORMS "linux amd64; linux ppc64le -ppc64le; linux s390x -s390x; linux arm64 -arm64; windows amd64 .exe nanoserver:1809 servercore:ltsc2019; windows amd64 .exe nanoserver:1909 servercore:1909; windows amd64 .exe nanoserver:2004 servercore:2004; windows amd64 .exe nanoserver:20H2 servercore:20H2" "Go target platforms (= GOOS + GOARCH) and file suffix of the resulting binaries" configvar CSI_PROW_BUILD_PLATFORMS "linux amd64; linux ppc64le -ppc64le; linux s390x -s390x; linux arm -arm; linux arm64 -arm64; windows amd64 .exe nanoserver:1809 servercore:ltsc2019; windows amd64 .exe nanoserver:1909 servercore:1909; windows amd64 .exe nanoserver:2004 servercore:2004; windows amd64 .exe nanoserver:20H2 servercore:20H2; windows amd64 .exe nanoserver:ltsc2022 servercore:ltsc2022" "Go target platforms (= GOOS + GOARCH) and file suffix of the resulting binaries"
# If we have a vendor directory, then use it. We must be careful to only # If we have a vendor directory, then use it. We must be careful to only
# use this for "make" invocations inside the project's repo itself because # use this for "make" invocations inside the project's repo itself because
@@ -138,7 +138,7 @@ kind_version_default () {
latest|master) latest|master)
echo main;; echo main;;
*) *)
echo v0.11.0;; echo v0.11.1;;
esac esac
} }
@@ -149,14 +149,15 @@ configvar CSI_PROW_KIND_VERSION "$(kind_version_default)" "kind"
# kind images to use. Must match the kind version. # kind images to use. Must match the kind version.
# The release notes of each kind release list the supported images. # The release notes of each kind release list the supported images.
configvar CSI_PROW_KIND_IMAGES "kindest/node:v1.21.1@sha256:fae9a58f17f18f06aeac9772ca8b5ac680ebbed985e266f711d936e91d113bad configvar CSI_PROW_KIND_IMAGES "kindest/node:v1.22.0@sha256:b8bda84bb3a190e6e028b1760d277454a72267a5454b57db34437c34a588d047
kindest/node:v1.20.7@sha256:e645428988191fc824529fd0bb5c94244c12401cf5f5ea3bd875eb0a787f0fe9 kindest/node:v1.21.1@sha256:69860bda5563ac81e3c0057d654b5253219618a22ec3a346306239bba8cfa1a6
kindest/node:v1.19.11@sha256:7664f21f9cb6ba2264437de0eb3fe99f201db7a3ac72329547ec4373ba5f5911 kindest/node:v1.20.7@sha256:cbeaf907fc78ac97ce7b625e4bf0de16e3ea725daf6b04f930bd14c67c671ff9
kindest/node:v1.18.19@sha256:530378628c7c518503ade70b1df698b5de5585dcdba4f349328d986b8849b1ee kindest/node:v1.19.11@sha256:07db187ae84b4b7de440a73886f008cf903fcf5764ba8106a9fd5243d6f32729
kindest/node:v1.17.17@sha256:c581fbf67f720f70aaabc74b44c2332cc753df262b6c0bca5d26338492470c17 kindest/node:v1.18.19@sha256:7af1492e19b3192a79f606e43c35fb741e520d195f96399284515f077b3b622c
kindest/node:v1.16.15@sha256:430c03034cd856c1f1415d3e37faf35a3ea9c5aaa2812117b79e6903d1fc9651 kindest/node:v1.17.17@sha256:66f1d0d91a88b8a001811e2f1054af60eef3b669a9a74f9b6db871f2f1eeed00
kindest/node:v1.15.12@sha256:8d575f056493c7778935dd855ded0e95c48cb2fab90825792e8fc9af61536bf9 kindest/node:v1.16.15@sha256:83067ed51bf2a3395b24687094e283a7c7c865ccc12a8b1d7aa673ba0c5e8861
kindest/node:v1.14.10@sha256:6033e04bcfca7c5f2a9c4ce77551e1abf385bcd2709932ec2f6a9c8c0aff6d4f" "kind images" kindest/node:v1.15.12@sha256:b920920e1eda689d9936dfcf7332701e80be12566999152626b2c9d730397a95
kindest/node:v1.14.10@sha256:f8a66ef82822ab4f7569e91a5bccaf27bceee135c1457c512e54de8c6f7219f8" "kind images"
# By default, this script tests sidecars with the CSI hostpath driver, # By default, this script tests sidecars with the CSI hostpath driver,
# using the install_csi_driver function. That function depends on # using the install_csi_driver function. That function depends on
@@ -292,7 +293,7 @@ tests_need_alpha_cluster () {
tests_enabled "parallel-alpha" "serial-alpha" tests_enabled "parallel-alpha" "serial-alpha"
} }
# Enabling mock tests adds the "CSI mock volume" tests from https://github.com/kubernetes/kubernetes/blob/master/test/e2e/storage/csi_mock_volume.go # Enabling mock tests adds the "CSI mock volume" tests from https://github.com/kubernetes/kubernetes/blob/HEAD/test/e2e/storage/csi_mock_volume.go
# to the e2e.test invocations (serial, parallel, and the corresponding alpha variants). # to the e2e.test invocations (serial, parallel, and the corresponding alpha variants).
# When testing canary images, those get used instead of the images specified # When testing canary images, those get used instead of the images specified
# in the e2e.test's normal YAML files. # in the e2e.test's normal YAML files.
@@ -795,7 +796,7 @@ install_snapshot_controller() {
kind load docker-image --name csi-prow ${NEW_IMG} || die "could not load the snapshot-controller:csiprow image into the kind cluster" kind load docker-image --name csi-prow ${NEW_IMG} || die "could not load the snapshot-controller:csiprow image into the kind cluster"
# deploy snapshot-controller # deploy snapshot-controller
echo "Deploying snapshot-controller" echo "Deploying snapshot-controller from ${SNAPSHOT_CONTROLLER_YAML} with $NEW_IMG."
# Replace image in SNAPSHOT_CONTROLLER_YAML with snapshot-controller:csiprow and deploy # Replace image in SNAPSHOT_CONTROLLER_YAML with snapshot-controller:csiprow and deploy
# NOTE: This logic is similar to the logic here: # NOTE: This logic is similar to the logic here:
# https://github.com/kubernetes-csi/csi-driver-host-path/blob/v1.4.0/deploy/util/deploy-hostpath.sh#L155 # https://github.com/kubernetes-csi/csi-driver-host-path/blob/v1.4.0/deploy/util/deploy-hostpath.sh#L155
@@ -832,8 +833,19 @@ install_snapshot_controller() {
echo "$modified" echo "$modified"
exit 1 exit 1
fi fi
echo "kubectl apply -f ${SNAPSHOT_CONTROLLER_YAML}(modified)"
done done
elif [ "${CSI_PROW_DRIVER_CANARY}" = "canary" ]; then
echo "Deploying snapshot-controller from ${SNAPSHOT_CONTROLLER_YAML} with canary images."
yaml="$(kubectl apply --dry-run=client -o yaml -f "$SNAPSHOT_CONTROLLER_YAML")"
# Ignore: See if you can use ${variable//search/replace} instead.
# shellcheck disable=SC2001
modified="$(echo "$yaml" | sed -e "s;image: .*/\([^/:]*\):.*;image: ${CSI_PROW_DRIVER_CANARY_REGISTRY}/\1:canary;")"
diff <(echo "$yaml") <(echo "$modified")
if ! echo "$modified" | kubectl apply -f -; then
echo "modified version of $SNAPSHOT_CONTROLLER_YAML:"
echo "$modified"
exit 1
fi
else else
echo "kubectl apply -f $SNAPSHOT_CONTROLLER_YAML" echo "kubectl apply -f $SNAPSHOT_CONTROLLER_YAML"
kubectl apply -f "$SNAPSHOT_CONTROLLER_YAML" kubectl apply -f "$SNAPSHOT_CONTROLLER_YAML"