dd51ac2844
The kube-sriov-device-plugin pod will mount HostPath:'/var/lib/kubelet'
If a pod mounting with rbd PVC is already running when
kube-sriov-device-plugin pod starts up, the kube-sriov-device-plugin
pod will refer to the rbd mountpoint which is under HostPath:
'/var/lib/kubelet'.
Even if the rbd is unmounted from the mountpoint on the host, the pod
will keep referring to it in its namespace.
So kubelet can't unmap the rbd and will fail to mount the volume when
pod with rbd PVC is recreated.
The kube-sriov-device-plugin doesn't need to use '/var/lib/kubelet' as
mountpath because its internal device socket is actually at
/var/lib/kubelet/device-plugins/.
Changing the kube-sriov-device-plugin mountpath to a less broad path
will preserve the rbd PVC mount point under /var/lib/kubelet/.
Test plan
PASS Installed AIO-SX
create SRIOV interfaces
create stateful pod with rbd PVC
delete kube-sriov-device-plugin
delete stateful pod
A new stateful pod will automatically be created
check if stateful pod was not stuck
confirm if stateful pod could mount the volume
PASS Create a SRIOV NetworkAttachmentDefinition
Launch a POD using the SRIOV interface
check if POD is running and if POD has connectivity.
PASS Upgrades testing (partial) - verified controller-1 is upgraded and device plugin/pod working with new location.
Closes-Bug: #2007596
Signed-off-by: Fabiano Mercer <fabiano.correamercer@windriver.com>
Change-Id: I7ef43a1c0ac4f7f0af1a366c298b4c1029d3e915
111 lines
3.1 KiB
Django/Jinja
111 lines
3.1 KiB
Django/Jinja
# SRIOV device CNI plugin version 3.4.0
|
|
# Based on:
|
|
# https://github.com/k8snetworkplumbingwg/sriov-network-device-plugin/blob/v3.4.0/deployments/k8s-v1.16/sriovdp-daemonset.yaml
|
|
#
|
|
# This file is licensed under Apache 2.0. You can obtain a copy of the license at:
|
|
# https://github.com/k8snetworkplumbingwg/sriov-network-device-plugin/blob/v3.4.0/LICENSE
|
|
#
|
|
# The following modifications have been made:
|
|
#
|
|
# - A nodeSelector of 'sriovdp' has been added to ensure the sriov device plugin
|
|
# pods only run on appropriately labelled nodes.
|
|
# - The config hostPath is explicitly set to 'File'
|
|
# - The daemonset is modified to tolerate all NoSchedule taints
|
|
# - An updateStrategy was added to the DaemonSet spec to allow controlled template
|
|
# updates of pods with "kubectl apply".
|
|
# - The image is set to a stable starlingX version
|
|
# - The default configMap is not used. Rather, a hostPath to the config.json file
|
|
# is used, as resources are populated and based on datanetwork names.
|
|
# - For k8s 1.19, the matchLabels are the same as the k8s 1.18 labels to
|
|
# allow a rolling update to succeed.
|
|
# - Set CPU requests to 0
|
|
#
|
|
# Copyright (c) 2022 Wind River Systems, Inc.
|
|
#
|
|
# SPDX-License-Identifier: Apache-2.0
|
|
#
|
|
---
|
|
apiVersion: v1
|
|
kind: ServiceAccount
|
|
metadata:
|
|
name: sriov-device-plugin
|
|
namespace: kube-system
|
|
|
|
---
|
|
apiVersion: apps/v1
|
|
kind: DaemonSet
|
|
metadata:
|
|
name: kube-sriov-device-plugin-amd64
|
|
namespace: kube-system
|
|
labels:
|
|
tier: node
|
|
app: sriovdp
|
|
spec:
|
|
selector:
|
|
matchLabels:
|
|
tier: node
|
|
app: sriovdp
|
|
updateStrategy:
|
|
type: RollingUpdate
|
|
rollingUpdate:
|
|
maxUnavailable: 1
|
|
template:
|
|
metadata:
|
|
labels:
|
|
name: sriov-device-plugin
|
|
tier: node
|
|
app: sriovdp
|
|
spec:
|
|
hostNetwork: true
|
|
nodeSelector:
|
|
beta.kubernetes.io/arch: amd64
|
|
sriovdp: enabled
|
|
tolerations:
|
|
- operator: Exists
|
|
effect: NoSchedule
|
|
serviceAccountName: sriov-device-plugin
|
|
imagePullSecrets:
|
|
- name: registry-local-secret
|
|
containers:
|
|
- name: kube-sriovdp
|
|
image: "{{ local_registry }}/{{ sriov_network_device_img }}"
|
|
args:
|
|
- --log-dir=sriovdp
|
|
- --log-level=10
|
|
securityContext:
|
|
privileged: true
|
|
resources:
|
|
requests:
|
|
cpu: "0"
|
|
memory: "40Mi"
|
|
limits:
|
|
cpu: 1
|
|
memory: "200Mi"
|
|
volumeMounts:
|
|
- name: devicesock
|
|
mountPath: /var/lib/kubelet/device-plugins/
|
|
readOnly: false
|
|
- name: log
|
|
mountPath: /var/log
|
|
- name: config
|
|
mountPath: /etc/pcidp/config.json
|
|
readOnly: true
|
|
- name: device-info
|
|
mountPath: /var/run/k8s.cni.cncf.io/devinfo/dp
|
|
volumes:
|
|
- name: devicesock
|
|
hostPath:
|
|
path: /var/lib/kubelet/device-plugins/
|
|
- name: log
|
|
hostPath:
|
|
path: /var/log
|
|
- name: device-info
|
|
hostPath:
|
|
path: /var/run/k8s.cni.cncf.io/devinfo/dp
|
|
type: DirectoryOrCreate
|
|
- name: config
|
|
hostPath:
|
|
path: /etc/pcidp/config.json
|
|
type: File
|
|
|