Skip to content

Commit 52be7ed

Browse files
tkatilatkatila
committed
Add tolerations support to operator and plugin CRDs
Signed-off-by: Tuomas Katila <tuomas.katila@intel.com>
1 parent 7d00cf0 commit 52be7ed

31 files changed

+626
-4
lines changed

‎deployments/operator/crd/bases/deviceplugin.intel.com_dlbdeviceplugins.yaml‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -72,6 +72,47 @@ spec:
7272
description: NodeSelector provides a simple way to constrain device
7373
plugin pods to nodes with particular labels.
7474
type: object
75+
tolerations:
76+
description: Specialized nodes (e.g., with accelerators) can be Tainted
77+
to make sure unwanted pods are not scheduled on them. Tolerations
78+
can be set for the plugin pod to neutralize the Taint.
79+
items:
80+
description: |-
81+
The pod this Toleration is attached to tolerates any taint that matches
82+
the triple <key,value,effect> using the matching operator <operator>.
83+
properties:
84+
effect:
85+
description: |-
86+
Effect indicates the taint effect to match. Empty means match all taint effects.
87+
When specified, allowed values are NoSchedule, PreferNoSchedule and NoExecute.
88+
type: string
89+
key:
90+
description: |-
91+
Key is the taint key that the toleration applies to. Empty means match all taint keys.
92+
If the key is empty, operator must be Exists; this combination means to match all values and all keys.
93+
type: string
94+
operator:
95+
description: |-
96+
Operator represents a key's relationship to the value.
97+
Valid operators are Exists and Equal. Defaults to Equal.
98+
Exists is equivalent to wildcard for value, so that a pod can
99+
tolerate all taints of a particular category.
100+
type: string
101+
tolerationSeconds:
102+
description: |-
103+
TolerationSeconds represents the period of time the toleration (which must be
104+
of effect NoExecute, otherwise this field is ignored) tolerates the taint. By default,
105+
it is not set, which means tolerate the taint forever (do not evict). Zero and
106+
negative values will be treated as 0 (evict immediately) by the system.
107+
format: int64
108+
type: integer
109+
value:
110+
description: |-
111+
Value is the taint value the toleration matches to.
112+
If the operator is Exists, the value should be empty, otherwise just a regular string.
113+
type: string
114+
type: object
115+
type: array
75116
type: object
76117
status:
77118
description: DlbDevicePluginStatus defines the observed state of DlbDevicePlugin.

‎deployments/operator/crd/bases/deviceplugin.intel.com_dsadeviceplugins.yaml‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -82,6 +82,47 @@ spec:
8282
the same DSA device.
8383
minimum: 1
8484
type: integer
85+
tolerations:
86+
description: Specialized nodes (e.g., with accelerators) can be Tainted
87+
to make sure unwanted pods are not scheduled on them. Tolerations
88+
can be set for the plugin pod to neutralize the Taint.
89+
items:
90+
description: |-
91+
The pod this Toleration is attached to tolerates any taint that matches
92+
the triple <key,value,effect> using the matching operator <operator>.
93+
properties:
94+
effect:
95+
description: |-
96+
Effect indicates the taint effect to match. Empty means match all taint effects.
97+
When specified, allowed values are NoSchedule, PreferNoSchedule and NoExecute.
98+
type: string
99+
key:
100+
description: |-
101+
Key is the taint key that the toleration applies to. Empty means match all taint keys.
102+
If the key is empty, operator must be Exists; this combination means to match all values and all keys.
103+
type: string
104+
operator:
105+
description: |-
106+
Operator represents a key's relationship to the value.
107+
Valid operators are Exists and Equal. Defaults to Equal.
108+
Exists is equivalent to wildcard for value, so that a pod can
109+
tolerate all taints of a particular category.
110+
type: string
111+
tolerationSeconds:
112+
description: |-
113+
TolerationSeconds represents the period of time the toleration (which must be
114+
of effect NoExecute, otherwise this field is ignored) tolerates the taint. By default,
115+
it is not set, which means tolerate the taint forever (do not evict). Zero and
116+
negative values will be treated as 0 (evict immediately) by the system.
117+
format: int64
118+
type: integer
119+
value:
120+
description: |-
121+
Value is the taint value the toleration matches to.
122+
If the operator is Exists, the value should be empty, otherwise just a regular string.
123+
type: string
124+
type: object
125+
type: array
85126
type: object
86127
status:
87128
description: DsaDevicePluginStatus defines the observed state of DsaDevicePlugin.

‎deployments/operator/crd/bases/deviceplugin.intel.com_fpgadeviceplugins.yaml‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,47 @@ spec:
7979
description: NodeSelector provides a simple way to constrain device
8080
plugin pods to nodes with particular labels.
8181
type: object
82+
tolerations:
83+
description: Specialized nodes (e.g., with accelerators) can be Tainted
84+
to make sure unwanted pods are not scheduled on them. Tolerations
85+
can be set for the plugin pod to neutralize the Taint.
86+
items:
87+
description: |-
88+
The pod this Toleration is attached to tolerates any taint that matches
89+
the triple <key,value,effect> using the matching operator <operator>.
90+
properties:
91+
effect:
92+
description: |-
93+
Effect indicates the taint effect to match. Empty means match all taint effects.
94+
When specified, allowed values are NoSchedule, PreferNoSchedule and NoExecute.
95+
type: string
96+
key:
97+
description: |-
98+
Key is the taint key that the toleration applies to. Empty means match all taint keys.
99+
If the key is empty, operator must be Exists; this combination means to match all values and all keys.
100+
type: string
101+
operator:
102+
description: |-
103+
Operator represents a key's relationship to the value.
104+
Valid operators are Exists and Equal. Defaults to Equal.
105+
Exists is equivalent to wildcard for value, so that a pod can
106+
tolerate all taints of a particular category.
107+
type: string
108+
tolerationSeconds:
109+
description: |-
110+
TolerationSeconds represents the period of time the toleration (which must be
111+
of effect NoExecute, otherwise this field is ignored) tolerates the taint. By default,
112+
it is not set, which means tolerate the taint forever (do not evict). Zero and
113+
negative values will be treated as 0 (evict immediately) by the system.
114+
format: int64
115+
type: integer
116+
value:
117+
description: |-
118+
Value is the taint value the toleration matches to.
119+
If the operator is Exists, the value should be empty, otherwise just a regular string.
120+
type: string
121+
type: object
122+
type: array
82123
type: object
83124
status:
84125
description: FpgaDevicePluginStatus defines the observed state of FpgaDevicePlugin.

‎deployments/operator/crd/bases/deviceplugin.intel.com_gpudeviceplugins.yaml‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -96,6 +96,47 @@ spec:
9696
the same GPU device.
9797
minimum: 1
9898
type: integer
99+
tolerations:
100+
description: Specialized nodes (e.g., with accelerators) can be Tainted
101+
to make sure unwanted pods are not scheduled on them. Tolerations
102+
can be set for the plugin pod to neutralize the Taint.
103+
items:
104+
description: |-
105+
The pod this Toleration is attached to tolerates any taint that matches
106+
the triple <key,value,effect> using the matching operator <operator>.
107+
properties:
108+
effect:
109+
description: |-
110+
Effect indicates the taint effect to match. Empty means match all taint effects.
111+
When specified, allowed values are NoSchedule, PreferNoSchedule and NoExecute.
112+
type: string
113+
key:
114+
description: |-
115+
Key is the taint key that the toleration applies to. Empty means match all taint keys.
116+
If the key is empty, operator must be Exists; this combination means to match all values and all keys.
117+
type: string
118+
operator:
119+
description: |-
120+
Operator represents a key's relationship to the value.
121+
Valid operators are Exists and Equal. Defaults to Equal.
122+
Exists is equivalent to wildcard for value, so that a pod can
123+
tolerate all taints of a particular category.
124+
type: string
125+
tolerationSeconds:
126+
description: |-
127+
TolerationSeconds represents the period of time the toleration (which must be
128+
of effect NoExecute, otherwise this field is ignored) tolerates the taint. By default,
129+
it is not set, which means tolerate the taint forever (do not evict). Zero and
130+
negative values will be treated as 0 (evict immediately) by the system.
131+
format: int64
132+
type: integer
133+
value:
134+
description: |-
135+
Value is the taint value the toleration matches to.
136+
If the operator is Exists, the value should be empty, otherwise just a regular string.
137+
type: string
138+
type: object
139+
type: array
99140
type: object
100141
status:
101142
description: |-

‎deployments/operator/crd/bases/deviceplugin.intel.com_iaadeviceplugins.yaml‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -81,6 +81,47 @@ spec:
8181
the same IAA device.
8282
minimum: 1
8383
type: integer
84+
tolerations:
85+
description: Specialized nodes (e.g., with accelerators) can be Tainted
86+
to make sure unwanted pods are not scheduled on them. Tolerations
87+
can be set for the plugin pod to neutralize the Taint.
88+
items:
89+
description: |-
90+
The pod this Toleration is attached to tolerates any taint that matches
91+
the triple <key,value,effect> using the matching operator <operator>.
92+
properties:
93+
effect:
94+
description: |-
95+
Effect indicates the taint effect to match. Empty means match all taint effects.
96+
When specified, allowed values are NoSchedule, PreferNoSchedule and NoExecute.
97+
type: string
98+
key:
99+
description: |-
100+
Key is the taint key that the toleration applies to. Empty means match all taint keys.
101+
If the key is empty, operator must be Exists; this combination means to match all values and all keys.
102+
type: string
103+
operator:
104+
description: |-
105+
Operator represents a key's relationship to the value.
106+
Valid operators are Exists and Equal. Defaults to Equal.
107+
Exists is equivalent to wildcard for value, so that a pod can
108+
tolerate all taints of a particular category.
109+
type: string
110+
tolerationSeconds:
111+
description: |-
112+
TolerationSeconds represents the period of time the toleration (which must be
113+
of effect NoExecute, otherwise this field is ignored) tolerates the taint. By default,
114+
it is not set, which means tolerate the taint forever (do not evict). Zero and
115+
negative values will be treated as 0 (evict immediately) by the system.
116+
format: int64
117+
type: integer
118+
value:
119+
description: |-
120+
Value is the taint value the toleration matches to.
121+
If the operator is Exists, the value should be empty, otherwise just a regular string.
122+
type: string
123+
type: object
124+
type: array
84125
type: object
85126
status:
86127
description: IaaDevicePluginStatus defines the observed state of IaaDevicePlugin.

‎deployments/operator/crd/bases/deviceplugin.intel.com_qatdeviceplugins.yaml‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -111,6 +111,47 @@ spec:
111111
description: ProvisioningConfig is a ConfigMap used to pass the configuration
112112
of QAT devices into qat initcontainer.
113113
type: string
114+
tolerations:
115+
description: Specialized nodes (e.g., with accelerators) can be Tainted
116+
to make sure unwanted pods are not scheduled on them. Tolerations
117+
can be set for the plugin pod to neutralize the Taint.
118+
items:
119+
description: |-
120+
The pod this Toleration is attached to tolerates any taint that matches
121+
the triple <key,value,effect> using the matching operator <operator>.
122+
properties:
123+
effect:
124+
description: |-
125+
Effect indicates the taint effect to match. Empty means match all taint effects.
126+
When specified, allowed values are NoSchedule, PreferNoSchedule and NoExecute.
127+
type: string
128+
key:
129+
description: |-
130+
Key is the taint key that the toleration applies to. Empty means match all taint keys.
131+
If the key is empty, operator must be Exists; this combination means to match all values and all keys.
132+
type: string
133+
operator:
134+
description: |-
135+
Operator represents a key's relationship to the value.
136+
Valid operators are Exists and Equal. Defaults to Equal.
137+
Exists is equivalent to wildcard for value, so that a pod can
138+
tolerate all taints of a particular category.
139+
type: string
140+
tolerationSeconds:
141+
description: |-
142+
TolerationSeconds represents the period of time the toleration (which must be
143+
of effect NoExecute, otherwise this field is ignored) tolerates the taint. By default,
144+
it is not set, which means tolerate the taint forever (do not evict). Zero and
145+
negative values will be treated as 0 (evict immediately) by the system.
146+
format: int64
147+
type: integer
148+
value:
149+
description: |-
150+
Value is the taint value the toleration matches to.
151+
If the operator is Exists, the value should be empty, otherwise just a regular string.
152+
type: string
153+
type: object
154+
type: array
114155
type: object
115156
status:
116157
description: |-

‎deployments/operator/crd/bases/deviceplugin.intel.com_sgxdeviceplugins.yaml‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -83,6 +83,47 @@ spec:
8383
the same SGX provision device.
8484
minimum: 1
8585
type: integer
86+
tolerations:
87+
description: Specialized nodes (e.g., with accelerators) can be Tainted
88+
to make sure unwanted pods are not scheduled on them. Tolerations
89+
can be set for the plugin pod to neutralize the Taint.
90+
items:
91+
description: |-
92+
The pod this Toleration is attached to tolerates any taint that matches
93+
the triple <key,value,effect> using the matching operator <operator>.
94+
properties:
95+
effect:
96+
description: |-
97+
Effect indicates the taint effect to match. Empty means match all taint effects.
98+
When specified, allowed values are NoSchedule, PreferNoSchedule and NoExecute.
99+
type: string
100+
key:
101+
description: |-
102+
Key is the taint key that the toleration applies to. Empty means match all taint keys.
103+
If the key is empty, operator must be Exists; this combination means to match all values and all keys.
104+
type: string
105+
operator:
106+
description: |-
107+
Operator represents a key's relationship to the value.
108+
Valid operators are Exists and Equal. Defaults to Equal.
109+
Exists is equivalent to wildcard for value, so that a pod can
110+
tolerate all taints of a particular category.
111+
type: string
112+
tolerationSeconds:
113+
description: |-
114+
TolerationSeconds represents the period of time the toleration (which must be
115+
of effect NoExecute, otherwise this field is ignored) tolerates the taint. By default,
116+
it is not set, which means tolerate the taint forever (do not evict). Zero and
117+
negative values will be treated as 0 (evict immediately) by the system.
118+
format: int64
119+
type: integer
120+
value:
121+
description: |-
122+
Value is the taint value the toleration matches to.
123+
If the operator is Exists, the value should be empty, otherwise just a regular string.
124+
type: string
125+
type: object
126+
type: array
86127
type: object
87128
status:
88129
description: SgxDevicePluginStatus defines the observed state of SgxDevicePlugin.

‎pkg/apis/deviceplugin/v1/dlbdeviceplugin_types.go‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -34,6 +34,9 @@ type DlbDevicePluginSpec struct {
3434
// InitImage is a container image with a script that initializes devices.
3535
InitImage string `json:"initImage,omitempty"`
3636

37+
// Specialized nodes (e.g., with accelerators) can be Tainted to make sure unwanted pods are not scheduled on them. Tolerations can be set for the plugin pod to neutralize the Taint.
38+
Tolerations []v1.Toleration `json:"tolerations,omitempty"`
39+
3740
// LogLevel sets the plugin's log level.
3841
// +kubebuilder:validation:Minimum=0
3942
LogLevel int `json:"logLevel,omitempty"`

0 commit comments

Comments
 (0)