Configuring ExoNodeClass and ExoNodePool
This guide walks you through configuring ExoNodeClass
and ExoNodePool
for Exostellar Karpenter (version 2.x).
Overview
Before submitting workloads to your cluster, two primary components must be configured:
ExoNodeClass
defines settings relating to AWS and xcompute profiles.ExoNodePool
enables features and defines sets of nodes that Exostellar Karpenter can create.
Sample Configuration
apiVersion: karpenter.k8s.exo/v1alpha1
kind: ExoNodeClass
metadata:
name: class-a
spec:
xcompute:
nodeImageName: "k8s-131"
version: "xspot-3.2.2"
securityGroupSelectorTerms:
- tags:
installer: x-install-<cluster-name> # Update with your cluster name.
subnetSelectorTerms:
- id: "subnet-083b46ca34c707209"
---
apiVersion: exokarpenter.sh/v1beta1
kind: ExoNodePool
metadata:
name: pool-a
spec:
disruption:
consolidateAfter: 1m
consolidationPolicy: WhenEmpty
template:
spec:
nodeClassRef:
kind: ExoNodeClass
name: class-a
requirements:
- key: "karpenter.k8s.exostellar/instance-family"
operator: In
values: ["m5"]
exostellar:
workloadOptimizer:
enabled: true
infrastructureOptimizer:
enabled: true
taints:
- key: exokarpenter.sh/x-compute
effect: NoSchedule
value: "true"
ExoNodeClass CRD
Field Name | Field Description | Default Value |
---|---|---|
subnetSelectorTerms | subnetSelectorTerms are the terms to select subnets. Multiple terms are OR'd. Terms can be id or a tag. | N/A (required value) |
subnetSelectorTerms.Id | Id is an ID matching a subnet. Example:
CODE
type: string | N/A |
subnetSelectorTerms.Tags | Tags is a map of key/value tags used to select a subnet. Specifying '*' for a value selects all values of the given tag key. | N/A |
securityGroupSelectorTerms | securityGroupSelectorTerms are the terms to select security groups. Multiple terms are OR'd. Fields in the same term are AND’d. Terms can be id, name, or tag. | N/A (required value) |
securityGroupSelectorTerms.Id | Id is an ID matching a security group. type: string | N/A |
securityGroupSelectorTerms.Name | Name is a security group name matching a security group. type: string | N/A |
securityGroupSelectorTerms.Tags | Tags is a map of key/value tags used to select a security group. Specifying '*' for a value selects all values of the given tag key. | N/A |
xcompute | xcompute specifies xcompute-related settings | N/A (required value) |
xcompute.version | version is the xspot version of the controllers and workers. Must follow this regex pattern: type: string | N/A (required value) |
xcompute.nodeImageName | nodeImageName is the image that nodes should use. Should match an image name on EMS. type: string | N/A (required value) |
xcompute.nodeVolumeSizeGb | NodeVolumeSizeGb is the size (Gb) of the volume for the node. type: int | 10 |
xcompute.logPath | logPath is the log filepath on the EMS. type: string |
|
xcompute.enableBalloon | enableBalloon enables or disables VM self-ballooning. Dynamically adjusts memory allocation within VM to optimize performance and resource utilization. type: bool | true |
xcompute.enableHyperthreading | enableHyperthreading enables or disables hyperthreading. Allows a physical core to act as two logical cores. | true |
xcompute.maxControllers | maxControllers is the maximum controllers that can be deployed in each profile defined by this class and a referencing ExoNodePool. type: int | 10 |
xcompute.prefixCount | prefixCount is the prefix count for the VM naming convention. type: int | 3 |
xcompute.tags | tags are key-value tags to be attached to worker and controller EC2 instances. | {} |
xcompute.controller | controller specifies controller-related settings. | listed below |
xcompute.controller.instanceProfile | instanceProfile is the iam instance profile for the controller. type: string | ““ (default value is set by helm flag) |
xcompute.controller.instanceType | instanceType is the instance type of the controller. type: string | c5.large |
xcompute.controller.volumeSizeGb | volumeSizeGb is the size of the controller's volume. type: int | 100 |
xcompute.controller.tags | tags are key-value tags to be attached to controller EC2 instances. |
|
xcompute.worker | worker specifies worker-related settings. | listed below |
xcompute.worker.instanceProfile | instanceProfile is the iam instance profile for the worker. type: string | ““ (default value is set by helm flag) |
xcompute.worker.instanceTypes | instanceTypes are the on demand instance type families and priorities that can be used to create a worker. Each entry in the instanceTypes list must match the regex pattern type: array[string] | {“m5:0”, “c5:0”, “r5:0”} |
xcompute.worker.spotTypes | spotTypes are the spot instance type families and priorities that can be used to create a worker. Each entry in the spotTypes list must match the regex pattern type: array[string] | {“m5:0”, “m5d:0”, “c5:0”, “c5d:0”, “r5:0”, “r5d:0”} |
xcompute.worker.tags | tags are key-value tags to be attached to worker EC2 instances. |
|
userdata | userdata defines the userdata script to be run on the node. type: string | ““ |
kubelet | kubelet defines args to be used when configuring kubelet on provisioned nodes. | {} |
ExoNodePool CRD
Field Name | Field Description | Default Value |
---|---|---|
template.spec | Template contains the template of possibilities for the provisioning logic to launch a NodeClaim with. | N/A (required) |
template.spec.taints | Taints will be applied to the NodeClaim's node. | {} |
template.spec.startupTaints | StartupTaints are taints that are applied to nodes upon startup which are expected to be removed automatically within a short period of time, typically by a DaemonSet. StartupTaints are ignored for provisioning purposes. | {} |
template.spec.resources | Resources models the resource requirements for the NodeClaim to launch. | {} |
template.spec.requirements | Requirements are layered with GetLabels and applied to every node. Entries consist of a label key, an operator, and a value. You can define the set of acceptable CPU and memory sizes for the nodes through the | N/A (required) |
template.spec.nodeClassRef | NodeClassRef is a reference to an ExoNodeClass that defines provider specific configuration | N/A (required) |
template.spec.exostellar | Exostellar configures the workload optimizer (WO) and infrastructure optimizer (IO) settings. | N/A (required) |
template.spec.exostellar.workloadOptimizer | WorkloadOptimizer configures Exostellar's workload optimizer. | N/A (required) |
template.spec.exostellar.workloadOptimizer.enabled | Enabled enables workload optimizer for the nodes in this pool. type: bool | N/A (required) |
template.spec.exostellar.infrastructureOptimizer | InfrastructureOptimizer configures Exostellar's infrastructure optimizer. | N/A (required) |
template.spec.exostellar.infrastructureOptimizer.enabled | Enabled enables infrastructure optimizer for the nodes in this pool. type: bool | N/A (required) |