Configuring ExoNodeClass and ExoNodePool

This guide walks you through configuring ExoNodeClass and ExoNodePool for Exostellar Karpenter (version 2.x).

Overview

Before submitting workloads to your cluster, two primary components must be configured:

ExoNodeClass defines settings relating to AWS and xcompute profiles.
ExoNodePoolenables features and defines sets of nodes that Exostellar Karpenter can create.

Sample Configuration

YAML

apiVersion: karpenter.k8s.exo/v1alpha1
kind: ExoNodeClass
metadata:
  name: class-a
spec:
  xcompute:
    nodeImageName: "k8s-131"
    version: "xspot-3.2.2"
  securityGroupSelectorTerms:
    - tags:
        installer: x-install-<cluster-name>   # Update with your cluster name.
  subnetSelectorTerms:
    - id: "subnet-083b46ca34c707209"
---
apiVersion: exokarpenter.sh/v1beta1
kind: ExoNodePool
metadata:
  name: pool-a
spec:
  disruption:
    consolidateAfter: 1m
    consolidationPolicy: WhenEmpty
  template:
    spec:
      nodeClassRef:
        kind: ExoNodeClass
        name: class-a
      requirements:
        - key: "karpenter.k8s.exostellar/instance-family"
          operator: In
          values: ["m5"]
      exostellar:
        workloadOptimizer:
          enabled: true
        infrastructureOptimizer:
          enabled: true
      taints:
        - key: exokarpenter.sh/x-compute
          effect: NoSchedule
          value: "true"

ExoNodeClass CRD

Field Name	Field Description	Default Value
subnetSelectorTerms	subnetSelectorTerms are the terms to select subnets. Multiple terms are OR'd. Fields in the same term are AND’d. Terms can be id or a tag.	N/A (required value)
subnetSelectorTerms.Id	Id is an ID matching a subnet. Example: CODE `spec: subnetSelectorTerms: - id: "subnet-09fa4a0a8f233a921" - id: "subnet-0471ca205b8a129ae"` type: string	N/A
subnetSelectorTerms.Tags	Tags is a map of key/value tags used to select a subnet. Specifying '*' for a value selects all values of the given tag key.	N/A
securityGroupSelectorTerms	securityGroupSelectorTerms are the terms to select security groups. Multiple terms are OR'd. Fields in the same term are AND’d. Terms can be id, name, or tag.	N/A (required value)
securityGroupSelectorTerms.Id	Id is an ID matching a security group. type: string	N/A
securityGroupSelectorTerms.Name	Name is a security group name matching a security group. type: string	N/A
securityGroupSelectorTerms.Tags	Tags is a map of key/value tags used to select a security group. Specifying '*' for a value selects all values of the given tag key.	N/A
xcompute	xcompute specifies xcompute-related settings	N/A (required value)
xcompute.version	version is the xspot version of the controllers and workers. Must follow this regex pattern: `^xspot-[0-9]+.[0-9]+.[0-9]+$` type: string	N/A (required value)
xcompute.nodeImageName	nodeImageName is the image that nodes should use. Should match an image name on EMS. type: string	N/A (required value)
xcompute.nodeVolumeSizeGb	NodeVolumeSizeGb is the size (Gb) of the volume for the node. type: int	10
xcompute.logPath	logPath is the log filepath on the EMS. type: string	`/xcompute/logs`
xcompute.enableBalloon	enableBalloon enables or disables VM self-ballooning. Dynamically adjusts memory allocation within VM to optimize performance and resource utilization. type: bool	true
xcompute.enableHyperthreading	enableHyperthreading enables or disables hyperthreading. Allows a physical core to act as two logical cores.	true
xcompute.maxControllers	maxControllers is the maximum controllers that can be deployed in each profile defined by this class and a referencing ExoNodePool. type: int	10
xcompute.prefixCount	prefixCount is the prefix count for the VM naming convention. type: int	3
xcompute.tags	tags are key-value tags to be attached to worker and controller EC2 instances.	{}
xcompute.controller	controller specifies controller-related settings.	listed below
xcompute.controller.instanceProfile	instanceProfile is the iam instance profile for the controller. type: string	““ (default value is set by helm flag)
xcompute.controller.instanceType	instanceType is the instance type of the controller. type: string	c5.large
xcompute.controller.volumeSizeGb	volumeSizeGb is the size of the controller's volume. type: int	100
xcompute.controller.tags	tags are key-value tags to be attached to controller EC2 instances.	`{` `exostellar.xspot-role: xspot-controller` `}`
xcompute.worker	worker specifies worker-related settings.	listed below
xcompute.worker.instanceProfile	instanceProfile is the iam instance profile for the worker. type: string	““ (default value is set by helm flag)
xcompute.worker.instanceTypes	instanceTypes are the on demand instance type families and priorities that can be used to create a worker. Each entry in the instanceTypes list must match the regex pattern `^[a-zA-z0-9]+:[0-9]+$` type: array[string]	{“m5:0”, “c5:0”, “r5:0”}
xcompute.worker.spotTypes	spotTypes are the spot instance type families and priorities that can be used to create a worker. Each entry in the spotTypes list must match the regex pattern `^[a-zA-z0-9]+:[0-9]+$` type: array[string]	{“m5:0”, “m5d:0”, “c5:0”, “c5d:0”, “r5:0”, “r5d:0”}
xcompute.worker.tags	tags are key-value tags to be attached to worker EC2 instances.	`{` `exostellar.xspot-role:xspot-worker` `}`
userdata	userdata defines the userdata script to be run on the node. type: string	““
kubelet	kubelet defines args to be used when configuring kubelet on provisioned nodes.	{}

ExoNodePool CRD

Field Name	Field Description	Default Value
template.spec	Template contains the template of possibilities for the provisioning logic to launch a NodeClaim with.	N/A (required)
template.spec.taints	Taints will be applied to the NodeClaim's node.	{}
template.spec.startupTaints	StartupTaints are taints that are applied to nodes upon startup which are expected to be removed automatically within a short period of time, typically by a DaemonSet. StartupTaints are ignored for provisioning purposes.	{}
template.spec.resources	Resources models the resource requirements for the NodeClaim to launch.	{}
template.spec.requirements	Requirements are layered with GetLabels and applied to every node. Entries consist of a label key, an operator, and a value. You can define the set of acceptable CPU and memory sizes for the nodes through the `karpenter.k8s.exostellar/instance-category`, `karpenter.k8s.exostellar/instance-family`, `node.kubernetes.io/instance-type`, and `karpenter.k8s.aws/instance-cpu` label keys using either the `In` or `NotIn` operators.	N/A (required)
template.spec.nodeClassRef	NodeClassRef is a reference to an ExoNodeClass that defines provider specific configuration	N/A (required)
template.spec.exostellar	Exostellar configures the workload optimizer (WO) and infrastructure optimizer (IO) settings.	N/A (required)
template.spec.exostellar.workloadOptimizer	WorkloadOptimizer configures Exostellar's workload optimizer.	N/A (required)
template.spec.exostellar.workloadOptimizer.enabled	Enabled enables workload optimizer for the nodes in this pool. type: bool	N/A (required)
template.spec.exostellar.infrastructureOptimizer	InfrastructureOptimizer configures Exostellar's infrastructure optimizer.	N/A (required)
template.spec.exostellar.infrastructureOptimizer.enabled	Enabled enables infrastructure optimizer for the nodes in this pool. type: bool	N/A (required)