Customizing Profiles for Slurm Integration

Edit the Profile for Your Purposes:

1. Because this profile was copied and will be added as a new profile, remove the ”Id”: so the EMS can automatically assign a new unique ID.

2. Tagging instances created by the backend is controlled by two sections, depending on the function of the asset:

   1. Controllers are On-Demand instances that manage other instances. By default, they are tagged as seen below.

      1. CODE

               {
                 "Key": "exostellar.xspot-role",
                 "Value": "xspot-controller"
               }

      2. To add additional tags, duplicate lines 1-4 as 5-8 below (as many times as you need), noting that an additional comma is added on line 4.

      3. CODE

               {
                 "Key": "exostellar.xspot-role",
                 "Value": "xspot-controller'
               },
               {
                 "Key": "MyCustomKey",
                 "Value": "MyCustomValue"
               }

      4. Don’t forget the comma between tags.

   2. Workers will be created by Controllers as needed and they can be On-Demand/Reserved instances or EC2 Spot. By default, they are tagged as seen below:

      1. CODE

               {
                 "Key": "exostellar.xspot-role",
                 "Value": "xspot-worker"
               }

      2. Add as many tags as needed.

      3. CODE

               {
                 "Key": "exostellar.xspot-role",
                 "Value": "xspot-worker"
               },
               {
                 "Key": "MyCustomKey",
                 "Value": "MyCustomValue"
               }

      4. Don’t forget the comma between tags.

3. InstanceType: Controllers do not generally require large instances.

   1. In terms of performance, these On-Demand Instances can be set as c5.xlarge or m5.xlarge with no adverse effect.

4. MaxControllers : This will define an upper bound for your configuration.

   1. Controllers will manage up to 80 workers.

   2. The default upper bound is 800 nodes joining your production cluster: notice line 20 "MaxControllers": 10,.

   3. If you plan to scale past 800 nodes joining your production cluster, MaxControllers should be increased.

   4. If you want to lower that upper bound, MaxControllers should be decreased.

5. ProfileName: This is used for your logical tracking, in the event you configure multiple profiles.

6. InstanceTypes here in the Worker section, this refers to On-Demand instances – if there is no EC2 Spot availability, what instances do you want to run on.

7. SpotFleetTypes : here in the Worker section, this refers to EC2 Spot instance types – because of the discounts, you may be comfortable with a much broader range of instance types.

   1. More types and families here, means more opportunities for cost optimization.

   2. Priorities can be managed by appending a : and an integer, e.g. m5:1 is a higher priority than c5:0.

8. NodeGroupName : This string appears in Controller Name tagging <profile>-NGN-count

9. All other field/lines can be ignored in the asset.