Creating LSF Compute Environment Base

Compute Environment

To prepare the nested-vm compute node we need to process an AMI. If you have an existing AMI for your compute environment we can leverage that. If the site already uses the AWS Provider for Resource Connector, using the AMI from the awsprov_templates.json will suffice. If an AMI needs to be created from an existing compute node the following steps walk through that process.

The key concepts to keep in mind about getting a good AMI is that it should boot fast and do little to no work, for example in bootstrapping or user_data. Also, it should have everything required to run the workflows and authenticate users.

To use the same AMI as AWS Provider for Resource Connector

• Find the AMI_ID from the configuration:

  CODE

  badmin rc view -c templates

  • Make a note of the AMI_ID for subsequent steps.

To create an AMI from an LSF compute node

• Allocate a compute node:

  CODE

  bsub -J ami-creation sleep 5000

• When the job is allocated, gather some information on the node running the job:

  • The bsub command above should have returned a JOB_ID.

  • The bjobs -l <JOB_ID> command should show the JOB_ID running on particular node or execution host.

  • Once it is possible to identify the node, navigate to AWS console EC2 Instances page and search for the Private IPv4 Address.

  • Select that instance and click “Create an image” from the Actions button in the upper right corner.

  • AMI Creation takes several minutes to complete. Give the AMI a unique name, optionally a description, and accept the default values provided by the prompts.

  • Make a note of the AMI_ID for subsequent steps.

  • NOTE : this action will reboot the node and kill our job, which is expected.