Booting an Instance from the LSF AMI

To ensure Exostellar can boot your AMI and run your jobs, we need to capture the boot parameters.

The following script has aws CLI examples to catalog AMI information (as in the previous step) as well as boot a temporary instance to grab the boot parameters.

The script assumes aws credentials and IAM permissions for describing instances and AMIs, as well as, starting and terminating instances. As written, it is designed to be used within the same subnet as the targeted LSF cluster, but it is only provided as an example.

An instance could be booted with relative ease in the AWS Console, as well.

amiprep.sh Script

BASH

#!/bin/bash

if [[ ! "$#" -eq 2 ]]; then                                                                                           
        printf "\n\t\tUSAGE:  $( basename ${0} ) <ami-id> <key-name>\n\n" >&2                   
        exit 1
else
        export LOGGER=./logging.$( basename ${0} .sh ).$$
fi

(
  #set -x
 
  AMI_ID=${1} 
  KEY_NAME=${2} 
  REGION=$( curl http://169.254.169.254/latest/meta-data/placement/region 2>/dev/null )
  MACS=$( curl http://169.254.169.254/latest/meta-data/network/interfaces/macs 2>/dev/null |sed 's#/##g' )
  SUBNET_ID=$( curl http://169.254.169.254/latest/meta-data/network/interfaces/macs/${MACS}/subnet-id 2>/dev/null )
  SEC_GRP_IDS=$( curl http://169.254.169.254/latest/meta-data/network/interfaces/macs/${MACS}/security-group-ids 2>/dev/null )

  echo "Grabbing ${AMI_ID} info and stashing it in ./logging.${AMI_ID}.$$.info"
  COMMAND="aws ec2 describe-images --image-id ${AMI_ID} --region ${REGION}"
  ${COMMAND} > ./logging.${AMI_ID}.$$.info
  LAST_EXIT_STATUS=$?
  echo 
  if [[ ! ${LAST_EXIT_STATUS} -eq 0 ]]; then
          echo "Exiting due to failure of previous command: ${COMMAND}"
          echo 
          exit ${LAST_EXIT_STATUS}
  fi

  echo "Starting an instance from ${AMI_ID} and stashing output in ./logging.new-instance.$$.info"
  TAGS="ResourceType=instance,Tags=[{Key=Name,Value=temporary-parse-ami},{Key=niv-dev-id,Value=deleteme}]"
  COMMAND="aws ec2 run-instances --image-id ${AMI_ID} --region ${REGION} --instance-type c5.xlarge --key-name ${KEY_NAME} --subnet-id ${SUBNET_ID} --tag-specifications ${TAGS} --security-group-ids ${SEC_GRP_IDS}"
  ${COMMAND} > ./logging.new-instance.$$.info
  LAST_EXIT_STATUS=$?
  echo 
  if [[ ! ${LAST_EXIT_STATUS} -eq 0 ]]; then
          echo "Exiting due to failure of previous command: ${COMMAND}"
          echo 
          exit ${LAST_EXIT_STATUS}
  fi

  INSTANCE_ID=$( grep InstanceId logging.new-instance.$$.info |awk ' {print $2} ' |sed 's/"//g; s/,//g' )
  echo 'To clean up, do not forget:'
  echo "aws ec2 terminate-instances --instance-ids ${INSTANCE_ID}"
  echo 

) |& tee -a ${LOGGER}

On the temporary instance:

Save the contents of the /proc/cmdline file for future use.
Based on the kernel version string from /proc/cmdline, we also need to copy the initrd or initramfs image.
1. Example /proc/cmdline output:
  1. BOOT_IMAGE=/boot/vmlinuz-3.10.0-1160.105.1.el7.x86_64 root=UUID=44a6a613-4e21-478b-a909-ab653c9d39df ro console=tty0 console=ttyS0,115200n8 crashkernel=auto net.ifnames=0 console=ttyS0 LANG=en_US.UTF-8
2. “kernel version string” from above:
  1. 3.10.0-1160.105.1
3. Command to list the initramfs or initrd image file:
  1. ls -la /boot/*3.10.0-1160.105.1*.img
4. That .img file and the /proc/cmdline output need to be saved to remote storage or someplace where they can be used later.