Using Cloud Instances for Your Cuda Loads

Introduction

Writing an algorithm employing Cuda does not require a state of the art GPU. However, to reap the benefits of massively parallel processing it is required a powerful one. How about using a cloud instance?.

Writen by: Santiago Hurtado

rack

Let’s do some quick numbers first:

An Intel i7-9700f processor should deliver 384 Gflops at a price of about 400 USD

Instance Creation

Ok, enough of that, So how do we get an Azure VM that gives the promise of a large number of computations per second. * The setup we will be performing is only recommended for the bare usage of Cuda*. A depth tutorial can be found at the Nvidia site. Also, if you are looking for a * Data science VM* there is an easier way, follow this tutorial from Microsoft.

Most of the time I recommend using the Azure Cli cli to have repeatable tasks. I also assume you use mac or Linux since the variables used were written for Bash.

Don’t spin up a full-fledged GPU instance for setting up the machine, you can use a free version for the initial image and when done, copy or resize the instance.

Setup

Some basic info for this tutorial, we use an initial Size of Standard_B2s and East US as the default location.

Lets first create a resource group, if you haven’t one already.

  • Find a location and verify the size you want exist in the location

      az account list-locations

  az vm list-sizes -l ${azure_region}

  • Create a resource group

      az group create --name ${resource_group} --location ${azure_region}

  • Now create the VM, make sure you setup the storage-sku for later rezise compatibility.

          az vm create \
          --resource-group ${resource_group}  \
          --name ${vm_name} \
          --image UbuntuLTS \
          --admin-username ${USERNAME} \
          --ssh-key-values <put the path or paths to your .pub ssh key here> \
          --size Standard_B2ms --storage-sku StandardSSD_LRS

Connect to your VM

To find the public IP address of your VM, the response of the cli has it, also you can get it as follows:

az vm show -d -g ${resource_group}  -n ${vm_name} --query publicIps -o tsv

Remember the machine the IP adress can change unless you set a pay static IP

  • SSH to your instance using you .pub key

      ssh <public_ip>

Install drivers

  • Upgrade all, it is a good idea to restart after.

      sudo apt -qq update && sudo apt -yqq upgrade
  sudo restart

  • Setup the repositories and drivers, see that we dont need the actual GPU on the VM just yet. check the nvidia website for any updates on these steps:

      wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/cuda-ubuntu1804.pin
  sudo mv cuda-ubuntu1804.pin /etc/apt/preferences.d/cuda-repository-pin-600
  sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/7fa2af80.pub
  sudo add-apt-repository "deb http://developer.download.nvidia.com/compute/cuda/repos/ubuntu1804/x86_64/ /"
  sudo apt-get -qq update
  sudo apt-get -yqq install cuda

While you wait you could write your cuda code on a new terminal.

  • Setup The Bash environment

      echo export PATH=/usr/local/cuda-10.2/bin:/usr/local/cuda-10.2/NsightCompute-2019.1'${PATH:+:${PATH}}' >> ~/.bashrc
  echo export LD_LIBRARY_PATH=/usr/local/cuda-10.2/lib64'${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}'>>~/.bashrc

  • Power off

      sudo poweroff

Running on a GPU instance

Warning: Please don’t forget to deallocate your instance it will keep billing you if you don’t.

The new size we will be using is:

    size = Standard_NC6_Promo

  • Deallocate the VM for resizing

      az vm deallocate --name ${vm_name} --resource-group ${resource_group}

  • Resize the VM

      az vm resize --resource-group ${resource_group}  --name ${vm_name} --size ${size}

  • Start the VM

      az vm start --resource-group ${resource_group} --name ${vm_name}

  • Get the new IP

      az VM show -d -g ${resource_group}  -n ${vm_name} --query publicIps -o tsv

Testing it works

  • Check what NVIDIA card you have

      nvidia-smi

  • Copy a cuda code, for example https://github.com/advt3/ParallelProgramming/blob/master/cuda/enumerate.cu

      wget https://raw.githubusercontent.com/advt3/ParallelProgramming/master/cuda/enumerate.cu

  • Compile

      nvcc -o enumerate enumerate.cu

  • Run

      ./enumerate

  • Shutdown and deallocate

      az vm deallocate --name ${vm_name} --resource-group ${resource_group}

  • Verify the status of your VM

      az vm list -d -o table

Cool we did it, I will recomend you scrip this steps so is a bit more natural an can be done often.

Final Thoughts

In this post, we have explained the complete environment setup using the latest Cuda driver and ubuntu 18.04, however, you can find a more complete but outdated description on the Microsoft documentation.

It will be nice to automatize the code running on a CI/CD pipeline so you can run your code only for the specific needed time. We will see when we have the time to test it.

Happy coding!

References