Tutorial: Installing a Virtual Mutli-node HPCC Cluster

Resources | June 20, 2013 1:20 pm

This post covers the installation of HPCC on a multi-node virtual cluster using Oracle VM VirtualBox. This enables you to take advantage of HPCC Systems’s Massively Parallel Processing without the hardware requirements.

Click (more…) to View the Tutorial

Installing Oracle VM Virtualbox

Download VirtualBox by browsing to VirtualBox.org and clicking on “Download” in the navigation sidebar. Once VirtualBox has been downloaded, follow the instructions necessary to install it on your system.

Virtual Multi-Node Cluster Tutorial - Homepage of VirtualBox.org

Downloading Guest OS ISO Image

You will need an ISO Image of the OS you wish to virtualize. Refer to the Installing & Running the HPCC Platform documentation for a list of supported operating systems. This document will be using 64-Bit Ubuntu 10.04 LTS which can be found at http://www.ubuntu.com.

Creating a Virtual Guest OS

  1. Open VirtualBox Manager
  2. Click on the New icon (or Menu Bar: Manager > New)
  3. Select an OS to virtualize
Installing Virtual Multi-Node Cluster - Creating a New Virtual Machine in Oracle VirtualBox

Note: Please Refer to the installing and running the HPCC Platform Documentation for Supported Operating Systems

Starting Virtual Image

  1. Select your newly created VM and click on the Settings icon.
  2. Select the Storage category and click on Empty under IDE Controller
  3. In the Attributes sub section, click on the Disc icon and select¬†“Choose a virtual CD/DVD disk file…“.
  4. Select the OS Disk Image that you download previously. Click OK.
  5. In the VirtualBox Manager, click on the now enabled Start icon. Follow the instructions to install the OS on the Virtual Machine.

Installing Multi-node virtual cluster - VM Settings Dialog

Installing the HPCC Platform on a Virtual Machine

Once the OS has installed on the Virtual Machine, download the package from the HPCC Systems website and install the package on the Virtual Machine. Please refer to the Installation & Running the HPCC Platform documentation for a walk-through on the installation process.

Once HPCC has been installed, shut-down the virtual image.

Cloning Virtual Machines

Using Oracle VM VirtualBox GUI Front-End

  1. In the VirtualBox Manager, right-click the virtual image and select Clone.
  2. Choose a name for the cloned Virtual Machine and Click Next.
  3. Select the Full Clone radio-button and click on the Clone button.

Using the Command Prompt/Terminal

  1. Open the Command Prompt/Terminal
  2. Type the following command (replacing the “path\to\etc\” with the actual path in question):Path\to\VirtualBox\VBoxManage.exe clonevid “path\to\existingvdi.vdi” “path\to\clonevdi.vdi”
  3. Open the VirtualBox Manager and click on Create.
  4. In the Create Virtual Machine Wizard, create a new Virtual Machine using the same configuration in the first Virtual Image created.
  5. In the Virtual Hard Disk dialog in the Create Virtual Machine Wizard select Use Existing Hard Disk and browse to the location of the cloned .vdi file created in the Command Prompt/Terminal.
  6. Click Create.

Repeat this section for the number of nodes you wish to have in your virtual cluster.

Networking Virtual Machines

For each virtual image:

  1. In the VirtualBox Manager, right-click on the virtual image and select Settings.
  2. Go to the Network category, make sure “Enable Network Adapter” is checked and select “Bridged Adapter” in the Attached to:” drop-down list.
  3. Click OK.
  4. Start all Virtual Machines.

Installing a Multi-Node HPCC Cluster

Refer to the Installation & Running the HPCC Platform documentation for a walk-through on the creation of a multi-node cluster.

And you’re done! Sometimes you want to test certain things out on your personal machine before throwing it on a production cluster. This was a godsend when I was figuring out the PIPE function (which I’ll probably write more about someday).