Virtual Appliance Marketplace

Features

Collegiate:
Yes

One-line Description:
Instant Virtual Cluster

Filename:
CondorVM.zip

Size Compressed:
252

Allocated Memory:
160

Username:
root

Password:
VMWchallenge

VMware Tools Installed?:
Yes

Operating System:
Debian Linux 3.1

Applications:
Condor 6.7.19
Condor Web Portal 0.0.2
Howl 1.0.0
Apache 2.0.54-5
PHP 4.3.10-16
Perl 5.8.4-8sarge4
Flam3 2.6

Description:
* Appliance function, uses and benefit to the intended audience in
the VMware community:
This appliance provides an instant cluster, an application
installable on one or more machines that allows its user to easily
perform distributed batch computing tasks using the Condor system.
Although it can be used on an individual machine for managing batch
jobs such as scientific or financial computations, the real strength
of this virtual appliance is when it is installed on a set of
machines that are connected via a network. We have designated one
machine as the "queen bee," and the other machines "drones." The
user submits some tasks to the queen using the provided web interface
or other custom tools, and the drone machines execute those tasks and
return the results.
This system should scale to thousands of drones, and is intended to
be both secure and easy to use. Deploying the Condor Batch
Scheduling system as a VM Appliance provides three main benefits:
1. Sandboxing Remote Execution: Resource owners may be hesitant
about allowing remote execution of arbitrary programs on their
systems. However, grid computing cannot function without this
ability. Deploying the remote execution component of Condor in a
virtual machine allows us to mitigate these fears by providing a
secure sandbox in which Condor can execute arbitrary user programs.
2. Homogeneity: Because the Condor VM appliances are all
identical, batch computing tasks need only be written and compiled
for a particular platform (in this case, Debian Linux, but more
generally, for any guest platform supported by both Condor and VMware).
3. Ease of deployment: Condor is a complex application with many
configuration options. By providing a pre-configured, pre-installed
appliance, users unfamiliar with Condor can still get a compute
cluster up and running in minutes.
At runtime the user promotes a single VM appliance to the status of
"queen bee" and the rest become drones by default. Each drone
requires only two things at runtime: an IP address and the location
of the appliance acting as the queen (queens also require an IP
address). To provide maximum flexibility of deployment, we support
several different methods of acquiring an IP address and the location
of the queen.
1. DHCP with DNS-SD over mDNS: Again, the appliance obtains its
IP address from the local DHCP service. However, this time the
location of the queen is discovered using DNS Service Discovery
[www.dns-sd.org] over Multicast DNS [www.multicastdns.org]. This
mode requires no user configuration, but limits the participation of
drones to appliances running within reach of multicast packets.
2. AutoIP with DNS-SD over mDNS: Like (2) the location of the
queen is determined using multicast DNS. However, in this case the
nervous network administrator is not required to issue VM appliances
global IP addresses. Instead, each appliance chooses its own IP
address using IPv4 Link-Local Addressing (IPv4-LL) [RFC3927]. This
mode doesn't require amiable network administrator, but limits the
participation of drones to appliances running on a single link-local
network.
* The following are the steps to build the final virtual appliance:
1. Download and create a new Virtual Machine. Configure the
Virtual Machine with 160MB memory, 2GB sparse virtual disk, two
Ethernet interfaces, USB, and CD-ROM. The first Ethernet interface
must be set to 'bridged' for outside networking and the other
Ethernet interface set to 'host-only' for communicating with the
local computer. The order of the network interfaces is very important
as our supplemental software makes assumptions about the ordering.
2. Download and install Debian's 'debian-31r1a-i386-
businesscard.iso' by mounting it as a CD for the virtual machine at
startup.
3. In order to maintain a small sparse virtual disk, all
installations must be made through an in memory file system. Start
by mounting tmpfs into /dev/shm. Then, to avoid Debain package
managers from using an on disk cache make a soft-link from /var/cache/
apt/archives to a directory under /dev/shm, e.g. /dev/shm/archives.
Finally, proceed to step 4/ and copy all packages into /dev/shm, e.g.
scp condor.tar user@vm:/dev/shm. When package installation is
complete remove the soft-link.
4. Install additional Debian packages such as apache2 and php4 in
addition to basic system packages. Also, download and install Howl
for autoip and multicast DNS, and Condor. Configure the Condor Web
portal to use a set of custom example jobs and display the number of
drones in the appliance pool. Provided examples use Flam3, a 3D
fractal generator. Then install the custom supplemental appliance
scripts to manage our appliance state machine and ensure all the
correct services are running.
* Installed OS and application's licenses:
Debian and its packages are under the GPL.
Howl comes under its own license and APSL.
Condor comes under the Condor Public License.
Flam3 comes under the GPL.
* Once VMs are up an running follow these steps:
1) Designate one virtual appliance to be the Condor Master
("queen bee"). Connect to it using the URL shown on its console from
a browser on the same machine (host only interface). Click the
"Become Central Manager" button.
2) At the top of the page, with a green background, click on
"Submit Job".
3) Three applications are shown. Click on the third, "Flam3 Set".
4) Click the button on the pink background "Submit Query". This
submits a job that generates 16 random fractal images. If you see an
error, simply wait a few seconds and refresh the page. Condor may
still be starting.
5) Click the link "Go to Job Status".
6) Scroll to the bottom, and click on the link "Results" for the
last job listed under "Completed Jobs."
7) Wait and watch as this page refreshes showing thumbnails of
the 16 images. As each job completes, the resulting images will be
returned to the queen.

Technical Specifications

Operating System:

Debian Linux 3.1

VMware Tools installed: No

Size: 252MB

Allocated Memory (RAM): 160

Applications Installed:

Condor 6.7.19
Condor Web Portal 0.0.2
Howl 1.0.0
Apache 2.0.54-5
PHP 4.3.10-16
Perl 5.8.4-8sarge4
Flam3 2.6

Virtual Appliance Account Information