Virtual Appliance Marketplace

Average user Rating:9 Ratings (Login to rate)


Cluster headnode ready to deploy a fedora core 5 based image. Includes tools for managing, administering and running a cluster.

Features

Collegiate:
No

One-line Description:
Cluster headnode ready to deploy a fedora core 5 based image. Includes tools for managing, administering and running a cluster.

Filename:
oscar_fc5x.tar.bz2

Size Compressed:
1464

Allocated Memory:
320

Username:
oscar

Password:
vmoscar

VMware Tools Installed?:
Yes

Operating System:
fedora core 5

Torrent?:
No

Applications:
OSCAR 5.0 SVN r4853
and all applications included into it:
LAM MPI 7.1.2
MPICH 1.2.7
OpenMPI 1.0.2
Ganglia 3.0.3
SystemImager 3.7.3
SystemInstaller-OSCAR 2.2.3
SystemConfigurator 2.2.2-11ef
NetBootManager 0.8
In addition
yume 1.6
Nagios 1.3

Description:
Virtual OSCAR Cluster Headnode
This virtual appliance is based on the OSCAR (Open Source Cluster Application Resources) cluster infrastructure, a collaborative effort to make beowulf-type clusters easy to use and manage. The author is core OSCAR developer and uses VMware's VMplayer on a regular basis for developing and testing OSCAR.
The virtual appliance is a fully working OSCAR headnode built on top of Fedora Core 5 i386 and prepared for the definition of client cluster nodes and their deployment.
A client node image based on Fedora Core 5 is predefined and ready built for deployment to cluster nodes. It is called fc5image. Additional images can be built or imported, with some tweaks even images of other architectures than x86 can be managed. OSCAR's strength in supporting multiple distributions and architectures and managing heterogeneous clusters can be fully used from this virtual appliance.
All one has to do in order to get a fully working Linux cluster including parallel execution libraries for high performance computing (HPC), a highly scalable batch scheduler and versatile resource manager as well as tools for managing and monitoring the cluster in scalable way is to define a set of cluster client nodes and install them. The client nodes can be virtual machines, themselves, all they need is sufficient disk-space to accommodate the installed OS which takes up approximately 1GB.
ATTENTION:
Make sure you know exactly which machine you are going to install! Picking up the wrong MAC address on a busy network means to install and overwrite the harddisk of the wrong machine!
The "nice" version of this page (with more html markup) can be accessed here.
Potential users of this appliance:

• Cluster administrators who want an easy start with OSCAR and want to benefit of the simple way of backing up, restoring and migrating virtual machines. Having a backup of the entire cluster headnode which can be restarted on a Windows laptop is increasing the availability of the cluster services a lot.
  
• Students taking parallel computing classes: the virtual headnode appliance makes it trivial to build a cluster even on their Windows laptops. All they need besides the headnode appliance is a copyable empty virtual machine which can easily be created with the mk_vcluster utility. The cluster is ready for developing and testing parallel applications using any of the three major MPI versions or PVM.
  
• OSCAR users and developers who need to test changes and restore easily a reproducible state of the headnode.
  

This OSCAR headnode is somewhat special because it uses online repositories for the distribution packages (a new feature in OSCAR) and includes a ready configured system health monitoring software (Nagios), which is not part of OSCAR.
Building and Configuring the Virtual Headnode
This appliance was built by installing a plain Fedora Core 5 system onto a virtual machine with 12GB virtual disk. The virtual machine has two bridged ethernet interfaces, eth0 is intended for communication with the cluster nodes, its configuration should not be changed. eth1 is for the external communication, it should be adapted to the user's LAN.
A very recent OSCAR build taken from SVN (version 5.0 alpha r4853) was installed into the virtual machine. It was configured to access the fedora core mirror sites instead of accessing the usually local package repositories. Some tweaks and fixes had to be applied to the OSCAR installation in order to get it running. These will be incorporated into the OSCAR SVN repository soon.
Nagios was installed into the virtual OSCAR headnode as an OSCAR package, including scripts for automatically configure it for cluster monitoring. The adaptation of Nagios to OSCAR is work which is not yet integrated into the OSCAR mainline.
The OSCAR installation wizard was invoked, an image was prepared and a test node was installed in order to check functionality. The test node was then deleted from the OSCAR database.
Components

1. Image based installation / node deployment:
   built with the SystemInstallation Suite: SystemImager, SystemInstaller-OSCAR, SystemConfigurator.
   
2. Resource Manager:
   the virtual headnode has the resource manager Torque installed and preconfigured with one queue. The resource scheduling is controlled by Maui. Use the command qsub for submitting jobs, qstat for checking their status, qdel for deleting them, pbsnodes for finding out about the availability of nodes.
   
3. Parallel message passing libraries: MPI, PVM:
   
   
   
   • MPICH 1.2.7
     
   • OpenMPI 1.0.2
     
   • LAM MPI 7.1.2
     
   • PVM 3.4.5+4
     
   
   
4. Environment switcher:
   allows switching easily between environments like different MPI versions, compilers, etc... Read the man-pages of switcher. All delivered MPIs come with preconfigured switcher/modules.
   
5. Live cluster management tools:
   C3 (Cluster Command and Control) and SC3 (Scalable (Sub-)Cluster Command and Control). Allow parallel cluster command execution with commands like cexec, cpush, ckill, cget, scexec, scpush, scrpm...
   
6. Cluster user database synchronization via opium/sync_files:
   pushes password databases from the master node to the cluster nodes periodically. If the master node is integrated into the enterprise LDAP or NIS+ environment, the client nodes will get the passwords assembled from the LDAP or NIS+ database on the master node. There is no need to integrate the cluster nodes into LDAP, too.
   
7. Shared /home filesystem across the cluster:
   the /home filesystem of the master node is exported to the client nodes via NFS. This allows them to share the SSH keys. SSH keys for accessing the cluster are automatically generated and allow accessing any node within the cluster without entering passwords.
   
8. Cluster performance monitoring system:
   Ganglia 3.0.3 is installed on the master node and made accessible over its web server. The generic URL is http://master_node_address/ganglia/
   
9. Cluster health monitoring system:
   Nagios 1.3 is installed on the master node and automatically configured for monitoring services on the master node and the accessibility of client nodes, their filesystems and swap space. Nagios will restart certain services on the master node if they die via an event handler, thus keeping them highly available.
   Nagios is extremely versatile in its configuration and the way it notifies the administrators of any failures. The default configuration for OSCAR is to send notification emails to the mail alias nagios-admin on the headnode. Edit the /etc/aliases file if you want to change this behavior.
   
10. NetBootManager for managing how and what client nodes will be booting if they are configured in the BIOS to boot via PXE. This application manages symbolic links to PXE configuration files for each cluster node.
    

Virtual Headnode Usage
Please check
http://oscar.openclustergroup.org/
for further documentation on OSCAR.
Headnode Accessibility
The first step after booting a fresh Virtual OSCAR Headnode and logging in as user “oscar” (password: vmoscar) is to integrate its external interface (eth1) into the local network. Select System -> Administration -> Network, enter the root password (default: vmOSCAR) and edit the “eth1” setting according to your needs. Please consider changing the default root password!
Then open a shell and restart the network:
$ sudo service network restart
Check the accessibility of the machine from outside.
Management Interface
The main interface to the OSCAR system is the Management Interface which can be started by clicking on the OSCAR icon located on the desktop of the user “oscar”. It offers following buttons:

• "Download Additional OSCAR Packages”: currently there are no downloadable packages for the OSCAR version included in the Virtual Appliance.
  
• "Build OSCAR Client Image”: build additional client images. Can be used for building images of other distributions after a bit of tweaking. A Fedora Core 5 based image named fc5image is delivered with the Virtual Appliance.
  
• "Add OSCAR Clients”: define additional OSCAR clients. The Virtual Appliance comes with no clients defined, use this button to define your cluster nodes. Their IP addresses should be defined in the same subnet as the headnode's eth0 subnet!
  
• "Delete OSCAR Clients”: delete OSCAR clients from the OSCAR database.
  
• "Install/Uninstall OSCAR Packages”: this step doesn't work currently.
  
• "Monitor Cluster Deployment”: start a monitoring panel for visualizing the cluster node deployment status.
  
• "Test Cluster Setup”: push this button to test the OSCAR installation after the deployment and the “Finish Cluster Setup” step.
  
• "Network Boot Manager”: control the next boot action of client nodes which are booting from the network. Offers several configurable options: install, boot from installed harddisk, boot remotely offered kernel,run memory test.
  
• "Ganglia Monitoring System”: open the ganglia system performance monitoring tool in a web browser.
  
• "Nagios Monitoring System”: open the nagios cluster health monitoring tool in a web browser. Nagios requires authentication, the htpasswd file is located in /etc/nagios. The Virtual Appliance comes with nagios configured for two users:
  
  
  
  • user: guest, password: guest : has only read access
    
  • user: nagiosadmin, password: testnagios : has read/write access.
    
  
  

Define and Deploy Nodes
The Virtual OSCAR Headnode appliance comes with no client nodes defined, therefore defining nodes by pressing the “Add OSCAR Clients” button in the management panel is one the first step towards getting an OSCAR cluster up and running.
Adding new nodes to the cluster is a 3-step process. You will be guided through this process by the panel opened after pressing the “Add OSCAR Clients” button.

1. Define Client Nodes: define the name prefix, the number of nodes, the starting IP address, etc...
   
2. Setup Networking: mainly you will need to collect the ethernet MAC addresses of the nodes you intend to add. This can be done by detecting them (i.e. press the “Detect MAC Addresses” button and let the new nodes try to PXE-boot), entering them one by one or loading them from a file.After the network has been prepared start client node deployment by network-booting the new nodes. They will get an IP address from the headnode and will install themselves by rsync-ing the node image to their local harddisk.
   Of course the client nodes can also be virtual machines, such that the entire cluster can be virtualized.
   ---
   Before starting the installation: start netbootmgr and make sure that the hosts defined (which you want to install) are switched to "Install" mode!
   ---
   ATTENTION: MAKE SURE YOU COLLECT THE RIGHT MAC-ADDRESSES AND INSTALL THE RIGHT NODES! WHEN INSTALLING A NODE ITS HARDDISK WILL BE ERASED AND OVERWRITTEN! THE AUTHOR TAKES NO RESPONSIBILITY FOR NODES AND HARDDISKS WHICH HAVE BEEN ERRONEOUSLY ERASED!
   
3. Finish Cluster Setup: this “post-install” step is needed in order to get the configuration files of the cluster applications finalized and in sync with the changed cluster state. Push this button when all client nodes were installed successfully.
   

Applications and Licenses
The virtual appliance is built on top of a Fedora Core 5 installation. License information of every installed package is available by invoking the command
$ rpm -qa –qf '%{NAME} %{LICENSE}n'
The packages within the OSCAR installation are all open source software. The licenses can be found in the files /opt/oscar/packages/*/doc/license.tex. Most packages are GPL licensed. The programs which have a different license are:

• the MPI and PVM parallel libraries: have BSD or BSD alike licenses,
  
• Ganglia: is BSD licensed,
  
• Torque and Maui: have BSD-alike licenses.
  

Nagios is licensed under GPL2.

Technical Specifications

Operating System:

fedora core 5

VMware Tools installed: No

Size: 1464MB

Allocated Memory (RAM): 320

Applications Installed:

OSCAR 5.0 SVN r4853
and all applications included into it:
LAM MPI 7.1.2
MPICH 1.2.7
OpenMPI 1.0.2
Ganglia 3.0.3
SystemImager 3.7.3
SystemInstaller-OSCAR 2.2.3
SystemConfigurator 2.2.2-11ef
NetBootManager 0.8
In addition
yume 1.6
Nagios 1.3

Virtual Appliance Account Information