Next Previous Contents

2. Implementation

2.1 Why Java?

Java is a robust and mature programming environment and provides a good platform for rapid application development.

In addition to the "free-as-in-beer" JDK (Java Development Kit) from Sun Microsystems, there are a number of "free-as-in-liberty" JVMs (Java Virtual Machines) available for UNIX-like platforms.

Java has also been criticized for being slow. This criticism is still valid if you're writing extremely time-critical software, (like streaming video or machine emulation code), but Java runs quite well on today's machines for most things, including a parallel batch scheduling system.

2.2 What's this about Python code?

Although the scheduler is implemented in Java, you can write your own components to override or enhance the scheduler in the Python language. Python is a natural language for high performance computing due to it's language and library support for numerical methods. Thanks to the Jython project, now you can run your Python scripts inside the Java runtime. We even supply example python code that can be slipped into the main scheduling loop of the Scheduler daemon!

2.3 Do I have to install a JVM (Java Virtual Machine) on all the nodes in my cluster?

The quick answer is yes. The long answer is... If you intend to roll-your-own for some of the subsystems, (namely the node daemon), then no; for example you could rewrite the node daemon in another programming language so you didn't have to use the JVM. (But at this point, maybe you really just want to write your own scheduler and resource manager anyway and we can agree upon a common communications protocol :)

2.4 Where do I install the various RPMs?

The Maui Scheduler "server" RPM should be installed on one node of your cluster that you've decided should be the control node.

The Maui Scheduler "client" RPM should be installed on interactive nodes of your cluster from which users will submit jobs and query system status.

The Maui Scheduler "drone" RPM should be installed on all compute nodes of your cluster where batch jobs are run.

2.5 mauid user issues

The mauid user is created on-the-fly when the RPMs are installed on each machine. If you use NIS, you may want to set up this user before installing the scheduler, and as long as NIS is exported properly to all nodes in your cluster, the scheduler will used the supplied user.

Otherwise, a mauid user is created locally on each machine with the next available UID. Note that this could lead to UID mismatch across nodes. But some people think this is a feature...

2.6 What network protocols does the Maui Scheduler use?

Between client commands and server, the scheduler uses an XML-based protocol. The client sends a request in the form of an XML document and the server responds in kind. These documents are encrypted when sent over the wire. You can read more about the protocol here.

Between server and node daemons, the scheduler currently sends serialized Java objects (also encrypted by the same mecanism as client/server protocol). Although this protocol is lightweight and efficient, we prolly need to design a more generic protocol so you can write your own node daemon objects... Anyway, you can weigh in on this issue on one of the mailinglists!

2.7 Do I have to use a MySQL database?

Currently, yes. In the future we may implement hooks in the scheduler to allow you to use a different database (like Oracle or Postgres). The Maui Scheduler has been designed to communicate directly with a database through an abstraction layer (and currently MySQL is the only one implemented).

2.8 Can I use a MySQL database on another machine besides the one running the Maui Scheduler server?

Yes you can. Remember to change the locations specified in the maui.properties file. In addition you should make sure that the mauid user has the appropriate access credentials to access the database from a remote machine.


Next Previous Contents