Magus is the MidnightBSD build cluster for mports. It tests mports and builds packages for use with different releases. Any package included on the MidnightBSD FTP server after 0.1.1-RELEASE was built with magus.
Chris Reinhardt started working on magus to replace the manual procedure we were using to test ports. Upon joining the MidnightBSD project, Chris felt it was a lot of wasted effort to setup a machine and manually build software packages.
The first version was tested in 2007 after a transition to "mports."
EMU Donates Machines
Eastern Michigan University's computer science department donated ten systems to use for magus cluster nodes from their collection of old lab machines. Each system is a Dell Optiplex gx240 with 512MB of RAM and one or two hard drives. The systems contain Pentium 4 processors.
In November 2007, we were able to test a run on one version of MidnightBSD in about ten hours on ten machines. There were around 1600 ports at that time.
I was able to test magus on an amd64 install in December, 2007. The run took approximately 2 days on a Pentium D 805 using one core.
The build master was hosted at Eastern Michigan University until March, 2008.
Chris is working on a new version of magus capable of testing multiple versions of MidnightBSD (0.1, 0.2, etc) and architectures (i386, amd64, sparc64). As of March 12, 2008, the software is stable, and runs are active on the EMU machines.
Why is magus useful?
Magus allows us to test most ports automatically. It can detect errors downloading software (fetch errors), compile errors, and changes to ports that have broken other ports.
With magus, we can build software for releases automatically saving developer time. One can browse magus results at http://www.midnightbsd.org/magus/
How does magus operate?
Magus requires a MySQL server to operate. A build master maintains the MySQL database. Each client connects to the MySQL database to fetch jobs and create locks on ports. The lock prevents another machine from working on the same port.
Before a run, an indexer is run on the master on a fresh checkout of the mports CVS repo. This indexer builds a table and determines relationships between ports.
The nodes use ssh to transfer completed packages to a "cache host." In the original version, the build master was also the cache host. This was changed to allow stargazer to be the build master. All i386 runs will be done at EMU, but other architectures must be done elsewhere. Chris will handle sparc64 runs and I will be completing amd64 runs.
The magus client code can print debug information on runs. It is specialized for the mports system and has not been written as a general purpose tool.
Where is the magus source code?
Chris has committed the code to the cvs server at mports/Tools/magus
It is under the BSD License.
How do you setup a machine for magus?
There is a meta port at mports/ports-mgmt/magus-utils
Do you need more nodes?
We need more machines to build amd64 and sparc64 packages on.