GridSweeper design overview
Although XgridDRMAA has not quite stabilized yet, it’s time to move on to serious work on GridSweeper. (I’ll use it as a test suite for XgridDRMAA—I can run the code using Grid Engine’s DRMAA and XgridDRMAA, and problems in the latter will no doubt emerge.)
I spent a while at my whiteboard scrawling a mind map; here’s a simplified version in more legible form:
Development priorities for this software:
- Parameter control This is sort of the point: converting compact representations of parameter combinations into big long lists of parameters settings to be run.
- Plug-in interface This is how parameter settings get translated into control parameters for specific classes of models—e.g., Repast models, general command-line parameters, etc.
- Grid control This is the other part of the point: submitting lists of parameter settings to the grid. Very straightforward, thanks to DRMAA.
- CLI I need some way of interacting with the system (aside from writing new main() methods) as early as possible.
- Preferences Good to be able to save settings to shortcut things for both the CLI and the GUI—e.g.,
- File transfer interface Unfortunately, you can’t count on having a shared filesystem. (In fact, I don’t have a shared filesystem for my “grid” of two Macs.) So you need a way to transfer output files that aren’t stdin/out/err (which is provided for by DRMAA). I think the simplest solution is to just support FTP servers, my previous ramblings about having a custom file-transfer daemon notwithstanding. Most bang for my coding time, thanks to the Jakarta Commons Net FTP library.
- GUI This is the most open-ended component, so I’ll leave it for the end, and it can be as sophisticated or as simple as I have time for.