Numpy job plugin¶
The Numpy plugin supports sending Numpy based python jobs to grid resources. The plugin supports submission of parameter sweeps with options of replacing parameters in the python files with information on, task id, job name and sweep size. Each specific job will be created in a folder inside the job definition directory.
Input files¶
Input file tab in the Numpy plugin.
Input files will be transferred to the selected resource by arcjobtool and can consist of python scripts, binary files and textfiles. The Input files tab always contains an inputfile called run.sh which is the man script that is executed by the job. run.sh is maintained by the plugin and should not be modified by the user.
Adding input files¶
Files are added by clicking on the Add button. This brings up a file selection dialog. Select the input file to be included with the job definition. When the file has been selected it is copied into the job definition by arcjobtool. This approach enables the user to move the entire job definition without any dependencies on local files.
Creating new input files¶
arcjobtool can also create new input files using the Create... button. Clicking on this button shows a dialog box asking for a filename. When the filename has been selected an system editor is shown and the file can be edited. arcjobtool will suspend its execution until the editor is closed.
Removing input files¶
An input file can be removed from the input file list by selecting it in the Input files listbox and clicking on Remove. This removes the input file from the job definition directory.
Clearing input files¶
All input files in the job definition can be cleared by clicking on the Clear button. This will remove all input files except the run.sh from the job definition directory.
Editing input files¶
Existing input files in the job definition can be edited by selecting the file in the Input files listbox and clicking on the Edit... button. This will bring up a system editor enabling the user to edit the input file.
Note
Editing of binary files is not supported. Editing a binary file in the system editor will only show “garbage” in the editor window. Please be careful not saving the binary file when it has been opened in the editor.
Assigning the main python file¶
Meaningful usage of this plugin requires that at least one of the input files is a main python file. To indicate main python script that should be executed by the job, use the Main python file drop down listbox to select the main script file.
Parameter sweep settings¶
Paramters tab in the script plugin.
The Parameters tab settings for the parameter sweep can be defined. arcjobtool can do parameter substitution of certain parameters in the script and input files. The following substitution parameters can be used:
| Parameter | Description |
| %(name)s | Parameter containing the job name |
| %(id)d | id of the current task 1-sweepsize |
| %(sweepSize)d | The size of the parameter sweep |
An example of a python-script with substitution parameters are shown in the following example:
from numpy import *
from pycalfem import *
# ----- ArcGui parameters ----------------------------------------
print "Jobname %(name)s"
print "Task id %(id)d"
print "Sweep size %(sweepSize)d"
arcjobtool will only do parameter substitution of files listed int the listbox Parameter substitution files.
Adding files for parameter substitution¶
Files which should be considered for substitution can be added by clicking on the Add... button. This will bring up a dialog showing the existing input files in the definition. Select the file to use for substitution in the list box and click OK
Removing files from parameter substituion¶
Files selected for parameter substitution can be removed by selecting the file in the Parameter substitution files listbox and clicking on the Remove button.
Clearing substitution files¶
All files selected for parameter substitution can be cleared by clicking on the Clear button.
Editing substitution files¶
Existing input files in the job definition can be edited by selecting the file in the Input files listbox and clicking on the Edit... button. This will bring up a system editor enabling the user to edit the input file.
Note
Editing of binary files is not supported. Editing a binary file in the system editor will only show “garbage” in the editor window. Please be careful not saving the binary file when it has been opened in the editor.
Setting the sweep size¶
The sweep size parameter controls how many jobs to submit in the parameter sweep. Changing this parameter will create a subdirectory in the job definition for each parameter. Each subdirectory is a single job that will be submitted to the grid. Changing the sweep size will automatically add and remove these directories. The sweep size is changed by changing the value in the Sweep size text box or using the spinner controls on the right side of the textbox.
Note
Currently arcjobtool has a limitation of 9999 jobs for a single parameter sweep.