tdub.batch¶
A module for running batch jobs (currently targets the US ATLAS BNL cluster).
Function Summary¶
|
Add an arguments line to a condor submission script. |
|
Create the preamble of a condor submission script. |
|
Create a condor workspace given a name. |
Reference¶
-
tdub.batch.
add_condor_arguments
(arguments, to_file)[source]¶ Add an arguments line to a condor submission script.
the arguments argument is prefixed with “Arguments = “ and written to to_file.
- Parameters
arguments (str) – the arguments line
to_file (typing.TextIO) – the open file stream
Examples
>>> import tdub.batch as tb >>> import shutil >>> ws = tb.create_condor_workspace("./some/ws") >>> with open(ws / "condor.sub", "w") as f: ... preamble = tb.condor_preamble(ws, shutil.which("tdub"), to_file=f) ... tb.add_condor_arguments("train-single ......", f)
-
tdub.batch.
condor_preamble
(workspace, exe, universe='vanilla', memory='2GB', email='ddavis@phy.duke.edu', notification='Error', getenv='True', to_file=None, **kwargs)[source]¶ Create the preamble of a condor submission script.
Extra kwargs create additional preamble entries. See the HTCondor documentation for more details on all parameters.
- Parameters
workspace (str or os.PathLike) – the filesystem directry where the workspace is
exe (str or os.PathLike) – the path of the executable that condor will run
universe (str) – the HTCondor universe
memory (str) – the requested memory
email (str) – the email to send updates to (if any)
notification (str) – the condor notification argument
to_file (typing.TextIO, optional) – if not None, write the string to file
- Returns
the submission script preamble
- Return type
Examples
>>> import tdub.batch as tb >>> import shutil >>> ws = tb.create_condor_workspace("./some/ws") >>> with open(ws / "condor.sub", "w") as f: ... preamble = tb.condor_preamble(ws, shutil.which("tdub"), to_file=f) ... tb.add_condor_arguments("train-single ......", f)
-
tdub.batch.
create_condor_workspace
(name, overwrite=False)[source]¶ Create a condor workspace given a name.
This will create a new directory containing log, out, and err directories inside. The workspace argument to the
condor_preamble()
function assumes creation of a workspace via this function.Missing parent directories will always be created.
- Parameters
name (str or os.PathLike) – the desired filesystem path for the workspace
overwrite (bool) – if True, an existing workspace will be overwritten
- Raises
OSError – if the filesystem path exists and exist_ok is False
- Returns
filesystem path to the workspace
- Return type
Examples
>>> import tdub.batch as tb >>> import shutil >>> ws = tb.create_condor_workspace("./some/ws") >>> with open(ws / "condor.sub", "w") as f: ... preamble = tb.condor_preamble(ws, shutil.which("tdub"), to_file=f) ... tb.add_condor_arguments("train-single ......", f)