tdub.batch

A module for running batch jobs (currently targets the US ATLAS BNL cluster).

Function Summary

add_condor_arguments(arguments, to_file)

Add an arguments line to a condor submission script.

condor_preamble(workspace, exe[, universe, …])

Create the preamble of a condor submission script.

create_condor_workspace(name[, overwrite])

Create a condor workspace given a name.

Reference

tdub.batch.add_condor_arguments(arguments, to_file)[source]

Add an arguments line to a condor submission script.

the arguments argument is prefixed with “Arguments = “ and written to to_file.

Parameters
  • arguments (str) – the arguments line

  • to_file (typing.TextIO) – the open file stream

Examples

>>> import tdub.batch as tb
>>> import shutil
>>> ws = tb.create_condor_workspace("./some/ws")
>>> with open(ws / "condor.sub", "w") as f:
...     preamble = tb.condor_preamble(ws, shutil.which("tdub"), to_file=f)
...     tb.add_condor_arguments("train-single ......", f)
tdub.batch.condor_preamble(workspace, exe, universe='vanilla', memory='2GB', email='ddavis@phy.duke.edu', notification='Error', getenv='True', to_file=None, **kwargs)[source]

Create the preamble of a condor submission script.

Extra kwargs create additional preamble entries. See the HTCondor documentation for more details on all parameters.

Parameters
  • workspace (str or os.PathLike) – the filesystem directry where the workspace is

  • exe (str or os.PathLike) – the path of the executable that condor will run

  • universe (str) – the HTCondor universe

  • memory (str) – the requested memory

  • email (str) – the email to send updates to (if any)

  • notification (str) – the condor notification argument

  • to_file (typing.TextIO, optional) – if not None, write the string to file

Returns

the submission script preamble

Return type

str

Examples

>>> import tdub.batch as tb
>>> import shutil
>>> ws = tb.create_condor_workspace("./some/ws")
>>> with open(ws / "condor.sub", "w") as f:
...     preamble = tb.condor_preamble(ws, shutil.which("tdub"), to_file=f)
...     tb.add_condor_arguments("train-single ......", f)
tdub.batch.create_condor_workspace(name, overwrite=False)[source]

Create a condor workspace given a name.

This will create a new directory containing log, out, and err directories inside. The workspace argument to the condor_preamble() function assumes creation of a workspace via this function.

Missing parent directories will always be created.

Parameters
  • name (str or os.PathLike) – the desired filesystem path for the workspace

  • overwrite (bool) – if True, an existing workspace will be overwritten

Raises

OSError – if the filesystem path exists and exist_ok is False

Returns

filesystem path to the workspace

Return type

pathlib.PosixPath

Examples

>>> import tdub.batch as tb
>>> import shutil
>>> ws = tb.create_condor_workspace("./some/ws")
>>> with open(ws / "condor.sub", "w") as f:
...     preamble = tb.condor_preamble(ws, shutil.which("tdub"), to_file=f)
...     tb.add_condor_arguments("train-single ......", f)