scine_puffin.jobs.templates.job

Functions

calculation_context(job[, stdout_name, ...])	A context manager for a types of calculations that are run externally and may fail, dump large amounts of files or do other nasty things.
is_configured(run)	A decorator to check if the job has been configured before running a method
job_configuration_wrapper(run)	A wrapping function for the run method of a Job instance

Classes

Job()	A common interface for all jobs in/carried out by a Puffin
breakable(value)	Helper to allow breaking out of the contex manager early

class scine_puffin.jobs.templates.job.Job

A common interface for all jobs in/carried out by a Puffin

archive(archive: str) → None

Archives all files existent in the job’s directory into tarball named with the job’s ID. The tarball is then moved to the given destination.

Parameters:

archivestr

The path to move the resulting tarball to.

capture_raw_output() → Tuple[str, str]

Tries to capture the raw output of the calculation context and save it in the raw_output field of the configured calculation. This should never throw.

Notes

• Requires run configuration

classmethod check_configuration(instance: T) → typing_extensions.TypeGuard[T]

static check_duplicate_property(structure: db.Structure, properties: db.Collection, property_name: str, model: db.Model) → db.ID | bool

Checks for a property that is an exact match for the one queried here. Exact match meaning that key and model both are matches.

Parameters:

propertiesdb.Collection (Scine::Database::Collection)

The collection housing all properties.

property_namestr

The name (key) of the queried property, e.g. electronic_energy.

modeldb.Model (Scine::Database::Model)

The model used in the calculation that resulted in this property.

structuredb.Structure (Scine::Database::Structure)

The structure to be checked in. The structure has to be linked to its collection.

Returns:

IDdb.ID (Scine::Database::ID)

Returns False if there is no existing property like the one queried or the ID of the first duplicate.

clear() → None

Clears the directory in which the job was run.

complete_job() → None

Saves the executing Puffin, changes status to db.Status.COMPLETE.

config: Configuration

configure_run(manager: db.Manager, calculation: db.Calculation, config: Configuration) → None

Configures a job for a given Calculation to do tasks in the run function

Parameters:

managerdb.Manager (Scine::Database::Manager)

The manager of the database holding all collections

calculationdb.Calculation (Scine::Database::Calculation)

The calculation to be performed

configConfiguration

The configuration of the Puffin doing the job

fail_job() → None

Saves the executing Puffin, changes status to db.Status.FAILED.

failed_file() → str

Returns the path to the file indicating a failed calculation, None if job has not been prepared

postprocess_calculation_context() → bool

Postprocesses a calculation context, pushing all errors and comments.

Returns:

True if the job succeeded, False otherwise.

prepare(job_dir: str, id: db.ID) → None

Prepares the actual job. This function has to be implemented by any job that shall be added to Puffins job portfolio.

Parameters:

job_dirstr

The path to the directory in which all jobs are executed.

iddb.ID (Scine::Database::ID)

The calculation that triggered the execution of this job.

abstract static required_programs() → List[str]

This function has to be implemented by any job that shall be added to Puffins job portfolio.

Returns:

requirementsList[str]

A list of names of programs/packages that are required for the execution of the job.

abstract run(manager: db.Manager, calculation: db.Calculation, config: Configuration) → bool

Runs the actual job. This function has to be implemented by any job that shall be added to Puffins job portfolio.

Parameters:

managerdb.Manager (Scine::Database::Manager)

The manager/database to work on/with.

calculationdb.Calculation (Scine::Database::Calculation)

The calculation that triggered the execution of this job.

configscine_puffin.config.Configuration

The configuration of Puffin.

stderr_path: str = 'stderr'

stdout_path: str = 'stdout'

store_property(properties: db.Collection, property_name: str, property_type: str, data: Any, model: db.Model, calculation: db.Calculation, structure: db.Structure, replace: bool = True) → db.Property | None

Adds a single property into the database, connecting it with a given structure and calculation (it’s results section) and also

Parameters:

propertiesdb.Collection (Scine::Database::Collection)

The collection housing all properties.

property_namestr

The name (key) of the new property, e.g. electronic_energy.

property_typestr

The type of property to be added, e.g. NumberProperty.

dataAny (According to ‘property_type’)

The data to be stored in the property, the type of this object is dependent on the type of property requested. A NumberProperty will require a float, a VectorProperty will require a List[float], etc.

modeldb.Model (Scine::Database::Model)

The model used in the calculation that resulted in this property.

calculationdb.Calculation (Scine::Database::Calculation)

The calculation that resulted in this property. The calculation has to be linked to its collection.

structuredb.Structure (Scine::Database::Structure)

The structure for which the property is to be added. The properties field of the structure will receive an additional entry, or have an entry replaced, based on the options given to this function. The structure has to be linked to its collection.

replacebool

If true, replaces an existing property (identical name and model) with the new one. This option is true by default. If false, doesnothing in the previous case, and returns None

Returns:

propertyDerived of db.Property (Scine::Database::Property)

The property, a derived class of db.Property, linked to the properties’ collection, or None if no property was generated due to duplication.

success_file() → str

Returns the path to the file indicating a successful calculation, empty string if job has not been prepared

verify_connection() → None

Verifies the connection to the database. Returns only if a connection is established, if it is not, the function will attempt to generate a connection every 10 seconds, indefinitely.

Notes

• Requires run configuration

work_dir: str

class scine_puffin.jobs.templates.job.breakable(value)

Helper to allow breaking out of the contex manager early

> with breakable(open(path)) as f: > print(‘before condition’) > if condition: > raise breakable.Break > print(‘after condition’)

exception Break

Break out of the with statement

args

with_traceback()

Exception.with_traceback(tb) – set self.__traceback__ to tb and return self.

scine_puffin.jobs.templates.job.calculation_context(job, stdout_name: str = 'output', stderr_name: str = 'errors', debug: bool | None = None) → Iterator

A context manager for a types of calculations that are run externally and may fail, dump large amounts of files or do other nasty things.

The executed code will be run in the working directory of the given job, the first exceptions will be caught and appended to the error output, the context will then close and leave behind a file called failed in the scratch directory. If no exceptions are thrown, a file called success will be generated in the scratch directory.

The output redirector part has been adapted from here [access date Jun 25th, 2019]

Parameters:

jobJob

The job holding the working directory and receiving the output and error paths

stdout_namestr

Name of the file that the stdout stream should be redirected to. The file will be generated in the given scratch directory.

stderr_namestr

Name of the file that the stderr stream should be redirected to. The file will be generated in the given scratch directory.

debugbool

If not given, will be taken from Job Configuration (config[‘daemon’][‘mode’]) If true, runs in debug mode, disabling all redirections.

Returns:

The context generates three files in the job.work_dir beyond any other

ones generated by the executed code.

The first two are the redirected output streams stderr` and ``stdout

(the name of these files are set by the context’s arguments), the third

file is either called failed or success depending on the

occurrence of an exception in the executed code or not.

scine_puffin.jobs.templates.job.is_configured(run: Callable[[T, P], AnyReturnType]) → Callable[[T, P], AnyReturnType]

A decorator to check if the job has been configured before running a method

scine_puffin.jobs.templates.job.job_configuration_wrapper(run: Callable)

A wrapping function for the run method of a Job instance

• Configures run (setting class members for specific database Calculation)

• Additionally Try/Catch safety to avoid dying pending jobs without error