woods.command_launchers module

Set of functions used to launch lists of python scripts

Summary

Functions:

`dummy_launcher`	Doesn't launch any scripts in commands, it only prints the commands.
`local_launcher`	Launch all of the scripts in commands on the local machine serially.
`slurm_launcher`	Parallel job launcher for computationnal cluster using the SLURM workload manager.

Reference

woods.command_launchers.dummy_launcher(commands)

Doesn’t launch any scripts in commands, it only prints the commands. Useful for testing.

Taken from : https://github.com/facebookresearch/DomainBed/

Parameters: commands (List) – List of list of string that consists of a python script call

woods.command_launchers.local_launcher(commands)

Launch all of the scripts in commands on the local machine serially. If GPU is available it is gonna use it.

Taken from : https://github.com/facebookresearch/DomainBed/

Parameters: commands (List) – List of list of string that consists of a python script call

woods.command_launchers.slurm_launcher(commands)

Parallel job launcher for computationnal cluster using the SLURM workload manager.

Launches all the jobs in commands in parallel according to the number of tasks in the slurm allocation. An example of SBATCH options:

#!/bin/bash
#SBATCH --job-name=<job_name>
#SBATCH --output=<job_name>.out
#SBATCH --error=<job_name>_error.out
#SBATCH --ntasks=4
#SBATCH --cpus-per-task=8
#SBATCH --gres=gpu:4
#SBATCH --time=1-00:00:00
#SBATCH --mem=81Gb

Note

–cpus-per-task should match the N_WORKERS defined in datasets.py (default 4)

Note

there should be equal number of –ntasks and –gres

Parameters: commands (List) – List of list of string that consists of a python script call