The Nextflow config file

The information about the resources needed, the scheduler used and the container options can be stored within the nextflow.config file. In this file we just include a number of single configuration files, each one for a different profile for different executors.

Here you the single config files that are included by the main one and used depending on the different Nextflow parameter -profile

Standard (default)

/*
* This configuration file is the default one used by the pipeline
*/

process {
    // definition of the local executor. Run the pipeline in the current computer.
    executor="local"

    // resources for default process execution
    memory='0.6G'
    cpus='1'
    time='6h'

       // resources for execution of processes / modules with the label "two cpus". This override the default ones.
        withLabel: 'twocpus' {
            memory='0.6G'
            cpus='2'
        }
}

SGE

You can activate using this command line:

nextflow run lucacozzuto/elixir_NF -r main -profile hpc_sge -with-docker

/*
* This configuration file is the one used when indicating the Nextflow parameter -profile hpc_sge
*/

process { 
    // definition of the SGE executor. Run the pipeline in a node able to submit jobs to a HPC via qsub
    executor="SGE"       

    // definition of the default queue name. 
    queue = "smallcpus"

    // resources for default process execution
    memory='1G'
    cpus='1'
    time='6h'
    
        
       // resources for execution of processes / modules with the label "two cpus". This override the default ones.
        withLabel: 'twocpus' {
           queue = "bigcpus"
           memory='4G'
           cpus='2'
       }   

} 

SLURM

You can activate using this command line:

nextflow run lucacozzuto/elixir_NF -r main -profile hpc_slurm -with-docker

/*
* This configuration file is the one used when indicating the Nextflow parameter -profile hpc_slurm
*/

process { 
    // definition of the slurm executor. Run the pipeline in a node able to submit jobs to a HPC via sbatch
    executor="slurm"       

    // resources for default process execution
    cpus='1'
    time='6h'

       // resources for execution of processes / modules with the label "two cpus". This override the default ones.
        withLabel: 'twocpus' {
           queue = "bigcpus"
           cpus='2'
        }   

} 

AWS BATCH

You can activate using this command line:

nextflow run lucacozzuto/elixir_NF -r main -profile cloud -with-docker

/*
* This configuration file is the one used when indicating the Nextflow parameter -profile cloud
*/

// Here we define some AWS parameters like the region, the aws executables and 
// the S3 bucket for being used as work directory for intermediate files
aws.region = 'eu-central-1'
aws.batch.cliPath = '/home/ec2-user/miniconda/bin/aws'
workDir = 's3://class-bucket-XXX/work'

process {
    // definition of the awsbatch executor. Run the pipeline in a AWS node able to submit jobs via batch submission
    executor = 'awsbatch'

    // definition of the default queue name. 
    queue = 'spot'

    // resources for default process execution
    memory='1G'
    cpus='1'
    time='6h'

       // resources for execution of processes / modules with the label "two cpus". This override the default ones.
       withLabel: 'twocpus' {
          memory='0.6G'
          cpus='2'
       }
}