Nextflow config profiles

geni submission create now takes --nextflow-profile.

Nextflow’s -profile selects a named profiles { } block from a workflow’s own nextflow.config — a way to switch parameters or per-process resources for a run without maintaining a second copy of the pipeline. Nothing GENI-specific here, just standard Nextflow config bundled at the root of your workflow ZIP:

profiles {
  test {
    params.samplesheet = 's3://lab-data/test/samples.csv'
    process {
      withName: 'GATK_HAPLOTYPECALLER' {
        cpus = 2
        memory = '8 GB'
      }
    }
  }

  production {
    params.samplesheet = 's3://lab-data/production/samples.csv'
    process {
      withName: 'GATK_HAPLOTYPECALLER' {
        cpus = 16
        memory = '64 GB'
      }
    }
  }
}

GENI now passes -profile straight through:

geni submission create \
  --workflow-name wgs-germline \
  --workflow-version v2.1.0 \
  --params-file sample.json \
  --engine-id <engine-id> \
  --queue-id <queue-id> \
  --nextflow-profile production \
  --output-folder s3://lab-results/run-4471/

Pass more than one as a comma-separated list — test,docker — and Nextflow merges them left to right.

A profile can set params.* and per-process cpus/memory/container. It can’t redirect execution off the queue or engine you submitted to: GENI’s own generated config is loaded last and always wins on process.executor, process.queue, and the aws.*/google.* keys, the same precedence rule that already lets a workflow’s withName:/withLabel: selectors coexist with GENI’s settings (see Routing tasks to a different queue). Retries reuse whatever profile the original submission was created with — you don’t pass the flag again.

Minimum engine version

--nextflow-profile requires a Nextflow engine on version 26.04.6-1 or later. An engine still on an older image — 25.10.4 or 26.04.1-s3/26.04.1-gcp — rejects the flag with 422 before a submission is even created, so there’s no risk of it being silently ignored.

To move an existing engine up:

geni engine update <engine-id> --engine-version 26.04.6-1

See Update an engine for what that changes (and what it doesn’t).

Full details, including the ZIP-with-nextflow.config requirement: Nextflow config profiles.