Managing submissions

Before submitting a sample you need the workflow registered, and the target environment enabled with an engine and a processing queue. The parameters file can be JSON or YAML.

List submissions

geni submission list

Use --help to see the filtering and sorting options — by provider, status, workflow, date range, environment, engine, queue or tag.

geni submission list --today --status failed
geni submission list --workflow wgs-germline --sort-by duration --sort-order desc

Submit a sample

First find the engine and queue available in the target environment:

geni engine list
geni queue list

Then submit:

geni submission create \
  --workflow-name <workflow-name> \
  --workflow-version <workflow-version> \
  --params-file <path to a JSON or YAML file> \
  --engine-id <engine-id> \
  --queue-id <queue-id> \
  --fallback-queue-id <fallback-queue-id> \
  --output-folder <path to a folder in a bucket>

Recommendations

--workflow-version is optional; without it the latest version is used. Always set it explicitly for production runs.

The input parameters file must be on your computer — the CLI uploads it to GENI.

The engine and queue must both be active and belong to the same environment. The engine needs read permission on every bucket holding input files, and write permission on the bucket named in --output-folder.

Output folder

Every submission must say where output files go, even if the workflow writes nothing. The path follows the cloud’s own convention — s3://bucket/prefix/. GENI passes it to Nextflow as both outdir and outputDir, so the workflow can derive per-task paths from it.

Use --cancel-time DD:HH:MM to cap the runtime of this submission. It overrides whatever the workflow version configures.

Use --tag key=value to label the submission. See Tagging.

Fallback queue

Pass --fallback-queue-id with an on-demand queue. If a job fails for spot-capacity reasons, Nextflow runs it again on the fallback queue.

Note

Cromwell does this on its own and does not need a second queue.

Cancel

geni submission cancel <submission-id>

GENI stops the Nextflow process and terminates every Batch job belonging to the submission.

Retry

geni submission retry <submission-id>

Optionally target a different engine or queue, or resume from the previous run’s work directory:

geni submission retry <submission-id> --queue-id <ondemand-queue-id> --resume

Logs and tasks

Nextflow logs for a submission:

geni submission logs <submission-id>
geni submission logs <submission-id> --follow

The tasks of a submission — add --status failed to see only the failures:

geni task list --submission-id <submission-id>
geni task list --submission-id <submission-id> --status failed

Logs of one task:

geni task logs <task-id>

Filter by tag, and show another tag as a column:

geni task list --tag <key1>=<value1> --show-tag <key2>

Output formats

Every list command accepts --format, which takes table (the default), csv or json.

geni submission list --today --format csv > runs.csv