Skip to content
GENI Workflows

Create an engine

An Engine configures which workflow orchestrator is used — Nextflow or Cromwell. That choice determines the supported workflow language: Nextflow or WDL respectively.

Engines do three more things:

  • Limit concurrency. An engine also acts as a queue, capping how many submissions run in parallel.
  • Scope bucket access. An engine restricts which GCS buckets the workflows it runs may touch — read only, write only, or read and write.
  • Hold scratch space. Each engine has its own GCS bucket for temporary files, with a configurable lifetime for those files.

Cost can be aggregated by engine as well as by queue. Creating engines and queues costs nothing, in cloud charges or in GENI Workflows licensing — so create as many as your work actually distinguishes between.

Create

Section titled “Create”
Terminal window
export ENVIRONMENT_ID=<environment-id>


geni engine create \
  --name "Nextflow" \
  --environment-id $ENVIRONMENT_ID \
  --engine-version 26.04.1-gcp \
  --read-write <gcs-bucket> \
  --read-only <gcs-bucket> \
  --write-only <gcs-bucket>
Argument Meaning
<environment-id> The ID GENI generated, for example 320e43e5-b171-4d0b-ada3-7cfcc71f6dd8
<gcs-bucket> A bucket the engine’s workflows may access, for example gs://geni-test-bucket

--read-write, --read-only and --write-only are repeatable, and each accepts a prefix as well as a whole bucket.

Next

Section titled “Next”

Create a queue.