Nextflow engine version 26.04.6-2 is out, and it is now the default for new engines on AWS
and GCP. It fixes the case where resume didn’t work when retrying a submission previous canceled.
What 26.04.6-2 changes
The engine now uses the submission ID as the Nextflow session ID. Every attempt of a submission — the first run and every retry — shares it, so a retry finds the cache no matter how the previous attempt ended:
- cancelled by you (
geni submission cancel) - cancelled by its cancel time
- killed for any other reason, such as the head job’s instance going away
Only tasks that finished are reused. A task that was still running when the run stopped starts over on the retry.
We tested both cancel paths on AWS and on GCP.
Submissions that started on an older engine version are covered too. Their retry on
26.04.6-2 resumes the most recent cached session in the work directory, so a run that was
already cancelled once can still reuse its finished tasks after you update the engine.
Cancelling now stops the running tasks
Alongside the new engine, GENI fixed what happens to a submission’s tasks when it is
cancelled by its cancel time. Before, only the Nextflow head job was stopped. Tasks already
running on AWS Batch or Google Cloud Batch kept going, and billing, after the submission
showed CANCELLED, and their results were thrown away. The automatic cancel now stops those
tasks the same way geni submission cancel always has.
This part needs no engine update: it applies to every engine, on both clouds.
Update your existing engines
New engines get 26.04.6-2 by default. Existing engines keep the version they were created
with until you change it.
List your Nextflow engines and their versions:
geni engine list --engine nextflowUpdate each one:
geni engine update <engine-id> --engine-version 26.04.6-226.04.6-2 accepts everything 26.04.6-1 did, including
--nextflow-profile, so no workflow or command needs to
change.
More on engine updates: Update an engine. More on retries: Managing submissions.