- We are adopting topic channels to replace the versions.yml.
- For now both options are valid, but this will change.
- No more mixing of version channels.
What are topic channels
Say goodbye to writing a version.yml for each module and mixing the channels together. To improve here, we will be adopting topic channels. Topic channels can collect values from multiple sources automatically, just by sharing the same topic name. This means no more tangled channel wiring or mix operator chains. With topic channels, you can now broadcast and collect data across your pipeline more naturally.
process {
output:
val('hello'), topic: my_topic
// ...
}
process bye {
output:
val('bye'), topic: my_topic
// ...
}See also the channels documention and the migration tutorial
How to adopt your module
- Adapt main.nf
- Perform semi-automatic meta.yml update
- Catch version in tests/main.nf.test
- Update snapshot
- Fix dependent nf-core/subworkflows
Let’s get to work. Grab any nf-core module
1. Adapt main.nf
No need to write to version.yml files any more, we emit a topic channel in the output section instead.
output:
tuple val(meta), path("*.html"), emit: html
tuple val(meta), path("*.zip") , emit: zip
-path "versions.yml" , emit: versions
+tuple val("${task.process}"), val('fastqc'), eval('fastqc --version | sed "/FastQC v/!d; s/.*v//"'), emit: versions_fastqc, topic: versions
︙
-cat <<-END_VERSIONS > versions.yml
-"${task.process}":
- fastqc: \$( fastqc --version | sed '/FastQC v/!d; s/.*v//' )
-END_VERSIONS2. Semi-automatic meta.yml update
Update the meta.yml using nf-core/tools. Then fill in the version information.
You can update the meta.yml conveniently using
nf-core modules lint <MODULE_NAME> --fix
versions_fastqc:
- - ${task.process}:
type: string
description: The process the versions were collected from
- fastqc:
type: string
description: The tool name
- fastqc --version | sed "/FastQC v/!d; s/.*v//":
type: string
description: The command used to generate the version of the tool
topics:
- versions:
- - process:
type: string
description: The process the versions were collected from
- tool:
type: string
description: The tool name
- version:
type: string
description: The version of the tool3. Catch version in tests/main.nf.test
This might be a little more tricky for some of your modules, but a good start to catch the version in the snapshot is:
process.out.findAll { key, val -> key.startsWith("versions")}4. Update snapshot
As no version.yml exists anymore, you will have to update the snapshot.
5. Fix dependent nf-core/subworkflows
All nf-core/subworkflows which are calling your module will fail now because no version.yml is created to use the mix operator on.
So, you should remove all occurences of:
ch_versions = ch_versions.mix(FASTQC.out.versions.first())Make sure to update the snapshots of the subworkflows as well.