Snakemake for HPC Workflows

Event content

Current HPC workflows can become quite complex, beyond the scope of simple bash scripting or scheduler job dependencies. Snakemake introduces robust tools to generate elaborate workflows, including stage-in and -out options, and interaction with different job schedulers. Snakemake is particularly popular in the field of Bioinformatics, so researchers of this field will profit from attending the course. AI/ML is another discipline that can benefit from organizing job dependencies. In general, the content and experience gathered in this course should be applicable to any discipline where complex interactions between files and jobs and different stages of analysis are involved.
The course will consist of an introduction to Snakemake, its concepts and capabilities, as well as 1 or 2 exercises where users will have to develop and apply their own workflows for a Bioinformatics and/or AI/ML task.

Learning Goal

• How to use snakemake to generate hpc workflows
• Examples from bioinformatics and machine learning fields, but content should be applicable to any discipline

Information about the event

Max. participants

20

Requirements

• Experience with working in HPC systems and with job schedulers is required, such as the Introduction to HPC courses regularly offered by the GWDG, or first-hand usage of the SCC cluster.
• Additionally, the basics of Python and command line usage are assumed, as well as how to edit a text file in-terminal or with an IDE with some connection to the cluster.
• Git or similar version control is not strictly required, but would help users manage their workflows and exercises.
• Course accounts or access to the SCC required.

Speakers

Martin Leandro Paleico