The Nextflow Podcast (Seqera)

In this episode of the Nextflow Podcast, we dive into highlights from the recent Nextflow Summit held in Barcelona in 2024. The summit featured an incredible range of talks, demonstrations, and discussions showcasing the latest advancements in workflows, data analysis, and community collaboration. With so much exciting content to explore, we’ve picked out some of our favourite moments to share with you. From impactful new features to inspiring keynote addresses and humorous presentations, this episode offers a curated selection of clips that capture the energy and innovation from the event. Join us as we relive the summit’s best bits and discuss their significance for the Nextflow community. 00:00 Start 00:10 Podcast introduction 00:47 Introductions: Group 1 01:56 Clip 1: Wave mirroring 04:32 Clip 2: Throwing cat food 06:00 Clip 3: crisprseq 08:03 Clip 4: nf-test 10:34 Clip 5: Workflow output definition 12:39 Clip 6: Open Science Software Foundation launch 15:08 Clip 7: FAIR code with nf-core 16:58 Clip 8: Teaching with Nextflow 18:56 Clip 9: Conda lock files 21:24 Clip 10: MultiQC in scripts 22:43 Clip 11: Small Nextflow Big Nextflow 25:17 Introduction: Team 2 26:11 Clip 12: Genomics England 29:47 Clip 13: Alphafold interactive reports 33:26 Clip 14: Data Studios 35:32 Clip 15: Covid and Ebola 37:35 Clip 16: Seqera AI 41:34 Clip 17: Nextflow VSCode DAGs 44:33 Clip 18: Fusion snapshots 48:38 Clip 19: Sustainable computing 52:38 Clip 20: nf-core/tools 56:37 Clip 21: Industry best practices with nf-core 58:58 Clip 22: Federated data queries 01:01:56 Clip 23: Nextflow Ambassadors 01:05:04 Conclusion

In this "News & Views" episode of Channels we talk about improvements to handling S3 file names, support for the Spack package manager, and local hackathon sites.

In this episode we explore the new features of Seqera's Data Studios and Data Explorer, with Phil Ewels, Rob Newman and Rob Syme from Seqera.

Discover how to use these tools for troubleshooting Nextflow pipelines, tertiary analysis and Nextflow development. We discuss the pain points that led to the creation of Data Studios and how it's designed to allow scientists to interactively and collaboratively work with data and complex workflows, without having to move large datasets around.

Rob Syme wows us with another fantastic practical demonstration, setting up and using Data Studios to write and test a Nextflow pipeline in VSCode running on the cloud in a Data Studio environment, including running the Nextflow CLI with task submission to AWS Batch.

We cover features like session persistence to save work states, and upcoming custom container support for your own specialized applications.

Learn how these tools can enhance your computational biology projects and make seamless cloud integration a reality.

00:00 Channels Podcast 43: Data Studios 00:26 Introductions 01:54 Data Studios 04:51 Move the compute to the data 06:13 Real-time collaboration 06:47 Data Explorer 09:41 Access to public data 10:45 Data Explorer demo 13:56 Data Studios setup 20:17 Session persistance 22:52 Data Studios RStudio demo 28:24 Nextflow development in Data Studios 36:17 Future development 37:01 Custom containers 40:01 Boston Summit demo 44:01 Lifetime management 47:14 Wrap up

In a special edition of the Channels Podcast, Phil Ewels chats to Geraldine Van der Auwera! Recorded in-person at the Nextflow Summit Barcelona, we hear a bit about Geraldine’s background - her work at the Broad with GATK and WDL, and the path that brought her to Seqera and the Nextflow community. Oh, and the WDL runner whose name got away is miniWDL, by Mike Lin!

Join us in this episode of the Channels podcast for a closer look at the Nextflow Ambassadors Program. Marcel Ribeiro-Dantas and Geraldine Van der Auwera discuss the origins of the program, recent highlights and next steps, including the open call for new Ambassadors, which will close June 14.

Relevant links:

• Call for applications: https://www.nextflow.io/ambassadors.html
• Original announcement: https://www.nextflow.io/blog/2023/introducing-nextflow-ambassador-program.html
• Marcel’s favorite Ambassador-contributed Nextflow blog post: https://www.nextflow.io/blog/2024/reflections-on-nextflow-mentorship.html

Join us in exploring the latest Nextflow release, 24.04. Phil Ewels and Ben Sherman dive deep into the new updates in Nextflow’s most recent version. We discuss advanced retry strategies, job arrays, resourceLimits, Singularity’s OCI mode, and the game-changing Workflow Output Definition.

We’re actively seeking feedback from the community on these new features. We would love for you to experiment with the new syntax and let us know your thoughts.

Please let us know what you think either via Nextflow GitHub issues or via the community forum.

In this episode we refer to a the recent blog post about the release, which you can find here: Nextflow 24.04 - Release highlights.

Intro - tweaks, fixes and new support in 24.04

Performance and Stability Improvements

Ben kicks off the discussion with key performance and stability improvements. The latest stable release, 24.04, includes numerous bug fixes and performance enhancements. There’s a significant focus on closing gaps and adding retry strategies for better stability, especially concerning cloud providers. Notably, if Nextflow’s API calls against cloud providers fail for any server-side reason, it will retry automatically, avoiding pipeline failures caused by temporary server issues.

Publishing Changes

Previously, if a file failed to publish, Nextflow would only issue a warning. It was possible for a pipeline to complete successfully without noticing missing output files. Now, by default, Nextflow will fail the pipeline if the publishing fails, though there’s an option to revert to the old behavior. Additionally, retry strategies for publishing have been introduced to ensure retry attempts if an issue arises.

Job Arrays

One highly requested feature is the introduction of job arrays. This allows users to submit many jobs as a single submission, alleviating strain on schedulers. The submission happens in a batch, and then the scheduler can process and plan effectively. Once a job is submitted as part of a job array, the jobs run independently. If any child job fails, it is resubmitted without affecting the rest of the array.

Singularity OCI mode and GA4GH TES

In the past, Singularity supported Docker images by converting them into SIF files, consuming storage and time. Now, both Singularity and Apptainer can run OCI images directly, saving valuable resources.

Additionally, the TES executor has seen significant improvements, now supporting TES 1.1, which brings broader compatibility and integration with existing workflows.

Major new syntax features

Topic Channels

A new feature called topic channels offers a more straightforward approach to collecting channel outputs across pipelines. Channels can emit data to a named topic, simplifying the collection and use of version information from various processes within Nextflow pipelines.

Eval Outputs

Eval outputs simplify the addition of shell commands to tasks. With eval outputs, necessary post-task commands can be defined neatly, avoiding repetitive code within process definitions.

Workflow Output Definition

The concept of workflow output definitions has been introduced. This new syntax streamlines the publication of files by defining publish targets within workflows. Instead of defining publication behaviors within process definitions, users can now manage them at the workflow level, ensuring better clarity and fewer repetitions in the code.

A special edition of Channels, Phil Ewels talks to Paolo Di Tommaso and Evan Floden about the past, present and future of Nextflow as the project celebrates its 10th anniversary.

Today in the Channels podcast, we're talking about computing resources... Ben Sherman takes us through new and upcoming runtime improvements in the Nextflow, leading to better performance fine tuning and simpler syntax for pipelines.

We cover:

• Fractional CPU requests
• resourceLimits directive (to replace check_max)
• Virtual threads in Groovy
• WebAssembly runtime (a lightweight alternative to containers)

Evan Floden (CEO and co-founder of Seqera) joins Phil Ewels (Product Manager for OSS at Seqera) to talk about two of the big announcements at the recent Nextflow Summit Boston 2024 - Seqera Pipelines and Seqera Containers. We dive into what these two new community offerings are, why we built them and how they work. Links:

• https://seqera.io/pipelines/
• https://seqera.io/containers/
• Announcement blog post
• Nextflow Summit Boston 2024 agenda
• Summit YouTube playlist
• Nextflow 24.04 blog post

Timeline:

00:00 - Introduction

00:37 - Nextflow Summit Boston 2024

01:14 - Summit talks available online

01:57 - Summit announcements

04:32 - Seqera Pipelines - motivations

05:59 - Seqera Pipelines - first look

06:48 - Testing requirements for community pipelines

10:05 - How we tested pipelines

11:53 - SRA-Explorer tangent

12:34 - Seqera Pipelines - live demo

16:14 - Getting Nextflow launch commands

19:12 - Seqera Containers - intro

24:03 - Seqera Containers - live demo

27:13 - Containers - ARM cpu arch builds

28:33 - Singularity containers

29:09 - Wave CLI

31:19 - Nextflow with Seqera Containers

33:22 - Seqera Containers - future usage

35:09 - Where to read more

36:53 - Thank you to the community

In this "News & Views" episode of Channels we talk about how support for Nextflow DSL1 is going to be dropped and what that means for you. We also explore the wondrous world of AI and ML with a few examples of recent pipelines we've written.

In this "News & Views" episode of Channels we talk about the new Sarus container engine, staging outputs using 'rclone' and 'fcp', and using GITHUB_TOKEN to access private repositories.

In this milestone 50th episode of the Nextflow podcast, host Phil Ewels sits down with Krešimir Beštak, a PhD student and active contributor to the nf-core community, to explore an exciting frontier in bioinformatics: microscopy and spatial omics.

While Nextflow is traditionally associated with genomics workflows, Krešimir shares how it’s being used to power image analysis pipelines like MCMICRO, supporting complex research into cardiovascular disease and cancer diagnostics. Based at the University Hospital Heidelberg, Krešimir discusses his transition from master’s student to PhD researcher, the translational applications of spatial proteomics, and how Nextflow enables reproducible workflows far beyond its original scope.

00:00 Podcast Ep 50: Krešimir
00:09 Welcome and introductions
03:13 Introduction to Spatial Omics
04:26 Multiplexing markers
06:38 Metabolite microscopy
07:23 Myocardial multiomics
08:58 Microscopy data analysis
11:32 nf-core/mcmicro
13:05 2D vs 3D microscopy
13:38 Computational bottlenecks within analysis
15:43 Other nf-core imaging pipelines
18:36 Downstream analysis after molkart
19:53 Manual interventions
23:27 Minerva
25:39 Google Maps for cells
27:16 Microscopy community around Nextflow
30:34 How to get involved
31:38 Changes in Nextflow for microscopy
32:43 Nextflow Ambassador program
33:17 Conclusion

In this episode of the Channels podcast, it’s all about developer experience! Ben Sherman takes us through new and upcoming language improvements in the Nextflow language which will help people writing Nextflow pipelines write code faster, with fewer errors, and of course with beautifully formatted indentation 🤩

We cover:

• What exactly was happening in the last episode, when Marcel, Chris and Phil ran wild with a beta version of breakpoint debugging in IntelliJ IDEA.
• Why informative error messages are difficult in Nextflow, and how we’re making things better (coming to your console in the next stable release!)
• How we’re building a Nextflow language server, for linting, formatting and code completion in your favourite IDE.

The first episode of a new format - meet Channels: the nextflow podcast. We'll be joining Rob Lalonde in bringing podcast episodes to your feeds with short News & Views episodes where you can hear us discuss hot topics in the Nextflow community.

Join us in this episode of the Channels podcast to get a recap of some of the biggest features to be added to Nextflow in 2023 and take a look at some of the things coming in 2024.

We tried to do this in Episode 27 but ended up spending nearly all the time discussing community and nf-core, so this episode is dedicated to just Nextflow features.

We cover Phil's top hits:

• 2023
  • Fusion support on Azure Batch, Google Batch, SLURM, LSF
  • Spack integration
  • Markdown docs, developer docs
  • New `nextflow inspect` command
  • Channel "topics"
  • AWS Fargate for compute tasks
• 2024
  • Job arrays
  • Garbage collection (aka work directory cleanup)
  • Command line interface v2
  • Improvements to Nextflow packaging
  • Workflow inputs and outputs schema
  • Module configuration / config v2

Today in the Channels podcast, we're talking about what happened this year in the Nextflow world, with particular emphasis on the many exciting developments in nf-core. We cover: - Community growth (with stats and graphs!) - The evolution of nf-core governance - The latest crop of nf-core pipelines that saw their first release this year and the expansion into domains beyond genomics (even beyond biology!) - Infrastructure updates including subworkflows and testing with nf-test - Community updates including mentorships and training - Upcoming events in 2024: nf-core online hackathon in March, Boston Summit in May and Barcelona Summit in October After today’s episode, we’ll be going on break for the winter holiday and will return in January 2024 with a fun mix of topics. Stay tuned!

A small but intrepid team of Seqerans recently attended the ISMB 2024 conference in Montreal, a great mix of computational biology and bioinformatics with a largely academic / research-focused audience. Join Geraldine Van der Auwera, Rob Syme and Florian Wuennemann for a lively discussion about the scientific themes of the conference, their experience running the booth, meeting with the ISMB and BOSC community, and resources for job seekers.

Tune to to Episode 16 of the Channels podcast to hear from the nf-core community infrastructure team: Matthias Hörtenhuber and Júlia Mir Pedrol. In a bumper episode, we discuss the new "nf-validation" Nextflow plugin (https://nextflow-io.github.io/nf-validation/) and talk nf-core components: modules and subworkflows.

In this episode of Channels, we talk to Josh Chorlton - CTO & cofounder of BugSeq Bioinformatics Inc. We talk about how BugSeq got into using Nextflow and MultiQC and the tips and tricks that they've employed to push scale and performace of their tools to the limit. We cover topics like testing and CI/CD, passing around structured data objects between Groovy and Python using Protobufs, building Docker images at scale and the ins and outs of using MultiQC for custom clinical data reports.

In this episode of the podcast, Geraldine and Ben talk about the Global Alliance for Genomics and Health (GA4GH) and its efforts to develop interoperability standards for pipelining infrastructure. If the words TES, WES, TRS and DRS are just as much alphabet soup to you, tune in to learn all about how organizations around the world are collaborating to streamline the process of sharing and analyzing data at scale.

Join us to hear Phil Ewels chat with Maxime Garcia and Franziska Bonath about how nf-core started at the National Genomics Infrastructure (NGI) at SciLifeLab in Sweden.We chat about how the nf-core/sarek pipeline grew alongside and eventually joined nf-core,as well as the early days of nf-core bytesize talks and outreach organisation. Links: - SciLifeLab - National Genomics Infrastructure - Maxime's Nextflow Summit Boston 2023 presentation about Sarek - nf-core bytesize talks

In this Podcast, Rob Lalonde of Seqera Labs speaks with Dr. Evan Floden, CEO and co-founder of Seqera Labs. Rob and Evan talk about the history of Nextflow, breaking bad, and the future of bioinformatics. Evan also provides a live demo of Nextflow Tower.

Software containers using Docker / Singularity are a core part of modern data science, found at the heart of Nextflow pipelines. On today's show, we bring on the creator of Nextflow – Paolo Di Tommaso – to tell us the good, the bad and the ugly of containers.

He talks about "Wave", one of the newest tools to come out of Seqera Labs and how this can make Nextflow pipelines scale on the cloud, work behind closed doors in protected environments and how it can build & augment images with additional requirements on the fly. And all this whilst reducing the amount of boilerplate code required within your pipeline.

As if that wasn't enough, we get a live demo of Wavelit - the open source command-line interface tool, that allows you to leverage the full power of Wave as a stand-alone tool, outside of Nextflow.

In today's episode of the Channels Podcast, we talk to Seqera software engineer Ben Sherman about what's going on "under the hood" with Nextflow. We touch on task provenance, cloud cache databases and task / sub-workflow grouping. Gotta cache 'em all! 0:00 - Introduction 2:21 - Task provenance 14:32 - Difference to the nf-prov plugin 17:13 - Quilt packages 22:30 - Cloud cache database 32:16 - Task grouping (horizontal) 45:18 - Workflow grouping (vertical) 1:00:22 - Upcoming events 1:03:31 - Close

In episode 44 of the Nextflow Channels podcast, Phil Ewels welcomes Alex Pelzer— a key figure in the Nextflow and nf-core community. Alex shares his journey from academia to the pharmaceutical industry, highlighting the critical role of regulatory frameworks in bioinformatics.

🌟 Key Highlights:

• The transition from working at QBiC to Boehringer Ingelheim.
• The importance of maintaining high-quality, reproducible pipelines in bioinformatics.
• Understanding regulatory requirements and their impact on clinical trials.
• How regulatory standards and open-source initiatives like @nf-core are shaping the future of bioinformatics.

Alex also discusses the formation of the @nf-core Regulatory Special Interest Group, which aims to bring together experts in the field to establish and promote best practices for ensuring pipeline quality and compliance with regulatory standards.

Looking forward to seeing many of you at the upcoming #NextflowSummit in Barcelona this October! We’ll be working with this #regulatory group during the nf-core hackathon.

00:00 Episode 44: Regulatory revolution 00:06 Introductions 00:33 Upcoming Nextflow Summit Announcement 01:34 Alex’s Background and Early Days with Nextflow 04:41 Formation and Growth of nf-core 06:18 Transition to Pharma and Regulatory Work 08:56 Regulatory Challenges and Practices in Bioinformatics 13:30 Tool Selection and Quality Assurance 15:23 Evaluating Software Trustworthiness 17:08 Overhead in Academia vs. Industry 18:11 Downstream analysis 20:57 Regulatory Special Interest Group 25:28 Special Interest Groups in nf-core 28:59 Future Directions and Community Involvement 30:15 Open source contributions in pharma 33:27 Conclusion and Upcoming Events

Join us as we discuss the brand new @nf-core / EuroFaang pre-print! 🔥 We chat to some of the key authors, covering the background to the paper, the main messages and how EuroFAANG and nf-core have collaborated. 🧬 👩🏻‍🔬

In this Podcast, Rob Lalonde of Seqera Labs speaks with Lorena Pantano, head of Computational Biology at NextRNA, a firm leading the next revolution in RNA-directed therapeutics located in Cambridge, Massachusetts. In this fascinating discussion, Rob and Lorena talk Bioinformatics, life in a startup, and women in technology.

Rob Lalonde of Seqera interviews Kelsey Florek, Senior Genomics and Data Scientist at the Wisconsin State Laboratory of Hygiene. In this interview, Rob and Kelsey discuss how genomic surveillance has changed since the COVID pandemic, and how Seqera Platform and Nextflow pipelines have substantially lowered the barrier to entry for bioinformaticians running workflows in public health labs and in the field. The viralrecon pipeline Kelsey mentioned in the interview can be found at https://nf-co.re/viralrecon

Join host Phil Ewels in Episode 49 of the Nextflow Podcast as he interviews Dr. Olga Botvinnik, founder and CEO of Seanome. Olga discusses her journey from studying at MIT to founding her non-profit research institute focused on marine genomics.

Discover her groundbreaking work in annotating unknown proteins and her efforts in collaborating with the Earth BioGenome Project.

Olga also shares her experience with the Nextflow and nf-core communities, and her vision for developing tools to aid research in understudied ocean organisms.

Join us as we chat about the vast potential of marine genomics and its applications.

00:00 Podcast Episode 49: Olga Botvinnik / Seanome

00:06 Welcome

01:18 Olga's background

07:30 Colonial tunicates

09:38 Starting Seanome with Arctic clams

11:07 Vision for Seanome

12:46 Earth BioGenome Project

13:46 Animal genomics not metagenomics

14:53 Building nf-core/proteinannotator

19:46 Software engineering in bioinformatics

22:21 nf-core early days

27:46 Bioinformatics Beyoncé

30:50 Seanome future steps

32:10 Final thoughts: Why the ocean

34:19 Goodbyes

In episode 47 of The Nextflow Podcast, Phil Ewels and Ben Sherman discuss the launch of the new Nextflow language server, a significant upgrade providing advanced code intelligence features such as code completion and error hints for VS Code users. They detail the benefits of formalizing Nextflow as its own programming language, reducing reliance on Groovy, and improving error messages and code clarity. They also cover new documentation, plans for future features like type annotations, and encourage community feedback and adoption. Find the new docs here: https://www.nextflow.io/docs/latest/vscode.html 00:00 Podcast Ep 47 - Version 2 - VSCode Extension 00:09 Welcome and introductions 00:44 Podcast rebranding 01:29 Nextflow Summit announcements 02:48 Language server background 06:27 Nextflow as a programming language 09:58 Building a language server 14:12 VS Code interactions with the language server 15:22 VS Code demo 15:50 Errors + warnings tray 17:10 Errors: Level 0 20:00 Errors: Level 1 22:32 Errors: Level 2 23:02 To def or not to def 24:46 Variable references 25:34 Number of arguments 25:57 Hover hints on workflows and processes 26:19 Code navigation 26:52 Output hints 27:17 Code peeks 27:56 Getting help 28:29 Code formatting 30:25 Nextflow schema params 32:03 Future warnings 34:02 Preview DAG 34:54 Fixing new syntax errors 35:42 Config files 37:07 Updating the plugin 37:33 Roadmap: Type checking 38:55 Conclusion

In this News and Views episode, Phil Ewels talks to Ben Sherman about upcoming features in core Nextflow development: a new CLI v2 interface; submission of job arrays and intermediate file cleanup during runs.

In this episode of Channels, we talk to Jakob Zeitler - Head of R&D at Matterhorn Studios. We dive into how they use Nextflow and Seqera Platform for material science research, paving the way for cheaper and more eco-friendly products for the future ⚗️👩🏻‍🔬🔬🌎 It may not be bioinformatics, but there's a lot that's in common!

In this "News & Views" episode of Channels we kick off the Summer 2023 series of Channels podcasts and go through some of the highlights from the latest Nextflow releases.

In a special edition of the Channels Podcast, Geraldine Van der Auwera interviews four members of the Nextflow community attending the 2023 nf-core hackathon: Raquel Manzano, Edmund Miller, Rieke Hanssen and Francesco Lescai. Recorded in-person at the Nextflow Summit Barcelona, we hear a bit about what they have been working on, how they came to use Nextflow and what motivated them to join the nf-core hackathon.

In this News and Views episode, Chris Hakkaart and Marcel Ribeiro-Dantas talk to Rob Syme about the exciting new Nextflow training material and ways to learn Nextflow in 2023.

The second "News & Views" episode of Channels - join us as we chat about the nf-test framework for testing Nextflow pipelines, the second round of the Nextflow & nf-core mentorship program and a new Nextflow blog post about how to figure out exactly why your pipeline won't resume!

In this episode of the Channels podcast we have Evan Floden (Seqera Labs CEO) and Harshil Patel (head of scientific development at Seqera Labs) on the show! We talk about: 0:00 - Introduction 1:42 - History of automated testing in nf-core 4:10 - Using Tower CLI with GitHub actions in nf-core 5:09 - Coverage of full-size tests in nf-core 8:00 - Nextflow Tower "pipeline showcase" automation 14:18 - Python library for Tower automation (tw-pywrap) 18:23 - Seqera roadmap: pipeline detail view and run linking 22:06 - Nextflow Tower - Data Explorer 28:01 - Nextflow Tower - Datasets 31:55 - Upcoming events 35:15 - Wrap up

In this Podcast, Rob Lalonde of Seqera Labs speaks with Cedric Notredame, head of the Notredame Lab at the Centre for Genomic Regulation in Barcelona, Spain. In this far-ranging discussion, Rob and Cedric talk about cloud, pipelines, and trace the evolution of Nextflow from its early days as a research project to its current role in the global COVID effort.

In this Podcast, Rob Lalonde speaks with Nils Homer, founding partner at Fulcrum Genomics. Nils shares his unique perspectives on breaking into the bioinformatics industry, advances in technology and tools, and what to expect in the next three to five years.

In this "News & Views" episode of Channels we talk about the new Flux executor for Nextflow, the initiative to add transcripts to all nf-core bytesize videos,  and some improvements for error messaging when using Nextflow.

In this episode of Channels, we talk to Paolo Di Tommaso (creator of Nextflow, Seqera CTO & cofounder) and Jordi Deu Pons (software engineer @ Seqera) about Fusion - a file system written specifically for Nextflow.

In Episode 51 of the Nextflow podcast, Phil Ewels and Ben Sherman discuss Nextflow's upcoming strict syntax - a significant evolution bringing clearer error messages and a more consistent language framework.

They explore key changes including the separation of code from declarations, removal of for/while loops in favour of operators, and changes to type annotations. Learn how these improvements lay the groundwork for better developer experiences with features like static type checking and auto-formatting.

The episode covers migration timelines, practical steps to prepare, and how these changes will ultimately lead to more robust, maintainable pipeline code. Check out the Nextflow docs for the "Updating Nextflow Syntax" guide to start preparing today.

00:00 Podcast Ep51: Strict Syntax
00:07 Welcome and introduction
01:20 Language server background
04:18 Syntax sugar is just empty calories
06:50 Bespoke language syntax
08:20 To DSL3 or not 2 DSL
10:59 Nextflow versioning
15:13 Diving into specifics
16:08 Mixing statements with script declarations
19:19 Better inspect command
21:08 Custom Classes
23:32 For loops, while and switch
26:02 Standard library
26:56 Type annotations
28:49 addParams
32:26 Process script section
34:20 Moving into lib/
36:36 Configuration syntax
41:14 Relieving pressure from the config
43:38 Timelines
44:55 The gain for the pain
47:54 Next steps
48:43 Conclusion

In this "News & Views" episode of Channels we talk to Nextflow developer Ben Sherman about some exciting new language features (hopefully) coming soon to Nextflow:
0:00 - Introduction
2:42 - Default values for process inputs
11:59 - Arity, nullable options for input and output paths
28:17 - Named arguments for processes and workflows
36:16 - Map inputs/outputs for processes
41:14 - Custom record inputs/outputs for processes
51:37 - New community content and upcoming events

A recap of our current training formulas, how we got here and where we’re heading to next.

Join us in this episode of the Channels podcast as Geraldine Van der Auwera and Chris Hakkaart discuss options for learning Nextflow in 2024. A recap of our current training formulas, how we got here and where we’re heading to next.

Upcoming training opportunities

• Community Foundational Nextflow Training — Online, 5-6 March 2024
• Training at Seqera Sessions — London, 21 March 2024
• Training at Nextflow Summit — Boston, 21-22 May 2024

Other upcoming events of interest

• nf-core Hackathon — Online + Local Hubs, 18-20 March 2024

In this episode of Channels, the Nextflow podcast, join us as we explore Nextflow and AI. Our special guest, Sasha from Seqera, shares his remarkable journey from the consumer tech industry to bioinformatics. Learn about the birth of TinyBio, its innovations in bioinformatics AI applications, and its recent acquisition by Seqera. We delve into AI concepts, the challenges faced, and the future of integrating AI with Nextflow. Tune in to discover how advancements in AI are reshaping the bioinformatics landscape. 00:00 Podcast 45: Tinybio : Nextflow x AI 00:09 Introduction 00:34 Sasha's background 03:17 How tinybio started 05:57 Tinybio co-founder: Vishal Patel 06:35 tinybio chat public launch 09:20 Failed feature launches 10:39 Running the code too 12:46 User trust 14:11 Accessing files via chat 15:08 Experiment idea generator 17:16 Pipeline generator 20:25 AI concepts: Foundational models 22:36 AI concepts: RAG 24:53 AI concepts: Langchain 26:37 AI concepts: Fine tuning 29:14 Challenges in bioinformatics AI 31:53 AI concepts: Q&A training 33:09 Motivation for joining Seqera 34:43 What happens to tinybio now? 35:36 Future plans 38:20 AI and open-source 40:50 Find more updates 41:37 Wrap up

Tune in to hear all about Pixelgen Technologies - creators of a novel technology for doing single-cell spatial proteomics.

We chat about how this fascinating technology works, what you can do with the data and why they chose to develop their analysis suite as an open-source nf-core pipeline.

Links:

• Software:
  • nf-core/pixelator pipeline
  • Pixelator Python package
• Science:
  • Nature Methods publication: “Molecular pixelation: spatial proteomics of single cells by sequencing”
• Presentations:
  • Nextflow Summit 2022 talk: “Pixelgen Technologies ❤︎ Nextflow”
  • Nextflow Summit 2024 Boston talk: “Nextflow allows Pixelgen innovation engine to reach the market quicker”
  • Nextflow Summit 2024 Barcelona
• Pixelgen:
  • Pixelgen Technologies homepage
  • @PixelgenTech on twitter / X and LinkedIn
• BioLizard:
  • BioLizard homepage
  • @BioLizard_nv on twitter / X and LinkedIn

Chapters:

• Start (00:00)
• Welcome (00:09)
• Introductions (00:50)
• Intro to Pixelgen Technologies (01:41)
• Lab methods (03:08)
• Nature Methods publication (06:15)
• Data visualisation (09:35)
• Working with single cell data (12:29)
• Cell interactions (15:06)
• Why Pixelgen chose open source (16:05)
• Pixelator Python package (20:10)
• Building within nf-core (21:40)
• Florian - Biolizard (23:01)
• Pipeline overview (25:19)
• Using nf-core from a commercial setting (26:50)
• Coming out of stealth (28:53)
• Extending Pixelator (32:24)
• Dealing with customers (34:58)
• Looking to the future (39:38)
• Interactive analysis (41:32)
• Advice for others (43:12)
• Pixelgen at the Nextflow Summit (45:16)

Ever wanted to peer inside Nextflow as it runs your pipeline? We play with an interactive debugger to do just that. Also, a first peek at a new Nextflow Ambassador program, plus a revamped hands-on Nextflow training course.

Rob Lalonde of Seqera Labs interviews Robert Petit, a senior bioinformatics scientist at Wyoming Public Health Laboratory. In this interview, Rob and Rob discuss public health challenges unique to Wyoming, genomic surveillance, the Bactopia pipeline, and the latest fashion trends among leading bioinformaticians in the state.

In this episode of Channels, the Nextflow podcast, host Geraldine from Seqera and her colleague Ben Sherman dive into the differences and similarities between Nextflow and the Workflow Description Language (WDL). They discuss the origins and development of WDL at the Broad Institute, the challenges of working with different workflow languages, and how Nextflow's channel-based data flow model compares to WDL's approach. The conversation covers key features, type systems, and runtime specifications, highlighting the strengths and limitations of both languages. Ben hints at upcoming improvements and enhancements for Nextflow, aimed at making the language more robust and user-friendly. The episode offers valuable insights for bioinformaticians and developers navigating the complexities of workflow management.

This podcast episode is quite different to normal. Rather than an interview or a discussion, you'll get to watch as Ben Sherman guides Phil Ewels through the process of creating a new Nextflow Plugin from scratch. Right from the first line of code to creating a release and publishing the plugin. Nextflow plugins have been getting more attention lately and we've been getting a lot of questions in the community. There is better documentation and resources planned, but in the mean time we hope that this can be a useful guide for anyone curious in how they work, and interested in getting started. After this we will be back to our usual routine in the podcast, but we'd love to hear what you thought of this taster!

• 00:00:00 Welcome
• 00:03:27 Nextflow docs
• 00:08:34 Starting by forking nf-hello
• 00:10:32 Overview of project files
• 00:17:13 Trying a first compile
• 00:18:12 Different publishing methods
• 00:19:54 nf-boost local publish method
• 00:23:05 Trying the new compile
• 00:24:00 Running locally with the plugin
• 00:27:32 Looking at the nf-hello plugin code
• 00:39:29 Deleting files we don't need
• 00:42:34 Finding event names in the Nextflow source
• 00:47:46 Writing some custom functions
• 00:49:41 Testing our new code
• 00:51:51 Modifying the test pipeline to create files
• 00:54:14 Coding up the JSON output
• 00:59:20 Looking at nf-prov code for BCO files
• 01:05:30 Testing JSON output
• 01:09:39 Automatic work dir cleanup / nf-boost
• 01:12:17 Publishing a release on GitHub
• 01:19:07 Custom plugin repositories
• 01:26:47 Publishing for all Nextflow users
• 01:29:52 Conclusion and end

In this Podcast, Rob Lalonde speaks with Phil Ewels of SciLifeLab Sweden, part of NGI Stockholm. In this fascinating discussion, they cover everything from nf-core to future bioinformatics challenges.