Methods for Reproducibility in Biomedical Informatics Research
Course Schedule & Assessment Plan
Summary
This course guides students through the process of developing a reproducible pipeline to support biomedical informatics projects. Each 3 hour class is structured so that each lesson will provide a 30-60 minutes of lecture, followed by a 1-1.5 hour lab based activity where students learn and practice implementing the lesson's main concepts. This course employs a scaffolding framework in that each concept builds upon the next for facilitating the student's personal project. In-class activities will account for a small portion of the grade (5%), and are really meant as a low-stakes way of ensuring students are staying on track. The higher weighted assessments include the midterm and final project. However, the course is designed to provide repeated opportunities for students to gain feedback from classmates and the instructor to ensure they are staying up to date on the course material. Moreover, students are given the choice of how to present their final projects. This allows students to leverage their personal strengths to demonstrate understanding of the material. For example, students can elect to present their final project in the form of an oral presentation or a written report.
| Week | Lesson | Activity | Rationale | Weight |
|---|---|---|---|---|
| 1 | Introduction to Reproducibility in Biomedical Informatics | In-class discussion to assess student’s current understanding of reproducibility. Homework: Research project abstract/proposal: Define data and question to answer |
Introduce the importance of reproducibility in biomedical informatics and introduce students to the course project. | 5% |
| 2 | Introduction to Github | In-class assignment to set up Github Repo and perform basic commands/functions | Ensures that students are familiar with GitHub, a mainstay method of version control. This class is a stepping stone to ensure they are set up to embark on their quarter long project. |
5% |
| 3 | Setting up project repository and pipeline | In-class assignment to review best practices for organizing Github repo and projects | Ensures that students have set up their project repository and begin to learn about best practices for pipeline development. | 5% |
| 4 | Data acquisition and cleaning | In-class assignment to build a pipeline to import data. Will review data storage options (AWS, SQL database) and APIs | Students will be able to apply data acquisition techniques for their own projct workflows. | 5% |
| 5 | Unit tests and peer review | In-class assignment to review and implement unit tests. Students will also conduct a peer review of each other's work | Students will learn the importance of unit tests for ensuring that projects are reproducible and that code is doing what they think it is. | 5% |
| 6 | Mid-term Review (Peer Review) |
In-class activity where students will be paired and tasked with reproducing each others work | Ensure that students are keeping up with the prior week's tasks and that project repo is currently runnable and reproducible by peers. | 10% |
| 7 | Configuration files | In-class activity to configure files to ensure reproducibility | Ensure that students learn best practices for implementing configuration files. | 5% |
| 8 | Establish modeling pipeline | In-class activity to establish functional modeling pipelines | This helps ties together all components taught throughout the quarter together. | F |
| 9 | README | In-class activity to develop a README file | Review the importance of documentation of how to run and reproduce analysis | 5% |
| 10 | Final projects | Submission / Presentation | Students will present their own project in the form of their choosing. Their final project will also be graded by the instructor to assess reproducibility |
30% / 20% |
Grading Scheme:
In-class assignments/homework (40%) – in-class assignments/homework throughout the quarter
Mid-term project assessment (15%)
Final project - reproducible pipeline deployment (25%)
Final project presentation (20%) - choice of oral presentation or write up.