GitHub as a team sport

DfT QA Month

Matt Dray

May 23, 2024

tl;dr

• ‘Quality’ isn’t just good code
• Teamwork makes the dream work
• GitHub is a communication tool

• ‘Too long; didn’t read’.
• GitHub isn’t just a dumping ground for code and Git history.
• It’s a platform for working with teammates to get things done.
• Quality is improved by good communication, organisation and reduction of something called the ‘bus factor’ that I’ll get to in a minute.

About us

• I’ll start with some context about the organisation so you understand how we work.
• There’s a link to the website at the bottom of each slide.

The Strategy Unit (SU)

• An ‘internal consultancy’
• Hosted by NHS Midlands and Lancashire
• Growing in size and reputation

• Initially a ‘start-up’ style operation that has expanded to 70+ staff.
• ‘We produce high-quality, multi-disciplinary analytical work – and we help people apply the results.’
• A lot of our work is on the important New Hospital Programme (NHP).
• ‘Our proposition is simple: better evidence, better decisions, better outcomes.’
• Expansion is tricky; how can we maintain quality?

The Data Science Team

• Expanded to 6, all remote
• Modelling, Quarto, Shiny
• New Hospital Programme (NHP)

• A new team, expanding rapidly from 2 to 6 in about a year.
• Remote across England.
• Experience from across the NHS and consultancy. I spent a decade in five central government departments before this.
• We’re helping to model and design apps for the NHP to help build hospitals.
• So: growing team, different experiences, important work, but few standardised processes. What to do?

GitHub at the SU

• We should be exemplars
• Aiming for open by default
• GitHub is on the homepage and there’s a Data Science site

• It’s not just the DS team.
• We have many other analysts eager to learn and contribute.
• How can we set good standards and encourage use across the organisation?
• We’re running Coffee & Coding sessions, teaching and encouraging talks and blogs on our site.
• We want to drive up quality by making code open too.
• It’s a statement of intent that the SU homepage links to our GitHub organisation.

This talk

• So that’s the context: we’re experienced, but fledgling as a team, and keen to do things well.
• These slides are some of the ways we’ve been working so far with a focus on GitHub, specifically.

What this is

• Low-tech, no code
• Tips and etiquette, not directives
• What’s been working for us

• But this is not a technical talk about how to use Git for version control.
• Mostly it’s about planning, workflows, standards and communication.
• It’s things that our team have been doing and the ideas are evolving.
• I’ve worked mostly alone on GitHub projects in my career and never worked in a data science team of even this size. So at worst these slides are a way for me to write down what I’m learning.

The ‘bus factor’ 🚍

• We should maintain quality
• We need redundancy
• Standardised processes can help

• Why do we care about discussing and ‘formalising’ these ideas?
• We should encourage standard practices in case someone is ill or away.
• This also makes it easier when new team members join.
• This helps us maintain quality.

‘Rules’

• It’s the spirit that counts
• Do as I say, not as I do
• Know why you’re breaking the rules

• To be clear though, nothing here is etched into stone.
• There will be times where rules can be broken.
• But we shouldn’t be complacent.

What we do

• So, let’s move onto some specific examples of how we’ve been using GitHub for our work.
• I haven’t included everything.
• Many of our best examples are currently on closed repos that will be opened with time, due to various sensitivities.
• Some ideas are literal features of the platform, others are more like ‘suggested best practice’.
• Hopefully there’ll be at least one thing that’s new to you and that you might want to use in your own team.

GitHub flow

1. Create a repository
2. Write issues
3. Plan
4. Create a branch
5. Make a pull request
6. Review
7. Release

• This is a fairly generic GitHub flow.
• I’ll talk through a few things in each of these categories.

Repositories

• Assign ‘owner’ and ‘deputy’ roles
• Add README and .gitignore
• Store data elsewhere

• Easy starter: tell people what the purpose of the repo is and how to use it. This is what a README is for. This is an absolute must to lower the bus factor.
• We should be prevent accidental file upload immediately. Use a .gitignore to exclude likely data files (as well as other unnecessary files). We’re thinking about common templates/cookiecutters.
• Communicative files (README, .gitignores) are good, but so is vigilance (code review).
• Owners/deputies are in charge of ‘GitHub gardening’ (keeping issues in order, labelling, milestones, etc).
• Deputies help with bus factor.
• The owner can be auto-selected as the reviewer. We’re experimenting with this for repos with external contributors, especially.
• Data is stored elsewhere, on Azure or Posit Connect, due to sensitivity and size. This should be planned before you begin and recorded in the README.

Issues

• Aren’t just ‘problems’
• Use labels, including MoSCoW
• Explain the need, be informative

• Issues can be reminders or questions for further discussion, not just features to build.
• Tickets should get two labels. We use a topic like ‘enhancement’, ‘bug’, ‘documentation’, ‘techdebt’, etc, plus MoSCoW (must, should, could, won’t) to help prioritisation.
• Issue templates can ensure certain info is provided, which is especially good for external contributors.
• Refer to other related commits by number (e.g. #1), which stops you repeating the same information.
• Prefer to reopen an issue if it doesn’t actually work.
• Issues can track separate sub-issues.
• You can add checklists with markdown checkbox: - [ ] (these appear in the issue preview).
• You can ‘hide’ comments if they’re out of date, etc.

Plan

• Talk, review and reflect
• Use labels to prioritise
• Sort into milestones

• We have a repo and issues, what do we do now? Where to start?
• We’ve begun working in sprints of about 4 weeks. We have sprint planning meetings to plan things out.
• Consider what needs to be done in the sprint period, what other issues support those goals?
• Is there time for other tasks, like clearing techdebt?
• All issues should be assigned to a milestone.
• Issues in milestones should be sorted in priority order/order of expected completion (MoSCoW labels will help with this).
• This helps focus the goals of the sprint and keep us on track.

Branches

• One issue, one branch, one assigned person
• Name them sensibly
• Burn them

• Only one person works on a branch at a time. This person is the one assigned to the relevant issue.
• Branch names should be numbered to match their issue, e.g. ‘123-add-filter’. This makes it obvious what issue is being fixed by that branch and should help identify if more than one person has a branch open for the same issue.
• If commits from someone else are required, then all parties must communicate about the current state of the branch to ensure they pull changes and avoid merge conflicts.
• Branches are ephemeral and die when the PR is merged. They should be deleted (this can be done automatically).
• The only branches to exist at all times should be main and a deployment branch, if necessary. All others should be active branches so it’s clear what’s being worked on.

Commits

• Don’t commit to main!
• ‘Small, early and often’
• Make messages meaningful

• There’s not a lot of earth-shattering advice to give here; this stuff is fairly standard.
• Do not commit directly to main. Your work must be independently checked first to limit the chance of mistakes.
• Make your commits small in terms of code and files touched, if possible. This makes the Git history easier to read and makes reviews easier too.
• Commit and push early and often into your branch. This can help others see progress and helps reduce the bus factor.
• Don’t dump your work into a commit because it’s the end of the day.
• Make your commit messages meaningful. What does the commit do? Start with a verb in present tense (‘adds’, not ‘added’). Or maybe use ‘conventional’ commits.

Pull requests (PRs)

• Small and closes an issue
• Select the assignee and reviewer
• The assignee merges

• PRs should solve the issue they’re related to. Occasionally one fix may solve another.
• They should be named to explain what they do. The issue might be ‘the red button doesn’t work’; the PR might be ‘fix the red button’.
• They should be small in terms of lines of code and files touched. This will make it easier and faster to understand and assess the changes.
• The submitter should mark themself as the ‘assignee’ and choose a reviewer. You may want to chat with the reviewer to let them know if they have time.
• For context, link to the issue(s) being closed with the magic words (‘closes’, ‘fixes’, etc), which will also close those issues as completed.
• Include a short explanation or bullet-points of what the PR does. Provide any extra information to make the reviewer’s life easier (areas of focus, maybe) or to ask a question about some aspect of what you’ve written.
• The PR submitter is the one who clicks the merge button. This is in case the submitter realises there’s something they need to add or change before the merge.

Reviewing PRs

• Be helpful, be kind
• Use GitHub suggestions
• Discuss if unclear

• The reviewer should typically check that the changes result in the issue being fixed. This may require pulling the branch and then testing it, but may not be necessary for small changes.
• The reviewer should seek clarification and add comments where something isn’t clear.
• Use ‘suggestions’ as a reviewer rather than committing to someone else’s branch.
• When working at pace (when aren’t we?), we should err towards approval if the issue is completed rather than an endless cycle of asking for small changes. The submitter and reviewer should decide whether smaller things like code style or change in approach should be added as a new issue with a ‘techdebt’ label.

Releases

• Use semantic versioning (1.2.3)
• Autofill notes with PR names
• Don’t release on a Friday 🙃

• Tag the history and release on GitHub concurrently to keep them in sync (this is done automatically if the release is done from the GitHub interface).
• Semantic (x.y.z where x is breaking, y is new features and z is patches for bugs).
• We typically just autofill the release description with the constituent PR titles. Which means it’s important to give them meaningful names.
• We align releases with sprints, though patches may occur more frequently.
• We link releases to deployment in many cases. Don’t release to prod on a Friday, lol.

Surprise twist…

GitHub is a team member

• Automate with Actions
• Provide issue and repo templates
• An all-in-one planner?

• I lied: we have 6 human team members. GitHub itself has features that can automate away some boring things and help prevent accidents or forgetfulness.
• GitHub Actions for continuous integration. R-CMD check at least for R projects. Start with r-lib examples as a basis.
• We’re looking towards things like templates at the issue and repo levels; again to remove drudgery.
• We use Trello to plan things and have to link to GitHub repos and issues in Trello cards. Can we use GitHub as our planner across multiple repos instead? Seems possible.

Be a good sport

Are we curling? 🥌

We:

• are a small team
• assume specialist roles
• work in sync

• You have been wondering: if this is a ‘team sport’, what sport is it?
• This is a terrible metaphor. But think about it.

The bottom line, actually

1. Communicate
2. Help each other
3. Be kind

• The ideas in this talk are things that have helped us, and could help you, to drive up and maintain quality. Some were obvious, some were specific features you might not have known about.
• But none of these are replacements for being good team members.
• GitHub just provides some affordances to help you.
• I am the guy falling over, the stones are tasks, my team mates are picking me up and dusting me off.
• Did you learn at least one thing? What has your team been doing? What works for you?