Data Science Blog

Lazily working with large data on Azure storage, with DuckDB or polars

Python

learning

Azure

polars

howto

(#lede) (use callout?)

We’ve Been Here Before

A brief history of our scepticism towards new technology, and what it means for AI today

AI

Data Science

Technology

History

In Plato’s Phaedrus, over 2,000 years ago, Socrates tells a story about a dangerous new technology that would erode memory, encourage shallow thinking, and give people the…

Pandas vs. Polars

Accelerating Healthcare Data Processing

Python

data science

performance

Data processing is at the heart of our work at the Strategy Unit. As our datasets have grown and NHP model has become more complex, we’ve found that our pipeline takes…

Verifying Git Commits

Git

GitHub

Ever looked at a list of commits on GitHub and noticed ✅ green checks next to some commits, but not others?
Wondered what they mean?

Positron for Product Owners

R

Positron

Git

The clever cookies at Posit (creators of RStudio) recently released Positron, their new ‘next generation’ integrated development environment (IDE), after 2+ years of…

Imputing missing data

(Sometimes) making things up can improve the quality of your analysis

R

imputation

learning

MICE

In the world of data analysis, missing values can feel like puzzle pieces that just won’t fit, leaving analysts frustrated and insights incomplete. But what if I told you…

What does a scrummaster do, anyway?

How we manage sprints at the Strategy Unit Data Science team

GitHub

Scrum

Agile

We use sprints to manage our workload developing the open source NHP Model and several adjacent products. We’ve written before about sprint roles: Claire as Product Owner and …

Data science as a product

Bringing together analysts, data scientists, and customers to better serve patients and the whole NHS

GitHub

Scrum

Agile

Data science in healthcare is booming. Every day new advances emerge that promise to deepen understanding, improve screening or optimise treatments. But as this panoply of…

Taking a Hackathon approach to exploring new methods in NLP

learning

NLP

AI

At the Strategy Unit, we’re lucky to have an awesome Evaluation team who often process and categorise large amounts of free text in the work that they do. This can be a…

Using GitHub Apps for Authentication

GitHub

learning

deployment

Recently, we’ve been working on building an internal dashboard to monitor the repositories in our GitHub organisation. The intention is to perform various checks, such as…

Mapping my R journey so far: ten things that I have done along the way

learning

This blog post follows up from a talk I gave last year at coffee and coding about my experiences of learning how to code using Rstudio. Here I build on that talk to share…

Introduction to text vectorization

NLP

Python

tutorial

What is it and why do we care? First steps in Natural Language Processing

Deploy previews with GitHub pages

GitHub

learning

deployment

Wouldn’t it be nice if when a PR is created, you automatically got a deployed version of your changes to look at?

Using GitHub to plan and organise Coffee & Coding

GitHub

learning

Coffee & Coding is a fortnightly hour-long session organised by the Data Science team, open to all members of the Strategy Unit with an interest in coding. It’s been well…

Map and Nest

purrr

R

tutorial

I want to share a framework that I like using occasionally for data analysis. It’s the nest-and-map and it’s helped me countless times when I’m working with related…

Storing data safely

learning

R

Python

UPDATED: Please see the Addendum to this blog, added 2025-04-03 regarding accessing data from SharePoint

YiWen Hon, Matt Dray, Claire Welsh

One year of coffee & coding

learning

The data science team have been running coffee & coding sessions for just over a year now. When I joined that Strategy Unit, I was really pleased to see these sessions…

RStudio Tips and Tricks

learning

R

In a recent Coffee & Coding session we chatted about tips and tricks for RStudio, the popular and free Integrated Development Environment (IDE) that many Strategy Unit…

Visualising participant recruitment in R using Sankey plots

learning

tutorial

visualisation

R

Sankey diagrams are great tools to visualise flows through a system. They show connections between the steps of a process where the width of the arrows is proportional to…

Nearest neighbour imputation

learning

Recently I have been gathering data by GP practice, from a variety of different sources. The ultimate purpose of my project is to be able to report at an ICB/sub-ICB level¹.…

Jacqueline Grout

Advent of Code and Test Driven Development

learning

Advent of Code is an annual event, where daily coding puzzles are released from 1st – 24th December. We ran one of our fortnightly Coffee & Coding sessions introducing…

Reinstalling R Packages

git

tutorial

R 4.3.0 was released last week. Anytime you update R you will probably find yourself in the position where no packages are installed. This is by design - the packages that…

Alternative remote repositories

Git

tutorial

It’s great when someone send’s you a pull request on GitHub to fix bugs or add new features to your project, but you probably always want to check the other persons work in…

Creating a hotfix with git

Git

tutorial

I recently discovered a bug in a code-base which needed to be fixed and deployed back to production A.S.A.P., but since the last release the code has moved on significantly.…