Text mining of patient experience data

May 15, 2023

Patient experience

  • The NHS collects a lot of patient experience data
  • Rate the service 1-5 (Very poor… Excellent) but also give written feedback
    • “Parking was difficult”
    • “Doctor was rude”
    • “You saved my life”
  • Many organisations lack the staffing to read all of the feedback in a systematic way

Text mining

  • We have built an algorithm to read it
    • Theme
    • “Criticality”
  • Fits alongside other work happening within NHSE
    • A framework for understanding patient experience

Patient experience 101

  • Tick box scoring is not useful (or accurate)
  • Text based data is complex and built on human experience
  • We’re not making word clouds!
  • We’re not classifying movie reviews or Reddit posts
  • The tool should enhance, not replace, human understanding
  • “A recommendation engine for feedback data”

Everything open, all the time

  • This project was coded in the open and is MIT licensed
  • Engage with the organisations as we find them
    • Do they want code or a docker image?
    • Do they want to fetch their own themes from an API?
    • Do they want to use our dashboard?

Phase 1

  • 10 categories and moderate performance on criticality analysis
  • scikit-learn
  • Shiny
  • Reticulate
  • R package of Python code

Golem all the things!

  • Opinionated way of building Shiny
  • Allows flexibility in deployed versions using YAML
  • Agnostic to deployment
  • Emphasises dependency management and testing
  • Separate “reactive” and “business” logic (see the accompanying book)

Phase 2

Making it useful

  • Accurately rating low frequency categories
  • Per category precision and recall
  • Speed versus accuracy
  • Representing the thematic structure

The future

  • Off the shelf, proprietary data collection systems dominate
  • They often offer bundled analytic products of low quality
  • The DS time can’t and doesn’t want to offer a complete data system
  • How can we best contribute to improving patient experience for patients in the NHS?
    • If the patient experience data won’t come to the mountain…

Open source FTW!

  • Often individuals in the NHS don’t want private companies to “benefit” from open code
  • But if they make their products better with open code the patients win
  • Best practice as code

The projects

  • https://github.com/CDU-data-science-team/pxtextmining
  • https://github.com/CDU-data-science-team/experiencesdashboard
  • https://github.com/CDU-data-science-team/PatientExperience-QDC

The team

  • YiWen Hon (Python & Machine learning)
  • Oluwasegun Apejoye (Shiny)

Contact:

  • chris.beeley1@nhs.net
  • https://fosstodon.org/@chrisbeeley