Large Language Models (LLMs): Is this AI?

Data Science team, Strategy Unit

Oct 10, 2024

Generative AI ✨

• Creates new content
• Trained on lots of examples
• Can mimic creativity

• Generative AI uses patterns from data (like text, images, or sound) to create new, similar content.
• It’s trained on large datasets and then generates things based on what it has learned.
• Can write stories, make art, create music, or even simulate human conversations.

Modalities 🖼️

• Images (e.g. DALL-E)
• Audio/video (e.g. inVideo AI)
• Text (e.g. ChatGPT)

• Written content like stories, code or conversations.
• Visuals, from drawings to photorealistic images.
• Music, voices, or even entire videos with sound and motion.
• These slides are about that last one, text, via Large Language Models (LLMs).

Text 🔤

• Spam filters
• Sentiment
• Topic detection
• Word prediction
• …

• Before we get into LLMs, it’s worth recognising how many tools we use that involve some kind of text processing (perhaps AI driven).
• Pattern detection, comparison to previous spam, keyword detection.
• Summarising customer reviews as positive or negative (sentiment analysis).
• Automatically identify key themes or topics within large collections of unstructured text (e.g. topic modelling).

Large Language Models (LLMs) 🦜

• A (fancy?) parrot
• Learns from lots of text
• Predicts next word

• Breaks down text: the model processes input text by breaking it into smaller units (like words or subwords).
• Finds patterns: it learns relationships between the units.
• Generates predictions: based on the input, it predicts the most likely next word or phrase to generate coherent responses or text.

In healthcare 🏥

• Organise medical documents
• Drug discovery research
• Chatbots
• …

• Automatically generate, organise, summarise patient notes.
• Analyse medical literature and data to help identify new potential treatments or drug interactions.
• Provide 24/7 support by answering common medical questions and guiding patients through symptom checks.

IRL models 🤖

• Healthcare-focused models
• Google’s MedLM
• Nature: LLMs in medicine and the future landscape

• Specifically-developed LLMs for healthcare applications already exist, many are openly available.
• Google’s MedLM has a price tag.
• There’s a couple of Nature articles on current applications and potential future landscape.

Case study 🧠

• NHS: chatbot for mental health referrals
• Used Limbic ‘Access’ ‘e-triage’ chatbot
• According to Limbic:

‘Nearly 40% of NHS Talking Therapies already trust Limbic to improve their services’

• There’s an admin burden.
• Took place in the Surrey and Borders Partnership NHS Foundation Trust

Case study 🧠

Patient

Clinician

• From the NHS-E Transformation Directorate write-up:
• ~99% of patients that left feedback said that Limbic was helpful
• the service has seen a 30% increase in referrals and initial evidence indicates that Limbic improved out of hours access
• on a pro-rata basis, a saving of 3000 hours (4 psychological wellbeing practitioners)
• nearly 20% of referrals were identified as ineligible and signposted to a more appropriate service

But… ⚠️

The effect of using a large language model to respond to patient messages (The Lancet)

…raises the question of the extent to which LLM assistance is decision support versus LLM-based decision making

…a minority of LLM drafts, if left unedited, could lead to severe harm or death

• Brigham and Women’s Hospital, USA.
• Investigated how ‘LLM [GPT-4] assistance for electronic patient portal messaging in electronic health record systems (ie, using an LLM to draft a response for a clinician to edit) might impact subjective efficiency, clinical recommendations, and potential harms’ for cancer patients.
• Found safety errors, and in one instance the advice given to a patient could have been fatal.

Pros ➕

• For providers: could reduce pressure
• For users: increases service accessibility
• Can be trained for domain specificity

• Could ease pressure by providing info/assistance to users anytime, making services like customer support available 24/7.
• Healthcare is very broad, but we can train on specific subsets of information to tailor the outcome. However, we do need plenty of data.

Cons ➖

• Ethical issues, like:
  • bias
  • computational cost
  • data origins
  • privacy
• Not human
• It lies

• Sometimes gives wrong or confusing information, especially on complex topics. May provide inaccurate medical advice or information, leading to potential misdiagnoses or harmful decisions.
• ‘Hallucination’ is perhaps a weasel word.
• Can reflect biases found in the data it was trained on, leading to unfair responses that could be plain wrong for the target audience.
• Lack of accountability: it can be unclear who is responsible, complicating patient care and trust issues.
• Requires significant computational power and resources, which can be expensive and environmentally taxing.
• Doesn’t understand emotions, context, or nuance, which are all important in clinical settings.

Consider 🤔

• Are there legal issues?
• Have you considered user needs?
• Should you follow policies (e.g. HM gov, NHS, trust)?

• Could your product technically be a medical device, for example?
• Have you performed a data protection impact assessment?
• Are there more classic, better understood AI approaches you could use? NLP?
• Test the limits of the system; where is it likely to fail or give bad information?
• Provide an alternative to users in certain cases (e.g. non-chatbot given the chance of ‘conversation loops’).
• Are you thinking about the humans at the end of the process? The users of your service?

To ponder ❓

• Is this AI?
• Are LLMs an appropriate tool in healthcare?
• How might you feel interacting with an LLM-driven service?
• How can we protect patient privacy?
• How do we deal with LLMs as tools for decision support vs decision making?
• Who is responsible for errors, or even death?

Further reading 📚

• NHS Knowledge and Library Services: AI
• 3 Blue 1 Brown (YouTube)
• Computerphile (YouTube)
• GOV.UK chatbot experiment