Captions and Summaries for Deaf Students Project

About this project

Live captioning for deaf secondary school students hardly exists yet, even though the technology is available and already in schools. The experience of Covid made schools more aware of captions with online learning. Existing widely used software such as Windows 10 and 11 allow live captioning which could be turned on in classrooms. The quality of this open-source software has improved dramatically over the past year.

Teachers of deaf children and young people (ToDs) often work from school to school. In secondary each deaf pupil may have 12 subject teachers. The ToD finds out in advance if possible what work is coming up so that they can plan for a tutorial session. Many deaf students receive regular tutorial support from ToDs to boost literacy skills and subject learning. It is rare for the ToD to receive curriculum information in advance of a tutorial from all teachers, which can lead to wasted time and lack of focus on the most demanding areas of study. A summary of the most important learning from each class is helpful for the deaf student and the visiting ToD.

Our aim is to build an AI tool which automatically produces a useful summary from the recording of the lesson. The deaf pupil receives captions live in class and a summary arrives in their email on the same day. The ToD also receives the lesson summary and can use them to focus in on the most demanding concepts to prepare for the tutorial.

To attain this goal, we need your help:

Perhaps you are a QToD working in a secondary resource base school where the teachers are already deaf aware?
Would some of your teachers agree to have their lessons recorded?
Do you have deaf students who would like to try out live captions in these classes?
Would you be prepared to use a transcript to create a summary of the information you regard as important for the deaf pupil to know? We are hoping for 5 summaries per QToD involved in the project.

If so, we would like to hear from you! Please sign up here and receive our regular newsletters.

Frequently Asked Questions

Aims of the project

Our project aims to produce a summarising system and refine it with feedback from teachers and pupils. We think that the learning from this project can inform local authorities, teaching organisations and EdTech companies to incorporate a lesson summarising feature in software packages for schools.

The funding for the project is from the Harmonisation Impact Acceleration Account, supported by major UK Research Councils. The projects bring together researchers from different areas to address unmet social needs.

Different projects relating to Captioning

These unaffiliated projects also look at captioning and may be of interest:

Caption Connect Captioning Pilot EANI
Sound Waves Foundation - Action for Captions
Oncall Language Services (theeducatoronline.com)

Technical advice

Find out more about how to use live captions with speech to text in Windows 10 or Windows 11 or Apple Mac. (See below, links to word documents with further technical info.)

	Apple Mac	Windows 10	Windows 11
Microsoft Teams	Instructions for classroom teachers to set up Microsoft Teams
Powerpoint	Instructions for subtitles on Powerpoint Apple Mac MS365 Web browser	Instructions for subtitles on Powerpoint Windows 10 MS365 Web browser	Instructions for subtitles on Powerpoint Windows 11 MS365 Web browser
Word	Word Apple Mac Instructions for classroom teachers on setting up captions	Word Windows 10 Instructions for classroom teachers to set up captions	Word Windows 11 Instructions for classroom teachers to set up captions

Windows 11 has built-in Live Captions which work across all applications, find out how they work:

Windows 11 Live captions (Microsoft Support website)

Guidelines for QToD summarisers

We are asking ToDs who have agreed to participate in the project to produce 5 summaries each of 5 transcripts. These transcripts will be from their own school or service. We are not being prescriptive about how the summaries should be made because we want the authentic needs of the ToD and the deaf pupil to shape them. The AI summariser will be designed to create summaries following these same priorities.

How to summarise from a transcript (docx)

Safeguards and ethics

There are safeguards in this project. The voices of children are not recorded. Teachers can decide not to use a recording at the end of the lesson. The summaries are hand produced with the help of the ToD in the school and the project researchers. We will keep the recordings, transcripts and summaries confidentially inside the university for only 2 years to build proof of the concept. We have prepared information and consent forms for you to download for QToDs, subject teachers, deaf young people and their parents.

Ethics and agreement forms

Please download the form relevant to you, complete and return to Rachel O'Neill.

Recording and the summaries

QToD information sheet	QTOD consent form
Subject teacher information sheet	Subject Teacher consent form

Evaluating the captions and summaries

Deaf Pupil information sheet (16-18)	Deaf Pupil assent form (16-18)
Parent information sheet	Parent consent form

Pupils will be involved in this project to give their views on the live captions and the summaries. Subject teachers and ToDs will also be asked for their experiences and views about live captions and summaries. This will help us improve the systems we are trialling and the AI summariser.

Research Team

How the project is managed

The project is setting up a reference group which will meet three times over the period May 2024 – January 2025. Represented on it will be deaf students, teachers of deaf children, BATOD, the Council partners and other interested people. This group will advise the project workers.

Rachel O'Neill and Professor Peter Bell are the investigators on the Project.

Dr Eilidh McEwan was the researcher working on the education side of the project. A further post-doctoral researcher will be appointed in the School of Informatics soon.