BLIS: Where imagination meets engineering

When new software solutions emerge from third-level institutions it is usually presumed that the source will be the computer science or engineering departments.

However, a new technology for automated subtitling of TV and video broadcasts has been developed by a team led by Prof Paul McKevitt of Ulster University's School of Creative Arts & Technologies.

The highly sophisticated software, which is being developed and tested with the support of Invest Northern Ireland through their Proof of Concept Programme and the EU Sustainable Competitiveness Programme for Northern Ireland, has the potential to revolutionise TV subtitling making the process far quicker and more accurate, greatly enhancing the viewing experience.

Creative

The School of Creative Arts & Technologies lies at the heart of what is known as the “imagineering quarter”, a physical and virtual network comprising seven adjacent buildings in the heart of Derry which are devoted to teaching various aspects of creative subjects including film, creative technologies, design, drama, and music.

Prof McKevitt has worked for many years at the intersection of arts and technology. His work has taken him into areas such as artificial intelligence (the recognition of emotions and sentiment within computer programs), and game-based learning.

“Computing is becoming so pervasive now,” McKevitt notes. “It’s coming into every field, medicine, law, security and so on. One of the biggest areas now is computing for humanities in areas like the arts and language. This department used to be called the School of Creative Arts but we added ‘technologies’ to the name because of the increasing convergence between the two.

“We are bringing together computing and the arts in areas like film, design, music and drama. Our new cinematic arts degree is one of the most subscribed courses across all campuses of Ulster University.”

That popularity is mirrored by the huge growth in the arts and entertainment sector in Northern Ireland.

"There is a lot of film production going on in Northern Ireland," says McKevitt. "Dracula and Game of Thrones are recent examples. Northern Ireland used to be known for shipyards and textiles but has now moved into the creative industries and creative technologies in particular. The film, music and games industries are key. People don't realise it but the global computer games industry is bigger than the film industry."

Crossover

It is that crossover between the humanities and computer technology which is opening up new research horizons for McKevitt and his colleagues in Ulster University.

One of his previous research projects involved the use of computers to sense students’ emotions in order to improve the learning experience. This combines game-based learning, intelligent tutoring systems, and the creation of computer tutors that recognise and respond effectively to students’ emotions in order to deliver a more effective and pleasant learning experience.

Underpinning that was the development of computer tutoring systems which could recognise what emotions are relevant to the learning situation, how and when those emotions arise, and what factors determine an emotion. And then it all goes back into game scenarios where students use their knowledge and skills to solve problems.

The subtitling software originated from another project known as BLIS (Broadcast Language Identification System) which can check and ensure that TV content is broadcast in the correct language for a given region.

Broadcasters such as Sky commonly broadcast TV content such as sports, news and adverts in multiple languages across multiple regions and there is a need to ensure that it is always in the correct language.

“If a broadcast goes out in the wrong language it has to be corrected very quickly,” McKevitt points out.

“There can be financial penalties and there is reputational damage as well which can be even more costly to the broadcaster concerned.”

Software which can automatically recognise the language and ensure rapid correction is therefore very valuable. But it has turned out that there is an even more valuable spin-off effect of the BLIS software in automated subtitling of broadcast content giving more accurate and quicker subtitles.

Current automated subtitling is patchy to say the least when it comes to speed and accuracy and often delivers downright odd and inadvertently hilarious results.

"Star Wars character Princess Leia has been called "Present Cesc lay ya", lemon transcribed as "lepl on", Margaret Thatcher as "margarine hatcher", and the phrase "be given to our toddlers" transcribed as "be given to ayatollahs".

Subtitles

UK broadcasting is highly regulated by the watchdog

Ofcom

which has set new targets not only for quality but for the quantity of programmes to be subtitled.

The BBC is required to subtitle 100 per cent of its programmes whilst Channel 4 and ITV, STV and UTV will all have to subtitle at least 90 per cent of broadcasts – up from a current level of around 20 per cent.

This presents huge opportunities for the BLIS software. “Very often the child becomes more successful than the parent,” says McKevitt.

“Our software can automatically recognise the language and convert it to text. We have tested it on some programmes which delivered 100 per cent accuracy. This is not only important for general programming but absolutely vital for advertising where there are very strict criteria in relation to language used, product claims and so on.”

Dr John MacRae of Ulster University’s research and innovation office saw the potential of the technology from the outset.

“My own background is in artificial intelligence and I could immediately see the benefits of this software for creative technology applications and industry watchdog-type bodies like Ofcom. The development of the technology has also been informed by industry expert mentors in the broadcast technology UK and international supplier base centred around Newbury and London.”

Prof Paul Moore, head of the School of Creative Arts & Technologies says BLIS is a perfect example of what this school is trying to achieve.

“We are bringing computing and coding together with the humanities. We have research and degree courses in film, creative technologies, music, drama and design and are positioning our students for exciting multi-disciplinary careers in the creative industries.”