New EU-funded safety tech will help reduce viewing and demand for child sexual abuse images and videos

• Development of unique on-device tool to launch in March 2023
• Two-year, €2m-funded project is a collaboration of EU and UK experts
• Tech will be voluntarily installed on devices of those at risk of viewing child sexual abuse material  
• User-centred design will operate in real-time to prevent viewing of material before it reaches the screen

A unique safety tech tool which uses machine learning in real-time to detect child sexual abuse images and videos is to be developed by a collaboration of EU and UK experts.  

Launching in March, the two-year Protech project will research, design and create an app that can be installed on the devices of individuals at risk of accessing child sexual abuse material.

The app will be deployed voluntarily, and users will have full knowledge of its purpose and its effect on their device.

The safety app will monitor both network traffic and images viewed on the user’s screen in real-time. After being installed, the app will run silently and will not require user interaction unless sexual images of children are detected and blocked.  

Collaborators behind the €2m (£1.8m) project, which is funded by the European Commission, believe the tool could help stem the growing demand for child sexual abuse material online.

It will prevent the revictimisation of child sexual abuse survivors who continue to suffer in the knowledge that others may still be able to view images and videos of them online.

The app’s uniqueness lies in its user-centred design which employs highly accurate machine learning models to provide effective intervention to individuals who fear they might offend against children. It will work in real-time to detect and halt the viewing of criminal content before it is seen by the user.

It could prove to be a vital tool for the sustainable, long-term prevention of child sexual abuse content, alongside current digital activities that tackle and remove the imagery, such as criminal investigations and the removal and hashing of images.  

The project is led by one of the largest university hospitals in Europe, Charité – Universitätsmedizin Berlin, in partnership with experts from diverse and wide-ranging fields including criminology, public health, developmental, clinical and forensic psychology, software engineering, child protection and internet safety.

Director of the Institute of Sexology and Sexual Medicine at Charité – Universitätsmedizin Berlin, Prof Dr Klaus M Beier said: “The increasing consumption and distribution of child sexual exploitation material is a problem of international significance and necessitates research into user behavior, particularly in cases not known to the legal authorities, which far outnumber those under juridical inquiry or after conviction. This has been largely neglected in the past, despite being where the potential for prevention is greatest.  

“Thus, with the development of Salus, Protech also targets self-motivated and cooperative, potential or real users of child sexual abuse images who want to avoid starting or continuing consumption.”

The app, named Salus after the Roman goddess of safety and wellbeing, is to be created by UK technology company SafeToNet which specialises in cyber safety, using innovative real-time monitoring technology.  

SafeToNet Chief Operating Officer Tom Farrell QPM said: “We are excited to provide the technical expertise on such a critical project. We believe that in the moment technical prevention has a huge role to play in tackling the consumption and demand for child sexual abuse material.”

To help design the app, project team members from leaders in impact-focused research and knowledge exchange, the Policing Institute for the Eastern Region, based at Anglia Ruskin University in the UK and the Department of Developmental Psychology at Tilburg University in the Netherlands, will investigate why and how offenders begin viewing sexual images of children and what could help them to stop.  

Participants in the study will be volunteers, recruited by the project team partners who provide critical community prevention services – Charité – Universitätsmedizin Berlin; the UK’s Lucy Faithfull Foundation; Stop it Now Netherlands which is part of the Centre for Expertise on Online Child Sexual Abuse; and the University Forensic Centre within the University Hospital Antwerp in Belgium. Interviews will be conducted with individuals at risk of viewing sexual images of children as well as with professionals at prevention support level.

The Internet Watch Foundation (IWF), Europe’s largest hotline dedicated to finding and removing images and videos of child sexual abuse from the internet, will provide a secure environment to train and test the app’s machine-learning software to correctly detect child sexual abuse material.

IWF Chief Technology Officer Dan Sexton said: “Sadly, the demand for images and videos of children being sexually abused is unceasing. In 2022 the IWF removed more than 255,000 URLs from the internet which contained confirmed child sexual abuse material.

“But we know that finding and removing this horrendous content is not enough in the ongoing, global fight to stop the sexual abuse of children, which is why we are glad to play our role in this project to train and test software which could prove vital in lowering the demand for the criminal material in the first place.

“By collaborating with expert organisations in the EU and the UK we are ensuring that the intended impact for this project is as far-reaching as possible to help children around the world.”

Prof Dr Kris Goethals, Director of the University Forensic Centre (UFC) within the University Hospital Antwerp said: “Since the COVID pandemic, there has been a spike in online abuse images and a large part of our treatment population at our outpatient centre (UFC) consists of individuals who are convicted of such acts or who are at risk of committing these acts.

“These problems are observed all over the world and require an international solution. Unfortunately, a purely repressive approach to this phenomenon does not bring much relief, therefore the Protech project can be a first crucial step in a more preventive approach.  

“This project thus provides an additional value in the fight against child sexual abuse, but also brings together the different worlds of law enforcement, community prevention providers and society in general.”

Director of the Policing Institute for the Eastern Region, Prof Sam Lundrigan said: “The online abuse of children is a global challenge that needs innovative thinking in our combined efforts to respond.  

“We know that academic findings through research such as ours, can provide the data needed to support projects such as this, with informed insight and evidence. This is an exciting project that we’re delighted to support, and one we hope will have a real impact, both on those at risk of offending and those who have already suffered abuse.”

Chair of the Department of Developmental Psychology at Tilburg University, Prof Dr Stefan Bogaerts said: “Sexual abuse online is a complex and multidimensional problem without simple solutions. Digital devices can play a role in reducing sexual abuse online by providing certain features and measures that enhance the safety of users, such as security and privacy settings, reporting and blocking options.  

“In addition, there needs to be continued investment in raising awareness of users about online safety and how to prevent sexual abuse. Preventing sexual abuse online is a joint responsibility of tech companies, science, government, and society. Together, we can work towards a cultural change and creating a safe online environment for all users."

Once designed, the safety intervention will be rolled out in a pilot stage in five countries – Germany, Netherlands, Belgium, Republic of Ireland and the UK – involving more than 50 professionals and at least 180 users over an 11-month period.

SafeToNet will gather feedback from users and professionals while the pilot is ongoing and use it to further improve and adapt the app’s software.

Part of the project will involve evaluating and assessing the potential reach and impact of the intervention in Europe, taking on board recommendations from experts on how it could be effectively implemented as part of public health prevention programmes.

Given the sheer scale of sexual images of children available online and the growing demand for the content, the project team believe the app and the intervention programme behind it will also help reduce the workload of law enforcement pursuing the criminals responsible for creating, distributing and, in some cases, profiting from the sale of the content.

Donald Findlater, Director of The Lucy Faithfull Foundation’s Stop It Now! UK and Ireland helpline, said: “Last year, nearly 5,000 people contacted our Stop It Now! UK and Ireland helpline concerned about their own sexual thoughts or behaviour towards children. They want help to manage this so that children aren’t harmed and they do not commit a crime. In addition, our online self-help resources had hundreds of thousands of visitors, looking for help to manage their own or a loved one’s online sexual behaviour.

“Salus would help many people who contact us to stop viewing sexual images of children. This project allows us to support these people and learn how to better tackle the problem of people viewing sexual images of children online. Salus has the prospect of becoming a major contributor in the global fight against online child sexual abuse.”

Arda Gerkens, Managing Director of the Centre for Expertise on Online Child Sexual Abuse said: “Stop it Now Netherlands is very excited to be a part of the Protech research project. Our helpline offers support, guidance and online self-help resources for people who use child sexual exploitation material (CSEM). However, individuals who contact our helpline are often in search for technical interventions that will keep them away from CSEM.

“In our mission to stop and prevent child sexual abuse (online) and work towards a safe online environment for everyone, we feel that research and continued improvement of support and prevention tools are essential.”