Starmer’s social media restrictions will mean the government can spy on every phone
In practice, the government is demanding that Apple install on-device AI nudity-detection software at the operating system level for iOS and that Google do the same for Android. This would involve constantly scanning the camera viewfinder, screen output, livestream, and stored files, and blocking or blurring nudity in real time. This amounts to surveillance technology on every device in the country, says Matthew Lesh
In the midst of an interregnum in Labour’s civil war, Keir Starmer is on the hunt for something extremely dangerous: a legacy. His latest play is a call for tech companies to introduce device controls that prevent children from taking, sharing or viewing sexually explicit images on devices.
“Because this is not an impossible challenge. These are some of the most innovative companies in the world,” Starmer told London Tech Week, “And I believe they can solve it. If they choose not to, then we will act, and we will change the law.”
Starmer’s proposal is, on the face of it, entirely reasonable. Nobody wants to see children threatened. But technically speaking, and for our basic civil liberties, this policy is extremely problematic.
In practice, the government is demanding that Apple install on-device AI nudity-detection software at the operating system level for iOS and that Google do the same for Android. This would involve constantly scanning the camera viewfinder, screen output, livestream, and stored files, and blocking or blurring nudity in real time.
This amounts to surveillance technology on every device in the country. It will make our devices much easier to exploit by bad actors, opening a whole new vector of attack by hackers and hostile foreign governments. There’s a reason world leaders and intelligence officials cover or remove cameras from their devices. We can only expect this policy to make things worse.
The Home Office has suggested that this scanning be turned on by default on all iOS and Android devices, with users required to verify their age via biometrics or ID to disable the block. But that, in itself, raises further concerns about our privacy, as it means handing over additional sensitive information, often to a third party.
Not just theoretical
This isn’t just a theoretical threat. Last October, hackers compromised a customer support vendor used by Discord, stealing around 70,000 government ID images, including passports and driving licences, along with users’ names, email addresses, IP addresses, and support messages. The data was then used to extort the company. As the Electronic Frontier Foundation puts it, “Age verification systems are surveillance systems.”
Children will also inevitably find workarounds to age verification, as Australia’s much-heralded under-16 social media ban demonstrates. Implemented only in December last year, surveys suggest a majority of Australian children under 16 admit to still having active social media accounts, with most children saying circumventing the ban was ‘easy’. Methods range from the mundane (using a parent’s account or an older sibling’s ID) to the creative. One Reddit thread suggested using a printed mesh face mask from Temu to defeat facial recognition. Others simply migrated to platforms outside the ban’s scope. Mandating on-device nudity scanning would fare no better; a child with access to any adult’s phone, a VPN, an older device, or a foreign handset is immediately outside its reach.
The proposal would normalise government’s power to control which images you are allowed to see and take on your own device
More creepily, the proposal would normalise government’s power to control which images you are allowed to see and take on your own device. We have already seen the UK’s Online Safety Act inspire authoritarians around the world to pursue even harsher ‘copycat’ measures. Eventually, this could come home to bite. It is not hard to imagine calls for this sort of technology to be used to prevent ‘hate speech’ or ‘disinformation’ during a crisis – in practice, anything that the censor dislikes.
It would also mean users being regularly accused of wrongful behaviour, with little to no ability to contest the AI’s conclusion. There would be inevitable significant false positives, including against medical, artistic and breastfeeding imagery, locking out entirely legitimate uses of devices. Not to mention the need to install similar software on digital cameras and the continued existence of analogue technologies, which predators would likely still exploit.
The government has previously justified the policy on the basis that 91 per cent of online child sex abuse material is self-generated by children groomed, tricked and exploited into abuse. This figure, which comes from the Internet Watch Foundation, is accurate but misleading. It refers to the percentage of material that is removed, rather than evidence that 91 per cent of child sexual abuse begins online with strangers.
The reality is actually scarier, and highlights why this technical solution is unlikely to work: the bulk (over 90 per cent according to the NSPCC) of contact child sexual abuse is perpetrated by people known to the child. Device-level fixes can only plausibly reduce sextortion of teens by remote predators, but they are no substitute for the unglamorous work of children’s safeguarding.
As Starmer looks to make something of what are likely to be his final months in the top job, don’t be surprised to hear even more announcements. These thought bubbles should be subject to heightened scrutiny, for they can be expected to come back to bite.
Matthew Lesh is a Public Policy Fellow at the Institute of Economic Affairs and Country Manager at Freshwater Strategy