AI scanner used in hundreds of US schools misses knives – BBC

A security firm that sells AI weapons scanners to schools is facing fresh questions about its technology after a student was attacked with a knife that the $3.7m system failed to detect.

On Halloween last year, student Ehni Ler Htoo was walking in the corridor of his school in Utica, New York, when another student walked up behind him and stabbed him with a knife.

Speaking exclusively to the BBC, the victim’s lawyer said the 18-year-old suffered multiple stab wounds to his head, neck, face, shoulder, back and hand.

The knife used in the attack was brought into Proctor High School despite a multimillion weapons detection system installed by a company called Evolv Technology.

Evolv Technology is a security firm that wants to replace traditional metal detectors with AI weapons scanners.

Instead of simply detecting metal, Evolv says its scanner “combines powerful sensor technology with proven artificial intelligence” to detect weapons.

When the system detects a concealed weapon – like knives, bombs or guns – it triggers an alert.

The company has publicly stated their system is highly accurate, and previously boasted its scanners can help to create “weapons-free zones”.

The company’s chief executive, Peter George, has also said that its systems “have the signatures for all the weapons that are out there”.

Previous press releases have listed the weapons the system can find – which include firearms, explosive devices, and knives.

The system is used in major stadiums across the US, and the Manchester Arena in the UK. The testers said Evolv should inform potential clients.

Despite this, the company has been expanding into schools, and now claims to be in hundreds of them across the US.

In March 2022, the Utica Schools Board bought Evolv’s weapons scanning system for 13 schools. It was installed over the summer holidays.

On 31 October, CCTV captured the perpetrator of the attack against Ehni Ler Htoo entering Proctor High School and passing through the Evolv weapons scanners, according to one source at the school who has seen the security footage.

“When we viewed the horrific video, we all asked the same question. How did the student get the knife into the school?” said Brian Nolan, Superintendent of Utica Schools.

The knife used in the stabbing was more than 9in (22.8cm) long.

The attack triggered an internal investigation by Utica’s school district.

“Through investigation it was determined the Evolv Weapon Detection System… was not designed to detect knives,” Mr Nolan said.

The scanners were removed from Proctor High School and replaced by 10 metal detectors. But the scanners are still operating in the district’s remaining 12 schools. Mr Nolan says the district cannot afford to get rid of Evolv’s system in its remaining schools.

Since that attack, Mr Nolan says three other knives have been found on students in other schools in the district where the Evolv systems continue to operate.

One of the knives was 7in long. Another was a curved blade with finger holes. Another was a pocket knife. Mr Nolan says they were all found because they were reported to staff – not because the weapons scanner had detected them.

“The kids [who had the knives] all said they walked right through the weapons detection system, we asked them about that… it truly, truly does not find knives,” he said.

After the stabbing, the wording on Evolv’s website changed.

Up until October last year, Evolv’s homepage featured a headline that boasted of “Weapons-Free Zones”. The company then removed that wording, and changed the text to “Safe Zones”. It has now been changed again and reads “Safer Zones”.

Evolv claims its system uses cutting-edge AI technology to find weapons. However, its critics say not enough is known about how the system works – or how effective this technology is at finding different types of weapons.

The BBC sent a detailed right of reply to Evolv, laying out what had happened at the school in Utica, and the decision of the school to stop using its system.

We also asked what Evolv had told schools about what its system could and could not detect, whether it had told schools that independent testing had found its systems could not reliably detect large knives, and whether it thought its systems were suitable for use in schools. Evolv did not answer the questions.

Conor Healy of IPVM, a firm that analyses security equipment, says Evolv has exaggerated how effective the system is.

“There’s an epidemic of schools buying new technology based on audacious marketing claims, then finding out it has hidden flaws, often millions of dollars later. Evolv is one of the worst offenders. School officials are not technical experts on weapons detection, and companies like Evolv profit from their ignorance.”

Playing fast and loose with marketing claims is unacceptable when you sell a security product used to protect young people, he added.

Although Evolv did not give the BBC a comment, it did direct it to a blog post from its CEO, Peter George, in which he defends the lack of detail in how much the firm has said about how the technology works.

“Marketing weapons detection security requires a delicate balance between educating stakeholders on new technology and not providing bad actors with the information they could use to do harm,” he said.

“So, while public-facing marketing materials are intentionally not specific, we communicate all aspects of the Evolv Express system – including limitations and capabilities – with the trusted security professionals at our customers, partners, and prospects”, the blogpost read.

The BBC contacted seven other school districts that all use Evolv weapons scanners. Five did not respond. Two said they did not wish to comment.