Many schools and universities have adopted remote proctoring software during the COVID-19 pandemic. These tools suffer from bias and lack of validity, enable surveillance, and raise other concerns. There has been an uproar against remote proctoring from students, non-profits, and even senators.

In November 2021, the Financial Times published an article on a product called Sciolink that presents an entirely one-sided view of remote proctoring. It almost exclusively quotes the creators of Sciolink and provides no context about the limitations and risks of remote proctoring tools.

See for yourself where the article goes wrong. The contents of the article are reproduced in their entirety on the left, and our annotations are on the right. Link to the original article.

For more details about the pitfalls, read the blog post which introduces this article here.

AI tested as university exams undergo digital shift
A woman behind a table, looking at her laptop screen
By exploiting a computer’s hardware, such as its camera and microphone, software can spot suspicious actions
For a few minutes earlier this year, a university entrance exam disrupted Slovakia’s phone network. As Covid-19 spread, Scio, a Czech education company that manages entrance exams for more than 80 faculties at Czech and Slovak universities, switched from holding its tests in person to carrying them out online. But, during one exam in Slovakia, the third-party system it used crashed for more than an hour, prompting thousands of worried students to call Scio’s Czech hotline.
“We were told that . . . for a few minutes on one provider it wasn’t possible to call Slovakia because 10,000 people had decided to call the Czech Republic to ask what was happening,” says Martin Drnek, a manager at Scio.
The episode was one of the many forms of disruption caused by the rapid shift from the physical to the online world ushered in by the pandemic. Education worldwide has been thrown into chaos by Covid-19, with lockdowns and other disruption affecting more than 220m tertiary-level students, according to Unesco, the UN’s cultural organisation. At universities, not only have classes moved online, but entrance exams have undergone a digital shift to adapt to a Covid-affected world.
One of the biggest challenges of such exams is that there are — or should be — no second chances. All candidates must take an exam under the same conditions and at the same time, for the sake of fairness and to avoid questions leaking. So, now that exams are being conducted online, artificial intelligence and machine-learning technology are being harnessed to identify and prevent cheating.
After negotiating the first year of the pandemic using software provided by other companies, Scio developed its own to test and monitor university applicants. It paired with Born Digital, a Czech start-up founded by Tomas Malovec in 2019, which specialises in AI and machine learning techniques. The resulting online exam system is due to go live this year.
Beyond the most basic problem of ensuring that the system is stable, one of the biggest challenges of switching to online exams, says Drnek, is familiarising students with a system they may never have used before — both so that they can perform to the best of their abilities and so they do not accidentally do anything that could be construed as cheating.

The software, called ScioLink, addresses this in various ways, says Malovec. To spare students any problems in downloading the software to monitor them during exams, it is embedded in the exam webpage.

Then, to ensure the right person is performing the test, ScioLink photographs the candidate’s face and identity document using their webcam. It requires them to perform a 360-degree scan of their surroundings by webcam, and monitors their behaviour during the test via the webcam, along with the computer’s other systems, including the microphone. If a candidate engages in actions deemed to be suspicious, the software warns them in real time.

“In most cases, people are not trying to cheat. They just don’t know they are doing something forbidden. For example, someone calls and they pick up their phone — it could be just their mother asking them something,” says Malovec. “[If they are warned] they can correct it immediately. ”

If someone is trying to cheat, however, Born Digital’s AI model has various ways of catching them. It can detect whether other people are present in the room, and has been trained to recognise if someone is wearing earphones to be fed information. It can also recognise forbidden aids such as books or devices, such as smartphones and calculators — something Malovec says was not straightforward. “A phone is frequently just a black rectangle, so it is not simple. There was a lot of playing with things like this [during development],” he says.

Born Digital’s software does not itself rule whether someone has cheated. Instead, it alerts examiners to the point during the exam at which a potential breach occurred. This allows them to review footage of the incident without trawling through hours of otherwise unimpeachable exam conduct.
Malovec says students will be “inventive” when it comes to evading the monitoring technology, but points out that the software is always learning . He also predicts universities will increasingly embrace online tests, saying this is only the beginning of a period of change.
Juraj Mokry, vice-dean of the medical faculty at Comenius University in Bratislava, says online entrance exams can also indicate whether candidates will be able to cope with online teaching, which he expects to continue once the pandemic subsides. “If we have online teaching, it’s good to accept students who have experience with online education and who know how to work online,” he says. “It’s not only about excellence in chemistry and biology. It’s also testing how they can handle problems with computers and everything.”
However, he stops short of arguing that online tests should completely replace in-person exams. “We should offer some variability,” he says. “Like with education, one person prefers to learn from a screen, another from a textbook, another prefers to listen. It should be the same here.”

Copyright to the original article belongs to the Financial Times, and the article can be read in its original form in full at ft.com. It is republished here for the purposes of critical commentary.

We re-used the code and the copyright notice above from Molly White et al.’s The Edited Latecomers’ Guide to Crypto to generate our annotated article.