Soccer stopped being ‘just a game’ a long time ago, and by now it has become impossible to imagine a soccer pitch without technology. That is why each World Cup features new innovations. Over the past years, there has been an increasing focus on refereeing, and on efforts to make the game fairer using new technology. This year’s World Cup in Qatar, during which the Netherlands will face the US tomorrow, also features a technological first. Because for the first time ever during a major tournament, off-the-field Video Assistant Referees will be assisted by artificial intelligence.

Some six years ago, a referee decided to reverse a decision during a Dutch Eredivisie game and changed a preciously drawn yellow card into a red one. It was the first intervention during a game in an official league by the video assistant referee, a system that allows on-pitch referees to review important decisions. The video assistant referee, or VAR, monitors video footage of the game away from the pitch, and helps the on-pitch referee make the right decision.

Offside software

On top of that,video assistant referees in the Netherlands have also been using offside software for a few years now to make more accurate offside decisions, Koen de Koning explains. He works as a system engineering, instrumentation and data acquisition & control engineering technician at the Dynamics and Control group of the department of Mechanical Engineering. Furthermore, he recently joined the Tech United team, where he focuses on new neural networks to make robot soccer more robust.

De Koning: “When it’s impossible to make a decision with the naked eye based on replay images, the VAR has a choice between a method with two-dimensional lines and a method with three-dimensional lines, depending on the situation. The latter method uses a digital red line to indicate the attacker’s position on the pitch, and a blue line to indicate the defender’s position. The VAR can draw these imaginary lines based on camera footage.”

Soccer robots, incidentally, don’t need to worry about being offside, Tech United’s hardware coordinator Harrie van de Loo says. “We apply FIFA rules from human soccer during games, but offside is still a bit too complicated in robot soccer. Much is preprogrammed, and every design and bit of software needs to be handed over after each tournament. That significantly minimizes the risk of cheating.”

False certainty

The dotted lines used by the VAR often lead to intense debates. In some cases, offside calls are a matter of millimeters. And that leads to a false certainty, because it simply isn’t possible to draw these lines at a millimeter-level accuracy. Not only does the accuracy of the VAR’s decision depend on when he or she decides to pause the image, it’s also impossible to determine the exact kick point with scientific certainty. In some cases, a player might even be in offside position from the perspective of one camera, but not when viewed from the perspective of another camera.

The latest semi-automated offside technology that has now been introduced in Qatar needs to help officials make more accurate offside decisions. The tracking cameras in the stadium track 29 data points of each individual player – including all limbs and extremities that are relevant for making offside calls – up to 50 times per second. The official World Cup ball is fitted with a special sensor that sends data about the ball’s position to the video operation room at a rate of 500 times per second. This makes it possible to detect the kick point much more accurately than any traditional camera could. When a player is in offside position, the VAR is notified automatically, after which he or she can make an official call.

Training networks

The most important thing, De Koning says, is to train the neural networks to recognize an offside position. “Not all camera images are equally suited. Certain types of lenses, for example, can cause a perspective distortion. You could solve that problem by generating synthetic data that you can then use to train the system with; that will also make the system a bit more accurate.”

De Koning is currently investigating whether it’s possible to use extremely fast You Only Look Once – YOLO – algorithms to track Tech United’s soccer robots. “Quick decisions are important in robot soccer too. I understand that supporters aren’t that happy with the ‘technologization’ of soccer, but artificial intelligence can definitely make the game fairer and faster. That means that it will take the VAR seconds to make a decision instead of minutes, and that makes a difference. A game of soccer is long enough as it is.”