ANTWERP, Belgium--A gymnastics world championships here showcased a next-generation look, featuring four to eight video cameras around each apparatus, including the horizontal bar and the pommel horse.

The cameras transmitted feeds to more than 10 computer monitors that lined a section of temporary stands looking down on the Artistic Gymnastics World Championships. 

Staff workers were seen watching the screens to see if the artificial intelligence system was operating without a hitch, during the competition held from September through October last year. 

The AI system used footage from those cameras to recognize the athletes’ bone and joint positions and immediately identify the gymnastic skills demonstrated.

The system was consulted only when judgments were divided among the judges and in the event of athlete inquiries on the scoring results.

The judges had only to press a button on a personal computer to jump instantaneously to scenes of the maneuver they wished to scrutinize. That immediately invoked 3-D imagery, along with numerical figures for the angles of different body parts and other data.

ONLY AN ASSISTIVE MEASURE FOR NOW

The judging support system powered by AI was introduced for all gymnastic apparatuses of the world championships for the first time during the competition in Antwerp.

Morinari Watanabe, president of the International Gymnastics Federation (FIG), told a news conference that he believes the system will allow athletes to be rated in a fair and transparent manner.

The scoring of gymnastics by humans is a demanding process.

A fixed set of referees serve from morning through evening so the standards remain consistent. The judges keep a constant eye on the athletes’ performances and jot down a string of different symbols for different skills on a sheet of paper ready at hand.

They sometimes have only 15 minute breaks in a day during competitions, which can last for a week in major sporting events such as the Artistic Gymnastics World Championships, sources said.

Masao Kondo, 55, an international judge who heads the Japan Gymnastics Association’s referee commission, said that gymnasts over the past 30 years have come to flip and twist more, and engage in immeasurably more difficult maneuvers.

Meticulous rules have also been set on body angles, hold times and other details, Kondo added.

“It’s quite a lot of work to keep up with what’s going on with your eyes and brain,” he said.

Kondo said that judges seldom misrecognize skills in performances of a familiar gymnast. 

“But if a gymnast you have never seen abruptly does a quadruple twist, you may fail to assess that correctly,” he said.

Judges therefore watch the training of athletes at the venue to prepare for their mission.

Refereeing errors still have been repeated.

For example, Kohei Uchimura of Japan performed a triple twisting double back tuck, aka the "Ri Jong Song," during the floor exercise final at the 2011 Artistic Gymnastics World Championships.

His twisting was initially misidentified as double until the decision was modified later after a coach lodged a protest.

There has been no end to similar incidents, even though Uchimura said he was personally happy because the controversy showed his twisting had been too quick to discern.

The judging support system was first introduced for some of the gymnastic apparatuses during the 2019 world championships and was implemented for all 10 apparatuses during the last competition.

“Some skills in still rings are not recognized as such if the shoulder angle deviates more than 45 degrees from the required position,” Kondo said. “The human eye has no way to distinguish between 44, 45 and 46 degrees. The system is therefore backing us up.”

During the last world championships, Japan’s national team didn't achieve the many high scores it had hoped in the still rings and floor exercise.

“All angle figures (such as the shoulder opening) are there because the system is working,” said Hiroaki Sato, head coach of the national team. “We were told those figures were not so good.”

The AI system has so far only been introduced as a supportive measure to help score the difficulty ratings of gymnastic skills.

It remains to be seen if its use will spread in the future to cover aesthetic scoring as well.

GYMNASTS OFTEN UPSET WITH JUDGES' SCORES

The initial push for the AI system development came in September 2015, when Watanabe, the current FIG president, saw Fujitsu Ltd.’s golf swing diagnostic tool and quipped, “Robots could be scoring gymnastics at the Tokyo Summer Olympics,” which were scheduled for 2020 and postponed to 2021.

Athletes who look unhappy with the judges' scores are often shown on TV in scored sports, including gymnastics.

“That negatively affects the corresponding sports,” Watanabe said. “Nobody has similar looks on the face in timed sports (such as track and field and swimming).”

He said he thought an AI-assisted system could provide a solution.

Technologies at the time could not even ascertain, by using sensors, if a person was looking forward or backward or if the person was standing normally or upside down.

More than 150 Fujitsu engineers were assigned to the development work. They sorted out movements that are common to a total of 1,400 or so men’s and women’s gymnastic skills and fed them into their AI system.

The pommel horse presented a particularly difficult challenge in deciding where one gymnastic skill ends and another begins. In still rings, the AI system mistook a moving strap as part of a human hand.

The engineers solved similar problems by improving the accuracy of their feed data. They have applied for more than 200 patents associated with their work.

Gymnastic skills are assigned difficulty ratings of A to J at 0.1-point intervals. The difficulty (D) score is determined additively, partly on the basis of those difficulty values and the combination of skills.

The sum of the D score and the execution (E) score, which evaluates aesthetic aspects, makes the final, aggregate score. An E score of 10 points is deducted for falls, poor landings and other mistakes.

The use of the judging support system was limited to the D score in last year’s world championships, but the E score also contains components on which judgments could be made with currently available AI technologies, such as body angles.

Views are divided within the gymnastic circles on how much should be entrusted to AI systems in the years to come.

One attendee at the news conference asked whether the day will come when human judges are replaced by an AI system.

In response, the president of the FIG Women’s Artistic Gymnastics Technical Committee, which overseas refereeing, said she believes that a similar system is only supposed to support judges for the consistency of ratings and will not replace them.

Watanabe, however, said he is weighing the option of allowing an AI system to take the lead in making judgments.

“Our purpose has always been, from the beginning, to produce aggregate scores,” the FIG president told The Asahi Shimbun. “The jury members are afraid of losing their jobs. The AI system is currently only supporting humans, but humans will be playing a supportive role in the end.”

ATHLETE NOT KEEN TO BE RATED BY AI SYSTEM

Gymnastics competitions previously had rating components such as “virtuosity” and “originality.” The corresponding ratings owed much to the subjective views of the judges.

Identical numbers of judges were appointed from the Western and Eastern blocs during the Cold War to ensure a balance and fairness. 

The rules have since been changed for reducing inconsistencies among the judges. It has now become possible to rate not only the D score but also most portions of the E score on the basis of objective standards, including numerical figures.

The E score still continues to vary among different judges.

“Performances cannot be measured with simple numerical figures,” said an international judge. “As a human, you may sometimes wish to turn a blind eye to an angle deviation of one degree if the movement is so superb or pay respect to a high school student who has taken the plunge at such a difficult level."

That characteristically human component could lead to unfairness, but athletes may view it differently.

“I have always been trying to give performances that move the hearts of the referees and the audience,” said a gymnast from a corporate team. “There would certainly be no human errors if everything were to be entrusted to an AI system, but I still wouldn’t want to be rated by such a system.”

The current Code of Points gives no strict tolerance ranges for angles.

Hisashi Mizutori, manager of the Japanese men’s national team, said he is realizing the need to have the Code of Points comply with the AI era so no angles will be defined in an AI system at the discretion of its configurators.

Athletes have been trying, in planning their performances, to assess trends in the rating standards from year to year and from event to event.

Mizutori said that, if system introduction costs were to drop in the future, allowing the use of AI systems in training and in more sporting events, “That would allow you to make daily efforts of the sort, ‘I will try this and that so the angle will be rated as 16 degrees.’ And you would no longer be disturbed by gaps in the scoring standards from one sporting event to another.”

Hidenori Fujiwara, 53, the head of Fujitsu’s sports business promotion division who was in charge of the system development, said he believes that AI systems will not completely replace human judges.

Facial expressions and synchronicity with music do not render themselves readily calculable by an AI system.

“Let AI systems watch over what they can watch over, while humans refine their sensitivity,” Fujiwara said. “I am hoping to help create a world where AI systems coexist with humans.”

Apart from fairness, members of the gymnastic circles have presented ideas on the applications of AI systems, such as shortening competitions, making abstruse gymnastics more intelligible to the audience by showing AI scores simultaneously on a big screen at the venue, and realizing an AI-assisted coaching system in the future.