Yoshiro Mori likely thought he had dodged a bullet when the International Olympic Committee and senior Japanese politicians expressed their opinion that his apology for his sexist remarks had settled the matter.

But the words of a man considered one of Mori’s closest overseas allies shattered that confidence and brought home the reality that he would have to resign as head of the Tokyo Olympic organizing committee.

Mori on Feb. 3 said at a Japan Olympic Committee Council meeting that female board members prolong meetings because they are competitive and talk so much. The following day, he apologized at a news conference and retracted the comments.

But he fueled public criticism toward him by appearing annoyed with reporters who peppered him with questions about whether he would resign.

Still, an IOC official the same day touched upon Mori’s apology and said the matter was concluded in the IOC’s view.

Prime Minister Yoshihide Suga said in the Diet that Mori’s comments hurt Japan’s interests, but he would not comment on whether he thought Mori should stay or go.

And while large numbers of Olympic volunteers said they no longer wanted to help during the Games because of Mori’s remarks, organizing committee officials and ruling coalition lawmakers urged him to remain in his post.

But things changed when Mori attended a Feb. 8 videoconference in Tokyo with Toshiaki Endo, a vice president of the Tokyo Organizing Committee of the Olympic and Paralympic Games, and Toshiro Muto, the director general and CEO of the committee.

Also joining was John Coates, chairman of the IOC Coordination Commission for the Tokyo Olympics.

As Tokyo Olympic organizing chief, Mori, 83, often communicated with Coates, and they are said to have developed a strong relationship.

Mori was not prepared for what Coates had to say at the videoconference.

Coates said in no uncertain terms that corporate sponsors at the highest levels had expressed strong reservations about Mori’s comments. Taken aback by the harsh tone Coates was using, Mori could only say he was “in a serene frame of mind,” indicating he had made up his mind to step down.

The following day, the IOC issued a statement that overturned the earlier comment about the matter being concluded. The new stance was that Mori’s comments “were absolutely inappropriate and in contradiction to the IOC’s commitments and the reforms of its Olympic Agenda 2020.”

NO ONE WITH A FUTURE CONSIDERED

Mori had often indicated in the past that he wanted to step down because of his continued treatment for lung cancer and hand over the baton to another former prime minister, Shinzo Abe.

But with public opinion polls showing a vast majority of voters opposed to holding the Tokyo Olympics this summer, Mori felt that Saburo Kawabuchi was the only one capable of pulling off a successful Games.

Not only was Kawabuchi, 84, the first chairman of the J. League and subsequent president of the Japan Football Association, but he was also involved in inaugurating the B. League as president of the Japan Basketball Association.

When the Japan Sports Agency was established in 2015, Mori lobbied to have Kawabuchi head the organization. Although Kawabuchi was not chosen because of his advanced age, he later led the Japan Top League, an organization tasked with energizing various team sports in Japan.

A source in the Japan Olympic Committee said that ordinarily the chairman of a globally known Japanese company or even Seiko Hashimoto, the state minister for the Tokyo Olympic and Paralympic Games, would be considered as Mori’s successor.

“But with the negative public opinion toward the Olympics, the organizing committee president would face a major decision of either going ahead with the Games or canceling it,” the source said. “It was likely difficult to ask someone who still has a future to take on the responsibility of making such a decision.”