A reader who used an artificial intelligence chatbot to fact-check a newspaper story questioned the accuracy of a quote from Prime Minister Shigeru Ishiba regarding the consumption tax.

That prompted the surprised Asahi Shimbun reporter who wrote the article, in turn, to check out Grok for its accuracy and effectiveness.

Users of X, formerly Twitter, have utilized Grok for fact-checking in a spate of cases.

The hope is that the built-in intelligent app on the social media platform will efficiently decide if the posts they are particularly interested in are accurate.

But is it possible for a generative AI system to accurately determine the veracity of online posts?

‘EXAGGERATED’ OR ‘FALSE’

The news article in question was published in the morning edition of The Asahi Shimbun on May 31 in Japanese. It retraces a fact-checking process for a comment made by Ishiba over the consumption tax.

The Asahi Shimbun in June received an opinion email from a registered reader regarding the article.

Ishiba had noted that changing the consumption tax rate could take “an entire year” to update supermarkets’ relevant systems and other facilities, underscoring his cautious stance toward a possible tax cut.

The Asahi report delves into the Japanese leader’s statement and concludes that what Ishiba asserted was “virtually correct.”

The email insists that “this article by The Asahi Shimbun appears to be related to vested interests” and that the newspaper publisher is obviously “mounting a campaign to defend the government.”

The email continues, “I got Grok to fact-check the story.”

The AI model reportedly decided that the “dialogue from Prime Minister Ishiba is likely to be exaggerated or factually false.”

Shortly after Ishiba made the comment, a commercial broadcaster’s infotainment show aired an interview with a store owner in a Tokyo shopping district who stressed that “the tax rate settings for our cash registers can be adjusted within a day.”

The clipped video and image of the TV program went viral through social media, fueling controversy.

Working on the article in question, the Asahi journalist first examined the prime minister’s remark in detail. Ishiba, to be precise, was speaking about not the cash registers but “an update to the supermarket system.”

The reporter contacted the Finance Ministry for confirmation. The ministry explained that Ishiba was referring to the “point-of-sale (POS) system” intended to tally sales from multiple cash register counters within a store.

The Asahi Shimbun reached out to Toshiba Tec Corp., Fujitsu Ltd. and NEC Platforms Ltd. The three major POS system providers said that such an update would require “several months to more than a year,” describing the prime minister’s view as “appropriate” or “simply natural.”

One of them similarly added that redoing its system in one day is “unrealistic.”

Wondering why the reader turned to Grok over the issue, The Asahi Shimbun telephoned the man who sent the email.

The subscriber said he is 70 years old and lives in Chiba Prefecture, close to the capital. He described himself as a long-time reader of The Asahi Shimbun.

According to the man’s accounts, his eyesight began to deteriorate five or so years ago. This prompted him to thumb through the news much more frequently on YouTube and Twitter (now X).

His wife died suddenly last year. The husband himself was diagnosed with a rare cancer.

The man, at a loss for what to do given the paucity of information about his illness, happened to seek the help of Grok. He immediately found himself accessing not only the data he wanted but also even the cited medical journals.

“My doctor admired my probe for its accuracy,” he recalled. “I succeeded in my research in a way that would have otherwise been totally impossible on my own.”

He has since been “hooked” on Grok.

Grok is a conversational AI mounted on X, previously Twitter. It is designed to communicate naturally with humans by gathering information via posts on X and elsewhere in a real-time basis.

The application was released in 2023 by xAI, a digital intelligence developer led by billionaire and entrepreneur Elon Musk.

While X users can use Grok free of charge, instances of the model repeating improper comments, such as praising Nazi leader Adolf Hitler, have been reported.

When the Asahi reader came across the fact-checking story in the newspaper, he felt that the “excuse of not reducing the tax just because updating the system would take time is not reasonable.”

This thought prompted him to ask Grok for an opinion. The AI app’s answer was different from the Asahi’s conclusion and aligned more closely with his “intuition.”

The man later realized his mistake after being informed of the detailed reasoning behind the article.

“I understand that the POS system differs from a cash register after listening to your explanation,” the man acknowledged.

PICK UP FALSE DATA ONLINE

An expert offers tips on the validity of utilizing generative AI for fact-checking purposes.

Atsushi Udagawa, an associate professor of media studies at Musashi University, said that generative AI models gather and store information from the internet. This mechanism allows them to produce texts based on the collected data.

Udagawa pointed out that AI applications can also pick up false opinions or biased views in the digital world as well.

He believes that is the likely reason that many people rely on generative AI despite this clear disadvantage.

“A lot of individuals hold the naive belief that machines--given the nature of machinery--must be unbiased,” Udagawa said. “The process by which generative AI derives its conclusions is like a black box. It is ultimately impossible to conclusively verify whether AI’s responses are based on sound evidence.”

Grok itself seemed to share this sentiment.

The Asahi reporter inquired of the AI model in early August about the “appropriateness of taking advantage of you for fact-checking.” The answer from Grok was that its views “may be biased or limited.”

Grok said toward the end of its conversation that it will be “safer for you to make judgments yourself and directly check reliable sources,” since “I am just a tool, not a perfect judge of everything.”