‘Social order could collapse, resulting in wars’: 2 of Japan’s top firms fear unchecked AI, warning humans are ‘easily fooled’

Japan’s largest telecommunications company and largest newspaper have issued a stark warning about the artificial intelligence, saying that if it's allowed to go unchecked, it could result in the collapse of society as we know it.  The technology is also "expected to improve labor productivity to a certain degree," they add.

Nippon Telegraph and Telephone (NTT) and Yomiuri Shimbun Group Holdings on Monday released a joint proposal on shaping generative AI, urging officials to revise laws surrounding the technology and saying AI is already having negative impacts on users.

“If generative AI is allowed to go unchecked, trust in society as a whole may be damaged as people grow distrustful of one another and incentives are lost for guaranteeing authenticity and trustworthiness,” the paper read. “There is a concern that, in the worst-case scenario, democracy and social order could collapse, resulting in wars.”

Humans, the two companies said, are incapable of fully controlling this technology, noting that AI “lies with confidence” and humans “are easily fooled.” The design of AI systems to grab users’ attention has also been detrimental, the paper argues, damaging “autonomy and dignity, which are essential values that allow individuals in our society to be free.”

The proposal is further critical of AI being used in schools, especially for younger students whose “ability to make appropriate decisions has not fully matured.”

The group calls on Japan’s government to take action, including the passage of laws that protect elections and national security and stronger copyright laws. Recognizing that any legal reforms will take time, the organizations are calling on the media and industrial leaders in the country to introduce rules surrounding the dissemination of AI technologies.

While the report was cautious of AI’s downsides, it was not dismissive of the technology as a whole, noting the genie is long out of the bottle—and there is no going back.

“AI technology itself is already indispensable to society,” the companies said. “If AI technology is dismissed as a whole as untrustworthy due to out-of-control generative AI, humanity’s productivity may decline.”

Japan, like the U.S., has been gridlocked on the matter of AI regulation. The European Union, meanwhile, has already ratified a provisional agreement setting AI rules that could set the tone for how other countries acts next.

The AI Act, as it is called, sets rules for a number of industries, from banking to transportation, as well as guidelines for how law enforcement can use AI in their duties. Also covered is how large language models can be built, so as to protect both individual privacy and corporate secrets.

This story was originally featured on Fortune.com