Japan’s AI paradox: Bridging the gap between technological capacity and commercial implementation

Critical examination of the AI landscape in the land of the rising sun

In the swiftly evolving AI landscape, Japan stands at a crossroads, marked by a paradox that encapsulates both its prowess and challenges. According to the Tortoise AI Index, the nation ranks 12th overall in the global AI race — a position that speaks volumes about its capabilities and potential.

Japan shines in terms of its AI infrastructure and development metrics, securing the 6th place in the Index by Scale. These stats are indicative of the nation’s high absolute AI capacity and foundational aspects necessary for AI innovation.

Yet, this impressive façade carries underlying issues in commercial implementation, research, and government strategy, where Japan finds itself at the 23rd, 20th, and 18th positions respectively. As a result, the country’s Intensity score — its AI capacity relative to the country’s economy — lands at the rather unremarkable 25th spot. 

Tortoise rankings unveil a critical insight: despite its technological resources and robust infrastructure, Japan’s AI ecosystem struggles with applying its advancements in a commercial context and aligning them with governmental support. To put it bluntly, the expectation of Japan as a tech powerhouse, given its sizable population and economic stature, is significantly higher, suggesting the need for a recalibration of the nation’s AI implementation strategies.

The tech core is there, definitely

Emphasizing Japan’s undeniable technological edge and vast potential, the Japan Patent Office reports a consistent rise in domestic applications for AI-related inventions since 2014, with approximately 9,000 applications filed in 2021 alone. Computer vision and image processing remain the most popular fields, revealing a strong focus on automation, precision, and efficiency.

In addition, Fujitsu leads in AI-related patent applications among companies, with over 1,400 domestic patents published as of June 2023, followed by NTT, Hitachi, and Canon.

What’s more, the end-user expenditure on AI systems in Japan, encompassing software, hardware, and services, stood at around $2.7 billion in 2022. Projected to grow at a compound annual growth rate of over 23% until 2027, the market size is expected to surpass $7 billion, reflecting a burgeoning sector poised for significant expansion. 

Strong vision, soft touch 

At the heart of Japan’s strategic vision for its technological future is Society 5.0, a bold initiative aiming to transform the nation into a “Super Smart Society.” This objective was crystallized within the framework of the 5th Science and Technology Basic Plan, endorsed by Japan’s Cabinet Office in January 2016, followed by the 6th Plan in 2021.

In a nutshell, Society 5.0 aspires to integrate cyberspace and physical space to an unprecedented degree, facilitating paradigm-shifting economic advancements, with AI playing a central role.

Image by Mayumi Fukuyama, Research & Development Group, Hitachi, Ltd.

This multi-layered large-scale initiative is grounded in the belief that a human-centered society can leverage AI to enhance quality of life across multiple domains. This includes healthcare for more personalized and efficient treatments; agriculture for optimized crop management and reduced labor; and urban planning for streamlined traffic, improved public services, and reduced energy consumption among others. 

Parallel to the vision of Society 5.0, Japan has also been forward-looking in establishing a legal and regulatory framework conducive to AI innovation. The Ministry of Economy, Trade and Industry laid out AI implementation guidelines in 2022, emphasizing proactivity and responsibility in AI development.

The country’s initiatives at the state level also include serious efforts to address military AI challenges, as well as this year’s establishment of AI Safety Institute to guard privacy and uphold ethical standards.

Having said that, Japan’s AI motto diverges from the more stringent regulatory models pursued by the EU. Preferring a substantially “softer,” more lenient approach, the Japanese government aims to foster an AI ecosystem that encourages more free-flowing creativity and experimentation, suggesting a belief in the capacity of industries and researchers to self-regulate.

Falling short 

Given its obvious technological edge and welcoming AI policies from the government, why is Japan lagging behind in Intensity metrics? The short answer is a discrepancy between its strong tech infrastructure and the actual application of AI in the commercial sector and research areas proportional to the nation’s resources. 

Interestingly, Nomura Research Institute forecasts that AI robots could replace nearly half of Japan’s workforce in the next 10 to 20 years, yet it anticipates simultaneous growth in creative and design jobs. Just as the automobile revolution led to the decline of horse-related jobs but fueled a much larger auto industry, AI expansion is driving Japanese companies to adopt or consider using generative AI (GenAI). In fact, 41 out of 100 companies are already implementing it, according to a recent survey by Asahi Shimbun, one of the biggest media brands in Asia.

The trouble is that this pace is apparently too slow. Having examined Japanese AI companies in the recently published Global GenAI Landscape — much to our own surprise — Japan didn’t place among Asia’s leaders when measured by the number of proprietary GenAI models.

Some sources argue that the country’s GenAI shortcomings stem from its relative deficiencies in deep learning and software development, coupled with a looming shortage of almost 800,000 engineers by 2030. The nation’s shortfall is also likely exacerbated by the absence of privately owned supercomputers essential for training LLMs.

Upping the ante

Nevertheless, Japan has been taking concrete steps to combat these GenAI deficits. First of all, the government is finally putting Fugaku, the world’s most powerful supercomputer based on the HPCG and Graph500 rankings, to work on developing new language models. This is part of a broader strategy that includes investing close to $50 million to build a new supercomputer in Hokkaido, specifically for the purposes of producing LLMs for GenAI development.

In line with this, Japan’s Education Ministry is also overseeing the creation of a novel GenAI tool designed to produce scientific hypotheses by analyzing research papers and experimental images. The cost is estimated to be around $210 million per each research field, with the project targeting initial applications in medicine, followed by broader scientific use. 

Prime Minister Fumio Kishida is actively supporting GenAI’s nationwide integration. This agenda is highlighted by a meeting with Sam Altman at the end of last year, leading to the establishment of OpenAI’s new branch in Japan. Recent discussions with other tech bellwethers, including Meta’s Mark Zuckerberg, underscore Japan’s public sector interest in forging global partnerships to advance its GenAI capabilities (which is especially relevant in view of Meta’s latest strides in the arena).

Corporate pursuits

On the corporate front, Japanese companies like SoftBank and NTT are beginning to produce their own GenAI solutions and ramping up AI investment volumes, while other big names like Toyota and Honda are now utilizing AI to refine production processes, increase productivity, and reduce labor costs.

At the same time, the newly founded Rapidus consortium, backed by eight firms including Sony, aims to mass-produce 2nm memory chips by 2027, committing to invest $36 billion over the next decade. This move is part of a strategy to triple the sales of domestically produced semiconductors, surpassing $105 billion by 2030, which is supposed to position Japan ahead of its archrival South Korea. 

Other startups in Japan are developing new AI-backed applications, including diagnosis assistance, drug discovery, and customized treatment in healthcare; improved customer service, fraud detection, and risk assessment in finance; and customer behavior analysis and inventory management in retail among others.

Incidentally, Japan remains a global leader in industrial robotics as measured by factory installations, ahead of the US and second only to China. Now, the nation is adding AI to the equation, thereby creating fresh solutions to longstanding challenges.

These AI-powered robotic initiatives range from providing companionship to dementia patients to addressing disability-related loneliness through innovative forms of socialization. Breakthroughs extend to the food industry, from agriculture to the culinary world, where AI robots are increasingly utilized for planting and harvesting, while some are even in charge of pasta stations. 

The sun’s glowing in the East? 

In navigating the gap between its technological know-how and commercial implementation, Japan exemplifies a unique paradox within the international AI landscape. Recent collaborations with global tech leaders and strategic investments in AI by Japanese companies signal a robust shift to finally harness the nation’s full AI potential. The same goes for the concurrent establishment of government-led policies addressing the nation’s AI shortcomings through hardware dominance.

Notably, as Japan continues to AI-innovate across sectors, from healthcare to robotics — thanks to the efforts of numerous startup entrepreneurs — the country’s journey mirrors the ongoing global trend of GenAI integration. This, combined with Society 5.0, is poised to change Japan’s Intensity score drastically in the near future, assuming these initiatives are effectively realized.