The 2021 AI Index: Major growth despite the pandemic

The past decade was a pivotal one for the artificial intelligence (AI) industry, and 2020 saw AI substantially increase its impact on the world despite the chaos brought about by the COVID-19 pandemic. Technologists made significant strides in massive language and generative models; the United States witnessed its first drop in AI hiring ever – pointing to a maturation of the industry – while hiring around the world increased; and more dollars flowed to government use of AI than ever before, while colleges and universities offered students double the AI courses they did a few years ago.

These are just some of the findings from the 2021 AI Index, an annual study of AI impact and progress developed by an interdisciplinary team at the Stanford Institute for Human-Centered Artificial Intelligence (HAI) in partnership with organizations from industry, academia, and government.

"The impact of AI this past year was both societal and economic, driven by the increasingly rapid progress of the technology itself," said AI Index co-chair, Jack Clark. "With the index, we can actually measure and evaluate the changes, enabling leaders and decision makers to take meaningful action to advance AI responsibly and ethically with humans in mind."

The 2021 AI Index is one of the most comprehensive reports about AI to date, analyzing and distilling patterns about AI's impact on everything from national economies to job growth to the analysis of technical progress within AI research itself, and analysis of the diversity (or lack of) among the people who create AI systems. The new index shows that, in 2020:

• Private investment in AI substantially increased – despite the COVID crisis negatively impacting the economy in other ways
• China surpassed the US in significant scholarly work. Chinese-affiliated scholars were cited in more peer-reviewed journals than any other country's scholars, indicating China's AI research has increased in quality and quantity. However, the United States has consistently (and significantly) more cited AI conference papers than China over the past decade
• Synthetic media, colloquially known as deepfakes, are on the rise, with breakthroughs in the generation of synthetic text, imagery, and video demonstrating the progress of AI but also highlighting the potential for unethical or dangerous use
• Ethical challenges of AI applications have become a bigger focus for the AI community, with a significant increase in papers mentioning ethics and related keywords between 2015 and 2020
• Diversity in AI is low – in 2019, 45% of new AI PhD graduates who stayed in the United States were white, while 2.5% were African American and 3.2% were Hispanic. AI researchers are forming more affinity groups to try to improve diversity in the field, and these groups are seeing significant growth in their membership and impact: Black in AI members had twice the number of papers accepted at the Conference on Neural Information Processing Systems in 2019 compared to 2017, and participation at workshops held by Women in Machine Learning has grown from fewer than 200 participants in 2014 to more than 900 in 2020
• Since Canada published a national AI strategy in 2017, other nations have followed, with more than 30 countries committing to national AI strategies by 2020
• More AI PhDs took jobs in private industry rather than academia, and professors continued to leave higher education for roles in corporations
• Corporations have come to dominate the tools that AI researchers use, with corporate-backed software libraries (Google's TensorFlow and Keras, and Facebook's PyTorch) becoming the most popular frameworks on GitHub
• Government interest in AI continues to be significant, with the US government spending billions of dollars across civil and non-civil uses of AI. The current Congress has mentioned AI three times more than the previous one did.

Along with telling what happened in 2020, the index also highlights what researchers don't yet know about AI development and deployment. While assembling this report, the authors found a few areas where data sources are lacking or aren't – yet – comprehensive. For instance, most government spending data is inconsistently structured and frequently untrustworthy. AI ethics as a field has made various qualitative advances, but it's challenging to quantify these or model them over time. And data relating to diversity in AI – especially about the makeup of people at universities and companies – is relatively scarce.

"It's clear from the data that, in 2020, AI started to have a more significant impact on the world, while the technology continued to evolve very rapidly," Clark said. "Making this report has also highlighted for us the need for governments to invest more in gathering data about AI, and for researchers to develop new, harder tests for contemporary AI systems. Our bibliometric analysis shows that the United States and China have become peer nations with one another on AI development, when analyzed through a variety of dimensions."

AI Index steering committee member Barbara Grosz added, "It's encouraging to see in this year's index an increased commitment to examining matters of diversity in the field. The report makes evident, however, the lack of data available from academic institutions, scientific societies, and industry about the diversity of the AI research and industry communities – of students, scientists, and AI professionals. Diversifying the field is essential if it is to yield systems that work for all of society, and success requires that the research community and industry collect and analyze data that tracks progress."

To learn more, read the 2021 AI Index.

The authors of this article are Jack Clark, AI Index co-chair, and Daniel Zhang, AI Index research manager, and was first published at hai.stanford.edu.