Artificial intelligence technologies: deep neural networks, large language models, fundamental models, what’s next?

Vorontsov, Konstantin Vyacheslavovich
D.Sc. in Physics and Mathematics, Professor of the RAS, Head of the Department of Machine Learning and Digital Humanities, MIPT University, Head of the Department of Mathematical Forecasting Methods—Head of the Laboratory of Machine Learning and Semantic Analysis, the Institute for Advanced Research of Artificial Intelligence and Intelligent Systems of MSU,
Moscow, Russia
voron@mlsa-iai.ru

Abstract
The last «winter of artificial intelligence» ended first with a breakthrough in the field of computer vision (2012), then in the field of natural language processing and understanding, with large language models (2022). Deep neural networks for processing and analyzing images, signals, texts, graphs—what do they have in common? Is it really possible to talk about intelligence, or is this just another imitation of it? Is it possible to predict trends in the further development of these technologies? What should you do first so as not to be forever catching up? Should all efforts be focused on creating artificial general intelligence (AGI), or are there more important things to do? Is it possible to spot new ideas that will change the rules of the game among the artificial intelligence hype?

Keywords: machine learning, deep learning, natural language understanding, generative pretrained transformers, learnable data vectorization, foundation models