StartofAI：2022年人工智能AI报告.pdf

State of AI ReportOctober 11,2022#stateofaistateof.aiIan HogarthNathan BenaichAbout the authorsNathan is the General Partner of Air Street Capital,a venture capital firm investing in AI-first technology and life science companies.He founded RAAIS and London.AI(AI community for industry and research),the RAAIS Foundation(funding open-source AI projects),and Spinout.fyi(improving university spinout creation).He studied biology at Williams College and earned a PhD from Cambridge in cancer research.Nathan BenaichIan Hogarth Ian is a co-founder at Plural,an investment platform for experienced founders to help the most ambitious European startups.He is a Visiting Professor at UCL working with Professor Mariana Mazzucato.Ian was co-founder and CEO of Songkick,the concert service.He started studying machine learning in 2005 where his Masters project was a computer vision system to classify breast cancer biopsy images.Introduction|Research|Industry|Politics|Safety|Predictions#stateofai|2stateof.ai 2022Artificial intelligence(AI)is a multidisciplinary field of science and engineering whose goal is to create intelligent machines.We believe that AI will be a force multiplier on technological progress in our increasingly digital,data-driven world.This is because everything around us today,ranging from culture to consumer products,is a product of intelligence.The State of AI Report is now in its fifth year.Consider this report as a compilation of the most interesting things weve seen with a goal of triggering an informed conversation about the state of AI and its implication for the future.We consider the following key dimensions in our report:-Research:Technology breakthroughs and their capabilities.-Industry:Areas of commercial application for AI and its business impact.-Politics:Regulation of AI,its economic implications and the evolving geopolitics of AI.-Safety:Identifying and mitigating catastrophic risks that highly-capable future AI systems could pose to us.-Predictions:What we believe will happen in the next 12 months and a 2021 performance review to keep us honest.Produced by Nathan Benaich(nathanbenaich),Ian Hogarth(soundboy),Othmane Sebbouh(osebbouh)and Nitarshan Rajkumar(nitarshan).stateof.ai 2022#stateofai|3 Introduction|Research|Industry|Politics|Safety|PredictionsThank you!Othmane SebbouhResearch AssistantOthmane is a PhD student in ML at ENS Paris,CREST-ENSAE and CNRS.He holds an MsC in management from ESSEC Business School and a Master in Applied Mathematics from ENSAE and Ecole Polytechnique.#stateofai|4Nitarshan RajkumarResearch AssistantNitarshan is a PhD student in AI at the University of Cambridge.He was a research student at Mila and a software engineer at Airbnb.He holds a BSc from University of Waterloo.Introduction|Research|Industry|Politics|Safety|Predictionsstateof.ai 2022Definitionsstateof.ai 2022#stateofai|5Artificial intelligence(AI):a broad discipline with the goal of creating intelligent machines,as opposed to the natural intelligence that is demonstrated by humans and animals.Artificial general intelligence(AGI):a term used to describe future machines that could match and then exceed the full range of human cognitive ability across all economically valuable tasks.AI Safety:a field that studies and attempts to mitigate the catastrophic risks which future AI could pose to humanity.Machine learning(ML):a subset of AI that often uses statistical techniques to give machines the ability to learn from data without being explicitly given the instructions for how to do so.This process is known as“training”a“model”using a learning“algorithm”that progressively improves model performance on a specific task.Reinforcement learning(RL):an area of ML in which software agents learn goal-oriented behavior by trial and error in an environment that provides rewards or penalties in response to their actions(called a“policy”)towards achieving that goal.Deep learning(DL):an area of ML that attempts to mimic the activity in layers of neurons in the brain to learn how to recognise complex patterns in data.The“deep”refers to the large number of layers of neurons in contemporary models that help to learn rich representations of data to achieve better performance gains.Introduction|Research|Industry|Politics|Safety|PredictionsDefinitionsstateof.ai 2022#stateofai|6Model:once a ML algorithm has been trained on data,the output of the process is known as the model.This can then be used to make predictions.Self-supervised learning(SSL):a form of unsupervised learning,where manually labeled data is not needed.Raw data is instead modified in an automated way to create artificial labels to learn from.An example of SSL is learning to complete text by masking random words in a sentence and trying to predict the missing ones.(Large)Language model(LM,LLM):a model trained on textual data.The most common use case of a LM is text generation.The term“LLM”is used to designate multi-billion parameter LMs,but this is a moving definition.Computer vision(CV):enabling machines to analyse,understand and manipulate images and video.Transformer:a model architecture at the core of most state of the art(SOTA)ML research.It is composed of multiple“attention”layers which learn which parts of the input data are the most important for a given task.Transformers started in language modeling,then expanded into computer vision,audio,and other modalities.Introduction|Research|Industry|Politics|Safety|PredictionsResearch-Diffusion models took the computer vision world by storm with impressive text-to-image generation capabilities.-AI attacks more science problems,ranging from plastic recycling,nuclear fusion reactor control,and natural product discovery.-Scaling laws refocus on data:perhaps model scale is not all that you need.Progress towards a single model to rule them all.-Community-driven open sourcing of large models happens at breakneck speed,empowering collectives to compete with large labs.-Inspired by neuroscience,AI research are starting to look like cognitive science in its approaches.Industry-Have upstart AI semiconductor startups made a dent vs.NVIDIA?Usage statistics in AI research shows NVIDIA ahead by 20-100 x.-Big tech companies expand their AI clouds and form large partnerships with A(G)I startups.-Hiring freezes and the disbanding of AI labs precipitates the formation of many startups from giants including DeepMind and OpenAI.-Major AI drug discovery companies have 18 clinical assets and the first CE mark is awarded for autonomous medical imaging diagnostics.-The latest in AI for code research is quickly translated by big tech and startups into commercial developer tools.Politics-The chasm between academia and industry in large scale AI work is potentially beyond repair:almost 0%of work is done in academia.-Academia is passing the baton to decentralized research collectives funded by non-traditional sources.-The Great Reshoring of American semiconductor capabilities is kicked off in earnest,but geopolitical tensions are sky high.-AI continues to be infused into a greater number of defense product categories and defense AI startups receive even more funding.Safety-AI Safety research is seeing increased awareness,talent,and funding,but is still far behind that of capabilities research.Executive Summarystateof.ai 2022#stateofai|7 Introduction|Research|Industry|Politics|Safety|PredictionsScorecard:Reviewing our predictions from 2021stateof.ai 2022#stateofai|8 Introduction|Research|Industry|Politics|Safety|PredictionsOur 2021 PredictionGradeEvidenceTransformers replace RNNs to learn world models with which RL agents surpass human performance in large and rich games.YesDeepMinds Gato model makes progress in this direction in which a transformer predicts the next state and action,but it is not trained with RL.University of Genevas GPT-like transformer model IRIS solves tasks in Atari environments.ASMLs market cap reaches$500B.NoCurrent market cap is circa$165B(3 Oct 2022)Anthropic publishes on the level of GPT,Dota,AlphaGo to establish itself as a third pole of AGI research.NoNot yet.A wave of consolidation in AI semiconductors with at least one of Graphcore,Cerebras,SambaNova,Groq,or Mythic being acquired by a large technology company or major semiconductor incumbent.NoNo new announced AI semiconductor consolidation has happened yet.Small transformers+CNN hybrid models match current SOTA on ImageNet top-1 accuracy(CoAtNet-7,90.88%,2.44B params)with 10 x fewer parameters.YesMaxViT from Google with 475M parameters almost matched(89.53%)CoAtNet-7s performance(90.88%)on ImageNet top-1 accuracy.DeepMind shows a major breakthrough in the physical sciences.YesThree(!)DeepMind papers in mathematics and material science.The JAX framework grows from 1%to 5%of monthly repos created as measured by Papers With Code.NoJAX usage still accounts for$100M in the next 12 months.YesHelsing(Germany)raised$100M Series A in 2022.2020NVIDIA does not end up completing its acquisition of Arm.YesDeal is formally cancelled in 2022.Introduction|Research|Industry|Politics|Safety|Predictionsstateof.ai 2022Section 1:Researchstateof.ai 2022#stateofai|11 Introduction|Research|Industry|Politics|Safety|Predictions In 2021,we predicted:“DeepMind releases a major research breakthrough in the physical sciences.”The company has since made significant advancements in both mathematics and materials science.stateof.ai 2022 One of the decisive moments in mathematics is formulating a conjecture,or a hypothesis,on the relationship between variables of interest.This is often done by observing a large number of instances of the values of these variables,and potentially using data-driven conjecture generation methods.But these are limited to low-dimensional,linear,and generally simple mathematical objects.#stateofai|122021 Prediction:DeepMinds breakthroughs in the physical sciences(1/3)In a Nature article,DeepMind researchers proposed an iterative workflow involving mathematicians and a supervised ML model(typically a NN).Mathematicians hypothesize a function relating two variables(input X(z)and output Y(z).A computer generates a large number of instances of the variables and a NN is fit to the data.Gradient saliency methods are used to determine the most relevant inputs in X(z).Mathematicians can turn refine their hypothesis and/or generate more data until the conjecture holds on a large amount of data.Introduction|Research|Industry|Politics|Safety|Predictions In 2021,we predicted:“DeepMind releases a major research breakthrough in the physical sciences.”The company has since made significant advancements in both mathematics and materials science.stateof.ai 2022 DeepMind researchers used their framework in a collaboration with mathematics professors from the University of Sydney and the University of Oxford to(i)propose an algorithm that could solve a 40 years-long standing conjecture in representation theory and(ii)prove a new theorem in the study of knots.#stateofai|13 DeepMind made an important contribution in materials science as well.It showed that the exact functional in Density Functional Theory,an essential tool to compute electronic energies,can be efficiently approximated using a neural network.Notably,instead of constraining the neural network to verify mathematical constraints of the DFT functional,researchers simply incorporate them into the training data to which they fit the NN.2021 Prediction:DeepMinds breakthroughs in the physical sciences(2/3)Introduction|Research|Industry|Politics|Safety|Predictions In 2021,we predicted:“DeepMind releases a major research breakthrough in the physical sciences.”The company has since made significant advancements in both mathematics and materials science.stateof.ai 2022 DeepMind repurposed AlphaZero(their RL model trained to beat the best human players of Chess,Go and Shogi)to do matrix multiplication.This AlphaTensor model was able to find new deterministic algorithms to multiply two matrices.To use AlphaZero,the researchers recast the matrix multiplication problem as a single-player game where each move corresponds to an algorithm instruction and the goal is to zero-out a tensor measuring how far from correct the predicted algorithm is.Finding faster matrix multiplication algorithms,a seemingly simple and well-studied problem,has been stale for decades.DeepMinds approach not only helps speed up research in the field,but also boosts matrix multiplication based technology,that is AI,imaging,and essentially everything happening on our phones.#stateofai|142021 Prediction:DeepMinds breakthroughs in the physical sciences(3/3)Introduction|Research|Industry|Politics|Safety|Predictions A popular route to achieving nuclear fusion requires confining extremely hot plasma for enough time using a tokamak.A major obstacle is that the plasma is unstable,loses heat and degrades materials when it touches the tokamaks walls.Stabilizing it requires tuning the magnetic coils thousands of times per second.DeepMinds deep RL system did just that:first in a simulated environment and then when deployed in the TCV in Lausanne.The system was also able to shape the plasma in new ways,including making it compatible with ITERs design.DeepMind trained a reinforcement learning system to adjust the magnetic coils of Lausannes TCV(Variable Configuration tokamak).The systems flexibility means it could also be used in ITER,the promising next generation tokamak under construction in France.stateof.ai 2022#stateofai|15Reinforcement learning could be a core component of the next fusion breakthrough Introduction|Research|Industry|Politics|Safety|Predictionsstateof.ai 2022Predicting the structure of the entire known proteome:what could this unlock next?Since its open sourcing,DeepMinds AlphaFold 2 has been used in hundreds of research papers.The company has now deployed the system to predict the 3D structure of 200 million known proteins from plants,bacteria,animals and other organisms.The extent of the downstream breakthroughs enabled by this technology-ranging from drug discovery to basic science-will need a few years to materialize.There are 190k empirically determined 3D structures in the Protein Data Bank today.These have been derived through X-Ray crystallography and cryogenic electron microscopy.The first release of AlphaFold DB in July 20221 included 1M predicted protein structures.This new release 200 xs the database size.Over 500,000 researchers from 190 countries have made use of the database.AlphaFold mentions in AI research literature is growing massively and is predicted to triple year on year(right chart).#stateofai|16 Introduction|Research|Industry|Politics|Safety|Predictions This is because ESMFold doesnt rely on the use of multiple sequence alignments(MSA)and templates like AlphaFold 2 and RoseTTAFold,and instead only uses protein sequences.stateof.ai 2022Researchers independently applied language models to the problems of protein generation and structure pred