Deep AI: Emergent Abilities Show Up

 Deep AI: Emergent Abilities Show Up

After pretraining on massive datasets scraped from the internet, the latest language models show unexpected capabilities, such as solving simple arithmetic problems. 

We're still in the early phases of building algorithms that generalize to different tasks. While producing excellent results, vision transformer models are pretty specialized for this particular task. On the other hand, the large text models are probably algorithms that will be the easiest to generalize to other tasks. 

One of the most positive developments in 2022 and before is a development of #languagebenchmarks that evaluate complex models e.g., #BIGbench, #SuperGLUE, #GSM8K, etc.

Machine Learning will inevitably evolve into areas where complex forms of reasoning will emerge. Currently, the algorithms work as "compression" systems for a large amount of information. We can produce fantastic chat results or generate visual art with a creative retrieval process. #AI #deeplearning #machinelearning

https://arxiv.org/abs/2206.07682