Next month's "AI Summit": In addition to B100, NVIDIA has also gathered the authors of the "Most Important AI Papers" for a conversation with the founder.

On March 18-21, NVIDIA will hold the GTC 2024 conference, mainly targeting AI developers. The conference will feature two major highlights: NVIDIA will unveil the brand-new B100 chip and invite the eight authors of the groundbreaking paper "Attention Is All You Need" to the meeting for a dialogue with Jensen Huang.

Published in 2017, "Attention Is All You Need" is arguably the most important paper in the field of AI to date, cited over 80,000 times. It introduced the Transformer language model based on the attention mechanism, replacing recurrent and convolutional neural networks.

Transformer Leading AI Development

The Transformer model has played a crucial role in the development of AI, with almost all AI products, including ChatGPT, relying on Transformer.

Why is Transformer so vital for AI development? Because its true power extends beyond language; it can generate any content with repetitive patterns or structures, including images generated by tools like Dall-E, Midjourney, and Stable Diffusion, as well as computer code generated by tools like GitHub CoPilot, and even DNA sequences.

One of the paper's authors, Ashish Vaswani, explained how the Transformer works, "The Transformer can quickly capture the interactions between different parts of the input. Once captured, it can learn some features from them." Vaswani added,

"This is a universal approach that can capture fragments in sentences, notes in music, pixels in images, components of proteins. It can be used for any task."

After the paper was published, Parmar discovered that the Transformer could not only be used for translation but also for generating long-form text, a feat previous models struggled with. Furthermore, she recognized a key feature of the Transformer: "the more data you give them, the better they learn." Parmar added, "Whatever people throw at it, it works well." This undoubtedly paves the way for the emergence of large models like GPT-4. Nowadays, Transformer is powering most cutting-edge applications in AI development. It is not only embedded in Alphabet-C search and Alphabet-C translation, but also supporting all major language models, including the models behind ChatGPT and Bard.

Jill Chase, a partner at Alphabet's growth fund CapitalG, focuses on investing in artificial intelligence ventures, stating that "all products rely on Transformer to exist."

Eight "defectors" from Alphabet-C, embarking on AI entrepreneurship

The research team behind Transformer comes from Alphabet-C, including eight individuals: Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin.

As the pace of AI research accelerates, Transformer is widely used in text and image generation fields. However, most of this research comes from startups like OpenAI, rather than Alphabet-C. They found that Alphabet-C's structure does not allow for entrepreneurial risk-taking or rapid product launches.

Subsequently, eight researchers left the team. Polosukhin left in 2017 and founded the blockchain startup Near. Gomez, the youngest member of the team at the time, left Alphabet-C in 2019 and founded Cohere, dedicated to using large language models to help banks and retailers solve customer service issues. Cohere is currently valued at over $2 billion.

Kaiser, Uszkoreit, Shazeer, Vaswani, and Parmar left Alphabet-C in 2021. Kaiser became a researcher at OpenAI. Uszkoreit co-founded Inceptive with others, a company using deep learning technology to design "biological software" to help researchers find new drugs and biotechnologies. Shazeer co-founded Character AI with others, focusing on creating personalized chatbots. Character AI is currently valued at over $1 billion. Vaswani and Parmar founded Adept AI and Essential AI respectively, aiming to build software for enterprises using large language models. Essential AI has raised $8 million to date. Jones didn't leave Alphabet-C until 2023, when he co-founded Sakana AI with others.