The next big step in AI is Action Transformers

Muthukumaraswamy
Searce
Published in
4 min readJul 23, 2023

--

The future of AI

Imagine a world where the boundaries between our daily life and work, where astonishing possibilities emerge, and where the very essence of how we interact with our devices undergoes a paradigm shift. You might initially perceive this as mere sensationalism, but your perspective will undoubtedly shift.

Step into a realm beyond Bard & ChatGPT for there exists an AI application with potential far greater than we can think of. This technology holds the power to revolutionize how we engage with our phones, tablets, and computers, breaking free from the constraints of our imagination.

This groundbreaking technology is quietly making its way into our lives. However, let me be clear — how you react to its possibilities lies solely in your hands.

Prepare yourself for the revelation of Action Transformers — a force so disruptive and transformative that it has the capacity to shape the very fabric of your future.

Widespread AI

In the realm of technology, AI has permeated every aspect of our lives, serving as an invaluable predictor and decision-maker for highly specialized tasks. Until now, AI models have excelled in catering to specific use cases with precision.

Surprisingly, the domain that has reaped the most significant economic benefits from AI’s prowess is Online Advertising. Behemoths like Google and Meta have built empires on the foundation of certainty, offering empirical, data-driven assurances of profitability for advertisers.

By leveraging AI’s capabilities, these tech giants ensured that marketing campaigns reached the precise customer personas, transforming the industry from a historical “hit-and-miss” approach to a seamless and targeted endeavor.

However, this remarkable success came at a price, as AI remained an exclusive technology accessible only to a privileged few due to its hefty investments.

Yet, the landscape is now shifting with the advent of Generative AI, marking a transformative moment when AI steps out of the shadows and becomes accessible to all in our day-to-day lives.

They all boil down to a fundamental concept: attention. In a world where maintaining focus becomes an ever-increasing challenge, it’s ironic that the force that is destined to revolutionize our reality hinges on something as simple as paying attention.

Embrace the forthcoming changes as AI embraces mainstream accessibility, and the world around us undergoes a metamorphosis, fueled by the power of attention.

Attention is all you need

In the realm of Artificial Intelligence, there emerged an extraordinary concept known as the “attention mechanism,”. Its significance in the field is unparalleled, making it a groundbreaking milestone.

Imagine a world where machines comprehend the context of phrases in a manner akin to how we humans do. The attention mechanism opened the door to such a possibility for the first time.

Let’s discuss an illustrative example to grasp its essence better. Suppose you’re tasked with translating a phrase into a different language. Ordinarily, you’d take it step by step, breaking the sentence into smaller chunks and holding the earlier parts in your memory to retain context as you progress.

Before the advent of the attention mechanism, neural networks relied on brute force to extract context from the entire sentence. As a consequence, as the length of the phrase to be understood increased, computational and memory demands soared astronomically.

But then came the game-changing attention mechanism.

Through the ingenious use of a weighting mechanism, we taught machines to “score” each word in a sentence concerning the other words around it, one step at a time. In simpler terms, for every word received, the machine could now comprehend which other words held greater or lesser importance in that specific context, just like how we, humans, do so intuitively.

This breakthrough ushered in a new era, giving rise to the transformer model, which underpins the likes of Bard,ChatGPT, DALL-E, Stable Diffusion, and the protagonist of today’s tale.

The attention mechanism revolutionized the way machines perceive language, opening doors to creativity and unprecedented possibilities. It marks a turning point in the history of AI, forever shaping the landscape of human-machine interaction.

The era of Transformers has arrived

In the dynamic landscape of AI, the attention mechanism proved to be a game-changer in 2017. A visionary group of researchers made a daring decision to break away from the conventional recurrence and convolution techniques that had long been the standard for training sequence models. Instead, they embarked on a journey to create something entirely fresh and powerful — a new encoder-decoder architecture, where two neural networks were intricately connected in a sequence, relying solely on the prowess of the attention mechanism.

Since then, these novel models have predominantly found their application in generating text or image outputs, with awe-inspiring success. Just take ChatGPT as an example, revolutionizing various industries and leaving its mark on creative workers, writers, marketers, and more.

Yet, hidden beneath the surface, lies a profound and awe-inspiring use case that remains largely undiscovered — a secret waiting to unveil, destined to reshape how we interact with our digital devices in ways beyond imagination. Brace yourself, for this revelation promises to transcend the boundaries of the known and open up a world of limitless possibilities. A transformative force lies in wait, ready to redefine the very essence of our digital experiences.

Are you prepared to embrace this extraordinary revolution?

--

--