What technology segments underpin generative AI?
Technology segments in generative AI include transformer-based models, diffusion models, variational autoencoders, autoregressive models, generative adversarial networks, and energy-based models.
The most well-known of the different models, it was introduced in 2017 by ex-Googler Ashish Vaswami in the paper "Attention Is All You Need". The transformer model is designed to process sequential data, such as natural language text, and can achieve state-of-the-art performance on a variety of natural language processing tasks. Transformer-based models can be deployed for language translation, text classification, question answering, text generation, and image and video processing.
Diffusion models in AI are a class of generative models used to model the spread of information or influence through a network. These models are based on the diffusion process that describes how information or influence spreads through a network over time.
Diffusion models are typically used in social network analysis, to model the spread of ideas, opinions, or behaviours through a network. They can also have applications in epidemiology to model the spread of diseases through a population.
Variational autoencoders (VAEs) are a type of generative model in machine learning used for unsupervised learning of complex data distributions.
VAEs are a versatile and powerful tool in machine learning, with applications in generative modelling, data compression, anomaly detection, and data augmentation.
Autoregressive models are a class of statistical models that are used to model time series data. These models are based on the idea that the value of a variable at a given time point is dependent on its previous values.
Autoregressive models are a powerful tool in time series analysis and forecasting, with applications in a wide range of domains, including finance, climate modelling, and signal processing.
Generative adversarial networks
Generative adversarial networks (GANs) are a type of deep learning model that are used for generative modelling. GANs consist of two neural networks: a generator network and a discriminator network. GANs have a wide range of applications, including image and video generation, natural language processing, and data augmentation
Energy-based AI models are a class of machine learning models that are based on the concept of energy function/s. These models are used for a variety of tasks, including classification, regression, and generative modelling.
Energy-based models have been used in a variety of applications, including image and video analysis, natural language processing, and anomaly detection. They are particularly useful for tasks where the data is complex and difficult to model using traditional machine learning models.
Foundation models are a powerful class of generative model, typically based upon a transformer, which serve as a starting point for a wide variety of applications.
These models are trained using unsupervised learning techniques, where they learn from vast amounts of text data without explicit human annotations. This training forms the basis for a “reusable” underlying model – a foundation – with considerable flexibility in how it could be deployed.
The models can then be fine-tuned with further data for specific downstream tasks, ranging from text classification and language translation, to sophisticated predictive and analytical tasks.