Foundation models

Foundation models are a transformative approach to artificial intelligence. Rather than being a single technology, they encompass a comprehensive process for designing and building AI models. These pretrained models are versatile foundations, enabling organizations to develop a wide range of AI and machine learning applications more efficiently and effectively.

By using foundation models, organizations can accelerate their AI initiatives and innovate across various domains. Discover how foundation models work or connect with our sales support team to discuss how we can help you.

What are foundation models?

Foundation models definition: Foundation models are a pivotal aspect of artificial intelligence technology, enabling the rapid development of new AI models by using preexisting knowledge and architectures.

Instead of dedicating extensive time and resources to create and train AI models from scratch, organizations can adapt pretrained foundation models at a significantly lower cost.

Foundation models are based on deep learning principles and are trained on vast datasets, resulting in a model that can be applied to a wide range of data types. By using foundation model applications, organizations can facilitate innovation and efficiency across different domains.

How do foundation models work?

Foundation models can take various forms, and they serve as an umbrella term for models that can be adapted and applied to more specific uses. While their construction may vary to meet different needs and applications, they share certain characteristics, such as being primarily based on neural networks, which mimic human thinking through interconnected nodes.

Neural networks are trained on datasets and deep machine learning technologies, enabling them to handle varied datasets to quantify, categorize and produce predictive outcomes.

The neural network of a foundation model is trained to identify patterns and relationships within a dataset. Instead of being equipped to identify only certain attributes of the data, a foundation model can look for patterns that may be replicated across a wide variety of data types.

What is the history of foundation models?

Foundation models have a relatively recent history as a technology. Their development arose out of the growing interest in and research surrounding artificial intelligence. As AI became more common, there was a greater need to make designing and building AI models more efficient.

The first notable appearances of foundation models were in the late 2010s, when deep learning became sophisticated enough to facilitate the development of foundational AI models.

With enough learnings and iterations of AI models to draw from, data engineers were able to cherry-pick the most successful elements of individual models to create a solid foundation that could be applied more generically.

What are key types of foundation models?

Foundation models have a wide range of characteristics, enabling them to power AI intended for a range of applications. Some are more common than others and can be broken down into these categories.

• Large language models can be applied across a large set of use cases as they are trained on linguistic content to identify specific elements of language and grammar. With this training, they can form foundation models for applications like translation and text generation. One example of a large language model adapted from a foundation model is Google's BERT.
• Vision models operate in similar ways to language models but apply the approach to visual media. Vision models are computer algorithms for processing visual data. The basic training for these models can be applied widely to image classification and object detection models. One example of a vision model based on a foundation model is DALL-E.
• Multimodal models are AI models trained on multiple datasets, offering versatile adaptability. Foundation models can take this form. The key benefit is that multimodal foundation models are already trained and, therefore, equipped to handle multiple data types, allowing for simpler data input.

How are foundation models used?

Foundation models are a widely applicable technology across all industries, supplying the basis for extremely adaptable AI applications.

Foundation models based on language processing have adaptable and strong capabilities to process and generate language, as seen with the AI chatbot ChatGPT from OpenAI. They are equipped with the training to translate text between languages, summarize the meaning of language, generate new textual content based on prompts and more. Generative AI, a subset of these models, is particularly notable for its ability to create new content, driving innovation in various fields.

Foundation models generate textual content and provide the basis for visual artificial intelligence. This powers computer vision, allowing for object recognition and image classification, which has expedited and streamlined processes across industries such as security and medicine.