14 Companies Pioneering AI Foundation Models in Pharma and Biotech
Foundation models represent a new paradigm in artificial intelligence (AI), revolutionizing how machine learning models are developed and deployed. As these models grow increasingly capable, they become useful for applications across a wide range of economic functions and industries, including biotech. Foundation models are a class of large-scale machine learning models, typically based on deep learning architectures such as transformers, that are trained on massive datasets encompassing diverse types of data. The most prominent examples of general-purpose foundation models are the GPT-3 and GPT-4 models, which form the basis of ChatGPT, and BERT, or Bidirectional Encoder Representations from Transformers. These are gigantic models trained on enormous volumes of data, often in a self-supervised or unsupervised manner (without the need for labeled data).
Their scalability in terms of both model size and data volume enables them to capture intricate patterns and dependencies within the data. The pre-training phase of foundation models imparts them with a broad knowledge base, making them highly efficient in few-shot or zero-shot learning scenarios where minimal labeled data is available for specific tasks.
This approach demonstrates their high versatility and transfer learning capabilities, adapting to the nuances of particular challenges through additional training.
Below we summarized a number of companies building domain-specific foundation models for biology research and related areas, like chemistry.
Deep Genomics
In September 2023, Deep Genomics unveiled BigRNA, a pioneering AI foundation model for uncovering RNA biology and therapeutics. It is the first transformer neural network engineered specifically for transcriptomics. BigRNA is informed by nearly two billion adjustable parameters and trained on thousands of datasets, totaling over a trillion genomic signals.
This model is designed to predict tissue-specific regulatory mechanisms of RNA expression, binding sites of proteins and microRNAs, and the effects of genetic variants and therapeutic candidates. By understanding these complex RNA interactions, BigRNA facilitates the discovery of new biological mechanisms and RNA therapeutic candidates that traditional approaches might miss, exemplifying its transformative potential in RNA-based drug discovery.
Ginkgo Bioworks
In August 2023, Ginkgo Bioworks and Google Cloud announced a 5-year partnership aimed at developing state-of-the-art large language models (LLMs) focused on genomics, protein function, and synthetic biology. Ginkgo’s AI foundation model will run on Google Cloud's Vertex AI platform, aiming to accelerate innovation in drug discovery, agriculture, industrial manufacturing, and biosecurity.
Furthermore, in February 2024 Ginkgo committed to building next-generation biological foundation models by acquiring key assets of Reverie Labs, a startup specializing in AI/ML tools for drug discovery. This acquisition includes Reverie's infrastructure and software for training large-scale AI models, enhancing Ginkgo's capabilities in developing comprehensive biological models.
Bioptimus
In February 2024, Bioptimus, a biotech startup based in France, announced the successful closure of a $35 million seed funding round to develop an AI foundation model targeting advancements across the biological spectrum, from molecular to organismal levels.
Led by Professor Jean-Philippe Vert, the company collaborates with Owkin to leverage extensive data generation capabilities and multimodal patient data from leading academic hospitals worldwide. Owkin's initiative, MOSAIC, represents one of the largest multi-omics atlases for cancer research, showcasing the potential of combining computational and experimental research methods.
This collaboration, supported by Amazon Web Services (AWS), is crucial for developing AI models capable of capturing the diversity of biological data.
Continue reading
This content available exclusively for BPT Mebmers
We use cookies to personalise content and to analyse our traffic.
You consent to our cookies if you continue to use our website. Read more details in our
cookies policy.
Comments