Beyond Transformers...Zyphra Unveils Zamba: A Compact 7B SSM Hybrid Model
Welcome to Nural's newsletter focusing on how AI is being used to tackle global grand challenges.
Packed inside we have
- Meta release Llama 3 - most powerful open source model to date
- AI startup Mistral to raise 500m EUR at 5bn EUR valuation after less than a year
- Zyphra Unveils Zamba: A Compact 7B SSM Hybrid Model... Beyond Transformers
- and Stability AI lays off 10% of staff
If you would like to support our continued work from £2/month then click here!
Marcel Hedman
Key Recent Developments
Introducing Meta Llama 3: The most capable openly available LLM to date
What: Meta released Llama 3: "the next generation of our state-of-the-art open source large language model". They have made sure to provide access across the most popular development platforms including AWS, Databricks, Google Cloud, Hugging Face, Kaggle, IBM WatsonX, Microsoft Azure, NVIDIA NIM, and Snowflake
Currently, the 8B and 70B parameter models have been made available with an anticipated 400B parameter model to come which should rival OpenAI's GPT4 performance.
Key Takeaway: In the battle of opensource vs closed source AI development, Meta have been the leading champion of state of the art open source release.
Open source offers extreme promise as it enables researchers, companies and anyone seeking to engage with LLMs to develop freely on top of highly expensive models. However, releasing such a powerful technology openly increases the risk for misuse. Will the open release approach be maintained as the 400B parameter model becomes available?
VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Time
TL;DR: single portrait photo + speech audio = hyper-realistic talking face video with precise lip-audio sync, lifelike facial behaviour, and naturalistic head movements, generated in real time.
Follow the link to check out videos generated using this methodology...
AI Ethics & 4 Good
🚀 [DeepMind] The ethics of advanced AI assistants
🚀 Using unlabeled data to enhance fairness of medical AI
🚀 Generative models improve fairness of medical classifiers under distribution shifts
🚀 Transparent medical image AI via an image–text foundation model grounded in medical literature
🚀Meta’s Oversight Board probes explicit AI-generated images posted on Instagram and Facebook
Other interesting reads
🚀 Zyphra Unveils Zamba: A Compact 7B SSM Hybrid Model
🚀 AI start-up Mistral in talks to raise €500mn at €5bn valuation
🚀 What Every CEO Needs To Know About The New AI Act
🚀 Stability AI Lays Off 10% Of Staff — Report
🚀 SAMMO: A general-purpose framework for prompt optimization
🚀 Grok-1.5 Vision Preview - Elon Musk's X release model
Papers
🚀 A Survey on Retrieval-Augmented Text Generation for Large Language Models
🚀 MEGALODON: Efficient LLM Pretraining and Inference with Unlimited Context Length
Cool companies found this week
AGI
Zyphra - Zyphra is a full stack AGI company building next-gen models, infrastructure and silicon inspired by principles from neuroscience and physics.
Reka - Building multimodal language models and recently released Reka core, a frontier-class multimodal language model on par with leading models in the industry today.
Best,
Marcel Hedman
Nural Research Founder
www.nural.cc
If this has been interesting, share it with a friend who will find it equally valuable. If you are not already a subscriber, then subscribe here.
If you are enjoying this content and would like to support the work financially then you can amend your plan here from £2/month!