The Technology Innovation Institute (TII) has introduced the Falcon Mamba 7B, a new addition to its Falcon series of large language models. The Falcon Mamba 7B is now the top-performing open-source State Space Language Model (SSLM) globally, as independently confirmed by Hugging Face.
This latest model marks a departure from previous Falcon models, which relied on transformer-based architecture, making the Falcon Mamba 7B the first SSLM in the Falcon series. This model showcases TII’s pioneering research and its commitment to providing groundbreaking tools and products to the community in an open-source format.
Abu Dhabi: A Global AI Hub
Faisal Al Bannai, Secretary General of ATRC and Adviser to the UAE President for Strategic Research and Advanced Technology Affairs, emphasized the significance of this achievement: “The Falcon Mamba 7B marks TII’s fourth consecutive top-ranked AI model, reinforcing Abu Dhabi as a global hub for AI research and development. This achievement highlights the UAE’s unwavering commitment to innovation.”
In terms of performance, the Falcon Mamba 7B outshines Meta’s Llama 3.1 8B, Llama 3 8B, and Mistral’s 7B on Hugging Face’s new benchmarks for transformer architecture models. Additionally, Falcon Mamba 7B leads the competition among other SSLMs on both the old and newly introduced tougher benchmark leaderboards.
Pioneering AI Innovation
Dr. Najwa Aaraj, Chief Executive of TII, highlighted the institute’s ongoing commitment to technological advancement: “The Technology Innovation Institute continues to push the boundaries of technology with its Falcon series of AI models. The Falcon Mamba 7B represents true pioneering work and paves the way for future AI innovations that will enhance human capabilities and improve lives.”
State Space models excel at understanding complex situations that evolve over time, making them ideal for tasks like processing large volumes of information, such as entire books, without requiring additional memory. In contrast, transformer-based models are highly effective at tasks like content generation but demand significant computational power.
SSLMs have a wide range of applications, including estimation, forecasting, and control tasks. They also perform exceptionally well in Natural Language Processing tasks, such as machine translation, text summarization, computer vision, and audio processing.
Dr. Hakim Hacid, Acting Chief Researcher of TII’s AI Cross-Centre Unit, praised the collaborative efforts within TII that led to the development of Falcon Mamba 7B: “This release represents a significant stride forward, inspiring fresh perspectives and further fueling the quest for intelligent systems. At TII, we’re pushing the boundaries of both SSLM and transformer models to spark further innovation in generative AI.”
Falcon LLMs have been downloaded over 45 million times, underscoring their success. The Falcon Mamba 7B will be released under the TII Falcon License 2.0, a permissive Apache 2.0-based software license that includes an acceptable use policy to promote the responsible use of AI.
