Language models have become increasingly sophisticated, with companies and researchers striving to create the most efficient and capable AI for natural language processing tasks.
Language models have become increasingly sophisticated, with companies and researchers striving to create the most efficient and capable AI for natural language processing tasks. Among these, Falcon, a new open-source language model, has emerged as a significant player. Its introduction has caught the attention of those invested in the development and application of AI language models, particularly because it offers a middle ground between the capabilities of GPT-3.5 and GPT-4 and yet is accessible to the public for further development.
Falcon, an open-source language model with 180 billion parameters, has recently made headlines as it takes the lead in the open-source language model space. This model, referred to as Falcon 180B, signifies its parameter count, outperforms the largest LLaMA model, which has 70 billion parameters, according to benchmarks on the Hugging Face Openllm leaderboard. Notably, Falcon has achieved the first rank on this leaderboard, which is a momentous achievement for an open-source model.
The emergence of Falcon 180B is significant as it approaches the performance levels of some of the most advanced proprietary models, such as Google's Palm two and OpenAI's GPT-4, albeit with a larger parameter count. Falcon 180B is observed to surpass GPT-3.5 and nearly match Palm two in performance, which is an impressive feat for an open-source offering.
Open-source models like Falcon 180B come with certain advantages, such as the ability to be fine-tuned by developers for specific use cases. However, running and fine-tuning a model of this size can be prohibitively expensive for casual users, although it is technically feasible.
When it comes to the licensing of Falcon 180B, it is not entirely unrestricted. While the model is touted as open-source, it comes with a stipulation that it cannot be used for "hosting use" without permission. This means that if one intends to offer the model as a service to others, such as via an API, they would need to obtain explicit permission, which currently seems to be a challenging task due to contact issues with the provided license email.
Despite the licensing caveat, Falcon 180B still stands as a significant contribution to the open-source community. It offers the most capable open-source model available to date. Developers and researchers interested in experimenting with Falcon 180B can access a demo on Hugging Face, though the demo limits the output to 256 tokens to conserve compute resources.
In practical tests, Falcon 180B demonstrated its prowess by generating social media content that diverged from the typical Chat GPT style, thereby providing unique and relevant outputs. For example, when tasked with crafting an Instagram caption for a tech-savvy audience, Falcon 180B produced captions that were more on-topic and adhered to the prompt's instructions better than GPT-4's attempts, which were criticized as generic.
It's worth noting that Falcon 180B has not undergone advanced tuning for alignment, which means it might produce less politically correct or more "open" responses compared to other models. This characteristic could be advantageous or problematic, depending on the use case and the need for content sensitivity.
Finally, Falcon 180B is particularly adept at translations involving European languages. This is attributed to the inclusion of the Refined Web Europe dataset in its training data, which is not as heavily represented in other models. As such, Falcon 180B could serve as a valuable tool for translation tasks involving European languages.
