Meta Unveils Llama 4 AI Series Featuring New Expert-Based Architecture

Date:

Share:


Image: Meta

Meta unveiled on April 5 its new AI model series: Llama 4, which includes Llama 4 Maverick and Llama 4 Scout, tailored for conversation and processing large files, respectively, along with an unreleased “teacher” model called Llama 4 Behemoth.

Llama 4 is Meta’s first series to adopt a “mixture of experts (MoE) architecture.” This approach activates only select parts of the neural network, referred to as the “experts,” to handle specific subtasks. The task will be broken down into subtasks and each routed to the most appropriate experts, improving resource efficiency.

What are the specifics about Llama 4 Maverick and Scout?

Llama 4 Maverick features 128 experts and 17 billion active parameters, which represent the portion of a model’s knowledge used to process a given input. Meta describes it as the “product workhorse model for general assistant and chat use cases,” specialising in image interpretation and creative writing.

Interestingly, Mark Zuckerberg’s company boasts that Maverick offers “a best-in-class performance to cost ratio” when it comes to conversations. Cost has been playing on the minds of AI giants since the surprise release of DeepSeek in January, which took only $5.6 million to train.

SEE: Meta’s $800M Offer To Chip Startup Was Rejected — Here’s Why

However, AI experts have noticed that the version of Llama 4 Maverick published on LMArena, which ranks major large language models across various tasks, is “optimized for conversationality” and performs differently from the publicly available version. This suggests that Meta submitted an altered version to LMArena that would rank higher on its leaderboard.

Llama 4 Scout also has 17 billion active parameters and just 16 experts, but Meta says it is the “best multimodal model in the world in its class.” It has an unusually large context window of 10 million tokens, which represent the amount of information it can process in a prompt, so it performs well when summarising large documents and in sequential reasoning.

Meta says that both Scout and Maverick are its “best yet” due to being distilled from Llama 4 Behemoth, with a whopping 28 billion active parameters and 16 experts. While it already ranks highly on LMArena, it is still being trained and has not been released. According to The Information, the Llama 4 announcement was delayed at least twice due to the models underperforming in technical benchmarks and conversationality.

How can you access LLama 4 Maverick and Scout?

Scout and Maverick can be downloaded on Llama.com and Hugging Face, or used through the Meta AI chatbots in WhatsApp, Messenger, and Instagram in 40 countries. Multimodal features can only be used in the U.S. and in English, currently.

Some partners have already announced integrations; developers can build and deploy AI applications with the Llama 4 models in Microsoft’s Azure AI Foundry and Azure Databricks.

Llama 4 is apolitical

Meta stated it has worked specifically to “remove bias” from the Llama 4 models. The refusal rate for questions on “debated political and social topics” is over 5% lower than that of Llama 3.3 and, among the questions it does decline, its responses are described as “dramatically more balanced.”

U.S. President Donald Trump’s team has voiced skepticism about the neutrality of AI models, with his AI and crypto czar David Sacks suggesting that OpenAI’s ChatGPT is “programmed to be woke” on a podcast. AI experts say that bias ultimately stems from training data and can lead to political leanings in any direction, not just the left.

Nevertheless, Zuckerberg’s firm has made a number of recent moves that suggest it wants to stay on the side with the U.S. administration. Republican strategist Joel Kaplan was hired as Meta’s policy lead shortly after Trump assumed office; he sees social media regulation as a direct challenge to free speech.

In January, Meta revealed the company was discontinuing its third-party fact-checking program and relocating its content moderation teams from California to Texas to “help remove the concern that biased employees are overly censoring content.” Meta has also eliminated its diversity, equity, and inclusion initiatives after Trump criticised such programs.

Furthermore, Meta said the Llama 4 models respond with a “strong political lean” on “contentious” topics at a similar rate to Grok, the chatbot produced by xAI, a company owned by current White House adviser Elon Musk.

Llama 4 cannot be used in the E.U.

According to the Llama 4 acceptable use policy, individuals “domiciled” or companies with a “principal place of business” in the European Union cannot use or distribute the models. Those individuals or companies can, however, use the Llama 4 models if they are incorporated into a product or service they have access to in the region.

This is likely the result of Meta’s issues with E.U. legislation, particularly when it comes to AI. In June 2024, Meta delayed the training of its large language models on public content shared on Facebook and Instagram after E.U., regulators suggested it might need explicit consent from content owners. Meta AI has still not been released within the bloc.

SEE: Meta Offers Less Personalised Ads for EU Users

Meta signed an open letter urging European regulators to address “inconsistent regulatory decision-making” and unpredictable compliance demands last September. Then, in February, Meta declared it was prepared to escalate its concerns over what the company sees as unfair E.U. regulations directly to Trump.

There are other restrictions when it comes to Llama 4 usage, as commercial entities with more than 700 million monthly active users must request permission from Meta before using its models. The Open Source Initiative has said that such a restriction takes the AI “out of the category of “open source,” despite Meta claiming otherwise.



Source link

━ more like this

Hollywood’s biggest filmmaker just came out clean about using AI in movies

Legendary filmmaker Steven Spielberg voiced concerns about the growing role of artificial intelligence in creative industries during an appearance at SXSW in Austin....

Windows 11 is readying support for 1,000+ Hz monitors, assuming you got one

Microsoft is quietly preparing Windows 11 for a new class of ultra-fast gaming monitors, even if most gamers won’t see them anytime soon....

ByteDance has reportedly suspended the global rollout of its new AI video generator

A month after Seedance 2.0's launch in China sparked cease-and-desist letters from Disney and Paramount Skydance over its use of copyrighted materials, its...

You might want to double-check before buying laptops from this Chinese brand

Independent testing has recently uncovered what it describes as a potential CPU mislabeling issue affecting multiple Chuwi laptops. In its latest report, Notebookcheck...

There’s a new global factor for a potentially serious price hike for PCs and mobile

A fresh supply chain shock may be brewing for the tech industry, and it could eventually affect the price of PCs, smartphones, and...
spot_img