Meta's Llama 3.1 leaks reveal significant upgrades to 8B and 70B models, plus a new 405B SOTA OSS model.

By PulseAugur Editorial · Summary by gemini-2.5-flash-lite from 1 source

Meta AI's upcoming Llama 3.1 models are reportedly set to feature significant performance improvements, particularly in the 8B parameter version. The 70B parameter model is also expected to see enhancements, though to a lesser extent. Additionally, a new 405B parameter open-source model is anticipated to achieve state-of-the-art performance. AI

Summary written by gemini-2.5-flash-lite from 1 source. How we write summaries →

RANK_REASON Leak of upcoming frontier-tier open-source models from a major AI lab.

Read on Smol AINews →

COVERAGE [1]

Smol AINews TIER_1 · 2024-07-23 01:12

Llama 3.1 Leaks: big bumps to 8B, minor bumps to 70b, and SOTA OSS 405b model

**Llama 3.1** leaks reveal a **405B dense model** with **128k context length**, trained on **39.3M GPU hours** using H100-80GB GPUs, and fine-tuned with **over 25M synthetic examples**. The model shows significant benchmark improvements, especially for the 8B and 70B variants, wi…

COVERAGE [1]

Llama 3.1 Leaks: big bumps to 8B, minor bumps to 70b, and SOTA OSS 405b model

RELATED ENTITIES

RELATED TOPICS