MetaAI's newest generation of their Llama models, Llama 3.1, is now available.
What's new with Llama 3.1?
- Longer context window supporting up to 128k tokens.
- Now available in 3 different sizes, including a new 405B parameter flagship model, and upgraded 70B & 8B versions.
- Available across 8 different languages, including English, German, French, Italian, Portuguese, Hindi, Spanish, and Thai.
- Competitive with other leading, closed-source foundational models, including GPT-4, GPT-4o, and Claude 3.5 Sonnet.
Llama 3.1 comes in three sizes: 8B, 70B, and 405B.
| Name | Description |
|---|
8B | Relatively small, fast model, and supremely capable LLM you can run on your laptop. |
70B | Medium-large variant that enables diverse use cases that may require more complex reasoning. |
405B | The most extensively trained open LLM to date. Requires an extremely capable setup to run |
How to run Llama 3.1 locally in your LM Studio
- Install LM Studio 0.2.28 from https://lmstudio.ai
- Search for Meta-Llama-3.1-8B-Instruct-GGUF or use this direct download link.
- When the download is complete, go ahead and load the model.
- Use the model in the chat interface or via the LM Studio API.
You can use Llama 3.1 via LM Studio's built-in OpenAI-like server. See the docs for more details.
To use MetaAI's Llama 3.1 in LM Studio, download or update to LM Studio 0.2.28 or later.
Download LM Studio from the LM Studio website.