There are two key decisions to make when you want to run a Gemma model: 1) what Gemma variant you want to run, and 2) what AI execution framework you are going to use to run it? A key issue in making both these decisions has to do with what hardware you and your users have available to run the model.
This overview helps you navigate these decisions and start working with Gemma models. The general steps for running a Gemma model are as follows:
Choose a framework
Gemma models are compatible with a wide variety of ecosystem tools. Choosing the right one depends on your available hardware (Cloud GPUs versus Local Laptop) and your interface preference (Python code versus Desktop Application).
Use the following table to quickly identify the best tool for your needs:
| If you want to... | Recommended Framework | Best For |
|---|---|---|
| Run locally with a Chat UI | - LM Studio - Ollama |
Beginners, or users who want a "Gemini-like" experience on their laptop. |
| Run efficiently on Edge | - LiteRT-LM - llama.cpp |