Gemma 4 on your own hardware: from choosing a variant to a first app

Your own model starts with choosing a version

Want to run a model yourself and see what happens outside a provider’s service? Gemma 4 is a family of open-weight models. Google provides the model parameter files for use in building applications. The documentation lists E2B, E4B, 12B, 31B and 26B A4B. The release log dates 12B Unified to June 3, 2026.

Your own hardware lets you plan how data is processed and learn the mechanics of running a model. Before downloading a version, check its format, documentation and the requirements of the tool that will run it. The variant name alone will not tell you enough.

Start with an inbox of fictional support requests

We suggest a small project: organizing support requests. The model assigns each message a category and briefly explains its choice. Add “needs review” so a vague description does not force a guess.

Label a small set of examples manually and hold some back for the final evaluation. Otherwise, it is easy to refine the instructions only for messages whose answers you already know. If the model confuses payments with refunds, revisit the category definitions. If it gets lost in long requests, check that it received the full text.

The outcome is easy to show a future user: a message lands in the right category or reaches someone who can review it. There is room in the demo for successful assignments and cases that need help.

Who will look after the service once it is running?

Once the model replies, record memory usage, time to the first answer and behavior with several concurrent tasks. A conversation on an idle machine is different from the whole team using the application at once.

Include updates, restarts after failures and log reviews in your calculations. With your own service, those jobs stay with you even if there is no longer a charge for each external API call.

Keep the model identifier, settings and evaluation set. You will have something to compare the next result with. Once the basic case works to the agreed standard, you can add an interface and more tasks.