At Google I/O 2024, Google unveiled Firebase Genkit, an open-source framework designed to help developers incorporate generative AI into web and mobile applications. Firebase Genkit supports models such as Google Gemini and Google Gemma, enabling functionalities like intelligent agents, automated customer support, semantic search, and converting unstructured data into actionable insights. The framework includes a developer UI that facilitates the prototyping, development, and testing of AI features within a local development environment.

Google partnered with NVIDIA to enhance inference performance for the Gemma model on NVIDIA RTX professional GPUs and NVIDIA GeForce RTX GPUs. This collaboration aims to boost developer productivity and improve application responsiveness by enabling the local execution of Firebase Genkit on these GPUs.

Developers can leverage Firebase Genkit to run AI models locally, using the Genkit plugin to integrate Ollama for hosting Gemma on their local machines. Ollama, an open-source framework, simplifies the deployment of large language models (LLMs) locally, thus providing enhanced inference performance directly on desktop systems.