Home Assistant

Google Generative AI Conversation

The Google Generative AI integration adds a conversation agent powered by Google Generative AI in Home Assistant.

This conversation agent is unable to control your house. The Google Generative AI conversation agent can be used in automations, but not as a sentence trigger. It can only query information that has been provided by Home Assistant. To be able to answer questions about your house, Home Assistant will need to provide Google Generative AI with the details of your house, which include areas, devices and their states.

This integration requires an API key to use, which you can generate here.

Configuration

To add the Google Generative AI Conversation integration to your Home Assistant instance, use this My button:

Generate an API Key

The Google Generative AI API key is used to authenticate requests to the Google Generative AI API. To generate an API key take the following steps:

  • Visit the API Keys page to retrieve the API key you’ll use to configure the integration.

Options

Options for Google Generative AI Conversation can be set via the user interface, by taking the following steps:

  • Browse to your Home Assistant instance.
  • Go to Settings > Devices & Services.
  • If multiple instances of Google Generative AI Conversation are configured, choose the instance you want to configure.
  • Select the integration, then select Configure.
Prompt Template

The starting text for the AI language model to generate new text from. This text can include information about your Home Assistant instance, devices, and areas and is written using Home Assistant Templating.

Model

Model used to generate response.

Temperature

Creativity allowed in the responses. Higher values produce a more random and varied response. A temperature of zero will be deterministic.

Top P

Probability threshold for top-p sampling.

Top K

Number of top-scored tokens to consider during generation.

Maximum Tokens to Return in Response

The maximum number of words or “tokens” that the AI model should generate.

Services

Service google_generative_ai_conversation.generate_content

Allows you to ask Gemini Pro or Gemini Pro Vision to generate content from a prompt consisting of text and optionally images. This service populates response data with the generated content.

Service data attribute Optional Description Example
prompt no The prompt for generating the content. Describe this image
image_filename yes File names for images to include in the prompt. /tmp/image.jpg 
service: google_generative_ai_conversation.generate_content
data:
  prompt: >-
    Very briefly describe what you see in this image from my doorbell camera.
    Your message needs to be short to fit in a phone notification. Don't
    describe stationary objects or buildings.
  image_filename: /tmp/doorbell_snapshot.jpg
response_variable: generated_content

The response data field text will contain the generated content.

Another example with multiple images:

service: google_generative_ai_conversation.generate_content
data:
  prompt: >-
    Briefly describe what happened in the following sequence of images
    from my driveway camera.
  image_filename:
    - /tmp/driveway_snapshot1.jpg
    - /tmp/driveway_snapshot2.jpg
    - /tmp/driveway_snapshot3.jpg
    - /tmp/driveway_snapshot4.jpg
response_variable: generated_content