Home Assistant

ESP32-S3-BOX voice assistant

This tutorial will guide you to turn an ESP32-S3-BOX, ESP32-S3-BOX-3, or an ESP32-S3-BOX-Light into a Home Assistant voice assistant. Note, the term ESP32-S3-BOX is used to refer to any of the 3 product variants.

Prerequisites

  • Home Assistant 2023.12 or later, installed with the Home Assistant Operating System. If you do not have Home Assistant installed yet, refer to the installation page for instructions.
  • Home Assistant Cloud or a manually configured Assist Pipeline
  • The password to your 2.4 GHz Wi-Fi network
  • Chrome or Edge browser on a desktop (not Android/iOS)
  • One of the three Espressif ESP32-S3-BOX variants:
    • ESP32-S3-BOX-3
    • ESP32-S3-BOX or ESP32-S3-BOX-Lite (not currently on the market)
  • USB-C cable to connect the ESP32-S3-BOX

Installing the software onto the ESP32-S3-BOX

Before you can use this device with Home Assistant, you need to install a bit of software on it.

  1. Make sure this page is opened in a Chromium-based browser on a desktop. The software installation does not work with a tablet or phone.

    • If you have an ESP32-S3-BOX-3, select the Connect button below to display a list of available USB devices. Do not connect the ESP32-S3-BOX-3 yet. We want to see the list of available USB devices first, so that it is easier to recognize the ESP device afterwards.

    • If your browser does not support web serial, you will see a warning message indicating this instead of a button.

    • If you have an ESP32-S3-BOX or ESP32-S3-BOX-Lite, open the ESPHome projects page, select your variant and follow the installation instructions.

    • For advanced users: The configuration files are available on GitHub:

  2. To connect the ESP32-S3-BOX to your computer, follow these steps:

    • In the pop-up window, view the available ports.
    • Plug the USB-C cable into the ESP32-S3-BOX and connect it to your computer.
      • If you have an ESP32-S3-BOX-3, plug it into the box directly, not into the docking station (not into the blue part).
      • Troubleshooting: If your ESP32-S3-BOX-3 does not appear in the list of devices presented by your browser, you need to manually invoke “flash mode”:
        • Hold the “boot” button (left side upper button) as you tap the “reset” button (left side lower button).
        • After a few seconds, the ESP32-S3-BOX-3 should appear in the list of USB devices presented by your browser.
        • Follow the steps until step 3. After selecting the Next button, on the S3-Box, tap the “Reset” button again.
        • Then, select the blue Connect button again, select the USB device and follow the prompts to configure the Wi-Fi.
    • In the pop-up window, there should now appear a new entry. Select this USB serial port and select Connect.

  3. Select Install Voice Assistant, then Install.

    • Once the installation is complete, select Next.
    • Add the ESP32-S3-BOX to your Wi-Fi:
      • When prompted, select your network from the list and enter the credentials to your 2.4 GHz Wi-Fi network.
      • Select Connect.
      • The ESP32-S3-BOX now joined your network. Select Add to Home Assistant.

  4. This opens the My link to Home Assistant.

    • If you have not used My Home Assistant before, you will need to configure it. If your Home Assistant URL is not accessible on http://homeassistant.local:8123, replace it with the URL to your Home Assistant instance.
    • Open the link. Open My link

  5. Select OK.

    Set up ESPHome

  6. To add the newly discovered device, select the ESP32-S3-BOX from the list.

    • Add your ESP32-S3-BOX to a room and select Finish.

  7. You should now see the ESPHome integration. New ESPHome device discovered

  8. Select the ESPHome integration. Under Devices, you should see the ESP32-S3-BOX listed. ESP32-S3-BOX-3 discovered

    • Your ESP32-S3-BOX is connected to Home Assistant over Wi-Fi. You can now move it to any place in your home with a USB power supply.

  9. If you want, you can process the wake word on the ESP32-S3 device, rather than on your Home Assistant server. (The server is the device where Home Assistant is installed, for example on Home Assistant Green).

    • Under Devices, on the ESP32-S3-BOX-3 entry, select Device to open the device page.

    • Under Wake word engine location, select On device, if you want your wake word to be processed on the device itself, and not in Home Assistant.

      • Local processing is faster.
      • The wake word is now Okay Nabu.

      ESP32-S3-BOX-3 on device wake word processing

  10. If you chose on-device wake word, but you do not want to use Okay Nabu, you can change the on-device wake word.

  11. Congratulations! You can now voice control Home Assistant via a ESP32 device with a display. Now give some commands.

Controlling Home Assistant

  1. Say your wake word. For this tutorial, use “OK, Nabu”.
  2. Say a supported voice command. For example, Turn on the light.
    • Once the intent has been processed, the LED lights up in green and Home Assistant confirms the action.
      • Make sure you’re using the area name exactly as you defined it in Home Assistant.
      • You can also ask a question, such as
        • Is the front door locked?
        • Which lights are on in the living room?
  3. Your command is not supported? Add your own commands using a sentence trigger.

Turning off microphone or screen

  1. If you do not want to Assist to listen for a while, you can turn off the microphone.

    • Go to Settings > Devices & Services and select the ESPHome integration.

      • Under ESP32-S3-BOX-3, select 1 device.
      • Enable Mute.
      • The screen of the ESP32-S3-BOX-3 will turn off, too.

      Toggle to enable/disable Mute

  2. If you want to just use the wake word, but do not want to use the screen, you can turn it off.

Related topics

Related links