ESP32-S3 Voice Assistant

A $20 ESP32-S3 voice assistant with OpenAI Realtime speech, phone-configured personas, animated faces and a printable enclosure.

Background

This open-source push-to-talk voice assistant was created by Udi Tirosh. It combines an ESP32-S3 N16R8, I²S microphone and speaker hardware with the OpenAI Realtime API, then puts the personality controls on a phone-hosted web page instead of baking them into firmware.

The project's roughly $20 target describes a low-cost core build. Optional audio and display hardware, cables, fasteners and printed parts can lift the all-in cost.

Component	Selection	Notes	Price
MCU	ESP32-S3 N16R8 development board	16 MB flash and 8 MB OPI PSRAM	$7
Microphone	INMP441 I²S MEMS microphone	—	$1.50
Audio	MAX98357A I²S amplifier	—	$2
Audio output	PCM5102A DAC	Optional; add about $4.50 for 3.5 mm output	$0
Speaker	1–3 W, 4/8 Ω speaker	Up to 57 mm diameter	$1.50
Display	GC9A01 240×240 round TFT	Optional in the minimal build	$3.50
Control	12×12 mm tactile push button	Momentary push-to-talk switch	$0.25
Power	USB-C data and power cable	—	$2
Hardware	M2/M3 self-tapping screw set	—	$0.75
Enclosure	3D-printed enclosure	Approximate filament cost; model supplied	$1.50
PCB	Two-layer custom PCB	Optional; fabrication cost varies	$0
Total	$20

Wiring

From Blueprint. Source-verified GPIO assignments and power distribution. The MAX98357A uses 5 V; the microphone, display and optional DAC use 3.3 V, with a common ground.

INMP441 microphone: SD to GPIO 40, WS to GPIO 41 and SCK to GPIO 42. Tie L/R to ground, power the module from 3.3 V and share ground with the ESP32-S3.

MAX98357A amplifier: DIN to GPIO 17, BCLK to GPIO 47 and LRC to GPIO 21. Use 5 V for VIN. The optional PCM5102A shares the same I²S data and clock pins, with XSMT on GPIO 39.

GC9A01 display: SCLK to GPIO 6, MOSI to GPIO 7, CS to GPIO 5, DC to GPIO 4 and RST to GPIO 2. The push-to-talk button uses GPIO 1 and the NeoPixel status light uses GPIO 48.

Unlike most DIY voice assistants, this one has no fixed personality. Its owner writes a system prompt on a phone and can swap personas without changing the firmware. The hardware combines push-to-talk input, streamed speech, an animated face and a printable circular enclosure.

Why this $20 ESP32-S3 voice assistant is different

Hold the button, speak, then release it. The device sends the conversation to the OpenAI Realtime API and plays the reply through its speaker. A NeoPixel shows whether it is listening, processing or speaking.

The optional 240×240 display shows an animated face with seven moods selected by the model. Device-side tools can also change the persistent volume or show network information.

The $20 target covers carefully sourced core electronics. The optional DAC, display, printed parts, cable and fasteners increase the final cost.

A phone becomes the personality editor

The device hosts a local configuration page with a system-prompt editor, ten voices, language selection, volume controls, persona import and export, and an animated-face editor. Settings remain in non-volatile storage after a reboot, while ZIP export makes personas easy to share.

The voice assistant beside a phone displaying its persona editor

The browser decodes uploaded media, resizes it and converts each frame to RGB565. The ESP32 receives ready-to-display 150×150 frames and stores them in LittleFS. New artwork is staged before activation, with factory faces retained as a fallback.

How realtime audio runs on an ESP32-S3

The firmware divides time-sensitive work across both ESP32-S3 cores:

Core 0 handles Wi-Fi, secure WebSockets and Base64 processing.
Core 1 handles I²S audio, the push-to-talk button and display updates.
PSRAM ring buffers move 24 kHz PCM audio between tasks.
A roughly 300 ms jitter buffer smooths playback.

An INMP441 captures audio and a MAX98357A drives the speaker. An optional PCM5102A adds a 3.5 mm output.

The project includes a patched ArduinoWebsockets library for its secure connection and uses Arduino_GFX_Library for the round display. Keep these supplied dependencies when first reproducing the build.

Breadboard first, custom PCB when ready

Start with a breadboard and jumper wires for easy debugging. The supplied two-layer KiCad PCB provides a cleaner permanent assembly once the circuit works.

Inside the enclosure, showing the speaker, controller and jumper wiring

The approximately 145 mm parametric enclosure includes printable mounts for the controller, microphone, amplifier, speaker, button, display and optional DAC.

Flash the sketch first, then upload the LittleFS image containing the factory animation frames. Uploading firmware alone leaves the display without its complete face library.

The classroom idea is the real invention

The project began as an educational tool. Students research a historical, literary or philosophical figure, turn that work into a system prompt, interview the result and revise weak or contradictory characterizations. Exportable personas let a small set of devices rotate between classes without reflashing.

Start with the original build instructions and project logs, then download the firmware, KiCad files and enclosure from the project’s source repository.

Project and images by Udi Tirosh. This article is based on the public Hackaday project, its components list and source repository. Hardware Part Picker has not independently reproduced the build.