FROM THE BUILD · PIP PROTOTYPEMeet Pip.
An AI desktop companion
Developing an ESP32-based AI desktop companion with wake-word detection, voice interaction, animated eyes, and persistent conversation context. Pip combines embedded hardware with online AI services to research questions, generate images, and send emails, including research results and generated pictures.
Takeaways & skills
- Skills developed
- ESP32 and Python integration, serial communication, audio input/output, wake-word handling, display animation, persistent context, and debugging response delays. Integrating online research, image generation, and email delivery with the conversational interface.
- Beyond the technical work
- Breaking a complex interaction into smaller stages makes failures easier to isolate. Iterating on listening cues and response timing reinforced that usability matters alongside technical functionality—and that each fix should be checked before adding another feature.
Project notes
Earlier Python-based prototypes integrated computer vision, sensor input, application control, and serial communication with ESP32 hardware. The standalone implementation brings the microphone, speaker, display, and conversational interface into a dedicated embedded device.
Current firmware work focuses on the transition from wake-word detection to recording, audible start/stop cues, and voice-command handling for display parameters such as eye color. Persistent memory retains conversation context across interactions; latency work targets the delay between user input and spoken output.
Beyond conversation, Pip can research topics online, generate images from spoken requests, and email results or pictures. Ongoing development connects these tools with persistent context so follow-up requests can refer back to earlier conversations and images.





