Beyond Pictures Under Glass: Rethinking Smart Home Control Through Everyday Objects
Picture this: you're settling into your couch after a long day, and instead of fumbling for your phone to dim the lights or adjust the temperature, you simply squeeze the pillow next to you or slide your finger on your coffee table. What sounds like science fiction is actually a return to something more human, i.e. using our hands the way they were meant to be used.
In 2011, designer Bret Victor published A Brief Rant on the Future of Interaction Design, criticising the growing reliance on flat touchscreen interfaces, which he referred to as "Pictures Under Glass." He argued that this model of interaction "denies our hands what they do best" and described it as "a Novocaine drip to the wrist," dulling our natural sense of physical engagement. At the same time, Hiroshi Ishii and the Tangible Media Group at MIT were exploring a different path through their concept of Tangible Bits, which aimed to give physical form to digital information and more closely link the worlds of bits and atoms.
Both Ishii and Victor shared a frustration: We've spent millennia developing sophisticated skills for sensing and manipulating the physical world, yet most modern interfaces barely tap into what our hands are capable of. As Victor put it,
our fingers have an incredibly rich and expressive repertoire, and we improvise from it constantly without the slightest thought.
So why are we limiting ourselves to sliding them across glass?
This question led me down a research rabbit hole that ultimately became the basis of my PhD work. What if everyday household objects around us, the pillows, plants, unused edges and surfaces, lamp shades, etc., could become our smart home user interfaces? Not as replacements for existing interfaces but as complementary ways to interact that feel more natural, more intuitive, and frankly, more fun.
Over the past few years, I have explored how our homes are arranged and what potential they hold for alternative types of interaction. This work led to the design of four functioning prototypes that turn everyday household objects into smart home controls. A vase becomes a volume knob. A coaster adjusts the lighting. A pillow controls the thermostat. An artificial flower opens and closes the curtains. Each object uses a different sensing method such as capacitive touch, pressure sensing, or mechanical input. These technologies are embedded so naturally that guests often do not realise they are interacting with a smart device at all.
The journey wasn't just about the technology, though. It was about understanding how the physical characteristics of objects influence our interactions with them. Through extensive user testing, I discovered that shape, material, size, and even colour profoundly influence whether people find these interfaces intuitive or confusing, delightful or frustrating.
Over the coming weeks, I’ll be sharing a four-part series on how I built a set of unconventional smart home interfaces. These posts will take you behind the scenes of each build, not just the polished results but the missteps, surprises, and design pivots along the way.
From ceramic vases that cracked under pressure to sensors that didn’t survive the seasons, this series documents the messy, hands-on process of turning abstract research into working artefacts. Each post will include build instructions, code, and practical insights, so you can try making your own versions at home.
But this is about more than DIY. It’s about reimagining how we interact with technology in our everyday spaces. What if our homes were filled with interfaces that responded to our touch in intuitive, human ways? What if technology blended into our routines instead of interrupting them?
Pillow Prototype
Coaster Prototype
Flower Prototype
Bibliography
- Hiroshi Ishii. 2008. Tangible bits: beyond pixels. In Proceedings of the 2nd international conference on Tangible and embedded interaction (TEI '08). Association for Computing Machinery, New York, NY, USA, xv–xxv. https://doi.org/10.1145/1347390.1347392
- Victor, Bret. "A Brief Rant on the Future of Interaction Design." Worrydream, 2011. Accessed June 2023. https://worrydream.com/ABriefRantOnTheFutureOfInteractionDesign/.
