Hidden Interfaces for Ambient Computing

As shopper electronics and internet-connected home equipment have gotten extra frequent, houses are starting to embrace numerous varieties of related units that supply performance like music management, voice help, and residential automation. A swish integration of units requires adaptation to current aesthetics and person kinds fairly than merely including screens, which may simply disrupt a visible area, particularly once they grow to be monolithic surfaces or black screens when powered down or not actively used. Thus there may be an rising want to create related ambient computing units and home equipment that may protect the aesthetics of on a regular basis supplies, whereas offering on-demand entry to interplay and digital shows.

Illustration of how hidden interfaces can seem and disappear in on a regular basis surfaces, equivalent to a mirror or the wooden paneling of a house equipment.

In “Hidden Interfaces for Ambient Computing: Enabling Interplay in On a regular basis Supplies by Excessive-Brightness Visuals on Low-Price Matrix Shows”, introduced at ACM CHI 2022, we describe an interface expertise that’s designed to be embedded beneath supplies and our imaginative and prescient of how such expertise can co-exist with on a regular basis supplies and aesthetics. This expertise makes it doable to have high-brightness, low-cost shows seem from beneath supplies equivalent to textile, wooden veneer, acrylic or one-way mirrors, for on-demand touch-based interplay.

Hidden interface prototypes exhibit vibrant and expressive rendering beneath on a regular basis supplies. From left to proper: thermostat underneath textile, a scalable clock underneath wooden veneer, and a caller ID show and a zooming countdown underneath mirrored surfaces.

Parallel Rendering: Boosting PMOLED Brightness for Ambient Computing
Whereas lots of right this moment’s shopper units make use of active-matrix natural light-emitting diode (AMOLED) shows, their value and manufacturing complexity is prohibitive for ambient computing. But different show applied sciences, equivalent to E-ink and LCD, don’t have ample brightness to penetrate supplies.

To deal with this hole, we discover the potential of passive-matrix OLEDs (PMOLEDs), that are based mostly on a easy design that considerably reduces value and complexity. Nonetheless, PMOLEDs usually use scanline rendering, the place lively show driver circuitry sequentially prompts one row at a time, a course of that limits show brightness and introduces flicker.

As a substitute, we suggest a system that makes use of parallel rendering, the place as many rows as doable are activated concurrently in every operation by grouping rectilinear shapes of horizontal and vertical strains. For instance, a sq. could be proven with simply two operations, in distinction to conventional scanline rendering that wants as many operations as there are rows. With fewer operations, parallel rendering can output considerably extra gentle in every prompt to spice up brightness and get rid of flicker. The method will not be strictly restricted to strains and rectangles even when that’s the place we see probably the most dramatic efficiency enhance. For instance, one might add further rendering steps for antialiasing (i.e., smoothing of) non-rectilinear content material.

Illustration of scanline rendering (high) and parallel rendering (backside) operations of an unfilled rectangle. Parallel rendering achieves vibrant, flicker-free graphics by concurrently activating a number of rows.

Rendering Person Interfaces and Textual content
We present that hidden interfaces can be utilized to create dynamic and expressive interactions. With a set of basic UI parts equivalent to buttons, switches, sliders, and cursors, every interface can present totally different primary controls, equivalent to gentle switches, quantity controls and thermostats. We created a scalable font (i.e., a set of numbers and letters) that’s designed for environment friendly rendering in only a few operations. Whereas we at present exclude letters “ok, z, x” with their diagonal strains, they could possibly be supported with further operations. The per-frame-control of font properties coupled with the excessive body price of the show permits very fluid animations — this functionality significantly expands the expressivity of the rectilinear graphics far past what is feasible on fastened 7-segment LED shows.

On this work, we exhibit numerous examples, equivalent to a scalable clock, a caller ID show, a zooming countdown timer, and a music visualizer.

Realizing Hidden Interfaces with Interactive {Hardware}
To implement proof-of-concept hidden interfaces, we use a PMOLED show with 128×96 decision that has all row and column drivers routed to a connector for direct entry. We use a customized printed circuit board (PCB) with fourteen 16-channel digital-to-analog converters (DACs) to straight interface these 224 strains from a Raspberry Pi 3 A+. The contact interplay is enabled by a ring-shaped PCB surrounding the show with 12 electrodes organized in arc segments.

Comparability to Present Applied sciences
We in contrast the brightness of our parallel rendering to each the scanline on the identical PMOLED and a small and huge state-of-the-art AMOLED. We examined brightness by six frequent supplies, equivalent to wooden and plastic. The fabric thickness ranged from 0.2 mm for the one-way mirror movie to 1.6 mm for basswood. We measured brightness in lux (lx = gentle depth as perceived by the human eye) utilizing a gentle meter close to the show. The environmental gentle was stored dim, barely above the sunshine meter’s minimal sensitivity. For easy rectangular shapes, we noticed 5–40x brightness enhance for the PMOLED compared to the AMOLED. The exception was the thick basswood, which didn’t let a lot gentle by for any rendering expertise.

Instance displaying efficiency distinction between parallel rendering on the PMOLED (this work) and a equally sized fashionable 1.4″ AMOLED.

To validate the findings from our technical characterization with extra practical and complicated content material, we consider the quantity “2”, a grid of checkboxes, three progress bars, and the textual content “Good Life”. For this extra complicated content material, we noticed a 3.6–9.3x brightness enchancment. These outcomes counsel that our strategy of parallel rendering on PMOLED permits show by a number of supplies, and outperforms frequent state-of-the-art AMOLED shows, which appear to not be usable for the examined situations.

Brightness experiments with further shapes that require totally different numbers of operations (ops). Measurements are proven compared to massive state-of-the-art AMOLED shows.

What’s Subsequent?
On this work, we enabled hidden interfaces that may be embedded in conventional supplies and seem on demand. Our lab analysis suggests unmet alternatives to introduce hidden shows with easy, but expressive, dynamic and interactive UI parts and textual content in conventional supplies, particularly wooden and mirror, to mix into folks’s houses.

Sooner or later, we hope to research extra superior parallel rendering methods, utilizing algorithms that might additionally help photos and complicated vector graphics. Moreover, we plan to discover environment friendly {hardware} designs. For instance, application-specific built-in circuits (ASICs) might allow a reasonable and small show controller with parallel rendering as a substitute of a big array of DACs. Lastly, longitudinal deployment would allow us to go deeper into understanding person adoption and conduct with hidden interfaces.

Hidden interfaces exhibit how management and suggestions surfaces of good units and home equipment might visually disappear when not in use after which seem when within the person’s proximity or contact. We hope this course will encourage the neighborhood to think about different approaches and situations the place expertise can fade into the background for a extra harmonious coexistence with conventional supplies and human environments.

Acknowledgements
Initially, we wish to thank Ali Rahimi and Roman Lewkow for the collaboration, together with offering the enabling expertise. We additionally thank Olivier Bau, Aaron Soloway, Mayur Panchal and Sukhraj Hothi for his or her prototyping and fabrication contributions. We thank Michelle Chang and Mark Zarich for visible designs, illustrations and presentation help. We thank Google ATAP and the Google Interplay Lab for his or her help of the challenge. Lastly, we thank Sarah Sterman and Mathieu Le Goc for useful discussions and solutions.