Gestural Interactions:
Button Press

Gestural Interactions:
Button Press

Overview:

This research project seeks to design novel gestures rooted in everyday interactions derived from diverse mediums and perspectives such as graphic design, motion design, sound design, haptics, machine learning, interactive typography, and speculative worldbuilding using Research through Design (RtD) methodologies as an ideation tool.

Initial Approach

+UI Animation

+Teachable
+Machine

+Implementation

When beginning to research UI based gestural interactions for ‘pressing a TV remote button’, I initially and quickly came to the realization that such a simple gesture was a bit exhausted to solely iterate. So I sought to explore variations that built upon a on-Tap gesture.

When beginning to research UI based gestural interactions for ‘pressing a TV remote button’, I initially and quickly came to the realization that such a simple gesture was a bit exhausted to solely iterate. So I sought to explore variations that built upon a on-Tap gesture.

Ideation

I looked to the relationship between the physical action of pressing a button and the subsequent TV’s reaction.

I looked to the relationship between the physical action of pressing a button and the subsequent TV’s reaction.

Initial Notes

Mimicking or visualizing tapping from the front end and back end of pressing a button.

Mimicking or visualizing tapping from the front end and back end of pressing a button.

Imagining a button being pressed from its interface on the remote and a signal meeting or matching that action with the activation of a feature of the TV allowed me to begin my research sketches.

Imagining a button being pressed from its interface on the remote and a signal meeting or matching that action with the activation of a feature of the TV allowed me to begin my research sketches.

Initial Sketches:

"Visualizing the weight or length of a tap through stark & bold shapes morphing."

Each iteration starts with a stark shape appearing on tap, then would animate to each frame below it based on the amount of time held and initial pressure upon ‘tap’.

The end state/frame of each iteration is distinctly different from the first state to distinguish its completion or initiation of a succeeding action.

Obserbations:

The next set of sketches were more interested in a pattern I realized I was naturally filtering into my gestures. Each iteration created so far was centered around a geometic shape that could be sectioned off, so I looked to mobile games to get inspired by clever solutions.

Forms that visually convey the path from action to action.

When trying to imagine the forms for these gestures it was helpful to look to video games designed for mobile and there for, touch based gestures. I specifically looked to staples of mobile games and pivotal experiences in mobile gaming.

Visual Research

The images here show a few examples that helped us create a library. Building off of forms and shape combinations that are inherent to navigational actions.

Mobile Touch Controls

Touch controls from the games below are intuitive and allow the user to envision their path before they even touch the screen.

GTA: Utility Wheel

Nintendo DS: Player Navigation

GTA: Utility Wheel

Ex Astris: Mobile Touch-Controls

Translating the system:

I then posed the question, How might your abstract gestures translate into screen-based interactions? How could we apply the same frame of thinking to variant one central physical gesture, into a bunch of smaller gestures and assign them a value to facilitate and a screen based action?

Teachable
Machine

Tap Variation Recognition:

Interactive
Typography

Tap Variation Recognition: