Interactions Lab > Research

Check out some of the recent projects from the Interactions Lab below, organised into four themes: Wearables, Tangible Interaction, Cognition and Embodiment and Social Computing. For a complete, comprehensive list of our research outcomes and interests, head over to the publications page.

You can also see some information about our lab and the equipment and facilities we have for realising projects in the areas of physical and tangible computing.


Finger ID on smartwatches

How we identify the finger touching a smartwatch? This project looks at whether the touch contact regions generated by different fingers during interaction with a smartwatch are distinct from one another. Using raw data from the touch screen driver of a modified Android kernel, we build machine learning models to distinguish fingers and show how these can be used to create interfaces where different functions are assigned to different digits.

Read more: Gil, H.J., Lee, D.Y., Im, S.G. and Oakley, I. (2017) "TriTap: Identifying Finger Touches on Smartwatches." To appear in Proceedings of ACM CHI'17, Denver, CO, USA.

Know more: Watch the teaser video from CHI 2017, or download source and binaries on github


Touch contact shape input

Can we use the shape of our finger touch to control smart watches? We developed a watch-format touch sensor capable of capturing the contact region of a finger touch and used this platform to explore the kinds of contact area shapes that users can make. We also investigated the design space of this technique and propose a series of interaction techniques suitable for contact area input on watches.

Read more: Oakley, I., Lindahl, C., Le, K., Lee, D.Y. and Islam, R.M.D. "The Flat Finger: Exploring Area Touches on Smartwatches". In Proceedings of ACM CHI'16, San Jose, CA, USA. [download article]

See more: Watch the teaser video from CHI 2016


Tapping gestures on watches

How can we quickly and easily control smart watches? This work explores how rapid patterns of two-finger taps can be used to issue commands on a smart watch. The goal of the this work is to design interfaces that give access to a wide range of functionality without requiring users to navigate through menus or multiple screens of information.

Read more: Oakley, I., Lee, D.Y., Islam, R.M.D. and Esteves, A. "Beats: Tapping Gestures for Smart Watches". In Proceedings of ACM CHI'15, Seoul, Republic of Korea. [download article]

See more: Watch the teaser video from CHI 2015


Touching the edge

How can we interact with very small mobile or wearable computers? With next generation personal computing devices promising more power in smaller packages (such as smart watches or jewellery) interaction techniques need to scale down. This work explores interaction via an array of touch sensors positioned all around the edge of a small device with a front-mounted screen. This arrangement sidesteps the "fat-finger" problem, in which a user's digits obscure content and hamper interaction.

Read more: Oakley, I. and Lee, D.Y. (2014) "Interaction on the Edge: Offset Sensing for Small Devices". In proceedings of ACM CHI 2014, Toronto, Canada. [download article]

See more: Watch the teaser video from CHI 2014

Tangible Interaction

A Tangible Reading Aid

How can we make reading on electronic devices better? We propose a tangible reading aid in the form of the eTab - a smart bookmark designed to scaffold and support advanced active reading activities such as navigation, cross-refereeing and note taking. The paper describes the design, implementation and evaluation of the eTab prototype on a standard Android tablet computer.

Read more: Bianchi, A., Ban, S.R. and Oakley, I. "Designing a Physical Aid to Support Active Reading on Tablets". In Proceedings of ACM CHI'15, Seoul, Republic of Korea. [download article]

See more: Watch the teaser video from CHI 2015


Tracking multiple magnets

How can we sense multiple objects on and around current mobile devices? While prior work has shown that embedding a magnet in one object allows it to be tracked, scaling this up to multiple objects has proven challenging. This paper proposes a solution - spinning magnets. By looking for the systematic variations in magnetic field strength this causes, we are able to infer the location of each of a set of tokens.

Read more: Bianchi, A. and Oakley, I. "MagnID: Tracking Multiple Magnetic Tokens". In Proceedings of ACM TEI'15, Stanford, CA, USA. [download article]

Know more: Watch the video from TEI 2015, check out the Hackaday article or the source on github


Magnetic Appcessories

How can physical, tangible interfaces be introduced to everyday computers? This work explores how mobile devices can be used as platforms for tangible interaction through the design and construction of eight magnetic appcessories. These are cheap, robust physical interfaces that leverage magnets (and the magnetic sensing built into mobile devices) to support reliable and expressive tangible interactions with digital content.

Read more: Bianchi, A and Oakley I. (2013) "Designing Tangible Magnetic Appcessories". In proceedings of ACM TEI 2013, Barcelona, Spain. [download article]

Know more: Watch the video on Youtube or check out the articles on Gizmodo, Slashdot and Make Magazine

Cognition and Embodiment

Assessing epistemic action

Physical, tangible interfaces are compelling, but what makes them better than conventional graphical systems? One answer might be that they facilitate epistemic action - the manipulation of external props as tools to simplify internal thought processes. To explore this idea, we developed the ATB framework, a video coding instrument for fine-grained assessment of epistemic activity. We present an initial user study that suggests it is reliable and that the number and type of epistemic actions a user performs meaningfully relates to other aspects of their task performance such as speed and ultimate success.

Read more: Esteves, A. Bakker, S., Antle, A., May, A., Warren, J. and Oakley, I. "The ATB Framework: Quantifying and Classifying Epistemic Strategies in Tangible Problem-Solving Tasks". In Proceedings of ACM TEI'15, Stanford, CA, USA. [download article]


Cognition and gameplay

Our minds don't work in isolation, but operate embedded and embodied in our bodies and the world. To understand the importance of this assertion, we are exploring how physical, tangible interfaces and representations - things that users can reach out and hold - impact user performance in problem-solving tasks such as puzzles and games. We seek to elaborate the ways in which the representation of a problem affects how our minds can conceive and deal with it.

Read more: Esteves, A., Hoven, E. van den and Oakley I. (2013) Physical Games or Digital Games? Comparing Support for Mental Projection in Tangible and Virtual Representations of a Problem Solving Task. In proceedings of ACM TEI 2013, Barcelona, Spain. [download article]

Social Computing

Motives for using Facebook

What does your Facebook profile really say about you? This work connected Uses and Gratifications (U&G) theory, a framework that aims to explain the how and why of media consumption, with data captured from Facebook. Specifically, motives captured via a systematic survey were linked to data summarising both an individuals friendship network and detailed site usage statistics. This work both expands the scope of U&G theory and highlights just how much the data Facebook stores can reveal about you.

Read more: Spiliotopoulos, T. and Oakley, I. (2013) "Understanding motivations for Facebook use: Usage metrics, network structure, and privacy". In proceedings of ACM CHI 2013, Paris, France. [download article]


Social Technology and Music

Technology is changing musical consumption, production and performance in unprecendented ways. In particular, social networking sites (SNS) are a 'disruptive force of change' catalysing the consumption, production and dissemination of music on one hand and on the sociality it enables or disables on the other. This project seeks a deeper understanding of the impact of social technologies on musical practices can inform design of novel paradigms of social interaction online and offline.

Read more: Karnik, M., Oakley, I., Venkatanathan, J., Spiliotopoulos, T. and Nisi, V. (2013) "Uses & Gratifications of a Facebook Media Sharing Group". In proceedings of ACM CSCW 2013, San Antonio, Texas. [download article]

Equipment and Facilities

The Interactions Lab designs, builds and evaluates novel interactive systems. It particularly focuses on physical, tangible and social computing systems. You can see some of the equipment we use to realise these projects below.


To build functional interactive prototypes, we have additive and subtractive digital fabrication tools including a Silhouette Cameo vinyl cutter, two 3D printers (an Ultimaker and a Stratasys Mojo) and a Roland MDX-20 mill.

Tangible and Surface Computing

To support our work on tangible interaction, we have two tabletop computers capable of tracking multiple fingers and objects: the Microsoft PixelSense (or Samsung SUR-40) and the ReacTable Live.

Physical Computing

We have electronics tools, equipment and general supplies to realise for physical computing projects based on Arduino, Raspberry Pi, Phidgets and Microsoft Gadgeteer platforms.

Interactions Lab, School of Design and Human Engineering
Ulsan National Institute of Science and Technology, UNIST-gil 50, Ulsan, 689-798, Republic of Korea