Liberating Aestheticodes

The idea of scanning visual codes to trigger digital interactions is well established. The use of specifically designed codes, containing many bits of information reaches back to the use of barcodes. More recently, 2D codes that are readable by mobile applications (e.g., QR codes), have become popular. These techniques are designed to be robust but come at the cost of a limited aesthetic.

As an alternative we have refined a technique, named Aestheticodes, to embed multiple visual codes in wider decorative patterns [M+13]. This is based on d-touch [CH09], a visual marker recognition system which uses topological markers that encode information in the structure of the image. The core drawing rule specifies how various black and white regions must be nested inside one another in order to form a valid visual marker: “A valid marker can be composed of a black region containing 3 or more white regions, and at least half of these white regions must contain one or more black regions. This makes exactly 3 levels of nesting – it must be no more and no less. However, there is no limit in the number and shape of the regions. We felt this to be especially appropriate because it embodies a bottom-up, drawing-based approach that we felt would fit very well with the drawing skills of professional designers.

In the Trackable Tableware project we have worked with ceramics designers to refine the vision technology and to produce a pattern book of designs (see figure 1). Using these designs we then manufactured place settings (e.g. see figure 2) and worked with a restaurant to explore how we can use the interactive tableware to enhance the dining experience. As part of a CEKE feasibility project, we are currently collaborating with textile designers to create Aestheticodes in lace (with the aim of enhancing the visitor experience of browsing lace collections with digital layers of information).

The aim of this project is to liberate Aestheticodes from the restricted range of markers that can be currently recognized and to liberate both the codes and the people who interact with them from being static. To achieve this we propose to establish a multidisciplinary team combining expertise in qualitative spatial representation and reasoning (Stell and Worboys), software systems (Mortier), design (Quinn and Thorn) and HCI (Koleva and Benford). We will focus on two key strands of work:

1. We will explore the feasibility of extending the underlying code space by using more complex topological relationships and by using granular (level of detail) effects in designs. This will extend the commercial application of Aestheticodes by having a larger number of unique ids that can be robustly detected.
2. We will explore the feasibility of creating dynamic Aestheticodes including both codes which are animated (code embedded in moving image on screen or fabric as it changes shape) and static codes whose appearance changes as they are scanned from different positions by a moving participant. Such codes will enable exciting new interactions with patterns such as zooming and panning on larger surfaces (wall papers, tiles, fabrics and furnishings) and lay the foundation for the use of Aestheticodes in future interactive environments. Imagine being able to wander in a space where dynamic images cover walls; as you move the Aestheticodes in the images are read by some form of wearable device (e.g. like Google Glass or part of clothing); depending on the codes and the individual, different effects are produced (e.g. visually on a personal display or via clothing that produces sensations, changes colour, etc). While this vision is well beyond the deliverables of this project, the proposed work will help us to understand the challenges that need to be addressed.