context
In this course we researched future-focused technologies and design for interactions that may arise when these technologies mature. An emphasis was placed on anticipating the societal impact of these technologies and accounting for potential pitfalls in our designs.
time, course
project duration: one semester
course: Innovation Design 1
semester: 3rd in 2019
kick-off
Technological development must be a social debate. The more people know, the more they can act. To create these debates we are working on an Adaptive User Interface with Artificial Intelligence.
definition
Adaptive User Interface are an evolution of standard static UIs, which present the same UI regardless of the user or context of use. Most interfaces we encounter today are static. AUIs are a stepping stone towards ubiquitous computing (ambient intelligence), a futuristic vision in which every object is equipped with computers, sensors, and networking power. In this vision, the border between the environment and computing interface is blurred – the entire environment becomes an interface.
An Adaptive User Interface (AUI) is a user interface that dynamically adjusts its layout, elements, functionality, and/or content to a given user's needs, capabilities, and context of use.
Advances in artificial intelligence, particularly computer vision and natural language processing, have endowed computers with remarkable sensory abilities. It is only a matter of time before computers are capable of perceiving information about their environments better than humans can.
interaction
An interface is always based on the communication between the user and the user interface, thus creating interaction. The first step for us was to analyze the interaction between different participants and systems. First questions we answered is how can an artificial intelligence get to know the user? And by whom is the the adaption made?
We differentiate between the Human-to-Human-Interaction where the humans communicate with each other. We are able to interpret each other draw conclusions about our environment and develop intelligent reactions is the core of human existence.
With a Human-to-Computer-Interaction the information of the communication is lost. As it stands, classical computers are simply incapable of interpreting humans as like we can interpret each other. In this project, we asked ourselves, what if human-computer interaction were as rich as human-to-human interaction? How could we design computers to intelligently interpret and react to human behaviour?
what are possible adaption parameters ?
To revive the question from our kick-off above, we worked on several interaction sketches for scenarios where adaptive UIs might be useful. We concluded different user models:
physical traits
In a public setting, it is impossible to design a system perfectly tailored to each individual's dimensions. An interface suited for a moderately tall user will prove cumbersome for both tall and short users. What if we didn't have to design a single interface for everyone? Adaptive UI systems could conceivable detect and adapt themselves to a user's height, vertically aligning and scaling the interface appropriately.
habits and abilities
A user's navigation habits gives insight into their goals, priorities, and individual preferences. Their navigation ability may reflect slight difference in micro-interactions or gross differences in motor ability. How can adaptive interfaces adjust themselves to accelerate common tasks and accommodate individual differences in ability?
*use case 1
This use case with the smartphone detects a very slow reader and suggests that the text be played using a reader tuner. The focus point runs along with the text in support of the voice.
*use case 2
In this use case, the system recognizes that the reader finds it difficult to read text with a very wide column of text. The system adapts the layout including the text width with a query that the user can confirm or reject.
context and surroundings
We are always on the move–surrounded by different people, navigataing different social contexts, traveling within our towns, across countries, or around the world. Despite our ever-changing environments, our digital devices always remain the same. How might we incorporate situational and contextual awareness into our devices?
*use case 2
The system preserves your privacy, as soon as the camera detects a stranger, options for editing and changes on your smartphone are blocked.
framework - magic mirror
We have developed a framework for adaptive user interface systems and an installation to demonstrate the current state of artificial perception. Our mirror was designed to playfully and intuitively communicate the present capabilities of machine perception to the general public and provoke critical discussion thereof. Equipped with facial recognition, keypoint tracking, and emotion recognition, the mirror could recognize past visitors, estimate a user’s age and gender, and display facial features and emotional cues in real-time.
A person positions themself in front of this installation. The camera detects different characteristics of the person and compares them with its own database to find out how old and what gender the person is.
Mockup mirror design (left),
Screenshot of real prototype (right)
https://www.schwaebische-post.de/ostalb/ostalbkreis/so-gefragt-sind-digitale-ideen-90311984.html
team partner
Kalle Robinson Reiter and Matthew Jörke
supervised by
Professor Jörg Beck
