TextARt

UX/UI in AR

Thesis question

'Why do people less care about TEXT in AR/VR than visuals?'

The state of UI/UX design in virtual worlds is still evolving. The use of typography and text is a prime example of this. At the moment, text elements in virtual worlds for the most part still follow the design conventions developed for 2D environments, such as print or flat web pages, rather than taking into account the more dimensional and experiential characteristics of 3D environments. I have been curious to explore new ways to visualize text in virtual spaces that move beyond traditional 2D graphic design.   

For my thesis project I have experimented with ways to make text come alive with AR by creating more animated and dimensional text examples in the process of learning simple verbs in a new language. For example, the word ‘SPIN’ could be visualized by displaying a spinning animation of the word, or the letters that make up the word, with related environmental elements that help provide a context for the spinning action.

Personal statement

My previous work experiences in Korea were very much text related. Having worked as an editor and copywriter, I am interested in the visual power of text and how words can be designed to enhance the telling of stories. After studying graphic design, I learned how typeface is often as important as other graphic elements in terms of delivering a clear and strong message. That is why I am interested in the role of text design in virtual worlds. Through experiments like mine, my hope is that text design will start to evolve as a more integrated part of the story and experience in virtual spaces. 

Research summary

A couple of projects in particular have inspired the design of my thesis prototype. WordWorld is an American animated television series for preschool children focused on word building. It reinforces the pre-reading concept that letters (and their sounds) make words, and that words have real meaning. The meaning of words is visualized by the arrangements of the letters. For example, the word ‘cow’ is depicted by the letters, c, o, w. arranged in the shape of a cow. WordWorld also introduces essential emergent literacy concepts—such as sounding out letters, “chunking” words, and rhyming—that all children encounter as they learn to read.  (http://www.wordworld.com/)   
Concrete poetry, which is defined as “poetry in which the meaning or effect is conveyed partly or wholly by visual means, using patterns of words or letters and other typographical devices” (definition by google) has also been in inspiration.

Experiments of 3D text

At the beginning, these tests were still based in traditional 2D graphic design forms. But eventually, my text evolved to have volume and movement. 

Content: Learning Vocabulary Words

Looking back to my very early days when I made the English language learning book in Korea, I thought that, in terms of content for my prototype, typography in virtual space might be a good way to learn a new language. Because text can really come alive with AR, which allows for the creation of more animated and dimensional letters. 

Format

1) AR APP

AR as a medium falls at the intersection of traditional 2D print and 3D virtual space. In a way, it is 2 ½ D. It allows for the display of information such as text in 2D and 3D at the same time and lets users experience how text might be enhanced and further brought to life through animation in virtual space. As a test case for exploring the design of text in virtual space, I started with some prototype examples of using AR for learning simple vocabulary words in a foreign language through animated visualizations and related audio support for pronunciations.  My goals here are to show a new way of visualizing text in AR/3D, to suggest that AR/virtual space is a great medium for experimenting with text design and to begin to establish new conventions for the use of text in virtual worlds.

2)  AR APP + A Learn-A-New-Word-A-Day Calendar

For a 2D print medium which could be repurposed for AR, I decided to focus on a daily calendar. First, because the form of such a calendar is very versatile  ground for free typographical experimentation - it is a bit like a poster. Secondly, a calendar is helpful in creating a routine, which is an impactful ingredient for language learning. 

Target Users

- Main target: designers who are interested in the creation of text and typography in virtual worlds

- Sub target: Beginner learners of Korean who want to extend their vocabulary in a fun, playful way

Because the I chose the vocabulary learning as a way of my prototyping, and it is not a educational learning book for serious learners, I did not want to go deeply in as an educational function. 

How to use

For my prototype, I’ve experimented with Korean verbs. Each page of the paper calendar prototype includes the day’s date written at the top, one of the verbs written in Korean letters in the middle and the phonetic pronunciation of the verb written in English at the bottom. When users hold the phone over the letters written on each calendar page, they see an AR animation of the letters that suggests the meaning of the verb, hear the pronunciation of the word in Korean, and see the equivalent English word. For creating the AR enhancements, I used Unity and 3D modeling programs

Calendar design

It uses the same color with inside papers. Visual elements on the corners are borrowed from the Korean flag. The name of the Korean alphabet is at the middle, and its design suggests volume and layering. Each page of the paper calendar prototype includes the day’s date written at the top, one of the verbs written in Korean letters in the middle, and the phonetic pronunciation of the verb written in English at the bottom. 

AR text sketches

AR text creation with Unity 3D

Example 1. Roast

Example 5. Catch

User tests and design revisions

I also did a user test for not only AR words but also calendar design to give better user experience. I found that the animations were viewed differently depending on whether they were viewed on a screen or in a virtual space. Several revisions were needed in the AR space in order to match the 3D text animation on the screen.