May 2, 2022
Virtual Typography
My students explored the use of typography in Virtual Reality
Virtual Reality was not made for reading long texts. So it is not surprising that typography does not play a prominent role in the design of virtual environments.
This, however, is a bit of a shame. As Oliver Reichenstein noted, web design is 95% typography and Frank Rausch added that quite often, the user interface is just text. Since even Virtual and Augmented Reality requires some sort of visual user interface, text will be part of any digital environment.
Furthermore, reading is still essential for conveying different kinds of information. Text cannot and should not always be replaced by videos, audio, images and avatars. No matter how the Metaverse plays out — written language will be part of it.
A few weeks ago, I published a rant on the silly obsession with simulation in Virtual Reality. This criticism holds up, especially regarding text and user interfaces. UI is often depicted as 2D windows, panels and buttons floating in 3D space. I generally dislike simulation — but this is just daft:
There have been a couple of interesting experiments with typography — especially in AR. But overall, very little has been done on how to design textual content in a virtual environment. The few examples that exist are more a formal exercise and not an exploration on how to design for real textual content in virtual worlds.
This is not only the fault of developers. Traditionally, most typographers are focused on a 2D surfaces — be it a page or a screen. Since letters are truly two-dimensional, designers are often quite uneasy about pushing them into the third dimension. Even in an architectural context, type is usually placed into 2D rectangles. So in order to create typographic design strategies for VR, we considered the fundamental qualities and characteristics of typography.
Typography is both structural and evocative. It is a rigid arrangement and an open choreography. Typography organises text, but it also uses visual expression in order to convey emotion and abstract ideas.
Typography is usually an invisible art form — but if it breaks with conventions, it can become a powerful amalgam of language and image.
In her excellent book ‘Thinking with Type’, Ellen Lupton aptly writes:
‘Readers usually ignore the typographic interface, gliding comfortably along literacy’s habitual groove. Sometimes, however, the interface should be allowed to fail. By making itself evident, typography can illuminate the construction and identity of a page, screen, place, or product.’
Later, she writes:
‘Designers provide ways into — and out of — the flood of words by breaking up text into pieces and offering shortcuts and alternate routes through masses of information. (…) Although many books define the purpose of typography as enhancing the readability of the written word, one of design’s most humane functions is, in actuality, to help readers avoid reading.’
All this is true for traditional typography — but even more so for experimental typography in a new medium like Virtual Reality. So I was very curious, what my students would do with typography in VR.
Virtual Typography Class
The Virtual Typography Class took place in the winter term 2021/22 at the Interface Design programme at the University of Applied Sciences Potsdam. I gave the class together with Tim Ebert and Sebastian Wloch who are currently running the Interface Lab at our university.
The assignment of the class was to design and prototype a typographic space in Virtual Reality. Based on the assignment, the overall aim was to interpret a text-based information system — like a book, wiki, Twitter feed, research databases, etc. — as an interactive and networked typographic structure in VR.
In October 2021, the Corona incidence was not too bad, so we started off on-site and were even able to have an XR party where the students could experiment with the equipment. This allowed us to establish a concise technological workflow. The VR project itself was created in Unity, using both TextMeshPro and assets from Blender.
Using TextMeshPro, it was pretty straightforward to create typographic structures in Unity. Since TextMeshPro is a component, the properties of the text can be dynamically changed in Unity and in the VR environment. This was great for interactively changing content, layout and appearance of the text.
However, text that is created with TextMeshPro is strictly two-dimensional. If you want to have words and letters with a more physical, three-dimensional presence, you have to create them in a tool like Blender. So we also had an intensive Blender workshop led by Levi Kollwitz. While I always maintained that letters should not be distorted, it was quite intriguing to see how type can be arranged in space and combined with blender modifiers.
In the design phase, we were guided by questions like: How can 2D text be displayed in a 3D environment? What is optimal reading typography in VR? What are the visual limits of a long text? How can spatial structures be used to organise a text? How can you interact with text in VR? How do you deal with perspective and vanishing points? Which ‘traditional’ typographic rules still apply to VR — and which do not? How can links and hierarchies be represented? What can a text network look like in virtual space?
All results from the class can be found here. However, here is a quick overview of the student projects:
Student Projects
Life of Escher VR
By Lewin Brumm, Milan Wulf, Benjamin Kaczynski
This VR project deals with the life of the Dutch artist Maurits Cornelis Escher. Based on his Wikipedia page, the life of Escher is presented in an experimental spatial structure that allows the users to move along different pathways with constantly changing orientation and alignment. The overall structure was inspired by Escher’s drawings of optical illusions and distorted spaces.
→ Watch a screencast of ‘Life of Escher VR’
The Fabric of Interface
By Helene Biebinger, Melvin-Eli Urulu, Leonie Schatter
The virtual installation ‘Fabrics of Interfaces’ is inspired by the second chapter of Stephen Monteiro’s book of the same name from 2017. In this chapter, Monteiro primarily examines the connections between technical terms that were taken from the textile industry and are now also used to describe digital interfaces. For the installation, some terms were selected and their meaning typographically visualised in the VR world. The colourfulness and structure of the room is based on the structure of pixel images and places the textile terms in a digital environment.
→ Watch a screencast of ‘The Fabric of Interface’
Die Traumnovelle VR
By Aaron Schwerdtfeger, Paul Morat, Jannis Hektor
This VR application concerns itself with the theoretical background of Arthur Schnitzler’s Traumnovelle. Initially, the users see an excerpt of the book that describes Albertine’s dream. However, hidden behind Schnitzler’s text are two texts by Sigmund Freund — ‘The Interpretation of Dreams’ and ‘The Unconscious’.
The primary text contains hidden portals that allow the user to enter and explore Freud’s theories. Since Schnitzler’s Traumnovelle is the first literary work to make direct reference to Freud’s theories, the VR application refers to the relation between the conscious and the unconscious.
→ Watch a screencast of ‘Die Traumnovelle VR’
The Sopranos
By Kerstin Humm, Elsa Woelk, Yasemin Yigit
This project examines how the dynamics of TV dialogue can be visualised as a typographic structure in VR. Based on the first episode of the popular TV series ‘The Sopranos’, the dialogue is arranged as a vast typographic space that is supported by a spatial elements. These provide details about locations or the entrances and exits of individual characters.
→ Watch a screencast of ‘The Sopranos’
Walking through Macbeth
By Hyeonji Kim, Diem Ngoc Vu, Anastasia Voloshina
The VR project ‘Walking through Macbeth’ is based on Macbeth, a well-known tragedy by William Shakespeare. Initially, the project team analysed the entire drama and selected relevant text passages for the visualisation. In order to relate the plot, a virtual stage with short quotes and animated objects was designed, just using typography. The user can follow the story of Macbeth and encounter important quotes, characters and even small details from the scenes. Additionally, various sounds, storms and fog effects accompany the user as she or he explores the virtual stage.
→ Watch a screencast of ‘Walking through Macbeth’
Kosmos
By Pascal Struck, Felix Tesche
‘Kosmos’ is an interactive WebXR application that visualises patterns of language in a vast word network. The project demonstrates the complexity of our language and illustrates the connection of individual words. The distance between words describes their relationship to each other. The closer two words are, the more often they are used in the same context.
→ Watch a screencast of ‘Kosmos’
Conclusion
Considering the technical and conceptual challenges, I am very happy with the outcome of the class! All students made an enormous effort to translate a textual structure into a virtual environment. They challenged the way how text is used in 2D and 3D spaces — and they created unique designs for virtual typography.
However, I also feel that we just scratched on the surface of the subject. In the next few years, I expect more use cases for text and typography in virtual reality. So I am very much looking forward to future developments — and I hope we demonstrated that typography in VR can contribute to a vision of the Metaverse that goes beyond simulation.
