Using Aero to make mobile AR learning experiences
A popular catchphrase for the moment involves the “metaverse,” which is conceptualized in part as an overlay of digital elements over the physical world (with aspects of both visual simultaneously).
A small aspect of a “metaverse” may be built for mobile augmented reality (AR) learning, using a popular application that enables end-to-end development-to-deployment. Mobile AR refers to the use of a mobile device (think smartphone, an iPad, or other device that can run the Aero app) to experience the augmented reality without the need for a head-mounted displayed (HMD) or smart glasses. This app is Adobe Aero, which is a tool that may be activated as part of the Adobe Creative Cloud Suite.
AR is not just about digital visuals overlaid over physical space. Oftentimes, AR includes sound effects. It can include interactivity. AR is about embodied experiences and experiential learning. The capabilities of the technologies are advancing, with advances in wireless and cell connectivity enabling ever more enriched communications.
This short article introduces some basic features of Aero, which was released in November 2019.
To introduce the tool, a few short videos have been made to show some of the visual qualities of the digital characters and objects. These videos are made with pre-builts inside the Aero app. The objects are three-dimensional (x, y, and z axes), and they are built using sparse polygons (given the connectivity needs to show the animation).
Some Light Animations using Aero Built-ins
Imported Videos
Video: Lineup
Video: Twirling AR Robot Next to a Laptop
Links to Videos as “Shorts” on YouTube
Video: Kid Grabbing for a Balloon (with sound)
https://youtube.com/shorts/IBnVQZ3nt0M
Video: Hummingbird
https://www.youtube.com/shorts/8JBGNIoaEKk
A Light Overview
For those who are familiar with Adobe Creative Cloud applications, Adobe Aero offers a clear basic interface, with a navigation bar at the top, a left toolbar, a Behavior Builder at the bottom left, a large workspace in the center, and a properties panel to the right. (Figure 1)
Figure 1: Adobe Aero Graphical User Interface (GUI)
As part of the app, users have access to Adobe Aero in the cloud, which hosts the respective published and unpublished Aero files. Each published file comes with a URL and a QR code, which spawns the AR experience. Aero files are .REAL file types. (Figure 2)
Figure 2: Adobe Aero in the Cloud
The Work Pipeline
In terms of a work pipeline, a designer/developer may conceptualize the teaching and learning experience that they want to deploy. They create the artworks, digital puppets, motions, animations, sound, and other elements in other software tools. Then, they import the digital objects into Aero, place them in 3d space, add animations, test, revise, and then deploy.
Aero takes still images [.png (raster), .svg (vector), .psd (Photoshop document, for editable layers to enable objects to be “set back” in visual space), .jpg / .jpeg (raster, lightweight, lossy/nonlossy),.tif / .tiff, .gif (animated .gif for animated sequences), and others. 3d objects may be imported as .obj (3d geometries), .glb (“3D scenes, models, lighting, materials, node hierarchy and animations”) (“What is a GLB file?..., Visao) / GL Transmission Format Binary (GLB) (standardized format), and .fbx (3d model file) (“Filmbox” format by Autodesk) (proprietary format). Animated characters may be exported as .glb or .fdx files, with animations already applied, animations imported separately, scripted animations to apply to existing characters in Aero, or animations to apply to character “skeletons”. Collision detection may be set to emulate in-world physics.
Various tools in the Adobe Creative Cloud suite—Illustrator, Photoshop, Dimensions—enable export to Aero, but many of the 3d visuals are actually 2d (flat plane) with the illusion of 3d. 3d objects may be created in Unity or other game engines and other software programs, with sparse polygons that may be used in Aero.
The AR experiences may be built on Aero running on a desktop machine and then displayed on an iPad, or they may be fully built and deployed and experienced on an iPad, at present.
AR for Teaching and Learning
The common “value-added” of AR for learning involves the ability to depict scenarios from the imagination, from extreme scales (nanoscale, galaxy scale), and the ability to simulate the world.
The mechanics of building an AR experience on Aero is straightforward, but structuring the AR experience for effective learning is more challenging. The visuals that may be used are limited, so they have to be purposefully selected. They have to be fairly accurately representational even in the absence of precise details. Interfaces of the AR have to be intuitive. The entire experience has to be accessible both virtually and physically. This means that information needs to be conveyed in multi-channel ways, and proper labeling is important.
It helps to begin with defined learning objectives and then building the AR experience to that objective or those objectives. Animations need to be informative (and occasionally, atmospheric or mood-setting). Sound should be informative (or occasionally mood-setting). In terms of interactions, users should be clear what may be interacted with (by proximity, by button pressing, and other actions). There should be clear digital signage if the AR experience is in an extended interior space, such as an art gallery or museum space. It helps to define opening scenes and exit scenes for learners. Such AR experiences are generally thought to be effective if they are brief and well-designed to avoid cybersickness or nausea (from the flicker and other aspects of the AR visual depictions).
Much more depth is included in some handouts created for those who would design immersive learning experiences using Adobe Aero.
URLs for Updated Adobe Aero Presentation Handout (in .docx and .pdf formats)
Word:
URL: http://www.k-state.edu/ID/IntroAdobeAeroHandout.docx
PDF:
URL: http://www.k-state.edu/ID/IntroAdobeAeroHandout.pdf
The cost of creation of AR experiences will vary, depending on the complexity of the teaching and learning. The research literature suggests mixed results in terms of AR efficacy for teaching and learning. There are many context-sensitive aspects that may determine effectiveness of the teaching and learning.
Most do agree that learning through AR involves higher cognitive load (more demands on attention and working memory).
Future-proofing the Digital Resources
As AR advances, which file types are mainstream, and which are more proprietary and closed will become clear. At present, there does not seem to be ways to “future-proof” .REAL files, which are inextricably linked to Aero. There may be some ways to transcode the file eventually to other file types that may “play well” with other systems.
Conclusion
Augmented reality (AR) may seem elusive in many instructional design shops, but it is not. Aero may serve as a bridge to AR as a proof-of-concept. It may serve as a bridge to more complex design, development, and deployment tools.
AR may be a portal to the metaverse.
About the Author
Shalin Hai-Jew works as an instructional designer at Kansas State University. Her email is shalin@ksu.edu.
