Gesture and Form

This project unites researchers in architectural design and technology, human computer interaction (HCI), cultural anthropology and material culture studies, and constructed textiles to investigate the integration of Augmented Reality (AR) in handcraft and educational design. The study adopts a two-pronged approach: first, developing and evaluating novel AR techniques to capture, encode, and represent gesture in handcraft; second, exploring new forms of communication through AR and how these enable, foreclose, or disrupt disciplinary roles, intergenerational communication, and forms of collaboration. 

Site

Explorations occur in two distinct sites linked by a modality of using coordinated gestures with textiles to build physical structures. The first site is an academic and non-profit collaboration with South Sudanese refugees documenting and sharing material heritage in the form of beaded accessories and the skills required for their fabrication. The second is an educational architectural design studio where novice architects use chord, rope, and fibers to make free-standing structures informed by beadwork stitches. In each context participants use AR headsets to encode and guide sequential, step-by-step actions, creating a finished bead-work piece or building element. In the South Sudanese refugee community, the AR application expands existing forms of heritage documentation, recording gestures to initiate a process of encoding hand, tool, and thread positions and their sequencing. In the architectural studio researchers learn from this embodied practice and explore communicating through sequentially stepped three dimensional instructions for building. 

Contributions

The investigation’s focus lies in attempting to bridge these two contexts and seeing how they speak to one another. Attached to this is the risk that the existing state of AR technology will limit the nuance of what is captured and transmitted, and disrupt organic person-to-person interactions. In return, the investigation contributes a sociotechnical method of AR mobilization. It supports marginalized or at-risk craft knowledge practices and innovative design training; it is disseminated in forums of digital humanities, computational design and construction, smart textiles, and tangible, embedded and embodied interactions; and it is mobilized in professional and non-profit communities of mixed reality application developers, cultural preservation, architecture and textile design, and technology education.