AIsop: Exploring Immersive VR Storytelling Leveraging Generative AI

Published March 16, 2024 in IEEE VR '24: 2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW) by Elia Gatti, Daniele Giunchi, Nels Numan, Anthony Steed

Abstract

We introduce AIsop, a system that autonomously generates VR storytelling experiences using generative artificial intelligence (AI). AIsop crafts unique stories by leveraging state-of-the-art Large Language Models (LLMs) and employs Text-To-Speech (TTS) technology for narration. Further enriching the experience, a visual representation of the narrative is produced through a pipeline that pairs LLM-generated prompts with diffusion models, rendering visuals for clusters of sentences in the story. Our evaluation encompasses two distinct use cases: the narration of pre-existing content and the generation of entirely new narratives. AIsop highlights the myriad research prospects spanning its technical architecture and user engagement.

Full-text PDF Publisher Page

Cite with BibTex

@inproceedings{gattiAIsop2024,
  title = {{AIsop}}: {{Exploring Immersive VR Storytelling Leveraging Generative AI}
  booktitle = {2024 {{IEEE Conference}} on {{Virtual Reality}} and {{3D User Interfaces Abstracts}} and {{Workshops}} ({{VRW}})},
  author = {Gatti, Elia and Giunchi, Daniele and Numan, Nels and Steed, Anthony},
  publisher = {IEEE},
  address = {Orlando, USA},
  doi = {10.1109/VRW62533.2024.00229}
}