Generative AI and the correlation of text and image in the intersemiotic translation of literary works

This paper investigates the intersection of generative AI, intersemiotic translation and collaborative translation, focusing on the creation of an audiovisual version of Virgil’s Aeneid through human-AI collaboration. The video project, which employs text-to-image (T2I) and image-to-video (I2V) generative models to reinterpret The Aeneid, highlights the significance of textual prompts in shaping AI outputs. By examining the framework for the collaboration between the researchers and AI tools, the paper elucidates the challenges and opportunities inherent in integrating AI into creative practices. It emphasises that the adaptation process necessitates a careful translation of literary language into prompts that AI can interpret and addresses the limitations of current generative technologies. The findings indicate that AI can facilitate innovative adaptations, but achieving fidelity to the source material requires a nuanced understanding of both the literary text and the capabilities of generative AI. This paper contributes to the emergent discourse on AI in Translation Studies and computational creativity, advocating for a multidisciplinary approach that draws from adaptation studies, generative culture and prompt engineering. The proposed collaborative model, which can serve as a framework for future explorations of AI in storytelling, underscores the essential role of human oversight and creative input in the generative process. Funded by the European Union – Next Generation EU, Mission 4 Component 1 CUP H46E22000000006.