Computer-Assisted Evaluation of Story-Driven Interactive Storytelling Systems

The evaluation of the narrative emerging from storytelling systems is not a trivial task, not even for linear ones. For interactive stories, the space of possible developments may grow very quickly and become easily unmanageable. The story designer has hardly any assistance at all in this process. We propose, expanding on previous attempts, a general methodology to evaluate story-driven interactive storytelling systems via clustering, tension curve extraction and user surveys. This procedure outputs a set of clusters, each with its own specific tension curve shape and average quality score. The story designer may inspect the resulting clustering and iterate over his/her storytelling system using the new knowledge acquired. This may also lead to an association between tension curves and quality of a story. We apply this methodology to our story-driven interactive storytelling system to show its current and potential achievements, respectively. Our results indicate that clusters, even if not well-formed, display different quality scores and that some tension curves are associated with better stories. While the method has proven to be valid, there is room for improvements.