Storyteller is now a sprawling ecosystem of software, with a full stack web application, native apps for Android and iOS, KOReader plugins, and upcoming macOS, watchOS, and tvOS apps. We're alpha and beta testing v3 releases of the above, which will bring updated UIs, a huge suite of new library management features, and more. We're really excited to show you what we've been working on! But at the core of Storyteller is its alignment algorithm.

Storyteller can take an ebook and an audiobook that you provide it and align them, finding where each word of the ebook is spoken in the audiobook. It does this automatically, without any input from you. Then it uses the EPUB specification's built-in audio synchronization system, called Media Overlays, to embed the audio and synchronization info into the EPUB.

That lets you read your book immersively, with your reader app highlighting each sentence as it's narrated by the narrator, just like in the demo above (which is using real Storyteller alignments!). Initially, that's all Storyteller was: one Python script. It took one audiobook file and one ebook file, and it output a new ebook file with audio synchronization metadata.

At the time, there were a very small number of ereader apps (and zero ereader devices) that could consume these files, which used the EPUB Media Overlay specification for their functionality. I would run the script on my computer, copy the resulting EPUB to my phone, and then use Book Fusion's fledgling Media Overlay support to read and listen to my books.