Rhizomes: Cultural Studies in Emerging Knowledge: Issue 37 (2021)

Intersections of New Media and Narratives: The Enhanced Ebook Reading Experience

Alejandra Ortega

Abstract: Since 2012, creators have advanced the technology used to make enhanced ebooks to include touch interactivity alongside animation and sound. The use of new media not only leads to the brain responding differently to the act of reading, but the physical engagement aspects leads to an individualized reading experience. This essay examines experimental author Mark Z. Danielewski’s The Fifty Year Sword and new media art collective Tender Claws’s Pry to show how enhanced ebooks critique the linear storytelling process and change the way our brain reads texts.

The move from reading novels by holding a printed text in our hands to reading digital texts on an electronic device has been an expected transition with the development of computer technology. As George P. Landow once wrote on the future of narratives: “Electronic processing marks the next major shift in information technology after the development of the printed book. It promises (or threatens) to produce effects on our culture, particularly on our literature, education, criticism, and scholarship, just as radical as those produced by Gutenberg’s movable type” (19). Therefore, the transition from printed texts to digital ebooks on Amazon’s Kindle or Apple’s iPad is understandably a natural next step in this progression. The original intent of this process was to retain the expected design of a print book and transform it statically into a digital format. Concurrently, writers have sought to use digital technology to its full potential to create new reading experiences. Most of this work is being done by authors of experimental literature—often referred to as “ergodic literature.” Espen Aarseth coined the term “ergodic literature” by drawing on the “Greek words ergon and hodos, meaning ‘work’ and ‘path.’ In ergodic literature, nontrival effort is required to allow the reader to traverse the text” (1). Experimental writers saw the potential in using technology to create new forms of electronic literature. In the early forms of what have since become known as “enhanced ebooks,” animation and sound were incorporated into the reading experience. This drive to make use of digital media in literature is rooted within a long history of erogidc narrative form. One famous early example of electronic literature that intersects ergodic narrative and digital media is hypertext fiction, such as Twelve Blue (1996) by Michael Joyce, where readers interacted with the text by clicking on embedded links. Shelley Jackson’s Patchwork Girl (1995), a hypertext fiction readers could purchase on a 3.5-inch floppy disk and read on their home computer, could be viewed as a precursor to our present-day enhanced ebooks. Where texts by Joyce and Jackson required readers to use their mouses to click on links or images, enhanced ebooks require readers to haptically engage with animation, digital pages, videos, and sounds. Mark Z. Danielewski’s The Fifty Year Sword (2012) is one of the earliest examples of the new enhanced ebook form. Because the novella was first written for the print form, the enhanced ebook notably retains much of the traditional narrative structure expected of a text but uses the digital elements of the enhanced ebook medium to further draw the reader into the story. Following this shift in storytelling, new media art collective Tender Claws created the novella, Pry (2014). Tender Claws developed the enhanced ebook technology further to include videos, spoken dialogue, and interactive touch with images and text. To enable the enhancements and progress the story, the reader must physically engage with the digital content. Rather than simply turn a page, readers must turn on animation, listen to sounds, interact with characters, or move text to unlock the entire narrative. The physical interaction with these enhancements employs multiple areas of the brain, creating new forms of cognitive immersion for readers that lead to personalized reading experiences.

Presently, the term “enhanced ebook” refers to any application of electronic literature that has been enhanced digitally to facilitate an interactive reading experience. In early manifestations of enhanced ebooks, the term “enhanced” solely referred to the incorporation of animation and sound. However, enhanced ebooks sold through Apple now have the capability to include videos, pre-recorded spoken dialogue, composed music or sounds, touch interactivity (with both text and images), animated images, and archived materials. The incorporation of these digital elements creates a unique multimedia reading experience.

Enhanced ebooks can be read through basic ereader applications, such as iBooks (on any ereader or tablet device) or can be accessed as downloadable standalone apps. As of this writing, The Fifty Year Sword, Pry, and other enhanced ebooks are limited to Apple devices due to the company’s unique widgets that assist creators in constructing these texts. Yet, in addition to the cost of developing specialized programming, some publishers are concerned about the cost of investing in a technical director to assist in transitioning books from the plain text format to enhanced ebook format. As Lisa McCloy-Kelley, head editor of the digital production group at Random House, commented, “we’re finding that the effort behind these types of books is a magnitude of somewhere between seven and fifteen times as much effort as a typical illustrated ebook” (qtd. in O’Connor). David Kudler further explains that there are questions of managing the cost of transporting the size of enhanced ebooks to devices. He notes that his draft of an enhanced ebook contained video clips that weighed over 200MB, making it difficult for even Apple to sell. Yet some companies have made progress with streamlining production of these texts. Recently this technology has been expanded for enhanced ebook versions of textbooks, nonfiction, and children storybooks. The sales of these texts have proven to justify the costs of creating and designing enhanced ebooks since there is clearly a captive audience for these texts. For example, incorporation of archival material can be seen in biographies such as that of Jacqueline Kennedy: Historic Conversations on Life with John F. Kennedy (2011), published by Hyperion. The enhanced ebook version of the biography includes transcripts of eight and a half hours of interviews, archival photographs, and original contributions from Caroline Kennedy and Michael Beschloss (the presidential biographer) on the background of the text (Osnos). Therefore, the inclusion of this extra audio and visual material is something that nonfiction readers would be willing to spend money on in comparison to receiving only the plain text version of the biography that does not include this material. The extra material, in the eyes of the reader, makes way for a fuller reading experience. While the debate over whether enhanced ebooks are a passing fad or an art medium to stay continues, it is clear that many publishers and authors are interested in working toward finding new ways to incorporate enhanced programming within their texts to develop new reading experiences. Ultimately, we can presume that while the shift in creating texts with enhanced elements will be a slow transition, it will significantly influence both the storytelling process and the reading experience. 

Interest in understanding the cognitive response to reading has been taken up by many scholars, such as Maryanne Wolf and Stamislas Dehaene. Similarly, at Michigan State University, Natalie M. Phillips and her research team have advanced our understanding of what they refer to as the “reading brain” in their Digital Humanities and Literary Cognition lab. By having subjects read eighteenth-century texts and Jane Austen novels while undergoing an MRI, they examine the difference between close reading and reading for pleasure to understand how the brain reacts to different modes of reading. Additionally, scholars such as N. Katherine Hayles have researched the cognitive effects of screen reading. She has noted that the increase in digital reading not only requires different reading strategies but can also change the architecture of the brain (Hayles, How We Think 62). Furthermore, Marie-Laure Ryan has addressed the value of considering a reader’s immersion within postmodern texts, arguing that this immersion is important because it helps create an emotional connection between readers and the characters within the narratives (170-71). This examination of immersion is an important aspect of the reading process for both print and digital texts. Yet despite these interests in reading and cognition, scholarship on the cognitive response to reading enhanced ebooks is limited. Therefore, I make a case in this essay for the necessity of examining the distinct reading experience of enhanced ebooks. After a brief look at how our brain reads texts, I discuss the design of two enhanced ebooks to consider how this new technology contributes to the way creators construct electronic experimental narratives. Through my close readings of these narratives, I address how reading these experimental texts on ereaders affects the reader’s cognitive experience. Ultimately, by analyzing two of the first enhanced ebooks on the market, I argue that this new form of electronic literature creates a unique cognitive immersion that is significantly different from a traditional text. Not only does the brain respond differently to the act of reading an enhanced ebook, but the physical engagement aspects of the medium results in an individualized experience.

Reading over the course of a human’s development affects brain patterns. Different types of reading create different brain patterns. Wolf explains that the process of reading rearranges the organization of the brain through groups of neurons, creating new connections and pathways among themselves. This process of rearrangement occurs whenever we learn a new skill (5). To address this, Dehaene has famously presented his “neural recycling” hypothesis, “which suggests that reading repurposes existing brain circuits that evolved independently of reading” (qtd. in Hayles, Electronic Lit 65). He points to the brain’s “letterbox” (located in the left occipto-temporal region) as a crucial place this rearrangement occurs. He explains that the reading process starts when our eyes physically scan the page of a book through a series of movements of small steps that both eyes take in the same direction, called saccades. Furthermore, we only identify ten or twelve letters per saccade (Dehaene 16). To recognize a word, our brain divides the visual labor into different categories, processed by a patch of our cerebral cortex. Specifically, the angular gyrus and the supramarginal gyrus regions of the brain are involved in recognizing and comprehending letter shapes that form words. The brain channels the process through two different neural routes. First, the phonological route (through the temporal lobes) works to decode strings of letters and translate them into a sound pattern. This sound pattern will then allow us to access the meaning of the word stored within our brain (Gabrieli et al. 115). Second, the direct route can bypass the sound pattern stage to instead match the word directly to a pre-understood meaning. This second route is used when the reader scans a frequently encountered word (also known as a sight word) (Gabrieli et al. 115). The Wernicke’s area of the brain, which is one of two parts linked to the cerebral cortex, is the area that comprehends or understands written and spoken language. However, making meaning of a literary text is more than just understanding individual words. In the early stages of their research at the Michigan State University Digital Humanities and Literary Cognition Lab, Phillips et al. note that when readers engage in different forms of reading, each reading style requires its own cognitive demands and creates distinct patterns in the brain (319). The brain activity that engages with both storytelling and listening are activated across multiple regions in order to perform these different reading styles (320). These brain patterns further reveal differences in reading and attention that are unique to individual readers. Accordingly, these scholarly discussions on our reading brain allow us to understand how brain patterns can adapt from reading on a printed page to reading on a screen.

Scholars have been debating about the cognitive effects of screen reading since Ladow’s enthusiastic support of the transition from print to digital texts in the 1990s. Although research makes it clear that reading a print text rewires the brain, scholars have questioned how screen reading restructures this learned reading process. Specifically, many have been concerned about how comprehension and retention is affected during screen reading. Nicholas Carr argues that the concern for comprehension and retention lies in the connection between working memory and long-term memory. As Hayles has explained, “For retention of more complex matters, the contents of working memory must be transferred to long-term memory, preferably with repetitions to facilitate the integration of the new material with existing knowledge schemas” (Electronic Lit, 64). Yet more recent research proves that comprehension in digital reading has mixed results for individuals, depending on the form of the digital reading taking place. PhD candidate Lauren M. Singer and Dr. Patricia A. Alexander point to specific design elements of screen reading that may affect visual legibility of digital texts (and, in turn, comprehension and retention), such as text size, screen resolution, backlighting, and luminance contrast (157). Furthermore, sequential versus continuous reading may also affect the reading process. Singer and Alexander write that, “frequent scrolling increases the cognitive demands on readers and, thus, may negatively affect recall” (157). Additionally, while some scholars, such as Jane M. Healy and Sven Birkerts, have argued that electronic reading is threatening sustained reading, others, such as Ziming Liu, have concluded that, over the years, readers have developed screen-based reading behavior (such as “more time on browsing and scanning, keyword spotting, one-time reading, non-linear reading, and more reading selectively” [705]) that helps with sustained reading. Because enhanced ebooks address these concerns in their design, the texts can encourage reader engagement and facilitate immersion within the narratives.

Many enhanced ebooks, by design, are structured to address concerns of comprehension and develop immersion through interactivity. Through this, readers develop a sense of agency that allows for an individualized reading experience. Carolyn Handler Miller encourages us to think of the interactivity in enhanced ebooks as a “conversation between the user and the content” (54). Through the established haptic directions provided to the reader at the start of an enhanced ebooks, the repetition of movement retains readers’ attention and allows them to develop their own engram. Furthermore, as Miller shows, readers are in complete control of what they see, when they see it, and the overall pathways of interaction. She argues that “essentially, agency gives the user the ability to make choices and to see and enjoy the results of those choices” (56). In early forms of enhanced ebooks, readers activate the enhanced elements through timers when they turn a digital page. Sara and Russ Weinzimmer, the inventors of the enhanced ebook, explain in their patent that this turn of the page begins a timer. Subsequent timers can adjust based on a person’s reading speed. This timed reading speed is a critical aspect of the digital reading process. If timed too fast, the programmed enhancements could disrupt the reading process through early activation. To address this, the enhanced ebook version of The Fifty Year Sword notifies the reader at the start of the book that while a turn of the page activates any timed enhancements, turning back a page reactivates the sequences, effectively restarting the page for the reader. In Pry, however, the reader is given the options to “restart” a whole chapter sequence. The reader, unfortunately, cannot go back to the start of the sequence the way they can with The Fifty Year Sword. Rather, small red diamonds are displayed underneath the chapter titles to indicate to the reader how much of the chapter was accessed or “unlocked” in the reading process. The display of these diamonds encourages the reader to revisit chapters and take new haptic paths to access more of the narrative. As these two different methods show, the capability of the enhanced ebook to learn a reader’s speed is crucial because it allows the technology to create an individualized experience. As the technology learns more about its readers, it can assist readers as they interact with the text, as well as affect how much information the reader receives. This technology leads to the individualized immersive reading experience that in turn leads to individualized interpretations of the overall narrative.

Danielewski’s The Fifty Year Sword takes place on one night in October at a foster home in East Texas. Chintana, a seamstress, was invited by the 112-year-old Mose Dettledown to attend the fiftieth birthday party of Belinda Kite. Chintana spends her time chaperoning five orphan children in attendance, who are being entertained by a visiting Storyteller. The Storyteller tells the children of his journey finding the titular Fifty Year sword. The sword is a weapon that never fails to cut but shows no wound until the victim’s fiftieth year of life. At one point, the Storyteller stops abruptly to allow the children to open the box that contains the sword. He picks up the handle of the sword and swings it at the five children’s necks, presumably marking them for death upon their fiftieth birthdays. Belinda interrupts the Storyteller and tries to calm the now frightened children by declaring the sword is fake. She makes a show of picking up the sword and cutting at herself multiple times. Because no damage is shown on her body, she believes she has disproved the Storyteller and his sword. However, at the stroke of midnight Belinda Kite turns fifty. The cuts she made on her body appear as if by magic, leading her to bleed out and fall apart. Despite Chintana’s efforts to help, Belinda dies.

Because The Fifty Year Sword was published as a print novella first, the transition from the material book to the enhanced ebook form gives the illusion of a solid yet movable piece. The novella was originally published in a limited market in Holland in 2005. While preparing to publish it in English in 2012, Danielewski chose to package it in an unconventional way. The book was made with aged cardstock, hand-stitched illustrations, Nepalese binding, and then placed in a sturdy slipcase that had five metal latches, one for each orphan in the story. The enhanced ebook has transformed the hand-stitched illustrations into animations. These embroidered images are embedded on the digital page to give the illusion of a three-dimensional thread despite the flat surface of the ereader. Thin lines of stray floss, as well as tiny pinholes where the embroidery floss was threaded through the page, are replicated within the animations. Structurally, most recto pages of the novella are completely blank, and text is only on the verso pages. This text primarily consists of brief lines of dialogue presented in a structure that resembles poetry. As Glyn White wrote when examining the print version of the text: “Though there is clearly room to say much more, the short lines, often imparting only a single word, show the text stretched thin” (112). Twelve recto pages display illustrations that visually represent the Storyteller’s story that either stand alone or bleed into the verso page to create a full landscape scene. The use of blank space in the print version proves to be strategic when the story is transitioned to the enhanced ebook version because screen reading is often performed in an “F” pattern. Hayles explains that “a person reads the first two or three lines across the page, but as the eye travels down the screen, the scanned length gets smaller, and by the time the bottom of the page is reached, the eye is travelling in a vertical line aligned with the left margin” (Electronic Lit 61). It is unclear if the placement of The Fifty Year Sword text on only the verso pages in a vertical poetry-like structure was pre-planned in anticipation of screen reading. However, planned or not, the use of blank space on verso pages allows the reader to stay focused in one specific location of the screen, thus encouraging comprehension and retention of the digital text. Artistically, the placement of text about the page allows for a play with negative space. Words fall off the page, disappear, or spontaneously appear as the reader engages with the enhanced ebook. These effects compliment the narrative.

The design of the enhanced ebook version of The Fifty Year Sword builds on the traditional linear reading experience by asking readers to both read text and engage with accented moving animations. The design of The Fifty Year Sword lends itself to continuous reading with the tap of a finger, allowing for limited physical demands. Yet while interruptions may occur with the onset of movable text, the interruptions are performed to positively engage the reader. These animative elements make use of the visual areas of the occipital lobes comparative to the stationary words typed on a page in order to process and make meaning of the text movement. For example, in the Storyteller’s description of his travels, text both appears and disappears. An elaborate version of this can be seen when he describes how in the Valley of Salt, “Only shadows remained” (90). Passages are blurred out, looking almost shadow-like on the screen, until the activated timer allows them to become readable (see Figure 1). After a moment, the text becomes blurry again, signaling to the reader that things are moving in and out of focus for the Storyteller as he travels through the valley. As the story builds, the movement of the text becomes more complex. When the Storyteller describes the Forest of Falling Notes, he states, “For you see in the peculiar Forest of / Falling / Notes, sounds could not hold / together. / Like pearls on a snipped / silk / thread, they / scattered wildly upon / the / ground” (108). Half of the text on this page is not safe (see Figure 2). The letters separate from each other and crash down over the page like a waterfall. The first set of letters that descend hit the page number on the bottom of the screen, causing it to flip over on itself and fall with the remaining letters.

In the process of making the enhanced ebook version of The Fifty Year Sword, Danielewski commented, “I’m concerned with image as well as text. …My vocation has always encountered this bifurcation between image and text” (Book Business Magazine). Danielewski has a long history of playing with images and text. His most famous novel, House of Leaves (2000), for example, not only makes use of color-coded words and themes (the word “house” is always in blue while all discussions of the Minotaur are in red), but also constructs images out of the written text (such as windows, keys, and trees). In the enhanced ebook version of The Fifty Year Sword this “bifurcation” becomes literal in its use of animation. Throughout the story, the text struggles to remain stable on the page. These moments provide a twofold experience for the reader. First, the animation visually conveys how the terrain behaved for the Storyteller as he traveled. The breaks in the poetry-like structure on the pages, utilizing a significant amount of blank open space, allow for a strategic disruption in both the usual flow of a story and that of the graphic surface. Second, the movement critiques the process of linear storytelling and narration. The Storyteller’s elaborate descriptions harken back to the process of telling ghost stories. The children and Chintana are sitting together in a dimly lit room, relying on candles and the Storyteller’s voice to guide them through the narrative. Whether or not these places exist does not matter. Rather, what does matter is that they exist in listeners’ minds. Therefore, the moving text programmed in the enhanced ebook further pulls the reader into the story, creating a distinct narrative environment, through the digital break between words and images. By combining the moving visual cues of the text, the areas of the limbic system associated with emotional response are heightened in a way a reader might not experience through reading static words alone. The process of storytelling breaks the narrative down to the essential basics of word and sentence creation, furthering a sense of anxiety in the inability of the text to hold together the narrative in a fixed state, both in time and for the reader.

In addition to the visual, sound is embedded within the narrative and timed equally with each digital page turn. The sounds are added to further connect the reader to the in-universe environment of the narrative. They grant the Storyteller a sense of power over his audience and perpetuate the sense of death shadowing the narrative. The use of sounds draws on classical conditioning, a learning procedure that helps us understand how humans learned to pair, for example, sounds with feelings. Social cognitive theory, however, pushes back on the simplicity of classical conditioning to address these individualized responses. Social Psychology theorists have argued that individual response is created through the intersection of personal experience and both external and internal stimuli. This is possible thanks to “scripts” (“grouped pre-interpreted actions”) that are stored in a person’s brain (Kafalenos 278). These scripts allow a person to cognitively respond to sounds by recalling stored memories associated with pre-made interpretations of chronological and causal relations (Kafalenos 278). In an interview, composer Christopher O’Riley describes how, when he was brought on to the project of adapting The Fifty Year Sword for the enhanced ebook form, he analyzed the characters and assigned personalized sounds and melodies to each of them. He explains that the personalization can help convey to the reader who the characters are in different moments of the text (O’Riley). The composer, therefore, is drawing on classical conditioning by integrating commonly associated feelings through distinct sounds for each character, thus allowing readers to tap into these learned responses. In using sound stimuli for each character, O’Riley seeks to guide readers toward a specific type of emotional response. However, as social cognitive theory shows us, composers such as O’Riley may desire to suggest a type of emotional response, but since the reader has a sense of agency (both in terms of their own learned responses to sound stimuli and how they are choosing to physically engage with the enhanced ebook), readers’ emotional responses to the sounds embedded within the text are completely their own. Readers’ agency not only allows them to generate an individualized emotional response, but also leads to an individual interpretive response, one that may not have been expected by the author.

In addition to composed pieces, O’Riley incorporated everyday sounds. These sound effects include, but are not limited to, an old door creaking open, strong gusts of wind, as well as a thud that echoes through a chamber as each latch of the box containing the sword is opened. These sound effects are used to further engage the listener’s cognitive response to the text. O’Riley chose to make use of natural sounds (e.g. the wind) and characteristic sounds (e.g. the creaking of the door being opened). This can be seen, for example, in the first appearance and illustration of the Storyteller (see Figure 3). The illustration grows with the increasing sound of echoes in a metal chamber, like ink being spilled, breaking into the verso page to cut through the narrative. The text on the verso page is animated so it gradually appears: “‘The orphans,’/ was all he said. / And Chintana showed him the way” (66). This blend of the intense echoes with the moving text can provide the reader with an ominous sense of dread that becomes associated with the Storyteller as a character. Without these effects, this emotional response may not fully emerge while reading the narrative. The inclusion of these sounds and compositions plays with the possibilities of technological engagements with text.

Danielewski’s The Fifty Year Sword was among the first to demonstrate how using the technology of enhanced ebooks can dramatically affect the process of storytelling. New media art collective Tender Claws, founded by Danny Cannizzaro and Samantha Gorman in 2014, has taken this technology further to create a more physically interactive experience. The original design by the Weinzimmers had an eye to the potential future of enhanced ebooks. In their patent, they gestured to the possibility of including videos and touch engagement. However, the data and device memory space required for this has resulted in enhanced ebooks having to be sold as a standalone app. Although not an app in the traditional sense of the word, enhanced ebooks can take advantage of the extra data space an app offers in order to provide their audience with a fuller experience. One such enhanced ebook released in November 2014 is Tender Claws’s Pry. In order to create the experience, Tender Claws had to push the Weinzimmers’s original program further to design custom gestures, multi-touch experiences, and a new audiovisual component. The inclusion of the tactile sensation stimulates the somatosensory regions of the brain’s parietal lobes, creating another level of engagement with a text. Pry incorporates storytelling aspects usually expected in video games and film to create what one reviewer termed as a “hybrid literary creation” (Jhave). The novella contains seven chapters, a prologue, an epilogue, and an appendix that includes a photo album. Each component has distinctly different interaction requirements. Some chapters ask you to watch a video while others require physical touch to activate texts, sounds, or animations. The bulk of the information provided to the reader is through non-linear text-based reading. The inclusion of images, film, sound, and digital interaction makes Pry one of the first interactive fictions to push the concept of enhanced ebooks to its technological limits.

Pry’s story focuses on James, a demolition consultant, who has returned from the 1991 Gulf War. He has vision impairment from wounds he sustained during his service and struggles with making peace with his time in the war. The enhanced ebook enables the reader to explore James’s mind through both his conscious experiences in the real world of the narrative and his subconscious thoughts and memories of his past. The movement in and out of James’s mind provides the reader access to his interiority. The app rewards a reader who is willing to explore each chapter multiple times. Unlike The Fifty Year Sword’s simpler design, no single readthrough of Pry allows for complete access to the narrative.

 To start, Pry’s prologue consists of a short video of James packing to deploy to the Gulf War. As he prepares to leave, he is interrupted by childhood memories. These images pull the reader into James’s mind as he considers the course of his life that leads him to enlist in the military, as well as provide some subtle backstory. These images are followed by a few scenes of James in the war. Images pile on top of each other in a tense montage, until the screen completely cuts to black, leading seamlessly to chapter one, where James is back from the war and lying in bed. The first request for touch engagement comes with a picture of two fingers that advises: “Spread and hold open to see through James’ eyes” (Pry). The process allows the reader to physically open James’s eyes (see Figure 4). After a stain on the ceiling above James becomes visually clear, the enhanced ebook then asks for the reverse of the reader: “Pinch and hold closed to enter James’ subconscious” (Pry). This results in the eyes closing to darkness, and then a sudden cascade of words rapidly pours over the screen to show what James is thinking in that moment. The words are a tangled web of phrases. The tighter the reader pinches, the faster and larger these words grow on the screen. This process, termed serial-flashing, is created from “a cognitive science technique for measuring attentional blink called RSVP: rapid serial visual presentation. …on first impact [the technique is] addictive, precipitating a strong desire to keep up” (Jhave). This “addictive” desire to keep up not only draws on the reader’s need to understand the narrative but also on the reader’s need to receive as much information as possible in order to read and interact with the text. Thus, from the start of the novella, the reader has an invested interest in engaging with the text in order to fully receive and interpret the narrative.  

After these two notes provided by Tender Claws, it is up to the reader to either open or close James’s eyes throughout the remainder of the book. All other physical interactions with the text must be discovered by the reader through exploration. This incorporation of haptic exploration provides readers with a sense of agency that is more interactive than the agency they have when reading The Fifty Year Sword. This agency allows readers to make their own narrative timeline and change points of view as they navigate the story. Through the multisensory interaction, readers may explore multiple aspects of the narrative. In Pry, the reader’s choices result in the appearance of different paragraphs, images, or videos to engage with either outside, through James’s eyes, or within James’s subconscious. The blending of these different visual mediums leads the text to reflect what Mark B. N. Hansen has referred to as “mixed reality” (2). In adapting the term coined by Monika Fleishmann and Wolfgang Strauss, Hansen is referring to an experience “by a technical dimension” that has “occurred as a cofunctioning of embodiment with technics” (9). For Pry, the use of mixed reality results in a blend of digital reproductions of real environments with the written word. Together, the two forms offer the reader another level of immersive reading. While Pry also uses sounds and animations as stimuli to guide the reader through emotional responses, the new additions of video, touch, and control of the narrative timeline draws readers further into the narrative, replicating James’s emotions and experiences of visual impairment and PTSD. Over the course of the novella, the reader is encouraged to virtually experience life through James. If the reader lets go of the screen during a sequence, whatever information remained is lost to readers unless they restart the chapter over again. This engagement of either prying open or pinching closed remains consistent throughout the following chapters, with only minimal breaks. Whichever the reader chooses creates different journeys through the text, thus leading to different interpretive responses.

Although Pry incorporates sound stimuli similar to The Fifty-Year Sword, it is purposefully used to further accentuate the themes of vision that are so crucial to the narrative. For example, chapter three is completely written in braille (see Figure 5). The reader must touch the screen the way a blind reader would touch the raised dots. As the reader’s fingers move across the braille, James reads the passage aloud. Images appear behind the fading braille as each word is read, displaying James’s memories. This blend of image, sound, and reading forces the reader to decide to focus on either the braille being read to them or the images appearing behind the text at different variables. Touching the braille too fast causes James to skip words, whereas touching the braille too slowly results in a difficulty in comprehension, as James literally sounds out each syllable of the words. The enhancements in this chapter seeks to replicate the experience of reading braille, thus further drawing the reader into James’s life and body. The dual-route pattern of sound and vision employed in this chapter encourages the reader to make an effort to focus on the narrative to ensure as much information is accessed and retained. A study by Han Zhang et al. examining the effects of sound on eye tracking notes that readers listening to sounds with words are more likely to slow down and re-read passages to further understand the text. Yet because the brain suppresses irrelevant stimuli in order to focus on what is placed before the eyes, readers must make a choice of what they will focus on within this chapter. In this case, the brain may suppress the images to favor the sounds, or vice versa. The choice that is made is one that will deeply affect the way the reader is cognitively responding to the text. Furthermore, any information that is suppressed may be lost to the reader and influence interpretations of the narrative.

The interactivity of Pry’s nonlinear narrative means that cathartic resolution may be unattainable for readers. Because the text encourages exploration, the epilogue serves more as a coda that extends into an unknown future, than as a final closing statement that offers psychological relief. The text intersects touch, vision, and sound one last time in order to ask the reader to pry open James’s eyes to see President George W. Bush on a television announcing his decision to enter the war in 2001. If the reader pinches James’s eyes closed, the reader enters his memory of seeing President George H. W. Bush announce the same thing ten years earlier (see Figure 6). The presidential announcements pair evenly to construct a new, hybrid announcement that replicates a PTSD triggered flashback memory James is experiencing. The duality of the two presidents on the screen allows for the narrative to have a circular feel, in that the ending can propel James back into the thick of the war with the Middle East, return home to work construction, slowly go blind, only to be propelled back into the war again. This circular narrative inspires a sense of being caught in a trap as James can never reclaim his life outside of war. The infinite loop of the narrative further encourages the reader to revisit the previous chapters in order to continue moving in and out of the narrative, creating a fluid reading experience.

When computers became a part of everyday life, the transition from the print to digital resulted in direct copies in form. As Cannizzaro poignantly explains, “Early adopters weren’t confident in people’s digital literacy. …That’s why developers can be so wedded to the familiar: From the trash bin to the notepad, everything on your computer has a real-life corollary” (Herstik). However, as the technology develops, creators who recognize the blending of printed literature with digital production are making interactive narratives that contain whole worlds to be not just read, but also physically experienced. Through a sense of touch or sound, the narratives come to life by employing multiple areas of the brain, creating a new form of readerly immersion. Examining the way enhanced ebooks are designed, as well as how readers interact with them, allows us to consider not only the way these texts are developing new intersections between digital media and storytelling, but also the way readers engage with and interpret digital narratives. As the implementation of enhancements in The Fifty Year Sword and Pry show, enhanced ebooks not only influence the way creators undergo the storytelling process, but also create a new individualized immersive reading experience.

Works Cited

Coyier, Chris. “CSS Sprites: What They Are, Why They’re Cool, and How to Use Them.” CSS-Tricks, 24. Oct. 2009, https://css-tricks.com/css-sprites/. Accessed 25 Nov. 2018.

Danielewski, Mark Z. The Fifty Year Sword. Penguin Random House, 2012.

Dehaene, Stanislas. Reading in the Brain. Penguin Group, 2009.

Elen, Paul. “Classical Conditioning: Classical Yet Modern.” Psychologica Belgica, vol. 58, no. 1, 2018, pp. 196-211.

Gabrieli, et al. “The Reading Brain.” Mind, Brain, & Education: Neuroscience Implications for the Classroom, edited by David A. Sousa, Solution Tree Press, 2010, pp. 113-38.

Hansen, Mark B. N. Bodies in Code: Interfaces with Digital Media. Routledge, 2006.

Hayles, N. Katherine. Electronic Literature: New Horizons for the Literary. U of Notre Dame P, 2008.

--. How We Think: Digital Media and Contemporary Technogenesis. U of Chicago P, 2012.

Herstik, Lauren. “Pry is a Novella-Meets-IPad. App That You Touch.” La Weekly, 28 Nov. 2014, https://www.laweekly.com/pry-is-a-novella-meets-ipad-app-that-you-touch/. Accessed 25 Nov. 2018. 

Howard, Brian. “Pub Buzz: The Revolution will be Digitized: A Look at the Ebook Conversion Process of Mark Z. Danielewski’s The Fifty Year Sword.” Book Business Magazine, 1 Jan. 2013, https://www.bookbusinessmag.com/article/pub-buzz-mark-z-danielewski-50-year-sword-enhanced-ebook/all/.  Accessed 22 Oct. 2018

“Interactive eBooks.” Follett, 2019, https://www.follettlearning.com/books-materials/learn/digital-content/interactive-ebooks. Accessed 28 Dec. 2020.

Jhave. “‘Prying: Jhave on Tender Claws’ New App.” Los Angeles Review of Books, 29 Dec. 2014, https://lareviewofbooks.org/article/prying-jhave-on-tender-claws-new-app/#. Accessed 25 Nov. 2018

Kafalenos, Emma. “Overview of the Music and Narrative Field.” Narrative Across Media: The Languages of Storytelling, edited by Marie-Laure Ryan, U of Nebraska P, 2004.

Kudler, David. “The Problem with Enhanced Ebooks.” The Book Designer, 18 May 2020, https://www.thebookdesigner.com/2020/05/the-problem-with-enhanced-ebooks/. Accessed 12 Aug. 2020.

Landow, George P. Hyper/text/theory. Johns Hopkins UP, 1994.

Liu, Ziming. “Reading Behavior in the Digital Environment: Changes in Reading Behavior over the Past Ten Years.” Journal of Documentation, vol. 61, no. 6, 2005, pp. 700-12.

McFadden, Christine. “Are Textbooks Dead? Making Sense of the Digital Transition.” Publishing Research Quarterly, vol. 28, no. 93, 2012, pp. 93-99.

Miller, Carolyn Handler. Digital Storytelling: A Creator’s Guide to Interactive Entertainment. Elinor Actipis, 2008.

Milligan, Caleb Andrew. “The Page is a Touchscreen: Haptic Narratives and ‘Novel’ Media” Paradoxa, no. 29, 2017, pp. 1-17.

Mott, Robert L. Sound Effects: Radio, Television, and Film. McFarland & Company, Inc., 2014.

O’Connor, Kim. “The Ghost in the Machine: Avant-garde Novelist Mark Z. Danielewski is changing the way we read e-books.” Slate, 3 Dec. 2012, https://slate.com/technology/2012/12/mark-z-danielewskis-the-fifty-year-sword-enhanced-e-book-will-change-kindle-and-ipad-reading.html. Accessed 22 Oct. 2018.

O’Riley, Christopher. “Mark Z. Danielewski” BOMB-Artists in Conversation, BOMB Magazine. 13 Jan. 2013, https://bombmagazine.org/articles/mark-z-danielewski/ 25 Nov. 2018.

Osnos, Peter. “Enhanced E-books and the Future of Publishing.” The Atlantic. 4 Oct. 2011, https://www.theatlantic.com/entertainment/archive/2011/10/enhanced-e-books-and-the-future-of-publishing/246111/. Accessed 25 Nov. 2018.

Pry. Tender Claws LLC, 2015. Vers.1.0. Apple App Store, https://itunes.apple.com/us/app/pry/id846195114?mt=8

Ryan, Marie-Laure. Narrative as Virtual Reality: Immersion and Interactivity in Literature and Electronic Media. Johns Hopkins UP, 2001.

Singer, Lauren M. and Patricia A. Alexander. “Reading Across Mediums: Effects of Reading Digital and Print Texts on Comprehension and Calibration.” The Journal of Experimental Education, vol. 85, no. 1, 2017, pp. 155-72.

Treon, Matthew. “The Fifty Year Sword: An Interview with Mark Z. Danielewski.” Lit Reactor, 16 Oct. 2012, https://litreactor.com/interviews/the-fifty-year-sword-an-interview-with-mark-z-danielewski. Accessed 10 Oct. 2018.

Weinzimmer, Sara and Russ Weinzimmer. “Sound-enhancing ebook with sound events triggered by reader progress” n/a. WO2012006256A2, 2010. 19 Oct. 2018.

Wilk, David. “Why it’s Too Early for Publishers to Give up on Media-Rich Ebooks.” Digital Book World. 2013, Link no longer available, essay attained by messaging author.

White, Glyn. “Reading the Graphic Surface of Mark Z. Danielewski’s The Fifty Year Sword.” Mark Z. Danielewski, edited by Joe Bray and Alison Gibbons. Manchester UP, 2011, pp. 105-22.

Wolf, Maryanne. Proust and the Squid: The Story and Science of the Reading Brain. Harper Perennial, 2008.

Zhang, Han, et al. “How Listening to Music Affects Reading: Evidence from Eye Tracking.”  Journal of Experimental Psychology: Learning, Memory, and Cognition, vol. 44, no. 11, 2018, pp. 1778-791.

Notes

1. The move from hypertext to enhanced ebooks makes way for what some scholars, such as Caleb Andrew Milligan, refer to as “haptic narratives.” The term may “invite readers to take stock of these fictions as touchable texts, just as much as they are readable texts” (1). While my focus on the use of The Fifty Year Sword and Pry are on their status as “enhanced ebooks,” Milligan’s consideration helps us consider how we have transitioned from early forms of electronic literature created by Joyce and Jackson to the enhanced ebooks of Danielewski and Tender Claws.
2. While some scholars may view Pry as an app or a game, I argue that the novella should be viewed as an enhanced ebook because of both its digital and narrative design. Pry demonstrates a clear narrative structure that seeks to tell the story of James through chapters that are structured like an expected print narrative. Through the intersecting of a textual narrative with digital enhancements, Pry makes use of both the reading process and haptic gestures to draw the reader into an immersive experience that engages with multiple modes of reading. Pry may be sold as a downloadable app due to the size of its digital data, however, it is marketed, studied, and engaged with in the same manner as other enhanced ebooks. Occasional collaborator and conceptual poet, Vanessa Place, commented that “what Pry does is expand the field of the book…This is new reading, though it’s reading we know how to do. As much as photography altered the landscape of landscapes, digital forms and their writings open narrative to other kinds of telling” (qtd. in Herstik). Unlike games or other interactive apps, Pry requires a person to read textually and digitally to receive and comprehend the novella (in the same way a person must read textually and digitally an accepted enhanced ebook, such as The Fifty Year Sword), thus qualifying it to be understood as an enhanced ebook.
3. Coding, such as Sprites, make this digital material accessible. Sprites are a two-dimensional (although they can be three-dimensional) image or animation. CSS Sprites were designed so “the computer could fetch a graphic into memory, and then only display parts of that image at a time, which was faster than having to continually fetch new images. The Sprite was the big combined graphic” (Coyier). The use of Sprites allows the animation data to be stored on the device without interrupting the reading process.
4. In house, Apple refers to the enhanced ebook as “multitouch” or “transmedia storytelling” (Wilk).
5. Textbook companies publish interactive textbooks for both college level courses and K-12 courses. Follett, for example, makes a distinction between digital textbooks that are native digital or enhanced print (McFadden 94). Native digital textbooks focus on developing a more interactive connection between the student and the textbook. Instructors can include customized products that help with the interaction with digital material. Enhanced print textbooks refer to digitally replicated textbooks and course materials. Some examples of K-12 Follett native digital textbooks include the Gareth Stevens Interactive eBooks that cover life science, physical science, and geography with integrated activities, or the Live Oak eReadalongs that include full performances, music, and sound effects to “reinforce word recognition while amplifying action” (Interactive eBooks).
6. Sara and Russ Weinzimmer describe this process in their patent: a “reader’s speed can be calculated by timing the interval between page turns. An updated estimate of reader speed can be calculated with each additional page turn event…reading is presumed to have begun with the first page turn, and the reading speed used is the reading speed calculated…with the second and subsequent page turns, the reading speed can be automatically adjusted” (Weinzimmer 2012).
7. Like all of Danielewski’s other work, color and typography play crucial roles within the text. The programming of the enhanced ebook exemplifies this. At the start of the narrative, the prologue notes that five different colors of quotation marks (deep yellow, orange, red-brown, mid-brown, and dark brown) are assigned to the five key speakers of the narrative. These speakers are made up of the five (now adult) children. These quotation marks are on every page in various sizes and, essentially, hold the narrative together. One may feel as if the narrative will unravel (much like the hand-sewn illustrations) if the quotation marks weren’t there to keep it together. However, although we learn the names of the five children (Micit, Iniedia, Sithiss, Ezade, and Tariff), it is impossible to tell which color is assigned to which child. The complications in tracing the voices of the narrators calls into question the process of storytelling—a question that is very prevalent throughout the text. Yet, ultimately, it does not matter how the voices are stitched together; rather, it is significant that they are there at all.
8. Paul Elen has recently pushed for us to think of this more in relation to attribution theory—a “study of the manner in which certain events are explained in terms of their potential causes” (204). By thinking of responses as attributes, we can consider a person’s causal analysis, or, “how does one make causal inferences between all sorts of events?”
9. Although not cited as a direct influence, this method of employing a distinct sound for a specific character is reminiscent of orchestral performances of fairy tales; Sergei Prokofiev’s 1936 composition of Peter and the Wolf is one notable example of this. Prokofiev’s version was later famously performed by the Philadelphia Orchestra and narrated by David Bowie in 1978. Similarly, O’Riely has performed The Fifty Year Sword live with Danielewski narrating.
10. Robert L. Matt addressed this in a similar way through the use of sound effects in television when he wrote that sound can “supply two important elements in drama and comedy shows—to inform and to appeal to some emotion” (77).
11. Carolyn Handler Miller notes six types of interactivity that digital storytelling employs. These include, stimulus and response (stimulus comes from the program and response comes from an action by the reader), the ability to navigate and move, the control of virtual objects, the control of communication between characters, the sending of information, and the receiving or acquiring items (Miller 60). All of these can be seen in some form in the enhanced ebook, Pry.
12. Although the study by Zhang et al. is focused on the effects of music with words on individual reading, this information can still be relevant here as it is the pattern of the words being heard are affecting the ability of the reader to cognitively process the written words. Through a series of tests, Zhang et al. analyze how “reading comprehension with music did not make first-pass reading more careless” (1786). However, despite this, they did find that music interference can affect readers, resulting in their need to re-read passages and compensate for the interruptions (1783). Additionally, the interruption of new forms of sounds resulted in the reader gazing at written text longer than participants in the study who were reading in silence (1785).
13. This process of touch and vision expands in chapter six, when two sentences of text continue to unfold over and over again as the reader pries the text open. Sentences can be pinched closed and made smaller, but if the reader wants to see more and access the narrative, they are required to stay focused and continue to literally pry the story out of the text.

Cite this Essay

Ortega, Alejandra. “Intersections of New Media and Narratives: The Enhanced Ebook Reading Experience.” Rhizomes: Cultural Studies in Emerging Knowledge, no. 37, 2021, doi:10.20415/rhiz/037.e06