problematic screen reading:
is it caused by our brain?

Esther de Groot  ■  Student of Book and Digital Media Studies at Leiden University

It is probably experienced by almost every adult reader that reading from any kind of screen can feel more complicated than reading from paper. Somehow screens do not give the reader the same kind of experience as a print copy would do. It is therefore often mentioned that people do not want to trade in their paper books for an e-reader or tablet, even though they hold several advantages over paper books. But what is it exactly that causes people to shy away from these new technologies?

In his book Reading in the brain: the new science of how we read, Stanislas Dehaene examines the way in which the brain analyses letters and words and gives them meaning. The next section will give a brief overview of Dehaene's explanation to see whether screens already cause problems on this basic level of reading letters and words. Afterwards other possible causes will be discussed.

How does reading work?

For the brain to give meaning to a letter, it first needs to be transferred into the brain via the eyes. The piece of the retina that is used for reading is called the fovea. This middle part of the retina is sensitive enough for light to see the letters. Because this part of the eye is very small it can only take in one or two words (depending on the number of letters in the word) at a time, causing the constant small movements of the eyes when reading.1 Enormous letters we read will cause us to read more slowly because one letter will take up more space in the fovea. More movements of the eyes are needed in this case to take all the letters in.2 The eye only sees the words focused on and maybe the initial letters of the next word to make sure our gaze begins at the right spot. The time needed to take in a word is only a mere one twentieth of a second.3 The visual system is equipped in such a way that it can disregard varieties in letter shapes. The recognition of a word is not impaired by a different size or shape of type of the same letter.4 For example, the brain does not see difference in meaning between ‘E’, ‘e’, ‘E’ and ‘e’ even though there are clearly differences in terms of shape. This is why we are able to read and understand different handwritings.

When the eye sees symbols that do not have any further meaning, only limited visual areas will be activated. When the symbol has a meaning though, it needs pathways that connect it to the visual and conceptual areas and visual and auditory specialisation areas.5 Before this happens the word is being placed in a mental lexicon of every word ever encountered to be encoded in a hierarchical order of letters, syllables, morphemes and words. Graphemes are placed together by our brain as having a single sound value, neglecting there are actually two letters there. In this stage it might spike the recognition of similar words that may have a completely different meaning before the right word is selected.

Fig. 1: The way a word is encoded in a hierarchical manner. Note that on the letter level the two t's are placed together as one.6

Afterwards the word can have two routes to get to its meaning. Firstly it can go straight from letter string to meaning. This is called the lexical route. The second option is called the phonological route in which it first has to be transferred into sound before coming to the meaning. This is usually the case with words that are new to the reader or very rare.7

Deheane describes the mental lexicon in a comparison to an assembly of daemons, each of which are assigned to a single word. When a word comes in resembling a particular one, its daemon will shout, as will others that have similar words. In this competition the best one will get chosen to represent the meaning of the word.8 In the case of a misspelled word, the best suited one is chosen, often leading to the reader probably not even noticing this if the shape is similar to the one that was expected.

Fig. 2: The places of the frontal and temporal lobes.

The region where the identification of letters takes place in the brain Dehaene calls the letterbox. This area is universally located in the same place regardless of nationality and language. The information from this part of the brain is sent to the temporal and the frontal lobes that encode sound and meaning.9 Different areas of these lobes are activated when hearing, reading, producing or associating words. These areas are again universal, no matter how we read, how we learned to read and what our cultural background may be.

The area referred to as the letterbox only responds to written language. It is thus a visual area. The recognition of faces and objects are handled in areas close by this letterbox.10 This makes clear that the brain deals with letters as if they were physical objects. In fact, this part we now use for reading is the same as the one which is still being used by primates to recognise objects. Since the invention of writing it has been adapted by human beings to be used for the recognition of letters, while it has been an object identifier for millions of years.11 Because there is a need to recognise every object no matter of its size, location, distance, if it is in light or shade, this also counts for letters, for the brain deals with them as a kind of object. The speed with which letters are encoded can also be linked to the history of the visual part of the brain. For our ancestors it was of the utmost importance to recognise shapes in nature (such as the footprints of a predator in the sand) that were vital for survival as quickly as possible.12

Primate brains do not respond very much to every single object, but to fragments of shape of these objects. These shapes could be called some sort of neuronal alphabet. Separate neurons react to the different shapes in objects and how they are arranged in space.13 It does not come as much of a shock to see that the neuronal alphabet is quite similar to the Hebrew, Greek and Roman alphabets.14 Humans have not so much invented the shapes of letters, but made use of the shapes that have been linked to the neurons in our brains for years.15 The connections made in the brain to make reading possible are, unlike those for speech and vision, not genetically organised. In every brain connections thus have to be made anew when learning how to read.16 It is in fact the ability of the human brain to make new connections that made it possible for humanity to read at all.17

The fact that this specific area of the brain has evolved into visual recognition of letters is partly due to its neurons that are connected with the fovea. They are the neurons best suited for the finer visual details of letters. Moreover it is placed in the left hemisphere, because this handles small local shapes better, while the right hemisphere deals with global shapes. Furthermore, the processing of speech is tied to the left hemisphere.18

Reading in the brain: the screen

After seeing how reading in the brain works, it seems that the level of reading separate letters or words is not what causes problems with reading from a screen. Reading begins with the recognition of an individual letter (which in its turn builds op graphemes, syllables and words) which is first visualised on the fovea before being transferred to the next stages of encoding. The only thing that matters here is the shape of the letter. And it does not even matter if there are varieties in the different ways to write this letter as long as its basic shape is still recognizable. For the fovea the shape of a letter does not change just because it appears on a screen of any kind. It is still the same letter and will thus be processed in exactly the same way as letters do when they are read from paper, vellum, parchment, etc.

One of the reasons for difficulties with screen reading may lie in some kind of unwillingness to accept the new medium

Screens that reflect light also do not interfere with this recognition either. This is due to the evolutionary history of the part of the brain which is now used to read. Objects needed to be recognised no matter if in light or shade. Because this is still the same now that this part of the brain is being used for the recognition of letters as objects, the light reflected from a screen should not interfere with the identification of letters.

The fact that the brain sees the letter as a physical object means that it does not matter where it is depicted. If the brain identifies a tree, it remains a tree, whether it stands far or near, in a forest or in a garden or in a picture book. The same goes for letters.

So what then is it that makes reading from a screen more difficult? To answer this question we need to take a look at the text as a whole instead of focusing on individual letters and words. Moreover some psychological reasons are in play.

Navigation

Besides interpreting letters and words as physical objects, the brain does the same with entire texts. How this precisely works remains unknown, but it is suggested that a text is seen as a landscape. When a book is read the brain maps the steps the reader has taken through the landscape. Because of this, it is fairly easy to remember where a specific passage is located in a book or magazine. The brain takes in information that can help to relocate the passage. For example how heavy the book was on each side when the passage was read. Was the left side lighter than the right side? Then it must be located more towards the beginning of the book instead of the end. Were both sides equally heavy? Then it is located somewhere in the middle. Moreover the brain remembers where on the page it is located (left page or right page, top or bottom corner, inner or outer corner). The flipping of the pages makes us physically feel the text, the progress and where we are located in the story.19 This can be compared with an actual landscape. If a person is blindfolded and then dropped somewhere in a town he has never seen before, he will have no way of navigating himself back to where he came from without the help of other people or navigational devices. But if he had been led there and seen every turn he made and all the buildings he passed it would be a lot less difficult to find his way back home.

The study showed that the participants preferred the computer less because it reminded them of the things that still had to be done

Reading from a screen can be compared to being a blindfolded man. Our minds have no idea where they are located. Either we scroll down on a computer, phone or tablet or tab to the next page on an e-reader. But never can both pages be seen, thus limiting the reference corners to four instead of eight. Also, the book always has the weight of the device that is being held. It does not matter if we are reading a 100-page book or one that has 1000 pages. The only way of knowing what is to come is checking the page numbers, but it cannot be physically felt. Just like the blindfolded man we cannot find our way back to a specific passage as easily as in a book because we do not know where we are placed in the landscape of the screen text. Moreover there was no right or left side that felt heavier than the other. The physical connection with the text as a landscape that can be mentally walked through and the context of the landscape are lost.

A study done by Noyes and Garland showed that there was no difference in correct answers and study time between students using paper and students using texts. However, when looking at the qualitative comprehension a difference was noted between remembering and knowing. The group that studied via a screen had a higher level of remembering than the paper group. This means that they remembered information connected to the last study session instead of really knowing it without having to attach it to a contextual association. The transition from the episodic memory to the semantic memory is not made.20 Another study suggests that this is due to the fact that the students find it more difficult to recollect the location of details in the text.21 A third study done by Anne Mangen shows that ‘if texts are longer than a page, scrolling and the lack of spatiotemporal markers of the digital texts to aid memory and reading comprehension might impede reading performance’.22 These results show that the navigation of a text also aids the memorisation of it.

The attitude towards screen reading

After years of reading from paper, parchment or vellum, one of the reasons for difficulties with screen reading may lie in some kind of (subconscious) unwillingness to accept the new medium for reading. When someone really wants to dig into a text in a critical way, to study or for work, that person will probably still go to a paper version. While people get articles more and more via the internet, when these articles need to be critically read the text will often get printed.

Text on screen is not taken as seriously as text on paper. This may be something that has been set in our minds for years. The printed book has authority because not just anyone can have a book published. Texts that can be read in books have been approved as good enough by a publisher. And while this may also count for (most) e-books, texts on the internet can be placed there by anyone and be of any quality. Moreover, texts that are read on screens can quickly be associated with short messages like e-mails, text messages and Facebook statuses. More serious (longer) texts are associated with paper. Maybe this is why people feel the urge to print articles when they want to critically read them.

This entails that people do not make as much of an effort when they are reading a text on screen. A study done at the San Jose University has also shown that people take a lot of shortcuts when they read from screen. They browse and scan, looking for a specific keyword.23 A study executed by Terje Hillesund has pointed out that expert readers who want to study a text do so with the paper edition. In this way they can flick back and forth and use a pencil or highlighter and make annotations to connect their reading to their writing. If an entire book is read from beginning to end they will all prefer the printed version.24 Although it is possible to scribble notes and underline in an article on a tablet, e-reader or on the computer these people still prefer to do this on paper. For this generation it is obviously easier to do it the old way, like they have been used to for several years.

Perhaps a future generation will have less difficulties reading a serious text on a screen as they grew up in a world where quality is less linked to a printed book with the approval of a publisher. Furthermore, they will probably study via a screen more often, because the technology to underline etcetera will come easier to them than to those of previous generations.

Screen light

In this case a distinction has to be made between the e-reader that works with so called e-ink and other screens. E-ink reflects light in the same manner as the letters in a paper book will do. On other screens such as the laptop or tablet however, the light will constantly shine on the readers face. This light can make that the reader can get a headache, eye strain or even a blurred vision. Studies have reported a higher level of tiredness and stress in comparison to reading from paper.25 The brain will still process the letters in the same way, but, logically, when having a headache or blurred vision this will not happen at a normal speed.

Distraction

Again a distinction needs to be made between e-readers and texts that are read online. Computers lack the transparency that is needed to enter the state of immersive reading as there are just too many distractions. As Hillesund describes in his study the web lacks this transparency due to toolbars, side panels and icons which are in their turn placed in the interface of a web browser or operating system. In digital reading there is also a kind of immersion, but this is a different kind than the one paper reading can create:

In imaginary reading and reflective reading, the text is fixed and the signs arbitrary and transparent; meaning and engagement are for the greater part created by internal processes in the user's mind. By contrast, online immersion is the result of external stimuli and the user's response to a flow of pictures, animations, videos, and text snippets.26

There is thus a greater deal of multimodality than we are used to in books. Sure, there can be pictures in books that have the power to create some distractions, but they will occur inside the text that is being read and be of a related topic. Pictures on the web do not need to be related to the text at all, as they are often of unrelated advertisements. Moreover, hyperlinks can lead the reader to a completely different text. It is psychologically very hard to resist the urge to follow such a link. A participant of Hillesund's study reported that when reading an online text he would first scroll down and then sideways to avoid seeing the advertisements keeping him from immersing in the text. He would preferably read the newspaper on his iPhone, because the only thing visible is the text itself.

Furthermore, the study showed that the participants preferred the computer less because it reminded them of the things that still had to be done. When the computer is switched on and is online, why not check your email and respond to questions regarding work? Paper does not have this contextual factor.27

In her book Proust and the Squid Maryanne Wolf also wonders if the way digital texts are presented has an influence on the way a text is processed:

The basic visual and linguistic processes might be identical, but would the more time-demanding, probative, analytical, and creative aspects of comprehension be foreshortened?

The web creates expanding amounts of information and the need to multitask. There is a higher cognitive load on the reader, which could effect the comprehension of the text.28

Conclusion

Studying how the brain works when reading letters and words has turned out to be not the cause of the difficulties with reading from a screen. The letters are first taken in by the part of the retina that is called the fovea. The visual system disregards variations in shape, because of which different shapes of the same letter can be recognised. The word will end up in a mental lexicon where it will be hierarchically encoded in letters, morphemes and eventually words. In this mental lexicon an analogy can be made with an assembly of daemons, each assigned to a specific word. The daemon will be activated if its word is seen by the fovea. The letters are identified in a part of the brain that is called the letterbox. This part of the brain is an area for visual recognition which has evolved into a reading area. It sees letters and words as if they were actual objects. The letters that are encoded by it rely on shapes that have long been linked to its neurons as a kind of neurological alphabet. For the encoding of sound and meaning the word is transferred to the temporal and frontal lobes. It can go to meaning via the lexical or phonological route.

If a letter is displayed on a screen it will still have the same basic shape. There is nothing different in the way the individual letter is seen by the fovea. Discrepancies caused by light, shade or sise should not matter, because the brain is equipped to recognise an object no matter where or how it is placed.

The problems start however at the higher level of the text. This has to do with the navigation of a text. Just like the brain sees letters as objects, it also sees a text as a whole as a physical object. This probably functions as a sort of landscape map. In a digital text the context of that landscape is lost. There is just a street, but the brain has no idea whether that street is located on the map. Finding back a specific passage is very difficult because the brain has fewer points to remember it by. There are no eight corners to orientate oneself with and the weight of the text does not change on the left and right side after proceeding in it. The physical connection where the text can actually be felt is lost.

Next to this neurological reason there is also a psychological one. This entails people's attitudes towards digital texts. In our society the printed book has been placed on a pedestal for years. It stands for quality that the mark of a publisher gives to it. Not just anyone or anything can be published. Online texts do not have this quality check and are associated with less serious matters. This is maybe the reason why people bring less attention and effort to reading texts online than on paper. A change in the attitude towards online texts might lift some of the difficulties that occur in screen reading.

Another reason for a preference to paper is the light produced by screen devices that does not work with e-ink. Long-term reading can lead to eye strain, headaches and blurry vision. These of course do not enhance the reading experience and the reading speed.

Computers also cause a high degree of distraction that make immersive reading very difficult. The presence of toolbars, side panels, icons, advertisements and hyperlinks cause a high level of multi modality and an urge to click (and thus leaving the page). These are not the best conditions to read longer texts in a serious manner.

In short, the brain does not experience difficulties when reading from a screen on the level of individual letters and words. It does cause problems when reading an entire text. The navigation of the text is made very hard, which can make the reader feel a bit lost. Extra difficulties occur due to the attitude towards digital texts, screen light and distractions.

E-readers could definitely have a bright future. They do not have the distractions the internet has and because of the e-ink there is no bright light. Moreover, unlike texts on the internet, books on the e-reader are taken seriously by the reader. It is not just any text, published by any kind of person, but by a writer whose story is approved by a publisher. If the producers of e-readers work on ways to improve the navigation of the texts, so that the brain can conceptualise the landscape of the book, nothing is making the reading any harder than a normal book would do.

Notes

  1. S. Dehaene, Reading in the brain: the new science of how we read, (New York: Viking Penguin, 2009), p. 13.
  2. Ibid., p. 14.
  3. Ibid., p. 17.
  4. Ibid., p. 19.
  5. M. Wolf, Proust and the Squid: the story and science of the reading brain, (Cambridge: Icon Books Ltd, 2008), p. 29.
  6. Dehaene, Reading in the brain, p. 25.
  7. Ibid., p. 28.
  8. Ibid., p. 43.
  9. Ibid., p. 53.
  10. Ibid., p. 74.
  11. Ibid., p. 125.
  12. Wolf, Proust and the squid, p. 12.
  13. Dehaene, Reading in the brain, p. 133–136.
  14. Ibid., p. 137.
  15. Ibid., p. 139.
  16. Wolf, Proust and the squid, p. 11.
  17. Ibid., p. 5.
  18. Dehaene, Reading in the brain, p. 166–167.
  19. F. Jabr, ‘Why the brain prefers paper’, Scientific American Magazine, November 2013, vol. 309, issue 5, (New York: Nature Pulishing Group, 2013), p. 49.
  20. A. Mangen et al., ‘Reading comprehension on paper versus computer screen: Effects on reading comprehension’, International Journal of Educational Research, November, vol. 58 (Amsterdam: Elsevier, 2013), p. 62.
  21. Ibid., p. 63.
  22. Ibid., p. 67.
  23. F. Jabr, ‘The reading brain in the digital age: the science of paper versus screens’, Scientific American, 11 April 2013, <http://www.scientificamerican.com/article/reading-paper-screens/> (Accessed 20 January 2014).
  24. T. Hillesund, ‘Digital Reading Spaces: How expert readers handle books, the Web and electronic paper’, First Monday: peer-reviewed journal on the internet, April 2013, vol. 15, no. 4-5, <http://firstmonday.org/article/view/2762/2504> (Accessed 20 January 2014).
  25. Jabr, ‘Why the brain prefers paper’, p. 50.
  26. Hillesund, ‘Digital Reading Spaces’.
  27. Ibid.
  28. Wolf, Proust and the Squid, p. 16.