eHealth literacy issues, constructs, models, and methods for health information technology design and evaluation

The concept of eHealth literacy is beginning to be recognized as a being of key importance in the design and adoption of effective and efficient health information systems and applications targeted to lay people and patients. Indeed, many systems such as patient portals and personal health records have not been adopted due to a mismatch between the level of eHealth literacy demanded by a system and the level of eHealth literacy possessed by end users. The purpose of this paper is to present an overview of important concepts related to eHealth literacy, as well as how the notion of eHealth literacy can be applied to improve the design and adoption of consumer health information systems. This paper begins with describing the importance of eHealth literacy with respect to design of health applications for the general public paired with examples of consumer health information systems whose limited success and adoption has been attributed to the lack of consideration for eHealth literacy. This is followed by definitions of what eHealth literacy is and how it emerged from the related concept of health literacy. A model for conceptualizing the importance of aligning consumers’ eHealth literacy skills and the demands systems place on their skills is then described. Next, current tools for assessing consumers’ eHealth literacy levels are outlined, followed by an approach to systematically incorporating eHealth literacy in the deriving requirements for new systems is presented. Finally, a discussion of evolving approaches for incorporating eHealth literacy into usability engineering methods is presented.

information systems. He has been involved in some of the first work in providing patients with Internet access to their own health information (as stored in hospital-wide electronic health records) and he continues to work with designers and evaluators on a wide range of health information systems. He holds undergraduate degrees in Psychology and Biology from Brock University, as well as an MSc in Computer Science from McMaster University and a PhD in Cognitive Psychology from McGill University.

Introduction
It has become increasingly recognized that the success or failure of health information systems and applications designed for laypeople, patients, and consumers depends on factors related to the match between the demands a system (or application) places on a user and the end user's level of eHealth literacy. It is now recognized as being critical that end users to identify, understand, and apply information provided to them through the growing number and range of electronic resources becoming available these systems must be usable and the content must be understandable. This has included a myriad of online web-based information resources, portals, and specific health sites. In addition, new types of information systems and applications are being made available to the public, including personal health records (PHRs), personal health portals (PHPs), patient clinical information systems. Further, the number of health related applications made accessible through mobile devices, such as smart phones, tablets, and laptops is rapidly growing.
A number of countries have launched major initiatives such as the development of national health information portals targeted for use by the general public. Despite these trends, the success of such initiatives (in terms of leading to information and applications that are understandable, useful, and usable for the end users) has been, and continues to be, mixed. For example, in the United Kingdom (UK) the ambitious national Health Space patient portal project, which aimed to provide health information to UK citizens, was eventually abandoned . In retrospective analysis, evidence of consideration of eHealth literacy and analysis of user needs for the Health Space project was found to be lacking in the 3000 pages of project documentation reviewed by the authors. According to Greenhalgh and colleagues, hopes of increasing the health literacy of users were not realized over the 3 years of the project. Furthermore, based on this analysis, Greenhalgh and colleagues argues that future projects of this nature aimed at consumers and the general population should be developed by applying user-centered methods, taking user literacy needs into account . Many other health applications targeted to the lay public have either failed or have seen very limited adoption levels. For example, a number of problems have been reported with Australia's Personally Controlled Electronic Health Record (PCEHR) project (Xu, Gao, Sorwar, & Croll, 2014). Issues have been reported related to usability problems, problems related to efficiency and issues around the best model to engage citizens to actually use the health record, including the need to better meet citizen's information needs regarding information needed to improve their health. In response to such trends, a new and emerging construct has appeared that promises to have considerable application in helping to explain these failures and improve system uptake, lead to positive health outcomes, and improved end user satisfaction. Specifically, the concept of "eHealth" literacy -a multi-faceted concept that is a hybrid situated at the intersection among health literacy and information technology literacy dimensions. In this paper we will examine a number of different perspectives on defining eHealth literacy, as well as characterizing and measuring it. In addition, we will explore how concepts, methods, and perspectives for understanding eHealth literacy have (or could be) applied for improving the effectiveness of healthcare information systems, applications and electronic resources designed for use by patients and the general public. As will be seen, the concepts have important implications for the analysis, design, and implementation of a wide range of health applications and information systems developed for use by health consumers or lay people.

Background: From health literacy to eHealth literacy
Health literacy has been defined as "the degree which people are able to access, understand, appraise and communicate information to engage with the demands of different health contexts in order to promote and maintain good health across the lifecourse" (Kwan, Frankish, & Rootman, 2006). According to Rootman, Frankish, and Kaszap (2007), health literacy encompasses more than merely comprehending information, but rather it involves a number of cognitive processes, it may vary as a function of context, and it is important for health promotion and throughout life. Other research has shown that low health literacy levels limit understanding about health and one's condition as we also setting barriers to patient education. Along these lines it has been found that low health status is highly related to low health literacy levels (Murray, Hagey, Willms, Shillington, & Desjardins, 2008).
With the proliferation of computers, mobile devices, health apps and the use of Internet to obtain health information, there is renewed interest in the importance of health literacy, particularly in relation to competency levels and literacy of end users in the of use of computers and technology (which is required in order to benefit from the information and knowledge deployed electronically). To obtain advantage from the wealth of health information that is now currently available electronically, it was found there was a need to create a new conceptnamely that of eHealth literacy. Eng (2001) defined eHealth as "the use of emerging information and communication technology, especially the Internet, to improve or enable health and health care" (p. 2). This definition was then integrated with the definition of health literacy by Norman and Skinner (2006b) who defined eHealth literacy as "the ability to seek, find, understand, and appraise health information from electronic sources and apply the knowledge gained to addressing or solving a health problem" (p. 3). Norman and Skinner (2006a) also developed the influential Lily Model that integrates six competencies believed to underlie eHealth literacy: health literacy, traditional literacy, numeracy, computer literacy, media literacy, science literacy, and information literacy.

eHealth literacy: The balancing act
Other models of eHealth literacy include work by  who proposed a model whereby the influence of the usefulness and usability of eHealth applications and systems are moderated (and potentially greatly affected by) two additional factors: (1) the consumer's level of eHealth literacy, and (2) system demands on eHealth Literacy. If there are high demands on eHealth literacy that exceeds the consumer's skills, they are less likely to see the value in these systems or use them effectively. Likewise, if the system's demands a high level of eHealth literacy, it is likely that users will not be able to benefit from the information contained in the application or system.

Fig. 1. Striking a balance between consumers' levels of eHealth literacy and system demands on eHealth literacy
Further,  noted that two approaches exist to mitigating discrepancies between demands systems place on eHealth literacy and the eHealth literacy skills of the users. Not that in Fig. 1, the imbalance that is problematic is when consumers' levels of eHealth literacy are insufficient or too low to meet the demands the system places on eHealth literacy on the right. First, consumers' can learn the skills to raise their eHealth literacy levels. This approach requires teaching lay people relevant terminology and techniques for them to better access, understand, appraise electronic health information and apply their knowledge to make better decisions about their health. If this approach is to be adopted, it is imperative to have tools to reliably measure improvements of consumers' eHealth literacy in order to determine how effective interventions are at bolstering eHealth literacy skills. Such measures will be discussed in the following section. Second, designers and developers can create consumer health information systems in such a way that they minimize the demands they place on users' eHealth literacy. This approach requires identifying and applying techniques and strategies of how to write and display content as well types of interaction that are more successful for users with people with lower levels of eHealth literacy. Emerging methods aimed at achieving this goal of reducing demands systems place on consumers' eHealth literacy will be outlined in a later section.

Measuring consumers' eHealth literacy
Although there are many established scales for measuring health literacy, eHealth literacy measures are in still in their infancy. Building on the definition of eHealth literacy, several attempts have been made to develop scales specifically focused on measuring eHealth literacy skills. Two predominant groups of researchers have developed scales for assessing level of eHealth literacy in end users of computing systems and applications. The first was the work of Norman and Skinner (2006a) who developed the first tool developed specifically for assessing eHealth literacythe eHealth Literacy scale (eHEALS). The eHEALS uses a five point likert scale (from strongly disagree to strongly agree) and poses the following eight questions: (1) I know how to find helpful health resources on the Internet (2) I know how to use the Internet (3) I know what health resources are available on the Internet (4) I know where to find helpful health resources on the Internet (5) I know how to use the health information I find on the Internet to help me (6) I have the skills I need to evaluate the health resources I find on the Internet (7) I can tell high quality from low quality health resources on the Internet (8) I feel confident in using information from the Internet to make health decisions (Norman & Skinner, 2006a).
A limitation of the eHEALS is that scores are based on one's perceived skills as opposed to objective measures. In addition, van der Vaart and colleagues (2011) did not find a significant correlation between eHEALS scores and eHealth tasks completed, leading to interest in validating such scales. As a result, there is interest in developing new measures of eHealth literacy. Towards this goal, Norgaard and colleagues (2015) have conducted pioneering work at a fundamental level aimed at developing an eHealth Literacy Framework (known as eHLF) that can be used for a range of functions. One such application is using the eHLF as the foundation for creating eHealth literacy instruments not only for screening, but also as guide to understanding eHealth literacy at a deeper level . The eHLF was based on concept mapping from qualitative analysis of a number of workshops that generated 450 statements that were grouped in to 128 clusters. Using inductive structured analysis, the following 7 domains of eHealth literacy were identified: (1) ability to process information (2) engagement in one's own health (3) ability to engage actively with digital services (4) feeling safe and in control (5) motivation to engage with digital services (6) having access to systems that work (7) digital services that suit individual needs . Kayser, Kushniruk, Osborne, Norgaard, and Turner (2015) have employed the 7 domains to inform the design of health information system design (discussed below) and are also working on using the approach to develop a new survey tool for assessing eHealth literacy levels. Thus, different approaches for measuring eHealth literacy skills are currently under development, investigation, and validation. More eHealth literacy measures are likely to emerge as well.

Leveraging eHealth literacy to create usable, useful, and understandable health information technology
In considering where and how concepts related to eHealth literacy can be applied to improve healthcare applications and systems it is useful to consider the System Development Life Cycle (SDLC) which considers design and development of systems and applications through several phases including: planning, analysis (involving requirements gathering), design, implementation, maintenance. Concepts, ideas and measurements related to eHealth literacy can be fruitfully applied at different stages of the SDLC. The concepts can be incorporated into user-centered design, where there is an early and continued focus on refining systems through evaluation of end user capabilities and information needs. Moreover, adopting user centered design and evaluation approaches at various stages in the SDLC is demonstrating promise in this area.
A new approach, developed by Kayser et al. (2015), was proposed for gathering requirements for consumer eHealth systems and applications, and for testing system prototypes early in the SDLC. In this work, a modelling formalism known as the usertask-context cube (which involves explicit description of expected users of a system, the tasks they are expected to carry out using the system, and the context of use) has been applied in a systematic way for gathering system requirements that take into account Norgaard and colleagues' (2015) 7 eHealth literacy domains (see Fig. 2).
In step 3 of the process outlined by Kayser and colleagues (see Fig. 2), the components of the user-task-context cube are considered in conjunction with Norgaard and colleagues' 7 domains. For example, regarding domain 1, can a particular user carry out tasks (in specified context) a way that allows them to appropriately process information? The approach has begun to be used for systems analysis and design of a number of consumer eHealth applications in Scandinavia and Australia. It holds considerable promise by providing a stepwise framework for explicitly considering eHealth Literacy in defining system requirements. This include defining the classes of users expected to adopt a new system or application, and describing key tasks and activities users of different levels of eHealth literacy would be expected to carry out effectively and efficiently.

Ensuring systems and applications place appropriate demands on users' eHealth literacy
In order to evaluate the match (or mismatch) between a consumer's level of eHealth literacy and system demands new approaches must be used. A variety of methods that have emerged from the field of usability engineering can be applied (Nielsen, 1994) and have begun to be adapted for taking into account eHealth literacy. Two of the main usability engineering methods include usability inspection and usability testing and both are described below, from an eHealth literacy perspective.

Usability inspection
Usability inspection methods involve an analyst stepping through and evaluating a user interface or system for its usability (Nielsen & Mack, 1994). In one of the main usability inspection methodsheuristic evaluation, one or more usability analysts systematically step through a user interface comparing it against a set of heuristics (or guidelines) for good user interface design, noting violations of the heuristics. In another method, the cognitive walkthrough, one or more analysts systematically step through the user interface noting steps taken, system responses, and potential user problems. In two lines of recent work, the usability inspection approach has been extended to include ability to assess level of eHealth literacy of an application or system.
Work is emerging with the objective of identifying potential issues related to eHealth literacy demands in consumer health information systems by developing new heuristics. Initially, Monkman and Kushniruk (2013a) attempted using a single health literacy heuristic to categorize content issues that were likely to create issues for lay people. However, it became apparent not just the content (e.g., terminology) but also as how it is displayed (e.g., long paragraphs, limited white space) can negatively impact the use of these systems by consumers' with limited eHealth literacy. Next, Monkman and Kushniruk (2013b) distilled advice from Health literacy online: A guide to writing and designing easy-to-use (U.S. Department of Health and Human Services, 2010) into a set of heuristics. These heuristics were subsumed from guidance on leading practices for the design of systems and written content as well as displays for lay people with limited health literacy. Subsequently, their work progressed by exploring and incorporating more evidence from published studies related to eHealth literacy and usability. Specifically, Monkman and colleagues (Monkman, Griffith, & Kushniruk, 2015) conducting a scoping review and generated a set of heuristics for assessing eHealth literacy and usability simultaneously based on evidence in on health and eHealth literacy and usability issues in the literature. This study outlined an approach for conducting such evaluations on health applications for consumers that includes using 8 general heuristics applicable to all consumer health information systems and three optional heuristics suitable for specific types of content or mobile applications.
Using a different usability inspection approach, Chan and Kaufman (2011) extended Norman and Skinner's (2006b) six competencies related to eHealth literacy to include six additional cognitive process dimensions: remembering, understanding, applying, analyzing, evaluating and creating. Analysts stepped through the user interface of applications and identified which cognitive competencies were required, the type of literacy needed to successfully complete tasks, and potential issues. In addition, 20 users were then recruited to carry out the same tasks, with their actions and verbalizations being audio recorded. Errors and problems encountered by users were related to the findings from carrying out the usability inspection approach.
Thus, multiple different approaches are beginning to be applied in evaluating the relation between eHealth literacy, usability, and successful and effective use of information technology targeted to lay people.

Usability testing
Usability testing is generally considered the gold standard for identifying system usability issues. This method involves studying representative users attempting to complete representative tasks using a system. Therefore, usability testing is very promising for its potential to identify system issues related to eHealth literacy. Further, the U.S. Department of Health Services (2010 recommends this approach for designing consumer health systems that are easy to use and appropriate for people with limited health literacy. It is extremely useful to conduct usability testing with representative end users (i.e., lay people who would be expected to use the application when completed) to identify eHealth literacy issues during the developing and testing phases of applications for lay people. Usability testing with considerations for eHealth literacy allows people expected to use the system (or prototype) and comment on the content and design and identify potential obstacles to understanding information or using the system. This approach has been shown to reveal opportunities where the eHealth literacy demands of the system exceed the skills of the user. For example, pilot usability testing of a Personal Health Record (PHR) revealed eHealth literacy problems such as the lack of a legend on the blood pressure graph and challenging medication terminology (Monkman & Kushniruk, 2013a). These issues would undoubtedly create problems for users' ability to understand and make use of their health information. Such testing is not only beneficial for existing applications, but can also assessing demands on eHealth literacy and identify problems in developing and prototype applications (as recommended by Monkman, Griffith, & Kushniruk, 2015). The methods used can vary from observing users interact with systems (or prototypes), interviewing end users, to using questionnaires, to application of a general methodology referred to as usability inspection.

Conclusion
Despite being a relatively new construct, eHealth literacy is gaining attention and traction quickly because of its potential implications for consumers. A variety of different tools are under development for measuring consumers' eHealth literacy skills. Similarly, researchers are also working to create new methods or adapt existing methods for the purposes of identifying where the demands of consumer health applications exceed users' capabilities. Additionally, opportunities for incorporating considerations for eHealth literacy in the requirements gathering phase to better inform the design of new consumer health information systems are also being explored. Moreover, eHealth literacy can be incorporated into design of systems targeted for a range of users, and can be adopting using a user centered design approach Thus, it is being increasingly recognized that eHealth literacy must be considered and examined throughout the entire SDLC and user centered design approaches should be used to ensure lay people can use and apply the health information they are provided.