Web Based Technology And Continuing Medical Education - Free Essay Example

Sample details Pages: 24 Words: 7332 Downloads: 8 Date added: 2017/06/26 Category Statistics Essay Did you like this example? This dissertation explores the use of Web based technology to enhance and maintain procedural skills in the context of continuing medical education. The research was initiated by the perceived need for novel and alternative methods of providing procedural skills training to health practitioners. This observation, supported by learning needs analysis, led to the design and implementation of a Web based educational resource aimed at doctors and other healthcare practitioners. The assessment part of the research focused on an empirical evaluation of the effectiveness of this Web based educational resource. This dissertation draws on a number of strands of Health Informatics: Don’t waste time! Our writers will create an original "Web Based Technology And Continuing Medical Education" essay for you Create order Principals of Heath Informatics Research Methods Clinical Information Systems Creating Online Educational Resources Whilst on a small scale, the results are relevant to medical educators involved in developing and evaluating web based educational resources. BACKGROUND Medical practitioners receive comprehensive procedural skills training and it is expected that this is maintained and regularly updated to limit skills decay and ensure clinical competency. Skills decay is defined as the loss of a trained or an acquired skill after a period of nonuse. Skills decay rapidly as the period of nonuse lengthens; and the extent of the decay is influenced by the characteristics of the skill and how and when these skills were learnt. Arthur et al., (1998) Skills are classified either as à ¢Ã¢â€š ¬Ã…“closed loopà ¢Ã¢â€š ¬? or à ¢Ã¢â€š ¬Ã…“open loop tasks.à ¢Ã¢â€š ¬? Arthur et al., (1998). Closed loop tasks are fixed sequence tasks with a defined beginning and end, for example, the preoperative anaesthetic machine check. à ¢Ã¢â€š ¬Ã…“Open loop tasksà ¢Ã¢â€š ¬? are tracking and problem solving tasks, for example, managing patientà ¢Ã¢â€š ¬Ã¢â€ž ¢s hypoxia. Arthur et al., (1998) in their review on the rate of skills decay and its influencing factors concluded that à ¢Ã¢â€š ¬Ã‹Å"closed loop tasksà ¢Ã¢â€š ¬Ã¢â€ž ¢ decay more slowly than à ¢Ã¢â€š ¬Ã‹Å"opened loop tasks.à ¢Ã¢â€š ¬Ã¢â€ž ¢ Arthur et al., (1998) also mentions that mental tasks decay more quickly than physical tasks and after 28 to 90 days of nonuse of the trained skills, task performance declines by 23% and by 40% after a year of nonuse. Clinical practice alone may be insufficient to prevent skills decay as indicated in a simulated airway management training study Kovacs et al., (2000). Skills decay quickly without practice; and procedural skills are only optimally retained when trainees regularly practiced the procedure on their own, in their own time and received periodic feedback. Training on simulation modalities, mannequins, fresh cadavers and live patients have the potential to successfully teach the procedural skill with significantly less skills decay over time as compared to didactic teaching alone. TI L et al., (2006). The traditional à ¢Ã¢â€š ¬Ã‹Å"one to oneà ¢Ã¢â€š ¬Ã¢â€ž ¢ apprenticeship model of medical procedural skills training and the in-hospital continuous medical education and maintenance of a skills base are often inefficient, expensive, and labour intensive. Patientà ¢Ã¢â€š ¬Ã¢â€ž ¢s, who are often used as practice tests subjects during skills training, safety is reliant on the medical practit ioner skills retention and task competency. Maintaining procedural skills competency may prove to be increasingly more challenging as expense, time constraints, available manpower, lack of resources and patientà ¢Ã¢â€š ¬Ã¢â€ž ¢s reluctance to be used as experimental models make this endeavor increasingly impossible to set up. The resultant worldwide move towards competency based training programs and self directed problem oriented based learning has made necessary the search for alternative valid and reliable educational methods for skills training and its maintenance. Fortunately, the last decade has seen an explosion in the use of technology to enhance medical education. Web-based educational programs, computer aided virtual reality situations, and high fidelity simulation has played an increasingly important role in medical education owning to its efficiency, ability to provide flexible learning experiences, multimedia capabilities, and economies of scale and power to distribute instructional content internationally. Vozenilik et al., (2004) In the last 5 to 10 years extensive empirical research has been conducted on the use of computer aided and web-based instruction in medical education where there has been overwhelming support for these mediums of instruction. Unfortunately the literature is strikingly sparse on the use of Web based instruction for procedural skills training and in the few studies where empirical research has been carried out; study designs were not robust enough to withstand interrogation or had inconclusive results. LITERATURE REVIEW A review of the literature was conducted to ascertain what work had been done in the field of Web based learning, medical education and procedural skills training. A CINAHL and Medline search was carried out exploring all citations up to June 2010. The search using Medical Subjects Headings (MeSH) à ¢Ã¢â€š ¬Ã…“Computer Aided Instructionà ¢Ã¢â€š ¬?, à ¢Ã¢â€š ¬Ã…“Internetà ¢Ã¢â€š ¬?, à ¢Ã¢â€š ¬Ã…“CMEà ¢Ã¢â€š ¬? returned 322 publications. Adding the MeSH term à ¢Ã¢â€š ¬Ã…“Reviewà ¢Ã¢â€š ¬? returned 21 reviews of which four were relevant. Replacing à ¢Ã¢â€š ¬Ã…“CMEà ¢Ã¢â€š ¬? with à ¢Ã¢â€š ¬Ã…“Procedural Skills Trainingà ¢Ã¢â€š ¬? produced only one noteworthy empirical research paper and 2 publications worthy of discussion. Relevant systematic reviews of the literature are summarized in the table below: 3 Title Author / Date Findings Review Conclusions Assessment of the Review Internet-Based Learning in Health Professionals: A Meta-analysis Cook et al., 2008 201 eligible studies with qualitative or comparative studies of Internet based learning accounting for 56 publications Internet formats were equivalent to non-Internet formats in terms of learner satisfaction and changes in knowledge, skills and behavior. Internet based learning is educationally beneficial. Comprehensive work with a robust study design. Skills outcomes included communication with patients, critical appraisal, medication dosing, cardiopulmonary resuscitation, and lumbar puncture. Unfortunately the study had many limitations as many publications were poorly designed with low methodological quality, without validity and reliability evidence for assessment scores and with widely varying interventions What the meta analysis did suggest was that no further studies comparing Internet based interventions with traditional methods or no intervention were merited as these types of studies would almost invariably be in favour of Internet Based interventions. The author of the review suggested that the questions that warranted further research would be when and should Internet based learning be used and how could it be effectively implemented giving impetus to the exploration of Internet based skills training and maintenance. Title Author / Date Findings Review Conclusions Assessment of the Review The Effectiveness of Computer-Aided (CAL) Self-Instructional Programs in Dental Education: Rosenberg et al. 2003 1024 articles systematically reviewed. 12 publications included in the final review. Five studies significantly favored CAL. CAL is as effective as other methods of teaching and can be used as an adjunct to traditional education or as a means of self-instruction. This study is a comprehensive review of controlled randomized studies with clear and relevant inclusion criteria assessed with good inter and intra rated reliability. The reviewer limited the study to dental students. Forms of Computer Aided Instruction was not clearly defined or specified in the inclusion criteria. It is unclear whether web based studies were included. The skills referred to in the studies were dental diagnostic not procedural skills. The apparent dearth of studies assessing procedural skills justified the investigation undertaken by this dissertation. Title Author / Date Findings Review Conclusions Assessment of the Review Internet-based medical education: a realist review of what works, for whom and in what circumstances. Wong et al. 2010 249 papers met their inclusion criteria. Learners were more likely to accept a course if it offered a perceived advantage over available internet alternatives, if it was easy to use technically, had elements of à ¢Ã¢â€š ¬Ã‹Å"interactivityà ¢Ã¢â€š ¬Ã¢â€ž ¢ and gave formative feedback. This study is a realist review and the methodology used answered the question of the study which aimed to provide a theory driven criteria to guide development and evaluation of Internet based tools. The findings and guidelines suggested in this review would later be incorporated in the design of the resource to be investigated in this dissertation. Title Author / Date Findings Review Conclusions Assessment of the Review eLearning: a review of Internet-based continuing medical education (CME). Wuton et al. 2004 16 studies met their eligibility criteria Internet based CME programs were as effective as traditional formats of CME A comprehensive and appropriate search of databases. Randomized controlled trials of Internet based education in practicing health care professionals. These results showed that Internet based interventions do have a place in CME and that these effects on skills behavior warrants further investigation. Title Author / Date Review Conclusions Assessment of the publication Procedures can be learned on the Web: a randomized study of ultrasound-guided vascular access training. Chenkin et al. 2008 Web based tutorial may be an useful alternative to didactic teaching for learning of procedural skills A randomized control trial with non inferiority data analysis. The non inferiority margin was specified at a 10% margin however the actual amount of improvement was not specified. Blinding bias was not assured and the trial relied on the reputation of the investigator. No mention of inter rated reliability was made. Despite its inherent weaknesses, the trial suggested that web based intervention is as good as the alternatives; however, the study incorporated the use of simulation and live models to teach the actual procedural skill. David Cook is a prolific writer of many reviews and publications investigating Internet based formats in medical education. His noteworthy publications à ¢Ã¢â€š ¬Ã…“Web based learning: pros, cons and controversiesà ¢Ã¢â€š ¬? Cook, (2007) and à ¢Ã¢â€š ¬Ã…“Where are we with Web based educationà ¢Ã¢â€š ¬? Cook,( 2006) extolled the benefits overcoming barriers of distance and time with novel instructional methods, and extenuated the disadvantages which included social isolation, upfront costs and technical difficulties of Web based education. He concluded that Web based instruction can be a potentially powerful tool and strongly recommended that the focus of future studies should concentrate on the timing and application of Web based learning tools. Summary of literature survey The review of the literature has outlined the use of Web based procedural skills training as an area that requires further research. Empirical research and systematic reviews that has been carried out thus far has been limited. The literature research conducted for this dissertation (though in its self may have been limited) was unable to find publications exploring the whole use of the Internet as a means of procedural skills training and skills maintenance. Justification and Learning needs analysis To assess the effectiveness of an Internet based learning resource in the context of procedural skills training, a skill had to be chosen that was relevant, involved both a physical and mental task, and had the potential of decaying. A procedural skill is defined as à ¢Ã¢â€š ¬Ã…“the mental (knowledge) and motor activities (behaviour) required to execute a manual taskà ¢Ã¢â€š ¬? and usually involves patient contact. Kovacs (1997). Furthermore, a learning needs analysis was undertaken to assess the value of this topic choice. Justification Intubation with a Laryngeal Airway Device (LAD) was chosen as the representative procedural skill. When a patient collapses from a cardiac or respiratory cause, timely control of the patientà ¢Ã¢â€š ¬Ã¢â€ž ¢s breathing and airway with prompt delivery of cardiopulmonary resuscitation (CPR) and defibrillation have resulted in life saving survival and neurological recovery. The LAD is a breathing maintenance device that can, with minimal training, be inserted effortlessly into the mouth of the patient allowing for breathing and oxygenation. It is increasingly being used in the repertoire of techniques available to frontline practitioners (practitioners first on call to resuscitation events) in emergencies where the technique has proven to be easy to use and life saving in the management of an airway crisis. Kette, (2005). In a survey of family medicine practitioners, all practitioners surveyed agreed that insertion of an LAD during a resuscitation procedure was a core procedural skill that most practitioners were required to perform in any setting; 86% admitting that they had been called upon at some point to perform the procedure. Wetmore et al., (2005). Insertion of a LAD with knowledge of the patientà ¢Ã¢â€š ¬Ã¢â€ž ¢s anatomy, indication and contraindication for use and technique of use both under a controlled setting and in an emergency is representative of a procedural skill a frontline practitioners is expected to perform. Learning needs analysis A key step in developing an effective educational website is performing a learning needs analysis to determine what the learning needs the resource hopes to address are and why these needs were not met by existing learning or teaching arrangements. Cook Dupras, ( 2004). A questionnaire not previously validated, making use of closed type questions, were used to assess three broad areas; knowledge and training, skills application and Internet accessibility. Eleven frontline practitioners were asked to provide an indication of how often they were called to attend resuscitation or airway management situations in the last two years. They were surveyed regarding training received in LAD usage and insertion, their desire to obtain more information or skills updating, and whether there was a perceived need for Internet-based continuing medical education courses on LAD usage and intubation. Their attendances at CME workshops in the last year were surveyed and the barriers to CME workshop attendance were assessed. The respondents were surveyed regarding access to the Internet and previous exposure to e-learning modules. This was done to assess whether the uptake of the resource would be biased towards participants with Internet access, frequent Internet uses or previous e-learning experiences. The results of the learning needs analysis showed that most respondents (90%) received exposure to the device. It is a requirement of their post as frontline practitioners, to be Acute Life Support (ALS) trained where usage of the device in resuscitation is taught. Half the respondents indicated that they were not comfortable with their level of knowledge; and 63% felt unconfident about inserting the device as they were on average, only exposed to two resuscitation scenarios per year. All had Internet access at work and at home; and half had previous experience of online learning. Only one percent of the respondents were able to attend a CME session in the last year, citing lack of time and convenience as the main reasons. 80% of respondents were interested in taking courses through the Internet, as continuous education credits are a requirement of a license to practice in medicine. Interest in the topic was high and given the above self-appraisal, it was felt that the course was nee ded and should appeal to this population. METHODS The method section is dealt with in two parts. The first will focus on the development and design of a Web based educational resource and the second on the evaluation of the resource. 1. Development and design The idea was to develop an educational resource that could be used to train, reinforce knowledge and maintain a procedural skill by employing and integrating principles of effective adult learning with the unique features of the web. The development was driven by educational needs and outcomes of learning needs analysis completed by participants in a previous part of this study. 1.1 Development Theories The course design reflected Adult Learning Principles and the aim of the course was to improve knowledge (cognition), integration of attitude changes (confidence) and in so doing result in a change in behaviour (competency). Gale (1986). With accessibility of the Web based educational resource, it was hoped that the resource would be accessed frequently until the task becomes automatic or accessed as a refresher when required or at regular intervals. The resource incorporated principals that were shown to be effective. It was centered on the learners needs, was focused on a specific task and recognised past experiences of the learner (Gale 1986). The theories used in the development of the resource included; Experiential Learning Theory, which concluded that experiential learning should have personal relevance, should be self-initiated and lead to pervasive effects on the learner. Rodgers (1969) Constructivist Theory where learning is an active process with learners constructing new ideas and concepts based upon past and current knowledge. Bruner (1966) Information Processing Theory where knowledge is presentation in sequences or à ¢Ã¢â€š ¬Ã‹Å"chunkingà ¢Ã¢â€š ¬Ã¢â€ž ¢ to accommodate short attention spans (Miller 1956). The educational resource strived to be pedagogically sound uniquely applying these principles online. Information was presented in small chunks in a sequential fashion, was self-contained, had interactive components and contained assessments with instant feedback. Online communication did not occur in real time as which happens with video conferencing and online chat rooms, instead the resource used communication that was asynchronous where participants logged on, viewed and downloaded course material, read postings and submitted interactive tasks. The advantage of using an asynchronous format was that learners and/or the instructor did not need to be online at the same time allowing the participant to work at his or her own pace. The asynchronous nature of this web based learning environment allowed for barriers of time, location and expense to be overcome. Sanoff (2005) 1.2 Moodle Description University College Londonà ¢Ã¢â€š ¬Ã¢â€ž ¢s (UCL) Moodle was the platform used to develop the educational resource. There are many applications offering free alternatives to the commercial software WebCTTM and BlackboardTM, however the UCL Moodle was chosen as a matter of convenience because it was accessible, independent of specific operating systems, fit for purpose and easy to use without much technical computer knowledge thereby potentially removing barriers to any future course design and development.. Moodle (Modular Object Oriented Dynamic Learning Environment) is software freely available to use and was developed by Dougiamas. Moodle. org (1999). The Moodle software was designed on pedagogical principles that encourage learner interaction in a virtual learning environment. Moodle is a course management system used to support Web-based courses and has a number of innovative tools that could be used to create courses that promoted collaborative learning. Moodle is able to run without modification on Unix, Linux, FreeBSD, Windows, Mac OS and Netware. (Moodle. Org). After an initial learning curve, the program was easy to use with simple but comprehensive online instructions. Moodle It did not require pre-existing computer programming knowledge, and in fact the author of this dissertation considered herself a novice computer user. Moodle is written in hypertext pre-processor (PHP) which is HTML embedded scripting language used to create dynamic Web pages.  PHP allows for connecting to remote servers, checking email, URL encoding and setting cookies. It offers good connectivity to many databases including MySQL, and PostgreSQL,  which Moodle uses as a single database. MySQL is a  relational database management system  that runs as a server providing multi-user access to a number of databases. (www.php.net). Moodle had the support for easily displaying multimedia aspects of the educational resource and the interface could be used in over 70 native language translations. The Web based educational resource was easily built up using multimedia activity modules and design elements, which included with easy navigation; Authentication and enrollment, Syndication with a chat forum made available to others as newsfeeds, Current evidence based didactic teaching, Interactive quizzes allowing import/export in a number of methods Hyperlinked resources to provide for branched learning, The use of a Wikipedia, A glossary of commonly used terms, Instructional video presentations. All the attributes of the Moodle made for an international transportable tool ideal for knowledge presentation, learner interaction, comments and reflection, dynamic and interactive assessments, flexibility, extendibility, and most importantly, support for autonomous learning and continued educational development around the world. The only noteworthy disadvantage of using the UCL Moodle was an imposed instructional design. 1.3 Resource Description The educational resource was named; VIRTUAL [emailprotected]/* */: Onà ¢Ã¢â€š ¬Ã¢â‚¬Å"line Laryngeal Airway Device training. à ¢Ã¢â€š ¬Ã¢â‚¬Å" à ¢Ã¢â€š ¬Ã…“Virtualà ¢Ã¢â€š ¬? suggesting both the virtual reality of an Internet based generated environment and the adjective, meaning à ¢Ã¢â€š ¬Ã‹Å"practicallyà ¢Ã¢â€š ¬Ã¢â€ž ¢ or à ¢Ã¢â€š ¬Ã‹Å"almostà ¢Ã¢â€š ¬Ã¢â€ž ¢ Collins English Dictionary ( 2008). The Web based educational resource was developed for distance learning and contained all the elements of a totally Internet delivered educational resource. The content of the course was drawn from the authorà ¢Ã¢â€š ¬Ã¢â€ž ¢s personal experience using the Laryngeal device; peer reviewed journal articles, manufactureà ¢Ã¢â€š ¬Ã¢â€ž ¢s product information and videos downloaded from the Internet. Permission for the use of copywriter-restricted material was sought and obtained where appropriate. 1.3.1 Screen Design The screen design refers to how the information was arranged and presented on the display screen. The guidelines used followed those (amongst others) suggested by DoD HCI Style Guide (1992). The screen was kept simple, orderly, clutter free and consistent with a limited, non-dominating colour palate of four colours à ¢Ã¢â€š ¬Ã¢â‚¬Å" blue, black, white and blue à ¢Ã¢â€š ¬Ã¢â‚¬Å" green, a combination that has been shown to cause little fatigue and distraction. Kelley (1988). The content of each lesson was presented on a plain white background with black text in a non-jarring informal style font that made the lessons easy to read. Clark (1997). One template was used and the navigation bar, top bar and individual lesson heading bars kept the same with only the content of each lesson changing. All the content was displayed statically on one screen with individual lessons accessed by scrolling vertically down to the individual lesson. Unnecessary menus and long selection lists were avoid ed. This allowed for an overall view of the content, minimized pointer and eye movements and caused less distraction with easy navigation. (Gruneberg 1978). A discussion forum, interactive quizzes and an end of resource examination were included to allow engagement and self-assessment. The quizzes and examination included a range of question types à ¢Ã¢â€š ¬Ã¢â‚¬Å" multiple choice questions, true/ false, photo matching and random short answer matching type questions. These varieties of questions were shown to improve the learning experiences of adult learners. (Mackway-Jones, 1998). Information was provided in chunks and the writing style kept informal, with plain, simple language and in conversational tone with some elements of humour. There were fewer than 60 à ¢Ã¢â€š ¬Ã¢â‚¬Å" character positions on a standard 80 à ¢Ã¢â€š ¬Ã¢â‚¬Å" character line, spacing between characters were 25 à ¢Ã¢â€š ¬Ã¢â‚¬Å" 50% of character height and spacing between lines were equal to the character h eight, this to increase reading efficiency. 1.3.2 Course Content The course material was presented as text, graphics, power point presentations, hyperlinks and video demonstrations of the procedural skill presented in animation and on an actual patient. Knowledge was provided in five short lessons that followed the natural sequence of usage and intubation with a Laryngeal device. Aims of the resource and objectives of each lesson were stated at the beginning of the course. Here too a glossary of commonly used terms and a baseline knowledge assessment quiz were included. Each lesson was kept succinct with hyperlinks to websites and folders for those seeking extra information. This was to limit download times. Each lesson was concluded with an interactive quiz used to reinforce and test the knowledge learnt. Instant responses were provided to the quizzes after submission with suggestions to either revisit the lesson or to continue depending on the results obtained in the quizzes. The resource was concluded with an end of course examination and the c ourse was predicted to take 1 to 2 hours to complete. The resource content was accessed with a secured password with all content downloadable by way of an Internet connection. All the participants were supplied with a secure company email address and all the ISTCs had Internet access. Permission was requested for the use of company time and resources e.g. airway device training mannequin and time during the working day for those who chose to access the resource at work. Participants were supplied with instructions on how to use Microsoft Word and how to log on to and navigate the Moodle site. The course material was available online for two weeks with access monitored. 1.4 Pilot Study The aim of the pilot study was to assess the ease of navigation, gauge the time it took to complete the course, the integrity of the hyperlinks and the validity and reliability of the content and examination questions. Font preferences, layout and download speeds were also assessed. A prototype of the resource was tested on a selected sample of five participants of similar profile to the participants used in the study. The participants of the pilot study were excluded from participating in the actual study. An external panel of three Consultant Anaesthetists and two trainee Registrar Anaesthetists where used to provided expert advice. The Consultant Anaesthetists were selected based on their special interest in emergency medicine or difficult intubation scenario teaching. A few typographic errors were corrected, aims and objectives were clarified, difficult navigational issues were corrected and some content deemed repetitive and lengthy by the pilot participants were excluded before rolling out the resource. These changes however, were minor and further usability studies were deemed unnecessary. 1.5 Content Validation The content presented was current, evidence based and peer reviewed for content validity by the panel of experts (made up of three Consultant Anaesthetists and two trainee Registrar Anaesthetists), who deemed the content to be relevant and appropriate. The panel of experts and the pilot participants also judged good face validity. 2. Evaluation of the Resource The study evaluates effectiveness and acceptance of a Web based educational resource used to train and maintain a learnt procedural skill in the context of continuous medical Education (CME). The evaluation of the resource was undertaken in two parts. First the effectiveness of the resource was evaluated and the endpoints measured were changes in knowledge, confidence and technical ability. This evaluation made use of a summative framework redefined by Saettler (1990); which takes place after interaction with the resource. A à ¢Ã¢â€š ¬Ã…“before à ¢Ã¢â€š ¬Ã…“and à ¢Ã¢â€š ¬Ã…“afterà ¢Ã¢â€š ¬? interventional ipsative assessment was undertaken where participantà ¢Ã¢â€š ¬Ã¢â€ž ¢s performance was compared to their own over a period of time. The second part of the evaluation was undertaken to assess the acceptability of the Web based educational resource as a medium for procedural skills training and this was done by way of an evaluation questionnaire completed by the participants after course completion. 2.1 Participants and Setting The participants and settings were specifically targeted, as they would ultimately be interested stakeholders and end users of this type of resource. The research was conducted at five Care UK TM Independent Centers (ISTCs) on practitioners employed at these facilities. The ISTCs are part of the governmentà ¢Ã¢â€š ¬Ã¢â€ž ¢s initiative to reduce long NHS waiting times for elective surgery by adding increased capacity and alternative treatment venues for patients. There are approximately 25 ISTCs in the United Kingdom with Care UK TM represents 20% of this market. The ISTCs were chosen as a setting because: They are not part of the UK NHS medical training scheme and therefore have no formal programs of medical training or teaching that similar grades of staff in the NHS would receive. Contractual obligations of the ISTC contract decreed that the ISTCs could not employ medical practitioners from the NHS; therefore, most of the medical staff employed at the ISTCs have trained abroad and are waiting either to enter a formal career path within the NHS or wanting UK work experience. This situation has resulted in a mixture of nationalities, non-uniform medical training and medical staff with differing levels of post qualification experience and more importantly, a high staff turnover. (ref) These resulting factors were conducive to a system of competency-based appraisals and continuous medical education, which could be addressed with Web, based educational resources. Participants in the study were all frontline practitioners employed at Care UK TM ISTCs, which employs 48 practitioners of this grade. This represents 50% of all frontline practitioners employed in ISTCs throughout the UK. This intended sample size of 48 adequately represented the wider population in this type of analysis. frontline practitioners are the first practitioners on call to the resuscitation of a collapsed patient where they would be called upon to secure the patientà ¢Ã¢â€š ¬Ã¢â€ž ¢s airway and ensure oxygenation until the Anaesthetists or the resuscitation team arrives. It is expected that frontline practitioners are trained and certified with acute cardiac and life support skills and confident in dealing with clinical emergencies. In reality, analysis has shown that frontline practitioners in these ISTCs, though some trained and certified, rarely use these skills due to the infrequent nature of resuscitation clinical emergencies, making these scenarios potentially high -risk events when they do occur. Frontline practitioners are made up of Resident Medical Officers (RMOs), Anaesthetic Assistants (ODAs) and Recovery Room Practitioners (RNs). RMOs are doctors who have completed their medical training and have at least two years post graduate work experience as qualified doctors. They are employed to provide 24 hours on site medical management of patients at the ISTCs and like general practitioners (RACGP 2006) and doctors outside NHS academic hospitals, are usually first on call for emergencies and the sole source of medical advice on the premises on which they work. Anaesthetic assistants and recovery room practitioners are nursing practitioners employed to assist the Anaesthetists in theatre and attend to patients just out of theatre recovering from the effects of the anaesthetic agents. Due to the academic isolation and constraints imposed by a busy and often inflexible rota, frontline practitioners find it difficult to attend CME sessions necessary for skills retention. This, coupled with infrequent use of critical resuscitation equipment like LADs, increases the risk of deskilling with potential harm to a patient in an arrest scenario. There is no formal training pathway or formalised program of Continuing Medical Education (CME) within the ISTCs and most of the frontline practitionerà ¢Ã¢â€š ¬Ã¢â€ž ¢s continued professional development is self-directed. This is likely to make them more receptive to Web based learning. It is important to remember that frontline practitioners employed at the ISTCs are in non-training posts with the ISTCs under no obligation to provide CME or time off for CME, therefore the onus and expense of CME is borne by the staff. It is expected that the convenience of time, place and curriculum will generate an interest in this method of gat hering CME credits and maintaining a procedural skills competency base. 2.2 Sampling There were 48 frontline practitioners of which each grade; RMOs (16), Anaesthetic assistants (16) and Recovery room practitioners (16) make up 33% each and were therefore proportionally represented within the Care UK ISTCs. Sampling was a matter of convenience as all 48 frontline practitioners invited to attend and those who choose to accept were recruited. Participants who choose not to interact with the resource or who choose not to undertake the pre and post written and practical examinations were excluded. It was intended that these be the only exclusion criteria. 2.3 Biases The study may be biased towards candidates interested in self- learning, continuing professional development and those interested or confident with e-learning. Experience suggested that the majority of medical practitioners recognised that continuous professional development is vital for career development and as a requirement of the GMCà ¢Ã¢â€š ¬Ã¢â€ž ¢s revalidation process, would therefore choose to participate. As an added incentive, completion of the learning resource was rewarded with internal CPD points. The research study may also be biased towards participants who were comfortable with negotiating the Internet. To minimise biases, the design of the online resource placed emphasis on simplicity, easy to follow instructions and easy navigation through the course. It was likely that most of the participants would have trained abroad with unknown learning styles and English as a second language introducing potential variation in responses. A telephone helpline, email assistance and Internet based FAQ were made available to assist the participants. It was unlikely that there would be a location bias as these treatment centers were equally spread randomly over 4 counties. 2.4 Ethics Ethical issues were considered broadly within the clinical academic research culture with management of ethics complying with Research Governance Framework standards DH ( 2008); Bowling (2002) Submission for ethics approval was made locally to the Care UK TM Companyà ¢Ã¢â€š ¬Ã¢â€ž ¢s Clinical Governance and Ethic Committee. NHS Ethics Committee approval was not needed, as the research conducted did not involve NHS patients or NHS staff. See appendix X Co-operation of each centreà ¢Ã¢â€š ¬Ã¢â€ž ¢s Medical Director was obtained to contact the frontline practitioners and to dispatch consent letters requesting informed consent to participate in the study. Each participant was provided with a description of the study, its purpose and methods, nature and reason for conducting the research, time scales involved and commitment expected. Participants were encouraged to participate and CPD points were offered on completion of the learning task. Participation however was voluntary with withdrawal from the study possible at any time thereby significantly limiting potential for coercion (see appendix). On acceptance, participants were asked to complete a consent form. Participants were assured that their information would be anonymous and that they would not be individually identified by nationality or age. Each participant was assigned a username and password, which was randomly assigned by way of a sealed unmarked envelope. The username and password was used to access the resource and was used as unique identifiers on the pre and post examination papers. The researcher conducted a blind trial with regards to the allocation of the usernames and passwords. All demographic data was treated confidentially within guidelines laid down by the Data Protection Act and all computer files were stored in a secure server which was password protected (DPA 1988). Findings of the study will be disseminated by way of the dissertation report and possible publication in a peer review journal to a wider audience. Results of the study will be offered to the participants and Care UK TM by way of an online report and formal presentation of the study results. 2.5 Summative Assessment Effectiveness of the resource was objectively measured using à ¢Ã¢â€š ¬Ã…“before à ¢Ã¢â€š ¬Ã…“ and à ¢Ã¢â€š ¬Ã…“afterà ¢Ã¢â€š ¬? written multiple choice type questions and a à ¢Ã¢â€š ¬Ã…“preà ¢Ã¢â€š ¬Ã¢â€ž ¢ and à ¢Ã¢â€š ¬Ã…“postà ¢Ã¢â€š ¬? practical skills test assessing change in knowledge, intubation skill performance and confidence. Procedural skills like behaviour are hard to change and difficult to measure, therefore a proxy measure like change in knowledge, which is easier to measure and quantify was used in the belief that an increase in knowledge is related to an increase in skill. To assess baseline knowledge, participants were randomly assigned to two groups (Group A, Group B), each group receiving a set (MCQ A1 or MCQ B1) of 10 written multiple-choice type questions which they were asked to complete before undertaking the resource. The use of multiple choice type questions has been recommended as a reliable method for use in competency based measures to measure the acquisition of knowledge. Association for Medical Education in Europe (1999) These questions were based on the content described in the lessons of the resource. Participants were requested to complete the questions on the answer sheet under examination conditions and within a time period of 15 minutes. The questions in the two sets were different, of varying degrees of difficulty and were worded in a way to prevent guessing and random answering. All the MCQS were one-best answer type with a few true false type questions included. There were no trick or extended-matching items and there was no negative marking. One point was scored for each correct answer with a maximum score of 10 per examination. An assistant was recruited to invigilate and collect the completed question sheets. Each answer sheet was identifiable only by way of the username randomly assigned to each participant and participants were requested not to discuss the questions. After a two-week period of interacting with the resource, a change in knowledge was assessed using à ¢Ã¢â€š ¬Ã…“afterà ¢Ã¢â€š ¬? written multiple choice type questions. The same format used to assess baseline knowledge was used and the same conditions were applied. Group A received MCQ B1 and Group B received MCQ A1 questions. In the event of participants discussing the questions the à ¢Ã¢â€š ¬Ã…“afterà ¢Ã¢â€š ¬? test questions, though the same were ordered and numbered differently. Psychomotor skills à ¢Ã¢â€š ¬Ã…“preà ¢Ã¢â€š ¬? and à ¢Ã¢â€š ¬Ã…“postà ¢Ã¢â€š ¬? completion of the course were evaluated using a specifically designed airway training mannequin on which the participants used the LAD. An initial à ¢Ã¢â€š ¬Ã…“preà ¢Ã¢â€š ¬? skills test was conducted before undertaking the resource and a à ¢Ã¢â€š ¬Ã…“postà ¢Ã¢â€š ¬? skills test two weeks after completing the resource. A modified, validated three pointà ¢Ã¢â€š ¬Ã¢â€ž ¢s global rating scale and a 22 point checklist was used to assess the procedural skill as suggested by the current gold standard for assessment of procedural skills Bould, (2009). Table 1 Modified 3 point global rating score of Laryngeal airway device insertion ability: Score 1 poor 2 competent 3 clearly superior Maneuvers Repeated, tentative, jerky Competent but occasionally awkward and stiff Fluid with no awkwardness Actual score Participants were presented with a scenario and their appropriate approach and performance were scored using a checklist of predetermined maneuvers with scores assigned to each step correctly performed (see appendix à ¢Ã¢â€š ¬Ã‚ ¦.). Time to ventilate the airway mannequin from initial positioning of the head and mouth opening to successful ventilation was considered a major outcome variable as the outcome was reproducible and easily quantifiable. An additional point was awarded if successful intubation was established within a time interval of 50 seconds. (ref)The researcher and a recruited assistant were used to independently assess the performance of individual participants. Participants were also asked to rate their confidence pre and post resource intervention. 2.6 Evaluation of Usability A previously validated (and modified for purpose) questionnaire evaluating learner satisfaction with the Web based educational resource was undertaken in the final part of the study (ref). Qualitative data by way of structured questions and quantitative data using a Likert Scale were collected after the post resource tests were completed. The participants were given a printed version of the questionnaire to fill in by hand and return anonymously as well as an attached word document for those wishing to return the questionnaire by way of an email. To improve the response rate, a collection box for responses were placed at each centre and a representative recruited to assist with reminders and collection. The questionnaire was made up of two parts (see appendix). The first was a set of structured questions used to collect demographic data about the participants e.g. age, gender, Internet user experience. This part of the survey was undertaken to assess whether there would be a relationship between age, gender, Internet user experience and preferences for Web based learning. Webb (2002). The second set of questions surveyed what the participants thought of the course design, course usability, the use of multimedia and interactive components, and the impact of the resource. The participants were also asked about exposure to similar learning through traditional means. This was to gain an understanding of how participants viewed technology as a means of continuing professional development and how the use of technology influenced their learning. The questionnaire consisted of 22 questions with choices ranked on a 5 point Likert scale where participants had to strongly agree, agree, disagree, strongly disagree or remain neutral. Questionnaire surveys are popular because of their low cost and ease of administration Mann (1998) and a well designed questionnaire can provide useful information in a standardised format and is widely used as a tool in continuing medical education. Pereles (1996), Morris (1997). The benefit of this type of closed-ended questions were that they were easy to standardise and analysis. The downside was that it was more difficult to design. Patton (2001). 2.7 Validity and reliability Validity and reliability describes the quality of a method of assessment and where possible previously validated tools were used. 2.7.1 Multiple Choice Questions The multiple choice type questions were drawn from a pool of unvalidated questions from past examination papers used in the training of Anaesthetic trainees. Examination questions from a pool of validated and reliable test questions would have been ideal to use however, as of this writing, availability of such a databank of questions do not exist. The panel of experts assessed the face and content validity of the questions and concurred that the questions were appropriate and tested knowledge. Concurrent validity was established by comparing the questions to a bank of questions used in the local Trust to examine trainees. The panel of experts independently categorized the test questions as easy or difficult. Where agreement could not be reached, the questions were graded as intermediate. Agreement correlation between the experts were calculated using a raw score formula for Pearsonian à ¢Ã¢â€š ¬Ã‚ ¦Ãƒ ¢Ã¢â€š ¬Ã‚ ¦Ãƒ ¢Ã¢â€š ¬Ã‚ ¦ To further establish the internal consistency or internal reliability of the questions, a difficulty score (DS) was used to define the degree of difficulty of each question against which the test papers from the pilot study was compared (Crocker 1986). Needs work à ¢Ã¢â€š ¬Ã¢â‚¬Å" speaking to Henry A degree of difficulty for each question was calculated and a Difficulty Score assigned to each grade of question: DS = the number of predicted correct answers that the participants would obtain divided by the number of all answers. Table 2 MCQA MCQB DS Difficult 3 out of 10 2 out of 10 0.25 Intermediate 2 out of 10 3 out of 10 0.125à ¢Ã¢â€š ¬Ã¢â‚¬Å" 0.25 Easy 5 out of 10 4 out of 10 0.55 Participants in the pilot study were given the two test papers (MCQA, MCQ B) to complete and a Difficulty Score was calculated on the actual answers received and compared to the predicted Difficulty Score. Analysis confirmed that different participants tended to do well or badly on the same parts of the tests thereby demonstrating internal consistency (see appendix). To test whether the MCQs were able to distinguish between pilot study participants who knew and understood the material and those who did not (that is to test the reliability of the MCQ), an overall Discriminative Index (DI) was calculated using the following calculation: DI = number of correct answers amongst the highest overall score. A DI score of 0.9 was obtained. Questions with a DI0.30 is considered as adequately discriminative. 2.7.2 Global Rating scale and modified checklist The global rating scale and modified checklist of LMA insertion were tools used in previous studies and were reported as validated, Bello (2005), Naik (2003). However, scanty to no information was provided on the validation process in the two publications and a review of checklists and global rating scales in assessment of procedural skills in anaesthesia did not report the establishment of construct or internal validity specifically for assessing intubation skills. Checklists have been shown to produce excellent reliability in trained observers, are easy to use after some training for optimal reliability, are potentially comprehensive depending on the checklist, Bould ( 2009) and using two experienced Consultant Anaesthetists as examiners of the participants performance resulted in good inter-rater agreement achieved. 2.7.3 Usability Questionnaire The questions used to assess the usability of the resource were a modified version of a validated questionnaire used in a previous study to assess technology (University of Maryland User Interaction Satisfaction 5.0 1995). These questions were specific enough to allow for meaningful, reliable measurement; however its ability to be generalised to wider groups allowed the findings to be reproduced on the participants used in this study (Maxwell 1992). 3 Data Collection and Analysis This study made use of quantitative analysis recorded on standardised data entry forms. Data from open-ended structured questions were categorised and data from closed à ¢Ã¢â€š ¬Ã¢â‚¬Å" ended Likert scale questions required quantitative analysis. 3.1 Power Calculation A power calculation using a web based power calculator for a paired t test was performed to determine the sample and the effect size to make the study significant. www.biomath.info/power/prt.htm). It was estimated that there would be a 30% mean difference of predicted change between pre and post course test scores based on the outcomes of the learning needs analysis. Using approximately one standard deviation in performance, a 5% significance level and a 70% power of the study, a sample size of 8 was suggested by the calculation to show an effect size of 1.1. However, given that it was possible to recruit more than eight participants, it was decided to operate on a smaller effect size and more participants. 3.2 Data presentation Descriptive statistics were used to describe the ordinal data derived from the Likert scale questionnaires. The data was presented as percentages in tables summarised as median or mode with the mode being most suitable for easy interpretation, variability was expressed with a range and the distribution of findings displayed in a dot-plot or bar chart. 3.3 Data Analysis The Wilcoxon signed-rank test, a non-parametric hypothesis test was applied to the ranked data from the à ¢Ã¢â€š ¬Ã…“beforeà ¢Ã¢â€š ¬? and à ¢Ã¢â€š ¬Ã…“afterà ¢Ã¢â€š ¬? MCQs and the à ¢Ã¢â€š ¬Ã…“preà ¢Ã¢â€š ¬? and à ¢Ã¢â€š ¬Ã…“postà ¢Ã¢â€š ¬? practical skills examination. This test was used to compare the two sets of scores from the same participant as an alternative to the paired Students t-test as the population could not be assumed to be normally distributed. The dependent variable was the examination scores and the independent variable the Web based resource. A SPSS data analysing program was used to generate descriptive and quartile statistics for the variables and ranks tables presented data of the comparison of participants à ¢Ã¢â€š ¬Ã…“Beforeà ¢Ã¢â€š ¬? and à ¢Ã¢â€š ¬Ã…“Afterà ¢Ã¢â€š ¬? course completion scores. Examinations of test statistic tables were used to show whether there were changes in examination scores and whether these changes were statis tically significant. Confidence Interval A 95% interval confidence level was chosen to generalise the results to that of the general frontline practitioner population. The selected sample of 48 participants as a proportion of the intended population in all the ISTCs corresponded with a Confidence Interval of ± 19%. That is to say that there was a 95% certainty that the true population proportion would fall into the range from 44% to 82% according to a Web based Confidence Interval for Proportions calculator.