Taking the show on the road: In-situ clinical simulations’ role in promoting teamwork

Clinical simulations are designed to increase communication and experience among all members of the healthcare team in a low stakes environment. In this study we investigate the current application of in-situ simulation for training and educational purposes at the University of Virginia Health System. One factor we examined includes the impact of the level of fidelity of the simulator on clinician experience. We also looked at the ability to document the situation and if generational differences exist among participants that determine their engagement. We examined types of data collection and examine what data might be useful to collect to determine if simulation improves patient outcomes. We interviewed several facilitators of this approach to divine its applicability to staff education and the potential impact on patient safety. We discovered several key themes including: levels of fidelity, generational differences in acceptance of simulation, difficulties in documentation in the scenario, improvement in communication and the difficulties in quantifying success. Implications included that the level of fidelity is less important than ensuring that the level of fidelity used is matched to the educational objectives and that the scenario created be supported and realistic.

and their impact on nursing student critical thinking and skill development.

Introduction
A primary guiding principle in medicine is to first do no harm. Most in-situ simulation situations seek to improve clinician's response to emergency situations; in most of these scenarios the patient has already died and they are seeking to make this a temporary condition. Several factors that we found to be common themes and worth exploring include that simulations be in-situ, varying levels of fidelity in the equipment used, and improvement of communication among participants. What became immediately apparent as to barriers to assessing these situations is the lack of ability to quantify improvement in clinician responses with measurable results and difficulties among the participants regarding realistic real-time documentation. The latter is likely an issue even in real life events.
First, we chose to explore the idea of in-situ simulation regarding improvements in practice. From herein in-situ will be defined as a simulation that takes place in the environment in which a real scenario could occur and preferably in moments when participants are unaware that the event is about to occur. Although benefits may very well exist even when a simulation is done in a laboratory or classroom type setting that may look very much like a hospital room we decided to examine simulations that took place in the location of actual patient care. We can also see benefits to pre-scheduled simulations, such as the ability to manipulate participants to include those less experienced and to avoid times when the scenario might interfere with patient care. However, we felt that examining scenarios that occurred in a manner that was reflective of a real situation was at the heart of in-situ simulations.
Regarding fidelity, a wide range of beliefs are held. We examined situations with high, mid, and low fidelity regarding the mannequin, equipment, and abilities. High fidelity is often noted as increasing realism in simulation scenarios however the high cost of these materials is a barrier for many institutions (Fickley, 2014). Some research exists that expresses the effectiveness of simulation even with low fidelity mannequins, however we were unable to find true research that demonstrated the true value of one level of fidelity over another.
There is a pervading myth among popular culture mediums where physicians bark orders at various staff that then carry out the orders. Thankfully, in real life this scenario does not generally exist as a team effort generally emerges. When strangers work together for the first time in any situation it is unlikely they will have any sort of collegiality or excellence in communication with one another. This is a gap that simulation seeks to fill. These scenarios can pull together healthcare workers of all levels to improve communication and promote teamwork.
Those who organize and facilitate these simulations hold the belief that they improve practice. It is a logical belief, practice and exposure should improve results however without results this is difficult to quantify. Most facilitators have surveys, sometimes pre and post, asking participants how they feel about their knowledge and their comfort level. Although this is good information it does not provide us with the answer of whether or not simulation actually improves patient outcomes in emergency situations. Even at a very large hospital it is difficult to gather and quantify this data because of the many variables.
Almost every hospital in America uses an electronic medical record (EMR). Many hospitals have non-production areas of these record that can be used for practice. Difficulties with using this are compounded though as clinicians are often quite unfamiliar with this area of documentation and with good reason. Emergency situations are not generally an everyday occurrence even on a high acuity unit, leaving clinicians with little practice time with this type of documentation. This is compounded with the stress of the situation and potentially with the discomfort of clinicians with using the EMR in general.
In-situ simulation is a strategy used to improve responses and communication among clinicians by allowing for practice caring for a patient in a simulated emergency situation. Simulation allows clinicians to experience potentially stressful and high-risk situations in a low risk method. It has become an increasingly important part of the education of healthcare workers and can therefore be transferred over into the work environment. Simulation gives clinicians the opportunity to practice skills and experience unusual situations in a moderately controlled manner with lower stakes than the real life situation.

Literature review
A thorough review of the literature was conducted using the EBSCO database with the search terms: ․ Simulation AND ․ Hospital AND ․ Emergency Limiters were applied to return scholarly articles published since 2010 in English. Duplicates were removed which returned 541 unique articles. Of these 487 were removed because they did not address the topic leaving us with 54 articles. Of these 29 were not in-situ simulations and so were removed. Eleven did not have the full text available and so were not reviewed. This left 14 articles for review.
Several of the themes that we identified in our preliminary finding emerged in the literature review as well. The themes noted include fidelity, documentation, communication, and difficulties in quantifying data.
Regarding fidelity it divides into two categories: the realism of the situation and the fidelity of the mannequin and supplies. One problem cited among the literature is difficulty in obtaining participation by hospital staff, which is particularly difficult for insitu simulation as it does take time away from their already busy schedules and patient care (Hill, Dickter, Van Daalen, 2010;Riley, Dalby, & Turner, 2012). Increasing the realism of the situation is thought to improve engagement from the staff. One study discussed providing time windows in which simulations might take place (two week time span) and having management encourage participation as a method of improving attendance and participation (Hill, Dickter, & Van Daalen, 2010). Among the studies it was noted that 6 used high fidelity simulators (Pak & Hardasmalani, 2015;Siassakos et al., 2011;Surcouf, Chauvin, Ferry, Yang, & Barkemeyer, 2013;Hamman et al., 2010;Delac, Blazier, Daniel, & N-Wilfong, 2013;Hunziker, Tschan, Semmer, Howell, & Marsch, 2010), 1 used a mid-fidelity simulator (O'Leary, Hokin, Enright, & Campbell, 2013), and 4 used low fidelity simulators/mannequins (Harris, Humphrey, & Cote, 2010;Hill, Dickter, Van Daalen, 2010;Fickley, 2014;Riley, Dalby, & Turner, 2012). Three studies did not disclose the level of fidelity used (Mellin & Poplawski, 2010;Davis, 2011;Olson & Maietta, 2014). Two of the studies used actors in place or in supplementation of mannequins, making this very high fidelity (Hamman et al., 2010;Siassakos et al., 2011). In the high fidelity studies several stated the belief that it increases the realism of the simulation and thus improves engagement (Pak & Hardasmalani, 2015;Hamman et al., 2010;Delac, Blazier, Daniel, & N-Wilfong, 2013). Studies that utilized low fidelity often cited a lack of funds as a barrier to simulation with higher fidelity. However, many sought to prove that the fidelity of the mannequin does not necessarily lead to decreased engagement and that a high level of realism is still possible even with lower cost equipment (Harris, Humphrey, & Cote, 2010;Hill, Dickter, & Van Daalen, 2010;Fickley, 2014;Riley, Dalby, & Turner, 2012). Specific problems noted with using high fidelity was the difficulty in using the equipment as well as the inability to make the mannequin match the intended scenario (Fickley, 2014). What we did not find among these articles was a comparison of high fidelity to low fidelity in similar circumstances.
As EMRs are becoming more prevalent in all aspects of patient care areas they are being used in emergency situations. Paper charting has prevailed in this area primarily because of the familiarity and speed that is so crucial in these situations. Clinical simulations are the ideal time to begin practicing using the EMR, which can allow for greater accuracy in data collection and improve the ability to retrospectively examine the actions of participants. Among the studies in our literature review only two cited the use of the EMR during simulation (Pak & Hardasmalani, 2015;Mellin & Poplawski, 2010), although another study provided electronic information to participants as the primary means of communication of information about the patient (Riley, Dalby, & Turner, 2012). One study noted that charting in the EMR was a potential patient safety concern as the computer removed the person charting from the area so much that they were not able to document accurately (Pak & Hardasmalani, 2015). Improvement of communication and teamwork among groups performing clinical simulations is a common goal and thus commonly cited among the literature. Many studies found that initially the participants of the simulations did not communicate well with one another (Fickley, 2014;Pak & Hardasmalani, 2015;Hill, Dickter, & Van Daalen, 2010;Delac, Blazier, Daniel, & N-Wilfong, 2013;Hunziker et al., 2010;Davis, 2011). However, several also noted that the discovery of this during a simulation allowed for interventions aimed at the improvement of communication should a real event occur (Fickley, 2014;Hunziker et al., 2010;Davis, 2011). Communication and teamwork is also a common focus in debriefing session which follow the simulations, which is generally aimed at identifying both successes and challenges identified during the scenario (Davis, 2011;Delac, Blazier, Daniel, & N-Wilfong, 2013;Olson & Maietta, 2014;Fickley, 2014;Harris, Humphrey, & Cote, 2010;Hunziker et al., 2010;Pak & Hardasmalani, 2015).
The ideal method for measuring the success of simulations is monitoring for improvement in patient outcomes in similar situations following the simulation. This is exceptionally difficult data to gather even in a large hospital since emergency situations do not always occur regularly and certainly not with the same team that participated in the simulation scenario. This has led researchers to develop other methods for obtaining data that is quantifiable or qualifiable, but does not fully measure the intended outcome of improvement in patient safety. We identified two primary methods in the literature that were used for analysing data to identify improvements. The first method is pre and post questionnaires that asked participants what they knew and how they felt about their performance in the simulation and/or emergency situations (Delac, Blazier, Daniel, & N-Wilfong, 2013;Siassakos et al., 2011;Surcouf et al., 2013;Riley, Dalby, & Turner, 2012;Olson & Maietta, 2014;Fickley, 2014;Harris, Humphrey, & Cote, 2010). This method of collecting data is good for measuring changes in thoughts and education and experience level changes, however it is impossible to correlate this to actual improvements in patient outcomes. The results can reveal important insights however, as one study found that residents reported less confidence after the scenario perhaps indicating inflated confidence that did not translate to practice (Surcouf et al., 2013). The other method of data collection that was identified was the identification of the time that interventions were implemented and looking for improvements in subsequent simulations (Pak & Hardasmalani, 2015;Hunziker et al., 2010;Davis, 2011;O'Leary, Hokin, Enright, & Campbell, 2013;Hill, Dickter, & Van Daalen, 2010). The American Heart Association (AHA) has identified the implementation of early cardiopulmonary resuscitation (CPR) and defibrillation as key points in improving patient outcomes so this type of data collection does allow the timing to be examined (American Heart Association, 2014). This method produces quantifiable data but still it is impossible to identify if improvement in practice times translates to an improvement in real life situations and an improvement in patient outcomes.

Data collection and methods
We conducted interviews with various simulation facilitators that each operate in different units and capacities and have examined their responses for themes and information that might provide insight into future improvements. We were particularly interested in how technology played a role in these simulations and so structured our interview around these key points. Interview questions are provided in appendix A, however as it was more of a discussion key points were noted outside of the structured questions. All interviews were conducted at the convenience and location choice of the facilitators. Please see Table 1 for list of facilitators and the setting the interview took place.

Results
After conducting the interviews we analyzed the text for keywords, classified the text by the keywords and then identified the themes that emerged. The key themes identified include levels of fidelity, generational differences in acceptance of simulation, difficulties in documentation in the scenario, improvement in communication and the difficulties in quantifying success. We only interviewed facilitators that conduct in-situ simulation but also examine how they feel this impacts their simulations. See Table 2 for a summarization of the themes that emerged in our interviews. It is important to note that each of the people interviewed have a different role and thus participants interact with them accordingly. The relationship varies from being a facilitator with no relationship to those on the unit to having a working relationship with most participants, with some situations being in between. This may color the participation and reactions that they receive, either positively or negatively. For instance, if the facilitator is someone they work with regularly they may be less apt to take the situation seriously. Conversely, if the facilitator is not someone that the participants know they may be reluctant to participate. One method identified to overcome this barrier is for management to mandate participation, which may not increase the amount of engagement but certainly increase the number of participants. Several of those interviewed expressed that if they get engagement from one key staff member then others were more likely to participate. Facilitator 4 stated "Once the one key nurse that the other staff look up to gets involved she will begin to draw the others in to participate and take the simulation seriously." Each of those interviewed provide in-situ simulation. There were many benefits cited to this type of simulation including increasing the realism and providing education in the moment. Another theme identified was that this type of simulation brought it to where the participants are both physically and educationally and allows them to work in a team that might actually occur. Some of the facilitators tailor when they perform these simulations based on what participants might be available, thus allowing them to give less experienced participants an opportunity to learn. Facilitator 1 said "I have found nurses tend to be experiential learners. Simulations allow the nurses to learn so many things at once. They learn how to prepare and administer code drugs, mix vasopressors, set up arterial lines, perform accurate CPR, learn ACLS algorithms, and learn about our ICU's patient population." Facilitator 3 and 4 seek to engage all healthcare workers in the scenarios allowing all staff the opportunity to be participate and be heard, from the certified nursing assistant performing compressions to the physician organizing the resuscitation efforts. In-situ simulation gives the opportunity for all who might be involved in an emergency situation the ability to be involved as opposed to pre-scheduled scenarios which might not include all types of staff in the correct proportions.
In-situ simulation also has unique challenges which must be considered. It is often difficult to find a physical location for the simulation if all beds in the hospital are full. Staff may be too busy with patient care to participate or disinterested as it interrupts the flow of their daily work routines. To overcome this it is necessary to produce a culture that accepts and embraces continuing education but this is also challenging in the fast paced healthcare system. Implementing simulation takes a great deal of effort from the facilitator, who also needs supporters since it can rarely be done by just one person, especially as fidelity increases. Each of the facilitators interviewed cited the amount of time to organize and implement the simulations as a challenge and barriers to having a higher quantity of simulations. Several found it challenging to bring these simulations to all areas of the hospital or even of their units and sometimes opt to perform simulations more frequently in the areas that are more likely to experience a medical emergency. Facilitator 4 also noted though that you cannot perform too many simulations in one area or staff may believe a real event to be a simulation and they may not respond. Facilitator 3 stated "On my unit participation in simulation is part of their evaluation and that can have good and bad consequences. It is good that there are plenty of participants but not good that there are commonly many there who are just there to check it off their to-do list and are not buying into the realism of the situation." Interestingly, none of those interviewed cited a lack of supplies as a barrier, indicating that their healthcare system supports them by providing adequate supplies.
When questioning facilitators about the types of data they collect the answers were varied. Facilitator 1 and Facilitator 2 administer pre and post surveys that examine how participants felt about their knowledge and what they thought about the scenario. Facilitator 2 also specifically focuses on communication, a key point in an ICU where staff are the Medical Emergency Team. See appendix B for the survey given by Facilitator 2. Facilitator 4 tracks participants and the LSLC tracks the outcomes of medical emergencies at UVAMC, however it is difficult to determine if outcomes are improved from this data. Facilitator 3 keeps track of verbal responses during the debriefing to minimize the formal structuring of responses, however this also yields data that is difficult to quantify. All facilitators noted that they feel that the simulations improve practice and the abilities and confidence of participants but that it is difficult to match this to improved patient outcomes. Facilitator 3 also stated that she was unsure if improved patient outcomes was the appropriate measure since sometimes staff efforts can be perfect and still result in a negative patient outcome simply because of compounding patient factors. When asked how they measure improvements in patient care and staff knowledge the response was very similar to the above, with questionnaires and statements made by the staff about increased confidence and knowledge. Participants also frequently cite an improvement in communication among team members and clarity of roles in emergency situations. This is again very difficult to quantify but as facilitators rarely have enough time to perform as many the simulations as they would like to the development of better tools and the collection of more pertinent data is likely a barrier.
With regards to fidelity among those we interviewed one uses high fidelity, two use mid fidelity and one uses low fidelity mannequins. Each facilitator supplements this with the most realistic supplies available to them, which generally include a real code cart supplied with expired or simulated medications and monitors for displaying vital signs. Facilitator 3 was using a mid-fidelity mannequin until the unit purchased a high-fidelity Sim-baby in 2008. When the mannequin was to arrive the unit named the baby and had a baby shower for her, which included gifts of clothing and diapers. Facilitator 2 believes that this has greatly increased the engagement of those on the unit. When talking with a nurse in the PICU that started in 2008 she could clearly recall participating in simulations during her orientation, which she believes has helped with staff engagement. Facilitator 2 has noticed more engagement over time which she attributes to generational changes. Facilitator 2 stated, "They have simulation as part of their education now so when they begin on the unit they are not surprised to have simulation as part of their training and that has really improved the response." She notes that the new generation of hospital staff have used simulation throughout their education which increases their comfort with this type of education.
When observing a simulation with high fidelity and taking into consideration past personal experience with this level of simulation we noticed some drawbacks. Staff felt for pulses in all locations of the mannequin even though pulses are only ever palpable in a couple of locations. Also, the mannequin displays some signs of cyanosis however as the facilitator described the symptoms of cyanosis increasing the staff did not respond to this since they could not visualize the change and they expected to. In a real life situation these discrepancies would not exist. Personal experience has also provided us with the insight that sometimes the mannequins responses either verbally or physically are unintentional and that perhaps this level of fidelity gives the less experienced facilitator less of an active role which can perpetuate this. High fidelity is intended to increase the realism however drawbacks and areas of concern must still be monitored.
Both Facilitator 1 and Facilitator 4 use mid-fidelity mannequins. Both would like to use high fidelity but cite the cost as a barrier. They also both feel very strongly that the benefits of simulation exist even without high fidelity equipment. Facilitator 4 says, "It is all about suspending their disbelief", pointing out that if you can get them to treat the mannequin as though it is a real patient then the level of fidelity is not important. Facilitator 3 uses low fidelity mannequins, however she has a mannequin that is overweight to increase the realism and give participants the opportunity to practice on a mannequin that more closely reflects their patient population. She cites the biggest barriers to using a low fidelity mannequin as decreasing the realism and the inability of participants to practice skills, however she also notes that benefits to the simulation still exist. Facilitator 4 considers what he wants participants to learn before he begins to build the scenario and then matches the supplies and level of fidelity to the learning objectives. This seems to be an excellent model for determining the level of fidelity that would be most beneficial.
Another point that we noted facilitators varied on is the ultimate outcome of their scenario. Facilitator 3 found that if there is a negative patient outcome in real life even if staff did everything correct they will have regrets about the situation. With this in mind her scenarios often end with a negative patient outcome regardless of whether or not participants did the right things. She also feels it is important that they have the sense of realism that if they don't do things right there will most likely be a negative patient outcome. Both Facilitator 2 and 4 do not allow the scenario to end with death of the mannequin. If participants are not making the correct decision they either provide some guidance or they continue to decrease the vital signs until the correct action is made. This way at the end of the scenario positive feelings about the situation exist. Facilitator 1 says that most of her participants have fun and that by making simulation enjoyable it encourages participation. Facilitator 2 always brings candy to end the scenario to relieve the stress of participants. She believes this is one of the most important expenditures in her budget.
Each of the facilitator's interviewed work for the University of Virginia Health System and use Epic as their EMR documentation system. They have access to a nonproduction environment of the EMR and have the new addition of the Code Narrator to document emergency procedures in. The facilitator's engage this new system and encourage participants to utilize this for the documentation of the event, however it was noted that participants struggle to actually use the EMR for documentation. With a lack of familiarity with this area of the EMR and the stress of the situation it becomes overwhelming however the practice in this low stakes environment is beneficial to their practice. Online training has been provided to users but more practice is needed. This is an area of practice that could be easily measured to determine if simulations are making improvements in practice by improving documentation in the EMR.

Discussion
In-situ simulation was identified as a method of providing simulation in the participants' practice environment. It is beneficial as it provides the opportunity for a realistic team and the ability to gain familiarity with the availability and location of supplies and support. Challenges to this type of simulation include availability of a location and hesitancy in participation because it disrupts the workday. We found that most facilitators collect data that is based on how participants feel both about the simulation and about the knowledge and experience. This type of data does not help to assess if this translates to improvement in patient outcomes or in real life clinical responses by clinicians, as this type of data is very difficult to obtain and analyze. Facilitator 3 noted that as she has performed simulations over the last decade that her participants are increasingly easier to engage. She attributes this to progression of educational programs to incorporate simulation into education. We found that the level of fidelity used varied but that all facilitators believed that high fidelity translated into greater realism. We appreciate Facilitator 4's response that he matches his scenario to what he wants participants to learn and tries to match the fidelity of the scenario to these outcomes. This implies that perhaps high fidelity is not necessarily the goal but making a match between the fidelity level and the educational objectives is. It is also notable that the mannequins' outcome varied based on the facilitators' goals and beliefs and not so much on the actions of the participants.

Weakness and recommendations for future research
Several weaknesses exist in our research and we wish to use them as suggestions for future research. In-situ simulation is an uncommonly documented type of simulation in research. This may be because of the challenges in gaining participants and interruptions in patient care, however we believe that it is a very useful simulation setting. With that in mind it was difficult to procure a large base for our literature review. We did not actually conduct and analyze simulations as part of our study which is both a strength and a weakness. It is a strength because it enabled us to be unbiased about what we learned but a weakness as most information was gained second hand from facilitators. Another weakness of this study is the lack of quantifiable data about the simulations conducted. This is not a weakness unique to our study but it does make an excellent challenge to the future research of simulation. Although the gold standard for determining the effectiveness of simulation is the improvement in patient outcomes we are now unsure if this is truly a good measure. Even if hospital staff do everything right a patient may still have a negative outcome simply due to the nature of their clinical condition. With this in mind improvement in response and intervention times may be a more accurate measure of staff improvement. Additionally, we only interviewed four facilitators which broadened our knowledge however a greater number would have improved our findings.

Conclusions
The focus of clinical simulation is clear: improve staff response to emergency situations to improve patient outcomes. From this point of knowledge the variables in types of simulation and its implementation vary widely. It is widely acknowledged in research and echoed in our findings that outcomes resulting from simulation include improvement in clinician confidence and communication among team members. What is less clear is how these results translate into patient outcomes or clinician responses in real life situations due to difficulties in data collection regarding these areas. Also identified in our literature review and our interviews was varying levels of fidelity and thoughts about how this translates into the realism of the scenario. Most facilitators believe that higher fidelity means increased realism but true data documenting that is lacking, at least among the literature we reviewed and in our interviews. Another area of emergency situations that is important yet difficult to practice is the documentation. Accurate documentation is important for the tracking of medical interventions but without the ability to practice this in the EMR staff are unlikely to be able to do so accurately when the time comes.