Buscar

... IMPACT OF COGNITIVE AIDS ON SIMULATION 1 Marian University Leighton School of Nursing Doctor of Nursing Practice The Impact of Anesthesia Machine Check Cognitive Aids on Student Registered Nurse Anesthetist Simulation Edward Erazo Submitted in partial fulfillment of degree requirements for the Doctor of Nursing Practice Nurse Anesthesia Marian University Leighton School of Nursing Chair: Dr. Sara Franco _________________________3/12/2023 (Signature) Committee member: (Date) Dr. Nicholas Jones _________________________3/10/2023 (Signature) Date of Submission: February 1st, 2023 (Date) IMPACT OF COGNITIVE AIDS ON SIMULATION Table of Contents Abstract ........................................................................................................ ...3 Introduction ................................................................................................................... 4 Background ......................................................................................................... 5 Problem Statement............................................................................................... 7 Needs Assessment/Gap Analysis ......................................................................... 7 Review of the Literature .................................................................................................. 8 Theoretical Framework.......12 Project Aim/Objectives...13 Project Design/Setting/Population...13 Data Collection Procedure ........................................................................ ..14 Ethical Considerations/Protection of Human Subjects ........................................ 15 Project Evaluation Plan ............................................. .15 Data Analysis and Results......16 Discussion...19 Conclusion ................................................................................................................... 19 References .................................................................................................................... 21 Appendices Appendix A ....................................................................................................... 26 Appendix B ....................................................................................................... 30 Appendix C.31 Appendix D.32 2 IMPACT OF COGNITIVE AIDS ON SIMULATION 3 Abstract Cognitive aids have been shown to enhance clinical performance and improve patient care by increasing the completion of essential tasks and facilitating clinical decision-making. Cognitive aids have also been demonstrated to be effective teaching tools. Simulation is a crucial teaching exercise that allows student registered nurse anesthetists (SRNAs) to practice clinical decision-making without risking patient harm. The anesthesia machine check simulation can be improved by adding cognitive aids. An anesthesia machine check cognitive aid would simplify the learning process and lead to a more productive simulation experience. This project aims to provide SRNAs with anesthesia machine check cognitive aids to assess if this would improve SRNA knowledge, satisfaction, and self-confidence regarding anesthesia machine check simulation exercises. Knowledge was assessed using a five-question test designed to measure anesthesia machine knowledge (Appendix C). SRNA satisfaction and self-confidence were measured using the National League for Nursings student satisfaction and self-confidence in learning survey (SSSL) (Appendix D). Participants were divided into control and experimental groups. Pretest and posttest knowledge and SSSL survey results were analyzed, and found no statistical significance among the groups (p < 0.05). Limitations of this study were the small sample size and poor coordination with the simulation faculty. Further research regarding the implications of cognitive aids in simulation is needed to determine their impact on knowledge, satisfaction, and self-confidence. Keywords: cognitive aids, simulation, anesthesia machine check, knowledge, satisfaction, selfconfidence. IMPACT OF COGNITIVE AIDS ON SIMULATION 4 Introduction Simulation is a significant component of an SRNA's education. Simulation provides a safe environment where an SRNA's clinical knowledge and decision-making skills can be challenged without placing a patient at risk. In a simulation, SRNAs can practice as independent anesthesia providers without the stressful psychological and emotional influences of working with difficult providers, staff members, and patients. Therefore, simulation is an excellent teaching tool for SRNAs to build foundational skills and help assess for potential improvements. Refining simulation through implementing an algorithm-based cognitive aid will facilitate the education of complex concepts. Every anesthesia provider encounters learning the lengthy steps of performing an anesthesia machine check and running through a pre-anesthesia checklist. It is an essential daily task performed by anesthesia providers to ensure the anesthesia machine is ready to provide safe care. Like the American Heart Association's adult cardiac arrest algorithm, an algorithm-based cognitive aid can be developed to facilitate SRNA anesthesia machine education (American Heart Association, 2020). This revolutionary learning tool can simplify the learning process of memorizing lengthy chronological tasks. A cognitive aid can streamline the anesthesia machine check learning process. This will enable SRNAs to learn this complicated, critical skill with decreased stress, resulting in improved knowledge, satisfaction, and self-confidence. A United Kingdom National Reporting and Learning Service review of 12,606 incidents found that 13% of the incidents reported were related to anesthesia device malfunctions (Catchpole et al., 2008). Anesthesia machine malfunctions and problems can be found by performing a thorough check. Improving the simulation of anesthesia machine checks is also imperative because research IMPACT OF COGNITIVE AIDS ON SIMULATION 5 studies have found that anesthesia providers of all experience levels still cannot detect all the defects when asked to check a machine with five preset faults (Larson et al., 2007). This research project aims to determine if an algorithm-based cognitive aid would improve anesthesia machine knowledge, self-confidence, and satisfaction in simulation. An anesthesia machine cognitive aid was developed and only distributed to half of the study's participants. Pretest, posttest, and survey scores were examined and compared between the experimental and control group. Background The anesthesia machine is a medical device designed to safely provide oxygen, air, volatile anesthetic agents, and ventilation to patients undergoing surgical procedures (Sherwin & Eisenkraft, 2020). The anesthesia machine is used to titrate inhaled anesthetic gas mixtures to produce ideal surgical conditions and provide patient comfort. Anesthesia machines are complex and can be divided into high, intermediate, and low-pressure systems. The difference in pressures directs the flow of oxygen and other gases. Anesthesia machines safely deliver precise anesthetic gas concentrations using calibrated vaporizers, flowmeters, carbon dioxide absorbers, breathing circuits, and check valves (Sherwin & Eisenkraft, 2020). Anesthesia machine parts must be thoroughly examined through an anesthesia machine check before providing anesthesia care. A faulty component can cause a leak, hypoxic mixture, or other malfunction, putting the patient at risk for harm. Therefore, it is paramount that anesthesia providers perform in-depth anesthesia machine checks to ensure patient safety. Inadequate anesthesia machine checks are an under-reported problem (Larson et al., 2007). The many different anesthesia machines available in the United States make the implementation of a single anesthesia machine check algorithm challenging. The Food and Drug IMPACT OF COGNITIVE AIDS ON SIMULATION 6 Administration (FDA) first issued a recommended pre-anesthesia checklist in 1993 to improve patient safety by promoting thorough anesthesia machine checks (Feldman et al., 2008). In 2008, the American Society of Anesthesiologists (ASA) released Recommendations for PreAnesthesia Checkout Procedures, a 16-page document with instructions on performing a full anesthesia machine check (Feldman et al., 2008). Despite these resources, many anesthesia providers still perform insufficient machine checks (Larson et al., 2007; Buffington et al., 1984). Anesthesia providers develop their own routine for performing the anesthesia machine check leading to an increase in unwarranted clinical variance. This variation in machine checks increases the risk that the provider may overlook a defect in the machine. For example, a study of 87 anesthesia providers found that the average anesthesia provider could only find 3.1/5 preset faults when performing an anesthesia machine check (Larson et al., 2007). The same study found that only 10 participants detected all five faults, and 3 found no flaws (Larson et al., 2007). This study reveals the alarming problem that numerous anesthesia providers fail to perform a complete and thorough anesthesia machine check (Larson et al., 2007). A similar study with 190 anesthesia providers at a New York State Society of Anesthesiologists meeting found that 7.3% of participants could not detect any faults. Only 3.4% of the providers could detect all five anesthesia machine faults (Buffington et al., 1984). An example of this problem occurred in Wisconsin when a patient had a coronary artery bypass graft and had to be bag-mask ventilated while the anesthesia machine was swapped out after induction (Ezaru, 2006). The anesthesiologist did not detect a leak from the carbon dioxide absorber because of an incomplete machine check (Ezaru, 2006). The variation in performing an anesthesia machine check leads to incomplete checks putting patients at risk for avoidable, unfortunate events. IMPACT OF COGNITIVE AIDS ON SIMULATION 7 The introduction of an algorithm-based cognitive aid for anesthesia machine checks may facilitate the learning process of this critical task and produce anesthesia providers who perform thorough machine checks consistently. Providing SRNAs with an anesthesia machine check cognitive aid will improve the education of this complex task, increasing SRNA knowledge, selfconfidence, and satisfaction. Problem Statement Anesthesia providers are responsible for ensuring that the anesthesia machine is ready to provide safe care before every procedure. Unfortunately, anesthesia providers may either not be consistent or knowledgeable in performing complete machine checks, increasing the risk of a faulty anesthesia machine being used (Larson et al., 2007). Various studies and anecdotes support the idea that anesthesia providers should receive periodic educational briefings on anesthesia machine checks (Ezaru, 2006; Larson et al., 2007). By developing an anesthesia machine check cognitive aid, anesthesia providers may adopt a more consistent routine of thorough anesthesia machine checks. In addition, improving the anesthesia machine check learning process may lead to novice anesthesia providers becoming knowledgeable in detecting all faults with anesthesia machines when conducting machine checks. Needs Assessment & Gap Analysis The project site does not utilize an anesthesia machine check cognitive aid to prepare SRNAs for anesthesia machine check simulation. The project sites practice prepares SRNAs for anesthesia machine check simulation with textbooks, online resources, hands-on practice, and professor instruction. A gap analysis was completed through an examination of the nurse anesthesia program. The gap analysis revealed no utilization of an anesthesia machine check algorithm-based cognitive aid because of a lack of awareness of the educational tool. Research IMPACT OF COGNITIVE AIDS ON SIMULATION 8 suggests cognitive aids facilitate the learning process and improve knowledge among anesthesia providers (Groves et al., 1994; Suet et al., 2021; Neal et al., 2012). Literature Review Search Methodology This literature review was conducted to examine articles regarding the utilization of algorithm cognitive aid teaching tools to improve anesthesia machine check education in adults studying anesthesia. The review was conducted using the keywords algorithm, concept map, cognitive aid, teaching tool, flowchart, anesthesia, education, simulation, knowledge, satisfaction, and self-confidence. This review was conducted in January 2022 using the databases PubMed and Medline-Ovid. The database searches were performed using the BOOLEAN phrases algorithm OR concept map OR flowchart OR cognitive aid OR teaching tool, and anesthesia OR education OR simulation. The seven hundred sixty-nine database search results were reduced to exclude secondary research designs, case studies, pediatric patients, and animal research. The remaining research studies were examined to determine if the studies met the inclusion criteria. The search inclusion criteria included articles written or translated into English. Of the seventy-two articles examined, fifty-nine articles were excluded due to unrelated research objectives and case study design. The research articles for this literature review were reduced to 14 level 2 and level 3 studies that emphasized the impact of cognitive aids, algorithms, and checklists on performance, knowledge, and participant perception. The literature review matrix is displayed in Appendix A. Impact on clinical performance Twelve of the fourteen studies examined the influence of cognitive aids on clinical performance. Eleven of the twelve studies found that cognitive aids enhanced clinical IMPACT OF COGNITIVE AIDS ON SIMULATION 9 performance and may facilitate superior patient care by increasing completion of essential tasks and improving clinical decision-making (Beck et al., 2018; Combes et al., 2004; Harrison et al., 2006; Hart & Owen, 2005; Heidegger et al., 2001; Neal et al., 2012; St. Pierre et al., 2017; Ward et al., 1997; Watkin et al., 2016; Wetmore et al., 2016; Zeiwacz et al., 2011). These studies evaluated performance in clinical and simulated settings by assessing numbers of omitted actions and errors. A randomized controlled trial consisting of 25 anesthesiologists participating in a simulated local anesthetic systemic toxicity scenario found that those equipped with the checklist cognitive aid were able to significantly outperform those without (P<0.001) (Neal et al., 2012). Participants with the cognitive aid averaged 16 essential tasks completed compared to the control group, who averaged 8.8 (Neal et al., 2012). Another randomized control trial of 26 anesthesia residents in two university hospitals found that implementing a checklist cognitive aid significantly increased the completion of safety checks immediately and four weeks after implementation (P=0.003) (Beck et al., 2018). Also, a study evaluating the impact of a cognitive aid on the clinical performance of 48 anesthesia residents during a malignant hyperthermia simulation using a malignant hyperthermia treatment score scale found that the residents who scored the highest were the ones who most frequently relied on the cognitive aid (P<0.001) (Harrison et al., 2006). Another simulation study found that implementing an operating room crisis cognitive aid checklist decreased the incidence of missed critical tasks six-fold (p=0.007) (Ziewacz et al., 2011). In this study, teams with the cognitive aid missed 2 out of 46 essential steps compared to those without the cognitive aid, who missed 11 crucial steps (Ziewacz et al., 2011). Two observational studies found that implementing a difficult airway cognitive aid resulted in decreased failed intubations (Combes et al., 2004; Heidegger et al., 2001). IMPACT OF COGNITIVE AIDS ON SIMULATION 10 Bould et al.'s (2009) neonatal resuscitation simulation randomized control trial was the only research study that resulted in no significant difference in clinical performance between anesthesia residents with and without cognitive aids (Bould et al., 2009). The researchers believed both groups performed similarly due to the experimental group's lack of cognitive aid use (Bould et al., 2009). All twelve studies evaluating cognitive aid's impact on clinical performance emphasized that further research on the implementation of cognitive aids in anesthesia is weak as most research has been performed in simulated settings. However, evidence provided by studies utilizing cognitive aids in simulation scenarios have found a strong relationship between the use of cognitive aids and improved performance and safety. Improved Knowledge Two of the fourteen articles examined the relationship between cognitive aids and participant knowledge (Neal et al., 2012; Suet et al., 2021). Both studies found that groups with cognitive aids had increased medical knowledge gain compared to those without. Neal et al. (2012) examined the knowledge retention of 25 anesthesia residents two months after the distribution of a cognitive aid and observed that those with the aid said scored higher knowledge test scores (P=0.031). Similar findings were found in an 89-anesthesia resident randomized control trial which examined the influence of a cognitive aid on an observer's medical knowledge gain after viewing a simulation scenario (Suet et al., 2021). This study found that those using the cognitive aid scored a mean of 11.4 on the medical knowledge test compared to the control group, who scored 9.6 (P=0.0008) (Suet et al., 2021). This is a critical study because it focused on the cognitive aids' influence on the education of anesthesia residents and found a significant IMPACT OF COGNITIVE AIDS ON SIMULATION 11 link to improved medical knowledge gain. This study provides evidence that cognitive aids can be used as teaching tools. Non-technical Skills Three of the fourteen studies evaluated cognitive aid's effect on the participant's nontechnical skills using the Anaesthetists' Non-Technical Skills (ANTS) scoring system (Marshall & Mehra, 2014; Neal et al., 2012; Suet et al., 2021). The ANTS scoring system is a measuring instrument used to assess the degree of team management, teamwork, situational awareness, and decision-making skills (Marshall & Mehra, 2014). A study of 64 participants undergoing a difficult airway simulation found that participants equipped with the cognitive aid had high ANTS scores in every category (p=0.002) (Marshall & Mehra, 2014). The study concluded that cognitive aids enhanced non-technical skills during airway emergencies. Similarly, Neal et al. (2012) analyzed 25 anesthesiologists undergoing a simulation with cognitive aids. They found that the cognitive aid group had higher ANTS decision-making scores averaging 5.2 compared to 4.0 (p=0.037) (Neal et al., 2012). Suet et al.'s (2021) study of 89 anesthesia residents found no difference in ANTS scores between residents who received cognitive aids and those who did not. These three studies reported different findings when measuring cognitive aid's impact on nontechnical skills. Provider Satisfaction Three of the fourteen studies examined cognitive aid's influence on the participant's satisfaction of the simulation (Hart & Owen, 2005; Suet et al., 2021; Ziewacz et al., 2011). Ziewacz et al. (2011) found that participants with checklist cognitive aid on average felt better prepared for emergencies (4.2+/- 0.95) and would utilize cognitive aids if present in medical centers (4.3+/-0.75). Similar findings were reported in an observational study where 20 IMPACT OF COGNITIVE AIDS ON SIMULATION 12 anesthesiologists experienced a general anesthesia simulation for cesarean delivery with and without a cognitive aid (Hart & Owen et al., 2005). 95% of participants felt the cognitive aid was helpful, and 80% of participants would want to use the cognitive aid if presented with a simulation again (Hart & Owen et al., 2005). The providers who experience the guidance of cognitive aids agree that they are useful for improving patient care and safety; therefore, increasing provider satisfaction. Theoretical framework Kolb's theory of experiential learning was the theoretical framework used for this project (Kolb, 1984). Kolb's theory of experiential learning consists of a learning cycle of four phases: phase 1 concrete experience, phase 2 reflective observation, phase 3 abstract conceptualization, and phase 4 active experimentation (Murray, 2018). Kolb theorized that students must encounter all four stages of learning to learn from their experiences (Murray, 2018). During this research project, students will undergo the concrete experience phase when simulating an anesthesia machine check by receiving a hands-on, psychomotor experience (Murray, 2018). Next, participants will experience the reflective observation phase when recollecting the simulation experience and reflecting on how they feel about it. Students will encounter the abstract conceptualization phase when critically thinking and making significant intellectual connections regarding anesthesia machine check troubleshooting. Lastly, in the active experimentation phase, the students will practice their new knowledge in simulation or clinical and reaffirm the concepts they learned in the beginning stages. A diagram of Kolb's theory of experiential learning can be seen in Appendix B. IMPACT OF COGNITIVE AIDS ON SIMULATION 13 Project Aim 1. The project aim is to determine if an algorithm-based cognitive aid teaching tool will improve anesthesia machine check simulation knowledge, self-confidence, and satisfaction in student registered nurse anesthetists. Project Objectives 1. Develop an algorithm cognitive aid teaching tool to simplify anesthesia machine check education 2. Provide this tool to the experimental group of students seven days before anesthesia machine check simulation exercise. 3. Design a five-question anesthesia machine check knowledge test (Appendix D) to collect data within 15 minutes before the start of the simulation exercise and again within 10 minutes after. 4. Utilize the National League for Nursings student satisfaction and self-confidence in learning survey (Appendix D) to collect data within 15 minutes before and 10 mins after the simulation exercise. 5. Perform statistical analysis to examine the differences in knowledge, satisfaction, and self-confidences test scores between both groups, pre- and post-simulation. Project Design/ Setting/ Population This project is a quasi-experimental research study designed to examine the relationship between an algorithm cognitive aid teaching tool and SRNA knowledge, satisfaction, and selfconfidence in anesthesia machine check simulation. The anesthesia machine simulation exercise would be held in a simulation lab in a nursing school in the Midwest of the United States. A nursing program simulation professor would conduct the simulation exercise. This research IMPACT OF COGNITIVE AIDS ON SIMULATION 14 project would be conducted using first year, third semester SRNAs recently introduced to an anesthesia machine. The SRNAs will already have a traditional anesthesia machine education through textbooks, online resources, and hands-on opportunities. Data collection procedure/ Measurement Instruments The study will consist of an anonymized demographic questionnaire, knowledge, satisfaction, and confidence surveys done before and after the anesthesia machine check exercise. Seven days before the scheduled simulation exercise, thirty-four SRNAs will receive a recruitment email with information on the DNP project. Embedded in this recruitment statement will be a Qualtrics link where potential participants will be directed to the implied consent page. Once participants acknowledge the study's implied consent, the participants will proceed to complete the pre-test consisting of the demographic questionnaire, knowledge test, and satisfaction and self-confidence survey. The demographic questionnaire collects information on the participants' age, gender, marital status, highest level of education, years of clinical experience. Finally, half the participants will be assigned to the experimental group and receive the algorithm cognitive aid teaching tool to study in preparation for the anesthesia machine check simulation. The remaining seventeen participants will be assigned to the control group will be performing the simulation without the cognitive aid. The anesthesia machine check simulation is a mandatory exercise as part of the NSG 607S course. Participation in the DNP project is completely voluntary. Immediately after the simulation exercise, the participants will complete the post-test answering the same questions as the pre-test. The instrument used to collect the pre- and post-test data is Qualtrics. Participant confidentiality will be maintained throughout the study using Qualtrics anonymize setting. The measurement tool used to measure knowledge will be a principal investigator-created 5-question IMPACT OF COGNITIVE AIDS ON SIMULATION 15 quiz validated by DNP committee members. The measurement tool to be used to evaluate satisfaction and self-confidence will be the National League for Nursing's Student Satisfaction and Self-Confidence in Learning Scale. Ethical Considerations/Protection of Human Subjects Involvement in this study was low risk. The students acknowledged the informed consent statement prior to participating in the study. Research project participation was voluntary and did not risk causing physical or mental health concerns. The study utilized anonymous surveys and did not request personal information. The information collected for this study was secured in a password-protected laptop only accessible to the research organizers. Internal Review Board approval was granted before beginning the DNP project. Project Evaluation Plan This research project's data collection consisted of demographic information, knowledge test scores, and Student Satisfaction and Self-Confidence in Learning Scale scores before and after the simulation exercise. The Qualtrics software will store all of the data to be translated into a statistical software program. This program will record all quantitative data from the test and survey scores and organize the demographic data into subgroups. A paired t-test will be used to examine pretest and posttest scores within the control and experimental group. An independent samples t-test will be used to evaluate if there is a significant relationship between the implementation of the algorithm cognitive aid and increased knowledge, satisfaction, and selfconfidence scores. Using the collective average score of each pre- and post-simulation survey question, independent sample t-test can determine if there is a correlation between the cognitive aid and improved scores. Data Analysis and Results IMPACT OF COGNITIVE AIDS ON SIMULATION 16 Demographics This project consisted of 13 participants who completed the knowledge test, SSSL pretest, and posttest surveys. Of the 13 participants, there were a total of 10 females (77%) and three males (23%). Seven of the 13 participants reported being 26-30 years old (54%), followed by 4 participants reporting being 31-35 years old (31%), and two participants reporting being over 35 years old (15%). In addition, eight participants (62%) reported being single, and five (38%) reported being married. One participant reported having a masters degree (8%), while the remaining 12 participants reported having a bachelors degree (92%) as their highest level of education. Seven of the participants were randomized into the experimental group (54%) and six were randomized into the control group (46%). Knowledge Test The five-item knowledge test was designed to examine the participants baseline and change in knowledge regarding anesthesia machine checks (Appendix C). The control groups average pretest score was 3.83, and the average posttest score was 3.83. When performing a paired t-test, the results showed no statistically significant change among the control groups pretest and posttest scores (p > 0.05). The experimental groups average pretest score was 3.71, and the average posttest score was 3.85. When performing a paired t-test, the results showed no statistically significant change among the experimental groups pretest and posttest scores (p > 0.05). The data revealed no statistical significance when performing an unpaired t-test comparing the experimental and control groups posttest scores (p > 0.05). Figure 1 Average knowledge pretest and posttest scores IMPACT OF COGNITIVE AIDS ON SIMULATION 17 Average Knowledge Scores 3.9 3.85 3.8 3.83 3.85 3.83 3.75 3.7 3.71 3.65 3.6 Control Pretest Control Posttest Exp Pretest Exp Posttest Control Group Pretest to Posttest Comparison When performing a paired t-test analysis of the control groups pretest and posttest results, the data revealed that the anesthesia machine check simulation without the cognitive aid resulted in statistically significant improved self-confidence (p < 0.05) and an insignificant change in satisfaction (p > 0.05). The control groups average satisfaction score increased from 17.5 on the pretest to 20.8 on the posttest (p > 0.05). In addition, the control groups average self-confidence scores improved from 29.8 on the pretest to 33.7 on the posttest resulting in a statistically significant change (p < 0.05). Of the 13 questions of the SSSL, questions six and eight resulted in a statistically significant change (p < 0.05) from the pretest to the posttest. Question six, I am confident that I am mastering the content of the simulation activity that my instructors presented to me, had an average pretest score of 3.17 and an average posttest score of 4 (p < 0.05). Question eight, I am confident that I am developing the skills and obtaining the required knowledge from this simulation to perform necessary tasks in a clinical setting, had an average pretest score of 3.67 and an average posttest sore of 4.33 (p < 0.05). Experimental Group Pretest to Posttest Comparison When performing a paired t-test analysis of the experimental groups pretest and posttest results, the data revealed an insignificant change in satisfaction scores (p > 0.05) and statistically significant improved self-confidence scores (p < 0.05). The experimental groups average IMPACT OF COGNITIVE AIDS ON SIMULATION 18 satisfaction score increased from 19 on the pretest to 21.4 on the posttest (p > 0.05). Also, the experimental groups average self-confidence scores improved from 29.5 on the pretest to 34.4 on the posttest, resulting in a statistically significant change (p < 0.05). Of the 13 questions of the SSSL, questions six, eight, and 12 resulted in a statistically significant change (p < 0.05). Question six, I am confident that I am mastering the content of the simulation activity that my instructors presented to me, had an average pretest score of 3.29 and an average posttest score of 4.29 (p < 0.05). Question eight, I am confident that I am developing the skills and obtaining the required knowledge from this simulation to perform necessary tasks in a clinical setting, had an average pretest score of 3.43 and an average posttest score of 4.29 (p < 0.05). Question 12, I know how to use simulation activities to learn critical aspects of these skills, had an average pretest score of 3.71 and an average posttest score of 4.43 (p < 0.05). Comparison of Control and Experimental Group Posttests The study revealed no statistically significant difference when performing an independent, unpaired t-test comparing the control and experimental groups satisfaction and self-confidence posttest scores (p > 0.05). In both posttests, the experimental group had a slightly higher average score, but not enough to be statistically significant. The average experimental group satisfaction and self-confidence posttest score were 21.4 and 34.4, respectively, compared to the control group, which was 20.8 and 33.6. Figure 2 Average SSSL satisfaction pretest and posttest scores Average Satisfaction Scores 25 20 15 10 5 0 17.5 Control Pretest 20.8 19 Control Posttest Exp Pretest 21.4 Exp Posttest IMPACT OF COGNITIVE AIDS ON SIMULATION 19 Figure 3 Average SSSL self-confidence pretest and posttest scores Average Self-confidence Scores 35 34 33 32 31 30 29 28 27 34.4 33.6 29.8 Control Pretest 29.5 Control Posttest Exp Pretest Exp Posttest Discussion The strengths of this project were simple, digital data collection, simulation faculty support, and coordinated anesthesia machine simulation schedule. Limitations of the study were small sample size, flexibility of students to coordinate simulation checkoffs earlier or later than scheduled, potential communication between students participating in the study, and student disinterest in study due to other academic responsibilities. Due to the limitations of this study, it is uncertain if cognitive aids improved SRNA knowledge, satisfaction, and self-confidence. Recommendations for this study are to have improved coordination and participation of the simulation faculty to increase the sample size. Another recommendation is to plan the anesthesia machine simulation exercise at a time when this exercise can be focused on so students are not distracted by concerns for exams occurring shortly after the simulation. Future studies on cognitive aids impact on simulation should include larger sample sizes. IMPACT OF COGNITIVE AIDS ON SIMULATION 20 Conclusion Cognitive aids simplify the learning process and make the memorization of crucial information less difficult for the student. An algorithm-based cognitive aid can streamline imperative simulation exercises like anesthesia machine checks. The improved method of education that cognitive aids provide enables SRNAs to participate in simulation with decreased stress, resulting in improved knowledge, satisfaction, and self-confidence. Simulation provides a safe environment where SRNAs can practice as independent anesthesia providers without the risk of patient harm. Therefore, simulation is an excellent educational exercise for SRNAs to build foundational skills and assess for improvements. The addition of cognitive aids may improve the learning process and provide students with better satisfaction and self-confidence after their simulation. After reviewing the analysis of the knowledge, satisfaction, and self-confidence pretest and posttest scores, the study results are inconclusive. This study did not find a statistically significant relationship between cognitive aids and improved knowledge, satisfaction, or selfconfidence for the anesthesia machine check simulation. Further research regarding the benefits of cognitive aids in simulation and anesthesia education is necessary to determine whether these tools can be utilized to improve knowledge, satisfaction, and self-confidence. IMPACT OF COGNITIVE AIDS ON SIMULATION 21 References American Heart Association. (2020). Adult cardiac arrest algorithm. https://cpr.heart.org//media/CPR-Files/CPR-Guidelines-Files/Algorithms/AlgorithmACLS_CA_200612.pdf Beck, S., Reich, C., Krause, D., Ruhnke, B., Daubmann, A., Weimann, J., Zollner, C., Kubitz, J. (2018). For beginners in anaesthesia, self-training with an audiovisual checklist improves safety during anaesthesia induction: A randomised, controlled two-centre study. European Journal of Anaesthesiology, 35, 527-533. https://dx.doi.org/10.1097/EJA.0000000000000781 Bould, M. D., Hayter, M. A., Campbell, D. M., Chandra, D. B., Joo, H. S., & Naik, V. N. (2009). Cognitive aid for neonatal resuscitation: A prospective single-blinded randomized controlled trial. British Journal of Anaesthesia, 103(4), 570575. https://doi.org/10.1093/bja/aep221 Buffington, C. W., Ramanathan, S., & Turndorf, H. (1984). Detection of anesthesia machine faults. Anesthesia and analgesia, 63(1), 7982. Catchpole, K., Bell, M. D., & Johnson, S. (2008). Safety in anaesthesia: A study of 12,606 reported incidents from the UK National Reporting and Learning System. Anaesthesia, 63(4), 340346. https://doi.org/10.1111/j.1365-2044.2007.05427.x Combes, X., Le Roux, B., Suen, P., Dumerat, M., Motamed, C., Sauvat, S., Duvaldestin, P., & Dhonneur, G. (2004). Unanticipated difficult airway in anesthetized patients: Prospective validation of a management algorithm. Anesthesiology, 100(5), 11461150. https://doi.org/10.1097/00000542-200405000-00016 IMPACT OF COGNITIVE AIDS ON SIMULATION 22 Ezaru C. S. (2006). Preinduction check-up of the anesthesia machine. Anesthesia and analgesia, 102(5), 15881589. https://doi.org/10.1213/01.ANE.0000215201.76420.57 Feldman, J. M., Olympio, M. A., Martin, D., & Striker, A. (2008). New guidelines available for pre-anesthesia checkout. Anesthesia Patient Safety Foundation, 23(1), 6-7. https://www.apsf.org/wp-content/uploads/newsletters/2008/spring/pdf/APSF200803.pdf Groves, J., Edwards, N., & Carr, B. (1994). The use of a visual aid to check anaesthetic machines. Is performance improved?. Anaesthesia, 49(2), 122125. https://doi.org/10.1111/j.1365-2044.1994.tb03367.x Harrison, T. K., Manser, T., Howard, S. K., & Gaba, D. M. (2006). Use of cognitive aids in a simulated anesthetic crisis. Anesthesia and Analgesia, 103(3), 551556. https://doi.org/10.1213/01.ane.0000229718.02478.c4 Hart, E. M., & Owen, H. (2005). Errors and omissions in anesthesia: a pilot study using a pilot's checklist. Anesthesia and analgesia, 101(1), 246-250. https://doi.org/10.1213/01.ANE.0000156567.24800.0B Heidegger, T., Gerig, H. J., Ulrich, B., & Kreienbhl, G. (2001). Validation of a simple algorithm for tracheal intubation: daily practice is the key to success in emergencies: An analysis of 13,248 intubations. Anesthesia and analgesia, 92(2), 517522. https://doi.org/10.1097/00000539-200102000-00044 Kolb, D. A. (1984). Experiential Learning: Experience as the Source of learning and development. Prentice-Hall. IMPACT OF COGNITIVE AIDS ON SIMULATION 23 Larson, E. R., Nuttall, G. A., Ogren, B. D., Severson, D. D., Wood, S. A., Torsher, L. C., Oliver, W. C., & Marienau, M. E. (2007). A prospective study on anesthesia machine fault identification. Anesthesia and analgesia, 104(1), 154156. Marshall, S. D., & Mehra, R. (2014). The effects of a displayed cognitive aid on non-technical skills in a simulated 'can't intubate, can't oxygenate' crisis. Anaesthesia, 69(7), 669677. https://doi.org/10.1111/anae.12601 Murray, R. (2018). An overview of experiential learning in nursing education. Advances in Social Sciences Research Journal, 5(1). doi:10.14738/assrj.51.4102 Neal, J. M., Hsiung, R. L., Mulroy, M. F., Halpern, B. B., Dragnich, A. D., & Slee, A. E. (2012). ASRA checklist improves trainee performance during a simulated episode of local anesthetic systemic toxicity. Regional anesthesia and pain medicine, 37(1), 815. https://doi.org/10.1097/AAP.0b013e31823d825a Poore, J. A., Cullen, D. L., Schaar, G. L. (2014). Simulation-Based Interprofessional Education Guided by Kolb's Experiential Learning Theory. Clinical Simulation in Nursing, 10(5) 241-247. Shttps://doi.org/10.1016/j.ecns.2014.01.004. Sherwin, M. A., & Eisenkraft, J. B. (2020). Anesthesia hazards: what is the role of the anesthesia machine? International anesthesiology clinics, 58(1), 2731. https://doi.org/10.1097/AIA.0000000000000264 St. Pierre, M., Breuer, G., Strembski, D., Schmitt, C., & Luetcke, B. (2017). Does an electronic cognitive aid have an effect on the management of severe gynecological TURP syndrome? A prospective, randomized simulation study. BMC anesthesiology, 17(1), 72. https://doi.org/10.1186/s12871-017-0365-8 IMPACT OF COGNITIVE AIDS ON SIMULATION 24 Suet, G., Blanie, A., De Montblanc, J., & Benhamou, D. (2021). Use of an observer tool to enhance observers' learning of anesthesia residents during high-fidelity simulation: A randomized controlled trial. Simulation in Healthcare: Journal of the Society for Medical Simulation, 1Advance online publication. https://doi.org/10.1097/SIH.0000000000000584 Ward, P., Johnson, L. A., Mulligan, N. W., Ward, M. C., & Jones, D. L. (1997). Improving cardiopulmonary resuscitation skills retention: effect of two checklists designed to prompt correct performance. Resuscitation, 34(3), 221225. https://doi.org/10.1016/s0300-9572(96)01069-6 Watkins, S. C., Anders, S., Clebone, A., Hughes, E., Patel, V., Zeigler, L., Shi, Y., Shotwell, M. S., McEvoy, M. D., Weinger, M. B. (2016). Mode of information delivery does not effect anesthesia trainee performance during simulated perioperative pediatric critical events: A trial of paper versus electronic cognitive aids. Simulation in Healthcare: The Journal of The Society for Medical Simulation, 11, 385-393. Wetmore, D., Goldberg, A., Gandhi, N., Spivack, J., McCormick, P., DeMaria, S. (2016). An embedded checklist in the anesthesia information management system improves preanaesthetic induction setup: a randomised controlled trial in a simulation setting. BMJ Quality & Safety, 25, 739-46. https://dx.doi.org/10.1136/bmjqs-2015-004707 Ziewacz, J. E., Arriaga, A. F., Bader, A. M., Berry, W. R., Edmondson, L., Wong, J. M., Lipsitz, S. R., Hepner, D. L., Peyre, S., Nelson, S., Boorman, D. J., Smink, D. S., Ashley, S. W., & Gawande, A. A. (2011). Crisis checklists for the operating room: development and IMPACT OF COGNITIVE AIDS ON SIMULATION pilot testing. Journal of the American College of Surgeons, 213(2), 212217.e10. https://doi.org/10.1016/j.jamcollsurg.2011.04.031 25 IMPACT OF COGNITIVE AIDS ON SIMULATION 26 Appendix A Literature Review Matrix Citation Beck, S., Reich, C., Krause, D., Ruhnke, B., Daubmann, A., Weimann, J., Zollner, C., Kubitz, J. (2018). For beginners in anaesthesia, self-training with an audiovisual checklist improves safety during anaesthesia induction: a randomised, controlled two-centre study. European Journal of Anaesthesiology, 35, 527-533. https://dx.doi.org/10.1097/EJA.0000000000000781 Bould, M. D., Hayter, M. A., Campbell, D. M., Chandra, D. B., Joo, H. S., & Naik, V. N. (2009). Cognitive aid for neonatal resuscitation: a prospective single-blinded randomized controlled trial. British journal of anaesthesia, 103(4), 570575. https://doi.org/10.1093/bja/aep221 Combes, X., Le Roux, B., Suen, P., Dumerat, M., Motamed, C., Sauvat, S., Duvaldestin, P., & Dhonneur, G. (2004). Unanticipated difficult airway in anesthetized patients: prospective validation of a management algorithm. Anesthesiology, 100(5), 11461150. https://doi.org/10.1097/00000542-200405000-00016 Research Design & Level of Evidence Randomized controlled trial; level 2 Population / Sample size n=x Major Variables Instruments / Data collection Results N=26 anesthesia residents during first month of residency Completion of essential tasks completed during anesthesia induction Observations of anesthesia induction immediately, at 4 weeks, and at 8 weeks. Immediately and 4 weeks after implementation of audiovisual cognitive aid checklist, residents with the cognitive aid had significantly greater number of safety checks compared to the control group with no cognitive aid. Randomized controlled trial; level 2 N=32 anesthesia residents volunteered for neonatal resuscitation protocol (NRP) simulation. Clinical performance, frequency of cognitive aid use The checklist scores were not significantly different. There was infrequent use of the NRP cognitive aid by the experimental group. Nonexperimental observational study; level 3 N=41 anesthesiologists who have received a 2month educational period on a difficult airway algorithm with cognitive aid. Total of intubations, difficult airways, deviations from the algorithm, number of laryngeal mask airways used, algorithm adherence rate, patient medical history, and airway management complications Video recordings of NRP exercise were examined by a neonatologist and anesthetist using an approved checklist Medical record review With the use of the difficulty airway algorithm for 18-month period, 98 unexpected difficult airways were resolved with waking the patient, inserting an LMA, or intubating with bougie. IMPACT OF COGNITIVE AIDS ON SIMULATION 27 Harrison, T. K., Manser, T., Howard, S. K., & Gaba, D. M. (2006). Use of cognitive aids in a simulated anesthetic crisis. Anesthesia and Analgesia, 103(3), 551556. https://doi.org/10.1213/01.ane.0000229718.02478.c4 Nonexperimental observational study; level 3 N=48, 24 first year anesthesia residents and 24 second year anesthesia residents who volunteered to participate in malignant hyperthermia simulation N=20 anesthesiologists who participated in a general anesthesia for cesarean delivery simulation with an electronic cognitive aid. N=80 physician and nurse anesthetists Hart, E. M., & Owen, H. (2005). Errors and omissions in anesthesia: a pilot study using a pilot's checklist. Anesthesia and analgesia, 101(1), . https://doi.org/10.1213/01.ANE.0000156567.24800.0B Nonexperimental observational study; level 3 Heidegger, T., Gerig, H. J., Ulrich, B., & Kreienbhl, G. (2001). Validation of a simple algorithm for tracheal intubation: daily practice is the key to success in emergencies--an analysis of 13,248 intubations. Anesthesia and analgesia, 92(2), 517522. https://doi.org/10.1097/00000539-200102000-00044 Nonexperimental observational study; level 3 Marshall, S. D., & Mehra, R. (2014). The effects of a displayed cognitive aid on non-technical skills in a simulated 'can't intubate, can't oxygenate' crisis. Anaesthesia, 69(7), 669677. https://doi.org/10.1111/anae.12601 observational study; level 3 N=64 anesthesia residents Neal, J. M., Hsiung, R. L., Mulroy, M. F., Halpern, B. B., Dragnich, A. D., & Slee, A. E. (2012). ASRA checklist improves trainee performance during a simulated episode of local anesthetic systemic toxicity. Regional anesthesia Randomized controlled trial; level 2 N=25 anesthesiologists, 13 without checklist Clinical performance, frequency of cognitive aid use Video recordings were examined using the Malignant Hyperthermia treatment score scale. The incidence of cognitive aid use was linked to significantly greater MH treatment score for the first year resident group (Spearman r = 0.59, P < 0.01) and second year resident group (Spearman r = 0.68, P < 0.001). Number of tasks not completed feelings about the checklist Direct observation and anonymous post-exercise questionnaire Participants missed a median of 13 (range, 7-23) of 40 tasks on the checklist. 95% participants felt that the checklist was useful. 80% would like to use it for practicing simulated scenarios. A checklist cognitive aid would improve patient safety by preventing vital checks from being missed. Number of intubations, type of intubation, surgical procedure, laryngoscopy grade, case details Non-technical skills Filemaker Pro 3.0 database Implementation of an algorithm lead to 6 failed intubations out of 13,248 cases (0.045%; 95% confidence interval 0.02%-0.11%). An algorithm cognitive aid for endotracheal intubation can lead to reliable successful intubations. Anaesthetists' Non-Technical Skills (ANTS) Anaesthetists' Non-Technical Skills (ANTS) scores were superior in all sections when a cognitive aid was supplied 10.4 (3.1) vs. 13.2 (2.4). The establishment of an infraglottic airway within 3 min in the cognitive aid group was also higher (control group 55.3% vs. intervention 76.9%, p = 0.076). Non-technical skills are enhanced when a cognitive aid is implemented during airway emergencies Number of tasks completed, knowledge, Participants were evaluated using the ASRA Checklist cognitive aid group displayed greater medical management of the simulated LAST event. The checklist group correctly performed 16.0 (2.6) tasks versus the no-checklist group's IMPACT OF COGNITIVE AIDS ON SIMULATION and pain medicine, 37(1), 815. https://doi.org/10.1097/AAP.0b013e31823d825a 28 cognitive aid and 12 with. nontechnical skills St Pierre, M., Breuer, G., Strembski, D., Schmitt, C., & Luetcke, B. (2017). Does an electronic cognitive aid have an effect on the management of severe gynaecological TURP syndrome? A prospective, randomised simulation study. BMC anesthesiology, 17(1), 72. https://doi.org/10.1186/s12871-017-0365-8 Randomized controlled trial; level 2 N=20 participants who were anesthesia providers, nurses, and physicians who were all part of anesthesia teams Completion of essential tasks, influence of cognitive aid Suet, G., Blanie, A., De Montblanc, J., Benhamou, D. (2021). Use of an observer tool to enhance observers' learning of anesthesia residents during high-fidelity simulation: a randomized controlled trial. Simulation in Healthcare: The Journal of The Society for Medical Simulation, https://dx.doi.org/10.1097/SIH.0000000000000584 Randomized controlled trial; level 2 N=89 anesthesia residents, 44 with the cognitive aid and 45 without. Knowledge, stress level, nontechnical skills questionnaire Ward, P., Johnson, L. A., Mulligan, N. W., Ward, M. C., & Jones, D. L. (1997). Improving cardiopulmonary resuscitation skills retention: effect of two checklists designed to prompt correct performance. Resuscitation, 34(3), 221225. https://doi.org/10.1016/s0300-9572(96)01069-6 Randomized controlled trial; level 2 N=169 undergraduate students Completion of essential tasks, accuracy of compressions and ventilations Watkins, S. C., Anders, S., Clebone, A., Hughes, E., Patel, V., Zeigler, L., Shi, Y., Shotwell, M. S., McEvoy, M. D., Weinger, M. B. (2016). Mode of information delivery does not effect anesthesia trainee performance during simulated perioperative pediatric critical events: a trial of paper versus electronic cognitive aids. Simulation Randomized controlled trial; level 2 N=89, 45 student registered nurse anesthetists, 44 anesthesia residents Completion of essential tasks, duration of simulation checklist, knowledge post-test, Anesthetists non-technical skill (ANTS) scoring system Observation of simulation, checklist of essential tasks, survey of the influence of the cognitive aid 8.8 (3.0) tasks (mean [SD], P < 0.001). The checklist group had superior decision-making scores on the anesthesiologists' nontechnical skills assessment (5.2 [1.8] versus 4.0 [1.35] summed rater score, P = 0.037) and had higher knowledge retention 2 months later (P = 0.031). Pre- and posttest for knowledge, researcher designed stress questionnairee, Anesthetists non-technical skill (ANTS) scoring system Researchers recorded completed tasks, and manikin recorded accuracy of compressions and ventilations. Scenariospecific checklists The mean medical knowledge score was higher in the cognitive aid group (P = 0.0008). The mean Anesthetists' Non-Technical Skill score and level of stress perceived did not differ between groups. Both groups reported equal the learning value and satisfaction related to the simulation course. The cognitive aid group considered evidencebased treatment steps significantly more frequent than teams of the control group (p < 0.001). The lengthy checklist cognitive aid group had superior performance on the procedural variables. The results support the hypothesis that cognitive aids help participants remember CPR steps. About 1/3 of anesthesia residents assigned to use a cognitive aid (CA) (electronic 29%, paper 36%) chose not to use it during the scenario. The total score was 6% higher in the paper CA group IMPACT OF COGNITIVE AIDS ON SIMULATION 29 in Healthcare: The Journal of The Society for Medical Simulation, 11, 385-393. Wetmore, D., Goldberg, A., Gandhi, N., Spivack, J., McCormick, P., DeMaria, S. (2016). An embedded checklist in the anesthesia information management system improves pre-anaesthetic induction setup: a randomised controlled trial in a simulation setting. BMJ Quality & Safety, 25, 739-46. https://dx.doi.org/10.1136/bmjqs-2015-004707 Ziewacz, J. E., Arriaga, A. F., Bader, A. M., Berry, W. R., Edmondson, L., Wong, J. M., Lipsitz, S. R., Hepner, D. L., Peyre, S., Nelson, S., Boorman, D. J., Smink, D. S., Ashley, S. W., & Gawande, A. A. (2011). Crisis checklists for the operating room: development and pilot testing. Journal of the American College of Surgeons, 213(2), 212217.e10. https://doi.org/10.1016/j.jamcollsurg.2011.04.031 and 8% higher (P = 0.03) in the electronic CA group. Randomized controlled trial; level 2 N=38 anesthesiology residents Completion of essential tasks, duration of simulation Pre-anesthetic Induction Patient Safety Checklist The checklist cognitive aid group scored significantly higher checklist scores by 7.8 points (p<0.01). Simulation duration was increased significantly by the use of the checklist in the experimental group. Quasiexperimental study; level 3 N=11, 2 surgical teams consisting of surgical physicians, residents, anesthesiologists, anesthesia residents, and circulating nurses Completion of essential tasks, perceptions of checklist cognitive aid Video recording, simulation checklist The Checklist cognitive aid group had a 6-fold decrease in omitted steps and errors. 11 of 46 fail steps in the control group compared to 2 of 46 in the cognitive aid group (p = 0.007). IMPACT OF COGNITIVE AIDS ON SIMULATION Appendix B Theoretical Framework: Kolbs Theory of Experiential Learning Poore, J. A., Cullen, D. L., Schaar, G. L. (2014). Simulation-based interprofessional education guided by Kolb's Experiential Learning Theory. Clinical Simulation in Nursing, 10(5), 241-247. https://doi.org/10.1016/j.ecns.2014.01.004. 30 IMPACT OF COGNITIVE AIDS ON SIMULATION Appendix C Anesthesia Machine Check Knowledge test 1. Which is most common site for anesthesia machine circuit disconnection? a. Defective absorber cannister b. Reservoir bag c. Y-piece d. Inspiratory limb 2. Which of the following is a common cause for anesthesia machine circuit leak? a. Defective absorber cannister b. Leak in bellows c. Oxygen analyzer d. Defective flowmeter 3. During high pressure leak tests, breathing circuit should be able to maintain a minimum of? a. 70 mmHg b. 50 mmHg c. 30 mmHg d. 20 mmHg 4. Gas pipeline pressures should be a minimum of ______. a. 45 psi b. 50 psi c. 35 psi d. 25 psi 5. The oxygen analyzer should detect the inspired oxygen concentration is? a. 15% b. 30% c. 21% d. 25% Dosch, M. P & Tharp, D. (2021). The Anesthesia Gas Machine. University of Detroit Mercy Nurse anesthesia. https://healthprofessions.udmercy.edu/academics/na/agm/index.htm Nagelhout, J. J. & Elisha S. (2017). Nurse Anesthesia. (6th ed.). Elsevier. 31 IMPACT OF COGNITIVE AIDS ON SIMULATION Appendix D National League for Nursing. (2005). Student Satisfaction and Self-Confidence in Learning. https://www.nln.org/docs/default-source/default-document-library/instrument2_satisfaction-and-self-confidence-in-learning.pdf?sfvrsn=0 32  ...

... DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 1 An Analysis of how Additional Simulation Scenarios Affect SRNAs Cognitive and Psychological Preparedness for Anesthesia Machine Malfunctions in the Clinical Setting Anthony Simon & Jessie Earle Doctorate of Nursing Anesthesia, Marian University NSG-706-MM01 Manuscript Dr. Susan Pepin February 23, 2023 Chair: Dr. Sara Franco, DNAP, CRNA _________________________ (signature) Committee Members: Dr. Derrianne Monteiro, DNP, CRNA _______________________________ (signature) DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 2 Table of Contents Acknowledgements..4 Abstract....4 Introduction..4 Background..5 Problem Statement...7 Needs Assessment and Gap Analysis..8 Aims and Objectives9 Theoretical Framework..10 Learning Environment...10 Simulated Scenarios...11 Patient-Centered Practice...12 SWOT Analysis.12 Review of the Literature13 Search Methodology..13 Synthesis of Literature...14 Project Design and Methods..24 Project Site and Population24 Measurement Instruments..25 Data Collection Procedure.25 Ethical Considerations...26 Data Analysis and Results.27 Discussion..29 DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 3 References..31 Appendices.35 Appendix A....35 Appendix B....36 Appendix C....38 Appendix D....39 Appendix E53 DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 4 Acknowledgements We would like to thank our project chair, Dr. Franco, for her encouragement and guidance during this project and throughout our time in the program. We would also like to thank Dr. Monteiro for sharing her passion for education and simulation lab with us. This project would not have been possible without their time, energy, and support. Abstract Simulation experiences are a mandatory part of nurse anesthesia programs. Simulations are intended to improve students skills in a safe environment, but their effects on cognitive knowledge and confidence are not well documented. This study provided 28 student volunteers with additional simulation opportunities. The experimental group participated in additional simulation experiences and the control group did not. Both groups took a pre- and post-test designed to measure confidence and test students knowledge. The pre- and post-test scores were compared between the two groups. Despite the perceived importance of simulation, no significant relationship was demonstrated between additional simulation scenarios and cognitive improvement. There was also no significant change in confidence levels regardless of simulation participation or frequency. Introduction Simulation is an integral part of many healthcare education programs. It provides an environment in which students can practice without the possibility of harming patients. It also enables instructors to evaluate performances they would otherwise be unable to directly supervise. For example, the use of simulation to evaluate students ability to prepare and trouble shoot anesthesia machine functions. The application of simulation before integration into the DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 5 clinical field has proven advantageous when encountering common anesthesia machine malfunction situations. Simulation courses are incorporated throughout Marian Universitys DNP CRNA program, each with its own syllabus and course expectations. Structured course calendars include one required lab hour weekly. However, many professors stress the importance of add itional lab hours, citing them as essential to passing test outs and being prepared for clinical experiences. While many programs offer open lab hours, they are not required at Marian University. These additional hours are often self-taught, overseen by upperclassmen, or lead by faculty. The time students allot to volunteer simulation can make an overall impact on readiness toward student integration into the clinical arena, as demonstrated in their response to common anesthesia machine malfunctions. Despite this, the current course syllabi for first year CRNA students at Marian University do not contain open lab requirements and there is little to no literature available to support voluntary simulation hours. This project attempted to determine if there is a statistical correlation between voluntary simulation experiences and students readiness to manage common anesthesia machine malfunctions. In addition, this project looks to assess participants overall readiness for integration into the clinical setting based on self-perception. This paper addresses this gap in literature and provides students and professors with data regarding the impact, or lack thereof, regarding extra simulation experiences. Background Simulations are utilized by graduate nursing programs throughout America. Regardless of the wide use of simulation experiences, the Council on Accreditation of Nurse Anesthesia Educational Programs, COA, does not have a set requirement for simulation time (COA, 2019). They do, however, allow for certain simulation experiences, such as central line placement, to DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 6 replace a limited number of clinical experiences. Despite the assumed importance of simulations within CRNA programs, little research is available regarding simulations within the DNP environment. Even less research is available to guide instructors on required practice hours in the simulation lab. It has been well documented that simulation experiences can improve self-confidence and perceived preparedness for clinical experiences within nursing programs (Basak et al., 2019, Tamaki et al., 2019, Li, J. et al., 2019). Marian University SRNAs are highly encouraged to attend open-lab hours to practice specific skills, such as machine checks and failures, but these are not required by the university nor is a specific number of hours recommended. Anesthesia machine tests and failure assessments are one of the first skills listed in the simulation curriculum for first year SRNAs at Marian University. Even with the basic familiarity obtained during required lab hours, SRNAs may be ill prepared to respond to anesthesia machine failures during the perioperative period. Loeb et al. (2019) state that misuse or malfunction of the anesthesia machine accounts for 1-2% of lawsuits involving anesthesia personnel. The majority of anesthesia delivery equipment claims, although infrequent, were a result of provider error. In fact, a study by Mehta et al. (2021) found that 85% of claims made regarding anesthesia equipment were due to provider error, and 35% were preventable by paranesthesia machine checks. Such malfunctions can result in very detrimental outcomes for the patient and the anesthesia clinician. Anesthesia machine and equipment malfunctions are often unpredictable. Therefore, diligence must be taken to ensure all proper equipment has been checked by the anesthesia provider before its use. Mehta et al. (2021) state that misconnections of the breathing circuit are among the most common preventable issues that lead to patient injury. Furthermore, anesthesia DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 7 gas delivery equipment constitutes 20% of reported events related to critical incidents (Mehta et al., 2021). Such events included ventilator problems (17.9%), vaporizer problems (5.1%), leaks in the circuit (9.6%), and gas supply issues (1.9%). These are all scenarios that can be simulated in a safe, positive environment. Light anesthesia, which can result in patient injury and recall, occurs in up to 71% of vaporizer malfunction (Mehta et al., 2021). These events were often due to user error resulting from unfamiliarity with equipment or memory lapse (Mehta et al., 2021). Most anesthesia machine malfunctions are related to user error, therefore establishing a fundamental understanding of how this equipment works is vital to patient safety. SRNAs must develop the ability to recognize and address anesthesia machine failure within the intraoperative period. This education can be provided in a safe, controlled environment with the use of highfidelity simulation scenarios. It is possible that students who volunteer to participate in simulation scenarios may be better prepared to prevent, recognize, and intervene in anesthesia machine failures sooner than those who do not, thus reducing the risk of harm to the patient. Problem Statement There is an inconsistency between professors expectations and course syllabi regarding simulation hours. While there is some evidence that simulation prepares students for clinical experience, there is no consistent recommendation for open lab hours. The skills students learn regarding machine checks and failure solutions directly affect patient safety. Considering the significance of anesthesia machine malfunctions, it is essential that students be as prepared as possible to trouble shoot issues while maintaining patient safety in the operating room. Students who attend simulation more often may establish a more robust understanding of the anesthesia machine components and respond to unpredictable machine equipment failures quicker. In order to address this, a quality improvement project was performed to assess how voluntary, additional DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 8 simulation scenarios affect perceived preparedness and technical skills in first year Marian University SRNAs. Needs Assessment & Gap Analysis Currently, Marian University SRNAs begin simulation courses in their first year, the semester prior to beginning clinical experiences. While they have a structured syllabi with a course calendar, they do not have a recommended number of practice hours. As is, students are expected to attend lab for one hour each week to learn a pre-determined skill. They are put through test-outs to assess their knowledge regarding that skill on a later date, usually a week to a month after their initial learning experience. During this time, they do not revisit that skill during formal lab hours. Additional simulation times are offered to become more familiar with the equipment throughout their first year. These extra hours are voluntary and are left open to students to participate at their discretion. Although DNAP staff encourage students to take advantage and utilize these open hours for more practice throughout their first year, current student participation is low. Discussions with DNAP staff revealed a general agreement that first-year anesthesia students should attend simulation lab more frequently to foster and solidify skills learned. In order to ensure student participation, Marian University needs to specify its simulation expectations, including open lab hours, within its syllabi. Prior to including additional requirements, however, data needs to be presented to validate the effectiveness of additional practice hours. This project has examined how additional simulation scenarios affect SRNAs cognitive and perceived progress towards being prepared for clinical experiences. DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 9 Aims & Objectives The first aim of this project was to assess the effect of additional simulation hours on SRNAs perceived preparedness and self-confidence during anesthesia machine malfunction scenarios. The second aim of this project was to determine if additional simulation experiences improve SRNAs technical skills by their ability to identify and resolve common anesthesia machine malfunctions. The purpose of this project was to provide support for participation in supplementary simulation time by demonstrating how it impacts SRNAs readiness to both prevent and address common anesthesia machine malfunctions in a clinical setting. These aims were approached by addressing the following objectives: 1. First year SRNAs will complete voluntary, additional simulation hours to practice anesthesia machine checks and machine failure scenarios prior to beginning clinical experiences. 2. SRNAs will perform a self-evaluation of their perceived preparedness and selfconfidence prior to attending additional simulation hours. 3. SRNAs baseline understanding of anesthesia machine malfunctions will be assessed by completion of a cognitive-based pre-test prior to attending additional simulation hours. 4. Volunteer First year SRNAs will attend a simulation workshop where they will work to address common anesthesia machine malfunctions. First year SRNAs will work with Junior SRNAs to develop comfort and self-confidence in solving simulated anesthesia cases. 5. SRNAs will participate in debriefing sessions immediately following each voluntary experience to document and reflect upon skills, knowledge, and confidence improvements. DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 10 6. After completing additional simulation hours, SRNAs will report higher levels of self confidence in addressing machine failures. 7. After completing additional simulation hours, SRNAs will score higher on a cognitive based assessment. Theoretical Framework The conceptual framework that most naturally aligns with this DNP project is Khalilis Clinical Simulation Practise Framework: A knowledge to action strategy in health profession education. This framework theorizes that simulation learning will help improve learners competence, confidence, and collaboration in the clinical setting (Khalili, 2015). It was used to establish and implement stimulation experiences for first year SRNAs. The theoretical framework clearly defines three components to make a simulation effective. First, simulations must be safe, positive, reflective, and fun in order to provide students with an environment that fosters education (Khalili, 2015). The next is a realistic, but challenging scenario that will push students to apply their knowledge and provide life-like situations. Lastly, all effective simulations should be interprofessional and interactive patient-centered scenarios (Khalili, 2015). Safe, Positive, Reflective and Fun Simulated Learning Environment This study utilized upperclassman-led simulation, which allowed first year students to participate in a safe and positive environment without fear of failing in front of faculty or clinical instructors. In order to further establish a safe, reflective, and fun learning environment, a nonjudgmental and non-threatening approach was needed. This was facilitated by providing learners with the educators expectations for the simulation while still maintaining the integrity of the simulation experience. Learners were instructed that perfection was not expected, but an emphasis was placed on respectful participation and critical thinking. As soon as the simulation DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 11 began, educators were no longer available for questions. However, debriefing and evaluation were conducted post simulation using a nonjudgmental, open-table approach. Participants were encouraged to share their experience and perspective of the scenarios provided. They were guided through a discussion of their interventions and rationales during the simulation. Following the debriefing, the educators revealed additional teaching points for future applications. The participants were encouraged to reflect on their experience before the debriefing was concluded. Challenging, but Realistic, and Integrated Simulated Scenarios This element centers on the integration of realism into the simulation. In order to achieve this goal, high-fidelity simulation was applied. High-fidelity simulation involves the incorporation of a manikin, or a high-fidelity human patient simulator (HPS), that has the ability to mimic or closely resemble human physiology. The HPS used replicated sounds (heart rate tones, breathing, talking), movement (pulsation, blinking), and visual waveforms on monitors. These were manipulated based on condition changes, treatment, and equipment (pulse oximetry, EKG, capnography, tidal volumes). Implementation of this high-fidelity simulation allowed the participants to benefit from the most realistic environment possible. Various procedures were performed on the HPS including intubation and bag-mask ventilation. Specific scenarios were chosen from anesthesia machine failure case studies to ensure that they remained realistic. Because the learning experiences took place in a closed and controlled environment, distractions and outside variables were able to be limited. These factors helped enable students to fully engage in their realistic learning environment. DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 12 Interactive, Inclusive, Interprofessional Patient- Centered Simulated Practice While this project did not involve professions other than nurse anesthetists, it included facility mentors and students from two different cohorts. Furthermore, communication and collaboration with team members was necessary in order complete the simulations. This satisfied the interprofessional requirement of Khalilis framework. In order to ensure the scenarios were patient-centered, machine failure scenarios were implemented that would impact the HPS physiology. Participants utilized hemodynamic monitors and physical assessment to evaluate overall patient status. This gave the SRNAs an opportunity to relate machine failures to patient care and safety in real time. Critical thinking and problem solving were also necessary in providing patient-centered care. Despite the complexity of the machine failure scenarios, the participants were primarily tested on prioritization of patient-centered care. SWOT Analysis Addressing the advantages of additional simulation time benefits a number of parties. Key stakeholders included simulation professors, directors, and SRNAs currently or soon to be enrolled in simulation courses. Marian Universitys CRNA program are also stakeholders since course and program curriculum can be adapted to include required additional hours if proven to be beneficial. Strengths of the project included using a convenience sample of volunteer students and required no financial investment. Student were not denied teaching or standard learning, leaving room for no present ethical concerns. Lastly, the project required one to two hours from its volunteers. The project also provided participants a helpful review of course material prior to the course final. These factors made the project appealing to potential volunteers. A detailed SWOT analysis chart can be seen in Appendix C. DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 13 Review of the Literature Literature Search Methodology A literature search was conducted on December 27, 2021, using PubMed, Ovid/Medline, Ebscohost/Medline, and CINAHL databases. Initially, three separate searches were conducted. The first keywords searched was BOOLEAN phrase anesthesiology or anesthesia or equipment failure. This resulted with a total article count of 106,460. A following separate search was conducted using BOOLEAN phrase high fidelity simulation training or patient simulation or simulation training. This resulted with a total article count of 10,861. A third separate search was conducted using keyword anesthesia machine malfunction. This resulted with a total article count of 946,784. The BOOLEAN phrase was adapted to combine these separate searches such as equipment failure AND simulation training AND anesthesia machine malfunction. A filter was also added to limit literature article results to include only English articles with studies conducted using humans. This search yielded 30 articles. Duplicate articles found between databases were identified and removed from the literature search. Inclusion criteria included primary sources and peer-reviewed studies. Exclusion criteria included secondary sources and equipment failures unrelated to anesthesia. After applying inclusion and exclusion criteria, 12 articles remained and were used for the review of literature. To find articles based on simulation within the BSN world, another search was conducted. Multiple databases were searched in December and January of 2022 to find articles about how simulations affect self-confidence in BSN students. Key words included simulation, nursing students, and self-confidence. Because self-confidence and self-efficacy are often used interchangeably, self-efficacy was also included as a key word. Eric was searched using the Boolean phrase nurs* AND simulat* AND self-efficacy OR confidence. From this, 29 articles DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 14 were found. Next, PubMed was utilized with filters set as publication date within five years or less, clinical trials, and randomized control trials. The Boolean phrase (nurs* AND student) AND simulat* AND confidence OR self-efficacy was used to find 63 articles. Additionally, two more articles were found through mining sources, making the grand total 94 articles. Of these, five were duplicates. The remaining articles were screened using exclusion and inclusion criteria. Sixty-one articles were excluded immediately based on abstract and/or title. Next, 28 articles were read. Inclusion criteria included: focus on the keywords, main subject regarding BSN simulation experiences with specific topics, pre and post-test evaluations over skills improvement and confidence, and pieces that juxtaposed simulations to traditional learning methods. Exclusion criteria included: virtual simulations, video supplements, emphasis on pre- or post-briefing, and comparison of as the main focus. Experiments based in hospitals and articles focused on workplace environments were also excluded. These criteria narrowed the 28 articles down to 9 articles, which were included in this review. Synthesis of Literature Review Much of the literature supports incorporating simulation-based learning as an adjunct to standard training in order to better prepare clinicians for anesthesia machine failures. Morgan et al. (2003) states that high fidelity patient simulation offers an ideal venue in which students can practice and incorporate classroom knowledge of critical events without jeopardizing patient safety. Errors and faults that result in this setting can be simulated, which allows the anesthesia student to have exposure to these encounters in a safe and controlled setting. Since anesthesia equipment malfunctions are primarily due to human error (Dalley et al., 2004), additional simulation participation can assist in lowering this occurrence by creating better clinically DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 15 prepared anesthetists. Through simulation, educators are able to indirectly assess students clinical performances and determine whether the educational objectives taught in the curriculum are matched by the performance in the OR setting (Morgan et al., 2003). Anesthesia simulationbased training scenarios also provide educators a chance to see if their curriculum teaching is effective as a whole. By evaluating an entire cohort, educators would be able to identify common problems and misunderstandings. This aids educators in identifying fundamental educational problems that would require revisitation on the topic (Weller et al., 2007). In anesthesia, patient safety depends on the interaction between the anesthetist and the equipment (Dalley et al., 2004). Foreignness to anesthesia equipment has been identified as a major factor in the development of adverse events (Dalley et al., 2004). Thus, unfamiliarity of differences in various machines may potentially contribute to the development of anesthesia malfunctions. Because set up and safety checks can vary from machine to machine, if anesthesia clinicians are not accustomed with individual systems, errors, machine malfunctions, and patient safety events may occur. Anesthesia machines differ in breathing circuit design, ventilator control, fresh gas delivery; all of which directly contribute to 14-30% of all intraoperative problems that develop (Dalley et al., 2004). In order to avoid patient harm, it is critical for the anesthesia clinician to be able to quickly identify and resolve these equipment failures in a timely fashion. Increased simulation training can facilitate improvements in troubleshooting unpredictable scenarios. Weller et al. (2007) states that technical malfunctions are rare in anesthesia but can result in major morbidity and possibly death when they do occur. Continual exposure in the simulation setting can become a primary preventative measure that could be exercised to reduce malfunction errors in the clinical setting. Larson et al. (2007) states that DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 16 human error and insufficient preanesthetic machine checks are a recurring theme. Students who volunteer in simulation more often may establish an in-depth preanesthetic machine check routine quicker, recognize machine errors sooner, and deliver faster intervention, thus reducing the risk of patient harm. Additional participation and practice in simulation scenarios prior to integration into the clinical setting may aid in prevention or faster resolve of these common malfunctions. There are often many measures that can be taken to resolve an anesthesia machine equipment malfunction or failure; some approaches are more effective than others. Weller et al. (2007) conducted a simulation study that revealed concerning results. In the simulated clinical scenario case study, anesthetists were presented with a motor vehicle accident victim that required a minimum of 70% inspired oxygen concentration. At some point d uring the simulation, the oxygen pipeline supply failed. All the anesthetists were able to continue managing ventilatory support. In order to maintain oxygenation, all the anesthetists turned on their backup cylinders when the pipeline failed. Only 30% of the anesthetists had recognized that the backup oxygen cylinder was empty pre-operatively. Upon recognition of the empty back up cylinder, all anesthetists quickly requested a full cylinder to resolve the issue. All the anesthetists in this simulation-based scenario would have saved the patients life. However, this scenario highlights the deficiencies of an inadequate preoperative machine check (Weller et al., 2007). In a similar study to evaluate management of a simulated oxygen pipeline failure, highfidelity simulation was used to teach and assess clinical skills of anesthesia students. This study found that management of an oxygen pipeline failure was poorly understood among the participants based on their performance. Many students primarily and solely relied on the use of the Ambu bag as the main source to ventilate and oxygenate the patient, disregarding the DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 17 available reserve tank on the back of the anesthesia machine (Lorraway et al., 2006). This showed evaluators that further education on the anesthesia equipment was necessary. Another study was conducted regarding oxygen pipeline crossover. The researchers reversed the connections between the oxygen and nitrous oxide pipelines and reviewed the response of the anesthesia subjects. Many of the anesthesia clinicians recognized an issue with the oxygen source, however 60% resorted to using the axillary flowmeter as a backup source of oxygen (Mudumbai et al., 2010). The auxiliary flowmeter receives its oxygen from the pipeline, which in this scenario is administering nitrous oxide due to the crossover. These studies emphasize the importance of a thorough pre-operative machine check prior to the use of the anesthesia machine. Components such as an empty backup cylinder or gas crossover may have been recognized prior to the initiation of patient care and delivery of anesthesia to the patient. These studies further advocate toward the benefits of additional simulation training. More exposure to these anesthesia machines in a simulation setting may assist with establishing a thorough and appropriately detailed machine pre-check, thus preventing the previously described scenarios. Weller et al. (2007) states that a high-fidelity patient simulation offers anesthetists the chance to test the students response to many critical equipment related events and assist in identifying common management errors. Simulation has proven to be useful in other settings outside the medical arena. It is especially useful in aviation to evaluate equipment and technological failure as well as explore human-machine interaction (Mudumbai et al., 2010). Simulation enables the ability to recreate rare situations that require prompt attention, rapid judgement, and quick decision making, producing similar high stakes conditions that occur in real life scenarios. Simulation has also been a valuable resource during the current COVID19 pandemic. The record-breaking spread DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 18 and the unprecedented nature of this virus has made healthcare providers anxious and fearful to say the least. The virus has forced many clinicians to adapt to unfamiliar departments due to shortages and high demand. At times, clinicians may not have the baseline foundational knowledge to work on such units, which in turn could risk harm to the patient. Simulation has been able to assist in alleviating these concerns. It offers a setting where clinicians can develop new skills in a controlled safe environment while simultaneously increasing confidence towards approaching real-world crisis (Ekert et al., 2020). Simulation has become a necessary source for training during these unpredictable times. In summary, the advantage of simulation implementation throughout every field is its ability to allow participants to be able to interact with the equipment by exploring its function. Participants and viewers are able to examine management strategies through replication of a scenario in a controlled environment where there is no risk of harm to another being (Mudumbai et al., 2010). Mudumbai et al. (2010) found that participants lack of knowledge of the anesthesia machine coupled with incidental shortcomings of equipment designs is a recipe for potentially lethal outcomes. Anesthesia machine and equipment malfunctions are often unpredictable. Therefore, diligence of the anesthetist must be taken to ensure all proper equipment is available, has been checked, and intact before it is ready to use. Potential ramifications of a novice anesthesia provider include patient awareness. Light anesthesia is one of the most common problems resulting from vaporizer delivery issues. The development of light anesthesia may be due to user error related to lack of familiarity with equipment. The majority of equipment failures develop from user error. Early exposure in the simulation setting may prove advantageous in further lowering these incidences in the clinical setting. DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 19 Despite having a reasonably low incidence, anesthesia machine malfunction can lead to severe injury. Mudumbai et al. (2010) reveals that the American Society of Anesthesiologists Closed Claims Analysis Database found that respiratory and equipment events constitute a significant cause of malpractice claims. The majority of anesthesia delivery equipment claims, although infrequent, were a result of provider error. One studys results displayed that anesthesia practitioners were not able to clearly identify preset machine faults (Larson et al., 2007). Additional provider training that outlines appropriate preanesthetic machine equipment and check-out procedures may be able to assist in correcting this issue. This education may be best provided in a high-fidelity simulation-based setting where anesthesia students can receive appropriate training in a similar environment where assessment and evaluation of their performance can be critiqued. Debriefing is a critical component of simulation training. It provides participants the opportunity to discuss rationales for behavior and decisions practiced during the simulation (Khalili, 2015). Increased time spent in simulation may also promote student cognitive retention for earlier detection and resolution of anesthetic errors that occur in the clinical setting. As previously mentioned, a critical component of high-fidelity simulation training is the post simulation debriefing. This is the time where the participants are afforded an opportunity to discuss their performance, rationales, and their overall simulation experience with the evaluators. Evaluators are also able to provide feedback and critique the participants simulation performance during this time. Edwards et al. (2018) state that debriefing is where 80% of the learning occurs and thus has been deemed the most important phase of simulation. During this phase, participants are able to identify their weaknesses and strengths. The general goal of debriefing is for students to gain more insight into their practice, ensure educational development DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 20 and emotional support, and become dynamic participants in their learning (Edwards et al., 2018). All this is achieved under the safety net of a calm, non-judgmental, and controlled environment where questions can be explored, and wrong answers are not penalized, but rather corrected with explanation through discussion. The result of these outcomes achieved by the implementation of simulation is a more robust student; one that is more confident and competent in performing professional practice in real life settings (Khalili, 2015). In order to understand the impact high-fidelity simulation can contribute, it would be pertinent to review previous studies and outcomes that compare participants that attend simulation versus those that do not. A study was conducted to compare nurses who underwent high-fidelity simulation over the use of Continuous Renal Replacement Therapy (CRRT) versus nurses that did not. Patients with acute kidney injury on CRRT were found to not be receiving the full benefits of CRRT due to frequent unplanned interruptions in its use. There were many reasons for these unplanned interruptions, some of which included filter clotting or clogging, decreased flow rates, catheter dysfunction, or coagulation. Most of these issues arose due to ineffective resolution of alarms that led to decreased blood flows (Lemarie et al., 2019). All nurses had previous simulation training on the use of the CRRT machine. However, the study hypothesized that additional simulation training may be able to minimize the number of unplanned interruptions that occurred with patients on CRRT which would increase its overall usefulness. An experimental research design was used, and nurses were randomly separated into an experimental group (nurses that attended the high-fidelity simulation training) and control group (nurses that did not attend high-fidelity simulation training for CRRT). They assessed outcomes using pre and post intervention knowledge tests and simulation evaluation. The results of the study found that nurses of the experimental group scored better results on their post DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 21 intervention knowledge test and required less calls for assistance during the simulation evaluation when compared to the control group. The experimental group also felt more confident and less stressed in starting and troubleshooting the CRRT machine (Lemarie et al., 2019). The outcomes of this study further advocate the many benefits behind the execution of additional simulation training. It is pertinent to take into consideration that some facilities utilize anesthesia technicians. Anesthesia technicians are typically tasked with restocking the medication and equipment carts, in addition to performing the daily pre-anesthesia check as well safety checks between each case. Many facilities advocate that their role promotes time efficiency and benefits the anesthesia provider by allowing them additional time to complete a more robust preoperative assessment of their next patient. However, one study found that the time saved by allowing technicians to complete the pre anesthesia machine check may be impacting anesthesia providers ability to perform a proper machine check. Armstrong-Brown et al. (2000) study looked at the ability for participants to detect machine faults based on their knowledge of the equipment. The study found that participants rate of fault detection was low. Armstrong-Brown et al. (2000) found that participants understood the function of the machine but struggled to apply the understanding to practical clinical skill. It is the legal responsibility of the anesthesia provider to assess and determine appropriate function machine equipment regardless of whether or not the technician has already completed this task. With that in mind, it is critical that all components of the machine checklist are checked and verified in order to assure proper function prior to use on patients. Incomplete or partial checks may hinder the ability for one to identify machine faults or failures. Armstrong-Brown et al. (2000) states that if pre-use machine checks are truly a patient safety concern, then they should not be optional. DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 22 How can the validity of simulations effectiveness be evaluated? One study looked to review how truly effective and beneficial the added experience of high-fidelity simulation truly is based on specific grading criteria measures. Using two different grading f orms, Schwid et al. (2002) reviewed footage of subjects taking part in a high-fidelity simulation to and used two grading forms to score their performance. Construct-related validity (determines whether simulation evaluation is a legitimate indicator of performance) and criterion-related validity (compares results of simulation to other measures of participant performance) were supported (Schwid et al., 2002). Participants rated the simulation as realistic which supports construct related validity as a simulator evaluation tool. The correlation between the simulator scores and department evaluations supported criterion-related validity (Schwid et al., 2002). The progression of simulator scores from novice to 4th year anesthesia resident participants further supported the reliability and validity of simulation. Another way to evaluate the effectiveness of simulation is to look not ahead into the professional world, but rather into undergraduate nursing programs. Although simulation-based research is severely lacking within the world of doctorate nursing programs, slightly more literature is available at the bachelors level. A majority of these studies are either experimental or quasi-experimental and involve convenience samples of students within the host university. By examining students perceived preparedness for clinicals and self-confidence levels pre and post-intervention, these studies evaluate the effectiveness of simulation experiences compared to traditional learning experiences (such as lectures or reading-based material). Although helpful, not all of the data is straight forward. Saied (2017) used a quasi-mixed methods study to examine the effects of pediatric simulations compared to traditional learning. His results showed that although simulation DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 23 participants had significantly increased knowledge scores, their self-efficacy scores were significantly lower than their counterparts scores (Saied, 2017). A randomized control trial showed a slight, but not significant improvement in students self-confidence following simulation when compared to traditional learning (Alamrani et al., 2018). Kahraman et al. (2019) performed a quasi-experimental study in which the experimental group participated in pediatric emergency simulations. They, however, also failed to demonstrate a significant improvement in simulation participants self-efficacy but did show that students had better attitudes towards pediatric emergencies going forward. A recent randomized control study showed that additional simulation experiences had no effect on students confidence levels (Svllingen et al., 2021). This study compared a control group that participated in a single round of simulations to an experimental group that participated in double simulations over three years. Other groups, such as Basak et al. (2019) however, found much more encouraging results. An experimental, randomized control trial showed that simulations on patient education significantly increased students self-confidence and critical thinking scores when compared to traditional learning methods (Basak et al., 2019). Li et al. (2019) also showed that self -efficacy was not the only significantly increased score: empathy and communication were also increased in simulation participants when compared to non-simulation-based learning. In fact, another quasi-experimental study showed that nursing students who participated in obstetrics simulations had better overall performances and significantly increased self-confidence than their counterparts that benefitted solely from traditional learning (Gray & Cavner, 2017). Tamaki et al. (2019) demonstrated significantly improved self-confidence, skill-performance, and knowledge scores following an end-of-life care simulation when compared to traditional learning. Furthermore, simulations may continue to build confidence over periods of time. One DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 24 correlational study showed that maternity exam simulations significantly improved students self-confidence not only at an initial post-test, but saw another significant improvement following another questionnaire two days later (Germain et al., 2018). The overwhelming support for further anesthesia machine training demands a change in SRNAs education. Simulations have been shown to improve the knowledge and confidence of anesthetists in the professional world. Although most evidence supports the use of simulations for undergraduate nursing students, it is not an overwhelming consensus. Therefore, there is a gap in the literature that cannot be filled based on the conclusions of current literature. The lack of support for simulations within the DNP world, however, needs to be addressed. In particularly, the effects of voluntary simulation experiences and simulation experiences regarding anesthesia machine malfunctions within SRNAs. Project Design and Methods Project Site and Population A convenience sample of SRNAs was selected to participate in this one-group, pretest posttest quasi-experimental design. The cohort consisted of 32 first year SRNAs attending the Marian University School of Anesthesia. SRNAs were not initially assigned to a control or experimental group. Students unwittingly self-selected their group based on their decision to participate in additional simulations offered by the researchers of this study. The simulation workshop took place in the Evans Center second floor simulation laboratory. The lab provided a familiar, high-fidelity learning environment without added cost to the participants or researcher. The simulation site was also a convenient location that allowed for students to participate with minimal interruption in their daily schedule. DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 25 Measurement Instruments The pretest and posttest used in the study were identical. They consisted of 10 questions that assessed baseline understanding, comprehension, judgment, and application of the anesthesia machine and equipment information received by the professor. They also included a modified version of the validated Student Satisfaction and Self-Confidence in Learning tool, borrowed from the National League for Nursing. The tests were distributed as a hard copy after the completion of a regularly scheduled class. Students were asked to work independently to complete the test and hand it back into the researcher prior to leaving. The correct answers to the first portion were not revealed until after the study was complete in order avoid alterations to data. The questionnaire is shown in Appendix E. Data Collection Procedure As required by the anesthesia simulation lab curriculum set forth by the simulation professor Dr. Monteiro, all 32 students attended an initial introductory session of the anesthesia machine and its major components. This was be led by faculty. Some weeks later, the architects of this study administered the pre-test that addressed performance and confidence. After the entire cohort completed the pretest, students were offered the opportunity to attend additional simulation practice led by the researchers. The topic of education focused on anesthesia machine equipment and common malfunctions experienced in the clinical setting. The simulation experiences, which consisted of two scenarios each, lasted one hour. During the first week, the scenarios were available for participation from 0800-1600 for two consecutive days. The same schedule was repeated with two new scenarios during the second week. This resulted with a total of four available scenarios and 32 available time slots. DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 26 The first scenario offered was a general anesthetic during which an oxygen pipeline failure occurred. Pipeline pressure failure was simulated during the maintenance phase of a general anesthetic. Participants were responsible for identifying the malfunction and using problem solving skills to ensure patient safety for the duration of the anesthetic. The second simulation, which featured an expired carbon dioxide absorber, presented with steadily climbing inspired and expired carbon dioxide. Once students identified the expired absorber, and increased fresh gas flows in order to maintain patient safety, the rest of the case proceeded uneventfully. Students who participated in the third scenario were presented with a circuit leak. The source of the leak was located at the y-piece, but students were encouraged to trace the length of the circuit during their debriefing. The final simulation required students to work through a complete power failure of the anesthetic machine. Students were expected to maintain patient safety by providing manual ventilation and switching to a total intravenous anesthetic. They were also expected to communicate all machine malfunctions with the operating team, anesthesia team, and engineering department. Participation was not limited to a set number of sessions. Students were grouped together based on open lab sign ups, in groups of two to six. A debriefing was conducted after each simulation session to discuss and review any questions the participants may have had. After the sessions were complete, the entire cohort, regardless of simulation participation, took the posttest assessment that addressed performance and confidence. After the tests were collected, the correct answers and rationales were made available via e-mail. Ethical Considerations In order to maintain the integrity of the study, in addition to providing protection of the participants, data and attendance was tracked using the last four numbers of each students DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 27 Marian ID number to maintain anonymity. This helped minimize the risk of biases that may result unintentionally. The pre and post simulation examination results were not released outside of the researchers possession. The pre and post tests were collected immediately upon completion and answers were not released until after the entire study was completed. Data Analysis and Results After implementation of the study, the information was documented on paper then coded into a statistical computing program called R. Each participant was represented by the last four numbers of their student ID number to maintain confidentiality. The frequency of participation in simulation education was also recorded and entered into the software. Other quantitative variables collected from the study were also plugged into the program. These variables included age, gender, ethnicity, marital status, years of critical care nursing experience, average weekly study hours, pretest and post test scores, and their average lab participation hours. Once all the data was inputted, a multiple regression was initially used to analyze the main predictor variable, testing the impact that attending the provided information sessions had on the post-test scores of the participants which is the outcome variable. The attendance was broken into two categories: students that attended at least once and students that attended at least twice. Both gave back very high p-values (p=0.5615 and p=0.9491, respectively), indicating that attending sessions did not have a statistically significant impact. DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 28 Multiple regression was also used to test the other predictor variables against the post test scores, including the pre-test score, self-reported confidence levels of the students during both pre-test and post-test, ICU experience, average hours spent studying in each week and average weekly lab attendance. Other demographic variables were also tested including age range, gender, ethnicity, and marital status. All predictor variable returned high p-values at the 95% significance level, except for the students confidence on the pre-test, which gave back a nearly significant pvalue of p=0.0533. Multiple regression was also used to calculate the impact of other various predictor variables on the confidence of students when taking their post-test. The main variable of interest was the students attendance to the offered session, which was broken into the same two categories as above: students that attended at least once and students that attended at least twice. When analyzed, both returned high p-values (p=0.7227 and p=0.8981, respectively), indicating that attending the information sessions did not have a statistically significant impact on the confidence of the student while taking the post-test. The same analysis was completed to test the other predictor variables, including the pretest and post-test scores, self-reported confidence levels of the students during the pre-test, ICU experience, average hours spent studying in each week and average weekly lab attendance. Other demographic variables were also tested including age range, gender, ethnicity, and marital status. DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 29 Most of the predictors returned high p-values at the 95% significance level, however, there was a statistically significant result with the self-reported confidence levels of students during the pretest. Discussion There were a variety of limitations that held back this study. One weakness of the project was the necessity to plan it around a pre-existing course syllabus. This limited the available time in which the project was able to be implemented. The project was also dependent entirely upon voluntary student participation, a weakness that ultimately limited sample sizes. An unforeseen weakness was the business of the simulation lab. Other junior level SRNAs were available for teaching during the weeks of this study. This pulled participants from the scenarios designated for the project. Other issues that arose included lag and sometimes inoperability with the control units communication with both the high-fidelity simulation mannequin and the anesthesia machine. Perhaps one of the biggest opportunities that benefited this project was the overall availability and proximity of the simulation lab. It is frequently open to students for additional practice hours and can be made available during other times by request. Because all first year SRNAs are required to enroll in simulation courses, they are also encouraged by their professors (and COA) to participate in additional hours. This study showed that attending open lab sessions any number of times did not increase the performance of the students, nor did it have an impact on students overall confidence. Despite the aforementioned limitations, participants were still able to effectively discuss resolution for the given anesthesia clinical scenario. Allowing students to effectively discuss the case as a group seemed to aid in a swifter response in the simulation. Open discussion with the DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 30 researchers after the simulation also seemed to increase further confidence. This information was passed on to the researchers by the participants during the post simulation debriefing session. This analysis addressed two aspects of attendance: first comparing participants that did attend open lab sessions with those that did not (Attend_1), and the second whether frequent session attendance improved overall performance (Attend_2). Both of these concepts revealed no significant results. However, in regard to confidence, there was significance found. If a student was confident while taking the pre-test, there was a greater likelihood that the student would be confident while taking the post-test as well. Although few significant effects were identified in the present analysis, it is likely due to the fact that the present sample size is only 28. A priori power analysis with 12 f actors was completed using an estimated effect size of 0.02. This power analysis was chosen due to Cohens 1988 guidelines (Cohen, 1988), which recommends using an 2 value of 0.02 for small effect sizes. Based off the power analysis it was suggested that a sample size of 877 is needed to properly power the present study. Increasing the sample size to appropriately power this study could elucidate effects that were not observed in the present, underpowered analysis. It would be pertinent for future research into this hypothesis to use an appropriate sample size with equipment that has been appropriately tested and deemed operable. This could potentially reveal a more robust and definitive outcome between the correlation of frequency in open lab simulation attendance and future performance for identifying and addressing anesthetic malfunctions. DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 31 References Alamrani, M. H., Alammar, K. A., Alqahtani, S. S., & Salem, O. A. (2018). Comparing the effects of simulation-based and traditional teaching methods on the critical thinking abilities and self-confidence of nursing students. The Journal of Nursing Research, 26(3), 152-157. https://doi.org/ 10.1097/jnr.0000000000000231 Armstrong-Brown, A., Devitt, J.H., Kurrek, M., & Cohen, M. (2000). Inadequate preanesthesia equipment checks in a simulator. Canadian Journal of Anaesthesia 47(10), 974-979. https://doi.org/10.1007/BF03024868 Basak, T., Demirtas, A., & Iyigun, E. (2019). The effect of simulation based education on patient teaching skills of nursing students: A randomized controlled study. Journal of Professional Nursing, 35, 417-424. https://doi.org/10.1016/j.profnurs. 2019.02.004 Cohen J. E. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum Associates, Inc. Council on Accreditation of Nurse Anesthesia Educational Programs. (2019). Standards for accreditation of nurse anesthesia educational programs. https://www.coacrna.org/wpcontent/uploads/2020/01/2004-Standards-for-Accreditation-of-Nurse-AnesthesiaEducational-Programs-revised-October-2019.pdf Dalley, P., Robinson, B., Weller, J., & Caldwell, C. (2004). The use of high-fidelity human patient simulation and the introduction of new anesthesia delivery systems. Anesthesia & Analgesia, 99(6), 1737-1741. https://doi.org/10.1213/01.ANE.0000136804.46675.EA Edwards, S., Lee, M., & Sluman, K. (2018). Student-led simulation: Preparing students for leadership. Nursing Management. https://doi.org/10.7748/nm.2018.e1778 DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 32 Ekert, J.O., Smith, A.L., Ramsey, C.L., Robinson, N., Love, J., Gothard, P., & Armitage, A.J. (2020). Medical student-led simulation in COVID-19 crisis. The Clinical Teacher 18(3), 252-257. https://doi.org/10.1111/tct.13308 Germain, M., OLeary-Kelley, C., Goyal, D., & Anand, S. (2018). Can clinical simulation increase confidence levels in performing postpartum care in a diverse group of baccalaureate nursing students. Nursing Education Perspectives, 39(2), 94-95. https://doi.org/ 10.1097/01.NEP.0000000000000233 Gray, B., & Cavner, J. (2017). Using simulation to enhance education in obstetrical nursing course. Nursing Education Perspectives, 38(1), 40-41. https://doi.org/ 10.1097/01.NEP.0000000000000099 Kahraman, A., Gumus, M., Binay, S., Zengin, D., Uzsen, H., Sevgili, S. A., Ozdemir, H. N. C., & Basbakkal, Z. (2019). The effect of simulation-based education on childhood epileptic seizure management knowledge, skills, and attitudes of nursing students. Epilepsy & Behavior, 100(A), 1-7. https://doi.org/10.1016/j.yebeh.2019.106497 Khalili, H. (2015). Clinical simulation practise framework. The Clinical Teacher, 12(1), 32-36. https://doi.org/10.1111/tct.12291 Larson, E.R., Nuttall, G.A., Ogren, B.D., Severson, D.D., Wood, S.A., Torsher, L.C., Oliver, W.C., & Marienau, M.E. (2007). A prospective study on anesthesia machine fault identification. Anesthesia & Analgesia, 104(1), 154-156. https://doi.org/10.1213/01.ane.0000250225.96165.4b Lemarie, P., Vidal, S.H., Gergaud, S., Verger, X., Rineau, E., Barton, J., Parot-Schinkel, E., Hamel, J., & Lasocki, S. (2019). High-fidelity simulation nurse training reduces unplanned interruption of continuous renal replacement therapy sessions in critically ill DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 33 patients: The simHeR randomized controlled trial. Anesthesia & Analgesia, 129(1), 121128. https://doi.org/10.1213/ANE.0000000000003581 Li, J., Li, X., Gu, L., Zhang, R., Zhao, R., Cai, Q., Lu, Y., Wang, H., Meng, Q., & Wei, H. (2019). Effects of simulation-based deliberate practice on nursing students communication, empathy, and self-efficacy. Journal of Nursing Education, 58(12), 681689. https://doi.org/10.3928/01484834-20191120-02 Loeb, R., Wahr, J.A., & Nussmeier, N.A. (2019). Anesthesia machines: Prevention, diagnosis, and management of malfunctions. UpToDate. Retrieved October 26, 2020, from https://bit.ly/3o2YVJ1 Lorraway, P.G., Savoldelli, G.L., Joo, H.S., Chandra, D.B., Chow, R., & Naik, V.N. (2006). Management of simulated oxygen supply failure: Is there a gap in the curriculum? Anesthesia & Analgesia, 102(3), 865-867. https://doi.org/10.1213/01.ane.0000195548.38669.6c Mehta, S.P., Eisenkraft, J.B., Posner, K.L., & Domino, K.B. (2013). Patient injuries from anesthesia gas delivery equipment: A closed claims. Anesthesiology 119, 788-795. https://doi.org/10.1097/ALN.0b013e3182a10b5e Morgan, P.J., Cleave-Hogg, D., & Tarshis, J. (2003). Identification of gaps in the achievement of undergraduate anesthesia educational objectives using high-fidelity patient simulation. Anesthesia & Analgesia, 97(6), 1690-1694. https://doi.org/10.1213/01.ANE.0000086893.39567.D0 Mudumbai S.C., Fanning R., Howard, S.K., Davies F.M., & Gaba D.M. (2010). Use of medical simulation to explore equipment failures and human-machine interactions in anesthesia DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 34 machine pipeline supply crossover. Anesthesia & Analgesia, 110(5), 1292-1296. https://doi.org/10.1213/ANE.0b013e3181d7e097 Schwid, H.A., Rooke, G.A., Carline, J., Steadman, R.H., Murrray, W.B., Olympio, M., Tarver, S., Steckner, K., & Wetstone, S. (2002). Evaluation of anesthesia residents using mannequin-based simulation: A multiinstitutional study. Anesthesiology 97, 1434-1444. https://doi.org/10.1097/00000542-200212000-00015 Saied, H. (2017). The impact of simulation on pediatric nursing students knowledge, self efficacy, satisfaction, and confidence. Journal of Education and Practice, 8(11), 95-102. https://files.eric.ed.gov/fulltext/EJ1139780.pdf Svellingen, A. H., Roykenes, K., Forstronen, A., Assmus, J., & Brattebo, G. (2021). Examining predictive factors of nursing students self-confidence in multiple simulation sessions: A randomized controlled study. Nurse Education in Practice, 57, 1-6. https://doi.org/10.1016/j.nepr.2021.103231 Tamaki, T., Inumaru, A., Yokoi, Y., Fujii, M., Tomita, M., Inoue, Y., Kido, M., Ohno, Y., & Tsujikawa, M. (2019). The effectiveness of end-of-life care simulation in undergraduate nursing education: A randomized controlled trial. Nurse Education Today, 76, 1-7. https://doi.org/ 10.1016/j.nedt.2019.01.005 Waldrop, W.B., Murray, D.J., Boulet, J.R., & Kras, J.F. (2009). Management of anesthesia equipment failure: a simulation-based resident skill assessment. Anesthesia & Analgesia, 109(2), 426-433. https://doi.org/10.1213/ane.0b013e3181aa3079 Weller, J., Merry, A., Warman, G., & Robinson, B. (2007). Anaesthetists' management of oxygen pipeline failure: room for improvement*. Anaesthesia, 62(2), 122-126. https://doi.org/10.1111/j.1365-2044.2006.04899.x DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 35 Appendix A Theoretical Framework Figure 1. Depicts a visual representation of Khalilis Clinical Simulation Practise Framework: A knowledge to action strategy in health profession education. (Khalili, 2015). DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 36 Appendix B GANTT Chart DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 37 DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 38 Appendix C SWOT Analysis DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 39 Appendix D Literature Review Matrix Reference Morgan, P.J., CleaveHogg, D., & Tarshis, J. (2003). Identification of gaps in the achievement of undergraduate anesthesia educational objectives using highfidelity patient simulation. Anesthesia & Analgesia, 97(6), 1690-1694. https://doi.org/10.1213/ 01.ANE.0000086893.3 9567.D0 Rese arch Desi gn & Leve l of Evid ence Case contr ol non exper iment al study (Lev el 4) Theo retic al/ Conc ept Fra mew ork N/A Purpo se/Ai m Populat ion/Sam ple n=x Variabl es To identif y key educati onal gaps in medica l student s knowl edge using human patient simula tion that should be addres sed in the anesth esia curricu lum 135 students Defibril late Instru ments/ Data Collect ion Checkli st was develop Manuall ed for y each ventilat scenari e with o 100% oxygen Note other vital signs Call for defibrill ator/cra sh cart Check lung fields for air entry Give intraven ous fluids Have plan if ventilati on unsucce ssful Verbali ze possible diagnos is/plan Result s Impli cation s for Futur e Resea rch Implic ations for Future Practi ce 75% of student s achiev ed 11 of 18 identifi ed objecti ves Inclus ion of machi ne check s or proble ms as an educat ional object ive Simula tion technol ogy therefo re present s educat ors with the opport unity to assess student s perfor mances using standar dized cases and to determ ine whethe r the educati onal objecti ves set out in the curricu lum are being matche d by perfor mance in a 50% of subject s did not confir m the blood pressur e before institut ion of treatm ent for tachyc ardia 40% of residen ts did not decrea se anesth etic DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 40 Initiate Basic or Advanc ed Cardiac Life Support protocol s Prepare to intubate Initiate treatme nt Increase oxygen concent ration Repeat laryngo scopy/s ome differen t airway maneuv er History focusin g on cardiov ascular or respirat ory system Turn off anesthet ic Call for help Confir m blood pressure Check endotra clinical setting DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 41 cheal tube through cords Dalley, P., Robinson, B., Weller, J., & Caldwell, C. (2004). The use of high-fidelity human patient simulation and the introduction of new anesthesia delivery systems. Anesthesia & Analgesia, 99(6), 17371741. https://doi.org/10.1213/ 01.ANE.0000136804.4 6675.EA rando mize d, contr olled, prosp ectiv e trial (Lev el 2) N/A To suppor t the use of simula tion with introdu ction of novel compl ex anesth etic equip ment to improv e the ability to manag e subseq uent critical incide nts and provid e insight into potenti al design errors withou t risk to patient 15 enrolled participa nts Tested particip ants underst anding of the pressure limiting function Tested the particip ants underst anding of the FGF control and the ability of the Drager Fabius GS to ventilat e the patient with entraine d room air Scenari o A: The times from intubati on to the recogni tion of severe bronch ospasm and initiatio n of effectiv e treatme nt were recorde d. Scenari o B: Times were recorde d from complet ion of the handov er of care to the provisi on of an adequat e FiO2, minute volume, and anesthe tic vapor Assess ments No statisti cally signifi cant differe nces betwee n groups for any of the questio n scores Three partici pants said that they would feel confid ent using the machi ne after a day of using it or after a period of familia rizatio n One partici pant felt confid ent to use the machi ne in a It would be prefer able to develo p practi cal famili arity with compl ex new equip ment in a safe enviro nment New equip ment should be design ed both to reduce the likelih ood of errors and to increa se the early detecti on of errors that do occur Anesth esiolog ists cannot reliabl y assess their ability to safely use the equipm ent in clinical practic e Simula tion provid es an excelle nt opport unity to assess equipm ent design withou t exposi ng patient s to potenti al risk HPS can improv e the ability of practiti oners to safely DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 42 of the times taken to crisis resoluti on reviewe d by videota pe analysis by an indepen dent observe r (CC) blinded to group allocati on Weller, J., Merry, A., Warman, G., & Robinson, B. (2007). Anaesthetists' management of oxygen pipeline failure: room for improvement*. Anaesthesia, 62(2), 122-126. https://doi.org/10.1111/ j.13652044.2006.04899.x Case contr ol non exper iment al study (Lev el 4) N/A To establi sh the benefit s of compu terized patient simula tion in identif ying deficit s in manag ement of equip 20 anesthet ists Familia rity with anesthe sia equipm ent and recognit ion of oxygen pipeline supply failure Preoperativ e check of their Errors made by study particip ants during the manage ment of the two clinical crises were recorde d Direct observa tion, video recordi ng, and automat ed monitor printout s (includi ng gas analysis ) normal case, but not in a crisis use new equipm ent Four partici pants did not feel confid ent to use the Machi ne In Scenar io A, no errors were made by Group 1 partici pants, wherea s 14 errors were made by Group 2 All anesth etists mainta ined ventila tion throug hout the case 70% had not discov ered backup Behav ior in the simula tor may not accura tely reflect the partici pants' norma l practi ce In ad eq uat e ma na ge me nt of an ev ent by an ent ire DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 43 mentrelated events equipm ent oxyge n cylind er was empty preoperati vely Provisi on of adequa te anesth esia during the pipelin e failure was variabl e No anesth etist discon nected the wall pipelin e supply co ho rt of an est het ist s wo ul d su gg est a fu nd am ent al ed uc ati on al pr ob le m re qu iri ng a sp eci alt ywi de res po ns e Data strongl y suggest that anesthe tists ought to be fully DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 44 Mudumbai S.C., Fanning R., Howard, S.K., Davies F.M., & Gaba D.M. (2010). Use of medical simulation to explore equipment failures and humanmachine interactions in anesthesia machine pipeline supply crossover. Anesthesia & Analgesia, 110(5), 1292-1296. https://doi.org/10.1213/ ANE.0b013e3181d7e0 97 Prosp ectiv e Desc riptiv e Stud y (Lev el 6) N/A To show how highfidelity medica l simula tion could both teach anesth esia residen ts about equip ment functio n and exami ne their manag ement strateg ies during an equip mentrelated crisis 3 rd yr anesthes ia resident s (1 4 months before graduati on) in groups of 2 (n=20) pipeline O2 and N2O supplie d from hose drops auxiliar y O2 flowmet er low O2 and high N2O alarms modes of ventilati on Video recordi ngs with microp hones One group was never able to definiti vely provid e O2 to the simula ted patient Two groups persist ed with the crosse d O2 flowm eter 5 groups ensure d adequa te oxyge nation either with an extern al O2 tank or Larger sampl e size and the use of only multip le types of anesth esia machi ne compet ent in the manag ement of oxygen failure and that there may be room for improv ement. May be useful to consid er a method such as highfidelity medica l simulat ion to expose potenti al causes of these crises and possibl e better manag ement strategi es DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 45 with air from the anesth esia machi ne itself Larson, E.R., Nuttall, G.A., Ogren, B.D., Severson, D.D., Wood, S.A., Torsher, L.C., Oliver, W.C., & Marienau, M.E. (2007). A prospective study on anesthesia machine fault identification. Anesthesia & Analgesia, 104(1), 154156. https://doi.org/10.1213/ 01.ane.0000250225.961 65.4b Prosp ectiv e Stud y (Lev el 6) N/A To see if there has been any improv ement in the ability of practiti oners to detect preset anesth esia machi ne faults and if duratio n of practic e is related to the ability to detect such faults 87 voluntee rs Five preset faults: data collecti on tool 1. Leak in the water trap Kruskal Wallis test 2. Empty oxygen cylinder , 3. Sticky exhalati on valve, 4. Dead backup battery 5. Remova l of the oxygen/ nitrous oxide fail-safe linkage pairwis e compar isons of the three experie nce groups only 3 groups recogn ized the high N2O alarm 74.7% of subject s detecte d the leak in the water trap and the empty oxyge n cylind er 50.6% and 49.4% found the dead battery and oxyge n/nitro us failsafe linkag e discon nect Ten partici pants succes Provid e famili arizati on with the provid ed machi ne before the test is condu cted Preve nt risk of comm unicat ion amon g partici pants There continu es to be proble ms detecti ng anesthe sia machin e faults despite the publica tion of multipl e checkli st There is a need for continu ed educati on of anesthe sia person nel regardi ng detecti on of anesthe DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 46 sfully found all five faults Lorraway, P.G., Savoldelli, G.L., Joo, H.S., Chandra, D.B., Chow, R., & Naik, V.N. (2006). Management of simulated oxygen supply failure: Is there a gap in the curriculum? Anesthesia & Analgesia, 102(3), 865-867. https://doi.org/10.1213/ 01.ane.0000195548.386 69.6c Coho rt study (Lev el 4) N/A To evaluat e the unders tandin g and manag ement of a simula ted oxyge n pipelin e failure by residen ts in an anesth esiolog y trainin g progra m 20 resident s Recogni zes loss of pipeline oxygen supply Recogni zes the O2 supply and pressure alarms Opens O2 cylinder on machin e Recogni zes O2 cylinder is empty Calls for a new O2 cylinder Change s O2 cylinder successf ully Ventilat es with Ambu bag Videota ped Three subject s detecte d zero faults Majori ty of subject s did not attemp t to open or change the cylind er even after being prompt ed by the surgeo n None of the subject s respon ded by increas ing the flow of air in the gas machi ne Most subject s did not recogn ize the origin of the alarms indicat sia machin e faults The presen ce of a secon d group would have helped to draw strong er conclu sions. Oxyge n supply failure is poorly underst ood and subopti mal manag ed by anesthe siology residen ts. Highfidelity patient simulat ion is a useful tool to identif y gaps in anesthe sia trainee knowle dge Patient simulat ion may be a useful tool to identif y areas of weakn DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 47 Anticip ates patient awakeni ng Waldrop, W.B., Murray, D.J., Boulet, J.R., & Kras, J.F. (2009). Management of anesthesia equipment failure: a simulationbased resident skill assessment. Anesthesia & Analgesia, 109(2), 426-433. https://doi.org/10.1213/ ane.0b013e3181aa3079 Coho rt study (Lev el 4) N/A To develo p a set of scenari os that effecti vely measur e skill in manag ing anesth esia equip ment failure and to evaluat e the psycho metric propert ies of the partici pants scores, includi ng their 56 resident s 8 Scenari os: 1) Endotra cheal tube cuff rupture 2) Large defect (tear)in plastic disposa ble gas circuit 3) Loss of central oxygen supply with empty O2reser ve cylinder s ing an oxyge n supply failure. Time it took until action was complet ed Most subject s did not know how to change the oxyge n cylind er or how to open the reserve supply. Each residen t increase d as a functio n of trainin g level. The residen ts with more trainin g and experi ence were able to accom plish these actions in a shorter period of time ess in a trainin g curricu lum in which gaps in knowle dge may lead to catastr ophic outcomes. Utilizi ng reside nts from variou s anesth esia progra ms to limit the genera lizabil ity of the results Use more scenar ios of the many types of equip ment failure s Reside nts with more trainin g and experie nce perfor med better Anesth esia residen ts should acquire the skills to recogni ze and manag ea numbe r of reporte d causes of DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 48 validit y and reliabil ity 4) CO2 absorbe nt canister misalig nment with no gas delivery 5) obstruct ed endotra cheal tube, 6) disconn ected Spo2an d endtidal CO2 monitor s with no immedi ate patient problem 7) Inhaled anesthet ic (isoflur ane) overdos e with associat ed bradyca rdia and hypoten sion 8) Absenc e of the expirato ry valve gasket with associat experi enced in anesth esiolo gy anesthe sia equipm ent failure during residen cy. Modeli ng these equipm ent failures in a simulat ed enviro nment, reliabl y evaluat e trainee s strengt hs and weakn esses, providi ng guidan ce concer ning relevan t feedba ck, and detect potenti al educati onal proble ms in the trainin g progra m DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 49 Armstrong-Brown, A., Devitt, J.H., Kurrek, M., & Cohen, M. (2000). Inadequate preanesthesia equipment checks in a simulator. Canadian Journal of Anaesthesia 47(10), 974-979. https://doi.org/10.1007/ BF03024868 Quas iexper iment al (Lev el 3) N/A Exami ne how differe nces in age or type of practic e impact s how anesth esiolog ists check their equip ment. 120 participa nts ed proxima l circuit leak High pressure system Videota ped Low pressure system Ventilat or function Scaven ging system Emerge ncy ventilati on system Airway equipm ent and suction Schwid, H.A., Rooke, A., Carline, J., Steadman, R.H., Murrray, W.B., Olympio, M., Tarver, S., Steckner, K., & Wetstone, S. (2002). Evaluation of anesthesia residents using mannequin-based simulation: A multiinstitutional study. Anesthesiology 97, 1434-1444. https://doi.org/10.1097/ 00000542-20021200000015 Casecontr ol Coho rt study (Lev el 4) N/A To assess the validit y and reliabil ity of the gradin g system by measur ing constr uctrelated validit y to 99 anesthes ia resident s diagnos tic observa tions announ ced time to reestabl ish ventilati on Time laryngo scope is remove d from mouth videota ped Medic al student s score on averag e lower than anesth esia residen ts or anesth esiolog ists No correla tion betwee n square root transfo rmatio n of score, age, or years of practic e constr uctrelated validit y of manne quinbased simula tor assess ment was suppor ted by an overall improv ement A more robust , compl ete checkl ist and less string ent time constr aint Requir e anesthe siologi sts to docum ent what checks they have perfor med at the start of each case to improv e compli ance with checki ng proced ure. Preve ntion of selecti on bias Use of a short form singlepoint checkli st gradin g forms is adequa te enough to determ ine reliabil ity and DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 50 Ekert, J.O., Smith, A.L., Ramsey, C.L., Robinson, N., Love, J., Gothard, P., & Armitage, A.J. (2020). Medical student-led simulation in COVID19 crisis. The Clinical Teacher 18(3), 252257. https://doi.org/10.1111/ tct.13308 Edwards, S., Lee, M., & Sluman, K. (2018). Student-led simulation: Preparing students for leadership. Nursing Management. https://doi.org/10.7748/ nm.2018.e1778 Casecontr ol study (Lev el 4) obser vatio nal study (Lev el 6) N/A N/A determ ine if the simula tor evaluat ion is a legitim ate indicat or of perfor mance To assess if clinica l simula tion session s genera ted by medica l student s can improv e confid ence in healthc are worker s To determ ine if student -led simula tion provid es a in simula tor scores 36 participa nts Undefin ed number of students Confide nce scores Learnin g outside their comfort zone Preand postsimulati on questio nnaire Selfreflecti on Peer Review validit y of simulat or perfor mance Respo ndents reporte d being signifi cantly more confid ent after the trainin g in all aspects of manag ing COVI D-19 patient s collect respon ses from all staff memb ers to genera te a larger sampl e size Studen ts enjoye d and apprec iated student led simula Provid e more time to carry out assess ments confid ence scales are not an accura te reflect ion of ones object ive perfor mance Medica l student s can rapidly design and deliver clinical simulat ion trainin g session s for a multidiscipli nary team of hospita l staff and signific antly increas e the confide nce of partici pants in times of crisis Demon strates that teacher s support student input. Studen DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 51 progre ssive learnin g enviro nment for later leaders hip roles, profess ional develo pment, and provid e exposu re to future profess ional role situati ons Lemarie, P., Vidal, S.H., Gergaud, S., Verger, X., Rineau, E., Barton, J., ParotSchinkel, E., Hamel, J., & Lasocki, S. (2019). High-fidelity simulation nurse training reduces unplanned interruption of continuous renal replacement therapy sessions in critically ill patients: The simHeR randomized controlled trial. Anesthesia & Analgesia, 129(1), 121128. https://doi.org/10.1213/ ANE.00000000000035 81 Rand omiz ed Contr olled, singl ecente r, open study (Lev el II) N/A To assess if additio nal nurse trainin g progra m based on highfidelity simula tion would reduce the rate of interru pted session s Compet ence levels tion approa ch and challen ges Collabo rative skills Reactio n to being led by peers and write up selfpeer assess ments Differ ent locati ons with visible hazard s Challen ge in role of student lead Decisio n making 30 nurses Stress levels Confide nce levels knowle dge questio nnaire test Satisfac tion with the training and apprehe nsion to begin CRRT evaluat ed using a numeri cal scale Interve ntion group had better results in the knowl edge test Confid ence was higher and stress was lower in interve ntions group blind study to caregi vers or study coordi nators to protec t from bias Provid e equal time for trainin g betwe en contro t led simulat ion allows student s to explore lived experie nces Uses student s stories in a way that offers insight into their field of practic e and the daily situatio n they will engage in Th use of highfidelity simulat ion can optimi ze care deliver y in ICU setting s and improv e patient safety DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 52 l and interv ention nurses DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 53 Appendix E Pre- and Post-Test Questionnaire DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 54 DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 55 DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 56 DocuSign Envelope ID: FA6F30A3-2A24-43B3-B87C-324DC716FD8B 57  ...

... PNEUMATIC TOURNIQUET CUFF PRESSURES AND IMPLICATIONS 1 Comparing Patient Systolic Blood Pressures with Tourniquet Inflation pressures and the Effects on Estimated Blood Loss. (A retrospective chart review) Anthony Dzigbordi Marian University Leighton School of Nursing Project Chair: Dr. Sara Franco DNAP, CRNA Site Mentor: Dr. Ryan Sexton DNP, CRNA PNEUMATIC TOURNIQUET CUFF PRESSURES AND IMPLICATIONS 2 Table of Contents Abstract....4 Introduction......4 Background..6 Problem Statement...6 Needs Assessment/Organizational Gap Analysis....7 Review of the Literature..8 Evidence Based Practice: Verification of Chosen Option.11 Theoretical Framework..12 Goals & Objectives....13 Project Design....14 Project site and Population.....15 Project site Strengths, Weaknesses, Opportunities and Threats (SWOT) Analysis...15 Strengths.15 Weaknesses.16 Opportunities..16 Threats16 Methods..17 Measurement Instrument18 Data Collection Procedure..18 Data Analysis..19 Results.20 Interpretation/Discussion21 PNEUMATIC TOURNIQUET CUFF PRESSURES AND IMPLICATIONS 3 Cost-Benefit Analysis/Budget23 Timeline.23 Ethical Considerations/Protection of Human Subjects..24 Conclusion.24 References..26 Appendix29 Appendix A29 Appendix B32 Appendix C33 Appendix D34 Appendix E35 Appendix F36 Appendix G37 PNEUMATIC TOURNIQUET CUFF PRESSURES AND IMPLICATIONS 4 Abstract During surgical interventions of upper or lower extremities, one of the methods in which blood loss can be decreased is through application of the pneumatic tourniquet. A bloodless field provides important advantages for the surgeon, and the pneumatic tourniquets provide a relatively good bloodless field to minimize intraoperative blood loss. (Naglehout & Elisha, 2018). Recent literature supports the use of personalized tourniquet inflation pressures (PTIP) as opposed to universal tourniquet inflation pressures. (UTIP) The purpose of this DNP project was to investigate what systolic blood pressures were measured on patients undergoing lower extremity surgeries prior to tourniquet inflation, what tourniquet inflation pressures were being used, and determine whether these tourniquet inflation pressures conform with PTIP, or UTIP standards, and further determine their effects on estimated blood loss (EBL), and other tourniquet related complications. This project comprises of a retrospective review of patient charts. A total of 186 electronic anesthesia records were reviewed and 75 patients were included in the DNP project. Identification of a project site was the first step. Approval was sought from the health facility and the various internal review boards (IRBs). A retrospective chart review was conducted to examine three full months of surgeries involving the lower extremities which required the use of pneumatic tourniquets. Data was collected and subsequently analyzed by use of Microsoft Excel. This data included the systolic blood pressures of patients prior to tourniquet inflation, tourniquet inflation pressures, ASA physical status, patient BMI, age and Gender. By completion of this retrospective chart review, it has been discovered that 75 patients received lower extremity surgeries with the use of pneumatic tourniquets. Of the 75 patients, tourniquets were inflated to 275mmHg on 57 (76%) of them regardless of their systolic blood PNEUMATIC TOURNIQUET CUFF PRESSURES AND IMPLICATIONS 5 pressure (SBP). 63 patients (84%) had a recorded EBL less than 100mL. No tourniquet related complications were recorded. Only one patient (1.3%) had a tourniquet inflation pressure greater than 300mmHg, with EBL recorded at 10mL. Keywords: Tourniquet inflation pressure, systolic blood pressure, estimated blood loss. PNEUMATIC TOURNIQUET CUFF PRESSURES AND IMPLICATIONS 6 Comparing Patient Systolic Blood Pressures with Tourniquet Inflation pressures and the Effects on Estimated Blood Loss. (A retrospective chart review) Background Despite the benefits to the surgeon, pneumatic tourniquets can cause multiple complications if specific criteria are not followed in their application. Some specific complications include tourniquet pain, hypertension, and postoperative neurapraxia. Evidence of nerve injury at the skin level at the edge of the tourniquet has also been reported. (Naglehout & Elisha, 2018). One significant safety measure for preventing tourniquet related complications is ensuring that only minimally effective pressure is used for occluding blood flow to the extremity. According to Naglehout and Elisha, tourniquet pressures for upper extremity should be kept at 70-90 mmHg more than the patients systolic blood pressure, and that of the lower extremity should be kept at twice the patient's systolic blood pressure or 300mmHg if twice the systolic is less. For the purposes of this research, focus will be placed solely on lower extremity surgeries. Problem Statement In actual practice however, some surgeons have requested tourniquet pressures much higher than these thresholds. Surgeons commonly inflate lower extremity tourniquets to a standard pressure of 350mmHg. (Kim & Kim, 2018). These practices are not ideal as according to Kim and Kim, there have been cases of tourniquet related complications such as nerve damage following the use of standard inflation pressures, which can be significantly high for some patients. There have also been multiple case reports of cardiac arrest following pneumatic tourniquet deflation after lower extremity tourniquets inflation to 350mmHg for one hour and forty-five minutes. (Spruce, 2017). PNEUMATIC TOURNIQUET CUFF PRESSURES AND IMPLICATIONS 7 Needs Assessment/Organizational Gap Analysis There is the need for an individualized tourniquet inflation pressure tailored to each patient based on systolic blood pressure, limb circumference and limb occlusion pressure. (Spruce, 2017). Other factors that should be considered for the safe application of tourniquets, based on individual needs rather than using facility standard tourniquet inflation pressures include; patients age, skin, shape and size of the extremity, and the dimensions of the cuff being used. (Kumar et al, 2016). According to Ding et al., (2019), the individualized pressure setting strategy can provide a lower inflation pressure (four studies), improve hemostatic effect (six studies) and reduce the incidence of related complications in patients undergoing orthopedic surgery. It has therefore become imperative to investigate what tourniquet inflation pressures were being used by surgeons at this facility, to find out if they mimic a pre-determined value for every patient or whether they were individualized pressure settings based on patient systolic blood pressure. This project will then investigate if there were any differences in the recorded estimated blood loss (EBL), based on the tourniquet pressures used. This has led to the development of the following PICOT question, In patients undergoing orthopedic surgery of the lower extremities, what is the effect of individualized tourniquet inflation pressures compared to facility standard inflation pressures on blood loss and other tourniquet related complications? This is to allow the conduction of a retrospective chart review, to compare the various tourniquet pressures that have been used for surgery of the lower extremities, and to determine their effects on blood loss and presence or absence of any tourniquet related complications. PNEUMATIC TOURNIQUET CUFF PRESSURES AND IMPLICATIONS 8 Review of the Literature A current literature review was conducted in October 2021. The search terms utilized in this literature search included tourniquet inflation pressures, lower extremity surgeries, blood loss and tourniquet related complications. The databases that were searched include CINAHL, PubMed and Google Scholar. Articles were included if they are primary research, systematic reviews, meta-analysis or randomized controlled trials (RCTs) which focused on the use of pneumatic tourniquets during lower extremity surgery, tourniquet inflation pressures, and effects on blood loss or tourniquet related complications in adult patients under general anesthesia. These articles must be most recent hence should have been published after the year 2015. A search conducted through PubMed database yielded a total of 128 articles initially. Ten (10) articles met the inclusion criteria and were included in the literature review. (See Appendix A). Four articles were randomized controlled trials, three were meta-analysis, two were systematic reviews while one article was primary research. Articles that did not meet the inclusion criteria were those that focused on tourniquet use in upper extremity surgery, tourniquet pain, tourniquet duration and associated complications, and articles that compared surgery with use of tourniquets versus surgery without tourniquets. Refer to Appendix C for Prisma diagram on literature search. Pneumatic tourniquets are commonly used to reduce intraoperative bleeding during surgery of the extremities. Many surgeons prefer to perform extremity surgery such as total knee replacement with the aid of a pneumatic tourniquet. A tourniquet is an occlusive device which restricts blood flow to the distal extremity to help create a bloodless field during the surgical procedure. (Ahmed et al., 2020). All ten studies, showed that tourniquet use can significantly decrease intraoperative blood loss, calculated blood loss, and the duration of surgery. (Ahmed et al., 2020, Cai et al., 2019, PNEUMATIC TOURNIQUET CUFF PRESSURES AND IMPLICATIONS 9 Ding et al., 2019, Kim et al., 2019, Spruce, 2017). However, one systematic review noted that despite these benefits, complications can arise from the use of tourniquets, and these may include nerve injuries, pressure injuries, pain, tissue necrosis, compartment syndrome and chemical burns. Other serious injuries such as deep vein thrombosis, severe ischemic injuries, thermal damage to tissues, and rhabdomyolysis can also occur. (Spruce, 2017). There are a wide variety of guidelines for deciding on the ideal tourniquet pressure during extremity surgery. However, three RCTs and one primary research showed that the practice of using fixed, high tourniquet pressures based on surgeon preference remains common. (Kim et al., 2019, Kukreja et al., 2018, Figueroa, 2021, Pinsornsak, 2021). All ten studies found that the use of high tourniquet pressures not based on limb occlusion pressure (LOP), lead to increased tourniquet related complications such as pain, excessive use of opioids, and prolonged PACU length of stay which further leads to increased healthcare costs. (Ahmed et al., 2020, Cai et al., 2019, Ding et al., 2019, Kukreja et al., 2018, Spruce, 2017). Two meta-analysis, four RCTs, one primary research and two systematic reviews confirmed that higher tourniquet pressures cause ischemia which has negative effects, such as the appearance of edema, stiffness, increased risk of deep vein thrombosis, wound problems, dysesthesia, a decreased range of motion, and nerve injuries resulting from excessive compression from the tourniquet. (Ahmed et al., 2020, Cai et al., 2019, Ding et al., 2019, Figueroa, 2021, Mateu et al., 2021, Pinsornsak, 2021). According to three RCTs and one meta-analysis, while some surgeons as a matter of personal preference use a uniform tourniquet inflation pressure (UTIP) regardless of the patient, others use personalized tourniquet inflation pressures (PTIP) based on the specific patients systolic blood pressure or the limb occlusion pressure. This is due to the fact that there exists no consensus regarding the optimal mode of inflating tourniquets during lower extremity surgeries. PNEUMATIC TOURNIQUET CUFF PRESSURES AND IMPLICATIONS 10 (Kim et al., 2019, Figueroa, 2021, Pinsornsak, 2021, Sun et al., 2022). Despite the absence of a consensus regarding which pressures to inflate tourniquets, nine out of the ten studies clearly stated that the PTIP provides a similar bloodless surgical field compared with the conventional UTIP while minimizing deleterious effects and complications (Ahmed et al., 2020, Cai et al., 2019, Ding et al., 2019, Kukreja et al., 2018, Spruce, 2017, Sun et al., 2022). One meta-analysis and two RCTs found no significant difference in terms of intraoperative blood loss (P = .48), and (100% vs. 99%, p = 1.000) between the use of PTIP and the UTIP. (Kim et al., 2019, Sun et al., 2022, Tuncali et al., 2018)). The literature overwhelmingly favors the use of PTIP in the sense that it provides less tourniquet pain intensity, optimal thigh circumference, lower rate of thigh ecchymosis, lower rate of limb ischemia or nerve damage, and better initial recovery of knee flexion in cases of lower extremity surgery. (Cai et al., 2019, Ding et al., 2019, Kukreja et al., 2018, Sun et al., 2022). Three RCTs and two systematic reviews further identified that patient populations such as those with osteoarthritis undergoing lower extremity surgeries benefit from PTIP since it provides a similar bloodless field as the UTIP but carries the added advantages of having a 20% lower rate of wound complications at discharge and a 24% lower rate of wound complications after two weeks when compared with the UTIP. (Figueroa, 2021, Mateu et al., 2021, Pinsornsak, 2021). A special study conducted on the obese population however showed that compared to non-obese patients, the obese patients required higher tourniquet inflation pressures during total knee arthroplasty since they have a wider extremity circumference in conjunction with a higher systolic blood pressure profile. (Tuncali et al., 2016). PNEUMATIC TOURNIQUET CUFF PRESSURES AND IMPLICATIONS 11 Finally, one study investigated the use of PTIP via the use of ultrasonic doppler for setting lower extremity tourniquet pressures during surgery which ensured great hemostatic effect, while providing optimal individual tourniquet pressure value for the patient. Meanwhile, it also effectively reduced the incidence of adverse reactions associated with tourniquet use and improved tourniquet safety. (Mu et al., 2018). All ten studies affirmed support for the use of the PTIP as opposed to the UTIP. There had been comparable outcomes in terms of limiting blood loss and the provision of a bloodless field by both the PTIP and the UTIP. Juxtaposing these results with the tourniquet related complications that are markedly increased in the use of the UTIP compared to the PTIP, it becomes evident that the PTIP provides the better option in terms of overall patient outcomes. Evidence Based Practice: Verification of Chosen Option The use of pneumatic tourniquets during extremity surgery is very common and a relatively safe practice. The surgical team's proper assessment, knowledge of the patient history, indications, contraindications and risks of tourniquet use can help prevent undesirable outcomes for the patient. (Jensen et al., 2019). According to the Association of peri-Operative Registered Nurses (AORN) guidelines for tourniquet use, it is imperative to confirm the intended tourniquet pressure setting during the time out, and before the tourniquet inflation. The tourniquet must be inflated to a minimum effective pressure as determined by the surgeon based on the patient's systolic blood pressure and limb circumference. (AORN, 2019). A review of current literature in relation to the use of pneumatic tourniquets showed that several studies favor the use of PTIP based on limb occlusion pressure or the patients systolic pressure. The literature overwhelmingly suggests that, PTIPs provide similar bloodless surgical field compared with the conventional UTIP while minimizing PNEUMATIC TOURNIQUET CUFF PRESSURES AND IMPLICATIONS 12 deleterious effects and complications. (Ahmed et al., 2020, Cai et al., 2019, Ding et al., 2019, Kukreja et al., 2018, Mateu et al., 2021, Pinsornsak, Spruce, 2017, Sun et al., 2022). This DNP project seeks to evaluate the overall adherence to the personalized tourniquet inflation pressures at the project site and if this recommendation is serving the intended purpose of minimizing blood loss while reducing tourniquet related complications. Theoretical (Conceptual) Framework One significant concept in nursing that had been under consideration for many years is reflection. It is a concept introduced by Dewey in the year 1933. (Galutira, 2018). The Theory of Reflective Practice in Nursing is a middle-range theory which emphasizes the necessity of nurses to practice reflection in four domains. These four domains include; reflection-before-action, reflection-in-action, reflection-on-action, and reflection-beyond-action. This is to help advance nursing practice considerably. (Galutira, 2018). Reflective practice has been found to possess the ability to impact positive outcomes such as professional or personal development, improvement in quality of care, and improved outcomes of care. (Redmond, 2017). Because of its undisputable ability to promote continuous development, reflective practice is considered as a crucial part of professional practice. When reflection is employed as a process of discovering alternative types of nursing knowledge such as aesthetic, empirical, ethical and personal forms, this leads to change in practice. The Theory of Reflective Practice in Nursing comprises of five concepts. These include; reflection, promoting factors, clinical experience, outcomes and hindering factors. Refer to appendix B for a diagram demonstrating how these five concepts are interrelated. The four domains of the theory of reflective practice have been defined thus; Reflection-before-action emphasizes a providers duty to reflect before engagement into clinical activity. Reflection-inaction comprises of active reflective thinking and moment to moment decision making during PNEUMATIC TOURNIQUET CUFF PRESSURES AND IMPLICATIONS 13 the course of the clinical activity. Reflection-on-action comprises the retroactive critical analysis of the clinical situation to recreate incidents. Finally, reflection-beyond-action comprises of critical thinking of the decision making and claims incorporated into the descriptions of nurses experiences with reference to clinical practice. (Galutira, 2018). Patient factors such as systolic blood pressure, body habitus and weight, as well as hemodynamic stability must be considered while deciding what tourniquet pressures to use for each patient. A retrospective chart review is a great example of reflection-beyond the action. This comprises of collection and review of data to help discover problems that may exist within clinical practice. When the Theory of Reflective Practice in Nursing is applied within this domain of anesthesia practice, it would encourage healthcare facilities to adopt new evidence-based strategies that support the use of patient centered tourniquet pressures based on systolic blood pressures instead of a generalized approach based on facility protocols or surgeon preference. Project Goals, Objectives and Expected Outcomes. This project was designed with the main objectives of investigating pneumatic tourniquet pressures that were being used for patients undergoing orthopedic surgeries of the extremities in comparison with their average systolic blood pressures and juxtaposing that with the estimated blood loss (EBL) reported. Since the main reasons for using pneumatic tourniquets for extremity surgery are to minimize blood loss through artery occlusion and also provide a bloodless field for the surgeon, it is imperative to use the minimum pressure that is patient specific to achieve these goals. The objectives of this project are therefore listed below: 1. To review the medical records of at least 60 patients that had undergone lower extremity surgeries that required the use of tourniquets by March 2022. PNEUMATIC TOURNIQUET CUFF PRESSURES AND IMPLICATIONS 14 2. Identify and record what the pre-tourniquet inflation systolic blood pressures of these patients were by March 2022. 3. Identify and record the tourniquet inflation pressures used for surgical procedures on these patients by March 2022. 4. Identify and record the EBL reported postoperatively by March 2022 5. Identify if tourniquet inflation pressures used were based on standard facility protocols or if they were tailored to patient specific needs based on their systolic blood pressures by April 2022. 6. Identify and record whether there were any significant differences in EBL between use of higher tourniquet inflation pressures based on standard facility protocols, and patient centered tourniquet pressures based on their systolic blood pressures by August 2022. The expected outcome is to determine if higher tourniquet pressures based on facility protocols directly relate to less estimated blood loss and better creation of a bloodless field Project Design This DNP project is designed to use a retrospective chart review to obtain data that has previously been recorded. This project has therefore employed a quantitative descriptive method to assess adherence to, and utilization of patient centered tourniquet inflation pressures at this project site, and whether the aims of minimizing blood loss while reducing tourniquet related complications were being met. The project used a retrospective convenient sample of adult patients who underwent surgical intervention of the lower extremity requiring the use of pneumatic tourniquets during the first three months of the year 2022. The project aimed to reach a sample size of at least 60 patients. The actual sample size achieved was 75 patients. During review of the patient charts, inclusion of patients in the project sample were determined by PNEUMATIC TOURNIQUET CUFF PRESSURES AND IMPLICATIONS 15 identifying patients who had lower extremity surgeries with documented tourniquet pressures and documented estimated blood losses. Additional data collected comprised of any related complications, patient demographic information such as age, gender, BMI, surgical procedure and ASA physical status. Project Site and Population The implementation of this DNP project was at a Midwestern Hospital in the United States of America. This non-profit organization is a level III Trauma Center with magnet designation. It is located in a predominantly white community and has 191 patient beds. (IU Health 2019). The retrospective chart review has been conducted on adult patients of 18 years or older who underwent lower extremity surgery with the use of pneumatic tourniquets during the stipulated time frame for this project. Children or individuals 17 years of age or younger were excluded, and adults 18 years of age or older who received upper extremity surgeries or lower extremity surgeries without the use of tourniquets were all excluded. Project site Strengths, Weaknesses, Opportunities and Threats (SWOT) Analysis Strengths This healthcare facility has an Electronic Medical Record (EMR) which facilitated a retrospective chart review. It is standard practice at this facility to document time of pneumatic tourniquet inflation, tourniquet pressure and time of tourniquet deflation hence data was readily available. This project does not require new experimentation with medications or procedures hence comes with a high level of safety for the subjects which facilitated the approval from the various Internal Review Boards (IRBs). PNEUMATIC TOURNIQUET CUFF PRESSURES AND IMPLICATIONS 16 No significant financial investments were required for this study since it involved the retrospective review of anesthesia records. Weaknesses Blood loss is usually estimated hence may not be accurately reported and recorded. There has not been adequate availability of data for both topics of comparison because most physicians adhered to the facility protocol of tourniquet inflation pressures as opposed to patient centered pressures based on systolic blood pressure. The success of this project depends on adequate data for both PTIP and UTIP for comparison in order to ensure a fair comparative study. The chart review required a significant time input to unearth all necessary data for this comparative study. Opportunities This healthcare facility is a busy one that sees a great number of orthopedic surgeries of the extremities yearly, hence, able to produce the necessary data for this study. Education and EBP is one of the priorities of this facility hence provided a great support for the project ideas with the necessary guidance. The facility mentor for this project was highly motivated about the prospects of this project and helped in any way possible to ensure a successful study. Threats It is unclear as to how long an anesthesia record remains accessible post patient discharge, hence, some charts may not be accessible for review purposes. PNEUMATIC TOURNIQUET CUFF PRESSURES AND IMPLICATIONS 17 Physician preference to the use of tourniquet inflation pressures may significantly influence which type of data are available and if data may be evenly acquired for both topics of comparison. Previous anesthesia records may not be readily accessible if there has been a recent shift in methods of documentation from paper-based charting to electronic medical records. See Appendix E. SWOT analysis. Methods The main intention of this project is to identify a gap in clinical practice and the extent to which evidence-based practice guidelines on tourniquet inflation pressures were being implemented. A retrospective chart review has been conducted at the chosen facility. A project mentor at the facility has gladly agreed and helped facilitate access to Electronic Health records and also provided guidance and direction when needed. An application for exemption was obtained from the Institutional Review Board (IRB) of Marian University before the commencement of data collection since this project involves data collection on human subjects. The retrospective chart review began once this exception was obtained. There was no need for an informed consent since only data that has previously been recorded have been reviewed and utilized. Charts were reviewed systematically beginning with surgical interventions involving the study group from January 1, 2022 through March 31, 2022. Adult patients who had lower extremity surgeries involving the use of tourniquets within this time frame were identified and data was collected in form of tourniquet inflation pressures, estimated blood loss, tourniquet related complications, surgical procedure, gender, BMI and age. This data was analyzed to determine the number of patients whose tourniquet inflation pressures were based on the PTIP method based on the AORNs guidelines versus the number of patients whose tourniquet PNEUMATIC TOURNIQUET CUFF PRESSURES AND IMPLICATIONS 18 inflation pressures were based on the UTIP method. The estimated blood losses were then compared between the two groups, and any tourniquet related complications reported were also compared. These data were analyzed and recommendations made regarding the use of PTIP versus the UTIP. Measurement Instrument Patient EHRs were the source of all data collected for this project. The obtained data was placed into a Microsoft Excel spreadsheet and analyzed. The systolic blood pressure of the patient and the pressure to which tourniquet was inflated were recorded to determine if the tourniquet inflation pressure was based on PTIP which is 100mmHG above systolic blood pressure or based on conventional UTIP of 300mmHG-400mmHG regardless of patient. Other patient variables such as weight, age, gender was collected and analyzed to determine if any of these variables has an impact on the results of estimated blood losses or tourniquet related complications. Other data that was collected include the types and location of the lower extremity surgery to determine if these played a role in the amount of estimated blood loss. Due to the project being a retrospective chart review, the validity and reliability of measurement devices cannot be determined. Estimated blood loss is usually based on surgeons judgement or the counting and weighing of sponges used during the surgical procedure. Since all these values are estimates, it would be difficult to determine their accuracy, hence the tendency of overestimation or underestimation greatly exists. Data Collection Procedures The data for this DNP project was systematically collected and manually recorded by conducting a retrospective chart review through access to patient EHRs. The facility mentor for this project facilitated access to the patient EHRs for this chart review to be conducted. Charts PNEUMATIC TOURNIQUET CUFF PRESSURES AND IMPLICATIONS 19 were reviewed for the period of January 1, 2022 through March 31, 2022 for patients18 years of age and older, who had lower extremity surgical procedures requiring to use of pneumatic tourniquets. This chart review was completed in April 2022. The acquired data was entered into a Microsoft Excel spreadsheet without any patient identifiable information. This data was then saved withing the Marian Universitys password protected One Drive cloud. The surgical schedule for the period spanning January 1, 2022 through March 31, 2022 was identified and scrutinized for surgeries of adults over the age of 18 receiving lower extremity surgeries. The anesthesia records of these patients were then reviewed to determine if pneumatic tourniquets were used during the surgical procedure. The anesthesia record is further scrutinized to determine what systolic blood pressures were documented prior to the inflation of tourniquets, and the pressures to which the tourniquets have been inflated. Estimated blood losses documented at the end of surgical intervention was also recorded. Other information such as ASA physical status, gender, age and weight was collected once patient meets all the above criteria. Accessing EHRs of surgical patients less than 18 years of age or patients who received upper extremity surgeries, any other surgery that is not orthopedic, or lower extremity surgeries without the use of pneumatic tourniquets was avoided. Data Analysis The main aim of this project is to identify what systolic blood pressures were recorded prior to tourniquet inflation and what tourniquet inflation pressures were being utilized for surgeries of the lower extremities at this facility and juxtapose that with the documented EBL. This will help determine if individualized tourniquet pressures or universal tourniquet pressures were being used and if there were any differences in EBL. Hence, data collected were placed into an excel spreadsheet and analyzed to determine the Mean, Median and Modes of continuous PNEUMATIC TOURNIQUET CUFF PRESSURES AND IMPLICATIONS 20 variables. Analyzing data in Microsoft Excel involved descriptive statistics. Counting was used to measure variables that are nominal such as, ASA physical status and gender. Mean, median and range was calculated for continuous variables such as age, tourniquet pressure and estimated blood loss via Microsoft Excel. The analyzed data was then scrutinized to determine what the facility protocol for tourniquet inflation was, and whether these protocols follow the PTIP or the UTIP methods of choosing tourniquet inflation pressures. Results The EMRs of 75 adult patients who underwent lower extremity surgery during the period of January 1, 2022 through March 31, 2022 were reviewed to determine if tourniquet inflation pressures were based on PTIP or UTIP. Overall, four different tourniquet inflation pressures were recorded at this facility for surgeries of the lower extremities on the 75 patients included in this study. These are; 250mmHg, 275mmHg, 300mmHg, and 350mmHg. Tourniquets were inflated to a pressure of 275mmHG on 57 patients, (76% of the sample size). This is regardless of their systolic blood pressures (SBP). Of the 57patients whose tourniquets were inflated to 275mmHg, 24 patients underwent total knee arthroplasty surgeries, 18 patients underwent knee arthroscopies, 9 patients underwent ankle fracture repair, and 6 patients underwent lower extremity I&D. Also, on these 57 patients, the highest SBP recorded for this group prior to tourniquet inflation was 168mmHg, while the lowest SBP recorded prior to tourniquet inflation was 103mmHg. Tourniquets were inflated to 250mmHg on 10 patients, (13.3% of the sample size). Of these 10 patients, 6 underwent ankle ORIF, and 4 underwent foot and toe I&Ds. The highest SBP recorded prior to tourniquet inflation for this group was 160mmHg and the lowest SBP recorded prior to tourniquet inflation was 96mmHg. Tourniquets were inflated to 300mmHg on 7 patients, (9.3% of the sample size). Of these 7 patients, 5 of them underwent total knee PNEUMATIC TOURNIQUET CUFF PRESSURES AND IMPLICATIONS 21 arthroplasties while 2 patients underwent knee arthroscopy procedures. The highest SBP recorded for this group prior to tourniquet inflation was 153mmHg, and the lowest SBP recorded was 97mmHg. Tourniquet was inflated to 350mmHg for one patient (1.3%). This patient underwent a foot and toe I&D surgery, and had a systolic blood pressure of 119mmHg prior to tourniquet inflation. See Appendix F for pie chart representation of tourniquet pressures used. Of all 75 patients included in the study. The mean SBP recorded prior to tourniquet inflation was 133mmHg, the median SBP was 133mmHg, and the modal SBP was 134mmHg. The mean tourniquet inflation pressure was 275mmHg, the median was 275mmHg, and the mode was 275mmHg. The mean age for sample size was 58 years old, the median age was 61 and the modal age was 66. The oldest age recorded was 86 and the youngest age was 18. Estimated blood losses (EBLs) were documented for all 75 patients. These range from a low of 5mL to a high of 250mL. Of the 75 patients in this study, the EBLs of 45 (60%) fell below 50mL. The EBLs of 18 patients (24%) fell between 51-100mL. The EBLs of 8 patients (10.6%) fell between 101-150mL. Also, the EBLs of 2patients (2.6%) fell between 151-200mL, while the EBLs of another 2 patients (2.6%) fell between 201-250mL. See Appendix G for a bar chart representation of EBLs. In general, the mean EBL recorded was 60.6mL, the median EBL was 50mL, and the modal EBL was 5mL. Discussion/Interpretation The results from this DNP project revealed that despite new evidence suggesting the use of PTIP for better patient outcomes, some providers still utilize the UTIP for inflating pneumatic tourniquets. This statement is true regarding this facility because, despite the diversity in systolic blood pressures recorded prior to tourniquet inflation, 76% of tourniquets were inflated to a PNEUMATIC TOURNIQUET CUFF PRESSURES AND IMPLICATIONS 22 UTIP of 275mmHg. This was regardless of patient systolic blood pressures ranging from a low of 103mmHg to a high of 168mmHg. Despite the use of this UTIP however, it is worth noting that this UTIP has been chosen very carefully to mimic the mean tourniquet inflation pressure of most patients if a PTIP were to be calculated for all of them. This is because, according to the method of calculating PTIP proposed by Naglehout and Elisha (2018), which is based on twice the patients SBP, then the PTIP tourniquet pressure for the highest recorded SBP of 168mmHg in the 275mmHG tourniquet pressure group would have been 336mmHg. And the PTIP tourniquet pressure for the lowest SBP of 103mmHg in this group would have been 206mmHg. This means that the tourniquet inflation pressure 275mmHg fell short by 61mmHg for the patient with SBP of 168mmHg, and was too high by 69mmHg for the patient with SBP of 103mmHg. However, considering Naglehout and Elisha recommends that the tourniquet should be inflated to 300mmHg if twice the patients SBP is less than this value, it means the ideal PTIP for the patient with SBP of 103mmHg should be set at 300mmHg. Hence, instead of viewing the UTIP of 275mmHg used on this patient as too high by 69mmHg, it is actually too low by 25mmHg. Also, taking all 75patients in the study into account, the mean SBP recorded was 133mmHg. A calculated PTIP for these patients based on the mean would therefore be 266mmHg. And since this fell below the 300mmHg tourniquet inflation pressure recommended by Naglehout and Elisha for lower extremity surgeries, it means the tourniquets should have been inflated to 300mmHg on most of these patients. Since the goal of the PTIP is to use the minimum pressure possible for limb occlusion which can minimize blood loss, provide a bloodless surgical field while minimizing tourniquet related complications, the choice of the 275mmHg UTIP at this facility seems to be the safest and best practice for these patients. The findings of this project demonstrated that these goals of PTIP use have been achieved. This is because, the recorded PNEUMATIC TOURNIQUET CUFF PRESSURES AND IMPLICATIONS 23 EBLs of 63 patients representing 84% of the Sample size fell below 100mL while 60% fell below 50mL. Also, as long as these same conditions for tourniquet use are met, the 10 patients whose tourniquets were inflated to 250mmHg received an even safer tourniquet inflation pressures which minimizes most complications of tourniquet use. The one patient whose tourniquet was inflated to a pressure of 350mmHg while having a SBP of 119mmHg prior to tourniquet inflation for a foot and toe I&D surgery, is a major outlier whose peculiar case could not be investigated beyond the scope of this study. It is unclear if this patient was taking any medications that placed him at a higher risk of bleeding which warranted a higher-than-normal tourniquet inflation pressure. Despite the higher pressure, no tourniquet related complications such as those enumerated throughout this study have been documented on this patient. Cost-Benefit Analysis/Budget The implementation of this DNP project did not incur any costs to the facility where the project was implemented. Practicum DNP student hours were utilized by the student to complete the implementation, analysis, and evaluation of this DNP project. However, if this project were to be an initiative of the organization where the project took place, cost of implementation by the organization would be the wages or salary of the person who completes the retrospective chart review. Timeline This DNP project is scheduled to be completed within a total of one calendar year. The project was initiated in August 2021, with the goal of completion by August 2022. (See GANTT chart, Appendix D. However, if this project had to be delayed for a few weeks or months due to any external or unforeseen circumstances, there is always room to adjust these timelines to accommodate such changes. PNEUMATIC TOURNIQUET CUFF PRESSURES AND IMPLICATIONS 24 Ethical Considerations/Protection of Human Subjects The main objective of this DNP project was to evaluate the necessity for practice changes to improve patient safety and optimize patient outcomes during surgical interventions requiring the use of pneumatic tourniquets. Prior to initiating the implementation phase of this project, an exemption was granted by the Institutional Review Board at Marian University on April 2022. No collection of any patient identifying information was made throughout the duration of this project. All the Health Insurance Portability and Accountability Act (HIPAA), standards were strictly maintained during this project. All values of Marian University, especially the value of maintaining the dignity of the individual were upheld during the implementation of this project through respect for the individual and by keeping patient information private. Conclusion During surgical interventions of upper or lower extremities, one of the methods in which blood loss can be decreased is through application of the pneumatic tourniquet. A bloodless field provides important advantages for the surgeon, and the pneumatic tourniquets provide a relatively good bloodless field to minimize intraoperative blood loss. (Naglehout & Elisha, 2018). One significant safety measure for preventing tourniquet related complications is ensuring that only minimally effective pressure is used for occluding blood flow to the extremity. For this reason, there is the need for an individualized tourniquet inflation pressure tailored to each patient based on systolic blood pressure, limb circumference and limb occlusion pressure. (Spruce, 2017). In actual practice however, and for that matter at this facility, some surgeons have requested tourniquet pressures much higher than these thresholds. Surgeons commonly inflate PNEUMATIC TOURNIQUET CUFF PRESSURES AND IMPLICATIONS 25 lower extremity tourniquets to a standard pressure of 350mmHg. (Kim & Kim, 2018). This DNP project was therefore designed to conduct a retrospective chart review to identify the use of patient centered tourniquet inflation pressures compared to universal tourniquet inflation pressures and juxtapose the values with the recorded estimated blood loss (EBL). This is to help identify if there is any difference in the amount of recorded EBL or if both methods provide similar outcomes with regards to the amount of EBL. The retrospective chart review found that a total of 75 patients received lower extremity surgery with pneumatic tourniquet use at this facility between January 2022 through March 2022. Of the 75 patients, the tourniquet inflation pressure was set at 275mmHg for 57 patients (76%). Therefore, the facility UTIP is identified as 275mmHg. This Tourniquet Inflation pressure is lower when compared with the recommendation of Naglehout and Elisha which indicated that tourniquets for lower extremity surgeries should be set at twice the patients SBP or 300mmHg whichever is greater. Despite a lower UTIP choice, EBLs of 63 patients 84% fell below 100mL, while 60% fell below 50mL. This lower UTIP therefore served the purpose of minimizing intraoperative blood loss and providing a bloodless surgical field for the surgeon while minimizing tourniquet related complications. This is because, no complication was documented with regards to the use of pneumatic tourniquets in these patients. Although the literature is overwhelmingly in favor of using the PTIP, an excellent UTIP choice by this facility which is primarily lower than most PTIPs served the same purpose of using the minimum pressure possible for artery occlusion to achieve a bloodless field, decrease blood loss and greatly minimize tourniquet related complications. PNEUMATIC TOURNIQUET CUFF PRESSURES AND IMPLICATIONS 26 References Ahmed, I., Chawla, A., Underwood, M., Price, A. J., Metcalfe, A., Hutchinson, C., Warwick, J., Seers, K., Parsons, H., & Wall, P. D. (2020). Tourniquet use for knee replacement surgery. The Cochrane database of systematic reviews, 12(12), CD012874. Association of peri-Operative Registered Nurses (AORN), (2019). Guideline quick view: Pneumatic tourniquet. AORN journal, 109(2), 266269. https://doi.org/10.1002/aorn.12619 Cai, D. F., Fan, Q. H., Zhong, H. H., Peng, S., & Song, H. (2019). The effects of tourniquet use on blood loss in primary total knee arthroplasty for patients with osteoarthritis: a meta-analysis. Journal of orthopaedic surgery and research, 14(1), 348. https://doi.org/10.1186/s13018-019-1422-4 Ding, L., Ding, C. Y., Wang, Y. L., Wang, M. L., Qian, X. H., Huang, L., Xie, X. E., & Ji, H. Z. (2019). Application effect of pneumatic tourniquet with individualized pressure setting in orthopedic surgery of extremities: A meta-analysis. Journal of advanced nursing, 75(12), 34243433. Figueroa D. (2021). In TKA, Lower versus higher tourniquet inflation pressures reduced postoperative pain at the tourniquet inflation site and surgical site at 24 and 48 hours and 2 weeks. The Journal of bone and joint surgery. American volume, 103(22), 2143. https://doi.org/10.2106/JBJS.21.01008 Jensen, J., Hicks, R. W., & Labovitz, J. (2019). Understanding and optimizing tourniquet use during extremity surgery. AORN journal, 109(2), 171182. https://doi.org/10.1002/aorn.12579 PNEUMATIC TOURNIQUET CUFF PRESSURES AND IMPLICATIONS 27 Kim, H., & Kim, Y. H. (2018). Two cases of pneumatic tourniquet paralysis: Points for prevention. Archives of Hand and Microsurgery, 23(4), 313-318. Kukreja, P., Lehtonen, E., Pinto, M. C., Patel, H. A., McKissack, H. M., & Shah, A. (2018). Postoperative tourniquet pain in patients undergoing foot and ankle surgery. Cureus, 10(12), e3678. https://doi.org/10.7759/cureus.3678 Kumar, K., Railton, C., & Tawfic, Q. (2016). Tourniquet application during anesthesia: "What we need to know?". Journal of anesthesiology, clinical pharmacology, 32(4), 424430. https://doi.org/10.4103/0970-9185.168174 Mateu Vicent, D., Sola Ruano, L., Cabr Serrs, J. L., Haro Fernandez, D., Luna, Gutirrez, R., & Torra Parra, M. (2021). Lower tourniquet pressure does not affect pain nor knee-extension strength in patients after total knee arthroplasty: a randomized controlled trial. Knee surgery, sports traumatology, arthroscopy: official journal of the ESSKA, https://doi.org/10.1007/s00167-021-06536-5 Mu, J., Liu, D., Ji, D., Li, B., Li, Z., Zhang, F., & Lineaweaver, W. C. (2018). Determination of pneumatic tourniquet pressure of lower limb by ultrasonic doppler. Annals of plastic surgery, 80(3), 290292. https://doi.org/10.1097/SAP.0000000000001247 Naglehout, J., & Elisha, S. (2018). Nurse Anesthesia. (6 th ed). Elsevier Inc. Pinsornsak, P., Pinitchanon, P., & Boontanapibul, K. (2021). Effect of different tourniquet pressure on postoperative pain and complications after total knee Arthroplasty: A Prospective, Randomized Controlled Trial. The Journal of arthroplasty, 36(5), 16381644. https://doi.org/10.1016/j.arth.2020.12.049 PNEUMATIC TOURNIQUET CUFF PRESSURES AND IMPLICATIONS 28 Spruce, L. (2017). Back to basics: Pneumatic tourniquet use. AORN journal, 106(3), 219-226. T http://dx.doi.org/10.1016/j.aorn.2017.07.003 Sun, C., Yang, X., Zhang, X., Ma, Q., Yu, P., Cai, X., & Zhou, Y. (2022). Personalized tourniquet pressure may be a better choice than uniform tourniquet pressure during total knee arthroplasty: A PRISMA-compliant systematic review and meta-analysis of randomized-controlled trials. Medicine, 101(8), e28981. https://doi.org/10.1097/MD.0000000000028981 Tuncali, B., Boya, H., Kayhan, Z., & Arac, S. (2018). Tourniquet pressure settings based on limb occlusion pressure determination or arterial occlusion pressure estimation in total knee arthroplasty? A prospective, randomized, double blind trial. Acta orthopaedica et traumatologica turcica, 52(4), 256260. https://doi.org/10.1016/j.aott.2018.04.001 Tuncal, B., Boya, H., Kayhan, Z., Ara, ., & amurdan, M. A. (2016). Clinical utilization of arterial occlusion pressure estimation method in lower limb surgery: effectiveness of tourniquet pressures. Acta orthopaedica et traumatologica turcica, 50(2), 171177. https://doi.org/10.3944/AOTT.2015.15.0175 Tuncal, B., Boya, H., Kayhan, Z., & Ara, . (2018). Obese patients require higher, but not high pneumatic tourniquet inflation pressures using a novel technique during total knee arthroplasty. Eklem hastaliklari ve cerrahisi = Joint diseases & related surgery, 29(1), 4045. https://doi.org/10.5606/ehc.2018.57973 PNEUMATIC TOURNIQUET CUFF PRESSURES AND IMPLICATIONS 29 Appendix A Citation Research Design & Level of Evidence Population / Sample size Major Variables Instruments / Data collection Results Ahmed, I., Chawla, A., Underwood, M., Price, A. J., Metcalfe, A., Hutchinson, C., Warwick, J., Seers, K., Parsons, H., & Wall, P. D. (2020). Tourniquet use for knee replacement surgery. The Cochrane database of systematic reviews, 12(12), CD012874. Systematic review 41 RCTs Knee replacement with tourniquet use, estimated blood loss, pain, thromboembolism Certainty of evidence was assessed using the GRADE approach Knee replacement with tourniquet is associated with increased risk of serious adverse events and post-op pain. No significant differences in estimated blood loss between higher tourniquet pressure use and personalized tourniquet pressures. Cai, D. F., Fan, Q. H., Zhong, H. H., Peng, S., & Song, H. (2019). The effects of tourniquet use on blood loss in primary total knee arthroplasty for patients with osteoarthritis: a metaanalysis. Journal of orthopaedic surgery and research, 14(1), 348. https://doi.org/10.1186/s13018-019-1422-4 Metaanalysis 11 RCTs Tourniquet pressures, calculated blood loss, risk of DVT, operation time The Cochrane collaborations tool Tourniquet use significantly decreased intraoperative blood loss (P < 0.002), calculated blood loss (P < 0.002) and time of operation (P < 0.002). No significant difference was found between higher tourniquet pressures and personalized tourniquet pressures (P > 0.05) Ding, L., Ding, C. Y., Wang, Y. L., Wang, M. L., Qian, X. H., Huang, L., Xie, X. E., & Ji, H. Z. (2019). Application effect of pneumatic tourniquet with individualized pressure setting in orthopedic surgery of extremities: A metaanalysis. Journal of advanced nursing, 75(12), 34243433. Metaanalysis 9 RCTs Individualized pressure setting, tourniquet inflation pressure, estimated blood loss, orthopedic surgery The Cochrane collaborations tool for quality evaluation An individualized inflation pressure is recommended when using the tourniquet in orthopedic surgery. Only minimum effective pressure is needed and can be determined by accessing the systolic pressure and limb circumference of the patient. n=x Level I Level I Level I PNEUMATIC TOURNIQUET CUFF PRESSURES AND IMPLICATIONS Figueroa D. (2021). In TKA, Lower versus higher tourniquet inflation pressures reduced postoperative pain at the tourniquet inflation site and surgical site at 24 and 48 hours and 2 weeks. The Journal of bone and joint surgery. American volume, 103(22), 2143. https://doi.org/10.2106/JBJS.21.01008 Randomized controlled trial Kim, T. K., Bamne, A. B., Sim, J. A., Park, J. H., & Na, Y. G. (2019). Is lower tourniquet pressure during total knee arthroplasty effective? A prospective randomized controlled trial. BMC musculoskeletal disorders, 20(1), 275. https://doi.org/10.1186/s12891-019-2636-7 Randomized controlled trial Kukreja, P., Lehtonen, E., Pinto, M. C., Patel, H. A., McKissack, H. M., & Shah, A. (2018). Postoperative tourniquet pain in patients undergoing foot and ankle surgery. Cureus, 10(12), e3678. https://doi.org/10.7759/cureus.3678 Retrospective cohort study Mu, J., Liu, D., Ji, D., Li, B., Li, Z., Zhang, F., & Lineaweaver, W. C. (2018). Determination of pneumatic tourniquet pressure of lower limb by ultrasonic doppler. Annals of plastic surgery, 80(3), 290292. https://doi.org/10.1097/SAP.0000000000001247 Clinical controlled trial 150 patients Tourniquet inflation pressures, estimated blood loss, postoperative pain Knee society scoring scale, visual analog scale Higher tourniquet inflation pressures increased wound complications at discharge 20% versus 4 percent with lower pressures. No significant difference between estimated blood losses in higher versus lower inflation pressures 160 knee surgeries Lower versus higher tourniquet pressures, estimated blood loss, quality of surgical field Computer generated randomization table, visual analog scale, 5point Likert scale. Surgical field has comparable quality in both groups. (100% versus 99%). (P=1). There was no difference in hemoglobin drop, estimated blood loss and drained blood volume 128 patients Higher versus lower tourniquet pressure, intraoperative blood loss, incidence of tourniquet pain Blood pressure measurement with cuff, Linear regression analysis Prolonged tourniquet times at higher pressures as opposed to limb occlusion pressures (LOP) leads to longer length of PACU stay and increased opioid use. Safety margin of tourniquets are improved by basing pressure on limb occlusion pressure of patient 96 patients Surgeon choice tourniquet pressure versus doppler measured LOP, calculated blood loss, incidence of tourniquet Measurement of LOP using ultrasonic doppler, Linear regression analysis Lower tourniquet pressures based on LOP ensures similar hemostatic effects and provision of bloodless field, while reducing the incidence of adverse reactions to tourniquet use and improve safety. Level II Level II Level IV Level III 30 PNEUMATIC TOURNIQUET CUFF PRESSURES AND IMPLICATIONS 31 complications, pain. Pinsornsak, P., Pinitchanon, P., & Boontanapibul, K. (2021). Effect of different tourniquet pressure on postoperative pain and complications after total knee Arthroplasty: A Prospective, Randomized Controlled Trial. The Journal of arthroplasty, 36(5), 16381644. https://doi.org/10.1016/j.arth.2020.12.049 Randomized controlled trial (RCT) Sun, C., Yang, X., Zhang, X., Ma, Q., Yu, P., Cai, X., & Zhou, Y. (2022). Personalized tourniquet pressure may be a better choice than uniform tourniquet pressure during total knee arthroplasty: A PRISMA-compliant systematic review and meta-analysis of randomizedcontrolled trials. Medicine, 101(8), e28981. https://doi.org/10.1097/MD.0000000000028981 Metaanalysis Tuncali, B., Boya, H., Kayhan, Z., & Arac, S. (2018). Tourniquet pressure settings based on limb occlusion pressure determination or arterial occlusion pressure estimation in total knee arthroplasty? A prospective, randomized, double blind trial. Acta orthopaedica et traumatologica turcica, 52(4), 256260. https://doi.org/10.1016/j.aott.2018.04.001 Randomized controlled trial 450 patients Higher versus lower tourniquet pressures, pain, blood loss, muscle damage, wound complication Visual analog scale for pain, Knee society score. Total blood loss showed no significant difference in the study groups. Similar results were noted in the quality of bloodless field. However, muscle damage, pain and wound complications were higher in the group with higher tourniquet pressures 13 RCTs Uniform tourniquet inflation pressure, personalized tourniquet pressure, blood loss, pain, nerve damage Visual analog scale Personalized tourniquet pressures provide a similar bloodless surgical field when compared to the uniform tourniquet pressures and also provides less pain intensity, less rate of ecchymosis, compared to the uniform pressures. 93 patients Tourniquet pressure, limb occlusion pressure, effectiveness of tourniquets, blood loss, pain, nerve damage Likert scale Tourniquet pressures based on limb occlusion pressure provides a similar surgical field compared to surgeon determined higher pressures while minimizing complications such as nerve damage, ecchymosis, tourniquet pain. Level II Level I Level II PNEUMATIC TOURNIQUET CUFF PRESSURES AND IMPLICATIONS 32 Appendix B Figure 1. The Theory of Reflective Practice in Nursing Conceptual Framework (Galutira, 2018) PNEUMATIC TOURNIQUET CUFF PRESSURES AND IMPLICATIONS 33 Identificatio Appendix C PRISMA 2009 Flow Diagram Records identified through database searching (n = 128) Additional records identified through other sources (n = 3) Screening Records after duplicates removed (n =75) Included Eligibility Records screened (n =75) Full-text articles assessed for eligibility (n = 28) Records excluded: older than 7 years, not in English (n =47) Full-text articles excluded: not tourniquet pressure related, not lower extremity related (n = 18) Studies included in the literature review: primary research, systematic reviews, RCTs, Metaanalysis (n =10) From: Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2009). Preferred Reporting Items for Systematic Reviews and Meta-Analyses: The PRISMA Statement. PLoS Med 6(7): e1000097. doi:10.1371/journal.pmed1000097 For more information, visit www.prisma-statement.org. PNEUMATIC TOURNIQUET CUFF PRESSURES AND IMPLICATIONS 34 Appendix D GANTT chart GANTT Chart 8/28/202110/17/202112/6/2021 1/25/2022 3/16/2022 5/5/2022 6/24/2022 8/13/2022 10/2/2022 Practice Question form DNP project team Faculty Chair Final PICO Clinical Practice Question Annotated Bibliography Literature Matrix CITI Training GANTT chart Proposal Draft 1 Proposal Draft 2 Proposal Draft 3 Theoretical framework SWOT Analysis Marian IRB approval IU Health IRB approval Complete Chart Review Analysis of Data Complete Project Report Complete DNP project PNEUMATIC TOURNIQUET CUFF PRESSURES AND IMPLICATIONS 35 Appendix E SWOT Analysis Clinical Site Strengths Facility has EMR to facilitate retrospective chart review Documentation of tourniquet pressures is standard practice No funding necessary Large healthcare facility sees a significant number of ortho cases. Data may not be equally available for both topics of comparison May require significant time input for chart review Acquiring enough data for both topics of comparison may be influenced by physician preference Weaknesses Opportunities Anthony Dzigbordi Education centered facility that supports research and EBP. Large number of orthopedic surgeries per year ensures data availability Use of EMR is conducive for retrospective chart review. S O W T Limitations on how far back a chart could be reviewed post discharge Available data may be dependent solely on physician preference of tourniquet pressures. Previous data may not be accessible if there was a recent transition from paper charts to EMR Threats PNEUMATIC TOURNIQUET CUFF PRESSURES AND IMPLICATIONS Appendix F Pie Chart (Tourniquet Pressures) Tourniquet Pressures used TP=350 TP=300 TP=250 TP=275 1 2 3 4 36 PNEUMATIC TOURNIQUET CUFF PRESSURES AND IMPLICATIONS 37 Appendix G Estimated Blood Losses 45 40 35 30 25 20 15 10 5 0 0-50 51-100 101-150 151-200 201-250  ...

... The Great Migration How I Learned to Stop Worrying and Love Tipasa Lynn Colbert Access Services Librarian Ancilla 2-year college FTE: 278 2 full-time librarian and 1-2 student workers Marian PhD granting institution + med school FTE: 3700 5 full-time librarians, 3 part-time associates, and 14 student workers Marian University ocially merged with Ancilla College. July 2021 Began discussions with Tipasa about whether it would work to meet user and staff needs. Ocially decided to migrate to Tipasa. Go live date. Aug 2022 migrated to a new authentication method. Spring 2022 July 2022 Fall 2021 Worked with IT department to determine what to do to make Illiad work with 2 campuses. Created workarounds to meet user needs at Ancilla campus. May 2022 Began migration including 2x/week training sessions and detailed migration process. Training Staff members involved Access Services Librarian Assistant Director (also manages systems) Librarian point person at Ancilla campus 2 part-time associates IT point person Considerations Learning styles Video tutorials Live training Written/visual documentation Schedules Staggered shifts Vacations Covid Potential shutdown of building for upgrades Strategy Designate a point person at each campus Regular check-ins Create atmosphere where staff can ask questions Document, document, document Libguides and Trello Provide plenty of practice and room for mistakes Be mindful of staff capacity, adjust as necessary/possible Conclusions It takes a village Gather your team Identify how they learn best Properly acknowledge their role and the amount of work required Pace their training if possible Timing is everything Be exible - something WILL go wrong Build in extra time for everything Reward the hard work Gave out thank you chocolate to everyone who helped in the process. Contact information Lynn Colbert Access Services Librarian lcolbert@marian.edu Slides available at:  ...

... PROPOFOL INFUSION TO DECREASE PONV Marian University Leighton School of Nursing Doctor of Nursing Practice Final Project Report for Students Graduating in May 2023 A Retrospective Study on the Use of Intraoperative Subhypnotic Propofol Infusion in Conjunction with Volatile Anesthetics to Decrease PONV Kerri Ann Paris Marian University Leighton School of Nursing Chair: Derrianne Monteiro, DNP, CRNA _______________________ (Signature) Project Team Member(s): Benjamin Cohen, DNP, CRNA _______________________ (Signature) Date of Submission: December 12th, 2022 1 PROPOFOL INFUSION TO DECREASE PONV Table of Contents Abstract ................................................................................................................................4 Introduction .........................................................................................................................5 Background ....................................................................................................................6 Problem Statement .........................................................................................................8 Gap Analysis ..................................................................................................................9 Review of the Literature ......................................................................................................9 Propofols Effect on PONV....10 Volatile Anesthetics Effect on PONV...11 Combination of Propofol and Volatile Anesthetic..12 Theoretical Framework.......................13 Goals, Objectives, Expected Outcomes ............................................................................14 Project Design and Methods.. ...15 Project Site and Population ..........................................................................................15 Measurement Instrument(s) ........................................................................................16 Data Collection Procedure ..........................................................................................17 Ethical Considerations/Protection of Human Subjects ...17 Analysis .....18 Results....18 Participants...18 PONV...19 Discussion..19 Conclusion ........................................................................................................................21 2 PROPOFOL INFUSION TO DECREASE PONV References ..........................................................................................................................22 Appendices 27 Appendix A ..................................................................................................................27 Appendix B ..................................................................................................................30 Appendix C ..................................................................................................................31 3 PROPOFOL INFUSION TO DECREASE PONV 4 Abstract Background and Review of Literature: Postoperative nausea and vomiting (PONV) frequently occur in patients after anesthesia, significantly impacting patient satisfaction and potentially leading to untoward complications. Multimodal PONV prophylaxis for patients with increased risk factors should be implemented to decrease stay in the PACU and healthcare cost. While research has been extensively conducted on the use of multimodal prophylaxis using antiemetics, such as ondansetron and dexamethasone, research on combining those therapies with a subhypnotic propofol infusion during anesthesia with a volatile anesthetic has been insufficient. Purpose: This DNP project was designed to determine whether the addition of a continuous subhypnotic propofol infusion in conjunction with a volatile anesthetic decreases the incidence of PONV. Methods: This project utilizes a quality improvement design by the evaluation of a practice intervention to improve the guidelines on preventing PONV after anesthesia. A retrospective chart review was conducted, and Microsoft Excel was used to perform all statistical analyses. Implementation Plan/Procedure: A total of 60 patient EMRs met the criteria for this project and were utilized in this study. The patient EMRs were separated into a control and experimental group. Those in the experimental group all received a subhypnotic propofol infusion at 0.1-0.5 mg/kg/hr. The incidence of PONV in the PACU was recorded and compared for both groups. Implications/Conclusion: The results of this study concluded that the addition of a continuous subhypnotic propofol infusion in conjunction with a volatile anesthetic presents no added benefit in decreasing the incidence of PONV in the PACU. Keywords: Subhypnotic Propofol Infusion, Volatile Anesthetic, PONV PROPOFOL INFUSION TO DECREASE PONV 5 A Retrospective Study on the Use of Intraoperative Subhypnotic Propofol Infusion in Conjunction with Volatile Anesthetics to Decrease PONV This project is submitted to the faculty of Marian University Leighton School of Nursing as partial fulfillment of degree requirements for the Doctor of Nursing Practice, Certified Registered Nurse Anesthetist track. Postoperative nausea and vomiting (PONV) frequently occur in patients after anesthesia, significantly impacting patient satisfaction and potentially leading to untoward complications, including delayed recovery, prolonged hospitalization, and increased healthcare costs. The incidence of PONV affects approximately 30% of patients undergoing general anesthesia and drastically increases to 80% for those with multiple risk factors (Jokinen et al., 2012). The Apfel simplified risk score is one of the most widely known scoring systems to predict the incidence of PONV, comprising of four risk factors: female gender, nonsmoking, history of motion sickness or PONV, and the use of postoperative opioids (Apfel et al., 1999). Due to the multifactorial nature of PONV, other risk factors that contribute to the increased incidence include age less than 50, type of surgery, such as cholecystectomy, gynecological, and laparoscopic surgeries, duration of surgery, general anesthesia, nitrous oxide, volatile anesthetics, and etomidate (Shaikh et al., 2016). The American Association of Nurse Anesthesiology (AANA) has published several articles demonstrating effective prophylactic regimes on the prevention of PONV, but there remains to be high variability with the proposed interventions, thus making PONV an unrelenting issue in the anesthesia community (AANA, 2021). Propofol, a widely used induction and maintenance medication during anesthesia, has several advantages, such as producing a dosedependent decreased level of consciousness, rapid onset of action, predictable duration, and an antiemetic effect (Folino et al., 2021). Volatile anesthetics (isoflurane, desflurane, and PROPOFOL INFUSION TO DECREASE PONV 6 sevoflurane) are commonly used for induction and maintenance of general anesthesia due to their advantageous profile of providing amnesia, immobility, and exerting cardioprotective effects with the most common disadvantage of causing PONV (Miller et al., 2021). The antiemetic properties of propofol allow it to be utilized in total intravenous anesthesia (TIVA) in patients at high-risk for PONV, but the effects of a subhypnotic dose of propofol as a continuous infusion in conjunction with a volatile anesthetic has not been extensively researched. Background The increased incidence of PONV in patients after anesthesia has led to extensive research on the efficacy of numerous antiemetics and alternative approaches and techniques to improve the quality of patient care. PONV is a complex problem with a multifactorial etiology. The stimulation of the vomiting reflex involves five afferent pathways, including the chemoreceptor trigger zone (CTZ), the vagal mucosal pathway in the gastrointestinal system, neuronal pathways from vestibular system, reflex afferent pathways from the cerebral cortex, and midbrain afferents (Shaikh et al., 2016). Stimulating any one of these afferent pathways sends inputs to the vomiting center located in the reticular formation in the brainstem controlling nausea and vomiting (Shaikh et al., 2016). While guidelines on the prevention of PONV exist, there are limitations to their efficacy due to the focus on specific patient populations, not addressing all aspects of management of PONV, or not providing results based on current literature (Gan et al., 2020). The fourth consensus guidelines for the management of postoperative nausea and vomiting based on published clinical evidence and reviewed by an international multidisciplinary expert panel provide strategies to reduce the risk for PONV, which include minimizing perioperative opioids, use of regional anesthesia, use of propofol infusions as the primary anesthetic, avoidance of volatile anesthetics, and adequate hydration (Gan et al., 2020). Multimodal PONV prophylaxis PROPOFOL INFUSION TO DECREASE PONV 7 for patients with increased risk factors should be implemented by every provider to decrease stay in the PACU, healthcare cost, dissatisfaction, and readmission. The fourth consensus guidelines recommended the use of multimodal prophylaxis in patients that present with PONV risk factors. Patients at an increased risk for PONV that have 1-2 risk factors should receive two prophylactic therapies, while those with greater than two risk factors should receive 3-4 prophylactic therapies (Gan et al., 2020). The prophylactic therapies include 5-HT3 receptor antagonists, corticosteroids, antihistamines, dopamine antagonists, propofol anesthesia, NK-1 receptor antagonists, acupuncture, and anticholinergics with an anti-emetic from a different class than the chosen prophylactic drug for rescue treatment (Gan et al., 2020). A common combination therapy for PONV includes a 5-HT3 receptor antagonist and dexamethasone. Ondansetron, the gold standard in management of PONV, is the most common 5-HT3 receptor antagonist utilized with similar antivomiting and antinausea effects (Gan et al., 2020). Dexamethasone, a glucocorticoid, not only improves PONV, but has also been shown to reduce the need for analgesics if given at the time of induction (Gan et al., 2020). While research has been extensively conducted on the use of multimodal prophylaxis using antiemetics, such as ondansetron and dexamethasone, research on combining those therapies with a subhypnotic propofol infusion during anesthesia with a volatile anesthetic has been insufficient. Vari et al. (2010) conducted a randomized study on the incidence of PONV in patients receiving propofol versus sevoflurane for anesthesia maintenance after thyroidectomy. Female patients who received sevoflurane during maintenance had a 70.6% incidence of PONV, while those who received propofol had a 42.4% incidence, concluding that the incidence of PONV was significantly higher in female patients that received sevoflurane (Vari et al., 2010). Vari et al. (2010) disclosed that maintenance of subhypnotic propofol by continuous infusion PROPOFOL INFUSION TO DECREASE PONV 8 may be effective in decreasing the incidence of PONV, but the lack of research in the topic is evident. Schraag et al. (2018) conducted a meta-analysis that revealed a reduction in PONV when utilizing propofol-based TIVA over volatile anesthetics with a 39% risk reduction thus improving patient satisfaction. Multiple studies have been conducted on the use of TIVA in preventing PONV, but research lacks in the efficacy of utilizing a subhypnotic propofol infusion with a volatile anesthetic to decrease PONV. Multiple studies have been published on the incidence of PONV with patients who received a volatile anesthetic during anesthesia compared with patients who received propofol-based TIVA, yet few studies exist that focus on the incidence of PONV with patients who received a volatile anesthetic combined with a continuous subhypnotic propofol infusion. The advantages of using volatile anesthetics may outweigh their disadvantage of causing PONV, especially when combined with a continuous subhypnotic propofol infusion. Problem Statement Patients continue to report PONV as one of the most unpleasant side effects of anesthesia, delaying discharge from the PACU by nearly 20 minutes with every episode of emesis (Gan et al., 2020). The significance of this study focuses on surgical patients undergoing general anesthesia with a volatile anesthetic and utilizing a subhypnotic propofol infusion during the length of the procedure to decrease the incidence of PONV. Along with other antiemetic medications for PONV prophylaxis in high-risk patients, this studys aim is to reveal whether the addition of a subhypnotic propofol infusion further decreases PONV in the PACU. A retrospective study was conducted utilizing patients electronic medical records from a hospital to determine the incidence of PONV in patients who received a volatile anesthetic and a continuous subhypnotic propofol infusion compared with patients who only received a volatile anesthetic. The results of this study provide PROPOFOL INFUSION TO DECREASE PONV 9 evidence to whether the addition of a continuous subhypnotic propofol infusion during anesthesia with a volatile anesthetic decreases the incidence of PONV. Gap Analysis Best practice highly recommends the use of multimodal prophylaxis in patients with one or more risk factors by using a combination therapy of antiemetics with different drug classes, such as a 5-HT3 receptor antagonist with dexamethasone, which is implemented as the current practice at the project site (Gan et al., 2020). Best practice includes approaches, such as the use of propofol infusions as the primary anesthetic and the avoidance of volatile anesthetics, to decrease baseline risk for PONV (Gan et al., 2020). Current practice at the project site differs from best practice due to the decreased incidence of using propofol infusions as the primary anesthetic and increased incidence of using volatile anesthetics. The increased incidence of using volatile anesthetics places patients at an increased risk for PONV. Current practice at the project site combines the use of a volatile anesthetic with a continuous subhypnotic propofol infusion during general anesthesia. Review of Literature The focus of this literature review is to present current evidence regarding the benefit of utilizing a subhypnotic propofol infusion in conjunction with a volatile anesthetic to decrease the incidence of PONV. The databases searched for literature were PubMed and the Cochrane Library. The search was conducted from November 2021 to January 2022. The keywords utilized were postoperative nausea and vomiting, propofol, subhypnotic propofol infusion, volatile anesthetics, inhalation, and anesthesia. BOOLEAN phrases were used to combine keywords such as postoperative nausea and vomiting AND propofol AND inhalation anesthesia, and postoperative nausea and vomiting AND propofol infusion AND volatile anesthetics. PROPOFOL INFUSION TO DECREASE PONV 10 Seventy articles resulted from the search, however 59 of those articles were eliminated after applying the inclusion and exclusion criteria. The inclusion criteria included systematic reviews, meta-analyses, and randomized controlled trials written between 2017-2021, including patients that received a volatile anesthetic combined with a propofol infusion who were assessed for postoperative nausea and vomiting. Exclusion criteria included patients that did not receive a volatile anesthetic combined with a propofol infusion and those who were not assessed for postoperative nausea and vomiting. The search was widened to a 10-year time frame to find additional relative articles ranging from 2011-2021, resulting in 3 articles. One article outside of the search range was utilized due to the significance of the article. After applying the inclusion and exclusion criteria, widening the search to a 10-year time frame, and adding the article of significance, 15 articles were utilized. The literature review matrix is located in Appendix A. The most widely known tool to predict the risk of PONV, the Apfel score, sets the foundation for this literature review. Apfel et al. (1999) designed an applicable model consisting of the four most important predictors of PONV, providing insight to who would benefit from prophylactic antiemetic therapy. Apfel et al. (1999) revealed that the incidence of PONV varies between 10-21% in patients with one or no risk factors, significantly increasing to 39-78% for patients with two risk factors. An increased risk of PONV implies a change or modification of the anesthetic technique, recommending prophylactic antiemetic therapy and/or the avoidance of volatile anesthetics by utilizing total intravenous anesthesia (TIVA) (Apfel et al., 1999). Propofols Effect on PONV Evidence has shown that a combination of antiemetic medications has a significant impact on the reduction of PONV than a single antiemetic alone (Gan et al., 2020; Muhly et al., 2020; Weibel et al., 2020). The Fourth Consensus Guidelines for the Management of PONV PROPOFOL INFUSION TO DECREASE PONV 11 signify a reduction in PONV with the use of a subhypnotic propofol infusion when combined with an antiemetic for general anesthesia (Gan et al., 2020) with Muhly et al. (2020) recommending the combination of ondansetron, dexamethasone, and a subhypnotic dose of propofol. Due to the antiemetic effect of propofol, TIVA with propofol has been widely utilized for general anesthesia, especially when there are concerns for PONV. Several studies have revealed a reduction in the incidence of PONV when utilizing TIVA with propofol compared to volatile anesthesia (Elbakry et al., 2018; Pang et al., 2021; Park et al., 2020; Schraag et al., 2018). Schraag et al. (2018) discovered a 39% risk reduction in PONV with TIVA, a significant reduction compared to 18.9% from previous studies. Apfel et al. (1999) revealed specific surgeries that possess an increased risk for PONV, one of which is laparoscopic. TIVA with propofol shows promising results during laparoscopic surgery in reducing PONV (Elbakry et al., 2017; Park et al., 2020). No significant difference in PONV was noted in one study when comparing TIVA with propofol versus desflurane, contradicting the results from multiple other studies (Aftab et al., 2019). Volatile Anesthetics Effect on PONV Guidelines for the management of PONV recommend avoiding the use of volatile anesthetics and nitrous oxide, although this recommendation is not always feasible (Gan et al., 2020). The benefit of using a volatile anesthetic is partially owed to its cardioprotective effects. Studies have revealed that there was not an increase in the incidence of PONV when a volatile anesthetic was used as an adjunct to propofol based TIVA (Chen et al., 2016; Lai et al., 2018; Uchinami et al., 2019). Sevoflurane is advantageous due to its cardioprotective effects, rapid uptake and elimination, inhibition of pulmonary irritant receptors, and smooth emergence, which justifies the importance of using it during anesthesia (Kawano et al., 2016; Lai et al., 2018; PROPOFOL INFUSION TO DECREASE PONV 12 Uchinami et al., 2019). Chen et al. (2016) discovered that patients had less changes in mean arterial pressure (MAP) and a smoother recovery from anesthesia when desflurane was used as an adjunct to propofol anesthesia. TIVA and volatile anesthesia are two common methods of anesthesia that have been extensively researched, but there are few studies on the combination of a volatile anesthetic with a subhypnotic propofol infusion. Volatile anesthetics and propofol each possess characteristics that may be beneficial if used as a combination instead of using each agent alone. Combination of Propofol and Volatile Anesthetic Volatile anesthetics and propofol are universally utilized for general anesthesia, both having their own benefits and side effects. PONV continues to be a common side effect of volatile anesthetics, with propofol possessing antiemetic properties. There are few studies on the effects of a subhypnotic propofol infusion in conjunction with a volatile anesthetic to decrease PONV during anesthesia. From the studies published, the use of propofol in combination with a volatile anesthetic revealed a reduction in PONV (Kawano et al., 2016; Wolf et al., 2021; Won et al., 2011). Kawano et al. (2016) showed a 66% decrease in PONV when volatile anesthesia was combined with propofol for laparoscopic gynecological surgeries. Other studies that have shown a decrease in PONV with propofol combined with a volatile anesthetic by comparing the use of propofol alone versus propofol with a volatile anesthetic (Chen et al., 2016; Lai et al., 2018; Uchinami et al., 2019). One study focused on comparing propofol target-controlled infusion (TCI) and propofol TCI with the addition of low-concentration desflurane during the maintenance phase of anesthesia for laparoscopic cholecystectomy surgery. The study revealed a low incidence of PONV in both groups with 96.2% of the propofol TCI group and 95.8% of the propofol TCI with desflurane group experiencing no PONV, most likely due to the antiemetic PROPOFOL INFUSION TO DECREASE PONV 13 effects of propofol (Chen et al., 2016). Uchinami et al. (2019) conducted a study comparing the incidence of PONV in patients that received propofol alone with patients that received propofol in conjunction with 0.8% sevoflurane. The results of the study revealed that the combination of propofol with a sevoflurane did not increase the incidence of PONV when compared to propofol alone, once again highlighting the importance of the antiemetic effect of propofol when combined with a volatile anesthetic (Uchinami et al., 2019). Overall, the incidence of postoperative nausea and vomiting was decreased when propofol was utilized in conjunction with a volatile anesthetic (Chen et al., 2016; Kawano et al., 2016; Uchinami et al., 2019; Wolf et al., 2021; Won et al., 2011). Theoretical Framework The theoretical framework utilized for this scholarly project is the Theory of Symptom Management. The Theory of Symptom Management, a middle range theory, comprises of three major concepts: symptom experience, symptom management strategies, and outcomes (Smith & Liehr, 2018). Applying the concepts of the Theory of Symptom Management to decrease the incidence of PONV, multimodal prophylaxis should be utilized for patients with PONV risk factors. Nausea and vomiting, especially in the postoperative period, are two of the most common side effects after anesthesia producing noteworthy patient dissatisfaction (Gan et al., 2020). A subjective experience, such as nausea and vomiting, would be described as a symptom, which can lead to disruptions in physical, mental, and social functioning (Smith & Liehr, 2018). The concept of symptom experience signifies perception and response to a change, whether that change be in frequency or severity (Smith & Liehr, 2018). PONV occur more frequently in females, nonsmokers, those with a history of motion sickness or PONV, and patients who receive postoperative opioids (Apfel et al., 1999). The increasing incidence and PROPOFOL INFUSION TO DECREASE PONV 14 severity of nausea and vomiting in the postoperative period can lead to a distressing situation for the patient, interfere with their care, and delay their recovery. Strategies for the concept of symptom management include efforts to minimize the symptom experience by reducing the incidence and severity of the symptom and alleviating the associated distress (Smith & Liehr, 2018). The framework specifies the intervention by asking who, what, when, where, how, and why. The what indicates the strategy and could include a combination of interventions, such as the multimodal approaches utilized for decreasing the incidence of PONV. The how and when specify the dose and timing of the intervention strategy. The fourth consensus guidelines for the management of postoperative nausea and vomiting recommend the combination of ondansetron and dexamethasone, with 4-8mg of dexamethasone at induction and 4mg of ondansetron at the end of surgery (Gan et al., 2020). Lastly, the concept of symptom outcomes refers to measurable outcomes to assess before and after implementation of the proposed intervention (Smith & Liehr, 2018). The goal of symptom outcome is to improve the patients symptom, leading to a shorter hospital stay, decreased healthcare cost, better physical and mental functioning, and an overall improved quality of life (Smith & Liehr, 2018). Goals, Objectives, and Expected Outcomes This DNP project was designed to determine whether the addition of a continuous subhypnotic propofol infusion in conjunction with a volatile anesthetic decreases the incidence of PONV. The goals of this project were to 1) identify the incidence of PONV in high-risk patients, including those of female gender, nonsmokers, patients undergoing gynecological and laparoscopic surgeries, and the use of a volatile anesthetic; 2) identify the differences in the need for antiemetic medication in the PACU between patients who received a subhypnotic propofol PROPOFOL INFUSION TO DECREASE PONV 15 infusion in conjunction with a volatile anesthetic and those who only received a volatile anesthetic; 3) evaluate if a subhypnotic propofol infusion in conjunction with a volatile anesthetic decreases the incidence of PONV. The expected outcomes of this project were to improve guidelines on the prevention of PONV, ultimately improving patient care. Project Design and Methods This project utilizes a quality improvement design by the evaluation of a practice intervention to improve the guidelines on preventing PONV after anesthesia. High-risk patients who received a continuous subhypnotic propofol infusion at 0.1-0.5 mg/kg/hr with a volatile anesthetic will be compared to patients who only received a volatile anesthetic. Incidence of PONV will be documented and recorded in the patients electronic medical record (EMR). Utilizing the EMR, a qualitative and quantitative evaluation will be made regarding the incidence of PONV and the dose of antiemetic given. Project Site and Population This DNP project was implemented at a 199 bed, Level III trauma center, private hospital in the Midwest United States. The full-service hospital offers a wide array of surgical procedures including general, orthopedic, neuroskeletal, gynecological, gastrointestinal, urology, plastics, healthy pediatrics, vascular, laparoscopic, and robotic surgeries. The facility employs full-time anesthesiologists and four certified registered nurse anesthetists under a medical supervision model with a physician-led team approach. The population being evaluated in this study were high-risk PONV patients undergoing general anesthesia. The inclusion criteria consisted of the female gender, age 20-50 years old, nonsmokers, gynecological and laparoscopic surgeries, patients that received ondansetron and dexamethasone (antiemetics) intraoperatively, the use of volatile anesthetics, and patients that PROPOFOL INFUSION TO DECREASE PONV 16 received a continuous subhypnotic propofol infusion at 0.1-0.5 mg/kg/hr and patients that did not receive a continuous subhypnotic propofol infusion. The exclusion criteria consisted of the male gender, age less than 20 or greater than 50 years old, current smokers, surgeries that are not gynecological or laparoscopic, patients that did not receive ondansetron and dexamethasone intraoperatively, and not using volatile anesthetics. The experimental group consisted of the patients that received a continuous subhypnotic propofol infusion and the control group consisted of patients that did not receive a continuous subhypnotic propofol infusion. A retrospective analysis of data was utilized for this study in the form of chart review. A total of 60 patient EMRs were utilized that met the criteria, 30 in the experimental group and 30 in the control group. Once all of the data was collected, a statistical analysis of the data was conducted to determine if there was a significant difference in the incidence of PONV between the two groups. Measurement Instruments In order to measure the outcomes of this DNP project, a project lead created tool was utilized. The measurement instrument consisted of a spreadsheet that was developed in Microsoft Excel. The project lead created tool is located in Appendix B. All of the variables were categorical, except for the antiemetic dose that was given in the postanesthesia care unit (PACU) for PONV. The variables utilized in the Microsoft Excel spreadsheet consist of age, categorized as 1 for 20-30 years old, 2 for 30-40 years old, and 3 for 40-50 years old; type of surgery, 1 for laparoscopic, 2 for gynecological, and 3 for a combination of laparoscopic/gynecological; volatile anesthetic, 1 for sevoflurane and 2 for desflurane; propofol infusion, 1 for yes if the patient received a continuous subhypnotic propofol infusion and 2 for no if the patient did not receive a continuous subhypnotic propofol infusion; and PONV, 1 for yes if the patient PROPOFOL INFUSION TO DECREASE PONV 17 experienced PONV in the PACU and received antiemetics and 2 for no if the patient did not experience PONV in the PACU and did not receive antiemetics. Data Collection Procedures A retrospective chart review was conducted at a private hospital in the Midwest to determine whether the addition of a continuous subhypnotic propofol infusion in conjunction with a volatile anesthetic decreases the incidence of PONV. 60 patient EMRs from August 2021 to December 2021 were utilized that met the criteria. The inclusion criteria for the experimental group consisted of the female gender, 20-50 years of age, nonsmokers, laparoscopic and gynecological surgeries, patients that received ondansetron and dexamethasone intraoperatively, the use of volatile anesthetics, and the use of a continuous subhypnotic propofol infusion at 0.10.5 mg/kg/hr. The inclusion criteria for the control group consisted of all of the variables in the experimental group except they did not receive a continuous subhypnotic propofol infusion. Evaluation of the experimental and control group occurred to determine whether the patients experienced PONV and needed antiemetics in the PACU. The results of the retrospective study concluded whether the addition of a continuous subhypnotic propofol infusion in conjunction with a volatile anesthetic decreased the incidence of PONV. Ethical Consideration/Protection of Human Subjects The Marian Internal Review Board (IRB) approval was obtained prior to initiating this DNP project. The project did not involve an intervention or interaction with living subjects. The data collected for this project was used to support a hypothesis. The data was de-identified, and patients remained anonymous. To ensure protection of human rights and HIPPA, the variables of each category that could potentially impact HIPPA were generalized to maintain confidentiality. A random ID number was assigned to each patient, age ranges were used instead of specific PROPOFOL INFUSION TO DECREASE PONV 18 ages, and a type of surgery instead of specific surgeries. All electronic files containing identifiable information were password protected to prevent access by unauthorized users. Informed consent was not needed due to the retrospective nature of the project. Refer to Appendix C for IRB approval. Analysis The data was analyzed using a Chi-Square Test of Independence. A Chi-Square Test of Independence was utilized to determine statistical significance between the control and experimental group results. All categorical data was evaluated in a frequency table. Frequencies and percentages were calculated for patient demographics that were categorical variables. Microsoft Excel was used to perform all statistical analyses. Results Participants A total of 60 patient EMRs met the criteria for this project and were utilized in this study. Half of the patients met the inclusion criteria for the control group, whereas the other half met the inclusion criteria for the experimental group. Those in the experimental group all received a subhypnotic propofol infusion at 0.1-0.5 mg/kg/hr. Most patients (36.7%) were between the ages of 30 and 40, had undergone a laparoscopic/gynecological surgery (41.7%), and received sevoflurane as the volatile anesthetic (71.7%). Refer to Table 1 to view the demographics of all patients. Table 1 Demographics and Characteristics of All Patients Characteristics Age Group n % PROPOFOL INFUSION TO DECREASE PONV 19 20-30 years 30-40 years 40-50 years 20 22 18 33.3 36.7 30.0 Type of Surgery Laparoscopic Gynecological Combined Laparoscopic and Gynecological 19 16 25 31.7 26.7 41.7 Volatile Anesthetic Sevoflurane Desflurane 43 17 71.7 28.3 Note. n=60 PONV To determine whether the addition of a continuous subhypnotic propofol infusion in conjunction with a volatile anesthetic decreased the incidence of PONV, the patients that met the criteria for the experimental group were compared to those in the control group. There was not a statistically significant difference in the incidence of PONV between those who received a continuous subhypnotic propofol infusion in conjunction with a volatile anesthetic and those who did not. A Chi-Square Test of Independence resulted in a P value of 0.640428787, indicating a lack of statistical significance between the two groups in this study. Discussion This DNP project was designed to determine whether the addition of a continuous subhypnotic propofol infusion in conjunction with a volatile anesthetic decreased the incidence of PONV. This study consisted of an experimental group that received a subhypnotic propofol infusion at 0.1-0.5 mg/kg/hr in addition to a volatile anesthetic and a control group that only received a volatile anesthetic. The results of this study concluded that there was not a statistical PROPOFOL INFUSION TO DECREASE PONV 20 difference in the two groups, signifying that the addition of a subhypnotic propofol infusion in combination with a volatile anesthetic has no impact on PONV. The results of this study did not correlate with a previous study by Kawano et al. (2016) that showed that the use of combined propofol and volatile anesthesia reduced the incidence of PONV. Kawano et al. (2016) did not follow the recommended multimodal approach to prevent PONV, therefore the high-risk patients in the study did not receive any prophylactic antiemetics. All of the patients in this study had received a multimodal approach to prevent PONV by receiving ondansetron and dexamethasone intraoperatively. The use of the multimodal approach could have had an effect on the results of this study. It could be useful to reconduct the study and focus on patients who received a subhypnotic propofol infusion in conjunction with a volatile anesthetic that did not receive prophylactic antiemetics. It has already been shown in studies the benefit of utilizing TIVA in patients that are high-risk for PONV. Volatile anesthetics and propofol both possess beneficial properties that make them favorable during anesthesia, verifying the importance of determining whether a subhypnotic propofol infusion decreases PONV in patients also receiving a volatile anesthetic. A study by Uchinami et al. (2019) showed the coadministration of sevoflurane and propofol did not increase PONV compared to TIVA. There are a limited number of studies available that show the significance of a reduction in PONV when propofol is used in conjunction with a volatile anesthetic, implicating the need for this study. Although the results of this study showed no statistical difference between the two groups, studies have been conducted that prove otherwise (Kawano et al., 2016; Uchinami et al., 2019). There were several limitations to this project. The first limitation was the retrospective nature of this project and the accuracy of the data retrieved from the patients EMRs. The quality PROPOFOL INFUSION TO DECREASE PONV 21 of the data collected is dependent on the familiarity of the user and therefore under-reporting or inaccurate documentation of nausea and/or vomiting presents a limitation. The study was subject to confounding as only specific factors were measured. The aim was to conduct research on high-risk PONV patients including women, nonsmokers, patients undergoing gynecological and laparoscopic surgeries, and the use of a volatile anesthetic; therefore, it is unclear whether the results from this study can be applied to other patient populations. The small sample size was an additional limitation. A prospective study is needed to extract significant data to determine whether the addition of a continuous subhypnotic propofol infusion in conjunction with a volatile anesthetic decreases the incidence of PONV. Conclusion In conclusion, the results of this study suggest that the addition of a continuous subhypnotic propofol infusion in conjunction with a volatile anesthetic presents no added benefit in decreasing the incidence of PONV in the PACU. Further studies are needed to show the effectiveness of a subhypnotic propofol infusion in conjunction with a volatile anesthetic to decrease the incidence of PONV in high-risk patients. PROPOFOL INFUSION TO DECREASE PONV 22 References Aftab, H., Fagerland, M. W., Gondal, G., Ghanima, W., Olsen, M. K., & Nordby, T. (2019). Pain and nausea after bariatric surgery with total intravenous anesthesia versus Desflurane Anesthesia: A double blind, randomized, controlled trial. Surgery for Obesity and Related Diseases, 15(9), 15051512. https://doi.org/10.1016/j.soard.2019.05.010 American Association of Nurse Anesthesiology (AANA) (2021). AANA Journal Article. Retrieved November 1st, 2021, from https://www.aana.com/search?keyword=PONV&it=article Apfel, C. C., Laara, E., Koivuranta, M., Greim, C. A., & Roewer, N. (1999). A simplified risk score for predicting postoperative nausea and vomiting: conclusions from crossvalidations between two centers. Anesthesiology, 91(3), 693-700. https://doi.org/10.1097/00000542-199909000-00022 Chen, P.-N., Lu, I.-C., Chen, H.-M., Cheng, K.-I., Tseng, K.-Y., & Lee, K.-T. (2016). Desflurane reinforces the efficacy of propofol target-controlled infusion in patients undergoing laparoscopic cholecystectomy. The Kaohsiung Journal of Medical Sciences, 32(1), 3237. https://doi.org/10.1016/j.kjms.2015.11.006 Elbakry, A.-E., Sultan, W.-E., & Ibrahim, E. (2018). A comparison between inhalational (desflurane) and total intravenous anaesthesia (propofol and dexmedetomidine) in improving postoperative recovery for morbidly obese patients undergoing laparoscopic sleeve gastrectomy: A double-blinded randomised controlled trial. Journal of Clinical Anesthesia, 45, 611. https://doi.org/10.1016/j.jclinane.2017.12.001 PROPOFOL INFUSION TO DECREASE PONV 23 Folino, T. B., Muco, E., Safadi, A. O., & Parks, L. J. (2021). Propofol. StatPearls. Retrieved November 1st, 2021, from https://www.ncbi.nlm.nih.gov/books/NBK430884/ Gan, T. J., Belani, K. G., Bergese, S., Chung, F., Diemunsch, P., Habib, A. S., Jin, Z., Kovac, A. L., Meyer, T. A., Urman, R. D., Apfel, C. C., Ayad, S., Beagley, L., Candiotti, K., Englesakis, M., Hedrick, T. L., Kranke, P., Lee, S., Lipman, D., Minkowitz, H. S., Morton, J., & Phillip, B. K. (2020). Fourth consensus guidelines for management of postoperative nausea and vomiting. Anesthesia and Analgesia, 131(2), 411-448. http://doi.org/10.1213/ANE.000000000000483 Jokinen, J., Smith, A. F., Roewer, N., Eberhart, L. H., & Kranke, P. (2012). Management of postoperative nausea and vomiting. Anesthesiology Clinics, 30(3), 481-493. DOI: https://doi.org/10.1016/j.anclin.2012.07.003 Kawano, H., Ohshita, N., Katome, K., Kadota, T., Kinoshita, M., Matsuoka, Y., Tsutsumi, Y. M., Kawahito, S., Tanaka, K., & Oshita, S. (2016). Effects of a novel method of anesthesia combining propofol and volatile anesthesia on the incidence of postoperative nausea and vomiting in patients undergoing laparoscopic gynecological surgery. Brazilian Journal of Anesthesiology (English Edition), 66(1), 1218. https://doi.org/10.1016/j.bjane.2014.07.005 Lai, H.-C., Huang, T.-W., Tseng, W.-C., Lin, W.-L., Chang, H., & Wu, Z.-F. (2018). Sevoflurane is an effective adjuvant to propofol-based total intravenous anesthesia for attenuating cough reflex in nonintubated video-assisted Thoracoscopic Surgery. Medicine, 97(42). https://doi.org/10.1097/md.0000000000012927 PROPOFOL INFUSION TO DECREASE PONV 24 Miller, A. L., Theodore, D., & Widrich, J. (2021). Inhalation anesthetic. StatPearls. Retrieved November 1st, 2021 from, https://www.ncbi.nlm.nih.gov/books/NBK554540/ Muhly, W. T., Ganley, T., Jantzen, E., Browne, P., Kerr, J., Gurnaney, H. G., Kraemer, F. W., Galvez, J., Keren, R., & Wells, L. (2020). Reducing postoperative nausea and vomiting in pediatric patients undergoing anterior cruciate ligament reconstruction: A quality report. Pediatric Anesthesia, 30(4), 446454. https://doi.org/10.1111/pan.13813 Pang, Q.-Y., Duan, L.-P., Jiang, Y., & Liu, H.-L. (2021). Comparison of outcomes after breast cancer surgery between inhalational and propofol-based intravenous anaesthesia: A systematic review and meta-analysis. Journal of Pain Research, Volume 14, 21652177. https://doi.org/10.2147/jpr.s315360 Park, J., Kim, M., Park, Y. H., Shim, J., Lee, H. M., Kim, Y., Moon, Y. E., Hong, S. H., & Chae, M. S. (2020). Comparison of the effects of intravenous propofol and inhalational desflurane on the quality of early recovery after hand-assisted laparoscopic donor nephrectomy: a prospective, randomized controlled trial. BMJ Open, 10(12). https://doi.org/10.1136/bmjopen-2020-039881 Schraag, S., Pradelli, L., Alsaleh, A. J., Bellone, M., Ghetti, G., Chung, T. L., Westphal, M., & Rehberg, S. (2018). Propofol vs. inhalational agents to maintain general anaesthesia in ambulatory and in-patient surgery: A systematic review and meta-analysis. BMC Anesthesiology, 18(1). https://doi.org/10.1186/s12871-018-0632-3 PROPOFOL INFUSION TO DECREASE PONV 25 Shaikh, S. I., Nagarekha, D., Hegade, G., & Marutheesh, M. (2016). Postoperative nausea and vomiting: a simple yet complex problem. Anesthesia, Essays and Researches, 10(3), 388396. https://doi.org/10.4103/0259-1162.179310 Smith, M. J. & Liehr, P. P. (2018). Middle range theory for nursing (4th ed.). Springer Publishing Company. Uchinami, Y., Takikawa, S., Takashima, F., Maeda, Y., Nasu, S., Ito, A., & Saito, T. (2019). Incidence of postoperative nausea and vomiting is not increased by combination of low concentration sevoflurane and propofol compared with propofol alone in patients undergoing laparoscopic gynecological surgery. JA Clinical Reports, 5(1). https://doi.org/10.1186/s40981-019-0292-4 Vari, A., Gazzanelli, S., Cavallaro, G., De Toma, G., Targuini, S., Guerra, C., Stramaccioni, E., & Pietropaoli, P. (2010). Post-operative nausea and vomiting (PONV) after thyroid surgery: a prospective, randomized study comparing totally intravenous versus inhalation anesthetics. The American Surgeon, 76(3), 325-328. Wolf, A., Selpien, H., Haberl, H., & Unterberg, M. (2021). Does a combined intravenousvolatile anesthesia offer advantages compared to an intravenous or volatile anesthesia alone: A systematic review and meta-analysis. BMC Anesthesiology, 21(1). https://doi.org/10.1186/s12871-021-01273-1 Won, Y. J., Yoo, J. Y., Chae, Y. J., Kim, D. H., Park, S. K., Cho, H. B., Kim, J. S., Lee, J. H., & Lee, S. Y. (2011). The incidence of postoperative nausea and vomiting after thyroidectomy PROPOFOL INFUSION TO DECREASE PONV using three anaesthetic techniques. Journal of International Medical Research, 39(5), 18341842. https://doi.org/10.1177/147323001103900526 Weibel, S., Schaefer, M. S., Raj, D., Rcker, G., Pace, N. L., Schlesinger, T., Meybohm, P., Kienbaum, P., Eberhart, L. H., & Kranke, P. (2020). Drugs for preventing postoperative nausea and vomiting in adults after general anaesthesia: An abridged Cochrane Network metaanalysis. Anaesthesia, 76(7), 962973. https://doi.org/10.1111/anae.15295 26 PROPOFOL INFUSION TO DECREASE PONV 27 Appendix A Literature Review Matrix Citation Research Design & Level of Evidence Randomized Controlled Trial, Prospective Level I Population / Sample size n=x n=183 1. Aftab et al., 2019 2. Apfel et al., 1999 Prospective Level II n=2,722 3. Chen et al., 2016 n=52 4. Elbakry et al., 2018 5. Gan et al., 2020 Randomized Controlled Trial, Prospective Level I Randomized Controlled Trial, Prospective Level I Expert Opinions Level V n=100 n=9,000 Major Variables Instruments / Data collection Results TIVA with propofol Desflurane PONV Pain Awakening time Peritoneal stretch PONV Gender Smoking status Motion sickness Duration of operation Opioids Type of surgery Clavien-Dindo Classification Stata/SE 15.1 No significant difference was found between TIVA with propofol versus desflurane on PONV Logistic Regression Model The risk factors can predict the incidence of PONV with at least two risk factors urging prophylactic antiemetic therapy Propofol Desflurane Laparoscopic cholecystectomy TIVA with propofol Inhalation anesthesia Morbidly obese Pearson x2 test Student t test Patients were more hemodynamically stable when a combination of propofol and desflurane was used for anesthesia versus propofol alone GraphPad CONSORT flow diaphragm SPSS None TIVA showed superiority over inhalation anesthesia and provided better postoperative recovery with less side effects and analgesic requirements PONV risk factors Antiemetics Dosing and timing To reduce the risk of PONV, the guidelines recommend the use of propofol infusions and avoidance of volatile anesthetics PROPOFOL INFUSION TO DECREASE PONV 6. Kawano et al., 2016 Randomized Controlled Trial, Prospective Level I n=42 7. Lai et al., 2018 Randomized Controlled Trial, Prospective Level I n=90 8. Muhly et al., 2020 Quality Improvement Project Level V n=817 9. Pang et al., 2021 Systematic Review of RCTs and MetaAnalysis Level I n=2,201 10. Park et al., 2020 Randomized Controlled Trial, Prospective Level I Systematic Review of RCTs and MetaAnalysis Level I n=80 11. Schraag et al., 2018 n=20,991 28 Sevoflurane Propofol PONV Antiemetic use Postoperative pain Sevoflurane Propofol-based TIVA Cough reflex PONV SPSS Bonferroni post hoc tests The incidence of PONV was decreased when propofol was combined with sevoflurane Schneiders Kinetic Model Mann-Whitney test Kruskal-Wallis test Chi-Square The incidence of PONV was not increased with sevoflurane when combined with TIVA Antiemetics (dexamethasone, ondansetron, and a low dose propofol infusion) PONV ACL reconstructions TIVA with propofol Inhalation anesthesia Rescue analgesia PONV IL-6 Survival rate TIVA with propofol Inhalation desflurane Recovery outcomes IHI Model of Improvement PDSA Cycles QlikView Patients experience lower emesis after surgery due to implementing standard PONV guidelines and reducing opioids Stata 12.0 Eggers test Plot-digitizer software TIVA showed a decrease in PONV and an increase in postoperative rescue analgesia compared to inhalation anesthesia Korean version of the Quality of Recovery-40 questionnaire Cochrane Collaboration Tool TIVA improves the quality of recovery postoperatively compared to desflurane PONV Propofol Inhalational agents Post-operative pain Emergence agitation Time to recovery Hospital length of stay Post-anesthetic shivering Hemodynamic instability There was a reduction in PONV when using TIVA instead of volatile anesthetics, with a 39% risk reduction PROPOFOL INFUSION TO DECREASE PONV 12. Uchinami et al., 2019 13. Wolf et al., 2021 14. Won et al. 2011 15. Weibel et al., 2020 Randomized Controlled Trail, Prospective Level I Randomized Controlled Trial, Prospective Level I n=77 Randomized Controlled Trial, Prospective Level I Randomized controlled trials (RCTs) Level I n=177 n=1,960 n= 97,516 29 Propofol 0.8% Sevoflurane and propofol PONV PONV Time to extubation Pain Movement Propofol Volatile anesthetics PONV Sevoflurane TIVA with propofol Antiemetics Antiemetic drugs (aprepitant, ramosetron, granisetron, dexamethasone, and ondansetron) PONV Mann-Whitney U test Fisher exact test PONV is not increased with sevoflurane with propofol compared to TIVA with propofol Model of Random Effects Comprehensive Meta-Analysis Version 3.0 The incidence of PONV was reduced with the combination of propofol and volatile anesthetics compared to only volatile anesthetic SPSS Bonferroni post hoc test The incidence of PONV was decreased with TIVA and combined anesthesia with propofol, sevoflurane, and remifentanil None The study found that combinations of antiemetics were more effective in preventing vomiting, with NK1 receptor antagonists the most effective. PROPOFOL INFUSION TO DECREASE PONV 30 Appendix B Project Lead Created Tool PROPOFOL INFUSION TO DECREASE PONV 31 Appendix C IRB Approval Institutional Review Board DATE: 3-24-2022 TO: Kerri Ann Paris & Derrianne Monteiro FROM: Institutional Review Board RE: S22.116 TITLE: SUBMISSION TYPE: A Retrospective Study on the Use of Intraoperative Subhypnotic Propofol Infusion in Conjunction with Volatile Anesthetics to Decrease PONV New Project ACTION: Determination of EXEMPT Status DECISION DATE: 3-24-2022 The Institutional Review Board at Marian University has reviewed your protocol and has determined the procedures proposed are appropriate for exemption under the federal regulations. As such, there will be no further review of your protocol and you are cleared to proceed with your project. The protocol will remain on file with the Marian University IRB as a matter of record. Please be mindful of the importance of reporting only de-identified, HIPPA-compliant information about the patient in any exhibit or publication. Although researchers for exempt studies are not required to complete online CITI training for research involving human subjects, the IRB recommends that they do so, particularly as a learning exercise in the case of student researchers. Information on CITI training can be found on the IRBs website: http://www.marian.edu/academics/institutional-review-board. It is the responsibility of the PI (and, if applicable, the faculty supervisor) to inform the IRB if the procedures presented in this protocol are to be modified of if problems related to human research participants arise in connection with this project. Any procedural modifications must be evaluated by the IRB before being implemented, as some modifications may change the review status of this project. Please contact me if you are unsure whether your proposed modification requires review. Proposed modifications should be addressed in writing to the IRB. Please reference the above IRB protocol number in any communication to the IRB regarding this project. __________________________________________ Amanda C. Egan, Ph.D.  ...

... School Leaders Instructional Leadership After a Change in Assignment: An exploration of school leaders instructional effectiveness in the transition between school settings _______________________________________ A Capstone Project Presented to The Faculty of the Fred S. Klipsch Educators College Marian University _______________________________________ In Partial Fulfillment of the Requirements for the Degree Doctor of Education in Organizational Leadership _______________________________________ by Corye J. Franklin [December 2022] ii Copyright by Corye J. Franklin All Rights Reserved [December 2022] iv DEDICATION This capstone project is dedicated to my family and supporters that have been routinely supportive during this entire process. Their source of support has been unwavering as they have pushed me to be the absolute best I can be. This work is also dedicated to my parents, Vicki and James, and my grandmother, Lois, who have always quietly pushed and challenged me to keep going. They have granted me the autonomy and freedom to be my absolute best as long as I was moving forward. v ACKNOWLEDGEMENTS I would like to first and foremost thank God for being with me every step of the way during this entire process. I owe Him all the praise for without Him, none of this would be possible. I would also like to thank my Chair, Dr. Nelson, who continued to push me and question me. Additionally, my committee, Dr. Cornelius and Dr. Powell have kept me inspired as their feedback and encouragement kept me motivated. I would also like to give a special thanks to my support group and a few of my fraternity brothers, and cohorts. Bro. Gary Peters who would also encourage me through humor. Bro. Dr. Corey Tashombe Hardys daily inspiration and push kept me motivated. We started our journey together and were determined to complete it, by any means necessary. My cohorts, Anthony, Ashley, David, and Mariama are true leaders. Each one of them epitomizes tenacity and a desire to make education better for students. vi LIST OF FIGURES Fig. 1 PIMRS Data Table: Framing School Goals Demographics 51 Fig. 2 PIMRS Data Table: Framing School Goals Leadership Setting. 51 Fig. 3 PIMRS Data Table: Communicating School Goals Demographics 53 Fig. 4 PIMRS Data Table: Communicating School Goals Leadership Setting 54 Fig. 5 PIMRS Data Table: Coordinating the Curriculum - Demographics. 62 Fig. 6 PIMRS Data Table: Coordinating the Curriculum Leadership Setting. 63 Fig. 7 PIMRS Data Table: Providing Incentives for Learning Demographics. 66 Fig. 8 PIMRS Data Table: Providing Incentives for Learning Leadership Setting.... 67 Fig. 9 PIMRS Data Table: Promoting Professional Development - Demographics 70 Fig. 10 PIMRS Data Table: Promoting Prof. Development Leadership Setting..71 vii TABLE OF CONTENTS Page DEDICATION iv ACKNOWLEDGMENTS.. v LIST OF FIGURES. vi ELEMENTS I. STUDY DESCRIPTION.... 1 Introduction Diagnostic Work Emerging Themes Importance and Significance II. LITERATURE REVIEW..10 Leadership Styles Shared Mission Culture of Learning Organizations Reflective Practice Nature of Leadership in Elementary and Secondary Schools Common Characteristics and Common Barriers in Urban Settings Discussion and Gaps III. METHODOLOGY..... 37 Participant Recruitment Process Intervention Rationale Timeline Potential Limitations IV. DATA, FINDINGS & THEMES 47 PIMRS and Reflection Question Results & Themes Identified Shared Mission Leadership Styles Managing the Instructional Program Maintaining a Positive Learning Climate Support viii V. CONCLUSIONS.......... 79 Conclusions Next Steps for Further Research APPENDICES 89 A. Solicitation Email B. Screening Tool/Questions C. Informed Consent Form D. Principal Instructional Management Rating Scale (PIMRS) E. Reflection Questions/Prompts F. Raw Data Table REFERENCES.. 128 STUDY DESCRIPTION 2 Introduction Instructional leadership can be conceptualized as the leadership approach wherein the school leader collaborates with teachers, to provide both support and guidance to establish the best teaching practices for students (Brolund, 2016). At its core, instructional leadership is student-oriented, as the collaboration between the teachers and the principal is intended to ensure that the student has the best learning experience that can create the desired competencies (Spillane et al., 2003). In instructional leadership, the role of the school leader is not solely to provide instruction to the teachers, but to work in conjunction with them and to provide them with the necessary professional development tools that can enhance their pedagogical approaches and practices. Against this background, the present exploration sought to understand the determinants of instructional leadership effectiveness, in the context of school administrators who were transitioning now, or who have transitioned between school settings within the last 5 years. For this study, school leaders can be either a principal or an assistant principal, specifically. Additionally, when referencing a transition in this study, the transition referenced is referring to the transition from elementary (PreKindergarten to 6th Grade) to secondary (7th Grade to 12th Grade) school or the vice versa, secondary school (7-12) to elementary school (K-6). Before conducting any research on the topic, it was important to ascertain the extent to which the concept of instructional leadership was deemed pertinent by school leaders, and whether there were any 3 noteworthy differences in how instructional leadership is employed in elementary versus secondary school environments. To this end, I reached out to school leaders who have experienced transition within the last 5 years for just an off-the-record chat. The topics of my conversations are expounded on subsequently, before a brief consideration of current knowledge on the topic and a statement on the importance and significance of the research. Diagnostic Work In order to ascertain school leaders perspectives about the transition from one school environment to another, administrators who had already made such a transition were asked in casual diagnostic conversation to give some detailed information about their experiences. Information was gathered from ten administrators who are experienced in leadership positions in both elementary and high school environments, transitioning from one to the other in either direction. A diverse group in terms of age, gender, and ethnic identity was selected to ensure the study understood multiple perspectives. At a general level, the questions sought to determine definitions of instructional leadership; how it is used in elementary and secondary school environments; any differences and parallels in instructional leadership in the two pedagogical environments; as well as the skills or other forms of preparation that the school leaders in transition deem effective in the elementary and/or secondary school contexts. Some of the questions asked in the informal discussion were: What does instructional effectiveness mean to you? 4 How long were you an elementary leader? Secondary leader? How would you describe elementary instructional practices? Secondary instructional practices? What is the leaders role instructional at each level? At which level do you feel most effective? (Elementary or secondary) What does instructional monitoring look like at the elementary level? Secondary? Did one experience (setting) prepare you for the other, instructional? What skills were you able to transfer from one setting to the other, contributing to your effectiveness? Were the instructional portions of your own administrator evaluation in each setting similar? In what ways did the evaluation expectations differ between levels? As an elementary/secondary school leader, describe the role of each team member: Department Chair, Literacy Coach, Teacher Leader, Assistant Principal? Principal? Dean? What support did you need during your transition? How did the district meet your needs? What was missing? Were you afforded/provided a mentor for your transition? (Yes, no, option 3) Did you have a transition plan? (Yes, No). 5 How many years of experience do you have as a principal/assistant principal/dean at the elementary level? Secondary level? How do you identify? (Male, Female, Other, I prefer not to answer) How do you identify? (Black, White, Latino, Other) Did you transition from administration in an elementary (K-6) to secondary (7-12) school or from secondary to elementary? (Elementary to Secondary; or Secondary to Elementary) When you transitioned from one level to the next, was it in the same district or a different district? (Same district; or Different district) How would you describe the setting where you currently lead? These diagnostic questions helped gain a better preliminary understanding of instructional leadership in elementary and secondary school environments from the perspective of school leaders who have recently experienced transition. This was done before starting the actual formal study. Moreover, this information was useful in highlighting some important themes that can be used to understand the determinants of instructional leadership efficacy before, during, and after the transition from elementary leadership to high school leadership, or vice versa, from high school to elementary leadership. 6 Emerging Themes on Instructional Leadership for School Leaders in Transition Instructional leadership implicitly underscores the importance of the leadership styles that are used by school leaders, as this determines the nature of their interaction with teachers, students, and other stakeholders. The leaders who were contacted also corroborated the importance of leadership styles in general, in response to probes on the determinants of institutional effectiveness. Therefore, understanding the leadership styles that are used by school leaders is pertinent to instructional leadership, as it is consistent with the what aspect of leadership that leads to instructional effectiveness. According to the existing literature, the leadership styles that are predominantly associated with instructional leadership success are transformational leadership and servant-oriented leadership (Adarkwah & Zeyuan, 2020; Dussault et al., 2008; Williams & Hatch, 2012). The value of these approaches is that they each instill a perspective that the school leader needs the ability to collaborate with followers, thereby differing starkly from leadership approaches that emphasize a hierarchical structure (Greenleaf, 1977; Burns, 1978). The second critical determinant of instructional leadership efficacy is the existence of a shared mission within the learning institution. A shared mission is important because it aligns the efforts of the teachers and school leaders with measurable and specific objectives, which is critical considering the definition of instructional leadership (Hoe, 2007; Castelli, 2011). As noted earlier, instructional leadership seeks to ensure that the best outcomes for the student are observed, thereby implicitly placing 7 emphasis on the institutional prerogative to learn what students need, and how they can succeed. Therefore, the third critical determinant of instructional leadership efficacy is the extent to which the school leader manages the instructional program and how those actions are embedded within the institutional culture. A learning organization is important because it shapes the nature of effort that organizational members will exert to achieve common objectives, in addition to determining the cultural environment that shapes attitudes, beliefs, and perceptions about learning (Heorhiardi et al., 2014; Fiol and Lyles, 1985). The fourth critical determinant of instructional leadership efficacy pertains to the school leader developing a positive learning climate. Developing a positive learning climate is one way the school leader can examine his or her own beliefs associated with instructional effectiveness. This is important because it ensures that the school leader can support alternative views that may be important, useful, or relevant (Castelli, 2011). Current research supports the idea that instructional leadership in elementary school environments is predicated on a process of social construction (Spillane et al., 2003). The implication is that school leaders in elementary school environments must have the ability to leverage the social, human, cultural, and economic capital available to achieve instructional leadership efficacy (Spillane et al., 2003). In contrast, instructional leadership within the secondary school environment is largely based on a model of distributional leadership, in which the school leader and the teachers share the 8 responsibility to make decisions, based on a thorough understanding of internal and external factors (Hulpia et al., 2009; Huong, 2020). Overall, the current knowledge on instructional leadership isolates specific themes that can determine its efficacy, as well as highlights some broad differences between the elementary and secondary school environments. Importance and Significance of the Study The importance of this study is to provide useful information that can be used as a practical guideline by school leaders in transition from either elementary school to high school or vice versa, high school to elementary. Moreover, the study specifies the methods and leadership themes that are positively correlated with instructional leadership, implying a degree of practical relevance. The study is also significant because it draws connections between self-reflections by the school leader and how they collectively influence instructional leadership. It, therefore, draws on a broad range of literature that is useful in providing a holistic and comprehensive understanding of instructional leadership. The study is also important because it embeds the narratives and experiences of school leaders who have transitioned from elementary to high school or high school to elementary school, thereby facilitating the ability to bridge the gap between theory and the reality experienced by school leaders who have transitioned. In addition, the study can shape pedagogical and leadership approaches that are used in both elementary and secondary school environments, which is important because the knowledge can be used to enhance students learning outcomes. This particular study is 9 significant because of its practical relevance and usefulness for school leaders who may wish to transition from one school environment to another, as they can become cognizant of the differences and overlaps that will influence their instructional leadership, during and after the transition. Finally, this can also help district leaders in that after transitioning themselves and having self-awareness and experience with the transition, they can pass on their own knowledge and assist others in their transitions. 10 LITERATURE REVIEW The purpose of this literature review is to examine prior research underlying the instructional effectiveness of school leaders, as well as explore the topics within the research and how they pertain to transitional leadership. Instructional leadership can be conceptualized as a collaboration between the school leader and the teachers, intended to provide support and guidance in the establishment of best pedagogical practices (Mehmet, 2016). Therefore, instructional leadership is oriented towards achieving the best learning outcomes for the students, by aligning pedagogical practices with the learning needs of the student population. The insights drawn from existent literature will be invaluable in shaping an understanding of how school leaders exercise instructional leadership in schools and adapt from one school environment to another, to maintain a degree of instructional leadership. Another critical contextual background of the literature review is the diagnostic work that was conducted with school leaders who have already transitioned from one school environment to another, to explore their perspectives on instructional leadership and how it differs in the two contexts, anecdotally. A total of ten school leaders were consulted, and the findings will be referred to intermittently in the literature review. The literature review is arranged thematically, with each section highlighting important leadership orientations that are critical to success. Emergent themes from already existing literature include those such as leadership styles, the significance of a shared mission, the culture of a learning organization to 11 include managing the instructional program, and development of the school learning climate. An exploration of how each of these themes affects the nature of leadership in the transition from elementary to secondary school, or vice versa. The final section of the literature review will provide a consolidated discussion of the main points, as well as highlight and discuss the emergent gaps in the literature review, and considerations on how to avoid gaps in our own exploratory research. Furthermore, the participants that will be included in this exploration will be primarily from an urban district, eliciting an urban lens on the entire exploration, and making the exploration look at how education is different between urban and rural areas. Special focus will be paid to exploring common characteristics and common barriers to success in an urban setting. Leadership styles, in particular, are an important theme in this exploration, and will be a main lens for looking at the results of both the PIMRS survey, and the reflection questions. In particular, transformational and servant leadership styles are the ones that came about the most often in research and anecdotal evidence from educators who have previously transitioned, so these two types of leadership are the ones that will be focused on here in depth. Leadership Styles The two leadership styles we will focus on in this literature review are transformational leadership and servant leadership. These two leadership styles are the focus of this examination for several reasons. The first rationale for focusing on these two 12 frameworks is that they are highly pertinent and popular leadership styles in school leadership, based on the understanding that school leadership involves the ability to cater to the needs of multiple stakeholders as well as ensure that others without the leadership roles have the adequate support to perform their responsibilities (Spillane et al., 2003). Both of these leadership styles assert that school leaders should be able to think about not only the needs of their students and those they serve, but also keep in mind the overall mission of the organization, specifically at the school where they serve and lead. Each of these leadership types are practiced in both of the settings that we will be focusing on in this exploration, transitions from elementary to high school, and high school to elementary school. These two styles of leadership can be useful to those in transition. In the case of transformational leadership, there is the way leaders set an example for their followers, making them believe they can achieve the same successes, and with servant leadership, it is seen in the way the leaders offer support to teachers. The second rationale for focusing on transformational and servant leadership is that during preliminary diagnostic work, school leaders who had previously transitioned alluded to both of these theories explicitly and implicitly. For example, one of the diagnostic sources noted the importance of feeling that leaders are cognizant of their own professional aspirations, such that they create conditions that are conducive to the development of their talent. This assertion resonates with transformational leadership through the implicit focus on inspirational motivation (Adarkwah & Zeyuan, 2020), while it resonates with servant leadership through the focus on commitment to the growth of individuals (Williams & 13 Hatch, 2012, p.39). The theories of both types of leadership can be applied to the transition from elementary to high school leadership, and the vice versa, which is the focus of this exploration. The third rationale of the literature review focus is that the context of transformational leadership from one school setting to another implies that the leader must not only possess innate traits that determine the possibilities of success but also have the ability to adapt to a new situational environment while meeting the mandate to serve the interests of the entire organization. Transformational leadership would therefore be relevant in a leadership transition context because it provides insight into how individuals and organizations can be successful. The focus is on not only the individual success, and the success of the students, but also the overall success of the organizations that are in charge of the schools. Similarly, servant leadership would be relevant in a leadership transition context because it underscores the importance of placing the needs of the organization and stakeholders before self. Both of these types of leadership focus on the success of the educator, the student, and the organization as a whole. These two types of leadership resonate with the exploration in that they reinforce the many themes identified in the literature itself. They promote the ideas of a shared mission, reflective practice, and creating a positive cultural environment for education, reinforcing that these leadership styles are pertinent to discuss when exploring the idea of leadership and transition in leadership. 14 Transformational Leadership One of the most celebrated leadership styles in research is transformational leadership. A transformational leader is defined as a person who comprehends a realistic vision of the future that can be conveyed and shared, motivates subordinates intellectually, and addresses individual differences among subordinates (Adarkwah & Zeyuan, 2020, p.18). This means that a leader is a person who has a vision of the future, and is able to convey it to others, inspiring them towards the same goal. The specific character traits associated with a transformational leader include the ability to make decisions beyond the rationale of self-interest, the capacity to motivate followers, the ability to exercise idealized influence, the ability to instill trust and confidence within all organizational members, and the capacity to serve the needs of the organization, within the context of broader community needs (Adarkwah & Zeyuan; Burns, 1978). A leader thinks about not only themselves, their own ideas, and their own cause, but also the overall reputation and perception of the community and organization they represent. Moreover, Dussault et al. (2008) also highlight the significance of promoting selfefficacy among the followers, or the extent to which the leader can instill a belief among followers that they are capable of performing a specific task. Part of the magic of this type of leadership is that they empower their followers, or in our case, students, to believe that they are capable of something, whether it is learning something, achieving a good grade on an assignment, or having a successful future. The character traits associated with transformational leadership are crucial in shaping the performance of followers, and by 15 extension, determining the efficacy of the organization (Burns, 1978). If the organization does not have successful, empowered followers, then the organizations efficacy comes directly into question. Research suggests that the successful transition between different organizational environments is largely influenced by the degree of adherence to transformational leadership canons. On the one hand, the school leader adopting transformational leadership will portray themselves in a manner that fosters a desire for emulation among the followers (Adarkwah & Zeyuan, 2020; Burns, 1978). Having a school leader who sets a good example of what students can achieve in their own life is a facet of this type of leadership. On the other hand, a transformational leader will indirectly influence student learning and achievement outcomes by affecting teachers perceptions of the teaching profession, their roles and responsibilities within the pedagogical environment, and their self-perceived capacity to deliver on learning objectives (Adarkwah & Zeyuan; Dussault et al., 2008). A transformational leader in education will not only empower their students, but also the colleagues around them, sharing with them an air of motivation and selfrespect that they can then pass on to their students. The nature of transformational leadership is therefore consistent with instructional leadership, as it connotes a degree of collaboration between the teachers and the school leaders in shaping the objectives and learning outcomes of the student body, showing the success and efficacy of the organization as a whole. 16 Servant Leadership Apart from transformational leadership, a significant amount of research has also been conducted to evaluate the efficacy of servant leadership. The traits of a servant leader include listening, empathy, healing, awareness, persuasion, conceptualization, foresight, stewardship, commitment to the growth of people, and building community (Williams & Hatch, 2012, p.39). The servant leader puts the welfare and success of their followers over their own welfare and success. The main avenue through which servant leadership affects organizational efficacy is through the channel of nurturing trust and consolidating relationships between the various stakeholders (Williams & Hatch; Greenleaf, 1977). Much like transformational leadership, we see the idea of creating positive relationships between the leader and their followers, but the difference here is that the leader in servant leadership will put the needs of the followers over their own, instead of emulating something that their followers can look up to. This type of leader would put their followers wants, needs, and successes, above their own. A servant leader is therefore one who places him or herself at the service of others in the organization and actively engages in practices that are intended to improve the morale and productivity of the group which is especially important in navigating difficult times in the organization (Greenleaf, 1977). The embrace of servant leadership can be useful in shaping a school leaders instructional efficacy in the transition from primary to secondary school environments or the other way, from secondary to primary school, specifically, for a number of reasons. 17 First, servant leadership places the needs of the students and teachers (among other subordinates), beyond the interests of the school leader, such that it is possible to identify and work towards common organizational objectives (Williams & Hatch, 2012). By prioritizing the needs of students, it creates an environment that prospers the common objectives of the organization. Everyone in the organization knows that they can count on their colleagues and superiors to have the same common goals, and all be working toward the same outcomes. Second, it can be useful in raising the morale of the employees, which in turn is associated with a higher degree of employee retention, overall job satisfaction, and higher levels of productivity (Williams & Hatch). If the teachers are happy with their students hard work and success, then teacher morale will be higher, creating a much more positive environment for everyone involved in the organization. Third, servant leadership fosters a relational dynamic of trust among followers, as well as between the leader and the followers, which enables all members of the organization to set ambitious targets and work collaboratively to attain the common objectives (Williams & Hatch; Greenleaf, 1977). It is common knowledge that there must be trust in every successful relationship, be it business, education, love, or another industry, and servant leadership certainly fosters this idea and can enhance morale. The two leadership styles discussed in this section attest to the approach undertaken by many school leaders in organizational transition, which is why this exploration focuses on the two styles specifically. The similarity between transformational leadership and servant leadership pertains to the capacity of both 18 leadership styles to improve the motivation levels of followers, and consequently increase productivity through that motivation, achieving efficacy on both the part of the organization, and those within it. The main difference is that transformational leadership focuses on engendering self-efficacy and self-empowerment within the followers, such that the success of a transformational leader is necessarily reflected by how followers perceive themselves, their occupations, and their capabilities (Adarkwah & Zeyuan, 2020; Dussault et al., 2008). The successes of those underneath the leader reflect the overall success of the leader themselves. In contrast, the success of a servant leadership approach is predicated on the extent to which the leader can meet the needs of others in the organization, intimating that the leader is critical in ensuring the success of meeting organizational imperatives. The leaders own individual successes do not matter in servant leadership, instead the leader is vital to, and will do anything in their power, including sacrificing their own needs for, the successes of their followers and the organization as a whole. Although both approaches have merit, research aligns transformational leadership more with instructional leadership, implying that it represents the best approach that can lead to organizational success (Adarkwah & Zeyuan, 2020). This is also reiterated in Smith & Smiths evaluations (2015), when they state, an effective leader must routinely engage in a number of important instructional leadership elements that give sustenance and life to their leadership practices in order to confidently engage in those practices (p. 21) meaning that to be an effective leader, administrators must use many different approaches, which can include qualities of transformational 19 leadership. [W]hat is needed is the careful integration of leadership theories of practice that support specific instructional leadership practices according to Smith & Smith (2015, p. 21), which can be interpreted as meaning that the leadership skills that educators should embody should be self-sustaining and positive ones, ergo, more geared toward transformational leadership, which does not require that one give away all their resources. This makes logical sense, in that if educators practiced too much servant leadership, then they would not be able to take care of themselves, instead constantly sacrificing everything they have for the good of their followers and the organization. The main drawback of servant leadership in this case is that a leader who practices this philosophy will foster success and self-efficacy in their followers, even at their own detriment. For this reason, transformational leadership is often associated with instructional leadership, but in our preliminary exploration of leaders who have transitioned from elementary to high school or vice versa, both transformational and servant leadership were often mentioned, making us focus on both of them for the purpose of this exploration of transitional experiences. Regardless of the leadership style adopted by the school leader in question, an equally important determinant of instructional leadership efficacy is the extent to which all members within the organization have a common objective or shared mission. The Significance of a Shared Mission A shared mission is a key concept associated with a learning organization and is an important aspect of each organization. A shared mission can be defined as a clear and 20 common picture of a desired future state that members of an organization identify with themselves essentially a mission that has been internalized by members of the organization (Hoe, 2007, p.12), and can be in the form of a physical mission statement, or metaphorically in the form of a shared school of thought. A shared mission also attests to the existence of a commonly agreed to and expected agenda for the positioning of the organization in the future (Hoe, 2007). A shared mission asserts that there is a common goal that everyone in the organization will be striving towards. The ability to cultivate a shared vision within an organization is considered a critical determinant of its success and performance. A shared vision is useful insofar as providing the organization with a clear sense of purpose and direction it gives followers an idea of where the founders and current leaders within the organization came from in founding the organization, and where exactly they want to take the organization in the future (Hoe, 2007). It unites every individuals motive going forward in the organization, with a unified purpose and similar goals. This in turn is important because it creates a sustainable mindset within the organization, such that all current efforts and endeavors are necessarily related to their ability to achieve the shared mission in the future. Every individual in the organization can understand the shared mission, allowing them to work together as a team, meeting the standards set by the organization, and working towards the same goals as the rest of the organization. Furthermore, a shared mission represents a broad picture that can be used to align current performance targets and objectives with tangible and measurable success outcomes (Hoe, 2007). It allows the organization to set measurable standards that can be 21 met in order to achieve future goals associated with shared mission. It also gives the organization something to aim for, allowing them to measure how successful they have been thus far, and lets them measure their success in the future. Organizational performance targets would not only be rooted in meeting present needs, but also be connected to a long-term organizational objective. Finally, a shared vision can be critical in providing the organization with guidance on activities or protocols that need to be changed, versus those that need to remain in place (Hoe, 2007). Organizations can look at what has been working for them thus far, and improve areas that need improvement, while focusing on the areas where they have seen success thus far. In this context, a shared mission cannot be separated from a learning organization, as it ensures that the organization is capable of scrutinizing and ascribing value to current processes based on how those processes can achieve the common objective set out in the shared mission (Castelli, 2011). A shared mission is paramount in determining the instructional leadership efficacy of a school leader in transition. This is because while previous successes and performance were directly connected to the environment and situational realities of one level of schooling, the school leader must necessarily adapt goals and expectations to resonate with the new school environment. A shared mission becomes the tool through which the school leader can contribute to the creation of a shared goal and even a physical mission statement for their new organization, considering the desires and objectives of both subordinates and students. By looking for the shared mission in the new environment that 22 the leader finds themselves in, they can focus their practice on the mission of their new school or organization, and bond with new colleagues over the shared mission. Moreover, a shared mission is instrumental in guiding the success of instructional efficacy because it aligns the leaders activities and roles within the organization with the desires and intended outcomes of all pertinent stakeholders within the institution. Through the shared mission, the entire team of leaders can synchronize their efforts towards the same goal. Furthermore, instructional leadership can be promoted by a shared mission in terms of influencing knowledge acquisition approaches, as the organization can draw on both internal and external resources to achieve the common purpose. Again, everyone can work together towards the same desired outcome, identifying what does and does not work for them, and adapting as needed. As noted earlier, the concept of a shared mission is closely associated with the learning organization to which it belongs one of the foundations of an organization is their goal for bettering the future, and this comes in the form of the shared mission. Another important determinant of the efficacy of instructional leadership is therefore the extent to which the school leader can cultivate the culture and shared mission of the learning organization within the new school environment. The Culture of a Learning Organization The criteria for a learning organization extend beyond solely the nature of training and educational opportunities that are available to the members of an organization (Heorhiardi et al., 2014). Rather, Heorhiardi et al. (2014) characterize a learning organization as predicated on creating a climate that rewards openness about ideas, with 23 a bent for examining data and assumptions; and helping people become more selfreflective (p. 9). A learning organization achieves the status of an official learning organization through the facilitation of transformative, rather than informative, learning within its members. The difference between the two is that informative learning is based on promoting the acquisition of knowledge that resonates with the leadership beliefs of the individual, while transformative learning is based on encouraging the acquisition of knowledge in a manner that challenges or changes the prevalent mental models (Heorhiardi et al.). Transformative learning advocates for promoting change and improvement in the general society as well as personally as informative learning does just that, informs. Fiol and Lyles (1985) offered a different conceptualization of organizational leadership, arguing that there is a conceptual difference between organizational adaptation and organizational learning, such that change is not necessarily contingent on learning. This, therefore, challenges the assertions by Heorhiardi et al. in relation to the primacy of transformative learning in orchestrating effective change within the organizational environment. The resolution of this conflict is that the organizational environment can be changed through transformative learning, which resonates with Fiol and Lyles (1985) conceptualization of organizational learning. However, organizational adaptation extends beyond transformative learning, to capture how the organization responds to external and internal environmental factors that may signal the need for change. 24 In evaluating the central tenets of organizational learning, Fiol and Lyles (1985) contended that there are three main areas of consensus in the literature. First, environmental alignment is critical in maintaining the competitiveness and relevance of an organization within its sectoral context. Environmental alignment is the extent to which the organization can cultivate and realize the potential to learn, relearn and unlearn its past activities and behaviors (Fiol & Lyles, 1985). The implication is that organizational adaptation is the essence of strategic management because it is the key activity for dealing with changes occurring in the environment and involves the continuous process of making strategic choices (Fiol & Lyles, 1985, p.804). Organizational adaptation is dependent on environmental alignment, and environmental alignment is dependent on the organizational adapting constantly changing and reassessing what is best for the organizations climate. The second area of consensus about the constituents of organizational learning is the inherent difference between organizational and individual learning. Organizational learning is not perceived as the cumulative result of each individuals learning within the organization. Instead, it is oriented towards systemic level changes, such that while it is partially influenced by the nature and results of individual learning within the organization, it serves to preserve certain behaviors, mental maps, norms and values over time (Hedberg, 1981 as cited in Fiol & Lyles, 1985, p. 804). In this case, organizational learning can be described as similar to having a shared mission; although 25 the organization is constantly changing based on the environment, remaining focused and committed to the organizations goals. The final area of consensus is in relation to contextual factors. The four contextual factors that determine organizational learning are a corporate culture that deliberately promotes learning, a strategic orientation that values flexibility and adaptation, an organizational structure that is conducive to new insights and innovation, as well as the operational environment (Fiol & Lyles, 1985). The corporate culture can promote learning through the shared values, behavioral expectations, and ideological norms that permeate the entire organization (Heorhiardi et al., 2014; Fiol & Lyles, 1985). A flexible and adaptable strategic orientation ensures that the organization can create boundaries to decision-making processes, such that every decision made within the organization remains consistent with the goals, objectives, and mission of the organization. Organizational structure is equally important in shaping the processes of learning. For instance, a bureaucratic organizational structure will likely impede the learning of the organization and its members, as individuals remain constrained by their occupational positions within the company hierarchy. A learning-promoting culture fosters an element of fluidity within the organization, such that members from different departments, and employees with different ranks across the organization, can effectively learn from others and contribute to the learning of others (Fiol & Lyles, 1985). The significance of the internal and external environment to the learning of organizational members is based on 26 its relative complexity, such that a more complex and dynamic environmental reality will inhibit organizational learning. Heorhiardi et al. (2014) instead posited that organizational learning is best facilitated through four approaches. The first is the mandate to foster a culture that supports transformative learning on the individual level and double-loop learning on the collective level (p.9). This has a high degree of resonance with the assertions by Fiol and Lyles (1985) that the purpose of organizational culture is to facilitate learning in the organization, with Heorhiardi et al. adding the specification of transformative rather than informative learning. The second strategy to achieve a learning organization is the development and promotion of leaders that support the culture and transformation process (Heorhiardi et al., 2014). The third strategy is the promotion of a culture in which leaders are amenable to being challenged by others, as consistent with the principle of reflective leadership practice within the organization (Castelli, 2011). The final strategy is the encouragement of employees within the organization to actively engage in critical reflective practice and to apply the double-loop learning framework (Heorhiardi et al., 2014). Regardless of its different iterations in the current literature, organizational learning can be viewed as an important determinant of a school leaders instructional leadership efficacy in the transition from elementary to secondary school, or vice versa, for a plethora of reasons. First, organizational learning is necessarily rooted within the past and current context of the organization, such that all decision-making processes are 27 based on the consideration of pertinent variables. Second, organizational learning necessitates a culture of learning within the organization, implying that the school leader in question should be amenable to learning new realities in the new occupational context. By extension, the school leader in question is best served by engaging in reflective practice, allowing the other organizational members the same, and being open to criticism without overreacting (Heorhiardi et al., 2014). Third, organizational learning stresses the importance of transformative learning, which must challenge the current practices used by the employees in the organization, or at least encourage thought processes that are not essentially compatible with defining the schools mission. Finally, this can also help district leaders in that after transitioning themselves and having self-awareness as well as personal experience with the transition, they can cascade their learning, reflections, and experiences in an effort to assist others in their transitions. Reflective Practice Reflective practice is the ability to critically self-evaluate past decisions, behaviors, and perceived norms, to achieve personal and professional development, and is extremely vital to any administrative educator transitioning in either direction. Reflective practice cannot be achieved when an individual perceives his or her mental models as static and objective frameworks for evaluating reality (Castelli, 2011). Instead, reflective practice essentially acknowledges the inherent limitations and biases that can affect behavior and action, intimating that an individual who exercises reflective practice is cognizant of the possibility of inherent flaws within the worldview. By having a self- 28 reflective practice, one can assess the positives and negatives of their own practice, while also keeping in mind the mission of their organization and the needs of their subordinates, while assessing their practice. Steiner (1998) also argues that organizational learning can be impeded in contexts where routine reflection adopted by teachers has significant differences from the reflections that are promoted by the administration or organizational leadership. Steiner (1998), therefore, argues that organizational learning is best promoted in instances where reflection used across the entire organization is consistent. This would seem contradictory to the contributions by Castelli (2011), who argues in support of adopting fluid conceptualizations of reflections such that organizational learning is facilitated by the fluid interaction between all organizational members regardless of inherent conflicts. The contradiction can be reconciled by considering that Steiners main argument can be consistent with Castellis approach in that fluid reflective practice as argued by Castelli can be promoted by aligning them across the organization, as suggested by Steiner. If the organization adopts a selfreflective practice that extends across the organization, then everyone can work together towards its common goals. Moreover, reflective practice can be critical to the success of a school leaders instructional leadership style. Reflective practice enables both the school leader and the teachers to learn from their mistakes, as well as embrace the possibility of acquiring more information from others within the school including the student body. Reflective practice allows the educator to be a sponge, absorbing everything in the environment 29 around them, assessing what works and what needs to change, and adapting to the same values and practices that are promoted in the organizations shared mission. Reflective practice also changes the relational dynamics within the organization, implying that the promotion of reflective practice within the student body is equally critical in ensuring that instructional leadership achieves its primary objective of attaining certain learning and achievement outcomes among the students. The Nature of Leadership in Elementary and Secondary School Environments Instructional leadership in elementary schools is based on a process of social construction, through which different forms of capital are used to cultivate instructional leadership (Spillane et al., 2003). Specifically, teachers within the elementary school environment can construct others as leaders, based on shared values along the lines of economic, human, cultural, and social capital (Spillane et al., 2003). Moreover, instructional leadership in elementary school settings is informed by the nature of interactions between teachers and the leaders, as well as the leaders own position within the organization. The overall implication is that to implement instructional leadership in elementary school settings, school leaders need to understand the role of the different forms of capital (that is, economic, social, human, and cultural) in constructing leadership, as this is critical in determining the efficacy of leadership within that environment (Spillane et al., 2003). As noted by Spillane et al. (2003), instructional leadership in elementary school environments expands beyond the roles and responsibilities of the school leader, 30 including the teachers. This assertion is also corroborated by Bond (2021), according to whom teachers in elementary schools often feel the need to increase their decisionmaking capacity, while not necessarily entering into an administrative position, which then underscores the importance of instructional leadership. The research by Bond (2021) notes that in elementary schools, instructional leadership can also be achieved by providing teachers with the opportunity to continue instructing their students, while simultaneously providing different forms of support outside the classroom. For instance, teachers can demonstrate instructional leadership by tailoring specific learning material for children with learning disabilities (Bond, 2021). Notwithstanding the reality that teachers can participate in instructional leadership in elementary school settings, school leaders also have an important role. According to Garcia et al. (2014), the leadership efficacy of an elementary school leader can be inferred from their relative strengths and weaknesses on the Big 5 personality trait scale. Specifically, the researchers note that Open, Agreeable, and Emotionally Stable principals were perceived to be Transformational Leaders, while Open and Emotionally Stable school leaders were also perceived as Transactional Leaders, and when school leaders were rated as Conscientious and Emotionally Unstable, they were perceived as Passive-Avoidant Leaders (Garcia et al., 2014, p.204). As the emphasis in this study is on the overall effectiveness of school leaders that have transitioned from one setting to another, it will be interesting to note whether school leaders applying transformational leadership have the personality traits identified by Garcia et al. (2014). 31 In the secondary school environment, leadership tends to be based on a model of distributional leadership, wherein teachers and other staff can participate in decisionmaking processes (Hulpia et al., 2009). Despite the importance of distributional leadership in the secondary school environments, Hulpia et al. (2009) noted that a strict leadership hierarchy is not necessarily related to organizational commitment on the part of followers, intimating that a more rigid structure is deemed more appropriate in secondary school environments. In evaluating instructional leadership in secondary school environments, Huong (2020) noted that leadership efficacy is based on both external and internal factors. Concerning external factors, the most important variables are the degree of autonomy of the school for instructional activities; awareness of the managers and teachers about leading teaching activities; and education innovation, while the important internal variables include a system of guiding documents of management levels on teaching activities; conditions of facilities and finance for school teaching activities; and principals training level (p.48). In the context of this study, it will be important to note the extent to which the identified variables determine the efficacy of instructional leadership within both elementary and secondary school environments. Common Characteristics & Common Barriers in Urban Settings The focus of this exploration will be primarily dealing with educators from an urban setting, and for this reason, urban education will be focused on in the literature review in order to connect it back to the different themes that emerged during primary 32 research. Academic activist Pedro Noguera asserts that there are many barriers that are present in urban settings, primarily for students of color, and he talks about equity rather than equality the idea that not every individual has the exact same needs, and therefore every individual will have different specific needs to be attended to (2009). When he was interviewed by Emily Kaplan in 2020, he reasserted this idea by explaining that other countries have different approaches to improving education, and they are finding much success, so why cant these same practices also be applied in the USA? Noguera asserts in this interview that other countries use trust and guidance, rather than threats, and that makes a huge difference (Kaplan, 2020). By specifically examining an urban setting and making sure to collect experiences from many different ethnicities of principals and leaders in schools, it will be interesting to see if this theme comes up in the Reflection Questions section of the exploration. Thinking about the use of trust and guidance, rather than threats can be connected back with the themes of having a shared mission, cultivating a positive environment, and self-reflective practice. Noguera (2009) also asserts that if organizations do not provide equity in the form of resources that individual students need, then students will continue to suffer, using his own son as an example, explaining that when his son reached the tenth grade, his friends started to drop out due to lack of support at home or at school, and his son began to suffer as well, feeling like there was no upward trajectory for him as a black individual (Noguera, 2009). This is not directly asked about in the reflection questions, but it will be interesting to see if any themes of inequality are evident for any of the participants in their experience. 33 In his work, Noguera explains often how race and a number of other factors can interfere with the goals of providing a shared mission, cultivating a positive educational environment, and self-reflective practice. One of the ways he describes improvement in education for everyone is when he asserts that Assessment is an essential part of education, because you have to know what kids are learning. So, you have to assess their growth, their progress. But assessments should be used for that purpose and to diagnose learning needs, not to rank people, which is what we are doing now (Kaplan, 2020). He asserts that things need to change in terms of our shared mission, cultivation of a positive environment, and self-reflective practice. In order to really give equal footing to students, he asserts that each student will have different needs and need different supports, thus asserting that these themes of education need to be reevaluated to be more inclusive to all learners (Noguera, 2009). It will be interesting to examine how exactly Nogueras assertions and beliefs interact with the results of the PIMRS survey and reflection questions, being that our participants are all going to be located in predominately urban settings. Looking at the answers to the Reflection Questions section, this lens will be applied, looking at how the leaders in these urban settings use equity to their advantage to give students a better chance and more support. Discussion and Gaps The main avenues through which school leaders in transition can maintain instructional leadership effectiveness are through their leadership style, the promotion of a learning organization, defining the school mission across the organization, managing 34 the instructional program, developing the schools learning climate, and reflective learning. The identified elements are interrelated and co-dependent in achieving leadership efficacy, as they combine the personal leadership traits and characteristics of the leader, the mandate to align decision-making protocols with the organizations internal and external environment, and the embrace of continuous learning as a pivotal element of guiding practice within the organization. These avenues constitute what it takes to transition from one environment to another, and although there is a lot of literature available, there are a few gaps in the research. The first gap in the literature pertains to the impact of experiential knowledge in shaping the success of the leader in transition. In other words, understanding how the experiences of leading an elementary school versus the realities of leadership in a secondary environment assume a degree of continuity rather than change. It will therefore be critical to consider any qualitative research approaches that can include school leaders who have transitioned from elementary to secondary school environments or vice versa from secondary to elementary school environment settings, to gauge the relevance and contribution of experiential value. The second gap in the literature pertains to the extent of consistency between instructional leadership (which resonates with the what of leadership) on the one hand, and the leadership styles that are typically used in organizational settings on the other (which resonate with the how of leadership). Although the present study attempts to redress this gap by considering similarities and differences between instructional 35 leadership, transformational leadership, and servant leadership, there is a lack of literature that considers the conceptual differences in practice. This would be important in highlighting the challenges and potential barriers that school leaders in transition can face between the elementary and secondary learning environments. A way to address this gap in our own exploration is to look into the similarities and differences between different types of leadership. In summary, the purpose of the literature review was to highlight the range of practices that are used to achieve organizational effectiveness. The main research topic is to understand how school leaders transitioning from one learning environment to another can ensure the success of instructional leadership. Instructional leadership was in turn defined as the collaboration between the school leader and the teachers in creating a learning environment that is conducive to meeting the learning and achievement outcomes of the student body. The first identified avenue to ensure the success of instructional leadership for a school leader in transition is the nature of the leadership style adopted. While servant leadership has its merits, the critical literature review noted that transformational leadership may be more appropriate as it has a higher degree of resonance with instructional leadership. The second avenue is shared mission, which alludes to common goals and objectives across the entire organization. A shared mission can augment instructional leadership by creating commonly accepted operational objectives, thereby aligning current practices with long-term operational objectives. The third avenue is the 36 promotion of organizational learning and managing the instructional program through which the entire organization can benefit from learning from its mistakes, adapting to situational realities, and creating a culture that is conducive to learning and innovation. The final avenue is developing the school learning climate and reflective learning. Each one of these tenets can be achieved through reflective practice, as the school leader and teachers can actively engage in constantly challenging their world views to expand their epistemological base. Overall, the literature review suggests that instructional leadership efficacy can be maintained in the transition from one school setting to another, provided the school leader adopts a positive leadership style, participates in the generation of a shared mission, fosters a culture of organizational learning, promotes reflective practice, and manages the instructional program. 37 METHODOLOGY The purpose of this mixed-methods study was to examine the determinants of instructional leadership for school leaders in transition from one school setting and environment to another, specifically in this case, transitioning from elementary to secondary school leadership positions or the vice versa of transitioning from high school to elementary school. This examination and exploration sought to note the extent to which instructional leadership methods used in the transition of elementary to secondary school (or vice versa) environments were similar or different, thereafter accounting for what determined the efficacy of instructional leadership in each respective setting. This exploration focused on the transition from elementary leadership positions to high school leadership positions, or the opposite, in the past five years through collecting qualitative and quantitative data from individuals in these positions. The focus was aimed toward those who have already transitioned and have been in their new position for at least a year, if possible, in order to gain an insight into their transition. With reflective practice, this allowed the individuals to give insightful responses. The study was framed diagnostically through informal conversations and started formally with a screening questionnaire. The study then collected qualitative data through an initial screening survey, followed by the analysis of quantitative data using the Principal Instructional Management Rating Scale (PIMRS) by Hallinger & Murphy (1985) (Appendix D) with the authors permission. The PIMRS was selected because it investigated a school leaders instructional effectiveness based on three themes identified 38 in the literature review: defining the school mission, managing the instructional program, and developing the school learning climate. These were relevant to this exploration, as they also aligned with the tenets of the efficacy of instructional leadership. The study concluded with written participant reflections. These surveys, the PIMRS scale, and the reflection allowed the researcher to analyze the experiences of those who have transitioned from a position of leadership in either setting and concluded what is similar or different between the two school settings. This chapter presented the methodology or intervention that was used. The first section detailed the participant recruitment process, followed by a detailed explanation of what the proposed intervention entailed. The final section concluded with a consideration of some of the methodological limitations, as well as how the identified limitations were addressed in the study of elementary and high school leadership. It should be noted that this exploration was done with no intention of any bias, despite any personal experiences of the researcher. The motivation behind this researcher wanting to explore this topic was that their experience transitioning in the school system made them want to further explore how to improve the experience for others going forward. As a Principal in an urban setting who has transitioned personally, the researchers motive was one of service unto others. The ultimate motivation behind this exploration was to improve the transition process for others in the future, and make the transition more seamless, and ultimately just easier for those who move in either direction. Surveys were conducted with no bias or influence on the part of the researcher, 39 and responses and data were interpreted with no personal bias or influence. Results included in this exploration were factual and quoted directly from participants. Participant Recruitment Process Considering the specific focus of this study, the participant sample was comprised of school leaders who have experience in both the elementary and secondary school environments, specifically those who have transitioned from positions of leadership in an elementary setting to leadership positions in a high school setting or the other way, going from high school leadership to elementary leadership. The participants included school leaders transitioning from elementary to high school, or vice versa, from elementary to high school. They were currently transitioning or have transitioned within the last 5 years. The aim was to select participants that have finished their transition, hoping that they have spent a year in their new position so as to have had time to reflect on their experience, and explain how it could have been better or different. This was not a requirement, as it could have been complicated to find individuals to participate. A total of 10 participants were recruited, and 15 were contacted to find 10 willing and eligible candidates. A solicitation email (Appendix A) was sent to potential participants within a single urban area in the Midwestern, USA, and was focused on school leaders, specifically principals, and assistant principals. The email detailed the nature and purpose of the study and asked interested participants to demonstrate their eligibility to participate by confirming that they have transitioned from an elementary school to a high school or a 40 high school to an elementary school, as well as where they were, and where they are now, as an administrator. The second stage of the recruitment process was to screen the potential participants (Appendix B). The method used was purposive sampling, defined as an approach where the researcher has the prerogative to select participants based on their ability to meet specific criteria relevant to the research (Campbell et al., 2020; Barratt et al., 2015). Since all potential participants were school leaders that have transitioned, purposive sampling was used to ensure that the sample could be generalized to the population. Therefore, the 10 participants selected had varying demographic characteristics in terms of age, gender, race or ethnic identity, professional experience, and leadership style. Ideally, the final sample was comprised of at least five males and five females of different ethnic identities (Caucasian, Black, Hispanic, Asian), different ages, and different years of professional experience. After the purposive sampling technique was applied to narrow down the participants, an informed consent form was emailed to the selected individuals. The purpose of the informed consent was to provide the participants with information about the study, their rights during the research process, and the boundaries that would apply to the researcher (Xu et al., 2020; Rodriguez-Patarroyo et al., 2021). The informed consent form (Appendix C) detailed that the participants had the right to withdraw from the study at any time with no sanction towards them; and that they had the right to confidentiality. In addition, the informed consent detailed how the researcher would secure the 41 confidentiality and privacy of the participants, for instance by storing all sensitive information in a password-protected and secure location, as well as using pseudonyms in the presentation of results in the final research document. There was complete confidentiality, and pseudonyms were used when discussing findings to maintain the anonymity of subjects. The informed consent also provided the contact details of the researcher and encouraged the participants to ask any questions or express any concerns they may have had. Intervention The focus of the study was on examining school leaders in transition from one school setting to another, specifically those who have transitioned, or are transitioning, from elementary to high school or high school to elementary. The literature review highlighted important themes that ostensibly affect both the transition experience, as well as the efficacy of instructional leadership in both contexts. The themes in question were leadership style, shared mission, the culture of a learning organization, reflective practice, and managing the instructional program. Specifically, each participant was asked to reflect on their leadership style and instructional effectiveness through a written reflection. The reflection consisted of questions about the school leaders experience during their transition from elementary to high school leadership or vice versa (Appendix E). The questions encouraged the school leader to consider the themes (school mission, managing the instructional program, developing the school learning climate, and leadership style). The reflection was a written one, with questions to prompt the educator 42 about their experience. The rationale for asking school leaders to identify their own approaches (including leadership styles) and explain their experience, was because the literature highlighted this as being integral to implementing instructional leadership. The data collection for the intervention entailed a qualitative screening survey that was administered to school leaders transitioning from one school environment to another. The survey was administered first, to identify some definitional aspects of instructional leadership in the elementary and secondary schools. This initial survey was intended to probe into the lived experiences of the research participants concerning the transition from one setting to another, going from elementary to high school or vice versa. Secondly, a written reflection regarding the leaders leadership style and instructional effectiveness was collected to determine the leaders experience during the transition. The reflection consisted of the school leaders experience during their transition from leadership in elementary to high school or high school to elementary school, comparing the two directly. Lastly, the Principal Instructional Management Rating Scale (PIMRS) was administered to each participant to correlate themes and actions related to the leaders instructional practice and effectiveness. The Principal Instructional Management Rating Scale (PIMRS) was administered to each participant (Appendix D). The use of this scale was used to help connect directly to determining the instructional effectiveness of school leaders that have made a transition. The dimensions noted in the scale connected with the themes mentioned throughout the literature review. PIMRS is a framework that highlights the various 43 dimensions of instructional leadership (Hallinger & Murphy, 1985). The framework is divided into three different dimensions. The dimensions include defining the school mission, managing the instructional program, and developing a positive learning climate. The dimensions are broken into 10 instructional leadership functions. Using this instrument allowed the researcher to verify the experiences of those in leadership positions as well as connect overall themes, using a reliable and universal scale to reinforce and connect the ideas brought up in the initial survey and reflection. The researcher secured permission to use the PIMRS scale from the author in order to reinforce the conclusions drawn through preliminary surveys and reflections. This scale was used to triangulate data, themes, and connections identified in the participant surveys and reflections. The researcher explored how the school leaders results from the PIMRS contributed to their overall instructional effectiveness. This exploration also considered any professional development the school leader experienced and how it supported or shaped their effectiveness and leadership style through the reflection questions. At the end of the study, the researcher dissected and analyzed the gathered information to draw conclusions about the transition between elementary and high school leadership positions, or high school to elementary school leadership positions. One of the main limitations of qualitative data is that it is subject to various forms of bias that can jeopardize the integrity and accuracy of the collected information (Maxwell, 2014). To correct this potential limitation, school leaders were given the 44 Principal Instructional Management Rating Scale (PIMRS). The survey and reflection questions were able to ascertain the educators perspectives on the efficacy of the instructional leadership approaches adopted by the respective school leaders. Rationale for Intervention The first rationale for the proposed intervention was to demonstrate whether school leaders in transition found themselves to be more or less effective instructionally as a result of transitioning from one setting to another, or whether other factors influenced their instructional leadership. This was also important in determining the leaders overall instructional effectiveness after they transition from elementary to high school leadership, or vice versa, going from high school to elementary school leaderships. The second rationale for the intervention was to highlight themes experienced by transitioning principals and assistant principals, as collected from the reflections and the PIMRS, thereby providing some practical utility. The intervention was also able to ascertain whether the transformational leadership style or servantoriented style was associated with a higher degree of instructional leadership efficacy. Moreover, the participants reflections provided valuable insight into the relative contribution of the themes in affecting instructional leadership efficacy. Timeline The timeline of methodology was: in July and August of 2022, participants were identified as a part of the participant recruitment process using purposive sampling in the form of the Screening Tool/Questions. Once participants were identified, Informed 45 Consent forms were administered in August 2022. Following this, identified participants (the school administrators) were emailed the link to an electronic version of the PIMRS scale in August/September of 2022. PIMRS results were then be analyzed, and themes that connected to the dimensions mentioned in the PIMRS and literature review were looked for. A written reflection was administered to all identified participants of the study in September 2022. The reflection includes the participants accounts of their lived experiences after a transition from one setting to the other, either way. The reflection also allowed the participants to share their reactions to the PIMRS, their experience during the transition, and what instructional needs or supports they needed, had, or wished they had available to them in their transition from elementary to high school or high school to elementary leadership positions. After the reflections were collected and all information gathered, the researcher coded qualitative data, analyzed the quantitative PIMRS results, and triangulated the two considering the relevant literature. The researcher then identified recommendations, next steps, and conclusions. Potential Limitations The first potential limitation of the methodology pertained to the reality that the intervention required multiple measures for determining the instructional effectiveness. This limitation was mitigated by using the validated PIMRS instrument, which has been widely used in a range of research contexts. The second potential limitation of the methodology was that the responses of the participants in the survey and reflections may have been biased, such that the results collected may have been inaccurate. This potential 46 limitation was redressed by checking the accuracy of the participants reports against the quantitative PIMRS measurement for instructional leadership efficacy. This was also useful when examining the extent to which instructional leadership was associated with bridging the gap between the empirical research in this proposed study, against existing anecdotal information on the significance of instructional leadership. 47 DATA, FINDINGS & THEMES 48 Data Findings & Themes In this section, data, findings, and the themes between them will be explored in detail from screening surveys, PIMRS scales, and reflection questions that have been administered to the participants. From there, an in-depth analysis will be done in regards to instructional leadership in detail from each participants experience in transition, exploring the ideas of servant and transformative leadership and their efficacy in instructional leadership, considering positives and negatives of their experience of transition, and then finally discussing the conclusions drawn from this research. Following this data analysis the conclusions chapter will present recommendations for next steps for the future to make transitions from elementary to high school leadership, or the vice versa, from high school to elementary school leadership a smoother experience for all involved. The qualitative data was coded deductively by themes in the research literature, based on common keywords found in each participants responses. Keywords include shared mission, which relates to other keywords such as collaboration, framing, and objectives. Keywords also include the words leadership style, which relate to other keywords such as firm-handed, and words relating to teaching and leadership style. The words organizational learning are also included in the coding, which relates to professional development, workshops, and skills, and finally climate and reflection, are included in the coded keywords, which relate back to other keywords such as incentives, rewards, visibility, and positive energy. All 49 of these keywords and themes are visible in both the PIMRS results and the Reflection Questions. 50 PIMRS and Reflection Question Results & Themes Identified The responses to the PIMRS survey allowed insight into the general efficacy of leadership at the elementary and high school level, and the reflection questions were very interesting to look at and think about, allowing reflection into the efficacy of leadership in transition, and also allowing exploration into the different experiences of different leaders. All results provided honest and insightful explorations into individuals in leadership transitioning from elementary to high school or the vice versa. After receiving the results of the reflections and comparing them to the PIMRS results, there were several themes that were identified in the responses, and these are easily related back to the literature explored previously. These themes include having a shared mission, leadership style, managing the instructional program, maintaining a positive learning climate, and support. First, a look will be taken at each theme in terms to the responses received in both the PIMRS survey and Reflection Questions survey, where data will be assessed and examined as it pertains to each theme that is the focus of this exploration. Furthermore, a look will be taken at how the data collected shows the different experiences of those who have transitioned, particularly how the different subgroups of participants found the experience, and how it differed between participants. 51 Shared Mission Fig. 1 PIMRS Data Table: Framing School Goals - Demographics Personal Demographics: Overall Male Female White Black Hispanic Avg. n=10 n=3 n=7 n=6 n=2 n=1 Asian Age 31-40 41-50 51-60 n=1 n=4 n=4 n=2 Framing School Goals (Overall) 4.42 4.43 4.40 4.33 5.00 4.50 4.00 4.25 4.69 4.38 1.Annual Goals 4.31 4.33 4.29 4.16 5.00 4.00 4.00 4.00 4.50 4.50 2.Staff Responsibilities 4.55 4.67 4.43 4.33 5.00 5.00 4.00 4.25 4.75 4.50 3.Needs Assessment/Staff Input 4.14 4.00 4.29 4.00 5.00 4.00 4.00 4.00 4.50 4.00 4.Use student data 4.38 4.33 4.43 4.33 5.00 4.00 4.00 4.25 4.50 4.50 5.Easily understood 4.69 4.67 4.71 4.67 5.00 5.00 4.00 4.50 5.00 4.50 Fig. 2 PIMRS Data Table: Framing School Goals Leadership Setting ES-Leadership Setting: >HS n=7 HS->ES n=3 Voluntary Request Same Dist n=8 n=2 n=7 Framing School Goals (Overall) 4.36 4.58 4.57 4.00 4.36 1.Annual Goals 4.20 4.33 4.25 4.50 2.Staff Responsibilities 4.43 4.67 4.50 3.Needs Assessment/Staff Input 4.14 4.33 4.Use student data 4.29 5.Easily understood 4.71 Diff Dist n=3 Dist 1 Dist 2 Dist3 Dist 4 Dist 5 n=4 n=1 n=2 n=2 n=1 4.67 4.25 4.00 4.88 4.38 5.00 4.14 4.67 4.25 4.00 4.50 4.00 5.00 4.50 4.43 4.67 4.25 4.00 4.50 4.50 5.00 4.38 3.50 4.00 4.67 3.75 4.00 5.00 4.00 5.00 4.67 4.62 3.50 4.43 4.67 4.00 4.00 5.00 4.50 5.00 4.33 4.75 4.50 4.71 4.67 4.75 4.00 5.00 4.50 5.00 52 The Reflection Questions begin with some background questions, pertaining to the last few years and the environment of the participants transition. The participants have all transitioned in the last 5 years or less, and the majority of them transitioned voluntarily. Among the participants, 60% transitioned from elementary to high school, and the remaining 40% transitioned from high school to elementary school leadership. The first main theme that comes about in both the PIMRS and Reflection Questions survey is the idea of having a shared mission. As previously explained, having a shared mission is an important part of having a successful and productive leadership experience, and the results of both surveys reiterate this fact, showing through explanations that each individual valued having a shared mission that allowed them to build the foundations for success in their new position. Looking at the data from the PIMRS survey pertaining to framing the school goals, there is a noted difference between the age group of 31 to 40-year-olds (Xennials) compared to other age groups in almost every question and other millennials. Also known as younger millennials, this age groups average answer was lower than the older age groups, which prompts many different insights and questions about how transition differs based on the individuals age. The younger millennials, 31 to 40-year-olds, responses and interactions were different. Participants in that age group responded somewhat faster and were eager to share their experiences. As posited by McPhee and Zaug (2000), millennials engage and process differently within an organization. Moreover, this age group assesses how they will likely fit both functionally and socially in most settings. 53 This also prompts insight into the experience of this age group in transition compared to the older age groups (Baby Busters, Gen X, Generation Jones- older millennials). According to this data, framing school goals is less of a focus to the younger participants, as the answers show lower ratings in response to the questions asked in this section of the PIMRS, compared to the same questions asked of 41-50 year-old (Baby Busters/Gen X) and 51-60 year-old (Younger Baby Boomers/Baby Busters/Generation Jones) age groups. This is not the only place in the data where we see this trend, as can be seen below. Fig. 3 PIMRS Data Table: Communicating School Goals - Demographics Personal Demographics: Communicating School Goals (Overall) 6. Communicate mission 7. Discuss goals at faculty meetings 8. Refer to goals in curricular decisions 9. Goals in visible displays 10. Goals used in student forums Overall Avg. Male Female White Black Hispanic Asian Age 31-40 41-50 51-60 n=10 n=3 n=7 n=6 n=2 n=1 n=1 n=4 n=4 n=2 4.42 4.40 4.43 4.40 4.60 4.60 4.00 4.00 4.40 4.50 4.31 4.00 3.86 4.00 4.00 4.00 3.00 4.00 3.75 4.00 4.55 4.00 4.43 4.33 4.50 4.00 4.00 4.25 4.25 4.50 4.14 4.33 4.43 4.33 4.50 5.00 4.00 4.25 4.50 4.50 4.38 5.00 4.86 4.83 5.00 5.00 5.00 5.00 4.75 5.00 4.69 4.67 4.57 4.50 5.00 5.00 4.00 4.50 4.75 4.50 54 Fig. 4 PIMRS Data Table: Communicating School Goals Leadership Setting Leadership Setting: ES->HS n=7 HS-- Voluntary Request Same >ES Dist n=3 n=7 n=8 n=2 Diff Dist 1 Dist 2 Dist3 Dist 4 Dist 5 Dist n=3 n=4 n=1 n=2 n=2 n=1 Communicating School Goals (Overall) 4.40 4.47 4.52 4.00 4.43 4.40 4.30 4.00 4.80 4.38 4.40 6.Communicate mission 4.00 3.67 4.00 3.50 4.00 3.67 4.00 3.00 4.00 4.00 4.00 7.Discuss goals at 4.29 faculty meetings 4.33 4.38 4.00 4.29 4.33 4.25 4.00 4.50 4.50 4.00 8.Refer to goals in curricular decisions 4.29 4.67 4.50 3.00 4.43 4.33 4.25 4.00 5.00 4.00 4.00 9.Goals visible in 4.86 displays 5.00 5.00 4.50 4.86 5.00 4.75 5.00 5.00 4.50 5.00 10.Goals used in student forums 4.67 4.75 4.00 4.57 4.67 4.25 4.00 5.00 4.50 5.00 4.57 Here, looking at the data, it can be seen that the age group of 31-40 year old participants scored answers related to communicating school goals lower on average than the older age groups. This again reiterates that this data shows, for this particular study, how younger leaders in transition value a shared mission less than the older participants. This idea does not reflect as significantly in the Reflection Questions, where each individual stresses how they find the idea of a shared mission important to their own success in transition. 55 Looking generally at the questions related to the school mission, particularly the ones related to framing and communicating school goals in the PIMRS, the average responses of the group are high, which is reaffirmed in the Reflection Questions survey, where participants acknowledge how important it is to have a shared school mission to keep in mind and work towards, as a team, with their entire school during, and after a transition. This data shows us that having a shared mission is one of the fundamental parts of having a positive experience in transition. This section of the survey reinforces the principles of transformative leadership over servant leadership, in that it inspires change among the students and faculty, making connections, and building a strong foundation for future learning, rather than promoting sacrifice, and giving selflessly to inspire learners. The school leaders collaborate with faculty to build the schools mission and set goals for the students, with the idea of the students futures in mind, helping them to learn the fundamentals that are needed to be successful and productive members of society. This builds a strong foundation for students in preparation for future learning. Having a shared mission includes setting goals, communicating goals to both faculty and students, setting attainable objectives, and rewarding faculty and students for meeting and exceeding expectations. Most of the participants scored setting goals and communicating them relatively high, and also explained this in their responses to their reflection questions. Participants commonly remarked that they enjoy setting goals and meeting/exceeding them with their faculty and students, and communicating essential goals, that contribute to successful leadership. This also relates back to the idea of having 56 a shared mission, and how it is vital to those entering a new environment, whether it is from high school to elementary leadership, or elementary to high school leadership. The idea of being a team and having the entire faculty working together towards a common goal is one that was reiterated consistently throughout this section. Participants who were able to find support and a shared mission in their new environment boasted of more positive experiences than those who did not have much support or teamwork. In terms of comparing the experience of the participants, the idea of a shared mission is evident in all of the participants experiences, as shown by the high overall average of 4.42/5.00 on the PIMRS scale from participants (Fig. 1). Here, there is an obvious contrast between the different demographics of the participants. In regard to the Caucasian participants, there is a lower average score in this section (4.33), compared to the responses of Black participants (5). Furthermore, it can be seen that other participants of color also highly value the idea of the shared mission, in that Asian participants and Hispanic participants rated the importance of having a shared mission as a 4.00 and 4.50 out of 5.00, respectively. It is interesting to compare this data and ascertain that those who are seen as a minority highly value their shared mission as a foundation to their efficacy as a leader. 57 Leadership Styles Figures 1 and 2 show the data collected about the importance of a shared mission in terms of demographic and leadership setting, and this brings about some interesting insights into the types of leadership that participants found most efficient in their experience. The two types of leadership that were explored in depth were servant leadership and transformational leadership. Servant leadership involves giving everything for the success of your subordinates, including sometimes giving away all of your personal resources. This type of leadership style was not the most popular in this exploration, with participants ideals lining up more with transformational leadership, wherein the leader sets an example for their subordinates to follow and show students what they can achieve if they continue to work hard, just like their leader does. Transformational leadership would take previous assessments into account to create attainable goals for the year, and servant leadership would include sacrifices on the part of the leaders and faculty to give their students a positive experience and prosper growth. When asked the question about describing leadership style in the reflection, there were quite a few interesting answers, and the common theme among them was being authoritative, maintaining a level of respect, but also being open and communicative with students and faculty. For example, participant Hannah described her leadership style as firm-handed in her reflection survey: I think I would describe my leadership style as a firm handed approach. I expect a lot from my students, because I want them to be successful adults in our society. I 58 try to teach my students that hard work is at the center of success, and I try to instill in them the values necessary to be a productive member of our society. Participant Carl described his leadership style as interesting in his reflection, He said: I am friends with my students and staff, but at the same time, I will hold them accountable for their actions as a person of authority. They know that no matter what, they can come to me with any issue, and although I will not judge them personally, I may have to make certain choices because of my position. These answers reinforce the practice of transformational leadership over servant leadership. Regarding transformational or transformative leadership, leaders want to encourage and inspire their students to be the best they can be by setting a good example for them to aspire toward, knowing that their students may eventually outgrow them and go beyond even what they know. In servant leadership, the leader gives their all (sometimes to their own detriment) to see their students succeed, but resources are finite. One of the responses from Hannah mentioned that their students and staff refer to them as stern but they still have a great relationship with their students at the high school level. This shows that they do use transformational leadership to set a good example and hold their students to a very high standard, and the students still respect their leader. This is shown through the students trying their best to do a good job and work hard for their Principal, showing respect for the authority and for the Principals position. 59 Furthermore, there were many examples of participants mentioning words such as authoritative, assertive, but some also mention nurturing, leadership styles, or being friendly with students and faculty, while still holding them accountable for their success, reinforcing the idea of transformative leadership in that they are making lasting changes in their students and faculty. Although none of the participants mentioned servant leadership explicitly, it is evident that some of them are giving a lot of themselves to help their students, as evidenced by Hannah, who mentioned that their therapist was a great support system and positively enriched the experience of the transition, allowing brainstorming and the implementation of new coping mechanisms. It follows that by taking care of themselves and making sure that they are mentally healthy and feeling good by seeing a therapist, they can serve their school better, making sure that they are balanced emotionally, mentally, and professionally. It is to this end that transformational leadership, also known as leading by example, is the primary focus of our participants. Another theme that was common amongst the idea of leadership style in this section of the survey was the idea of meaningful connections, and making a lasting impression on students. Most of the participants mentioned this idea, in particular, participant Abby mentioned how much she enjoyed professional development focused around making meaningful connections with her students, the in-services and workshops that I attended throughout the last couple years of being at this new school have been very helpful, especially ones about connecting with students (I particularly liked a PD that I did recently about getting students speaking), and managing time/self-care. It is 60 clear that most of the participants of this survey have similar values when it comes to their leadership style. Comparing transformational leadership and servant leadership in this way, it is evident that most of the participants see their leadership style as more transformative, rather than serving their students and faculty. By making meaningful connections the leaders are inspiring a true change in their students, transforming them into members of society, and preparing them for the future. Furthermore, if we examine and compare the experiences of the different participants in term of the environment of their transition in leadership, there are some more interesting conclusions to be drawn. Looking at the difference between those participants who transitioned voluntarily and comparing it to those who were requested to change, there is also an interesting fluctuation in the data. In terms of framing the school goals, the participants who transferred voluntarily had a significantly higher score than those who were requested to transfer, especially for the questions pertaining to staff input, and using student data. The participants responses in both the PIMRS and the Reflection Questions survey furthermore reinforce the idea of transformational leadership. An example is how the participants whose transition was voluntary rated the need for framing the school goals as 4.57/5.00, and the participants who were requested to transition rated it as a 4.00/5.00. This reinforces transformational leadership in that the idea of a shared mission to build a foundation for the future is one that is extremely important in the implementation of transformational leadership. Transformational leaders set an example for their followers to look up to and hopefully achieve or exceed in their 61 own endeavors. If the results reinforced servant leadership over transformational leadership, the idea of having a shared mission to work toward would not be present, instead the idea of giving up everything that a leader possibly can to help their followers get ahead would be present. 62 Managing the Instructional Program Managing the instructional program is another important theme that is observed in both the PIMRS and the Reflection Questions, and it brings about an interesting set of data to explore. This can be seen in Figures 5 and 6, where coordinating the curriculum is looked at in particular, allowing the exploration of how the data presents. Fig 5. PIMRS Data Table: Coordinating the Curriculum Demographics Personal Demographics: Coordinate the Curriculum (Overall) 16. Make clear who is coordinating curriculum 17. Results of testing for curricular decisions 18. Monitor classroom curriculum for objectives 19. Assess overlap between objectives and achievement 20. Participate in reviewing curricular materials Overall Avg. Male Female White Black Hispanic Asian Age 31-40 41-50 51-60 n=10 n=3 n=7 n=6 n=2 n=1 n=1 n=4 n=4 n=2 4.20 4.27 4.17 4.20 4.40 4.20 3.80 4.00 4.35 4.30 4.50 4.67 4.43 4.67 4.50 4.00 4.00 4.25 4.75 4.50 3.90 4.33 3.71 3.67 4.50 4.00 4.00 3.75 4.00 4.00 4.20 4.00 4.30 4.17 4.50 4.00 4.00 4.00 4.25 4.50 4.00 4.00 4.00 4.17 4.00 4.00 3.00 3.75 4.25 4.00 4.40 4.33 4.43 4.33 4.50 5.00 4.00 4.25 4.50 4.50 63 Fig. 6 PIMRS Data Table: Coordinating the Curriculum Leadership Setting ES-Leadership Setting: >HS n=7 Coordinate the Curriculum 4.20 (Overall) 16. Make clear who is coordinating 4.57 curriculum 17. Results of testing for 3.86 curricular decisions 18. Monitor classroom 4.14 curriculum for objectives 19. Assess overlap between objectives 4.14 and achievement 20. Participate in reviewing 4.29 curricular materials HS->ES n=3 Voluntary Request Same Dist n=8 n=2 n=7 Diff Dist n=3 Dist 1 Dist 2 Dist3 Dist 4 Dist 5 n=4 n=1 n=2 n=2 n=1 4.20 4.25 4.20 4.20 4.20 4.05 3.80 4.60 4.20 4.40 4.33 4.50 4.50 4.60 4.50 4.50 4.00 4.50 4.50 5.00 4.00 4.00 3.50 3.71 4.33 3.50 4.00 4.50 4.00 5.00 4.33 4.25 4.00 4.14 4.33 4.00 4.00 5.00 4.00 4.00 3.67 4.00 4.00 4.14 4.00 4.00 3.00 4.50 4.00 4.00 4.67 4.50 4.00 4.43 4.25 4.75 4.00 5.00 4.50 4.00 Looking at coordinating the curriculum, there is some interesting data that prompts the conclusion that this theme of managing the instructional program is very important to leadership and the transition from one setting to another, which also reinforces transformational leadership; making a noticeable change in the students to help them prosper. If the overall average of 4.20 out of 5.00 is considered, it can be seen that this theme is very important to the participants, and therefore to those in leadership. One place where a fluctuation can be seen is in Fig. 6, looking at district two. The overall average for this section of the PIMRS is significantly lower than the other districts, as are the majority of the answers to the questions. This brings about a question 64 of whether or not this district values coordinating the curriculum in the same way as those in other districts whose scores were similar and rated significantly higher. There is an additional section of the PIMRS where there is some discrepancy. This is between the male and female participants, particularly in terms of question two, pertaining to using the results of testing for curricular decisions. The data revealed that female participants rated the question pertaining to using results of testing for curricular decisions lower (3.71/5.00 and 4.17 average for this section of the PIMRS) than the male participants (4.33/5.00 and 4.27 average for this section of the PIMRS). This interesting difference between males and females prompts the question of how both males and females can be better supported (by gender) in transition so that they are better able to embody the ideas of their style of leadership. Based on the female participants scores in this section, this singular variable could help to support the exploration that female participants might view themselves more as servant leaders in regard to this particular PIMRS question. As proffered by Williams & Hatch (2012), servant leaders build community and often put the needs of others before themselves. Conversely, it can be suggested for further research if male participants might exercise transformational leadership in this particular PIMRS area, as evidenced by the minute difference for this question in this study between the genders. In transformational leadership, one takes the resources that they are given, in this case the results of testing, to better support their students and faculty. It would be interesting to understand if there are valid and reliable 65 gender differences in the way that participants are using or reasons they are not using the resources or fully embodying the idea of transformational of servant leadership. 66 Maintaining a Positive Learning Climate Another overarching theme of both the PIMRS survey and the Reflection Questions survey is the idea of developing and maintaining a positive learning climate. The results of the PIMRS survey questions in this section varied in an interesting way, which will be compared to the reflection survey answers to make conclusions and draw connections to consider some of the questions. Fig 7 PIMRS Data Table: Promote Professional Development Demographics Personal Demographics: Promoting Professional Development (Overall) 41. Ensure that PD activities are aligned with school goals 42. Support the use in classroom of skills and PD 43. Get participation of school staff in PD activities 44. Lead/attend teacher PD regarding instruction 45. Make time during meetings for teachers to share PD ideas Overall Avg. Male Female White Black Hispanic Asian Age 31-40 41-50 51-60 n=10 n=3 n=7 n=6 n=2 n=1 n=1 n=4 n=4 n=2 3.88 3.67 3.97 4.03 3.70 3.60 3.60 3.80 3.90 4.00 4.50 4.00 4.71 4.67 4.50 4.00 4.00 4.25 4.75 4.50 4.20 4.00 4.29 4.33 4.00 4.00 4.00 4.25 4.25 4.00 3.40 3.33 3.43 3.50 3.50 3.00 3.00 3.25 3.25 4.00 3.20 3.33 3.14 3.33 3.00 3.00 3.00 3.00 3.25 3.50 4.10 3.67 4.29 4.33 3.50 4.00 4.00 4.25 4.00 4.00 67 Fig. 8 PIMRS Data Table: Providing Incentives for Learning Leadership Setting Personal Demographics: Providing Incentives for Learning (Overall) 46. Recognize students who do superior work with rewards 47. Use assemblies to honor students for academics and behavior 48. Recognize students in the office for their work 49. Contact parents to share good performance and contributions 50. Support teachers in their recognition of students in class Overall Avg. Male Female White Black Hispanic Asian Age 31-40 41-50 51-60 n=10 n=3 n=7 n=6 n=2 n=1 n=1 n=4 n=4 n=2 3.88 3.67 3.97 4.03 3.70 3.60 3.60 3.80 3.90 4.00 4.50 4.00 4.14 4.67 4.00 4.00 4.00 4.25 4.75 4.50 4.20 4.00 4.29 4.33 4.00 4.00 4.00 4.25 4.25 4.00 3.40 3.33 3.57 3.50 3.50 3.00 3.00 3.25 3.25 4.00 3.20 3.33 4.00 3.33 3.50 3.00 3.00 3.00 3.25 3.50 4.10 3.67 4.14 4.33 3.50 4.00 4.00 4.25 4.00 4.00 In this part of the survey, questions about incentives and professional development are raised, and we see middle-scoring ratings when it comes to things like taking the time to talk informally with colleagues and students and visiting classrooms to discuss school issues with teachers. The predominant number of participants rated most of these questions with a score of 3.00 or above. This also included the PIMRS questions related to covering classes or tutoring/providing direct instruction. Furthermore, many participants talked about incentivizing goals for both the faculty and the students. One participant in particular, Daisy, mentioned that they had 68 great success with incentivizing goals, with their school meeting and exceeding the goals set at the beginning of the year, earning some kind of reward: I connect most with the section about providing incentives for learning. To me, this is such a great idea, and it is something that I personally have had a lot of success with. Offering a reward of some sort for good, hard work has been very successful at our school. The consistent theme of meaningful incentives is pervasive in educational literature, as it is common practice that members of a school community tend to work harder if there is an incentive that is practical and associated with the goals and mission of the school or organization. In terms of leadership types, the data supported that participants leadership styles were challenged in certain areas, which can directly affect the efficacy of transformational and servant leadership. In reviewing this section only, it appears that the overall ratings were lower in the areas of recognizing students in the office for their work (3.40/5.00) and contacting parents to share good performance and contributions (3.20/5.00). Whether it is the cause of too much servant leadership, prompting leaders to need a break to replenish their own personal resources or something else that resulted in the ratings, or other variables, this fluctuation was interesting and surprising. Transformative leaders ultimately preserve their personal resources, setting a model example for a healthy and successful future for their students and faculty. These transformational leaders work diligently to provide positive incentives for student in an effort to promote a positive school culture and climate. Ultimately, this includes talking 69 informally and providing tutoring and instruction directly to students, as a way to impact instruction. In terms of providing incentives for teachers and students in this section, we see relatively high scores, with the lowest being rated at an average of 3.20 out of 5.00 and highest being rated at 4.50 out of 5.00, meaning that our participants are comfortable with incentives for hard work paying off, and do their best to reinforce morale when their colleagues and students are doing a good job of maintaining the standards of the shared mission. Transformative leadership is demonstrated by leaders inspiring change throughout their schools by meeting required goals to achieve success in their school communities, society, all while building a foundation for their futures. 70 Support The surveys go on to ask about describing transition from one setting to the next, asking participants to consider what factors made their transition easier or more difficult. The answers here had a common theme: support, which was essential in their transition. Fig 9 PIMRS Data Table: Promoting Professional Development - Demographics Personal Demographics: Promoting Professional Development (Overall) 41. Ensure that PD activities are aligned with school goals 42. Support the use in classroom of skills and PD 43. Get participation of school staff in PD activities 44. Lead/attend teacher PD regarding instruction 45. Make time during meetings for teachers to share PD ideas Overall Avg. Male Female White Black Hispanic Asian Age 31-40 41-50 51-60 n=10 n=3 n=7 n=6 n=2 n=1 n=1 n=4 n=4 n=2 3.96 3.80 4.03 3.97 4.30 3.80 3.40 3.80 4.15 3.90 4.00 3.67 4.14 3.83 4.50 4.00 4.00 4.00 4.00 3.50 4.20 4.00 4.29 4.17 4.50 4.00 4.00 4.00 4.25 4.50 3.600 3.67 3.57 3.50 4.00 4.00 3.00 3.50 3.75 4.00 3.9 3.67 4.00 4.00 4.50 3.00 3.00 3.50 4.50 4.00 4.10 4.00 4.14 4.33 4.00 4.00 3.00 4.00 4.25 4.00 71 Fig. 10 PIMRS Data Table: Promoting Professional Development Leadership Setting Leadership Setting: Promoting Professional Development (Overall) 41. Ensure that PD activities are aligned with school goals 42. Support the use in classroom of skills and PD 43. Get participation of school staff in PD activities 44. Lead/attend teacher PD regarding instruction 45. Make time during meetings for teachers to share PD ideas ES->HS n=7 HS-- Voluntary Request Same >ES Dist n=3 n=8 n=2 n=7 Diff Dist n=3 3.86 4.20 4.10 3.40 3.94 3.71 4.67 4.25 3.00 4.00 4.67 4.25 3.56 3.67 3.86 4.14 Dist 1 Dist 2 Dist3 Dist 4 Dist 5 n=4 n=1 n=2 n=2 n=1 4.00 3.60 3.40 4.30 4.50 4.20 3.86 4.33 3.50 4.00 4.50 4.50 4.00 4.00 4.14 4.33 4.00 4.00 4.50 4.50 4.00 3.75 3.00 3.57 3.67 3.25 3.00 4.00 4.00 4.00 4.00 4.00 3.50 3.86 4.00 3.50 3.00 4.00 4.50 5.00 4.00 4.25 3.50 4.28 3.67 3.75 3.00 4.50 5.00 4.00 Participant Ernie stressed that support was key in his successful transition, and he wished he had had even more support: During my transition, the main thing I needed was someone telling me that I was doing the right things, or just explaining how they had handled situations and experiences as the principal. It was great to have a supportive faculty and supportive students, but I wish I had had time to sit down with someone who had done this exact job before so they could impart some of their own experiences on me. 72 It seems like every single participant mentioned that a support system is super important, whether you already have one, or looking for one once you arrive in your new setting. Each participant had their own experience with a support of some sort or wished that they had had a support that made their transition easier, and reinforced that they were on the right track. Participants mentioned that faculty were helpful, and one participant, Hannah mentioned that although it was not easy to gain the respect of the students immediately, she was able to win them over and have them respect her: My transition was, honestly, relatively uneventful. I had a great faculty that welcomed me into the new school, and although it took a little bit of time to gain respect from the students, we have a good rapport now after a few years, and they are meeting expectations regularly. I am very proud of their hard work. When asked what they needed or wished they had had during their transitions, the participants all mentioned one thing: some kind of support. Many wished that they had had a guide of some sort, whether it be a workshop or in-service, or a support group of some sort with educators who have made similar transitions, the answers all varied on this same theme of supporting those in transition. This was something that was particularly interesting, because many of the participants made their own support systems, because they found what was offered lacking. Some, such as participant Daisy even went as far as to look on social media for support from teachers also in the State: During my transition, I found a group on social media that helped me a lot. It was a bunch of Indy educators who all support each other, and this was 73 a huge asset to me in my transition. Having a group that I could bounce ideas off of, and who had been through similar experiences was super helpful and I dont think I would be as comfortable as I am now without their help! I have also made some lifelong friends who all share similar interests to me, which is great! In the final section of the Reflection Questions, participants were asked what kind of feedback they would give to a leader experiencing a transition, and what they would say to district-level leaders who support leadership assignments. In terms of feedback for those in transition, the majority of answers included things such as just go for it! or take the leap! Some participants said that they were happy that they made the transition that their gut was telling them to do so, and that others should listen to their gut if they feel or felt like something needed to change. Others mention that reaching out for support should not be as difficult as it is and finding a support system made their transition much smoother. Participant Daisy stressed how important it is to just take the leap in her response: I would tell someone in transition to just take the leap if you feel its time! Making the change was the best thing possible for my mental health, and I feel like I am doing a much better job in this position than my previous one. Here, I feel like I am really making a difference, instead of just trying to keep my head above the water. In reference to the district-level leaders, Daisy goes on to say: 74 To those who support leadership assignments, I would say there should be more mental health support for our teachers and administrators. We deal with a lot of heavy topics quite often, and this can be detrimental if we do not have someone to talk to. There was a common theme of support, yet again found in these responses. Many participants mentioned that there should be some sort of support for those transitioning between education settings, and how that would have been a game-changer for their transition. Felix stressed this in his response: To leaders who support leadership assignments, I think it would be helpful to offer more support or some kind of workshop about transitioning from elementary to high school, or even from high school to elementary school. This could be a great asset to administrators all over the district and the State. Most participants built their own support system, but they all mentioned that had there already been one in place, that would have been great. One area where a discrepancy can be seen in the PIMRS results are the questions pertaining to ensuring that professional development activities are aligned with school goals, and making time during meetings for teachers to share professional development ideas. The fluctuation occurs between the participants in districts 1 and 2, and the participants in districts 3, 4 and 5. The participants in the first two districts show much lower scores than the participants in other districts. This prompts many questions about 75 the support being provided to participants in each district. Perhaps the participants in districts 1 and 2 are feeling less supported than the participants in other districts, which would mean that more support can be provided to those in the first two districts so that they feel more comfortable listening to input regarding professional development. If the leaders are truly embodying the idea of transformational leadership, they would take into account the needs and wants of their teachers, taking into consideration the feedback that teachers are giving them in order to plan professional development workshops that will benefit their teachers in their journey to transform their learners into functioning members of society. Comparing the experience of the different participants, it can be seen by looking at the averages of the importance of having a shared mission and proper support, that every single participant counts this as an important part of their journey. The PIMRS results and the reflection results bring about many common themes, but there is one that every single participant conveyed in both the PIMRS and the reflection questions: support. In both the PIMRS and the reflection questions, participants expressed that support was the biggest factor in determining whether their transition was positive or negative. Those who had a built-in support system boasted a positive experience, and those who did not have a big support system around them explained how they made their own, and how it was more difficult of a transition without a lot of support. Because working in education is such an intense commitment, it is important that administrators find places to get support if needed, especially if they are feeling worried and stressed. 76 One participant in particular, Hannah, mentioned that their therapist was vital in helping them through their transition, teaching them coping mechanisms and strategies for dealing with stress in their new environment. The common theme here of support is one that could definitely be taken into consideration when making decisions in the future to help not only instructional leaders in transition, but anyone who transitions from one job to another in any field. In reference to whether the new level of education was more or less difficult than the previous, one particular answer stood out, and it would be interesting to explore this further. Participant Isabel mentioned that they did not find the new setting more or less challenging at all, which many participants touched on in their response, but they described instead that the new setting was just a completely different environment, needing a completely different approach: I wouldnt say that leadership at this level is more or less challenging than the elementary level, but it is definitely different. You are dealing with a whole different circus here, with more mature themes, and sometimes rather heavy situations. Although I wouldnt say it is more or less challenging, I would say that it has changed me and made me more passionate about turning my students into good humans. One example that participant Felix described was that that they had to manage their expectations because of the change of level of education and maturity: 77 One of the difficulties I had transitioning to high school leadership was navigating the different experiences of all the students. Our student population varies, with some students coming from wealthy families, and others coming from not-sowealthy families, and navigating the politics of the high school can be a little overwhelming. One thing that really helped me with this transition from elementary to high school was my colleagues the new and the former ones! I am still in contact with many of my mentors and former colleagues, and I was able to reach out to them for advice, which was priceless! Furthermore, it was mentioned on many occasions that a big change was going from high school to elementary school, where educators are working with teenagers, who are about to entire society, and they have to readjust their expectations of their elementary students, who are just learning the ways the world works. Judging from these responses, it is clear that a different approach is needed in the two different education settings. Participants were asked to describe their leadership style now, post-transition, and there was a common theme of change. Almost every participant mentioned that they had to re-think their approach to instruction and leadership in order to best handle their new environment. Again, participants mention that they have to adjust their expectations based on which level they are transitioning to. Participant Jenny had an experience like this, where she had to manage her expectations based on her transition from one level to another: 78 Instructionally, my leadership is less extreme than when I was working in the high school, because you simply cannot approach elementary students in the same way you would approach a high school student. I do not find it more challenging than the high school level, but it is a very different environment that warrants a very different approach. When the participants were asked which of the themes of the PIMRS resonated with them. The answers here varied, with quite a few different choices. A few chose the framing school goals and communicating goals section, citing that communicating the goals of the school to students and faculty was a good way to set their expectations and set personal and professional goals for the year. For example, participant Abby said: The theme that connected to me the most was the framing and communicating the school goals sections. These resonated with me because I have been at this school for a few years now, and I think the environment is a positive one that students can really thrive in, and for this reason, I try to frame and communicate the goals to the students and faculty on a regular basis, keeping them on track for success. Others chose the visibility section, saying they felt that they led by example, so that section really resonated with them. An example of this is participant Bonnie who wrote: The theme that connected to me the most was the framing and communicating the school goals sections. These resonated with me because I have been at this school for a few years now, and I think the environment is a positive one that students 79 can really thrive in, and for this reason, I try to frame and communicate the goals to the students and faculty on a regular basis, keeping them on track for success. A particularly interesting trend was that a few participants chose the incentives for learning section and incentives for teachers, saying that they have had great success in setting goals with their students and faculty, and rewarding them for meeting and exceeding expectations. Participant Jenny was one of the participants who felt this way: I think that I connect most to the PIMRS theme of providing incentives for teachers. Especially in todays day and age, teachers and faculty at schools go above and beyond their usual duties, often sacrificing their own personal time, making sure that their students are having the best experience possible, and learning as much as they can. 80 CONCLUSIONS 81 Conclusions When beginning this exploration, instructional leadership was defined as the leadership approach wherein the school leader collaborates with teachers, to provide both support and guidance to establish the best teaching practices for students (Brolund, 2016). This means that instructional leadership is student-oriented, as the school leaders and teachers collaborate to ensure the students have the best possible learning experience, graduating with the required skills and competencies to continue in their learning journey, and beyond (Spillane et al., 2003). To this end, the role of the school leader is not only to provide instruction to the teachers, but to work in partnership with them to provide the necessary professional development tools that can complement their pedagogical practices and different approaches. In the conclusion, the results of our research and exploration will be looked at in conjunction with the theory of instructional leadership to give ideas as to how the transition process can be improved. There were a number of themes that were present in both the PIMRS and the Reflection Questions survey. Many of these will be touched on, helping exemplify how those who choose school leaders can improve the transition, before discussing next steps for further research. This exploration provided insights into instructional leadership in transition between elementary to high school, or high school to elementary school, that will allow those choosing school leaders to find ways to better help their colleagues in transition. The common theme of support was present in all the PIMRS and Reflection Questions 82 results, leading to the conclusion that more support is needed for those transitioning from either level to the other. Furthermore, support could include things like communicating a shared mission, and making school-wide goals for leaders as well as student, with incentives to keep their morale up, among many other things. Further looking at the idea of support, and comparing that to the definition of instructional leadership, it is evident that this part of transition can be improved in the future. If instructional leadership is not only supporting teachers personally, but also supporting their pedagogical practices and different approaches, then there can definitely be some improvement in the future for supports for those transitioning to another level of education. Some of the participants made their own support group via social media, so maybe this is something we can learn from and take into the future of instructional leadership in transition. Having a place for those who are transitioning now to talk to those who have already transitioned would be an invaluable resource. Those who have already experienced transition have a different point of view from those who have not, and can impart vital wisdom unto those newly transitioning, allowing them to almost have a cheat sheet of what they can do to make the change smoother. Many of the participants mentioned that either the support that they had was vital in their successful transition, or the lack of support really hindered them in their transition, which is why they went and found their own support. To this end, the conclusion can be drawn that support for transitioning leaders is vital and is something that can be focused on in the future to improve the environment for all involved. If the 83 teachers and school leaders are not properly supported, it follows that they would not be able to do their best at work, and therefore, cause the students to suffer, and not be prepared for future education and/or entering general society. The idea of a shared mission is one that was also evident in the exploration as being vital to the transition of individuals in leadership, and this was reinforced in both the PIMRS results, as well as the Reflection Questions responses. In terms of instructional leadership, a shared mission is vital to success. If those in leadership do not have a shared mission, then how will they know what the ultimate goal they are working toward is? By having a shared mission and reinforcing that by sharing it with all school leaders and faculty, successful instructional leadership, where teachers are being supported by their higher-ups, allows the teachers to do a better job of maintaining their pedagogical practices, and therefore achieve the goal of educating the students so that they are ready to graduate to the next level of education or even life. Furthermore, a shared mission leans toward the theory of transformational leadership that has been discussed previously, in that it inspires a change in students and faculty, giving them goals and objectives to work toward, and allowing leaders to impart their wisdom on the students and faculty, setting a strong foundation for future endeavors, all the while preserving the personal resources of leaders, so that they can also continue their own journey. Through this self-reflective practice, which is essential for growth as an educator, it is evident that transformative leadership is present in many educators practices, even if 84 they do not describe themselves as a transformative leader. Although the terms of transformative leader were not explicitly used in the participants responses, each of them embodied at least one aspect of transformative leadership, in that they are trying to preserve their own personal and mental resources by finding a support system to help them through the transition, all the while setting a standard for the students to meet by showing them what it means to be a good person and a successful and productive member of this society. Leadership style determines how an educator interacts with their students, and by using transformational leadership rather than servant leadership, our participants are allowing themselves to continue to grow and achieve great things through their students and faculty being successful. Next Steps for Further Research In the future, there are many interventions and further exploration that can be done to continue this type of research and hopefully make a positive impact on the experience of educators in transition. Looking at the results of this exploration, it can be seen that support is the most important part of a leaders transition from elementary to high school, or high school to elementary school. Going forward, further research can be done about leaders in transition, looking into what particular supports are needed or could be created for those transitioning. There are many different types of support that could be needed by those transitioning, including mental, physical, or financial support. Many of the participants mentioned that a support group of some sort was extremely beneficial to them, whether it was one that they created themselves, or one that they found on social 85 media, so this is something that could be explored further to see what exactly those choosing leaders can do to help with transition. If this is done, and administrators and those in leadership can create support groups for transitioning leaders, then higher morale and more information about transitioning can be provided to leaders transitioning educational levels, allowing them to preserve their personal resources, and successfully use transformative learning to set goals and expectations for faculty and students. By allowing those who have already transitioned educational levels to share their wisdom with those in transition, it could positively affect the turnover rate of educators, and help with those who have been needing to take mental health breaks such as stress leave. If those in education administration and leadership are able to stop leaders from experiencing things like burnout, then they will ultimately be more productive and have more success with their subordinates. More research into the section of the PIMRS about monitoring student success would also be an interesting exploration to continue to research. This section of the PIMRS had much lower average answers of 3.20 to 4.00, unlike other sections with higher average scores. It would be very interesting to compare this section of the PIMRS survey to a future studys reflection answers, as one could explore why individual meetings regarding student success and performance, discussing academic performance results with the faculty, informing faculty of schools performance in written form, as well as using rest and other performance measures to assess progress toward school goals, and informing students of schools academic progress were rated with such 86 fluctuation, the average falling as low as 3.20 in some sections. It is suspected personally that this section rated so interestingly because leaders are practicing some servant leadership, and potentially burning themselves out to give their students the best experience and the best leaders they can have. With transformative leadership, there would be some inspired change, as well as forms of self-care to preserve personal resources. If the leaders are unable to give feedback about performance, there must be a reason why. Furthermore, more exploration into the different styles of leadership is recommended. It is evidenced by this exploration that transformative/transformational leadership is one that is more self-preserving than servant leadership, in that transformative leadership allows the leader to lead by example, showing their students and faculty what they can achieve if they set their minds to it and work hard, rather than servant leadership where the leader is just doing anything in their power to help their students and faculty to succeed, sometimes at their own detriment. What could be explored here is: which is the more appropriate practice? One could argue after this exploration that transformative leadership helps leaders take care of themselves in order to help their faculty and students thrive, better than servant leadership. But under what circumstances is servant leadership an appropriate leadership style for transitioning leaders to practice? Another aspect that would be interesting to explore further would be how the experience in each environment is different from the other. 87 This study examined the leadership transition of a small number of participants in a single midwestern urban city. In the future, a larger sample size could be explored, which would allow even more exploration into the leadership styles and transition experience of those in leadership. Future research could explore the sample size of an entire state, in order to explore whether the same pattern occurs in a broader setting. In this sample, it is evident that transformative leadership was the more prominent practice for the participants, rather than servant leadership, in that the participants talked about making connections with their students and faculty, building a foundation for their futures, and preserving their own personal resources. With a larger sample size, it could be explored whether the same pattern is evident, or if servant leadership would be practiced by any educators. After exploring this topic of leadership in transition, and exploring the efficacy of different types of leadership, the data shows us the participants experience of transitions in leadership from elementary to high school, or high school to elementary school. It was evident that transformational leadership is the more prominent practice among our sample group, but this begs the question: does this mean that transformative leadership is better than servant leadership? If more leaders are finding a positive experience from the practice of transformational leadership, where they can save their own personal resources, does that mean that servant leadership should be re-evaluated for its efficacy? What benefits come from servant leadership, when one must give everything that they have to their subordinates, even if it means their own detriment? Are there more benefits 88 to transformative leadership than servant leadership? Could it be the other way around? It would be interesting to compare the two directly and survey leaders about their thoughts. One piece of data that is particularly interesting is the idea that the younger leaders in transition who participated in this study do not value the idea of a shared mission as much as the older participants. The data shows on more than one occasion that the 3140-year-old participants scored the importance of a shared mission and communicating that shared mission much lower than the participants of older age groups (Fig.1, Fig. 2). This does prompt some questions that would be interesting to explore. If one were to take a larger sample size, would this same theme reflect in the data? Does this mean that a new type of movement is coming, and the older individuals will soon be seen as old school and obsolete? Is there something that can be done for younger leaders in transition to help them reinforce the idea of a shared mission? More research could certainly be done into this idea to further support young leaders who are transitioning in any direction. Finally, more interactive research for exploring leadership in transition could be conducted in the future, with more probing survey questions for participants, which would allow further exploration into the types of leadership that are best used in practice, as well as how to improve the process of transition for leaders in education, going from any level to another. This could include the impact of gender and millennial groups during a transition in assignment. The data from an exploration such as this one, or more research could greatly benefit those in the future who transition between any level of 89 education. This exploration provides evidence as to how we can improve the transition for those in the future, and along with more research, concrete steps can be taken to make positive changes and add more supports for those who need it. 90 APPENDICES 91 Appendix A: Solicitation Email Subject: Instructional Leadership Research Subhead: You can contribute to Educational Research Dear (NAME), I am conducting research on the determinants of instructional leadership efficacy for school leaders in transition from the elementary to the secondary school environments. Specifically, I wish to probe into the instructional leadership approaches that are used by school leaders in transition; draw parallels and distinctions between instructional leadership approaches in the primary and secondary school environments, and understand the specific strategies that are used in transition from elementary to high school leadership or high school to elementary school leadership. The purpose of the research is to gather actionable evidence that can be used to promote more effective instructional leadership strategies by school leaders in transition, which in turn has significant practical, theoretical, and pedagogical value. Would you like to assist in this study? Please kindly review the eligibility criteria below and inform me of your interest if you qualify: Who can participate: Principals and/or assistant principals in Indiana Principals and/or assistant principals who are transitioning, or who have transitioned from elementary to high school, or vice versa in the last 5 years Why should you participate: To contribute towards current knowledge on effective instructional leadership strategies To gain access to informative resources that can be used to support and enhance instructional leadership during a school setting transition Participation in the proposed research will allow participants to provide feedback on their relative efficacy during, and perhaps after, a transition from elementary to high school (or vice versa) leadership. Kindly respond to this email with an expression of interest within 3 to 5 days. Subsequently, I will contact you with a screening survey as well as an informed consent form that will be critical prerequisites for participation in the study. Sincerely, Corye Franklin 92 Appendix B: Screening Tool/Questions The screening tool will be in the form of a brief survey administered to school leaders expressing an interest in the study. Its purpose is to ensure the school leaders meet the eligibility criteria and have some foundational knowledge that underscores their ability to participate in the study. The survey questions are as follows: How did you find out about the study? (possible answers: solicitation email; colleague). Have you worked as a school leader in an elementary school environment? When did you work as a school leader in an elementary school environment? Have you worked as a school leader in a high school environment? When did you work as a school leader in a high school environment? Are you familiar with instructional leadership? In one to two sentences, please explain what instructional leadership means to you. What age are you? (possible responses: 20-30; 31-40; 41-50; 51-60; 60-64; 65 and over) What is your ethnic identity (possible responses: Asian; Hispanic/Latino; African American/Black; Caucasian; Other; Prefer not to say) What is your gender? (possible responses: Male; Female; Non-binary; Prefer not to say) Have you transitioned from an elementary to a high school environment or vice versa as a school leader? When? Are you familiar with the broad purpose of the research? Are you interested in participating in the research? Are you aware there are no financial benefits to participating in the research? Do you have any preliminary concerns about participating in the research? If so, could you state them as briefly as possible? In one or two sentences, please explain what you seek or anticipate gaining from participating in the research. Do you have access to a personal and secure computer? 93 Appendix C: Informed Consent Form Title of Study: School Leaders Instructional Leadership After a Change in Assignment: A comparative analysis of school leaders instructional effectiveness in the transition between school settings. Principal Investigator, Affiliation and Contact Information: Corye Franklin, Marian University EdD candidate corye_f@yahoo.com 317.413.8847 Institutional Contact: Kurt Nelson, Ph.D., Faculty Advisor 317-955-6421 knelson@marian.edu 1. Introduction and Purpose of the Study I am conducting research on the determinants of instructional leadership efficacy for school leaders in transition from one school setting to another (elementary to secondary or vice versa). Specifically, I wish to probe into the instructional leadership approaches that are used by school leaders in transition; draw parallels and distinctions between instructional leadership approaches in the primary and secondary school environments, and understand the specific strategies that are used in transition. The purpose of the research is to gather actionable evidence that can be used to promote more effective instructional leadership strategies by school leaders in transition, which in turn has significant practical, theoretical, and pedagogical value. 2. Description of the Research A survey will be given out first to find appropriate candidates, who will then be asked to reflect on their instructional effectiveness, and to provide feedback on their relative efficacy in their transition over the last ten years. Participants will then be given the PIMRS to identify specific strategies and themes associated with instructional efficacy. 3. Subject Participation Participation criteria are as follows: Principals and/or assistant principals in the state of Indiana 94 Principals and/or assistant principals who have transitioned from an elementary to a secondary school environment Participants will be asked to complete a preliminary screening survey (approximately 10 minutes) before participating. 4. Potential Risks and Discomforts No known risks 5. Potential Benefits To contribute towards current knowledge on effective instructional leadership strategies To gain access to informative resources that can be used to implement instructional leadership in the transition from elementary to secondary school environments 6. Confidentiality All information taken from the study will be coded to protect each subjects name. No names or other identifying information will be used when discussing or reporting data. The investigator will safely keep all files and data collected in a secure location/area. Once the data has been fully analyzed it will be destroyed. 7. Compensation Subjects will not be compensated for participation in this study. 8. Voluntary Participation and Withdrawal Your decision to participate in this study is completely voluntary. If you decide to participate in this study, you may withdraw from your participation at any time without penalty. 95 10. Cost/Reimbursements There is no cost for participating in this study. I voluntarily agree to participate in this research program Yes No I understand that I will be given a copy of this signed Consent Form. Name of Participant (print): Signature: Date: 96 Appendix D: Principal Instructional Management Rating Scale (PIMRS) PRINCIPAL INSTRUCTIONAL MANAGEMENT RATING SCALE Principal Form Published by: Dr. Philip Hallinger 199/43 Sukhumvit Soi 8 Bangkok, 10110 Thailand www.philiphallinger.com Hallinger@gmail.com All rights are reserved. This instrument may not be reproduced in whole or in part without the written permission of the publisher. Principal Form 2.1 THE PRINCIPAL INSTRUCTIONAL MANAGEMENT RATING SCALE PART I: Please provide the following information if instructed to do so by the person administering the instrument: (A) District Name: _____________________________ (B) Your Schools Name: _______________________ (C) Number of school years you have been principal at this school: ___ 1 ___ 5-9 ___ 2-4 ___10-15 ___ more than 15 (D) Years, at the end of this school year, that you have been a principal: ___ 1 ___ 5-9 ___ 2-4 ___ 10-15 (E) Gender: ___ Male ___ Female ___ more than 15 97 PART II: This questionnaire is designed to provide a profile of your leadership. It consists of 50 behavioral statements that describe principal job practices and behaviors. You are asked to consider each question in terms of your leadership over the past school year. Read each statement carefully. Then circle the number that best fits the specific job behavior or practice as you conducted it during the past school year. For the response to each statement: 5 represents Almost Always 4 represents Frequently 3 represents Sometimes 2 represents Seldom 1 represents Almost Never In some cases, these responses may seem awkward; use your judgement in selecting the most appropriate response to such questions. Please circle only one number per question. Try to answer every question. Thank you. Principal Form 2.1 To what extent do you . . . ? I. FRAME THE SCHOOL GOALS 1. Develop a focused set of annual school-wide goals ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 2. Frame the school's goals in terms of staff responsibilities for meeting them ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 3. Use needs assessment or other formal and informal methods to secure staff input on goal development ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 4. Use data on student performance when developing the school's academic goals ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 98 5. Develop goals that are easily understood and used by teachers in the school ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 II. COMMUNICATE THE SCHOOL GOALS 6. Communicate the school's mission effectively to members of the school community ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 7. Discuss the school's academic goals with teachers at faculty meetings ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 8. Refer to the school's academic goals when making curricular decisions with teachers ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 9. Ensure that the school's academic goals are reflected in highly visible displays in the school (e.g., posters or bulletin boards emphasizing academic progress) ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 10. Refer to the school's goals or mission in forums with students (e.g., in assemblies or discussions) ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 III. SUPERVISE & EVALUATE INSTRUCTION 11. Ensure that the classroom priorities of teachers are consistent with the goals and direction of the school ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 12. Review student work products when evaluating classroom instruction ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 99 13. Conduct informal observations in classrooms on a regular basis (informal observations are unscheduled, last at least 5 minutes, and may or may not involve written feedback or a formal conference) ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 14. Point out specific strengths in teacher's instructional practices in post-observation feedback (e.g., in conferences or written evaluations) ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 15. Point out specific weaknesses in teacher instructional practices in post-observation feedback (e.g., in conferences or written evaluations) ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 IV. COORDINATE THE CURRICULUM 16. Make clear who is responsible for coordinating the curriculum across grade levels (e.g., the principal, vice principal, or teacher-leaders) ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 17. Draw upon the results of school-wide testing when making curricular decisions ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 18. Monitor the classroom curriculum to see that it covers the school's curricular objectives ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 19. Assess the overlap between the school's curricular objectives and the school's achievement tests ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 20. 20. Participate actively in the review of curricular materials ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 V. MONITOR STUDENT PROGRESS 21. Meet individually with teachers to discuss student progress ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 22. Discuss academic performance results with the faculty to identify curricular strengths and weaknesses ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 100 23. Use tests and other performance measure to assess progress toward school goals ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 24. Inform teachers of the school's performance results in written form (e.g., in a memo or newsletter) ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 25. Inform students of school's academic progress ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 VI. PROTECT INSTRUCTIONAL TIME 26. Limit interruptions of instructional time by public address announcements ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 27. Ensure that students are not called to the office during instructional time ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 28. Ensure that tardy and truant students suffer specific consequences for missing instructional time ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 29. Encourage teachers to use instructional time for teaching and practicing new skills and concepts ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 30. Limit the intrusion of extra- and co-curricular activities on instructional time ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 VII. MAINTAIN HIGH VISIBILITY 31. Take time to talk informally with students and teachers during recess and breaks ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 32. Visit classrooms to discuss school issues with teachers and students ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 33. Attend/participate in extra- and co-curricular activities ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 34. Cover classes for teachers until a late or substitute teacher arrives ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 101 35. Tutor students or provide direct instruction to classes ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 VIII. PROVIDE INCENTIVES FOR TEACHERS 36. Reinforce superior performance by teachers in staff meetings, newsletters, and/or memos ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 37. Compliment teachers privately for their efforts or performance ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 38. Acknowledge teachers' exceptional performance by writing memos for their personnel files ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 39. Reward special efforts by teachers with opportunities for professional recognition ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 40. Create professional growth opportunities for teachers as a reward for special contributions to the school ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 IX. PROMOTE PROFESSIONAL DEVELOPMENT 41. Ensure that inservice activities attended by staff are consistent with the school's goals ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 42. Actively support the use in the classroom of skills acquired during inservice training ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 43. Obtain the participation of the whole staff in important inservice activities ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 44. Lead or attend teacher inservice activities concerned with instruction ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 102 45. Set aside time at faculty meetings for teachers to share ideas or information from inservice activities ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 X. PROVIDE INCENTIVES FOR LEARNING 46. Recognize students who do superior work with formal rewards such as an honor roll or mention in the principal's newsletter ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 47. Use assemblies to honor students for academic accomplishments or for behavior or citizenship ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 48. Recognize superior student achievement or improvement by seeing in the office the students with their work ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 49. Contact parents to communicate improved or exemplary student performance or contributions ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 50. Support teachers actively in their recognition and/or reward of student contributions to and accomplishments in class ALMOST NEVER ALMOST ALWAYS 1 2 3 4 5 As described in the Technical Report there are over 200 PIMRS studies that have been conducted around the world. If you are doing research with the PIMRS it will benefit you greatly to identify other studies that are focusing a similar topic (e.g., gender, effects of instructional leadership on school climate etc.). Similarly, if you are doing research outside of the USA, you may find related studies in your own country (e.g., Thailand, China, Pakistan, Zambia). Again, it will benefit you to read those studies. To facilitate your search process, you may down load the full LIST of PIMRS STUDIES by clicking on this link. Once you have identified studies that interest you, you may be able to find the relevant pdf file in the links below. I have organized the pdf files by decade and saved them into zipped files. If you download the zip files, you should be able to find most of the studies that you are looking for. I strongly recommend using and citing these studies in your own dissertation. 103 PIMRS Studies 1980s PIMRS Studies 1990s PIMRS Studies 2000s PIMRS Studies 2010s PIMRS Studies 2013-15 Download Download Download Download Download ABOUT THE AUTHOR Professor Dr. Philip Hallinger, author of the Principal Instructional Management Rating Scale (PIMRS), received his doctorate in Administration and Policy Analysis from Stanford University. He has worked as a teacher, administrator, and professor and as the director of several leadership development centers. He has been a consultant to education and healthcare organizations throughout the United States, Canada, Asia, and Australia. The PIMRS was developed with the cooperation of the Milpitas (California) Unified School District, Richard P. Mesa, Superintendent. As a research instrument, it meets professional standards of reliability and validity and has been used in over 200 studies of principal leadership in the United States, Canada, Australia, Europe, and Asia. The scale is also used by school districts for evaluation and professional development purposes. It surpasses legal standards for use as a personnel evaluation instrument and has been recommended by researchers interested in professional development and district improvement (see, for example, Edwin Bridges, Managing the Incompetent Teacher, ERIC, 1984). Articles on the development and use of the PIMRS have appeared in The Elementary School Journal, Administrators Notebook, NASSP Bulletin, and Educational Leadership. The PIMRS is copyrighted and may not be reproduced without the written permission of the author. Additional information on the development of the PIMRS and the rights to its use may be obtained from the publisher (see cover page). 104 Appendix E: Reflection Questions/Prompts 1. What year did you transition? Was your transition voluntary, or requested by the district? 2. Please describe the setting of your transition. Was your transition from elementary to high school, or high school to elementary school? Was it in an urban, suburban, rural, Catholic, charter, or district school? Did you remain in a similar environment post-transition? Did you remain in the same district area? 3. How would you describe your leadership style in general? 4. Describe your transition from one setting to the next. Think specifically about your experiences and adjustments you made or are making, considering what factors made it easier or more difficult to transition levels (i.e., personal qualities/traits, leadership style, district support/PD, school staff). 5. Instructionally, after your transition, how would you describe your leadership? In what ways is leadership at your new level more or less challenging than the previous level? 6. What did you need, or wish you had, during or after your transition from one setting to the next? 7. Considering the PIMRS and the identified themes, to which theme did you connect the most? Please explain. 105 8. What advice or feedback would you give to a leader experiencing this transition, or to district-level leaders who support leadership assignments? 106 Appendix F: Raw Data Table 107 108 REFERENCES Adarkwah, M., Zeyuan, Y. (2020). The Paradoxical Relationship between Principals Transformational Leadership Styles and Teachers Motivation. International Journal of Educational Excellence, 6 (2),15-46. Barratt, M. J., Ferris, J. A., & Lenton, S. (2015). Hidden populations, online purposive sampling, and external validity: Taking off the blindfold. Field Methods, 27(1), 321. Bond, N. (2021). Teacher leadership in an elementary school: A case study of instructional support specialists. Education, 141(4), 201-213. Brolund, S. (2016). Student success through instructional leadership. BU Journal of Graduate Studies in Education, 8(2), 42-45. Burns, J. M. G. (1978). Leadership. New York: Harper & Row. Castelli, P. (2011). An integrated model for practicing reflective learning. Academy of Educational Leadership Journal, 15, 15-30. Campbell, S., Greenwood, M., Prior, S., Shearer, T., Walkem, K., Young, S., . . . Walker, K. (2020). Purposive sampling: Complex or simple? Research case examples. Journal of Research in Nursing, 25(8), 652-661. Dussault, M., Payette, D., Leroux, M. (2008). Principals transformational leadership and teachers collective efficacy. Psychological Reports, 102(2), 401-410. Fiol, C., Lyles, M. (1985). Organizational learning. Academy of Management Review, 10(4), 803-813. 109 Greenleaf, R. K. (1977). Servant leadership: A journey into the nature of legitimate power and greatness. Paulist Press. Hallinger, P., Wang, W-C., Chen, C-W. (2013). Assessing the measurement properties of the Principal Instructional Management Rating Scale: A meta-analysis of reliability studies. Educational Administration Quarterly, 49(2), 272-309. Heorhiardi, A., Venture, K., Conbere, J. (2014). What do organizations need to learn to become a learning organization? Organizational Development Practitioner, 46(2), 5-9. Hoe, S. (2007). Shared vision: a development tool for organizational learning. Development and Learning in Organizations; Bradford, 21(4), 12-13. Hulpia, H., Devos, G., & Rosseel, Y. (2009). The relationship between the perception of distributed leadership in secondary schools and teachers and teacher leaders job satisfaction and organizational commitment. School Effectiveness and School Improvement, 20(3), 291-317. Huong, V. T. M. (2020). Factors affecting instructional leadership in secondary schools to meet Vietnams general education innovation. International Education Studies, 13(2), 48-60. Kaplan, E. (2020, September 30). Pedro Noguera: The work is not yet done. Edutopia., from https://www.edutopia.org/article/pedro-noguera-work-not-yet-done Maxwell, J. A. (2014). Qualitative research design: An interactive approach. Thousand Oaks, Calif: Sage Publications. 110 McPhee, R. D., & Zaug, P. (2000). The communicative constitution of organizations: A framework for explanation. Electronic Journal of Communication, 10(12). http://www.cios.org/getfile/MCPHEE_V10N1200 Mehmet, S. (2016). Factors related to instructional leadership perception and effect of instructional leadership on organizational variables: A meta-analysis. Educational Sciences: Theory & Practice, 16(5), 1761-1787. Noguera, P. (2009). The trouble with black boys: And other reflections on race, equity, and the future of Public Education. Jossey-Bass. Okolie, A. (2016). Exploring the implementation of everyday anti-racism education by elementary teachers in their classroom instruction. Ontario Institute for Studies in Education of the University of Toronto, 1-77. Rodriguez-Patarroyo, M., Torres-Quintero, A., Vecino-Ortiz, A., Hallez, K., FrancoRodriguez, A., Rueda Barrera, E., Ali, J. (2021). Informed consent for mobile phone health surveys in Colombia: A qualitative study. Journal of Empirical Research on Human Research Ethics, 16(1-2), 24-34. Smith, A. (2022). Indianapolis public schools launches new recruitment and retention initiative. WRTV. https://www.wrtv.com/news/hiringhoosiers/education/110ndianapolis-public-schools-launches-new-recruitment-andretention-initiative 111 Smith, J., & Smith, R. (2015). The architecture of instructional leadership ability. In J. Smith, & R. Smith The Architecture of Instructional Leadership Ability (pp. 2144). Corwin, https://dx.doi.org/10.4135/9781483384955 Spillane, J. P., Hallett, T., & Diamond, J. B. (2003). Forms of capital and the construction of leadership: Instructional leadership in urban elementary schools. Sociology of Education, 76(1), 1-17. Steiner, L. (1998). Organizational dilemmas as barriers to learning. The Learning Organization: Bradford, 5(4), 193-201. Williams, S., & Hatch, M. (2012). Influences of school superintendents servant leadership practices to length of tenure. Journal of Organizational Learning and Leadership, 10(2), 36-58. Xu, A., Baysari, M. T., Stocker, S. L., Leow, L. J., Day, R. O., & Carland, J. E. (2020). Researchers views on, and experiences with, the requirement to obtain informed consent in research involving human participants: A qualitative study. BMC Medical Ethics, 21(1), 93-112.  ...

... A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 1 A d ult P ri m a r y C a r e P r o vi d e rs A d v e rs e C hil d h o o d E x p e ri e n c e s ( A C Es) K n o wl e d g e, I m pl e m e nt ati o n, a n d P e r c ei v e d B a r ri e rs : A D N P p r oj e ct Ell e n M. M c C all a L ei g ht o n S c h o ol of N ursi n g, M ari a n U ni v ersit y S u b mitt e d i n p arti al f ulfill m e nt of d e gr e e r e q uir e m e nts f or t h e D o ct or of N ursi n g Pr a cti c e F a mil y N urs e Pr a ctiti o n er C h air: _ T ar a R. F o x, D N P, C P N P _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _1 _2/_1_8/_ _2 _0 _2 _2 _ _ _ _ _ _ _ _ _ _ _ ( Si g n at ur e) C o m mitt e e M e m b er: ( D at e) _ S h a n a R. Al e x a n d er, M D _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _____________________________________________________ ( Si g n at ur e) D at e of s u b missi o n: D e c e m b er 1 3, 2 0 2 2 ( D at e) A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S T a bl e of C o nt e nts A bstr a ct 3 I ntr o d u cti o n .. 4 B a c k gr o u n d .. 6 Pr o bl e m St at e m e n t . ..9 G a p A n al ysis.. . . ... 1 1 R e vi e w of t h e Lit er at ur e . .. . .. 1 2 T h e or eti c al Fr a m e w or k .. .. 1 8 Ai ms, O bj e cti v es, a n d H y p ot h es es.. 1 9 M e t h o ds .. ... ... 2 0 Pr oj e ct Sit e a n d P o pul ati o n . .. ... 2 1 M e as ur e m e nt I nstr u m e nt .. . .. 2 2 Et hi c al C o nsi d er at i o ns .. . .. 2 4 D a t a A n al ysis a n d R es ults .. ... .. 2 5 Dis c ussi o n.. .. ... .. 3 3 R ef er e n c es . 3 6 A p p e n di c es ( List e d s e q u e nti all y i n or d er t h e y a p p e ar i n p a p er) .. 4 5 A p p e n di x A ... 4 5 A p p e n di x B 4 6 A p p e n di x C 4 7 A p p e n di x D 4 8 A p p e n di x E .. .. 5 1 A p p e n di x F .. .. 5 2 2 A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 3 A bstr a ct O bj e cti v e: A d v ers e C hil d h o o d E x p eri e n c es ( A C Es) ar e h ar mf ul or distr essi n g e v e nts o c c urri n g wit hi n a c hil ds s o ci al or f a mil y e n vir o n m e nt w hi c h disr u pt ps y c h ol o gi c al a n d p h ysi c al d e v el o p m e nt. T h e g o al of t his D N P pr oj e ct is t o a ns w er t his q u esti o n: H o w d o pri m ar y c ar e pr o vi d ers ( P C Ps) p er c ei v e t h eir k n o wl e d g e, tr ai ni n g, s cr e e ni n g, a n d i nt er v e nti o ns w h e n utili zi n g A C Es r es e ar c h i n t h eir pr a cti c e ? T his D N P pr oj e ct s e e ks t o a ns w er t his q u esti o n usi n g a mi x e d m et h o d q u a ntit ati v e d esi g n. M e t h o d: A 2 0 it e m el e ctr o ni c q u esti o n n air e w as distri b ut e d t o 3 3 a d ult o ut p ati e nt pri m ar y c ar e pr o vi d e rs. Q u a ntit ati v e d at a w as c oll e ct e d fr o m 1 9 of t h e q u esti o ns. T h es e i n cl u d e d i n q uiri es c o n c er ni n g k n o wl e d g e of A C Es, s cr e e ni n g h a bits, a n d p er c ei v e d b arri ers t o i nt e gr ati o n of A C Es i nto a d ult pri m ar y c ar e pr a cti c e as w ell as p arti ci p a nt d e m o gr a p hi cs. T h e fi n al it e m of t h e q u e sti o n n air e w as a n i n vit ati o n t o dis c uss a d diti o n al m oti v ati o ns a n d p er c e pti o ns wit h t h e i n v estig at or. R es ults: Mi ni m al st atisti c all y si g nifi c a nt d at a is d u e t o t h e s m all s a m pl e si z e. D at a w as a n al y z e d uti li zi n g a c hi-s q u ar e t est, m e a n a n d m e di a n r es ult s, a n d i nf er e nti al st atisti cs t o t est pr oj e ct h y p ot h es es. D es cri pti v e st atisti cs w er e utili z e d t o d es cri b e t h e p o p ul ati o n, d e m o gr a p hi cs, a n d d a t a, vi a gr a p hs a n d t a bl es. C o n cl usi o n: T h e i m pl e m e nt ati o n a n d r es ults of t his pr oj e ct pr o vi d e f e w si g nifi c a nt fi n di n gs. H o w e v er, a d diti o n al dis c ussi o n of P C P m oti v ati o ns a n d p er c e pti o ns off er a u ni q u e p ers p e cti v e t o t h e c o n c e pt of w h at is r e q uir e d t o tr a nsl at e t h e A C Es i nf or m ati o n i nt o a d ult pri m ar y c ar e. K e y w or ds: A d v ers e c hil d h o o d e x p eri e n c es, a d ult o ut p ati e nt, t o xi c str ess, pri m ar y c ar e pr o vi d ers, m ot i v ati o ns, a n d tr a u m a-i nf or m e d c ar e. A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 4 Pri m ar y C ar e Pr o vi d ers A d v ers e C hil d h o o d E x p eri e n c e K n o wl e d g e, I m pl e m e nt ati o n, a n d P er c ei v e d B arri ers : A D N P Pr oj e ct I ntr o d u cti o n T h e ps y c h os o ci al i m p a ct o n h e alt h is a c o m m o n c o n c e pt i n m o d er n m e di ci n e ( B et h ell et al., 2 0 1 7; D u n p h y et al., 2 0 1 9; M erri c k et al., 2 0 1 9). First d es cri b e d b y H a ns S el y e as al ar m r e a cti o ns dis pl a y e d i n a ni m als, ps y c h ol o gi c al c o n n e cti o n t o bi ol o gi c al f u n cti o ns c o nti n u es t o fi n d s u p p ort i n s ci e ntifi c lit er at ur e ( P etr u c h elli et al., 2 0 1 9; S el y e, 1 9 3 7; S el y e , 1 9 9 8; S z a b o et al., 2 0 1 7). S el y e r ef err e d t o t h e c o m pl e x i nt er a cti o ns of str ess ors o n t h e b o d y i n 1 9 3 6 as g e n er al a d a pt ati o n s y n dr o m e ( S el y e, 1 9 3 7; S el y e, 1 9 9 8). O v er ti m e, S el y e diff er e nti at e d t h e t er ms e ustr ess a n d dist r ess d u e t o t h e i m p a ct of w or ks fr o m S w e dis h r es e ar c h er L e vi L e n ar d ( S el y e, 1 9 7 5; S z a b o et al., 2 0 1 7). I n t h e mi d - 8 0s, t h e b e h a vi or al s ci e n c es d e v el o p e d t h e st u d y of ps y c h o n e ur oi m m u n ol o g y t h e i nt ers e cti o n of t h e ps y c h e, t h e n er v o us s yst e m, a n d t h e i m m u n e syst e m ( V oll h ar dt, 1 9 9 1). N ot u ntil t h e e arl y 1 9 9 0s di d i n v esti g at ors a p pl y t his i nf or m ati o n t o pri m ar y c ar e ( F elitti et al., 1 9 9 8). F elitti et al. w er e i niti al r es e ar c h ers w h o q u a ntifi a bl y assi g n e d a n u m b er t o d et e ct a n ass o ci ati o n b et w e e n a d ult c hr o ni c ill n ess a n d f or m er e x p eri e n c es of c hil d h o o d a d v ersit y ( F elitti et al., 1 9 9 8). R e c e nt s ci e ntifi c a d v a n c e m e nts c o nfir m t h e c orr el ati o n of a d v ers e c hil d h o o d e x p eri e n c es ( A C Es) h a vi n g eff e cts i nt o a d ult h o o d ( B et h ell et al., 2 0 1 7; B h us h a n et al., 2 0 2 0; B or a et al., 2 0 2 1; H u g h es et al., 2 0 1 7). A C Es ar e h ar mf ul or distr essi n g e v e nts o c c urri n g wit hi n a c hil ds s o ci al or f a mil y e n vir o n m e nt w hi c h disr u pt ps y c h ol o gi c al a n d p h ysi c al d e v el o p m e nt ( Al h o w a y m el et al., 2 0 2 1). R es e ar c h ers cl assif y t h es e tr a u m ati c e v e nts i nt o t hr e e c at e g ori es (s e e T a bl e 1; B or a et al., 2 0 2 1; F elitti et al., 1 9 9 8; N ati o n al C e nt er f or Inj ur y Pr e v e nti o n a n d C o ntr ol, 2 0 2 2). A c orr el ati o n al r es p o ns e e xists b et w e e n e x p os ur e t o A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S c hil d h o o d tr a u m a a n d t h e d e v el o p m e nt of c hr o ni c dis e as e ( B or a et al., 2 0 2 1; F elitti et al., 1 9 9 8). A p pr o xi m at el y 6 1 % of a d ults i n t h e U nit e d St at es of A m eri c a ( U. S. A.) r e p ort e x p eri e n ci n g at l e ast o n e A C Es ( M erri c k et al., 2 0 1 9). Si xt e e n p er c e nt of U. S. a d ults h a v e e x p eri e n c e d f o ur or m or e ( M erri c k et al., 2 0 1 9). A C Es s c or es of f o ur or m or e ar e ass o ci at e d wit h e x p o n e nti al h e alt h ris ks an d is c o nsi d er e d a p u bli c h e alt h crisis ( B h us h a n et al., 2 0 2 0; F elitti et al., 1 9 9 8; H u g h es et a l., 2 0 1 7: P etr u c c elli et al., 2 0 1 9). T a bl e 1 C at e g ori es of A d v ers e C hil d h o o d E x p eri e n c es A b us e Ps y c h ol o gi c al or N e gl e ct H o us e h ol d d ysf u n cti o n P h ysi c al n e gl e ct M e nt al ill n ess of h o us e h ol d m e m b er E m oti o n al n e gl e ct D o m esti c vi ol e n c e t o w ar ds a p ar e nt E m oti o n al a b us e P h ysi c al a b us e S e x u al a b us e I n c ar c er ati o n of a h o us e h ol d m e m b er Di v or c e, s e p ar ati o n, or l oss of a p ar e nt S u bst a n c e a b us e: I n cl u d es ill e g al str e et or pr es cri pti o n us e b y h o us e h ol d m e m b er a n d pr o bl e m ati c dri n ki n g or al c o h olis m of h o us e h ol d m e m b er * N ot e: T his t a bl e off ers a s y nt h esis of t h e est a blis h e d c at e g ori es of A C Es; s c or es r a n g e fr o m 01 2 . T hr e e c at e g ori es e xist a n d t h e s u b c at e g ori es ar e n ot a n e x h a usti v e list of t y p es of c hil d h o o d a d v ersit y ( B or a et al., 2 0 2 1; F elitti et al., 1 9 9 8; N ati o n al C e nt er f or I nj ur y Pr e v e nti o n a n d C o ntr ol, 2 0 2 2 ). 5 A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 6 A C Es e x p os ur e is e m er gi n g i n r es e ar c h wit h wi d es pr e a d i m pli c ati o ns f or pri m ar y c ar e pr o vi d e rs ( Al h o w a y m el et al., 2 0 2 1; B h us h a n et al., 2 0 2 0; H u g h es et al., 2 0 1 7). Al m ost t w e nt y y e ars p ost t h e i niti al w or k, Dr. F elitti r efl e cts t h at t h e pri m ar y c ar e m e di c al c o m m u nit y h as n ot a p pli e d t his r es e ar c h ( F elitti, 2 0 1 7). W hil e a d ult pri m ar y c ar e h as b e e n w aiti n g t o o bt ai n a str e a mli n e d r e c o m m e n d ati o n, s o ci al w elf ar e h as d e v el o p e d pr ot o c ols a n d pr o c e d ur es ( F elitti, 2 0 1 7; P ar d e e et al., 2 0 1 7; P ur k e y et al., 2 0 1 8). T h e ci vi c s p h er e, l a w e nf or c e m e nt, a n d t h e s o ci al s ci e n c es h a v e b e e n q ui c k t o a p pl y A C Es r es e ar c h ( F elitti, 2 0 1 7). Pri m ar y c ar e p e di atri cs h as als o g ai n e d a n a w ar e n ess of t h e i m p a ct of t his d at a ( Al b a e k et al., 2 0 1 8; A m a y a-J a c ks o n et al., 2 0 2 1; F elitti, 2 0 1 7; Ki n g et al., 2 0 1 9). H o w e v er, t h e a p pli c ati o n of A C Es t o a d ult pri m ar y c ar e h as n ot b e e n w ell r es e ar c h e d ( F elitti, 2 0 1 7; K al m a kis et al., 2 0 1 7; R o b erts et al., 2 0 1 9). T h e q u esti o n r e m ai ns: w h at pr e v e nts a d ult pri m ar y c ar e fr o m utilizi n g A C Es r es e ar c h i n pr a cti c e ? T h e b asis of i n q uir y of t his mi x e d- m et h o d D N P pr oj e ct is t w of ol d: 1) A n al y z e t h e d e v el o p m e nt of A C Es s cor es a n d t h e i m pli c ati o ns f or pri m ar y c ar e pr o vi d ers us e ; 2) e x pl or e t h e c urr e nt i m pl e m e nt ati o n of A C Es r es e ar c h i nt o a d ult pri m ar y c ar e. B ac k gr o u n d I n t h e l at e 1 9 9 0s, a l o n git u di n al r es e ar c h st u d y of h e alt h pr e v e nti o n a n d o b esit y q u a ntifi e d t h e d efi niti o n of pr e vi o us c hil d h o o d a d v ersit y ( F elitti et al., 1 9 9 8). Dr. F elitti, Dr. A n d a, a n d t h eir c oll e a g u es c oi n e d t h e t er m A C Es ( F elitti et al., 1 9 9 8). F elitti et al. ( 1 9 9 8) utili z e d a 1 0- c at e g or y t o ol w hi c h d es cri b e d v ar yi n g s o ur c es of tr a u m a. T h es e e v e nts ar e c at e g ori z e d r el ati n g t o a b us e, n e gl e ct, a n d f a mil y d ysf u n cti o n ( B or a et al., 2 0 2 1; F elitti et al., 1 9 9 8; Gil g off et al., 2 0 2 0; N ati o n al C e nt er f or I nj ur y Pr e v e nti o n a n d C o ntr ol, 2 0 2 2 ). T his st u d y ill ustr at e d t h e i m p a ct of a d v ersit y o n l o n g t er m h e alt h c o ns e q u e n c es ( F elitti et al., 1 9 9 8). Si n c e t his i niti al st u d y, t h e u n d erst a n di n g of t h e i m p a ct of A C Es h as e x p a n d e d ( B or a et al., 2 0 2 1; A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 7 F elitti et al., 1 9 9 8; Gil g off et al., 2 0 2 0; N ati o n al C e nt er f or I nj ur y Pr e v e nti o n a n d C o ntr ol, 2 0 2 2). T h e c h a n c es of s e v er e dis e as e, a d di cti o n, m or bi dit y, a n d e v e n m ort alit y i n cr e as e d wit h a n i n cr e as e of a n i n di vi du als A C Es s c or e ( B h us h a n et al., 2 0 2 0 ; F elitti et al., 1 9 9 8; H u g h es et al., 2 0 1 7; P etr u c c elli et al., 2 0 1 9). A d v e rs e c hil d h o o d e x p eri e n c es h a v e a n i m p a ct o n all- c a us e m ort alit y as r e v e al e d i n w o m e n a n d p e o pl e wit h di a b et es ( C a m p b ell et al., 2 0 1 9; C h e n et al., 2 0 1 6). O v er a 2 0 y e ar p e ri o d, a d ults wit h di a b et es a n d A C Es d e m o nstr at e d 2. 3 ti m es hi g h er m ort alit y r at e ( C a m p b ell et al., 2 0 1 9). Ki d n e y f u n cti o n a n d A C Es is ass o ci at e d wit h i n cr e as e d r at es of m ort alit y as w ell ( O z ei h et al., 2 0 2 0). A n i m p ort a nt f a ct or fr o m O zi e h et al. ( 2 0 2 0) is t h at d e cr e as e d r e n al f u n cti o n or s c ori n g a n A C Es al o n e w as n ot ass o ci at e d wit h m ort alit y. R at h er, t h e c o m or bi dit y of t h e t w o i n cre as e d t h e o v er all m ort alit y r at e ( O zi e h et al., 2 0 2 0). T h e ris k of b ei n g o v er w ei g ht or o b es e, c o ntr a cti n g c hr o ni c o bstr u cti v e p ul m o n ar y dis e as e ( C O P D), b ei n g a c urr e nt s u bst a n c e a b us er, a n d u n e m pl o y m e nt i n cr e as e d a m o n g a d ults wit h hi g h er A C Es e x p os ur e ( M erri c k et al., 2 0 1 9). A d diti o n al n e g ati v e o ut c o m es i n cl u d e di a b et es, h e art dis e as e, str o k e, m e nt al ill n ess, br o k e n b o n es a n d s e x u all y tr a ns mitt e d i nf e cti o ns ( S TIs) ( B or a et al., 2 0 2 1; J u o n al a et al., 2 0 1 9). T h es e f a ct ors i n cr e as e i n d e p e n d e nt of tr a diti o n al ris k f a ct ors s u c h as s m o ki n g ( P ur k e y et al., 2 0 1 8). I n a d diti o n, a d ol es c e nt mis us e of pr es cri pti o n dr u gs i n cr e as e d b y 6 2 % f or e a c h a d diti o n al A C Es ( F orst er et al., 2 0 1 7). T h e m al a d a pti v e c o pi n g m e c h a nis ms of a d ol es c e nts is attri b ut e d t o eff e cts fr o m A C Es i m p a ct ( P ar d e e et al., 2 0 1 7). H o w e v er, t h e pr ot e cti v e f a ct or f or t h es e a d ol es c e nts is a p ositi v e a d ult i nfl u e n c e ( e. g., t h e i nfl u e n c e of a t e a c h er; F orst er et al., 2 0 1 7). Wit h o ut a p ositi v e a d ult r el ati o ns hi p f or a c hil d t o b uil d r esili e n c e, A C Es e x p os ur e l e a ds t o to xi c str ess i n t h e b o d y ( Gil g off et al., 2 0 2 0). T o xi c str ess is d efi n e d as c hr o ni c d ysr e g ul ati o n A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 8 of t h e n e ur o e n d o cri n e a n d i m m u n e s yst e ms ( Gil g off et al., 2 0 2 0). T h e t y pi c al h u m a n br ai n r es p o ns e t o s ur vi v al is t h e pr o d u cti o n of str ess h or m o n es t o r e a ct t o lif e-t hr e at e ni n g sit u ati o ns ( P ar d e e et al., 2 0 1 7). T h e br ai n r es ets w h e n t h er e is n o ne e d f or fi g ht or fli g ht ( P ar d e e et al., 2 0 1 7). H o w e v er, t h e s ur vi v al br ai n i n a tr a u m ati z e d i n di vi d u al is h y p ers e nsiti v e a n d c o nti n u all y a cti v at e d ( P ar d e e et al., 2 0 1 7). T h e h y p ot h al a mi c- pit uit ar y a xis is t h e pri m ar y bi ol o gi c al pr o c ess es disr u pt e d a n d d ysr e g ul at e d i n a t o xi c r es p o ns e ( Gil g off et al., 2 0 2 0). T his r es ults i n a m al a d a pti v e r es p o ns e w hi c h al ar ms f or t y pi c all y b e ni g n a cti viti es ( P ar d e e et al., 2 0 1 7). T h es e l e v els of t o xi c str ess l e a d t o l o n g t er m a d v ers e si d e eff e cts wit h r el at e d d os e r es p o ns es s u c h as di a b e t es, r es pir at or y dis e as e, ris k t a ki n g, a n d vi ol e n c e ( Gil g off et al., 2 0 2 0; P etr u c c elli et al., 2 0 1 9). In r e c e nt y e ars, A C Es r es e ar c h h as e x p a n d e d wit h v ar yi n g vi e ws f or i m pli c ati o ns of pr a cti c e ( H u g h es et al., 2 0 1 7). A C Es s c or es h a v e b e e n str o n gl y c orr el at e d t o c hr o ni c ps y c h os o ci al a n d m e di c al dis e as e ( B o ers m a et al., 2 0 2 0; P etr u c c elli et al., 2 0 1 9 ). T h e m or e A C Es a n i n di vi d u al r e p orts, t h e hi g h er t h eir ris k of dis e as e ( B h us h a n et al., 2 0 2 0; F elitti et al., 1 9 9 8; H u g h es et al., 2 0 1 7). T his i n cl u d es b ot h c o m m u ni c a bl e a n d n o n- c o m m u ni c a bl e dis e as es s u ch as h e art dis e as e, c a n c er, r es pir at or y ill n ess, a n d h ar mf ul b e h a vi ors ( B h us h a n et al., 2 0 2 0 ; F elitti et al., 1 9 9 8; H u g h es et al., 2 0 1 7; P etr u c c elli et al., 2 0 1 9). Al z h ei m er s dis e as e or s ui ci d e a tt e m pts i n cr e as e d e x p o n e nti all y i n c orr el ati o n wit h A C Es ( B h us h a n et al., 2 0 2 0). Of U. S. a d ults 5 1. 8 % h a v e a c hr o ni c dis e as e ( B o ers m a et al., 2 0 2 0). I n t h os e a d ults 2 7. 2 % h a v e m or e t h a n o n e di a g n osis ( B o ers m a et al., 2 0 2 0). O n e i n si x pri m ar y c ar e p ati e nts utili z e h e alt h c ar e s er vi c es wit h a ris k f a ct or f or c hr o ni c dis e as e ( M erri c k et al., 2 0 1 9). T h e d efi ni n g c h ar a ct eristi c of c hr o ni c dis e a s e is a n ill n ess r e q uiri n g l o n g t er m s u p p ort, m ai nt e n a n c e, a n d tr e at m e nt ( R e y n ol ds et al., 2 0 1 8). T h e f o c us of a pri m ar y c ar e s etti n g is c o nti n uit y, c o or di n ati o n, a n d c o m pr e h e nsi v e c ar e A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 9 ( R ey n ol ds et al., 2 0 1 8). T his m a k es t h e pri m ar y c ar e s e ct or i d e al f or att e n di n g t o t h e i m p a ct of c hr o ni c dis e as e a n d A C Es. P r o bl e m D es pit e ri g or o us e vi d e n c e of t h e i m p a ct of A C Es s c or es, tr a nsl ati o n of t his r es e ar c h t o a d ult pri m ar y c ar e pr a cti c e is li mit e d ( K al m a kis et al., 2 0 1 7; P etr u c c elli et al., 2 0 1 9). Pri m ar y c ar e i niti ati v es ar e i d e al f or s cr e e ni n gs of p ati e nts f or c hil d h o o d a d v ersit y wit h i m p a ct o n h e alt h ( Gil g off et al., 2 0 2 0). H o w e v er, dis c o n n e cti o n e xists b et w e e n i n q uir y fr o m pri m ar y c ar e pr o vi d ers ( P C P) a n d p ati e nt p ers p e cti v es of t h e pr o vi d er r es p o nsi bilit y ( Gil g off et al., 2 0 2 0; K a l m a kis et al., 2 0 1 7). P ati e nts r e p ort it is wit hi n t h e r ol e of t h e P C P t o i n q uir e a n d a d dr ess pr e vi o us tr a u m a ( G ol dst ei n et al., 2 0 1 7). B ut m ulti pl e st u di es of P C Ps r e p ort a l a c k of c o nfi d e n c e t o pr o vi d e tr a u m a-i nf or m e d c ar e ( B or a et al., 2 0 2 1; K al m a kis et al., 2 0 1 7; Ti n k et al., 2 0 1 7; W ei nr e b et al., 2 0 1 0). D es pit e A C Es A w ar e c a m p ai g ns, c o ntr o v ers y s urr o u n ds i m pl e m e nt ati o n i nto pr a cti c e ( B h us h a n et al., 2 0 2 0; Fi n k el h or, 2 0 1 8). T h e c o ntr o v ers y is n ot o nl y h o w t o i mpl e m e nt t h e A C Es q u esti o n n air e b ut als o if t h es e r es e ar c h fi n di n gs s h o ul d b e g e n er ali z e d (D u b e, 2 0 1 8; Fi n k el h or, 2 0 1 8). F or e x a m pl e, criti cs cl ai m a t o ol s h o ul d n ot b e us e d if n o s p e cifi c i nt er v e nti o n is cl e arl y d efi n e d a n d r e c o m m e n d e d ( Fi n k el h or, 2 0 1 8; Gl o w a et al., 2 0 1 6). I n a d diti o n, e v e n if a n A C Es s c or e is est a blis h e d it m a y n ot c h a n g e t h e pl a n of c ar e ( Gl o w a et al., 2 0 1 6) . T h e s cr e e ni n g m e c h a nis ms r el at e d t o c hil d h o o d tr a u m a h a v e e v ol v e d. T h e ori gi n al A C Es t o ol w as d e v el o p e d f or r es e ar c h a n d w as n ot i nt e n d e d f or us e i n a pri m ar y c ar e cli ni c al pr a cti c e f or a n y p o p ul ati o n ( F elitti et al., 1 9 9 8; Fi n k el h or, 2 0 1 8; P ar d e e et al., 2 0 1 7). A n a d diti o n al A C Es s cr e e ni n g t o ol w as d e v el o p e d f or us e i n a d ol es c e nts wit h r e visi o ns t o t h e ori gi n al t o ol ( P ar d e e et al., 2 0 1 7). Si mil ar t o t h e s e mi n al st u d y, t h e a d a pt e d A C Es t o ol w as pri m aril y d esi g n e d f or A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 10 r es e ar c h p ur p os es ( P ar d e e et al., 2 0 1 7). O n e pr e vi o usl y g ol d st a n d ar d t o ol t o ass ess a d ol es c e nts h as n ot b e e n u p d at e d si n c e 1 9 9 8 a n d is n o l o n g er i n di c at e d ( P ar d e e et al., 2 0 1 7). T his a d ol es c e nt r es e ar c h t o ol e v ol v e d i nt o t h e c urr e nt cli ni c all y utili z e d s cr e e ni n g; T h e R a pi d A d ol es c e nt Pr e v e nti o n S cr e e ni n g ( R A A P S) ( P ar d e e et al., 2 0 1 7). A d diti o n al t o ols ass ess c hil dr e n a n d p ar e nts s u c h as S af e E n vir o n m e nt f or E v er y Ki d ( S E E K) or s o ci al n e e ds pr o gr a ms ( Fi n k el h or, 2 0 1 8; G ottl ei b et al., 2 0 1 6). Tr a u m a ass ess m e nt t o ols i n cl u d e P T S D r e a cti o n i n d e x, Tr a u m a S y m pt o m c h e c klist, Str u ct ur e d Tr a u m a- R el at e d E x p eri e n c es a n d S y m pt o ms S cr e e n er, a n d Bri g ht F ut ur es ( A m a y a-J a c ks o n et al., 2 0 2 1). H o w e v er, t h es e t o ols ar e pri m aril y dir e ct e d t o w ar ds p e di atri cs a n d a d ol es c e nts u p t o 2 4 y e ars of a g e ( B or a et al., 2 0 2 1; Fi n k el h or, 2 0 1 8; P ar d e e et al., 2 0 1 7). A d ult s cr e e ni n g t o ols pri m aril y f o c us o n t h e a p pli c ati o n of t h e ori gi n al A C E s r es e ar c h t o ol ( R ari d e n et al., 2 0 2 1). Pr o vi d ers e x pr ess h esit a n c y t o t h e a p pli c ati o n of t o ols wit h o ut s p e cifi e d i nt er v e nti o ns ( A m a y a-J a c ks o n et al., 2 0 2 1; B or a et al., 2 0 2 1; Fi n k el h or, 2 0 1 8). A n a d diti o n al c o n c er n e xists r e g ar di n g t h e p ossi bilit y of tri g g eri n g r el ati n g t o s cr e e ni n g ( P ur k e y et al., 2 0 1 8). Tri g g eri n g is e m oti o n al distr ess or a n xi et y eli cit e d b y sit u ati o ns r e mi nis c e nt of p ast tr a u m a ( P ur k e y et al., 2 0 1 8). A n yt hi n g fr o m p h ysi c al t o u c h t o f e eli n g dis miss e d c a n eli cit a tri g g er distr ess a n d is hi g hl y s p e cifi c t o t h e i n di vi d u al ( P ur k e y et al., 2 0 1 8). Si d e eff e cts of tri g g eri n g i n cl u d es i n a bilit y t o a d v o c at e f or s elf i n a h e alt h c ar e s etti n g , l a ck of p arti ci p ati o n i n c ar e, a n d l oss t o f oll o w u p ( P ur k e y et al., 2 0 1 8 ). T h e pr o bl e m r e m ai ns t h at d es pit e r es e ar c h t o i nf or m t h e e xist e n c e of A C Es i n pri m ar y c ar e, cl arit y is l a c ki n g f or h o w t o utili z e t his i nf or m ati o n i n tr e at m e nt of a d ults. T h e c urr e n t r e c o m m e n d ati o n is t o t ar g et A C Es c o m or bi diti es a n d utili z e est a blis h e d s cr e e ni n g t o ols f or d e pr essi o n, a n xi et y, P T S D, a n d ot h er c o m or bi diti es of tr a u m a i n w hi c h tr e at m e nts h a v e b e e n est a blis h e d ( A m a y a-J a c ks o n et al., 2 0 2 1; Fi n k el h or et al., 2 0 1 8). F elitti ur g es a r ef o c us of A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 11 pri m ar y c ar e fr o m a bi o m e di c al t o bi o ps y c h os o ci al a p pr o a c h ( F elitti, 2 0 1 7, p. 2 0 6). C urr e ntl y, K a is er P er m a n e nt e is c o n d u cti n g a st u d y c o nsisti n g of 1 3 0, 0 0 0 p ati e nts t o e x a mi n e t h e l ar g e s c al e i m pli c ati o ns of i nt e gr ati n g A C Es r es e ar c h i nt o pri m ar y c ar e ( F elitti, 2 0 1 7; F elitti, 2 0 1 9). N urs e pr a ctiti o n ers ar e a k e y c o m p o n e nt of t h e r e c e nt i nt e gr ati o ns i nt o pr a cti c e wit h K ais er P er m a n e nt es a p pli c ati o n of t h e ori gi n al A C Es r es e ar c h ( F elitti, 2 0 1 9). T h e c all is t o i n c or p or at e m ulti dis ci pli n ar y t e a ms t o a d dr ess g a ps i n h e alt h c ar e ( F elitti, 2 0 1 9). G a p A n al ysis A C Es i nf or m ati o n h as pri m aril y b e e n a p pli e d i n p e di atri c s etti n gs, utili zi n g a pr e v e nt ati v e fr a m e w or k ( S h erfi ns ki et al., 2 0 2 1; S zil a g yi et al., 2 0 1 6). H o w e v er, li mit ati o ns e xist d u e t o t h e i m p a ct t h at pri or e x p os ur e t o A C Es h as i n a d ult pri m ar y c ar e s etti n gs. T h us, t h e f o c us of t h e g a p a n al ysis is t o a d dr ess t h e r e c o m m e n d ati o ns of e vi d e n c e- b as e d A C Es e v al u ati o ns uti li z e d f or pri m ar y c ar e (s e e A p p e n di x A). I n a d diti o n, e d u c ati o n h as b e e n s u g g est e d as a str at e g y t o o v er c o m e t h e g a ps b et w e e n A C Es r es e ar c h fi n di n gs a n d a p pli c ati o n t o pr a cti c e ( B or a e t al., 2 0 2 1). T h er ef or e, a d diti o n al i nf or m ati o n fr o m t his pr oj e ct assists i n t h e r e c o m m e n d ati o ns f or f ut ur e e d u c ati o n al i niti ati v es t o m e et t h e g a p e xist e nt b et w e e n e vi d e n c e- b as e d c ar e a n d c urr e nt pr a cti c es i n t h e a d ult pri m ar y c ar e s etti n g. T his g a p w as i d e ntifi e d pri m aril y t hr o u g h dis c ussi o n wit h k e y st a k e h ol d ers wit hi n t h e pri m ar y c ar e or g a ni z ati o n. T h e g o al of t h e a d diti o n al a c a d e mi c sit e as a s u bs e q u e nt s etti n g w as ass ess m e nt of t h os e w h o tr ai n n urs e pr a ctiti o n ers a n d pr a cti c e pri m ar y c ar e. I n a d diti o n, t h e sit e w as i nt e n d e d t o i n cr e as e t h e p o w er of r es ults wit hi n t h e coll e cti o n of d at a. A Mi d w est er n a d ult a n d f a mil y pri m ar y c ar e cli ni c n et w or k a gr e e d t o c o ntri b ut e t o t his pr oj e ct r el at e d t o b arri ers of A C Es r es e ar c h i n t h e pri m ar y c ar e s etti n g. S W O T a n al ysis a n d g a p a n al ysis w as c o m pl et e d t o assist i n affir mi n g t h e i m p a ct of t h e pr oj e ct (s e e A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 12 A p p e n di x B; B o n n el & S mit h, 2 0 1 8). T h e pr o vi d ers s er vi ci n g a d ult pri m ar y c ar e i n t h e Mi d w est m a d e t his pr oj e ct a p pr o pri at e t o a d dr ess t h e g a p i n k n o wl e d g e. L it e r at u r e R e vi e w Li mit e d s uffi ci e nt e vi d e n c e e xists i n t h e lit er at ur e r e g ar di n g t h e b arri ers pr e v e nti n g f a mil y m e di ci n e cli ni ci a ns fro m i m pl e m e nti n g A C Es r es e ar c h ( B or a et al., 2 0 2 1; K al m a kis et al., 2 0 1 7 ). A n al ysis of f a ct ors c o ntri b uti n g t o h e alt h c ar e pr a ctiti o n ers n e gl e ct of A C Es s c or es i n t h eir pr a cti c e l e d t o t h e d e v el o p m e nt of t h e f oll o wi n g PI C O q u esti o n: H o w d o a d ult pri m ar y c ar e p r o vi d ers p er c ei v e t h eir k n o wl e d g e, tr ai ni n g, s cr e e ni n g, a n d i nt er v e nti o ns r e g ar di n g A C Es r es e ar c h i n t h eir pr a cti c e ? Se arc h S yst e m ati c s e ar c h es utili zi n g CI N A H L, C o c hr a n e, Ps y cI nf o, a n d P u b M E D w er e c o n d u ct e d fr o m S e pt e m b er 2 0 2 1 t hr o u g h N o v e m b er 2 0 2 2 (s e e A p p e n di x C). Utili z ati o n of t h e f oll o wi n g c at e g ori es of k e y w or ds pr o vi d e d r el e v a nt arti cl es: A d v ers e c hil d h o o d e x p eri e n c es, t o xi c str ess, pri m ar y c ar e pr o vi d ers, m oti v ati o ns, a d ult, o ut p ati e nt, a n d tr a u m a-i nf or m e d c ar e. Initi al s e ar c h es w er e n arr o w, t er ms t o o s p e cifi c, a n d eli cit e d f e w a p pli c a bl e r es ults. T h er ef or e, s e ar c h es br o a d e n e d a n d t h es e k e y w or ds l e d t o B o ol e a n s e ar c h es ( usi n g A N D a n d O R ) i n cl u di n g, b ut n ot li mit e d t o, v ari ati o ns of t h e t er ms: Tr a u m a, a d v ers e e v e nts, c hil d h o o d a b us e, p h ysi ci a ns, pr o vi d er, n urs e pr a ctiti o n ers, p h ysi ci a n assist a nts, k n o wl e d g e, s cr e e ni n g, l e gisl ati o n, pri m ar y c ar e, pr a cti c es, g ui d eli n es, i nt er v e nti o ns, tr ai ni n g, a n d e d u c ati o n. I n cl usi o n a n d e x cl usi o n crit eri a o c c urr e d pri m aril y aft er utili z ati o n of s e ar c h e n gi n e t o i n cl u d e s cr e e ni n gs of a d ult pri m ar y c ar e P C Ps. Als o, e x cl usi o n crit eri a w er e t o pi cs r el at e d t o p e di atri ci a n o nl y pr o vi d e rs, s o ci al w or k pr a ctiti o n ers, pr e g n a n c y/ pr e n at al c ar e, e m er g e n c y d e p art m e nt/ ur g e nt c ar e/i n p ati e nt h os pit al s etti n gs, a n d m e di c al or n urs e pr a ctiti o n er st u d e nts. T his i n cl u d e d li miti n g A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 13 p u bli c ati o n d at es b et w e e n 2 0 0 7- 2 0 2 2 wit h t h e e x c e pti o n of s e mi n al st u di es. T h e c h oi c e t o e x p a n d t h e s e ar c h l e n gt h t o 2 0 0 7 w as d u e t o t h e li mit e d r es e ar c h c o m pl et e d wit hi n a n arr o w er ti m eli n e. T h e li mit ati o n of t o pi c w as i nt e n d e d t o mi ni mi z e t h e e xtr a n e o us arti cl es n ot r el at e d t o t h e PI C O. Aft er d el eti n g d u pli c at es, e x cl u di n g b o o ks, a n d arti cl es n ot i n E n glis h, a t ot al of 4 6 0 r es ults s urf a c e d. T h e 4 6 0 arti cl es w er e f urt h er s cr e e n e d b y titl e or a bstr a ct t o i d e ntif y t h os e t h at i n cl u d ed p o p ul ati o ns s p e cifi c t o a d dr essi n g t h e a p pli c ati o n a n d p er c e pti o ns of A C Es r es e ar c h i n a d ult pri m ar y c ar e s etti n gs. A t ot al of 1 1 2 arti cl es w as f urt h er s cr e e n e d t o e v al u at e w hi c h of t h e a rti cl es a p pli e d t o t h e PI C O q u esti o n. T hr e e arti cl es a d h er e d t o t h e c o nfi n es of t h e pr oj e ct q u e sti o n (s e e A p p e n di x D; B or a et al., 2 0 2 1; K al m a kis et al., 2 0 1 7; W ei nr e b et al., 2 0 1 0 ). As t his lit er at ur e s e ar c h r es ult e d i n a l a c k of us a bl e p e er r e vi e w e d arti cl es, ot h er str at e gi es w er e i m pl e m e nt e d. In or d er t o e x p a n d i niti al fi n di n gs, a c o ns ul t ati o n wit h a m e di c al li br ari a n pr o vi d e d t h e r e c o m m e n d ati o n t o a p pl y in cl usi o n a n d e x cl usi o n crit eri a p ost s e ar c h e n gi n e. I n a d diti o n, a f urt h er lit er at ur e s e ar c h w as i m pl e m e nt e d utili zi n g a ut h ors, r ef er e n c e lists, a n d w or ks cit e d, t o eli cit a d diti o n al r es ults. T h e i niti al s e ar c h a n d s u bs e q u e nt m e as ur es pr o d u c e d t h e f oll o wi n g r el e v a nt thi rt e e n r es ults; a c as e s c e n ari o ( R a vi & Littl e, 2 0 1 7), a mi x e d-m et h o d st u d y ( K al m a kis et al., 2 0 1 7), t w o o pi ni o n arti cl es ( C a m p b ell, 2 0 2 0; Jo n e s et al., 2 02 0 ), a l ett er t o t h e e dit or (M c L e n n a n et al., 2 0 2 0 ), a f e asi bility st u d y ( Gl o w a et al., 2 0 1 6), t w o s c o pi n g r e vi e ws (F or d et al., 2 0 1 9 ; S h erfi ns ki et al., 2 0 2 1) , a n d fi v e cr oss -s e cti o n al s ur v e ys ( B o d e n d orf er et al., 2 0 2 0 ; B or a et al., 2 0 2 1; M a u n d er et al., 2 0 2 0; St or k et al., 2 0 2 0; W ei nr e b et al., 2 0 1 0). N o p ur e l y q u alit ati v e st u di es r es ult e d r e g ar di n g p er c e pti o ns of pri m ar y c ar e pr o vi d ers o n A C Es, a p pli e d i m p a ct of A C Es , or A C Es s cr e e ni n gs. In a d diti o n t o t h e mi x e d -m et h o d arti cl e, t h e i n cl usi o n of t h e o pi ni o n pi e c es, l ett er t o t h e e dit or, a n d t h e c as e st u d y w as i nt e n d e d t o pr o vi d e A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 14 s o m e q u alit ati v e b asis i n t h e lit er at ur e ( C a m p b ell, 2 0 2 0; J o n es et al., 2 0 2 0; K al m a kis et al., 2 0 1 7; M c L e n n a n et al., 2 0 2 0). R es ults Of t h e 6 arti cl es t h at s ur v e y e d pri m ar y c ar e pr o vi d ers, att e n di n g a n d r esi d e nt p h ysi ci a ns, p h ysi ci a n assist a nts, n urs e pr a ctiti o n ers, m e di c al st aff, ps y c hi atrists, p e di atri ci a ns, a n d pr o vi d ers of a d diti o n al s p e ci alti es w er e i n cl u d e d ( B o d e n d orf er et al., 2 0 2 0; B or a et al., 2 0 2 1; K al m a kis et al ., 2 0 1 7; Ma u n d er et al., 2 0 2 0; St or k et al., 2 0 2 0; W ei nr e b et al., 2 0 1 0). T h e c o m m o n d e n o mi n at or b et w e e n all of t h es e arti cl es w as t h e s ur v eill a n c e of f a mil y pr a cti c e pri m ar y c ar e ( B o d e n d orf er et al., 2 0 2 0; B or a et al., 2 0 2 1; K al m a kis et al., 2 0 1 7; M a u n d er et al., 2 0 2 0; St or k et a l., 2 0 2 0; W ei nr e b et al., 2 0 1 0). A u ni q u e f e at ur e of t h e arti cl e fr o m B o d e n d orf er et al. ( 2 0 2 0) w as t h e ass ess m e nt of t h e A C Es c o n v ers ati o n. W hil e n o a ct u al A C E s cr e e ni n g w as c o n d u ct e d fr o m t h e ori gi n al st u d y, a c o n v ers ati o n o c c urr e d b et w e e n pr o vi d ers a n d p ar e nts i n a pri m ar y c ar e offi c e ( B o d e n d orf er et a l., 2 0 2 0). O ut of 2 3 8 p ar e nts a n d g u ar di a ns, 9 7 % m ar k e d t h e pr ef er e n c e t o dis c uss t his t o pi c wit h t h eir pri m ar y c ar e pr o vi d ers ( B o d e n d orf er et al., 2 0 2 0). Fr o m t h e pr o vi d ers p e rs p e cti v e, a n ot h er f e at ur e of t h e st u d y w as t h e p er c e pti o n t h at p ar e nts f elt a c c us e d or off e n d e d b y t h e t o pi c ( Bo d e n d orf er et al., 2 0 2 0). B ut a str o n g c o n cl usi o n is t h at A C Es r es o ur c es f or p ar e nts ar e r eq uir e d i n pr a c tic e ( B o d e n d orf er et al., 2 0 2 0). A f e at ur e of t h e s ur v e y c o n d u ct e d b y B or a et al. ( 2 0 2 1) is t h e v ari a bilit y n ot e d b et w e e n s p e ci alti es p er c ei v e d b arri ers a n d k n o wl e d g e of A C Es w er e si g nifi c a ntl y diff er e nt b et w e e n f a mil y m e di ci n e pr a ctiti o n ers, p e di atri ci a ns, a n d i nt er n al m e di ci n e pr o vi d ers. Si mil arl y, t h e o nli n e s ur v e y fr o m M a u n d er et al. ( 2 0 2 0) d e m o nstr at e d a str o n g ass o ci ati o n b et w e e n pr o vi d er A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 15 s p e ci alt y a n d fr e q u e n c y of s cr e e ni n g. Ps y c hi atrists r o uti n el y s cr e e n e d, f a mil y m e di ci n e p h ysi ci a ns s cr e e n e d w h e n i n di c at e d, a n d ot h er s p e ci alti es r e p ort e d r ar e o c c asi o ns of s cr e e ni n g ( M au n d er et al., 2 0 2 0). K al m a kis et al. ( 2 0 1 7) ill ustr at e d t h at 3 4 % of n urs e pr a ctiti o n ers us u all y s cr e e n e d f or hist or y of c hil d a b us e. N urs e pr a ctiti o n ers w h o r e p ort e d a c o nfi d e n c e i n t he t as k h a d m u c h hi g h er r at es of s cr e e ni n g ( K al m a kis et al., 2 0 1 7). A d diti o n all y, f or e a c h fi v e y e ars of y e ars i n t h e fi el d, t h e o d ds of s cr e e ni n g i n cr e as e d b y 1 7 % ( K al m a kis et al., 2 0 1 7). F e at ur es of t h e el e ctr o ni c s ur v e y distri b ut e d b y St or k et al. ( 2 0 2 0) is t h e dir e ct e d s ur v e y of p h ysi ci a ns w or ki n g i n a r e gi o n wit h p o or h e alt h o ut c o m es. T h e e vi d e n c e pr o d u c e d b y t his arti cl e ill ustr at e d t h e l a c k of f a mili arit y wit h A C Es b y pr o vi d ers 8 0. 5 % h a d n o pri or e x p eri e n c e wit h A C Es ( St or k et al., 2 0 2 0). I n a d diti o n, t h e i n cr e as e d pr e v al e n c e of A C Es i n w o m e n w as s u p p ort e d ( St or k et al., 2 0 2 0). W ei nr e b et al. ( 2 0 1 0) w as t h e ori gi n al st u d y utili z e d t o ass ess p er c e pti o ns of a d ult pri m ar y c ar e pr o vi d ers r e g ar di n g A C Es i n pr a cti c e. T his i niti al st u d y ill ustr at e d t h e c o n c e pt t h at c o nfi d e n c e, g e n d er, a n d k n o wl e d g e i m p a ct e d t h e s cr e e ni n g h a bits ( W ei nr e b et al., 2 0 1 0). T his d at a c o ntri b ut e d t o t h e ar g u m e nt t h at e vi d e n c e- b as e d g ui d eli n es is n e e d e d wit hi n t h e a d ult pri m ar y c ar e s e ct or ( W ei nr e b et al., 2 0 1 0). T h e r e m ai ni n g lit er at ur e is b est ass ess e d b y r e vi e wi n g t h os e w h o pr o m ot e, a n d t h os e w h o c a uti o n, a g ai nst t h e pr e m at ur e i nt e gr ati o n of A C Es i nt o pri m ar y c ar e. P r o m oti o ns I n g e n er al, h e alt h c ar e pr o vi d ers r e q uir e a n a w ar e n ess of A C Es ( C a m p b ell, 2 0 2 0; M c L e n n a n et al., 2 0 2 0). H o w e v er, t h e d at a r e g ar di n g t h e k n o wl e d g e of pr o vi d ers a b o ut A C Es is A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 16 s ur prisi n gl y l o w ( B or a et al., 2 0 2 1; S h erfi n ksi et al., 2 0 2 1; St or k et al., 2 0 2 0). O nl y 2 0 % of pr o vi d e rs h a d a w ar e n ess A C Es e xist e d i n a n ar ticl e p u blis h e d o v er 2 0 y e ars aft er t h e i niti al A C Es st u d y ( St or k et al., 2 0 2 0). I n g e n er al, f e w P C Ps e x pr ess a hi g h l e v el of f a mili arit y wit h t h e t o pic ( B or a et al., 2 0 2 1; K al m a kis et al., 2 0 1 7; S h erfi n ksi et al., 2 0 2 1; St or k et al., 2 0 2 0; W ei nr e b et al., 2 0 1 0). M ulti pl e st u di es affir m t h e c orr el ati o n b et w e e n k n o wl e d g e of t h e i m p a ct t h at A C Es h a v e o n p h ysi c al ill n ess a n d s cr e e ni n g pr a cti c es a m o n g p h ysi ci a ns ( B or a et al., 2 0 2 1; M a u n d er et al., 2 0 2 0; W ei nr e b et al., 2 0 1 0). A d diti o n al f e at ur es of i n cr e as e d r at es of s cr e e ni n g ar e g e n d er, d e m o gr a p hi cs, c o nfi d e n c e i n s cr e e ni n g a bilit y, a n d k n o wl e d g e of s o m e o n e wit h a hist or y of c hil d a b us e ( B or a et al., 2 0 2 1; K al m a kis et al., 2 0 1 7; M a u n d er et al., 2 0 2 0; W ei nr e b et al., 2 0 1 0). H o w e v er, e d u c ati o n r el at e d t o A C Es is o n e of t h e m ost fr e q u e nt r e c o m m e n d ati o ns f or f ut ur e r es e ar c h a n d pr a cti c e i m pli c ati o ns ( B or a et al., 2 0 2 1; K al m a kis et al., 2 0 1 7; M a u n d er et al., 2 0 2 0; W ei nr e b et al., 2 0 1 0). B arri ers t o t h e i m pl e m e nt ati o n of A C Es w er e v ari e d. T h es e i n cl u d e d l a c k of ti m e, g a ps i n k n o wl e d g e, dis c o mf ort wit h s u bj e ct m at eri al or f e ar of r etr a u m ati zi n g p ati e nts, l a c k of a d e q u at e r ef err als or s u p p ort st aff, li mit e d r ei m b urs e m e nt, a n d l a c k of str e a mli n e d pr a cti c e r e c o m m e n d ati o ns ( B o d e n d orf er et al., 2 0 2 0; B or a et al., 2 0 2 1; C a m p b ell, 2 0 2 0; F or d et al., 2 0 1 9 ; Gl o w a et al., 2 0 1 6; J o n es et al., 2 0 2 0; K al m a kis et al., 2 0 1 7; M c L e n n a n et al., 2 0 2 0; R a vi & L ittl e, 2 0 1 7; S h erfi ns ki et al., 2 0 2 1; St or k et al., 2 0 2 0; W ei nr e b et al., 2 0 1 0). Pr o vi d er A C Es s cor e, f e eli n g h el pl ess, a n d ris k of off e n di n g w as als o a n ot e d b arri er i n s e v er al arti cl es ( C a m p b ell, 2 0 2 0; K al m a kis et al., 2 0 1 7; St or k et al., 2 0 2 0). M a n y of t h e arti cl es t h at s ur v e y e d t h e attit u d es of pr a ctiti o n ers di d s o i n c o nj u n cti o n wit h s cr e e ni n g ( B or a et al., 2 0 2 1; Gl o w a et al., 2 0 1 6; K al m a kis et al., 2 0 1 7; M a u n d er et al., 2 0 2 0; A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 17 W ei nr e b et al., 2 0 1 0). O n e of t h e first p e er r e vi e w e d p u bli c ati o ns e x a mi ni n g t h e s cr e e ni n g of A C Es s h o w e d t h e t o ol a d d e d 5 mi n ut es i n 9 0 % of p ati e nts wit h a n y ris k ( Gl o w a et al., 2 0 1 6). T his s p e a ks t o t h e c o n c er n of ti m e a s a n ot e d b arri er i n a d diti o n al arti cl es ( B or a et al., 2 0 2 1; W ei nr e b et al., 2 0 1 0). H o w e v er, s cr e e ni n g is t h e pr e cis e c o n c er n of t h e c a uti o n ar y. C a uti o ns C riti cs a gr e e t h at s cr e e ni n g s h o ul d b e c o n d u ct e d wit h c a uti o n ( C a m p bell, 2 0 2 0; M c L e n n a n et al., 2 0 2 0 ; S h erfi ns ki et al, 2 0 2 1). T h e A m eri c a n A c a d e m y of P e di atri cs is fr e q u e ntl y q u ot e d as h a vi n g n o s p e cifi c r e c o m m e n d ati o n f or s cr e e ni n g of A C Es ( C a m p b ell, 2 0 2 0; M c L e n n a n et al., 2 0 2 0 ). A n ur g e nt c all f or m or e e vi d e n c e, s yst e m ati c r e vi e ws, a n d a d diti o n al d at a o n o ut c o m es , ar e pr o m ot e d as t h e r e as o n t o d el a y s cr e e ni n g ( C a m p b ell, 2 0 2 0; F or d et al., 2 0 1 9 ; M c L e n n a n et a l., 2 0 2 0; R a vi & Littl e, 2 0 1 7). C alif or ni as p oli c y t o all o c at e milli o ns of f u n ds t o s cr e e ni n g pr a cti c es ar e vi e w e d wit h d erisi o n ( C a m p b ell, 2 0 2 0). H ar ms ar e list e d as off e ns e a n d s e nsiti viti es of t h e r es p o n d e nt, t h e a d v ers e eff e ct of l a b eli n g a p ati e nt f or a c hr o ni c c o n diti o n, a l a c k of e vi d e n c e, a n d a l a c k of a d esir e t o g o t h er e ( C a m p b ell, 2 0 2 0; F or d et al., 2 0 1 9 ; K a l m a kis et al., 2 0 1 7). T h e m ost c o m p elli n g q u alit ati v e d at a c o m es fr o m t h e mi x e d -m et h o d st u d y fr o m K al m a kis et al. ( 2 0 1 7). N urs e pr a ctiti o n ers p er c e pti o ns i n cl u d e d t h e b eli ef t h at s cr e e ni n g w as of b e n efit t o t h e p ati e nt, a r es p o nsi bilit y of t h e pr a ctiti o n er , a n d c a m e wit h pr o vi d er e x p eri e n c e ( K al m a kis et al., 2 0 1 7). E d u c ati o n w as a si g nifi c a nt c o n c er n f or t h e pr a ctiti o n ers ( K al m a kis et al. , 2 0 1 7). I n a d diti o n, a n urs e pr a ctiti o n ers c o n c e r n t o r es p o n d c orr e ctl y w as i n di c at e d as a si g nifi c a nt as p e ct of s cr e e ni n g a cli e nt wit h a p ers o n al hist or y of c hil d h o o d tr a u m a ( K al m a kis et al., 2 0 1 7). A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 18 D es pit e s u p p ort fr o m n urs e pr a ctiti o n ers f or t h e r es p o nsi bilit y t o s cr e e n, a m bi g uit y c o nti n u es t o e xist f or pr a cti c e g ui d eli n es ( C a m p b ell, 2 0 2 0; F or d et al., 2 0 1 9 ; K al m a kis et al., 2 0 1 7; M c L e n n a n et al., 2 0 2 0 ; R a vi & Littl e, 2 0 1 7). H o w e v er, a c o m m o n s ol uti o n n a m e d i n t h e lit er at ur e is tr a u m a-i nf or m e d c ar e (C a m p b ell, 2 0 2 0; F elitti, 1 9 9 8; J o n es et al., 2 0 2 0 ; K al m a kis et al., 2 0 1 7; R a vi & Littl e, 2 0 1 7; St or k et al., 2 0 2 0). T his c o n c e pt l e a ds t o t h e dis c ussi o n of t h e fr a m e w or k w hi c h h ol ds t h e f o u n d ati o n f or pr o visi o n of s e nsiti v e i m pl e m e nt ati o n of A C Es r es e ar c h i nt o a d ult pri m ar y c ar e. T h e o r eti c al F r a m e w o r k T h e t h e or y u n d er pi n ni n g t his pr oj e ct is i nt er pr eti v e et h n o gr a p h y ( M el n y k & Fi n e o utO v e r h olt, 2 0 1 9). T his is a h y bri d of t h e ori es d e v el o p e d fr o m b e h a vi or al, or g a ni z ati o n al, s yst e m, a n d s o ci al s ci e n c es. T h e c o n c e pt u al m o d el (s e e A p p e n di x E) utili z e d f or t his pr oj e ct is R o b erts a n d c oll e a g u es' m o d el f or tr a u m a-i nf or m e d pri m ar y c ar e ( TI P C) ( R o b erts et al., 2 0 1 9). Tr a u m ai nf or m e d c ar e i niti at e d i n t h e m e nt al h e alt h a n d j u v e nil e j usti c e ar e n as ( D u b e, 2 0 1 8; T h o m ps o nL ast a d et al., 2 0 1 7). R o b erts et al. ( 2 0 1 9) d e v el o p e d a m o d el a p pl yi n g t his c o n c e pt t o pri m ar y c ar e. T his m o d el g ui d e d t h e d e v el o p m e nt of t h e q u a ntit ati v e a n d q u alit ati v e di m e nsi o ns of t his pr oj e ct e. g. pr o vi d ers e x p eri e n c es, k n o wl e d g e, a n d us e of tr a u m a r el at e d r es e ar c h. All of t h e m e nti o n e d t h e ori es f or m e d t h e f o u n d ati o n of t his pr oj e ct. A n e e d e xists f or r e pr es e nt ati v es of pri m a r y c ar e t o a d dr ess t h e m oti v ati o ns of t h eir utili z ati o n of A C Es r es e ar c h. Wit h o ut a n u n d erst a n di n g of w h y P C Ps o p er at e as t h e y d o, a n y s u g g esti o ns f or i m pr o v e m e nt l a c k cl arit y a n d dir e cti o n ( M el n y k & Fi n e o ut- O v er h olt, 2 0 1 9). T h e TI P C m o d el g ui d e d t his pr oj e ct b y c asti n g a visi o n f or w h at pri m ar y c ar e s ho ul d r es e m bl e, t h us pr oj e cti n g p ot e nti al e xisti n g d efi ci e n ci e s t o w ar d a n i d e al fr a m e w or k t o i m pl e m e nt A C Es r es e ar c h i nt o t h e pri m ar y c ar e s etti n g ( R o b erts et al., 2 0 1 9). A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 19 Ai ms, O bj e cti v es, a n d H y p ot h es es T h e pri m ar y ai m w as t o d et er mi n e a d ult pri m ar y c ar e pr o vi d ers l e v el of f a mili arit y a n d pr o vi d e r utili z ati o n of A C Es r es e ar c h i n pr a cti c e. S e c o n d ar y ai ms w er e t h e ass ess m e nt of b a rri ers t o t h e a p pli c ati o n of A C Es r es e ar c h a n d p ot e nti al e d u c ati o n al t o ols f or P C Ps. T h e ai m of t h e dis c ussi o n w as t o e x pl or e t h e p ers p e cti v es of pri m ar y c ar e pr o vi d ers r e g ar di n g t h eir e x p eri e n c e, k n o wl e d g e, a n d utili z ati o n of A C Es r es e ar c h i n pri m ar y c ar e a d ult pr a cti c e. T h us, t h e f oll o wi n g o bj e cti v es w er e d e v el o p e d. O bj e cti v e 1: Ass ess k n o wl e d g e of A C Es s c or es vi a q u esti o n n air e a d a pt e d fr o m B or a et a l. 2 0 2 1 a n d K al m a kis et al. 2 0 1 7, (s e e A p p e n di x F) distri b ut e d t o p arti ci p a nts i n S e pt e m b er 2 0 2 2. O bj e cti v e 2: Ass ess tr ai ni n g v ari a n c es b et w e e n cli ni ci a n s p e ci alti es s p e cifi c t o A C Es s cor es vi a q u esti o n n air e a d a pt e d fr o m B or a et al., distri b ut e d t o p arti ci p a nts i n S e pt e m b er 2 0 2 2. O bj e cti v e 3: I n q uir e a b o ut s cr e e ni n g h a bits a n d b arri ers i n pr a cti c e t o i nt er v e nti o ns r el at e d t o A C Es s c or es vi a q u esti o n n air e a d a pt e d fr o m B or a et al. a n d K al m a kis et al., distri b ut e d to s a m e p arti ci p a nts S e pt e m b er 2 0 2 2. O bj e cti v e 4: I n q uir e a b o ut us e of pr a cti c e i nt er v e nti o ns t o a d dr ess A C Es s c or es vi a q u e sti o n n air es a d a pt e d fr o m B or a et al. a n d K al m a kis et al. distri b ut e d t o p arti ci p a nts i n S e pt e m b er 2 0 2 2. O bj e cti v e 5: E x pl or e t h e p ers p e cti v es of P C Ps r e g ar di n g utili z ati o n of A C Es r es e ar c h i n pr a cti c e vi a dis c ussi o n wit h r es p o n d e n ts fr o m q u a ntit ati v e q u esti o n n air e (s e e A p p e n di x G) c o m pl et e d i n D e c e m b er 2 0 2 2. O bj e cti v e 6 : Ass ess pr o vi d ers pr ef er e n c es f or a d diti o n al e d u c ati o n t o off er r e c o m m e n d ati o ns r e g ar di n g f ut ur e r es e ar c h. A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 20 T h e o bj e cti v es of t his pr oj e ct w er e e v al u at e d q u a ntit ati v el y vi a t h e f oll o wi n g h y p ot h es es: P ri m ar y c ar e pr o vi d ers p er c ei v e t h eir k n o wl e d g e of A C Es as a d e q u at e. T h e diff er e n c es b et w e e n n urs e p r a ctiti o n ers, p h ysi ci a n assist a nts, p h ysi ci a ns, a n d ot h er P C P s ar e r efl e ct e d i n t h e p arti ci p a nts s cr e e ni n g pr a cti c es. T h e d e m o gr a p hi c diff er e n c es of p arti ci p a nts ar e r efl e ct e d i n t h e p arti ci p a nts s cr e e ni n g pr a cti c es. N ur s e p r a ctiti o n ers a n d p h ysi ci a n s diff er i n tr ai ni n g r el at e d t o A C Es. S c r e e ni n g pr a cti c es diff er b et w e e n n urs e p r a ctiti o n ers a n d p h ysi ci a ns. P ri m ar y c ar e pr o vi d ers mi ni m all y utili z e A C Es r es e ar c h i n t h eir pr a cti c e. Pri m ar y c ar e pr o vi d ers p er c ei v e b arri ers e xist t o t h e utili z ati o n of s cr e e ni n g i n pri m ar y c ar e. T hi s mi x e d m et h o d pr oj e ct s o u g ht t o o bt ai n t h es e o bj e cti v es vi a t h e f oll o wi n g d esi g n. M et h o ds T h e g o al of t his pr oj e ct w as t o a ns w er t h e PI C O q u esti o n utili zi n g a s e q u e nti al, mi x e d m et h o ds, q u a ntit ati v e d esi g n wit h t h e i n cl usi o n of dis c ussi o n wit h pr o vi d ers. S a m pli n g w as a m ul ti dis ci pli n ar y o nli n e q u esti o n n air e of p h ysi ci a ns, n urs e pr a ctiti o n ers, p h ysi ci a n assist a nts, a n d pr a ctiti o n ers pr o vi di n g pri m ar y c ar e s er vi c es. T h e p o p ul ati o n w as a c o n v e ni e n c e s el e cti o n of a n o ut p ati e nt c li ni c e m pl o yi n g pri m ar y c ar e pr o vi d ers ( P C Ps) a n d a u ni v ersit y d e p art m e nt of a d v a n c e d pr a cti c e n urs es i n t h e Mi d w est er n r e gi o n of t h e U. S. (s e e A p p e n di x A) wit h t h e o pti o n of i n di vi d u al r ef err als t o ai d i n distri b uti o n. D at a w as a n al y z e d utili zi n g a c hi-s q u ar e t est, m e a n a n d m e di a n r es ults, a n d i nf er e nti al st atisti cs t o t est pr oj e ct h y p ot h es es. A c hi-s q u ar e t est is a st atisti c al t est us e d t o d et er mi n e w h et h er t h er e is a r el ati o ns hi p b e t w e e n t w o c at e g ori c al v ari a bl es ( S c h o b er & V ett er, 2 0 1 9). T h e g o al of t his t est is t o i d e ntif y A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 21 w h et h er a dis p arit y b et w e e n a ct u al a n d pr e di ct e d d at a is d u e t o c h a n c e. O r t o id e ntif y if a li n k b et w e e n t h e v ari a bl es is u n d er c o nsi d er ati o n. T h e n ull h y p ot h esis is t h at t h er e is n o r el ati o ns hi p b et w e e n t h e c at e g ori c al v ari a bl es i n t h e p o p ul ati o n - m e a ni n g t h e v ari a bl es ar e i n d e p e n d e nt. If t h e res ulti n g p- v al u e is l ess t h a n 0. 0 5, t h e n ull h y p ot h esis c a n b e r ej e ct e d. C hi-s q u ar e t esti n g w as pri m a ril y us e d t o c o m p ar e t h e diff er e n c es b et w e e n pr o vi d er r ol es i n r e g ar ds t o p arti ci p a nt r es p o ns es. P r oj e ct Sit e a n d P o p ul ati o n T h e pr oj e ct sit e o ri gi n at ed i n 2 l o c ati o ns. T h e p o p ul ati o n c o nsist e d pri m aril y of 2 1 pri m a r y c ar e pr o vi d ers a n d 9 a d v a n c e d pr a cti c e n urs es i n a n a c a d e mi c s etti n g t o i nf or m p e rs p e cti v es of k n o wl e d g e a n d b arri ers i n pri m ar y c ar e s etti n gs. T hr e e a d diti o n al pr a ctiti o n ers w er e r e cr uit e d a n d t h e q u esti o n n air e w as f or w ar d e d t o t h e m vi a e m ail. T h e first sit e is a criti c al a c c ess h os pit al wit h f i v e o ut p ati e nt cli ni cs e m pl o yi n g m or e t h a n 2 0 pri m ar y c ar e pr o vi d ers. T his sit e s er vi c es r ur al c o m m u niti es i n t h e Mi d w est. Int er n a l m e di ci n e a n d f a mil y pr a cti c e p r o vi d ers w er e s a m ple d. T h e q u esti o n n air e w as dist ri b ut e d t o p h ysi ci a ns, n urs e pr a ctiti o n ers, an d p h ysi ci a n a ssist a nts. T h e a d diti o n al sit e is a pri v at e li b er al arts u n i v ersit y i n t h e Mi d w est c o m p os e d of a d v a n c e d pr a cti c e n urs es i n a n a c a d e mi c s etti n g t o i nf or m a d diti o n al p ers p e cti v es a n d off er a d diti o n al p o w er t o q u esti o n n air e r es ults. Inf or m ati o n r e g ar di n g l o c ati o n of pr a cti c e a n d d e m o gr a p hi cs of p arti ci p a nts w as e m b e d d e d i nt o t h e q u esti o n n air e t o a c c o u nt f or a d diti o n al p arti ci p a nts w h o c o ntri b ut e d a n d w er e n ot a p art of t h es e 2 sit es. D at a C oll e cti o n P r o c e d u r e A Q u altri cs ( o nli n e s ur v e y t o ol) q u esti o n n air e w as a d mi nist er e d t o p arti ci p a nts t o o bt ai n t h e q u a ntit ati v e m e as ur es (s e e A p p e n di x F). T h e pr oj e ct i nf or m ati o n a n d q u esti o n n air e li n k w er e distri b ut e d vi a e m ail. T h e o pti o n t o f or w ar d w as i n cl u d e d wit hi n t h e e m ail f or t h e A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 22 p ot e nti al t o g ai n a d diti o n al p arti ci p a nts. N o w a y of tr a c ki n g e m ails f or w ar d e d w as a v ail a bl e t o m o ni t or a d diti o n al distri b uti o ns of t h e q u esti o n n air e. T h e q u esti o n n air e w as als o distri b ut e d t o 9 a d v a n c e d pr a cti c e n urs e pr o vi d ers at a n a c a d e mi c i nstit uti o n i n t h e Mi d w est vi a e m ail. T h es e pr a ctiti o n ers, t h e i n di vi d u als e m ail e d, as w ell as t h e pri v at e pri m ar y c ar e cli ni c, c o nsist e d of a t otal of 3 3 k n o w n pr o vi d ers of v ari o us dis ci pli n es w h o w er e s ur v e y e d (s e e att a c h e d q u e sti o n n air e e x e m pl ar i n A p p e n di x F). Aft er t h e 6 w e e ks of d at a c oll e cti o n w as c o m pl et e, all r es ults w er e a n al y z e d a c c or di n g t o t h e af or e m e nti o n e d m et h o ds. T h e q u esti o n n air e c o nt ai ne d a n o pti o n f or p arti ci p a nts t o pr o vi d e c o nt a ct i nf or m ati o n t o v ol u nt e er f or t h e q u alit ati v e s e cti o n b y p h o n e. T h e q u alit ati v e m e as ur e w as a s e mi-str u ct ur e d, s cri pt e d, p h o n e i nt er vi e w of t h e 2 r es p o n d e nts w h o v ol u nt e er e d. O nl y o n e r es p o n d e nt w as a bl e t o c o m pl et e t h e i nt er vi e w. T his i nt er vi e w w as r e c or d e d, t h e pr o c ess of tr a ns cri pti o n i niti at e d b y t h e h e a d i n v esti g at or, b ut d u e t o c o nstr ai nts of ti m e a n d r es o ur c es, f ull a n al ysis w as n ot c o m pl et e d. N o s p e cifi c i d e ntifi ers w er e c oll e ct e d o utsi d e of t h e n a m e a n d p h o n e or e m ail c o nt a ct i nf or m ati o n f or t h e q u alit ati v e d at a. T h e diss e mi n ati o n of fi n di n gs w as distri b ut e d as a c o m pl et e m a n us c ri pt a n d s h ar e d wit h k e y st a k e h ol d ers. Str e n gt hs a n d w e a k n ess es of q u esti o n n air e w er e e v al u at e d i n t h e a n al ysis fr o m A p p e n di x B. M e as u r e m e nt I nst r u m e nt T h e q u a ntit ati v e m e as ur e m e nt t o ols i n cl u d ed a q u esti o n n air e d e v el o p e d b y B or a et al. ( 2 0 2 1) a n d K al m a kis et al. ( 2 0 1 7). K al m a kis et al. w a s utili z e d o nl y as a r ef er e n c e f or q u esti o n n air e str u ct ur e. T h e p er missi o n s a n d b ul k of t h e q u esti o n n air e w as i nf or m e d b y B or a et al. ( 2 0 2 1). T his t o ol w as a d a pt e d fr o m c urr e nt lit er at ur e o n t h e t o pi c ( B or a et al., 2 0 2 1). I n a d diti o n, t h e l e n gt h an d pr e vi o us us e c o ntri b ut e d t o t h e v ali dit y of t h e d at a ( B or a et al., 2 0 2 1). T h e q u esti o n n air e utili z e d m ulti pl e q u esti o n str u ct ur es s u c h as a 3 a n d 4 -p o i nt Li k ert s c al e, A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 23 di c h ot o m o us v ari a bl es, s el e ct all t h at a p pl y , a n d m ulti pl e off ers t o i n cl u d e c h oi c es n ot s p e cifi e d. C h a n g es of t h e q u esti o n n air e i n v ol v e d t h e a d diti o n of s e v er al d e m o gr a p hi c q u esti o ns i nf or m e d b y K al m a kis et al. ( 2 0 1 7) a n d a dj ust m e nt of s o m e of t h e l a n g u a g e f or cl arit y a n d n arr o w e d s c o p e f or t h e p ur p os es of t his pr oj e ct. T his w as d u e t o pref er e n c e of t h e i n v esti g at o r a n d a d esir e t o k e e p t h e q u esti o n n air e s h ort t o pr o m ot e p arti ci p ati o n. T h e q u esti o n n air e utili z e d w as h e a vil y r eli a nt o n t h e B or a et al. ( 2 0 2 1) a n d K al m a kis et a l. ( 2 0 1 7) f or s e v er al r e as o ns. T h e first w as t h at t h e a c c e pt a bl e t o ols i n t h e r es e ar c h pri m aril y f o c us o n p e di atri ci a ns or r e gist er e d n urs es ( Ki n g et al., 2 0 1 9; St o k es et al., 2 0 1 7). Si n c e t h e f o c us of t his pr oj e c t w as t o i nf or m a d ult pri m ar y c ar e pr o vi d ers, t h e a p pli c a bilit y of p e di atri ci a ns a n d r egist er e d n urs es l a c k e d pr oj e ct c o m p ati bilit y. Si mil arl y, a n ot h er t o ol s ur v e y ass ess e d n ursi n g a n d f a mil y n urs e pr a ctiti o n er st u d e nts o nl y ( Z h a n et al., 2 0 2 1). W h e n B a k er et al. ( 2 0 2 1) st u di e d T h e Attit u d es R el at e d t o Tr a u m a-I nf or m e d C ar e S c al e ( A R TI C), t his t o ol l a c k e d t h e s p e cifi cit y of a ss essi n g attit u d es of pri m ar y c ar e pr o vi d ers al o n e. T h e us e of A R TI C i n t h at w or k ass ess e d e d u c at ors a n d h u m a n s er vi c e pr o vi d ers i n a d diti o n t o h e alt h c ar e pr o vi d ers ( B a k er et al., 2 0 2 1). T his l e d t h e i n v esti g at or t o dis c ar d t h e A R TI C t o ol r el at e d t o l a c k of s p e cifi cit y t o t h e a d ult pri m ar y c ar e pr o vi d er. T h es e f a ct ors, t h e a p pr o pri at e n ess of t h e s c al e t o a d ult pri m ar y c ar e, as w ell as t h e r es p o nsi v e n ess of t h e r es e ar c h ers w h o p er mitt e d us e of t h e s ur v e y, i nf or m e d t h e c h oi c e of t his q u esti o n n air e f or t h e pr oj e ct. T h e q u alit ati v e q u esti o ns w er e d e v el o p e d b y t h e pr oj e ct i n v esti g at or . D e p e n d a bilit y w as affir m e d b y r e pr o d u ci n g t h e 8 q u esti o n s cri pt e a c h ti m e t h e i nt er vi e w w as c o n d u ct e d b y t h e h e a d i n v esti g at or. T h e res p o n d e nt w as gi v e n t h e o pti o n t o e x p a n d o n t h e t o pi c or off er a d diti o n al c o m m e nts b y t h e fi n al q u esti o n of t h e i nt er vi e w (s e e A p p e n di x G) . Pri or t o t h e c oll e cti o n of t h e i nter vi e w d at a t h e i n v esti g at or i d e ntifi e d 3 c at e g ori es of bi as. T h e bi as w as t h at 1) r es p o n d e nts A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 24 w o ul d e x pr ess a m o d er at e k n o wl e d g e of A C Es, 2) it w o ul d b e e as y f or pr o vi d ers t o off er st ori es of p a ti e nts w h o h a v e b e e n i m p a ct e d b y A C Es, a n d 3) e x p eri e n c es w o ul d i n cl u d e fr ustr ati o n wit h b a rri ers i n r ef err al of p ati e nts wit h hi g h A C Es s c or es. T h es e bi as es w er e d et er mi n e d i n c o nj u n cti o n wit h t h e k n o w n q u a ntit ati v e d at a fr o m t h e lit er at ur e ( B o d e n d orf er et al., 2 0 2 0; B or a e t al., 2 0 2 1; C a m p b ell, 2 0 2 0; F or d et al., 2 0 1 9; Gl o w a et al., 2 0 1 6; J o n es et al., 2 0 2 0; K al m a kis e t al., 2 0 1 7; M c L e n n a n et al., 2 0 2 0; R a vi & Littl e, 2 0 1 7; S h erfi ns ki et al., 2 0 2 1; St or k et al., 2 0 2 0; W ei nr e b et al., 2 0 1 0). T h es e i n cl u d e d p ers p e cti v es si mil ar t o i d e ntifi e d d at a r el at e d t o l a c k of ti m e, diffi c ult y i n r ef err als, a m bi g uit y of i nt er v e nti o ns, a n d n e e d f or m or e e d u c ati o n. Si n c e t h e it e ms w er e dir e ct e d at pr o visi o n of c ar e a n d n ot dir e ct e d t o w ar d A C Es e x p eri e n c es of t h e r es p o n d e nt, n o c o n c er ns f or tri g g eri n g w as e x p e ct e d. N o n e e d ar os e f or c o u ns eli n g r es o ur c es t o b e off er e d h o w e v er, t his w as a c o nsi d er ati o n of t h e pr oj e ct. Et hi c al C o nsi d e r ati o ns Et hi cs is a n i nt e gr al c o m p o n e nt of a n y r es e ar c h i niti ati v e ( M el n y k & Fi n e o ut -O v er h olt, 2 0 1 9). I nt er n al R e vi e w B o ar d (I R B) a p pr o v al w as o bt ai n e d pri or t o i niti ati n g t his D N P pr oj e ct t o a d dr ess a n y p ot e nti al et hi c al c o nfli cts. A n i m p ort a nt c o n c er n i n r es e ar c h is c o ns e nt ( M el n y k & Fi n e o ut -O v er h olt, 2 0 1 9). T his w a s pr o vi d e d f or b y a ss e nt wit h dis cl ai m er s p e cifi e d at t h e o p e ni n g of t h e q u esti o n n air e . Si nc e t h e q u alit ati v e d at a w as pr o vi d e d v ol u nt aril y, c o ns e nt w as b uilt i nt o t h e q u esti o n n air e b ef or e off eri n g t h e o pti o n t o pr o vi d e i d e ntif yi n g i nf or m ati o n. I n a d diti o n, a v er b al c o ns e nt w as o bt ai n e d pri or t o r e c o r di n g of t h e i nt er vi e w. A d diti o n al c o nsi d er ati o ns w er e b e n efi c e n c e a n d n o n m al efi c e n c e ( M el n y k & Fi n e o ut -O v er h olt, 2 0 1 9). N o ris ks w er e ass o ci at e d wit h t h e q u esti o n n air e o utsi d e of t h e p ot e nti al f or s u bj e ct m at eri al t o p ot e nti all y c a us e ps y c h ol o gi c al dis c o mf ort f or p arti ci p a nts w h o m a y h a v e a hist or y of a d v ers e c hil d h o o d e x p eri e n c es. H o w e v er, t his w a s pr o vi d e d f or b y t h e p arti ci p a nts a bilit y t o wit h dr a w at A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 25 a n y p oi nt i n ti m e wit h l ess t h a n mi ni m al ris k. I n a d diti o n, t h e a bilit y t o wit h dr a w w as s p e cifi c all y dis cl os e d at t h e o ns et of t h e q u esti o n n air e . A d diti o n al v er b al c o ns e nt w as pr o vi d e d u p o n i niti ati o n of t he i nt ervi e w wit h a n otifi c ati o n f or h o w t h e r e c or d e d d at a a n d r es p o n d e nt i nf or m ati o n w o ul d b e utili z e d . Wit h a n y t y p e of d at a c oll e cti o n , pri v a c y a n d c o nfi d e nti alit y is a si g nifi c a nt c o nsi d er ati o n ( M el n y k & Fi n e o ut-O v er h olt, 2 0 1 9). T h e first pr o visi o n f or c o nfi d e nti alit y w a s b uilt i nt o t h e q u esti o n n air e . F or t h e g e n er al q u a ntit ati v e d at a , t h e s ur v e y t o ol a n o n y mi z e d t h e d at a ( e. g. n o n a m es or i d e ntif y i nf or m ati o n w er e s p e cifi c all y st or e d). O b vi o usl y, t his w as i m p a ct e d b y t h e r es p o n d e nts w h o off er e d t h eir i nf or m ati o n f or t h e q u alit ati v e i nt er vi e w d at a c oll e cti o n. I n or d er t o tr a c k d at a att a c h e d t o q u alit ati v e i nf or m ati o n, e a c h q u esti o n n air e w as li n k e d t o t h e l ast 3 di gits of e a c h r es p o n d e nts p h o n e n u m b er. I n a d diti o n, th e i nt er vi e w d at a g at h er e d w as st or e d o n a p ass w or d pr ot e ct e d c o m p ut er. T h e o nl y i n di vi d u al wit h a c c ess t o t h e d at a o utsi d e of t h e pri n ci p al i n v esti g at or w as th e q u a ntit ati v e st atisti ci a n. H o w e v er, t h e i d e ntifi ers w er e r e m o v e d a n d t h e q u alit ati v e i nf or m ati o n tr a nsf err e d t o t e xt pri or t o a n y t y p e of a c c ess . T h e g at h er e d i nf or m ati o n will n ot b e st or e d a n y l o n g er t h a n 1 2 m o nt hs fr o m i niti al d at e of s u b missi o n. In a d diti o n, t h e pr oj e ct i n v esti g at or c o m pl et e d tr ai ni n g fr o m t h e C oll a b or ati v e I nstit uti o n al Tr ai ni n g I niti ati v e ( CI TI pr o gr a m) t o affir m et hi c al i niti ati v es of t h e pr oj e ct. D at a A n al ysis a n d R es ults T h e pr oj e ct e v al u ati o n pl a n w a s m ultif a c et e d. First, t h e q u a ntit ati v e d at a w as a n al y z e d wit h d es cri pt i v e st atisti cs usi n g Mi cr os oft E x c el t o r e vi e w t h e c h ar a ct eristi cs of t h e q u esti o n n air e p arti ci p a nts. T h e q u a ntit ati v e d at a w as a n al y z e d utili zi n g a c hi -s q u ar e t est. T h e Li k ert s c al e d at a w as a n al y z e d utili zi n g t h e m e a n a n d m e di a n r es ults. I nf er e nti al st atisti cs w er e als o utili z e d t o t est t h e af or e m e nti o n e d h y p ot h es es . De s cri pti v e st atisti cs w er e utili z e d t o A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 26 d es cri b e t h e p o p ul ati o n a n d d at a vi a gr a p hs a n d t a bl es . S e c o n d, t h e q u alit ati v e i nt er vi e w d at a w as r e c or d e d a n d t h e tr a ns cri pti o n pr o c ess i niti at e d. Bi as w as i d e ntifi e d pri or t o c o di n g b ut t h e pr o c ess n e v er r e a c h e d c o m pl eti o n t o w arr a nt r e vi e w b y a n i n d e p e n d e nt a u dit or. T h er ef or e, t h e q u alit ati v e c o m p o n e nt w a s i n cl u d e d o nl y i n dis c ussi o n of fi n di n gs as it di d n ot pr o vi d e tr ust w ort h y r es ults. R es ults Of t h e 3 3 v erifi e d q u esti o n n air e s s e nt, 1 2 w er e c o m pl et e d wit h p arti al c o m pl eti o n of 2 f or a r es p o ns e r at e of 3 6. 3 %. T h e m aj orit y of t h e r es p o n d e nts w er e f e mal e 7 7. 8 % (s e e T a bl e 2 ). O nl y 2 of t h e r es p o n d e nts i d e ntifi e d as m al e. O n e m al e a n d o n e f e m al e pr o vi d er v ol u nt e er e d t o i nt er vi e w. T h e a g e of t h e s a m pl e p o p ul ati o n r a n g e d as dis pl a y e d i n T a bl e 3 wit h f airl y e v e n distri b uti o n of a g es 3 0 a n d a b o v e. T a bl e 2 T a bl e 3 G e n d er o f Q u esti o n n air e P arti ci p ants A g e of Q u e sti o n n air e P arti ci p a nts T h e m aj orit y of r es p o n d e nts pr a cti c e d i n r ur al or s m all t o w n l o c ati o ns a n d ur b a n pr a cti c e , r e pr es e nt e d at 7 7. 8 % a n d 2 2. 2 % r es p e cti v el y. P h ysi ci a ns a n d n urs e pr a ctiti o n ers w er e t h e o nl y r ol es w h o r e pli e d t o t h e q u esti o n n air e wit h al m ost e v e n s plit 5 5. 6 % p h ysi ci a ns a n d 4 4. 4 % n urs e pr a ctiti o n ers r e pli e d. A v ari et y of s p e ci alti es w er e r e pr es e nt e d fr o m f a mil y m e di ci n e, i nt er n al m e di ci n e, g er o nt ol o g y, a n d p e di atri cs (s e e T a bl e 4). A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 27 T a bl e 4 S p e ci alti es of Q u esti o n n air e P arti ci p a nts K n o wl e d g e T h e m aj orit y of p arti ci p a nts e x pr ess e d a f a mili arit y wit h A C Es (s e e T a bl e 5). H o w e v er, th e k n o wl e d g e b as e of pri m ar y c ar e pr o vi d ers a p p e ar e d q u a ntifi a bl y i n a d e q u at e. T a b le 5 Q u esti o n n air e P arti ci p a nts F a mili arit y wit h A C Es T a bl es 6 a n d 7 ill ustr at e t h e us e of c hi-s q u ar e i n m a ni p ul ati o n of d at a. Dat a dis pl a y e d is t h e o bs e r v e d a n d e x p e ct e d r es ults f or h o w n urs e pr a ctiti o n ers a n d p h ysi ci a ns r es p o n d e d t o q u esti o n 2 o n tr ai ni n g. T h e n ull h y p ot h esis is t h at t h e r esp o ns es ar e i n d e p e n d e nt o f on e a n ot h er. Ass u mi n g n o diff er e n c e b et w e e n t he t w o pr a cti c e ar e as, t h e e x p e ct ati o n is f or t h e p arti ci p a nts t o b e A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 28 pr o p orti o n at el y distri b ut e d . B e c a us e 8 i n di vi d u als r es p o n d e d N o a n d 4 of t h e 9 p arti ci p a n ts w er e n urs e p r a ctiti o n ers, t h e n 4/ 9 of t h e 8 is e x p e ct e d t o b e a r es p o ns e fr o m a n urs e pr a ctiti o n er. T a bl e 6 T a bl e 7 C hi-s q u ar e: O bs er v e d R es ults of T r ai ni ng C hi-s q u ar e: E x p e ct e d R es ults of Tr ai ni n g T h us, 4/ 9 x 8 = 3. 5 6 r o u n d e d t o 2 d e ci m als is t h e r es ult. Als o, 5/ 9 x 8 = 4. 4 4 is t h e e x p e ct e d r es ult f or p h ysici a ns. Si mil arl y, b e c a us e 1 p arti ci p a nt r es p o n d e d Y es , 4/ 9 is e x p e ct e d t o b e n urs e pr a ctiti o n ers a n d 5/ 9 p h ysi ci a ns. H e n c e, 0. 4 4 a n d 0. 5 6 ar e t h e r es ults r es p e cti v el y. Wit h a rri v al at t h e e x p e ct e d r es ult s, Mi cr os oft E x c el h a s a f u n cti o n C H I T E S T w hi c h r et ur ns t h e p v a l u e. F or t h e a b o v e o bs er v e d a n d e x p e ct e d v al u es , t h e p -v a l u e is 0. 3 6 r o u n d e d t o 2 d e ci m al pl a c es. B e c a us e it is gr e at er t h a n 0. 0 5 t h e n ull h y p ot h esis c a n n ot b e r ej e ct e d . P h ysi ci a ns e x pr e ss e d m or e k n o wl e d g e o n t h e i m p a ct of c hil d h o o d tr a u m a o n a d ult w ell b ei n g wit h 6 0 % r e p orti n g s o m e w h at f a mili ar a n d 4 0 % r e p orti n g v a g u el y f a mili ar wit h A C Es r es e ar c h. N urs e pr a ctiti o n ers c o n v ers el y r e p ort e d s o m e w h at f a mili ar at 2 5 % a n d 7 5 % r e p ort e d b ei n g v a g u el y f a mili ar wit h A C Es. N o o n e d es cri b e d t h e ms el v es as b ei n g v er y f a mili ar wit h t h e A C E s st u d y. T h e a n al ysis of t h e a d diti o n al c o m p o n e nts of t h e Li k ert s c al e t o i d e ntif y c o m p o n e nts of k n o wl e d g e b as e a n d s u bs e q u e nt pr a cti c es pr o vi d e d littl e v ari a n c e. A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 29 I m pl e m e nt ati o n T h e r es ults r e g ar di n g i m pl e m e nt ati o n t o pr a cti c e i n cl u d e i nf or m ati o n r el at e d t o s cr e e ni n gs, p er c e pti o ns of i m p a ct, a n d p oli ci es i n pl a c e t o a d dr ess tr a u m a. W hil e f e w P C Ps s p e cifi c all y s cr e e n f or A C Es ( 3 6 %) a v ari et y of ot h er s cr e e ni n g pr a cti c es e m er g e d. A mi n ut e diff er e n c e w as c al c ul at e d b et w e e n n urs e pr a ctiti o n er a n d p h ysi ci a ns s cr e e ni n g pr a cti c es wit h a c hi -s q u ar e p v al u e of 0. 3 6. T h es e i n cl u d e d r o uti n e s cr e e ni n g f or p h ysi c al, s e x u al, or e m oti o n al a b us e, s u bst a n c e a b us e, a n d a n xi et y or d e pr essi o n (s e e T a bl e 8). T a bl e 8 V ari ati o ns of S cr e e ni n g Pr a cti c es b y Pr of essi o n al R ol e 6 5 4 3 2 1 0 N urs e Pr a ctiti o n er P h ysi ci a n P e r c e pti o ns P er c e pti o ns of A C E s d at a w er e hi g hl y f a v or a bl e 1 0 0 % of p arti ci p a nts e x pr ess e d t h e b eli ef t h at a d diti o n al tr ai ni n g o n t h e A C E s r es e ar c h w o ul d b e h el pf ul. P C Ps m e nti o n e d pr ot o c ols e xist e d f or s cr e e ni n g, ass ess m e nt, a n d r ef err al pr o c ess es b ut a d e vi ati o n w as n ot e d f or tr e at m e nt pr ot o c ols. 7 8 % of p arti ci p a nts m ar k e d n o tr e at m e nt pr ot o c ols e xist e d at t h ei r pl a c e of pr a cti c e. A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S T his n e g ati v e diff er e n c e w as n ot n ot e d i n t h e p att er n f or tr e at m e nt pr ot o c ols wit hi n i nt er n al m e di ci n e a n d pri m ar y c ar e (s e e T a bl e 9 ). T a bl e 9 E xisti n g Pr ot o c ols at Pr a cti c e L o c ati o ns L a c k of c ar e c o or di n ati o n s er vi c es , L a c k of tr ai ni n g , a n d I n a d e q u at e ti m e , w er e cit e d as m ost fr e q u e nt b arri ers (s e e T a bl e 1 0). T h e h y p ot h esis of t h e diff er e n c es b et w e e n pr a ctiti o n ers w as m ar k e d i n r e g ar ds t o p er c ei v e d b arri ers as o p p os e d t o s cr e e ni n g pr a cti c es. T a bl e 1 0 B arri ers P er c ei v e d b y Q u esti o n n air e P arti ci p a nts A s u btl e diff er e n c e e xist e d b et w e e n n urs e p r actiti o n er s r es p o ns es a n d ph ysi ci a n s r esp o ns es t o b arri ers (s e e T a bl e 1 1 ). F or ph ysi ci a ns, i n a d e q u at e ti m e w as m ost oft e n cit e d as a b arri er. P h ysi ci a ns als o cit e d i n a d e q u at e r ei m b ur s e m e nt w h er e n o n e of t h e n urs e p r a ctiti o n ers 30 A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 31 c o nsi d er e d t h e r ei m b urs e m e nt p orti o n of c ar e. Si mil arl y, al m ost all of t h e pr o vi d ers m ar k e d r o uti n e m e nt al h e alt h s er vi c es as e xtr e m el y diffi c ult f or r ef err e d p ati e nts t o a c c ess. T a bl e 1 1 V ari a n c es Dis pl a y e d B et w e e n Dis ci pli n es N urs e Pr a ctit i o n er P h ysi ci a n Dis c o mf ort i n as ki n g s e nsiti v e q u esti o ns I n a d e q u at e r ei m b urs e m e nt I n a d e q u at e ti m e L a c k of c ar e c o or di n a ti o n s er vi c es t o li n k p ati e nts or fa mili es wit h c o m m u nit y r es o ur c es L a c k of l o c al r es o ur c es L a c k of tr ai ni n g i n m a n a gi n g a p ati e nt w h o h as e x p eri e n c e d a d v ers e c hil d h o o d tr a u m a P ati e nt r el u ct a n c e t o dis c uss s e nsiti v e/ p ers o n al t o pi cs 1 3 5 2 4 2 3 4 2 2 3 2 A d diti o n al h y p ot h es es t h at w er e n ot c o nfir m e d b y t h e d at a i n cl u d e littl e v ari a n c e b et w e e n pr a cti c e l o c ati o ns or a g e ass o ci ati o ns r e g ar di n g pr o vi d er utili z ati o n of d at a. L i mit ati o ns M ulti pl e si g nifi c a nt li mit ati o ns e xist e d i n t his pr oj e ct. T h e first li mit ati o n w as t h e s m all s a m pl e si z e. W hil e t h e r es p o ns e r at e w as si mil ar t o m a n y s ur v e ys, t h e s m all s a m pl e si z e of t his d a t a m a d e it i nsi g nifi c a nt a n d diffi c ult t o g e n er ali z e e. g. n o n e of t h e c hi-s q u ar e r es ults a p pr o a c h e d si g nifi c a n c e. I n a d diti o n, it c a n n ot b e e x p e ct e d t o r efl e ct t h e vi e ws of t h e c o m m u nit y as a w h ol e n eit h er i n pri m ar y c ar e, t h es e sit es, n or t h e pr a cti c e l o c ati o ns. Si mil arl y, t h e s p arsit y of di v e rsit y wit hi n t h e s a m pl e li mite d t h e c o n cl usi o ns. P h ysi ci a n assist a nts, a cti n g as pri m ar y c ar e pr o vi d ers, w er e n ot r e pr es e nt e d i n a n y f or m. T h e s el e cti v e i niti al pr oj e ct sit es li mit e d t h e g e n er ali z a bilit y of pr oj e ct. F urt h er, t h e s ur v e y a d a pt e d fr o m B or a et al. ( 2 0 2 1) w as n ot a v ali d at e d s ur v e y a n d t h er ef or e r eli a bilit y w as s us p e ct. I n a d diti o n, t h e r es ults i n di c at e d t h os e w h o r es p o n d e d t o t h e q u esti o n n air e h a d s o m e l e v el of f a mili arit y wit h A C Es. T his m a y h a v e A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 32 i n di c at e d a l e v el of bi as i n h er e nt wit hi n t h e q u esti o n n air e t h at t h os e w h o r es p o n d e d alr e a d y v a l u e d t h e t o pi c. Fi n all y, t h e ti m e c o nstr ai nts of t h e pr oj e ct a d v ers el y aff e ct e d pr oj e ct ri g or. T h e f e w p arti ci p a nts of t h e q u esti o n n air e i n di c at e d r e mi n d ers c a n b e a n ai d i n c oll e cti o n of d at a. S i mil arl y, if a d diti o n al ti m e h a d b e e n p er mitt e d t o c oll e ct m or e r es p o n d e nt s p ot e nti all y m or e v ol u nt e ers c o ul d h a v e b e e n i n cl u d e d t o pr o vi d e r o b ust q u alit ati v e d at a. Si n c e t h e q u alit ati v e f e at ur e pr es e nt e d wit h w e a k p o w er it w as e x cl u d e d as a q u alit ati v e r es ult. H o w e v er, s o m e of t h e c o m m e nts fr o m t h e r es p o n d e nt ai d e d i n t h e dis c ussi o n of t h e a p pli c ati o n of t his pr oj e ct. I nt e r vi e w wit h R es p o n d e nt O bj e cti v e 5 m e as ur e d t h e ai m t o g ai n p ers p e cti v es of P C Ps i n a d ult pri m ar y c ar e. T h e g o al w as t h at a d diti o n al cr e di bilit y w o ul d b e assist e d b y a mi ni m u m of 8 -1 0 q u alit ati v e r es p o n d e nts. H o w e v er, i ns uffi ci e nt s at ur ati o n w as r e a c h e d a n d a d diti o n al r e cr uit m e nt w as o utsi d e t h e s c o p e of t his pr oj e ct. B e c a us e 8 -1 0 v ol u nt e er s di d n ot e m er g e fr o m t h e d at a , t h e o bj e cti v e w as n ot m et. H o w e v er, t h e i nt er vi e w pr o c ess di d eli cit s o m e u n e x p e ct e d fi n di n gs fr o m t h e o n e c o m pl et e d i nt er vi e w. M u c h of t h e c o n v ers ati o n b et w e e n t h e pri n ci p al i nv esti g at or a n d t h e r es p o n d e nt f oll o w e d t h e p atter n of b arri ers n ot e d i n t h e lit er at ur e e. g. ti m e, e d u c ati o n, a p pr o pri at e r ef err als , et c. H o w e v er, t h e r es p o n d e nt m a d e a st at e m e nt t h at a p p e ar e d m e a ni n gf ul t o t h e i n v esti g at or. T h e c o m m e nt w as m a d e i n r es p o ns e t o t h e i n q uir y r e g ar di n g e x p eri e n c e of pr o vi di n g c ar e t o a d ults wit h hi g h A C E s s c or es. T h e r es p o n d e nt m a d e t h e st at e m e nt, Its n ot v er y s atisf yi n g. B e c a us e b y t h e ti m e y o u s e e t h e m, it s b ec o m e s o m u c h a p art of t h eir p ers o n alit y a n d lif e t h at m ost of t h e m m ost of t h e m ar e afr ai d t o d e al wit h it. T his st at e m e nt w as a p er c e pti o n t h at w as miss e d i n t h e q u a ntit ati v e d at a of utili z ati o n of A C Es i n pr a cti c e. T his r es p o n d e nt i m pli e d t h at A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 33 t h e l ac k of gr atifi c ati o n of w or ki n g wit h p e o pl e wit h a hist or y of c hil d h o o d tr a u m a i m p a cts t h e pr o visi o n of c ar e. A r es p o ns e b y t h e pr o vi d er t o t h e c o m pl e x p ati e nt w h o pr es e nts t o t h e offi c e is t o gr o a n . T his pr a ctiti o n er st at e d, Y o u g et y o ur h a n ds dirt y a n d e x pr ess e d t h e tr e m e n d o us a m o u nt of ti m e a n d e n er g y ass o ci at e d wit h c ari n g f or a d ults wit h c hil d h o o d hist ori es of tr a u m a. T h e p ers p e cti v e w as s h ar e d t h at e v e n m e nt al h e alt h pr o vi d ers l a c k t h e e n er g y a n d ti m e t o a d dr ess t h es e iss u es. T his pr es e nts a u ni q u e c h all e n g e t o f ut ur e r es e ar c h. T h e r ol e of gr atifi c ati o n, m oti v ati o ns of pr o vi d ers, a n d b eli efs a b o ut t h e ti m e a n d e n er g y r e q uir e d t o a d dr ess A C Es i n t h e a d ult p ati e nt m a y c o ntri b ut e t o t h e d el a y of A C Es i nt e gr ati o n i nt o pri m ar y c ar e. A r e c o m m e n d ati o n of t his pr oj e ct is t o d e v el o p a ri g or o us u n d erst a n di n g of t h e b eli efs a n d m oti v ati o ns t h at i m p a ct pr o vi d er e n g a g e m e nt i n t h e tr e at m e nt of t h e a d ult p ati e nt wit h a hist or y of c hil d h o o d tr a u m a. Dis c ussi o n O bj e cti v e 1, 3 , a n d 4, e v al u at e d t h e k n o wl e d g e, s cr e e ni n g h a bits, a n d pr a cti c e i nt er v e nti o ns r e g ar di n g A C Es s c or es a m o n g pri m ar y c ar e pr o vi d ers. D es pit e t h e s m all s a m ple si z e, k n o wl e d g e a n d tr ai ni n g s h o w e d c o m p ar a bl e d at a t o t h e lit er at ur e r el at e d t o A C Es k n o wl e d g e. F a mili arit y wit h A C Es a p p e ar e d li mit e d a m o n g t his s m all c o h ort a n d affir m e d t h e r e c o m m e n d ati o n t o pr o vi d e a d diti o n al e d u c ati o n t o a d ult pri m ar y c ar e pr o vi d ers. H o w e v er, t h e s m all p er c e nt a g e of p h ysi ci a ns w h o e x pr ess e d m or e f a mili arit y wit h A C Es, s u p p ort e d t h eir m or e di v ers e s et of s cr e e ni n g h a bits. T h e c h all e n g e pr es e nt e d b y t his d at a r e m ai ns w h at t o d o wit h a p ati e nt w h o al erts f or e x p os ur e t o c hil d h o o d tr a u m a ? A d ult P C Ps d e n ot e pr ot o c ols f or g ui d a n c e is n e e d e d f or ass ess m e nt, s cr e e ni n g, a n d r ef err al. Str e a mli n e d r e c o m m e n d ati o ns f or i n- offi c e r es p o ns es is n e c ess ar y t o assist P C Ps i n u n d erst a n di n g t h eir r ol e r el ati n g t o a d ult p ati e nts wit h A C Es. A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 34 T h e v ari a n c e s b et w e e n cli ni ci a n s p e ci alti es i n o bj e cti v e 2 w as dis pl a y e d pri m a ril y i n s cr e e ni n g pr a cti c es a n d vi e w of b arri ers. S e v er al u ni q u e t h e m es e m er g e d fr o m t h e b arri ers P C Ps e x p eri e n c e i n r e g ar ds t o utili zi n g A C Es i n pr a cti c e. N urs e pr a ctiti o n ers p er c ei v e d diff e r e nt b arri ers t o t h e utili z ati o n b y a s m all m ar gi n. A c c or di n g t o t his d at a, p h ysi ci a ns s cr e e n e d f or a wi d e r arr a y of tr a u m a-r el at e d b e h a vi ors. T his w o ul d s u g g est t h at e d u c ati n g pr a ctiti o n ers o n s er vi c es a n d a d diti o n al tr ai ni n g c o ul d i n cr e as e eff e cti v e n ess of A C E s tr a nsl ati o n t o pr a cti c e. I n a d diti o n, a n a ut o m at e d s cr e e ni n g pr o c ess c o ul d ai d pr a ctiti o n ers w h o fi n d t h e ti m e r e q uir e m e nt of a n A C Es c o n v ers ati o n t o o d a u nti n g. H o w e v er , c o ns e ns us a m o n g pr o vi d ers a p p e ar e d at t h e o v er w h el mi n g r es p o ns e t h at r o uti n e m e nt a l h e alt h s er vi c es ar e e xtr e m el y diffi c ult f or r ef err e d p ati e nts t o a c c ess. T his s u p p orts t h e c o n c er n m e nti o n e d i n t h e lit er at ur e r e vi e w t h at s cr e e ni n g p ositi v el y f or A C Es will si m pl y o v er w h el m a n alr e a d y b ur d e n e d h e alt h c ar e s yst e m. A d diti o n al i nf or m ati o n fr o m t h e pr oj e ct assists i n t h e r e c o m m e nd ati o ns f or f ut ur e e d u c ati o n t o m e et t h e g a p e xist e nt b et w e e n e vi d e n c e -b a s e d c ar e a n d c urr e nt pr a cti c es i n t h e a d ult pri m ar y c ar e s etti n g. C o n cl usi o n T h e pri m ar y r e c o m m e n d ati o n of t his pr oj e ct, a n d m u c h of t h e lit er at ur e, is t o d e v el o p a str ea mli n e d r e c o m m e n d ati o n of h o w t o i m pl e m e nt A C Es i n a d ul t pri m a r y c ar e ( D u b e, 2 0 1 8; J o n es et al., 2 0 2 0; K al m a kis et al., 2 0 1 7; S h erfi ns ki et al., 2 0 2 1; St ori k et al., 2 0 2 0). T h es e i n cl u d e a v ali d at e d t o ol f or ass ess m e nt a n d s cr e e ni n g, a p pr o pri at e i nt er v e nti o ns i n t h e pri m ar y c ar e s etti n g, p ot e nti al a nti ci p at or y g ui d a n c e f or a d ults, c o m m u nit y r es o ur c es t o ai d i n a d dr essi n g tr au m a c ar e, eff e cti v e str at e gi es t o i m pr o v e t h e s yst e m, a n d q u alit y e d u c ati o n f or fr o ntli n e w or k ers ( B or a et al., 2 0 2 1; J o n es et al., 2 0 2 0; K al m a kis et al., 2 0 1 7; S h erfi ns ki et al., 2 0 2 1; S t ori k et al., 2 0 2 0). T h e p ur p os e of t his pr oj e ct w o ul d b e r e miss wit h o ut i n cl u di n g t h e r ol e of d o c t or al pr e p ar e d n urs e pr a ctiti o n ers ( D N Ps) t o ai d i n t h e tr a nsl ati o n of A C Es r es e ar c h i nt o A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 35 pr a cti c e a n d pr o m oti o n of tr a u m a-i nf or m e d c ar e. A d v a n c e d pr a cti c e n urs es ar e a cti v e p a rti ci p a nts i n d e v el o pi n g r es e ar c h of a d v ers e c hil d h o o d e x p eri e n c es ( Es d e n, 2 0 1 8; K al m a kis et a l., 2 0 1 7; P ar d e e et al., 2 0 1 7; R ari d e n et al., 2 0 2 1; R o b erts et al., 2 0 1 9; Z h a n et al., 2 0 2 1). It is wit hi n t h e s c o p e of t h e D N P t o utili z e t h eir e x p ertis e t o i nt e gr at e e vi d e n c e- b as e d pr a cti c es i nt o h e alt h s yst e ms ( B o n n el & S mit h, 2 0 1 8; M el n y k & Fi n e o ut- O v er h olt, 2 0 1 9). T h e r ol e of t h e D N P a s a tr a nsl at or of e vi d e n c e- b as e d pr a cti c e is p ositi o n e d t o assist wit h i nt e gr ati o n of A C E s r es e ar c h i nt o a d ult pri m ar y c ar e pr a cti c e. I n a d diti o n, n urs e pr a ctiti o n ers ar e tr ai n e d i n t h e p ar a di g m t h at h e alt h c ar e is s o m et hi n g gr e at er t h e n bi o m e di ci n e ( D u n p h y et al., 2 0 1 9). T h e c urr e nt pr oj e ct ass erts t h at a k e y f a cilit ati o n of Dr. F elittis tr a nsiti o n fr o m bi o m e di c al t o bi o ps y c h os o ci al is t h e c o nti n u e d i nt e gr ati o n of a d v a n c e d pr a cti c e n urs es wit hi n a d ult pri m ar y c ar e ( F elitti, 2 0 1 7). A C Es is a c o n c er n f or pri m ar y c ar e. Aft er a t h or o u g h b a c k gr o u n d a n d lit er at ur e r e vi e w of A C Es, t h e i m pl e m e nt ati o n a n d r es ults of t his pr oj e ct pr o vi d e f e w si g nifi c a nt fi n di n gs. M or e r es e ar c h is n e e d e d t o d et er mi n e pr o vi d ers k n o wl e d g e, i m pl e m e nt ati o n, a n d p er c ei v e d b arri ers t o utili z ati o n of A C Es i n pri m ar y c ar e. H o w e v er, t h e a d diti o n al dis c ussi o n of P C P m oti v ati o ns a n d p e r c e pti o ns off er a u ni q u e p ers p e cti v e t o t h e c o n c e pt of w h at is r e q uir e d t o tr a nsl at e t h e A C Es i nf or m ati o n i nt o a d ult pri m ar y c ar e. It is a m ost vit al r e c o m m e n d ati o n t h at f urt h er r es e ar c h t o s u p p ort str e a mli n e d e d u c ati o n a n d pr a cti c e r e c o m m e n d ati o ns m ai nt ai n pri orit y i n h e alt h c ar e. Att e n di n g t o t h e d y n a mi cs of c hil d h o o d tr a u m a a n d a d ult b e h a vi ors m a y l e a d t o i m p ort a nt a n d s ust ai n a bl e i m pr o v e m e nts of q u alit y of h e alt h. Pr e v e nt a bl e m or bi dit y a n d m ort alit y m a y r es ult fr o m i g n ori n g t h e c o ntri b uti o ns of A C Es i n a d ult c hr o ni c dis e as e. I n a d diti o n, t h e r ol e of n urs e pr a ctiti o n ers t o e d u c at e a n d assist i n a p pli c ati o n of t his e vi d e n c e is a visi o n of h o p e f or t h e f ut ur e. A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 36 R ef e r e n c es Al b a e k, A. U ., Ki n n, L. G., & Mil d e, A. M. ( 2 0 1 8). W al ki n g c hil dr e n t hr o u g h a mi n efi el d: H o w pr of essi o n als e x p eri e n c e e x pl ori n g a d v ers e c hil d h o o d e x p eri e n c es. Q u alit ati v e H e alt h R es e ar c h , 2 8 ( 2), 2 3 1 2 4 4. htt ps:// d oi. or g/ 1 0. 1 1 7 7/ 1 0 4 9 7 3 2 3 1 7 7 3 4 8 2 8 Al h o w a y m el, F., K al m a kis, K., & J a c el o n, C. ( 2 0 2 1). D e v el o pi n g t h e c o n c e pt of a d v ers e c hil d h o o d e x p eri e n c es: A gl o b al p ers p e cti v e. J o ur n al of P e di atri c N ursi n g , 5 6 , 1 8 2 3. htt ps:// d oi. or g/ 1 0. 1 0 1 6/j. p e d n. 2 0 2 0. 1 0. 0 0 4 A m a y a-J a c ks o n, L., A bs h er, L. E., G errit y, E. T., L a y n e, C. M., & H all a d a y G ol d m a n, J. ( 2 0 2 1). B e y o n d t h e A C E s c or e: P ers p e cti v es fr o m t h e N C T S N o n c hil d tr a u m a a n d a d v ersit y s cr e e ni n g a n d i m p a ct. N ati o n al C e nt er f or C hil d Tr a u m ati c Str ess. htt ps:// w w w. n cts n. or g/sit es/ d ef a ult/fil es/r es o ur c es/s p e ci al-r es o ur c e/ b e y o n d-t h e- a c es c or e- p ers p e cti v es-fr o m-t h e- n cts n- o n- c hil d-t a u m a- a n d - a d v ersit y-s cr e e ni n g- a n di m p a ct. p df B a k er, C. N., Br o w n, S. M., O v erstr e et, S., & Wil c o x, P. D. ( 2 0 2 1). V ali d ati o n of t h e attit u d es r el at e d t o tr a u m a-i nf or m e d c ar e s c al e ( A R TI C). Ps y c h ol o gi c al Tr a u m a: T h e or y, R es e ar c h, Pr a cti c e, a n d P oli c y , 1 3 ( 5), 5 0 5 5 1 3. htt ps:// d oi. or g/ 1 0. 1 0 3 7/tr a 0 0 0 0 9 8 9 B et h ell, C. D., C arl e, A., H u d zi a k, J., G o m b oj a v, N., P o w ers, K., W a d e, R., & Br a v e m a n, P. ( 2 0 1 7). M et h o ds t o ass ess a d v ers e c hil d h o o d e x p eri e n c es of c hil dr e n a n d f a mili es: T o w ar d a p pr o a c h es t o pr o m ot e c hil d w ell- b ei n g i n p oli c y a n d pr a cti c e. A c a d e mi c P e di atri cs , 1 7 ( 7). htt ps:// d oi. or g/ 1 0. 1 0 1 6/j. a c a p. 2 0 1 7. 0 4. 1 6 1 B h us h a n, D., K ot z, K., M c C all, J., Wirt z, S., Gil g off, R., D u b e, S. R., P o w ers, C., Ols o nM or g a n, J., G al est e, M., P att ers o n, K., H arris, L., Mills, A., B et h ell, C., & B ur k e H arris, N. ( 2 0 2 0). R o a d m a p f or r esili e n c e: T h e C alif or ni a S ur g e o n G e n er als r e p ort o n A d v ers e A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 37 C hil d h o o d E x p eri e n c es, t o xi c str ess, a n d h e alt h. Offi c e of t h e C alif or ni a S ur g e o n G e n er al. htt ps:// d oi: 1 0. 4 8 0 1 9/ P E A M 8 8 1 2 B o d e n d orf er, V., K o b all, A. M., R as m uss e n, C., K l e v a n, J., R a mir e z, L., & Ols o n-D orff, D. ( 2 0 2 0). I m pl e m e nt ati o n of t h e a d v ers e c hil d h o o d e x p eri e n c es c o n v ers ati o n i n pri m ar y c ar e. F a m il y Pr a cti c e, 3 7 ( 3), 3 5 5 3 5 9. htt ps:// d oi. or g/ 1 0. 1 0 9 3/f a m pr a/ c m z 0 6 5 B o ers m a, P., Bl a c k, L. I., & W ar d, B. W. ( 2 0 2 0). Pr e v al e n c e of m ulti pl e c hr o ni c c o n diti o ns a m o n g U S a d ults, 2 0 1 8. Pr e v e nti n g C hr o ni c Dis e as e , 1 7 . htt ps:// d oi. or g/ 1 0. 5 8 8 8/ p c d 1 7. 2 0 0 1 3 0 B o n n el, W. E., & S mit h, K. V. ( 2 0 1 8). Pr o p os al writi n g f or cli ni c al n ursi n g a n d D N P pr oj e cts . S pri n g er P u blis hi n g C o m p a n y. B or a, N., J o n es, T. R., S al a d a, K., & Br u m m el, M. ( 2 0 2 1). I nt er- cli ni ci a n v ari a bilit y i n pri m ar y c ar e pr o vi d ers a d v ers e c hil d h o o d e x p eri e n c e k n o wl e d g e, tr ai ni n g, s cr e e ni n g pr a cti c es, a n d p er c ei v e d i nt er v e nti o n b arri ers: A n e x pl or at or y cr oss-s e cti o n al st u d y. J o ur n al of C hil d & A d ol es c e nt Tr a u m a . htt ps:// d oi. or g/ 1 0. 1 0 0 7/s 4 0 6 5 3- 0 2 1- 0 0 3 6 5- x C a m p b ell, J. A., M osl e y -J o h ns o n, E., G ar a c ci, E., W al k er, R. J., & E g e d e, L. E. ( 2 0 1 9). T h e c o o c c urr e n c e of di a b et es a n d a d v ers e c hil d h o o d e x p eri e n c es a n d its i m p a ct o n m ort alit y i n U S a d ults. J o ur n al of Aff e cti v e Dis or d ers , 2 4 9 , 2 0 2 5. htt ps:// d oi. or g/ 1 0. 1 0 1 6/j.j a d. 2 0 1 9. 0 2. 0 1 6 C a m p b ell, T. L. ( 2 0 2 0). S cr e e ni n g f or a d v ers e c hil d h o o d e x p eri e n c es ( A C Es) i n pri m ar y c ar e: A c a uti o n ar y n ot e. J A M A , 3 2 3 ( 2 3), 2 3 7 9 2 3 8 0. htt ps:// d oi. or g/ 1 0. 1 0 0 1/j a m a. 2 0 2 0. 4 3 6 5 C h e n, E., T uri a n o, N. A., Mr o c z e k, D. K., & Mill er, G. E. ( 2 0 1 6). Ass o ci ati o n of r e p orts of c hil d h o o d a b us e a n d all -c a us e m ort alit y r at es i n w o m e n. J A M A Ps y c hi atr y , 7 3 ( 9), 9 2 0 9 2 7. htt ps:// d oi. or g/ 1 0. 1 0 0 1/j a m a ps y c hi atr y. 2 0 1 6. 1 7 8 6 A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 38 D u b e S. R. ( 2 0 1 8). C o nti n ui n g c o n v ers ati o ns a b o ut a d v ers e c hil d h o o d e x p eri e n c es ( A C Es) s cr e e ni n g: A p u bli c h e alt h p ers p e cti v e. C hil d A b us e & N e gl e ct, 8 5 , 1 8 0 1 8 4. htt ps:// d oi. or g/ 1 0. 1 0 1 6/j. c hi a b u. 2 0 1 8. 0 3. 0 0 7 D u n p h y, L. M. H., Wi nl a n d- Br o w n, J. E., P ort er, B. O., & T h o m as, D. J. ( 2 0 1 9). Pri m ar y c ar e: t h e art a n d s ci e n c e of a d v a n c e d pr a cti c e n ursi n g - a n i nt er pr of essi o n al a p pr o a c h. F. A. D a vis C o m p a n y. Es d e n, J. L. ( 2 0 1 8). A d v ers e c hil d h o o d e x p eri e n c es a n d i m pl e m e nti n g tr a u m a -i nf or m e d pri m ar y c ar e. T h e N urs e Pr a ctiti o n er , 4 3 ( 1 2), 1 0 2 1. htt ps:// d oi. or g/ 1 0. 1 0 9 7/ 0 1. N P R. 0 0 0 0 5 4 7 5 5 0. 4 8 5 1 7. e 9 F elitti, V. J., A n d a, R. F., N or d e n b er g, D., Willi a ms o n, D. F., S pit z, A. M., E d w ar ds, V., K oss, M. P., & M ar ks, J. S. ( 1 9 9 8). R el ati o ns hi p of c hil d h o o d a b us e a n d h o us e h ol d d ysf u n cti o n t o m a n y of t h e l e a di n g c a us es of d e at h i n a d ults. A m eri c a n J o ur n al of Pr e v e nti v e M e di ci n e , 1 4 ( 4), 2 4 5 2 5 8. htt ps:// d oi. or g/ 1 0. 1 0 1 6/s 0 7 4 9- 3 7 9 7( 9 8) 0 0 0 1 7- 8 F elitti, V. ( 2 0 1 7). F ut ur e a p pli c ati o n of t h e a d v ers e c hil d h o o d e x p eri e n c es r es e ar c h. J o ur n al of C hil d & A d ol es c e nt Tr a u m a, 1 0 ( 3), 2 0 5- 2 0 6. htt ps:// d oi. or g/ 1 0. 1 0 0 7/s 4 0 6 5 3- 0 1 7- 0 1 8 9- 1 F elitti V. J. ( 2 0 1 9). H e alt h a p pr ais al a n d t h e a d v ers e c hil d h o o d e x p eri e n c es st u d y: N ati o n al i m pli c ati o ns f or h e alt h c ar e, c ost, a n d utili z ati o n. T h e P er m a n e nt e J o ur n al , 2 3 , 1 8 0 2 6. htt ps:// d oi. or g/ 1 0. 7 8 1 2/ T P P/ 1 8 -0 2 6 Fi n k el h or D. ( 2 0 1 8). S cr e e ni n g f or a d v ers e c hil d h o o d e x p eri e n c es ( A C Es): C a uti o ns a n d s u g g esti o ns. C hil d A b us e & N e gl e ct , 8 5 , 1 7 4 1 7 9. htt ps:// d oi. or g/ 1 0. 1 0 1 6/j. c hi a b u. 2 0 1 7. 0 7. 0 1 6 F or d, K., H u g h es, K., H ar d c astl e, K., Di L e m m a, L. C. G., D a vi es, A. R., E d w ar ds, S ., & B ellis, M. A. ( 2 0 1 9). T h e e vi d e n c e b as e f or r o uti n e e n q uir y i nt o a d v ers e c hil d h o o d e x p eri e n c es: A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S A s c o pi n g r e vi e w. C hil d A b us e & N e gl e ct , 9 1 , 1 3 1 1 4 6. htt ps:// d oi. or g/ 1 0. 1 0 1 6/j. c hi a b u. 2 0 1 9. 0 3. 0 0 7 F orst er, M., G o w er, A. L., B or o ws k y, I. W., & M c M orris, B. J. ( 2 0 1 7). Ass o ci ati o ns b et w e e n a d v ers e c hil d h o o d e x p eri e n c es, st u d e nt -t e a c h er r el ati o ns hi ps, a n d n o n-m e di c al us e of pr es cri pti o n m e di c ati o ns a m o n g a d ol es c e nts. A d di cti v e B e h a vi ors , 6 8 , 3 0 3 4 . htt ps:// d oi. or g/ 1 0. 1 0 1 6/j. a d d b e h. 2 0 1 7. 0 1. 0 0 4 Gil g off, R., Si n g h, L., K oit a, K., G e ntil e, B., & M ar q u es, S. S. ( 2 0 2 0). A d v ers e c hil d h o o d e x p eri e n c es, o ut c o m es, a n d i nt er v e nti o ns. P e di atri c Cli ni cs of N ort h A m eri c a , 6 7 ( 2), 2 5 9 2 7 3. htt ps:// d oi. or g/ 1 0. 1 0 1 6/j. p cl. 2 0 1 9. 1 2. 0 0 1 Gl o w a, P. T., Ols o n, A. L., & J o h ns o n, D. J. ( 2 0 1 6). S cr e e ni n g f or a d v ers e c hil d h o o d e x p eri e n c es i n a f a mil y m e di ci n e s etti n g: A f e asi bilit y st u d y. J o ur n al of t h e A m eri c a n B o ar d of F a mil y M e di ci n e : J A B F M , 2 9 ( 3), 3 0 3 3 0 7. htt ps:// d oi. or g/ 1 0. 3 1 2 2/j a bf m. 2 0 1 6. 0 3. 1 5 0 3 1 0 G ol dst ei n, E., At h al e, N., S ci oll a, A. F., & C at z, S. L. ( 2 0 1 7). P ati e nt pr ef er e n c es f or dis c ussi n g c hil d h o o d tr a u m a i n pri m ar y c ar e. T h e P er m a n e nt e J o ur n al , 2 1 . htt ps:// d oi. or g/ 1 0. 7 8 1 2/t p p/ 1 6- 0 5 5 H u g h es, K., B ellis, M. A., H ar d c astl e, K. A., S et hi, D., B ut c h art, A., Mi kt o n, C., J o n es, L., & D u n n e, M. P. ( 2 0 1 7). T h e eff e ct of m ulti pl e a d v ers e c hil d h o o d e x p eri e n c es o n h e alt h: A s yst e m ati c r e vi e w a n d m et a- a n al ysis. T h e L a n c et P u bli c H e alt h , 2 ( 8). htt ps:// d oi. or g/ 1 0. 1 0 1 6/s 2 4 6 8- 2 6 6 7( 1 7) 3 0 1 1 8- 4 J o n es, C. M., M erri c k, M. T., & H o ur y, D. E. ( 2 0 2 0). I d e ntif yi n g a n d pr e v e nti n g c hil d h o o d e x p eri e n c es: I m pli c ati o ns f or cli ni c al pr a cti c e. J A M A, 3 2 ( 1), 2 5- 2 6. htt ps:// d oi: 1 0. 1 0 0 1/j a m a. 2 0 1 9. 1 8 4 9 9 39 A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 40 J u o n al a, M., Pit k n e n, N., T ol o n e n, S., L a a ks o n e n, M., Si e v n e n, H., J o ki n e n, E., L aiti n e n, T., S a bi n, M. A., H utri -K h n e n, N., L e hti m ki, T., T aitt o n e n, L., J ul a, A., L o o, B. M., I m pi v a ar a, O., K h n e n, M., M a g n uss en, C. G., Vii k ari, J. S. A., & R ait a k ari, O. T. ( 2 0 1 9). C hil d h o o d e x p os ur e t o p assi v e s m o ki n g a n d b o n e h e alt h i n a d ult h o o d: T h e C ar di o v as c ul ar ris k i n Y o u n g Fi n ns St u d y. T h e J o ur n al of Cli ni c al E n d o cri n ol o g y a n d M et a b olis m , 1 0 4 ( 6), 2 4 0 3 2 4 1 1. htt ps:// d oi. or g/ 1 0. 1 2 1 0/j c. 2 0 1 8- 0 2 5 0 1 K al m a kis, K. A., C h a n dl er, G. E., R o b erts, S. J., & L e u n g, K. ( 2 0 1 7). N urs e pr a ctiti o n er s cr e e ni n g f or c hil d h o o d a d v ersit y a m o n g a d ult pri m ar y c ar e p ati e nts. J o ur n al of t h e A m eri c a n Ass o ci ati o n of N urs e Pr a ctiti o n ers , 2 9 ( 1), 3 5 4 5. htt ps:// d oi. or g/ 1 0. 1 0 0 2/ 2 3 2 76 9 2 4. 1 2 3 7 8 Ki n g, S., C h e n, K. -L. D., & C h o ks hi, B. ( 2 0 1 9). B e c o mi n g tr a u m a i nf or m e d: V ali d ati n g a t o ol t o ass ess h e alt h pr of essi o n als k n o wl e d g e, attit u d e, a n d pr a cti c e. P e di atri c Q u alit y & S af et y , 4 ( 5), e 2 1 5. htt ps:// d oi. or g/ 1 0. 1 0 9 7/ p q 9. 0 0 0 0 0 0 0 0 0 0 0 0 0 2 1 5 M a u n d er, R. G., H u nt er, J. J., T a n n e n b a u m , D. W., L e, T. L., & L a y, C. ( 2 0 2 0). P h ysi ci a ns k n o wl e d g e a n d pr a cti c es r e g ar di n g s cr e e ni n g a d ult p ati e nts f or a d v ers e c hil d h o o d e x p eri e n c es: a s ur v e y. B M C H e alt h S er vi c es R es e ar c h , 2 0 ( 1), 3 1 4. htt ps:// d oi. or g/ 1 0. 1 1 8 6/s 1 2 9 1 3- 0 2 0 - 0 5 1 2 4- 6 M c L e n n a n, J. D., M a c mill a n, H. L., Afifi, T. O., M c T a vis h, J., & G o n z al e z, A. ( 2 0 2 0). Pr o bl e ms wit h t h e r e c o m m e n d ati o n t o i m pl e m e nt A C Es s cr e e ni n g. P a e di atri cs & C hil d H e alt h , 2 5 ( 1), 6 4 6 5. htt ps:// d oi. or g/ 1 0. 1 0 9 3/ p c h/ p x z 1 2 9 M el n y k, B. M., & Fi n e o ut- O v er h olt, E. ( 2 0 1 9). E vi d e n c e- b as e d pr a cti c e i n n ursi n g & h e alt h c ar e: a g ui d e t o b est pr a cti c e ( 4 t h e d). W olt ers Kl u w ers. A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 41 M erri c k, M. T., F or d, D. C., P orts, K. A., G ui n n, A. S., C h e n, J., Kl e v e ns, J., M et zl er, M., J o n es, C . M., Si m o n, T. R., D a ni el, V. M., Ottl e y, P., & M er c y, J. A. ( 2 0 1 9). Vit al si g ns: Esti m at e d pr o p orti o n of a d ult h e alt h pr o bl e ms attri b ut a bl e t o a d v ers e c hil d h o o d e x p eri e n c es a n d i m pli c ati o ns f or pr e v e nti o n 2 5 st at es, 2 0 1 5 2 0 1 7. M M W R. M or bi dit y a n d M ort alit y W e e kl y R e p ort , 6 8 ( 4 4), 9 9 9 1 0 0 5. htt ps:// d oi. or g/ 1 0. 1 5 5 8 5/ m m wr. m m 6 8 4 4 e 1 N ati o n al C e nt er f or I nj ur y Pr e v e nti o n a n d C o ntr ol. ( 2 0 2 2, A pril 6). F ast f a cts: Pr e v e nti n g a d v ers e c hil d h o o d e x p eri e n c es | vi ol e n c e pr e v e nti o n|i nj ur y C e nt er| C D C . C e nt ers f or Dis e as e C o ntr ol a n d Pr e v e nti o n. htt ps:// w w w. c d c. g o v/ vi ol e n c e pr e v e nti o n/ a c es/f astf a ct. ht ml O zi e h, M. N., G ar a c ci, E., C a m p b ell, J. A., W al k er, R. J., & E g e d e, L. E. ( 2 0 2 0). A d v ers e c hil d h o o d e x p eri e n c es a n d d e cr e as e d r e n al f u n cti o n: I m p a ct o n all -c a us e m ort alit y i n U. S. a d ults. A m eri c a n J o ur n al of Pr e v e nti v e M e di ci n e , 5 9 ( 2), e 4 9 e 5 7. htt ps:// d oi. or g/ 1 0. 1 0 1 6/j. a m e pr e. 2 0 2 0. 0 4. 0 0 5 P ar d e e, M., K u z m a, E., D a hl e m, C. H. G. Y., B o u c h er, N., & D arli n g -Fis h er, C. S. ( 2 0 1 7). C urr e nt st at e of s cr e e ni n g hi g h -A C E y o ut h a n d e m er gi n g a d ults i n pri m ar y c ar e. J o ur n al of t h e A m eri c a n Ass o ci ati o n of N urs e Pr a ctiti o n ers , 2 9 ( 1 2), 7 1 6 7 2 4. htt ps:// d oi. or g/ 1 0. 1 0 0 2/ 2 3 2 7- 6 9 2 4. 1 2 5 3 1 P etr u c c elli, K., D a vis, J., & B er m a n, T. ( 2 0 1 9). A d v ers e c hil d h o o d e x p eri e n c es a n d ass o ci at e d h e alt h o ut c o m es: A s yst e m ati c r e vi e w a n d m et a -a n al ysis. C hil d A b us e & N e gl e ct , 9 7 , 1 0 4 1 2 7. htt ps:// d oi. or g/ 1 0. 1 0 1 6/j. c hi a b u. 2 0 1 9. 1 0 4 1 2 7 P ur k e y, E., P at el, R., B e c k ett, T., & M at hi e u, F. ( 2 0 1 8). Pri m ar y c ar e e x p eri e n c es of w o m e n wit h a hist or y of c hil d h o o d tr a u m a a n d c hr o ni c dis e as e: Tr a u m a -i nf or m e d c ar e A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 42 a p pr o a c h. C a n a di a n F a mil y P h ysi ci a n M e d e ci n d e F a mill e C a n a di e n , 6 4 ( 3), 2 0 4 2 1 1. htt ps:// w w w. cf p. c a/ c o nt e nt/ 6 4/ 3/ 2 0 4.l o n g R ari d e n, C., S mit h B attl e, L., Y o o, J. H., Ci b ul k a, N., & L o m a n, D. ( 2 0 2 1). S cr e e ni n g f or a d v ers e c hil d h o o d e x p eri e n c es: Lit er at ur e r e vi e w a n d pr a cti c e i m pli c ati o ns. T h e J o ur n al f or N urs e Pr a ctiti o n ers: J N P, 1 7 ( 1), 9 8-1 0 4. htt ps:// d oi. or g/ 1 0. 1 0 1 6/j. n ur pr a. 2 0 2 0. 0 8. 0 0 2 R a vi, A., & Littl e, V. ( 2 0 1 7). Pr o vi di n g tr a u m a -i nf or m e d c ar e. A m eri c a n F a mil y P h ysi ci a n , 9 5 ( 1 0), 6 5 5 6 5 7. htt ps:// w w w. a af p. or g/ p u bs/ af p/iss u es/ 2 0 1 7/ 0 5 1 5/ p 6 5 5. ht ml R e y n ol ds, R., D e n nis, S., H as a n, I., Sl e w a, J., C h e n, W., Ti a n, D., B o b b a, S., & Z w ar, N. ( 2 0 1 8). A s yst e m ati c r e vi e w of c hr o ni c dis e as e m a n a g e m e nt i nt er v e nti o ns i n pri m ar y c ar e. B M C F a mil y Pr a cti c e, 1 9 ( 1 1), 1-1 3. htt ps:// d oi. or g/ 1 0. 1 1 8 6/s 1 2 8 7 5- 0 1 7- 0 6 9 2- 3 R o b erts, S. J., C h a n dl er, G. E., & K al m a kis, K. ( 2 0 1 9). A m o d el f or tr a u m a-i nf or m e d pri m ar y c ar e. J o ur n al of t h e A m eri c a n Ass o ci ati o n of N urs e Pr a ctiti o n ers , 3 1 ( 2), 1 3 9 1 4 4. htt ps:// d oi. or g/ 1 0. 1 0 9 7/j x x. 0 0 0 0 0 0 0 0 0 0 0 0 0 1 1 6 S c h o b er, P., & V e tt er, T. R. ( 2 0 1 9). C hi-s q u ar e tests i n m e di c al r es e ar c h. A n est h esi a a n d A n al g esi a , 1 2 9 ( 5), 1 1 9 3. htt ps:// d oi. or g/ 1 0. 1 2 1 3/ A N E. 0 0 0 0 0 0 0 0 0 0 0 0 4 4 1 0 S el y e H. ( 1 9 3 7). T h e si g nifi c a n c e of t h e a dr e n als f or a d a pt ati o n. S ci e n c e ( N e w Y or k, N. Y.) , 8 5 ( 2 2 0 1), 2 4 7 2 4 8. htt ps:// d oi. or g/ 1 0. 1 1 2 6/s ci e n c e. 8 5. 2 2 0 1. 2 4 7 S el y e H. ( 1 9 7 5). Str ess a n d distr ess. C o m pr e h e nsi v e T h er a p y , 1 ( 8), 9 1 3. htt ps:// e ur o p e p m c. or g/ arti cl e/ m e d/ 1 2 2 2 5 6 2 S el y e, H. ( 1 9 9 8). A s y n dr o m e pr o d u c e d b y di v ers e n o c u o us a g e nts. T h e J o ur n al of N e ur o ps y c hi atr y a n d Cli ni c al N e ur os ci e n c es : Offi ci al J o ur n al of t h e A m eri c a n N e ur o ps y c hi atri c Ass o ci ati o n, 1 0 ( 2), 2 3 0 2 3 1. htt ps:// d oi. or g/ 1 0. 1 1 7 6/j n p. 1 0. 2. 2 3 0 a A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 43 S h erfi ns ki, H. T., C o n dit, P. E., Willi a ms Al -K h ar us y, S. S., & M or e n o, M. A. ( 2 0 2 1). A d v ers e c hil d h o o d e x p eri e n c es: P er c e pti o ns, pr a cti c es, a n d p ossi biliti es. W M J : Offi ci al P u bli c ati o n of t h e St at e M e di c al S o ci et y of W is c o nsi n, 1 2 0 ( 3), 2 0 9 2 1 7. htt ps:// w mj o nli n e. or g/ w p- c o nt e nt/ u pl o a ds/ 2 0 2 1/ 1 2 0/ 3/ 2 0 9. p df St o k es, Y., J a c o b, J. D., Giff or d , W., S q uir es, J., & V a n d y k, A. ( 2 0 1 7). E x pl ori n g n urs es' k n o wl e d g e a n d e x p eri e n c es r el at e d t o tr a u m a -i nf or m e d c ar e. Gl o b al Q u alit ati v e N ursi n g R es e ar c h , 4 , 2 3 3 3 3 9 3 6 1 7 7 3 4 5 1 0. htt ps:// d oi. or g/ 1 0. 1 1 7 7/ 2 3 3 3 3 9 3 6 1 7 7 3 4 5 1 0 St or k, B. R., A ks el b er g, N. J., Qi n, Y., & Mill er, D. C. ( 2 0 2 0). A d v ers e C hil d h o o d E x p eri e n c es ( A C Es) a n d c o m m u nit y p h ysi ci a ns: W h at w e' v e l e ar n e d. T h e P er m a n e nt e J o ur n al , 2 4 , 1 9. 0 9 9. htt ps:// d oi. or g/ 1 0. 7 8 1 2/ T P P/ 1 9. 0 9 9 S z a b o, S., Y os hi d a, M., Fil a k o vs z k y, J., & J u h as z, G. ( 2 0 1 7). " Str ess " is 8 0 y e ars ol d: Fr o m H a ns S el y e ori gi n al p a p er i n 1 9 3 6 t o r e c e nt a d v a n c es i n GI ul c er ati o n. C urr e nt P h ar m a c e uti c al D esi g n , 2 3 ( 2 7), 4 0 2 9 4 0 4 1. htt ps:// d oi. or g/ 1 0. 2 1 7 4/ 1 3 8 1 6 1 2 8 2 3 6 6 6 1 7 0 6 2 2 1 1 0 0 4 6 S zil a g yi, M., K er k er, B. D., St orf er -Iss er, A., St ei n, R. E., G ar n er, A., O' C o n n or, K. G., H o a g w o o d, K. E., & M c C u e H or wit z, S. ( 2 0 1 6). F a ct ors ass o ci at e d wit h w h et h er p e di atri ci a ns i n q uir e a b o ut p ar e nts' a d v ers e c hil d h o o d e x p eri e n c es. A c a d e mi c P e di atri cs , 1 6 ( 7), 6 6 8 6 7 5. htt ps:// d oi. or g/ 1 0. 1 0 1 6/j. a c a p. 2 0 1 6. 0 4. 0 1 3 T h o m ps o n -L ast a d, A., Y e n, I. H., Fl e mi n g, M. D., V a n N att a, M., R u bi n, S., S hi m, J. K., & B ur k e, N. J. ( 2 0 1 7). D efi ni n g tr a u m a i n c o m pl e x c ar e m a n a g e m e nt: S af et y -n et pr o vi d ers' p ers p e cti v es o n str u ct ur al v ul n er a bilit y a n d ti m e. S o ci al S ci e n c e & M e di ci n e ( 1 9 8 2) , 1 8 6 , 1 0 4 1 1 2. htt ps:// d oi. or g/ 1 0. 1 0 1 6/j.s o cs ci m e d. 2 0 1 7. 0 6. 0 0 3 A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 44 Ti n k, W., Ti n k, J. C., T uri n, T. C., & K ell y, M. ( 2 0 1 7). A d v ers e C hil d h o o d E x p eri e n c es: S ur v e y of r esi d e nt pr a cti c e, k n o wl e d g e, a n d attit u d e. F a mil y M e d i ci n e, 4 9 ( 1), 7 1 3. htt ps:// w w w.stf m. or g/ F a mil y M e di ci n e/ V ol 4 9Iss u e 1/ Ti n k 7 V oll h ar dt, L. T. ( 1 9 9 1). Ps y c h o n e ur oi m m u n ol o g y: A lit er at ur e r e vi e w. A m eri c a n J o ur n al of Ort h o ps y c hi atr y , 6 1 ( 1), 3 5 4 7. htt ps:// d oi. or g/ 1 0. 1 0 3 7/ h 0 0 7 9 2 2 6 W ei nr e b, L., S a v a g e a u, J. A., C a n di b, L. M., R e e d, G. W., Fl et c h er, K. E., & H ar gr a v es, J. L. ( 2 0 1 0). S cr e e ni n g f or c hil d h o o d tr a u m a i n a d ult pri m ar y c ar e p ati e nts. T h e Pri m ar y C ar e C o m p a ni o n t o t h e J o ur n al of Cli ni c al Ps y c hi atr y . htt ps:// d oi. or g/ 1 0. 4 0 8 8/ p c c. 1 0 m 0 0 9 5 0 bl u Z h a n, L., M o u, X., & Gill, M. ( 2 0 2 1). A d v ers e c hil d h o o d e x p eri e n c es ( A C Es): D e v el o p m e nt of a n A C Es k n o wl e d g e s c al e. J o ur n al of N ursi n g M e as ur e m e nt , 3 0 ( 3), 1-1 9. htt ps:// d oi. or g/ 1 0. 1 8 9 1/J N M- D- 2 0- 0 0 1 1 7 A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 45 A p p e n di x A K e y St a k e h ol d e rs a n d G a p A n al ysis Sit e C o nt a ct p ers o n F or m of d istri b uti o n C e d ar vill e U ni v ersit y Dr. A n g eli a Mi c kl e E m ail M ar g ar et M ar y Dr. N a n c y K e n n e d y E m ail H e alt h * N ot e: A p pr o v al d o c u m e nts a v ail a bl e u p o n r e q u est. N ot es T o A P Ns o n st aff T o all o ut p ati e nt cli ni c P C Ps G a p A n al ysis T o ol Pr oj e ct: Pri m ar y C ar e Pr o vi d ers A d v ers e C hil d h o o d E x p eri e n c e K n o wl e d g e, I m pl e m e nt ati o n, a n d P er c ei v e d B arri ers : A D N P pr oj e ct I n di vi d u al C o m pl eti n g F or m: Ell e n M. M c C all a B est P r a cti c e St r at e gi es H o w P r a cti c e Diff e rs f r o m B est P r a cti c e P e r c ei v e d B a r ri e rs P ri o r t o P r oj e ct E d u c at e c ar e t e a ms o n i m p a ct of A C Es f or p o p ul ati o ns s er v e d. I m pl e m e nt tr a u m ai nf or m e d h e alt h s yst e ms. P er e v al u ati o n wit h pr a cti c e st a k e h ol d ers A C Es ar e pri m aril y p er c ei v e d t o b e a p e di atri c iss u e. Littl e t o n o f or m al a p pli c ati o n of tr a u m a-i nf or m e d i niti ati v es i n cli ni c L a c k of a w ar e n ess, l a c k of e d u c ati o n, L a c k of a w ar e n ess, l a c k of e d u c ati o n, l a c k of pr a cti c al a p pli c ati o n. * N ot e: B est pr a cti c e str at e gi es i m pl e m e nt e d fr o m A m a y a-J a c ks o n et al., 2 0 2 1. A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 46 A p p e n di x B S W O T A n al ysis * See A p p e n di x C f or k e y st a k e h ol d ers I nt e r n al P ositi v e N e g ati v e St r e n gt hs S p e cifi cit y of p o p ul ati o n / w ell d efi n e d ni c h e ( Pri m ar y car e pr o vi d ers) Di v ers e cli ni ci a n s a m pli n g (i e p h ysi ci a ns, n urs e pr a ctiti o n ers, p h ysi ci a n assist a nts, et c as pri m ar y c ar e pr o vi d ers) A bilit y o f p arti ci p a nts t o f or w ar d s ur v e y t o ot h ers i e s n o w b all eff e ct E as y i nf or m e d c o ns e nt/ c o nfi d e nti alit y Aff or d a bl e A bilit y t o c oll e ct m ulti pl e f or ms of i nf or m ati o n ( mi x e d m et h o ds d esi g n) M ulti pl e st a k e h ol d ers * Si m pli cit y of s ur v e y S el e cti v e a n d o p e n e n d e d q u e sti o ns S p arsit y of d at a i n lit er at ur e W e a k n ess es S el e cti v e i niti al pr oj e ct sit es li miti n g g e n er ali z a bilit y of pr oj e ct D esir e f or s h ort er s ur v e y l e a di n g t o p ot e nti al i n a c c ur aci es of d at a c oll e cti o n S elf -r e p ort i n h er e ntl y c o ntri b ut es t o s a m pl e bi as I n di vi d u al p ers p e cti v e d o es n ot a d dr ess or g a ni z ati o n al p ers p e cti v es of li mit ati o ns t o A C E a p pli c ati o n i n pr a cti c e. Dis c ussi o n p osts vs e m ail l e n d ris k f or l o w s ur v e y r es p o ns e r at es D i v ersit y of sit es a n d s n o w b all eff e ct p ot e nti all y aff e cts a bilit y t o a c c ur at el y pr oj e ct r es p o ns e r at es. F e es r e q uir e d fr o m i n v esti g at or b y s o m e sit es P ot e nti al a m bi g uit y f or a p pli c ati o n of r es ults Mi x e d m et h o ds d esi g n i n di c ati v e of a d diti o n al r e q uir e m e nts f or ri g or/si g nifi c a n c e. E xt e r n al O p p o rt u niti es P ot e nti al f or i m pr o v e m e nt of d at a c oll e cti o n t o ols, s ur v e y, a n d a n al ysis. P ot e nti al f or a d diti o n al st a k e h ol d ers A bilit y t o e x p a n d o n d at a c oll e ct e d ( q u alit ati v e f e at ur e) Mi ni m al r e g ul ati o ns f or s ur v e y d a t a c oll e cti o n A dj ust m e nts t o q u esti o n n air e b y pr et esti n g s ur v e y s yst e m L e n gt h a n d d e v el o p m e nt of s ur v e y T h r e ats S ur v e y f ati g u e P ot e nti al f or l o w r es p o ns e r at e Ti m e c o nstr ai nts of pr oj e ct P ot e nti al f or t e c h ni c al iss u es L a c k of i nt er est i e s n o w b all eff e ct i n a d e q u at e f or t o pi c P ot e nti al f or bi as i n p arti ci p a nts P ot e nti al str u ct ur al c h a n g es i m p a cti n g distri b uti o n (i e st a k e h ol d er c o nt a ct l e a v es p ositi o n) Or g a ni z ati o n al d e ni al t o p arti ci p at e L a c k of k n o wl e d g e of t o pi c c o ntri b uti n g t o p ot e nti al c o nf usi o n of t er ms. Q u alit ati v e f e at ur e d e p e n d e nt o n p arti ci p a nts willi n g n ess A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 47 A p p p e n di x C P RI S M A di a g r a m I d e ntifi c ati o n P RI S M A 2 0 0 9 Fl o w Di a gr a m R e c or ds i d e ntifi e d t hr o u g h d at a b as e s e ar c hi n g (P u b M e d = 1 6 0 Ps y cI nf o = 1 5 9 C o c hr a n e = 2 1 4 CI N A H L = 4 0 ) A d diti o n al r e c or ds i d e ntifi e d t hr o u g h ot h er s o ur c es (n = 33) I n cl u d e d Eli gi bilit y S cr e e ni n g R e c or ds aft er d u pli c at es r e m o v e d (n = 417) R e c or ds s cr e e n e d ( n = 4 1 7) F ull -t e xt arti cl es ass ess e d f or eli gi bilit y (n= 96) Arti cl es i n cl u d e d i n lit er at ur e r e vi e w (1 = mi x e d m et h o ds 3 = o pi ni o n 0 = q u alit ati v e 9 = q u a ntit ati v e/r e vi e w) R e c or ds e x cl u d e d pri m ari l y d/t u nr el at e d c o nt e nt ( n = 3 2 1) F ull -t e xt arti cl es e x cl u d e d, wit h r e as o ns ( St u d y i n pr o gr ess = 4 N o nr el at e d pr o vi d er i. e. p e di atri ci a n, ps y c hi atrist, r es e ar c h er, et c. = 3 9 D o es n ot ass ess cli ni ci a n p ers p e cti v e = 3 0 Ot h er * = 1 0 ) * Ot h er i n cl u d es: N ot r el at e d t o A C Es, p oli c y e v al u ati o ns, or st u di es c o n c er ni n g r e p orti n g c hil d a b us e, I P V, et c. A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 48 A p p e n di x D Lit e r at u r e R e vi e w M at ri x Lit er at ur e r e vi e w: A p pli e d A C Es r es e ar c h i n Pri m ar y C ar e St u d e nt N a m e: Ell e n M c C all a R ef e r e n c e R es e a r c h D esi g n & L e v el of E vi d e n c e T h e o r eti c al / C o n c e pt u al Fr a me w or k P u r p os e / Ai m P o p ul ati o n / S a m pl e n =x V a ri a bl e s I nst r u m e nt s / D at a c oll e cti o n R es ults I m pli c ati o n s f o r f ut u r e r es e a r c h I m pli c ati o n s f o r f ut u r e p r a cti c e B o d e n d orf er et al., 2 0 2 0 Cr oss s e cti o n al q u esti o n n air e L e v el 4 ACE c o n v ers ati o n T o e x a mi n e p ar e nt/ g u ar di a n a n d pr o vi d er a c c e pt a bilit y of A C E c o n v ers ati o n d uri n g w ell c hil d visits i n pri m ar y c ar e 2 3 1 p h ysi ci a n assist a n ts, r esi d e nts a n d pr o vi d ers. 7 1 % w er e at t e n di n gs. P ers p e cti v e, c o mf ort, pr e p ar ati o n, m oti v ati o n , v al u e, i m p a ct o n c ar e. N O ACE S C R E E NI N G. 8 q u esti o n s ur v e y t o e x a mi n e a c c e pt a bilit y a n d f e asi bilit y of A C E c o n v ers ati o n. P C Ps d esir e r es o ur c es f or p ar e nt s. R es e ar c h t h at ACE c o n v ers ati o n miti g at es ris k. E x pl or at or y cr oss -s e cti o n al st u d y L e v el 4 C oll a b or ati v e c ar e N =99 att e n di n g p h ysi ci a ns a n d r esi d e nts K n o wl e d g e, tr ai ni n g, s cr e e ni n g pr a cti c es, p er c ei v e d i nt er v e nti o n b arri ers S ur v e y I nt er-cli ni ci a n v ari a bilit y e xists, f e w f or m al tr ai ni n gs, e xists a n d li mit e d f a mili arit y e xists. D e v el o p t o ols a n d tr ai ni n g f or pr o vi d ers. C a m p b ell, 2020 E x p ert o pi ni o n L e v el 7 Tr a u m a i nf or m e d c are E x pl or e P C Ps A CEs k n o wl e d g e tr ai ni n g, s cr e e ni n g pr a cti c es a n d p er c ei v e d i nt er v e nti o n b arri ers, i n cl u di n g i nt er-c li ni ci a n v ari a bilit y. C a uti o n a g ai nst r o uti n e s cr e e ni n g An A C E c o n v ers ati o n c o ul d a v oi d dis c o mf ort. D o n ot ass u m e pr o vi d ers h a v e ACE k n o wl e d g e. E d u c at e all st aff. E d u c ati o n al t o ols, effi ci e nt m o d els, a n d c oll a b or ati v e c ar e n et w or ks. B or a et al., 2021 n = 1 f a mil y m e di ci n e M D n/ a n/ a R o uti n e s cr e e ni n g f or A C Es s h o ul d n ot b e c o n d u ct e d. M or e e vi d e n c e o n r o uti n e s cr e e ni n g f or A C Es. F or d e t al., 2019 S c o pi n g R e vi e w L e v el 5 Ar ks e y a n d O M all e y Fr a m e w or k f or s c o pi n g r e vi e ws N = 4 st u di es a d dr essi n g pr o vi d ers a d mi nist eri n g r o uti n e e n q uir y S etti n g f or r o uti n e e n q uir y, pr o c ess f or e n q uir y a n d s er vi c e i m pli c ati o ns, a c c e pt a bilit y a n d o ut c o m es of e n q uir y CI N A H L, M E D LI N E, a n d Ps y cI N F O fr o m 1 9 9 72018 I n g e n er al, willi n g n ess t o e n q uir e E vi d e n c e b as e is li mit e d, e x pl or e w h o e n q uir y s h o ul d t ar g et, t o ols t h at ar e a p pr o pri at e a n d s etti n gs t h at ar e a p pr o pri at e. Gl o w a et al., 2016 F e asi bilit y st u d y L e v el 6 A C Es r es e ar c h T o e x pl or e t h e e vi d e n c e b as e f or r etr os p e cti v e r o uti n e e n q uir y i n a d ults f or A C Es i n cl u di n g f e asi bilit y a n d a c c e pt a bilit y a m o n gst pr a ctiti o n ers, s er vi c e us er a c c e pt a bilit y , a n d o ut c o m es . T o e x pl or e t h e f e asi bilit y of i m pl e m e nti n g t he A C E s cr e e ni n g of a d ults d uri n g r o uti n e f a mil y N = 111 cli ni ci a n s ur v e ys I nf or m ati o n a b o ut p ati e nt, c h a n g e i n c ar e pl a n, c h a n g e i n r ef err als, i d e ntifi c ati o n of iss u es, a n d S ur v e ys 100 % A C E s ur v e y di d n ot i nt erf er e wit h visit, p er c ei v e d a c c e pt a bilit y t o p ati e nt: 9 8 %, Rec o m me n d li n ki n g p ositi v e A C E s c or es wit h s cr e e ni n g f or c urr e nt h e alt h ris k b e h a vi ors M a k e s ur e k e y crit eri a f or h e alt h s cr e e ni n gs h a v e b e e n m et b ef or e i m pl e m e nti n g. C a uti o n o n r o uti n e e n q uir y, s h o ul d b e m ulti -f a c et e d a n d tr a u m a i nf or m e d. I nt er v e nti o ns to a d dr ess A C E ris ks is p ossi bl e f or a pri m ar y c ar e s etti n g. A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S J o n es et al., 2020 E x p ert o pi ni o n L e v el 7 Tr a u m a i nf or m e d c ar e K al m a kis et al., 2 0 1 7 Mi x e d m et h o d c r oss-s e cti o n al c orr el ati o n al d esi g n L e v el 4 N ot st at e d M a u n d er et al., 2020 C o n v e ni e nt o nli n e s ur v e y L e v el 6 A C Es r es e ar c h M c L e n n a n et al., 2 0 2 0 L ett er t o E dit or L e v el 7 C o ns oli d at e d pri n ci pl es f or s cr e e ni n g R a vi & Littl e, 2017 C as e St u d y L e v el 6 Tr a u m a i nf or m e d c ar e S h erfi ns ki et al., 2 0 2 1 S c o pi n g r e vi e w L e v el 5 A C Es r es e ar c h St or k et al., 2020 C o n v e ni e n c e sur v e y L e v el 6 Tr a u m a i nf or m e d c ar e m e di ci n e offi c e visits I d e ntif yi n g a n d pr e v e nti n g A C Es T o e x a mi n e N Ps pr a cti c es s kills attit u d es a n d p er c ei v e d b arri ers t o s cr e e ni n g a d ult p ati e nts T o d et er mi n e if s cr e e ni n g is r el at e d t o k n o wl e d g e or m e di c al s p e ci alt y a n d t o ass es p er c ei v e d b arri ers E x pr ess c o n c er n t o t h e r e c o m m e n dati onto i m pl e m e nt A C Es i n cli ni c al pr a cti c e. Ho wto s e nsiti v el y a d dr ess a p ati e nt wit h si g nifi c a nt s e x u al a b us e h ist or y. E x a mi n e h o w c urr e nt lit er at ur e d es cri b es t h e p er c e pti o ns attit u d es, a n d pr a cti c es of h e alt h pr of essi o n als a n d tr ai n e es r e g ar di n g c hil d h o o d tr a u m a T o s ur v e y a c o n v e ni e n c e s a m pl e of c o m m u nit y b as e d p h ysi ci a ns a n d r esi d e nt p h ysi ci a ns t o ass ess f or f a mili arit y N = 3 a p h ar m a cist, M D, a n d P h D mi n ut es a d d e d t o visit. n/ a n/ a mi ni m al ti m e a d d e d t o visit. R e c o g ni z e t h e r ol e of cli ni ci a ns i n pr e v e nti o n of A C E s an d i m pl e m e nt ati o n of tr a u m a i nf or m e d c ar e. 49 Pr os p e cti v e, l o n git u di n al st u di es t o e x a mi n e t h e ass o ci ati o n of pr e v e nti n g A C Es a n d s p e cifi c h e alt h o ut c o m es a n d fi n a n ci al s a vi n gs. Rec o m me n d d e v el o p m e nt of s cr e e ni n g t o ol D at a s h ari n g, t e a m-b as e d a p pr o a c h, a d dr ess u n d erl yi n g tr a u m a t hr o u g h str at e gi es s u c h as c o g niti v e b e h a vi or al t h er a p y. N = 1 8 8 n urs e pr a ctiti o n ers K n o wl e d g e, s cr e e ni n g, b arri er s S ur v e y Ti m e a n d dis c o mf ort as b arri ers Rec o m me n d s cr e e ni n g i n pri m ar y c ar e N = 8 9 f a mil y p h ysi ci a ns, 4 8 ot h er s p e ci alists a n d 46 ps y c hi atrists D e m o gr a p hi cs, k n o wl e d g e, us u al pr a cti c e, a n d p er c ei v e d b arri ers O nli n e s ur v e y R el ati o ns hi p b et w e e n s p e ci alt y a n d k n o wl e d g e of A C Es as w ell as s cr e e ni n g pr a cti c es T est t h e ass u m pti o n t h at s cr e e ni n g a d ult p ati e nts f or A C Es i m pr o v es p ati e nt o ut c o m es E n c o ur a g e e d u c ati o n . N = 5 a ut h ors, 2 p e di atri ci a ns a n d r e m ai ni n g w er e s o ci al s ci e n c e pr of essi o n als n/ a n/ a S p e cifi c a n d r el e v a nt A C E q u esti o ns c a n b e i nt e gr at e d as n e e d e d i nt o hist or y t a ki n g. S yst e m ati c all y f or m a n e vi d e n c e b as e t o i nf or m s cr e e ni n g of A C Es N = 1 pri m ar y c ar e pr o vi d ers e x p eri e n c e wit h a n a d ult p ati e nt wit h hist or y of c hil d h o o d tr a u m a N = 1 7 st u di es fr o m v ari o us dis ci pli n es Si g ns or s y m pt o ms a n d a p pr o a c h es t o c ar e n/ a Tr a u m a i nf or m e d c ar e a cti o ns f or p h ysi ci a ns B est pr a cti c e g ui d eli n e d e v el o p m e nt K n o wl e d g e, attit u d es a n d p er c ei v e d b arri ers t o w ar d A C Es. S cr e e ni n g, E d u c ati o n al o p p ort u niti es , I m pl e m e nt ati o n str at e gi es, a n d s cr e e ni n g c a uti o ns Pri m aril y P u b me d A l ar g e pr o p orti o n of pr o vi d ers ar e u n a w ar e of t h e eff e cts of A C Es. Tr ai ni n g is g o o d, b ut n ot r es e ar c h b as e d. B arri ers ar e v ari e d. D e v el o p A C E e d u c ati o n n t h at is s ust ai n a bl e a n d e n d uri n g. R es e ar c h i m p a ct of s cr e e ni n g. W h at is t h e c ost b ur d e n r el at e d t o A C Es/ c ar e ? S e nsiti v el y a n d s af el y c o nsi d er t h e n e e ds of t h e p ati e nt wit h dis cl os ur e of c o nfi d e nti alit y li mits Tr a u m a i nf or m e d c ar e t o pro vi d e s e nsiti v e, i n di vi d u ali z e d c ar e fr o m t h eir f a mil y p h ysi ci a n. E d u c at e. S cr e e n if t h e a p pr o pri at e r es o ur c es ar e i n pl a c e. A d v o c at e f or a p pr o pri at e r es o ur c es. N = 226 p h ysi ci a ns pl us r esi d e nts a n d m e di c al st aff T h e 1 0 p oi nt ACE q u esti o n n air e, 6 b asi c d e m o gr a p hi c q u esti o ns, a n d 2 q u esti o ns a b o ut f a mili arit y wit h a n d us e of S ur v e y Q u altri cs 8 0. 5 % h a d n o k n o wl e d g e of A C Es, 3. 5 % us e d t h e A C E q u esti o n n air e i n pr a cti c e , 5 5. 5 % of p h ysi ci a ns r e p ort e d n o W h at ar e b est pr a cti c es f or s cr e e ni n g a n d ho wto c o or di n at e c ar e f or p ati e nts E d u c at e. B e c o m e a p art of a tr a u m a i nf or m e d c ar e t e a m. A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S W ei nr e b et al., 2010 Cr oss -s e cti o n al s ur v e y L e v el 4 A C E r es e ar c h * F elitti et al., 1998 R etr os p e cti v e and pr os p e cti v e c o h ort L e v el 4 Tr a u m a i nf or m ed c ar e wit h a n d us e of t he A C E q u esti o n n air e i n cli ni c al pr a cti c e a n d t o m e as ur e t h e pr e v al e n c e of t h eir o w n A C Es. T o d es cri b e t h e pr a cti c es, s kills, attit u d es, a n d p er c ei v e d b arri ers of a l ar g e s a m pl e of f a mil y p h ysi ci a ns i n s cr e e ni n g a d ult p ati e nts f or c hil d h o o d s e x u al or p h ysi c al a b us e T o d es cri b e t h e l o n g-t er m r el ati o ns hi p of c hil d h o o d e x p eri e n c es t o i m p ort a nt m e di c al a n d p u bli c h e alt h pr o bl e ms A C Ein pr a cti c e 50 c u m ul ati v e A C E s c or e N =833 p h ysi ci a ns Pr a cti c es, s kills, attit u d es, b arri ers S ur v e y N = 9, 5 0 8 a d ult p ati e nts Se ve n c at e g ori es of a d v ers e c hil d h o o d e x p eri e n c es Q u esti o n n air e C o nfi d e n c e i n s cr e e ni n g, p er c ei v e d r ol e, a n d k n o wl e d g e of tr a u m a pr e v al e n c e w er e ass o c i at e d wit h r o uti n e/t ar g et e d s cr e e ni n g. Wo men a nd p h ysi ci a ns r e p orti n g f e w er b arri ers w er e m or e li k el y t o r o uti n el y s cr e e n a d ult p ati e nts. Str o n g gr a d e d c orr el ati o n b et w e e n c hil d h o o d a d v e rsit y a n d a d ult dis e as e D e v el o p tr ai ni n g pr o gr a ms. L e ar n eff e cti v e pri m ar y c ar e a p pr o a c h es t o a d dr ess p ati e nts wit h A C E b ur d e n. P h ysi c al a n d b e h a vi or al eff e cts of A C Es o n p h ysi ci a ns E d u c at e p h ysi ci a n c o m m u nit y a b o ut t h e h e alt h a n d b e h a vi or al s e q u el a e of c hil d h o o d tr a u m a. * N ot a n arti cl e r e g ar di n g a p pli c ati o n of A C Es i n a d ult pri m ar y c ar e s u m m ar y of t h e ori gi n al r es e ar c h A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S A p p e n di x E M o d el f o r T r a u m a-I nf o r m e d P ri m a r y C a r e ( R o b erts et al., 2 0 1 9) 51 A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 52 A p p e n di x F S ur ve y A d ult P ri m a r y C a r e P r o vi d e rs A d v e rs e C hil d h o o d E x p e ri e n c e (A C E) K n o wl e d g e, I m pl e m e nt ati o n, a n d P e r c ei v e d B a r ri e rs : A D N P p r oj e ct. T h e p ri m a r y p u r p os e of t his q u esti o n n ai r e is t o d et e r mi n e p ri m a r y c a r e p r o vi d e rs le v el of f a mili a rit y a n d utili z ati o n of A C E r es e a r c h i n t h ei r p r a cti c e. Y o u a r e b ei n g as k e d t o ta k e p a rt i n t his p r oj e ct t o i nf o r m c u r r e nt u n d e rst a n di n g of this r es e a r c h int o t h e p ri m a r y c a r e s etti n g. P a rti ci p ati o n is v ol u nt a r y a n d t h e s u r v e y c a n b e e xit e d at a n y ti m e wit h n o c o ns e q u e n c es t o y o u rs elf. N o m o r e t h a n 2 0 diff e r e nt t y p es of q u esti o ns will b e p os e d. Pl e as e a ns w e r t o t h e b est of y o u r a bilit y. S o m e of t h e q u esti o ns m a y b e s e nsiti v e i n n a t u r e b ut d at a is a n o n y mi z e d b y t h e s u r v ey t o ol a n d o nl y li n k e d t o y o u r e m ail/ p h o n e n u m b e r wit h y o u r c o ns e nt. Pl e as e e m ail p ri n ci p al i n v esti g at o r Ell e n M c C all a at e m o o r e 8 4 7 @ m a ri a n. e d u w it h a d diti o n al q u esti o ns. B y cli c ki n g n e xt y o u a g r e e t o p a rti ci p at e i n t his s u r v e y: * N ot e: T his s u r v e y h as b e e n a d a pt e d a n d us e d wit h p e r missi o n b y B o r a et al. (2 0 2 1). F or cl arit y, a n a d v ers e c hil d h o o d e x p eri e n c e is a tr a u m ati c e x p eri e n c e i n a h u m a ns lif e b ef or e 1 8 y e ars of a g e. T h es e i n cl u d e: P h ysi c al a b us e S e x u al a b us e E m oti o n al a b us e M e nt al ill n ess of h o us e h ol d m e m b er Pr o bl e m ati c dri n ki n g or al c o h olis m of h o us e h ol d m e m b er Ill e g al str e et or pr es cri pti o n dr u g us e b y a h o us e h ol d m e m b er Di v or c e or s e p ar ati o n of a p ar e nt D o m esti c vi ol e n c e t o w ar ds a p ar e nt I n c ar c er ati o n of a h o us e h ol d m e m b er 1. H o w f a mili ar ar e y o u wit h t h e A d v ers e C hil d h o o d E x p eri e n c es ( A C E) St u d y, a n o n g oi n g st u d y c o n d u ct e d b y t h e C e nt er f or Dis e as e C o ntr ol a n d Pr e v e nti o n ( C D C) a n d K ais er P er m a n e nt e t o ass ess t h e ass o ci ati o ns b et w e e n c hil d h o o d str ess ors an d l at er -lif e h e alt h a n d w ell-b ei n g ? o V er y f a mili ar (I c o ul d d es cri b e it f airl y a c c ur at el y) o S o m e w h at f a mili ar (I h a v e a g e n er al c o n c e pt of t h e c o nt e nt) o V a g u el y f a mili ar (I t hi n k I v e h e ar d of it, b ut I m n ot s ur e I k n o w t h e d et ails) o N ot at all f a mili ar ( I h a v e nt h e ar d of it) 2. H a v e y o u r e c ei v e d f or m al tr ai ni n g o n tr a u m a i nf or m e d c ar e or a d v ers e c hil d h o o d e x p eri e n c es ? Y es No A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 53 3. H a v e y o u e v er d o n e A C E s cr e e ni n g wit h y o ur p ati e nts ? Y es, I h a v e us e d a f or m al A C E s cr e e ni n g t o ol Y es, I h a v e i n q uir e d a b o ut A C E r is k f a ct ors No U ns ur e 4. If y es, w h o h a v e y o u s cr e e n e d f or A C Es (s el e ct all t h at a p pl y) ? P ar e nts of p e di atri c p ati e nts a b o ut t h eir o w n A C Es P ar e nts of p e di atri c p ati e nts a b o ut t h eir c hil ds A C Es C hil dr e n a b o ut t h eir o w n A C Es A d ult p ati e nts a b o ut t h eir o w n A C Es Ot h er ( pl e as e s p e cif y) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 5. Pl e as e s el e ct t h e r es p o ns e(s) y o u p er c ei v e as b arri ers t o y o ur a bilit y as a pri m ar y c ar e pr o vi d er t o a d dr ess A C Es (s el e ct all t h at a p pl y): I d o n ot p er c ei v e a n y b arri ers t o a d dr ess A C Es I n a d e q u ate ti m e L a c k of l o c al r es o ur c es L a c k of c ar e c o or di n ati o n s er vi c es t o li n k p ati e nts/f a mili es wit h c o m m u nit y r es o ur c es Dis c o mf ort i n as ki n g s e nsiti v e q u esti o ns P ar e nt al r el u ct a n c e t o dis c uss s e nsiti v e/ p ers o n al t o pi cs L a c k of tr ai ni n g i n m a n a gi n g a c hil d w h o h as e x p eri e n c e d a d v ers e c hil d h o o d tr a u m a I n a d e q u at e r ei m b urs e m e nt Ot h er ( pl e as e s p e cif y) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 6. E x p os ur e t o si g nifi c a nt a d v ersit y i n c hil d h o o d is ass o ci at e d wit h t h e f oll o wi n g h e alt h o ut c o m es l at er i n lif e: O b esit y Li v er dis e as e C a n c er L u n g dis e as e S e x u all y tr a ns mitt e d i nf e cti o ns H e art dis e as e Di a b et es A gr e e A gr e e A gr e e A gr e e A gr e e A gr e e Dis a gr e e Dis a gr e e Dis a gr e e Dis a gr e e Dis a gr e e Dis a gr e e 7. I n g e n er al, h o w diffi c ult is it f or p ati e nts w h o n e e d r o uti n e or e m er g e n c y m e nt al h e alt h s er vi c es t o b e s e e n b y m e nt al h e alt h pr o vi d ers i n y o ur c o m m u nit y ? V er y diffi c ult S o m e w h at diffi c ult R o uti n e E m er g e n c y N ot diffi c ult 8. F or e a c h of t h e f oll o wi n g c at e g ori es, d o es y o ur pr a cti c e s etti n g h a v e pr ot o c ols i n pl a c e f or cli e nts w h o h a v e e x p eri e n c e d tr a u m a or vi ol e n c e ? Y es S cr e e ni n g Ass ess m e nt R ef err al Tr e at m e nt No A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 54 9. Pl e as e m ar k w hi c h (if a n y) of t h e f oll o wi n g pr o bl e ms/ c o n diti o ns y o u r o uti n el y i n q uir e a b o ut: C hil d d e pr essi o n A d ol es c e nt d e pr essi o n B e h a vi or pr o bl e ms B ull yi n g A n xi et y dis or d ers S u bst a n c e us e D o m esti c vi ol e n c e e x p os ur e P h ysi c al, s e x u al, or e m oti o n al a b us e N e gl e ct P ar e nt al m e nt al ill n ess P ar e nt al s u bst a n c e a b us e I n c ar c er at e d c ar e gi v er N ei g h b or h o o d vi ol e n c e 1 0. H o w k n o wl e d g e a bl e ar e y o u a b o ut t h e eff e ct of tr a u m ati c e arl y c hil d h o o d e x p eri e n c es o n c hil d w ell b ei n g a n d a d ult o ut c o m es ? N ot k n o wl e d g e a bl e S o m e w h at k n o wl e d g e a bl e K n o wl e d g e a bl e V er y k n o wl e d g e a bl e 1 1. W o ul d y o u b e n e fit fr o m tr ai ni n g o n A C Es ? Y es No 1 2. W hi c h f or m at w o ul d b e b est f or y o ur pr a cti c e s etti n g i n w hi c h t o l e ar n m or e a b o ut A C Es ? Att e n d s e mi n ar/ w or ks h o p L e ct ur e pr es e nt ati o n R e c ei v e e d u c ati o n al m at eri als O n -li n e c o nti n ui n g e d u c ati o n m o d ul es Ot h er ( pl e as e s p e cif y) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 1 3. W h at A C E -r el at e d r es o ur c es w o ul d y o u fi n d us ef ul i n y o ur pr a cti c e (s el e ct all t h at a p pl y) ? P ati e nt h a n d o uts P ar e nt h a n d o uts C hil d/ y o ut h r es o ur c es T e c h n ol o g y r es o ur c es ( w e bsit es, Q R c o d es, h el pli n es) Vi d e o/ P o w er p oi nt f or pr es e nt ati o n " T o ol kit " wit h m at eri als, i d e as, a n d r es o ur c es Tr ai ni n g i n A C Es s cr e e ni n g i nstr u m e nts E vi d e n c e -b as e d r es o ur c es C o m m u nit y r es o ur c es Ot h er ( pl e as e s p e cif y) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 1 4. W h at is y o ur ar e a of s p e ci alit y (s el e ct all t h at a p pl y) ? F a mil y m e di ci n e P e di atri cs I nt er n al m e di ci n e A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 55 W o m e ns H e alt h G er o nt ol o g y Ot h er ( pl e as e s p e cif y) _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 1 5. W h at is y o ur p ositi o n ( S el e ct all t h at a p pl y) ? N urs e pr a ctiti o n er P h ysi ci a n assist a nt R esi d e nt p h ysi ci a n Att e n di n g p h ysi ci a n F a c ult y/ A c a d e mi c st aff A d mi nistr ati o n 1 6. Pr a cti c e l o c ati o n: Ur b a n S u b ur b a n R ur al/ S m all t o w n 1 7. W h at is y o ur g e n d er ? M al e F e m al e Ot h er: _ _ _ _ _ _ _ _ _ _ _ _ _ 1 8. W h at is y o ur a g e ? 29 3 0 -3 9 4 0 -4 9 5 0 -5 9 60 1 9. W o ul d y o u b e willi n g t o v ol u nt e er f or a s h ort p h o n e i nt er vi e w t o c oll e ct q u alit ati v e d at a r e g ar di n g A C E r es e ar c h a n d b arri ers t o i m pl e m e nt ati o n i n pr a cti c e ? Y es No 2 0. If y es, pl e as e pr o vi d e n a m e, c o nt a ct n u m b er a n d e m ail a d dr ess. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ N O T E: i n pr o vi di n g y o ur i nf or m ati o n y o u a gr e e t o b e c o nt a ct e d b y Ell e n M c C all a f o r s ai d i nt er vi e w. Y o ur i nf or m ati o n will b e e n cr y pt e d, pr ot e ct e d, a n d n e v er us e d f or a n y a d diti o n al us e o utsi d e of c o nt a cti n g f or a d diti o n al d at a c oll e cti o n. It is n ot n e c ess ar y t o pr o vi d e t his i nf or m ati o n i n or d er f or y o ur f or m er r es p o ns es t o b e i n cl u d e d in t h e d at a c oll e cti o n f or t his D N P pr oj e ct. * Us e d wit h p er missi o n fr o m B or a et al., 2 0 2 1; K al m a kis et al., 2 0 1 7 A D U L T P RI M A R Y C A R E P R O VI D E R S A D V E R S E C HI L D H O O D E X P E RI E N C E S 56 A p p e n di x G Q u alit ati v e Q u esti o ns Dis cl os ur e a n d p er missi o ns: Y o u will b e as k e d a s et of ei g ht q u esti o ns r e g ar di n g y o ur e x p eri e n c e, k n o wl e d g e, a n d utili z ati o n of A C E r es e ar c h i n y o ur pr a cti c e. T h es e q u esti o ns ar e fr o m a s cri pt a n d t h e o nl y a d diti o n al q u esti o ns m a y b e a cl arifi c ati o n t o a r es p o ns e gi v e n. Y o u h a v e t h e ri g ht t o r ef us e t o a ns w er a n y q u esti o n a n d t h er e is n o c o ns e q u e n c e t o wit h dr a wi n g fr o m t his i n q uir y at a n y ti m e. O ur c o n v ers ati o n is r e c or d e d a n d will l at er b e c o n v ert e d t o t e xt wit h p orti o ns p u blis h e d i n a d o ct or al p a p er. I n a d diti o n, y o ur i d e ntif yi n g i nf or m ati o n will n ot b e att a c h e d t o t h e d at a i n a n y w a y - d o y o u c o ns e nt t o off e r y o ur p ers p e cti v e f or m y pr oj e ct ? 1. T ell m e a b o ut y o ur k n o wl e d g e of A C E s c or es. T his c a n i n cl u d e t h e d e gr e e t o w hi c h y o u ar e f a mili ar wit h t h e ori gi n al r es e ar c h st u d y, t h e i nf or m ati o n y o u h a v e l e ar n e d a b o ut i n pr a cti c e, or h o w f a mili ar y o u ar e wit h A C Es i n g e n er al. 2. W h at is y o ur p ers p e cti v e o n t h e i m p a ct of t h e A C Es i nf or m ati o n y o u h a v e t h us f ar ? 3. W h at d o y o u p er c ei v e as t h e i m p a ct t h at A C Es h a v e o n y o ur p o p ul ati o n of cli e nts ? 4. W h at h as b e e n y o ur e x p eri e n c e as a pr o vi d er i n pr o vi di n g c ar e t o a d ults wit h tr a u m a/ hi g h A C E s c or es ? 5. C a n y o u gi v e m e a n e x a m pl e of h o w A C Es i m p a ct y o ur c ar e as a pri m ar y c ar e pr o vi d er ? 6. C o nsi d eri n g A C E r es e ar c h, w h at d o y o u b eli e v e is i m p ort a nt f or y o ur r ol e as a pri m ar y c ar e pr o vi d er ? 7. If y o u c o ul d dr e a m, w h at w o ul d y o u w a nt t o a d dr ess A C E s c or es i n y o ur p ati e nt p o p ul ati o n ? 8. W h at ot h er c o m m e nts w o ul d y o u li k e t o off er ?  ...

... 1 The Identification and Evaluation of the Knowledge and Skills in a Professional Selling Program at a Small University A Capstone Project Presented to the Faculty of the Fred S. Klipsch Educators College Marian University _______________________________________ In Partial Fulfillment of the Requirements for the Degree Doctor of Education in Organizational Leadership _______________________________________ Lori Rumreich May 26, 2022 2 Fred S. Klipsch Educators College Marian University Indianapolis, Indiana APPROVAL OF THE CAPSTONE PROJECT This capstone project, The Identification and Evaluation of the Knowledge and Skills in a Professional Selling Program at a Small University, has been approved by the Graduate Faculty of the Fred S. Klipsch Educators College in partial fulfillment of the requirements for the degree of Doctor of Education. Chair, Jeffery Kaufman, Ph.D Committee Member Robert Schuttler, Ph.D. Committee Member Robert Kubacki, MBA 3 Table of Contents List of Tables .................................................................................................................................. 8 List of Figures ............................................................................................................................... 10 Executive Summary ...................................................................................................................... 11 Chapter 1: Introduction ................................................................................................................. 13 Background ............................................................................................................................... 13 State of Higher Ed: Enrollment Challenges, Financial Challenges, and Industry Needs ...... 13 Employment for Business Graduates .................................................................................... 14 Sales Education in U.S. Higher Education ............................................................................ 15 Problem of Practice ................................................................................................................... 16 The Status of Sales Education in Business................................................................................ 18 Significance and Purpose of the Project .................................................................................... 19 Significance of the Project ..................................................................................................... 19 Purpose of the Project ............................................................................................................ 20 System Diagnosis ...................................................................................................................... 21 Needs Assessment ................................................................................................................. 21 Stakeholders, their Loyalties, and Interactions to Understand their Positions ...................... 23 Conceptual Framework ............................................................................................................. 23 Research Question ..................................................................................................................... 24 Limitations and Delimitations ............................................................................................... 24 4 Definition of Terms ............................................................................................................... 25 Chapter 2: Literature Review ........................................................................................................ 28 General Background of Sales and Sales Programs ................................................................... 28 Market Trends for the Sales Profession and University Sales Education ............................. 28 Why Students Study Sales ..................................................................................................... 32 Industry Partnerships ............................................................................................................. 33 Sales Program Development ..................................................................................................... 36 How to Run, Manage, and Grow a Sales Program ................................................................ 36 Sales Program Curriculum Development .............................................................................. 39 Expected Knowledge and Skills and Typical Pedagogy in Sales Education ........................ 45 Sales Technology and Ethics in Professional Selling Programs ........................................... 49 Evaluation of Sales Education Programs within Business Schools .......................................... 52 Accrediting Agencies for evaluation Sales Program in Higher Education ........................... 52 Other Examples of Sales Program Evaluation in Higher Education ..................................... 54 Chapter 3: Methodology ............................................................................................................... 58 Methodology Part 1: Benchmarking best practice universities ............................................. 59 Methodology Part 2: Qualitative inquiry for program development ..................................... 60 Methodology Part 3: Quantitative survey of program students ............................................. 62 Methodology Part 4: Program evaluation .............................................................................. 63 5 Sampling ................................................................................................................................ 65 Research Instruments and Data Collection ............................................................................ 65 Data Analysis Plan .................................................................................................................... 66 Part 1: Benchmarking best practice universities.................................................................... 66 Part 2: Qualitative inquiry for program development ........................................................... 66 Part 3: Quantitative survey of program ................................................................................. 67 Part 4: Program evaluation .................................................................................................... 67 Chapter 4: Study Results ............................................................................................................... 70 Sales Program Design and Implementation, Parts 1 through 3 ................................................. 70 Program Design Part 1 Results: Benchmarking Similar, Regional, and Aspirational Programs ................................................................................................................................ 70 Sales Program Design Part 2 Results: Qualitative Inquiry .................................................... 73 Sales Program Curriculum Development and Implementation ............................................. 74 Sales Program Design Part 3 Results: Quantitative Survey of Students ............................... 75 Sales Program Evaluation, Part 4 of the Methodology ............................................................. 86 Level 1 Reaction .................................................................................................................... 87 Level 2 Learning .................................................................................................................... 89 Level 3 Behavior ................................................................................................................... 91 Level 4: Results ..................................................................................................................... 95 Chapter 5: Discussion and Evaluation of the Capstone Project .................................................... 97 6 Summary ................................................................................................................................... 97 Summary of the Project and Significance of the Outcome ................................................... 97 Discussion of the Knowledge and Skill Requirements Quantitative Survey ........................ 99 Discussion of the Program Evaluation Results.................................................................... 101 Limitations ........................................................................................................................... 107 Evaluation of the Intervention ............................................................................................. 107 Implications and Reflection ................................................................................................. 108 Future Research ................................................................................................................... 109 References ................................................................................................................................... 111 Appendix A ................................................................................................................................. 121 USCA Criteria for Membership .............................................................................................. 121 Appendix B ................................................................................................................................. 122 Research Instruments .............................................................................................................. 122 Qualitative Interview Guide ................................................................................................ 122 Survey Instrument for Sales Students .................................................................................. 124 Appendix C ................................................................................................................................. 126 Rubrics .................................................................................................................................... 126 Level 1: Reaction, Student Evaluation of Instruction Questions......................................... 126 Level 2: Learning, Role Play Rubric ................................................................................... 127 7 Level 3: Behavior/Transfer, Supervisor Internship Evaluation ........................................... 129 Appendix D ................................................................................................................................. 130 Display Matrix of Knowledge, Skills, and Pedagogy Findings .............................................. 130 Display Matrix of Knowledge, Skills, and Pedagogy Findings .............................................. 131 Course Framework for Knowledge and Skills ........................................................................ 132 Course Framework with Pedagogy Use .................................................................................. 132 8 List of Tables Table 1: Key Milestones in Sales Education ................................................................................ 30 Table 2: Number of Students Participating in Sales at Universities Offering a Sales Major ....... 30 Table 3: Common Sales Program Types at USCA Member Universities .................................... 42 Table 4: Stetson Universitys Sales Major and Minor Course List .............................................. 44 Table 5: Winona State Universitys Sales Courses for a Minor ................................................... 45 Table 6: Sales Practitioner Skill Inventory Categorized as Critical, Important, and Useful ........ 46 Table 7: Sales Practitioner Pedagogy Inventory Categorized as Critical, Important, and Useful 48 Table 8: Summary of Business School Accreditation Agencies .................................................. 52 Table 9: Mapping Course Components for Experiential Learning and Blooms Taxonomy ....... 55 Table 10: Sales Program Evaluation Model using the Adapted Kirkpatrick Four Levels............ 64 Table 11: Sales Program Characteristics ...................................................................................... 71 Table 12: Knowledge and Skills Outcomes at Comparison Schools ............................................ 72 Table 13: Sales Course Types Offered by Program...................................................................... 73 Table 14: Sampling Data from Student Survey ............................................................................ 76 Table 15: Survey Participant Demographics ................................................................................ 76 Table 16: Sales Courses Compeleted ............................................................................................ 77 Table 17: Competition Experience ............................................................................................... 77 Table 18: Sales Program Outcomes .............................................................................................. 78 Table 19: Sales Program Outcomes: Introductory and Advances Sales Course Comparison ...... 79 Table 20: Mean Results for Selling Skills .................................................................................... 81 Table 21: Mean Results for Self-Management and Customer Relationship Management ........... 83 Table 22: Mean Results for Perceptions of the Selling Profession ............................................... 84 9 Table 23: Mean Results for Perceptions about a Selling Career .................................................. 85 Table 24: Interest in a Sales Career .............................................................................................. 85 Table 25: Mean Results for Importance of Sales Program Features ............................................ 86 Table 26: Student Evaluation of Instruction Results .................................................................... 88 Table 27: Introductory Sales Course; Within Group Role Play Evaluation ................................. 90 Table 28: Advanced Sales Course; Within Group Role Play Evaluation ..................................... 90 Table 29: Introductory and Advanced Course Between Group Final Role Play Comparison ..... 91 Table 30: Advanced Sales End of Course Team Project Evaluation ............................................ 93 Table 31: Negotiation End of Course Project Evaluation ............................................................. 94 Table 32: Sales Internship Supervisor Evaluation ........................................................................ 94 Table 33: Career Outcomes and Comparison ............................................................................... 96 10 List of Figures Figure 1: Conceptual Framework for Sales Program Development.. 24 Figure 2: Map of Universities Offering a Sales Major 31 11 Executive Summary This capstone documented the process of defining, developing, and evaluating a curriculum for professional selling within a business school in a small liberal arts university. Specifically, it addressed the research question: What knowledge and skills do students need in preparation for a professional selling career, and how does implementing these in a formal curriculum improve students' sales competencies? Identification of the required knowledge and skills and the associated new curriculum were developed through secondary research, benchmarking other programs, and qualitative interviews with sales curriculum experts. Competency was defined as the ability to implement the knowledge and skills. The model used to evaluate student competencies throughout this new curriculum was a variation on the four-level Kirkpatrick model of reaction, learning, behavior, and results. Specifically, Level 2, Learning and Level 3, Behavior, were used to measure sales competencies and answer the research question being studied. Level 1, Reaction and Level 4, Results were identified as necessary for a comprehensive review of the new program being implemented, although they did not explicitly address the research question. The newly approved curriculum requires the completion of three, 3-credit hour courses, including an introductory level sales course and an advanced sales course. Students then choose additional courses from a list of options to complete the specialization. The program was defined with three overall learning objectives. These objectives were met through the completion of this new curriculum, including nine core knowledge and skill areas. These nine core knowledge and skill areas were introduced in the required introductory course and reinforced in the advanced level course. Ten additional, more advanced skills were introduced and practiced in the advanced course and elective courses. 12 The new curriculum was implemented during fall 2020, spring 2021, and fall 2021 semesters. Prior to the formal evaluation, students assessed the new curriculum through a survey instrument and their feedback provided direction for adjustments within individual courses. These results showed a high level of satisfaction among the students in the program. The Level 2 and 3 formal evaluation results were obtained at the end of each of the three semesters. A significant increase in sales competency was observed, as measured by (a) within group knowledge and skill development gains using a pretest; post-test role play exercise in the introductory and advanced sales courses; (b) end of course project evaluations in two courses; and (c) internship supervisor evaluations. Additional results of interest to the program design included the Level 4 findings of positive career outcomes among program graduates. Some lessons learned during this project include the importance of 1) using adaptive leadership principles in a projects early stages, especially when there are a variety of collaborators and stakeholders with differing values; 2) communicating needs and constraints to collaborators and stakeholders; 3) anticipating difficulties and accepting limitations within the organization, its structure, and its processes; and 4) developing leadership capacity to effectively manage the outcome while generating excitement to engage with a vision for the future. Keywords: professional selling, curriculum design, Kirkpatrick model, sales competency, sales training, university sales. 13 Chapter 1: Introduction This project identified the knowledge and skills needed by students as part of a new program for professional selling and reviewed the newly established curriculum that delivered the knowledge and skills that were defined. An evaluation plan to measure the effectiveness of the knowledge and skills gained by the students in the program was implemented and the results are presented in this document. This implementation project was performed within the context of the adaptive leadership framework. Background State of Higher Ed: Enrollment Challenges, Financial Challenges, and Industry Needs Higher education has changed, and the pandemic accelerated the trends that were already occurring. Undergraduate enrollment was down 4.4% in the fall of 2020 as compared to fall 2019, with community colleges leading the loss in enrollment, and all institution types seeing declines (Williams, 2020). Leading business journals such as Forbes and the Wall Street Journal routinely publish about the looming financial crisis in higher education and the likely closure of hundreds of colleges and universities in the next ten years (Belkin, 2020; Craig, 2020). These experts point to many factors. Some include the demographic cliff of college-aged students and the resulting lack of bodies available to sustain the number of colleges and universities in operation, the unsustainable tenure structure of universities, untenable prices and growing student debt, and a lack of alignment between what universities teach and what industry requires (Nietzel, 2020). Craig (2020) wrote that addressing student debt, or even free college, is not enough. it may not lead to increased economic opportunity as it omits a fairly important component of the employment equation: the employer (para. 5). Craig went on to write that employers and 14 colleges lack alignment, with the former seeking candidates with specific skill sets, especially digital skills. He recommended a model of higher education where faculty are tasked with building relationships with industry that integrate opportunities for networking and employment throughout a students college career. Supporting Craigs opinion, Belkin (2020) also predicted that private industry will play a key role in this reshaping of the workforce and that for colleges to remain relevant, they must develop closer relationships with companies. Employment for Business Graduates Business education remains a potential growth area for universities. According to the Bureau of Labor Statistics (BLS) Occupational Outlook Handbook (2019), employment opportunities in business fields continue to increase, driven by the strong median annual wage of nearly seventy thousand dollars annually as of May 2019. According to the Handbook, employment in business and financial operations occupations is projected to grow five percent from 2019 to 2029, faster than the average for all occupations, adding about 476,200 new jobs. It also projects that employment in accounting, market research, general management, sales, and supply chain management will grow faster than average. For sales professions specifically, BLS predicted that professional sales employment would remain strong through 2029, with insurance sales growing at a five percent rate 2019 2029 and a mean annual income of $50,940, and financial services sales positions growing at the market average, four percent, and a median pay of $62,270 per year. U.S. News and World Report, in the Best Job Rankings 2020 report, stated a median salary for all types of sales representatives of $58,510. A study by Emsi (2020) used 100 million professional profiles to identify job types among college degree holders. The study reported that among those with a formal degree in 15 business, 18% were employed in sales, 15% in business and financial analysis, 14% in accounting, 9% in marketing, and 6% in human resources. The study also reported that sales was the most common career outcome among all degree holders, including among graduates in humanities, social sciences, and engineering. In the study definitions, sales included technical sales, financial sales, and business to business sales, since these are the sales occupations most relevant to college graduates. The study did not include retail sales employment in the sales category. Research showed that students with a family history in sales tended to show a greater interest in a sales career than did students without this background (Crawford & Lumpkin, 1983; Spillan, Totten, & Chaubey, 2011). Students who studied business also tended to show more interest in a sales career than did students in other majors, suggesting the role that major plays in the selection of sales as a future career (Spillan, Totten, & Ziemnowicz, 2007; Spillan et al., 2011). Sales Education in U.S. Higher Education According to the Sales Education Foundation (SEF), the strong market demand, employment growth, compensation levels, and its position as a springboard to more senior management roles in marketing, finance and other functions have led to increased interest and job placement in professional selling careers among college graduates (SEF Key Statistics, 2020). This has in turn led to an increase in interest among employers in connecting with universities and has resulted in the recent growth in sales courses within the marketing curriculum at the university level (SEF Annual, 2020; Spiller, Kim, & Aitken, 2019). In 2002, the University Sales Center Alliance (USCA) was born in response to this increased market demand. Nine universities collaborated to form this organization with the mission, To advance 16 the selling profession through setting and monitoring sales program standards, sharing best practices, enhancing sales curricula, and preparing students for a career in sales (USCA, 2020, About Us section, para. 2). This organization has continued to gain prestige and today represents 35 full members and 23 associate members (USCA, 2020). While the USCA website provides direction on requirements for sanctioned sales centers within universities, a review of their member list shows that they primarily serve large universities with significant resources and dedicated, tenured sales faculty (USCA, 2020). A comparison of smaller universities and colleges to these USCA members shows that smaller programs are unlikely to possess the faculty and resources needed to meet the USCA requirements (SEF Annual, 2020). At the same time, smaller universities may be uniquely positioned to bring value to sales education for students and employers if they are able to deliver on a curriculum that prepares students with the required knowledge and skills to be successful in professional selling roles. Hallmarks of sales education include small class sizes, significant one-on-one interaction with faculty using role plays and other individual and small group practice, and close industry partnerships (Forbes, Loe, Patterson, & Erffmeyer, 2014; Spiller, Kim, & Aitken, 2019; Weilbaker, 2004). Most small universities already support small class sizes and the resulting closer relationships with students. In addition, smaller schools may be able to act more nimbly when it comes to rapid technological change as long as they are able to align their curriculum and course outcomes with industry needs. Problem of Practice There is a need for professional sales representatives in the labor force (BLS, 2019; Manpower Group, 2018; SEF Key Statistics, 2020). This need continues to grow due to the limited number of university professional selling programs in place to help prepare students for 17 professional selling roles upon graduation (Bolander, Bonney, & Satornino, 2014; Fogel, Hoffmeister, Rocco, & Strunk, 2012; Spiller, et al., 2019; USCA, 2020). The challenge of preparing students for the workplace is not a new phenomenon in business education, but what is new are the differences occurring in sales education. Namely, the lagging sales student pipeline, the accelerated rate and scope of change in the knowledge and skills needed to prepare students for a selling role in the marketplace, and the lack of research available to assess that the required knowledge and skills have been attained by new graduates (Peltier, Cummins, Pomirleanu, Cross, & Simon, 2014; Spiller, Kim, & Aitken, 2019). Although many programs claim successful sales student outcomes, how they identified current and emerging knowledge and skill requirements and incorporated them in their programs, and how they evaluated the efficacy of their programs has not been well documented. Although a modest body of research exists on sales education pedagogies, scholars in the field have identified the need for more research on sales education from smaller schools, identification of an ideal sales curriculum, and, for more research studies that use direct measures of performanceto advance the sales education field (Spiller et al., 2019, p. 229). To address the need for professional sales representatives who are prepared for professional selling roles at graduation, understanding the current knowledge and skills needed to prepare them, implementing these competencies in a formal sales curriculum, and evaluating student performance on these competencies is critical. Faculty at the university being studied for this project had identified a need for new programs with both student and market interest to ensure future student enrollment and job placement success. Internally, students had shown interest in professional selling coursework, with approximately 60% of marketing and 20% of management business graduates from the past 18 two years finding employment in a sales role upon graduation (Job Placement Statistics, 2019). However, the business school administration and faculty of these graduates have information that not all graduates had been successful in their first year, suggesting that the problem may relate to their academic preparation. The Status of Sales Education in Business To maintain current enrollment levels and financial returns in the face of the dwindling market opportunity, business schools must remain focused on attractive programs that will support recruitment efforts while satisfying employer demand for highly qualified graduates (Craig, 2020). To implement a successful new sales program, identification of business partner needs within the local geographic market under study for this project was needed. An understanding of student knowledge and skills needed for new sales graduates was critical to achieving strong partnerships and their associated financial support to maintain the program (Forbes et al., 2014). Finally, although there was general agreement on the need for a sales program, there are potential issues to be resolved related to program management and control, budget authority, and the partner management process. These issues will be further explored in the System Diagnosis section of this paper. Mission and culture are advantages within the business school being studied for this project. The school serves a population with higher than average numbers of minority students and economically disadvantaged students as compared to universities overall (Office of Institutional Research, 2020). Sales careers, with strong market demand, high entry-level salaries, and promotion potential within businesses, align well with the business school mission to serve these students and develop business leaders. While the business school is a teaching 19 institution and lacks the AACSB research-oriented accreditation typical of top sales schools, it has a culture based on a hands-on, experiential curriculum with significant industry professional interaction and faculty who have industry experience in their fields. These factors, along with the previously outlined market demand and growth, expose an opportunity for educational offerings like professional sales. Until 2019, the business school in this study offered only a single introductory sales course. A second sales course was introduced in late 2019, Advanced Selling. It resulted in positive student feedback, internship and employment opportunities for participating students, and early success in national sales competition events. A third course, Negotiation, was piloted in the spring of 2020. The business school also enjoyed an advantage with its nascent sales program, as the only university in the immediate geographic area with a sales program. A concern with the current sales offerings was that they were piloted with little input from potential partner businesses. Students who pursue careers in sales found themselves interviewing alongside students from schools with strong national rankings, brand recognition, and employer agreements, such as USCA schools. To compete with these schools, business school students must be well prepared in knowledge and skills and must make connections with potential employers during their time in school. Significance and Purpose of the Project Significance of the Project Professional selling was identified as a program that would prepare students to make an impact in a professional selling role upon graduation (Bolander, Bonney, & Satornino, 2014). This study addressed a local problem by focusing specifically on the development and evaluation of a professional selling program for students in the university under study. This project was 20 unique because it addressed an under-researched area of higher education within business schools (Bolander et al., 2014; Cummins et al., 2013; Deeter-Schmelz & Kennedy, 2011; Peltier et al., 2014; Spiller et al., 2019) in a high employment growth area (BLS, 2019; Manpower Group, 2018; SEF Key Statistics, 2020). The results of this study provided much-needed insights for the study university into the curriculum and pedagogy of a new sales program, along with establishing a clear evaluation process. Insights from this study aided the business school in helping students to succeed academically, in their internship and job searches, and after graduation, thus supporting enrollment goals and student graduation rates, industry funding, and business school reputation (Bolander et al., 2014; Cummins et al., 2013; HR Daily Advisor, 2018; Lilly & Stanley, 2016; Newberry & Collins, 2015; SEF Key Statistics, 2020; Weilbaker, 2004). Further, higher education has long been a tool used to address social inequality. This study also supported the successful attainment of professional sales employment among underrepresented student groups and increased diversity in the types of individuals gaining these positions. Purpose of the Project The purpose of this study was to identify the knowledge and skills needed to meet the educational objectives and market needs in the design and delivery of a professional selling program at the university under study and create and implement an evaluation framework for the measurement of program effectiveness. Although the business school under study was traditionally a strong revenue producer for the university, new student enrollment overall had been stagnant for several years and future enrollment projections nationally portend declines (Anderson & Douglas-Gabriel, 2022). Even with strong revenues, the business school had difficulty in meeting growing budget needs and 21 had not increased the number of faculty, staff, or administration in over ten years due to budget pressures. If, as projected nationally, enrollment is flat or even declines, consequences for the business school and the university could be severe. For these reasons, identifying new programs like sales, with strong employer demand and potential industry funding, and then designing a curriculum to meet these needs, presented opportunities for growing student enrollment and increasing external funding for the school. A program in professional selling was well aligned with the goals of strong employment prospects, good opportunities for industry funding and support, and high student interest. However, to be successful, the knowledge and skills required by graduates must be well understood, implemented, and measured effectively in the curriculum. The lack of research in these areas, specifically in schools similar in size and geographic location to the school under study, formed the argument for this project. System Diagnosis Adaptive leadership concepts were applied to (a) assess the needs of the organization; (b) understand stakeholder positions and document interventions; and, (c) summarize the learning that occurred to address the complex challenge that was presented by this project. Needs Assessment The business school marketing program has had success over the past 12 years in marketing-oriented competitions but had little focus on sales courses or competitions. The competition success resulted in positive public relations for the school. Student feedback was also positive, with many students requesting additional opportunities to include a wider range of competition options. 22 To identify new opportunities, a review of competition opportunities was performed. Several ideas emerged, such as sales competitions offered through the American Marketing Association (AMA), as well as university-sponsored sales competitions. With only a single sales course being offered in the curriculum before the fall of 2019, further study was done to identify if sales competitions would be appropriate. Job placement statistics for recent graduates were reviewed. It was determined that many business graduates were being employed in some type of sales role as their first professional position after graduation. To test the viability of pursuing additions to the sales curriculum, a faculty member invited a small group of students to participate in a regional beginners level sales competition. Student feedback was tremendously positive. A SWOT analysis was performed to help evaluate the abilities and potential for success of the business school to create, launch, and manage a new program in professional selling. Strengths of the school were that it was relatively small, with good communication among the faculty and administration. The school had a history of successfully implementing new courses and even new major programs of study. Another strength was the experiential nature of a sales program. The hands-on format aligned well with the current culture in the business school. Space was not expected to be an issue since the capacity for growth already existed. A potential weakness, or concern, was the financial resources that might be needed to run a new program, including new faculty. Opportunities were that a new program in sales would enjoy a competitive advantage since there were no universities in the local area offering sales majors, minors, or other programs of study devoted to sales. Another opportunity was employer demand and the willingness of local business partners to support a program financially. Strengthening 23 business partnerships provides opportunities for student internships and job placement. A threat to a new program was that another local university might decide to offer something similar. Stakeholders, their Loyalties, and Interactions to Understand their Positions The adaptive leadership process (Heifetz, Grashow, & Linsky, 2009) was used to understand the formal and informal dynamics within the organization. Stakeholders were identified and multiple interactions occurred over a six month period to clarify the needs and goals of each individual or group. Formal meetings and informal discussions occurred with the business school dean, faculty, adjunct faculty, university advancement staff, and external business professionals who expressed an interest in recruiting from or donating to a future selling program. University policies were also examined to ensure adherence to all requirements for the new program and course development and approval. Conceptual Framework Using the adaptive leadership process for system diagnosis allowed a more in-depth understanding of the needs of the organization, the marketplace, and the stakeholders. It resulted in a framework for how to construct a new program in professional selling. The research and learning from the process were then used to form the conceptual framework, as shown in Figure 1. A combination of adaptive and technical processes were used to implement the solution of desired knowledge and skills. This ensured all faculty teaching within the new program were part of program design and achieved agreement on the framework and implementation plan. Both adaptive leadership and technical processes were used as part of the ongoing program evaluation. 24 Figure 1 Conceptual Framework for Sales Program Development Research Question Based on the information presented, the project research question was stated as: What knowledge and skills do students need in preparation for a professional selling career, and how does implementing these in a formal curriculum improve students' sales competencies? Limitations and Delimitations Some limitations on this studys evaluation methods included one interview being held by telephone due to Covid restrictions, obtaining adequate contact information for the collection of outcomes measures among a small number of graduates, and the ongoing changes to university processes, people, and organizational structure during the project timeframe. Moving forward, limitations regarding running this program are that the business school is small and has limited faculty and staff available to assist if the program continues to grow. Faculty turnover could also impact program viability. 25 A delimitation for the program evaluation was the small student population being studied. This study required the measurement of unique populations of current and recently graduated students, with small sample sizes, indicative of case studies. Another delimitation was that this analysis included only a single institution, the university in this study, with a single student population of sales program students, and no opportunity for a comparison of outcomes with other new programs. Definition of Terms a. Adaptive Leadership: a practical leadership framework that helps individuals and organizations adapt to changing environments and effectively respond to recurring problems. It is the activity of mobilizing people to tackle tough challenges and do the adaptive work necessary to make progress and thrive. Research by Heifetz, Grahsow, and Linsky (2009) at Harvard University formed the basis for adaptive leadership. b. Benchmarking: the practice of comparing processes and performance metrics to best practices from other organizations, schools, or departments. c. B2B: The term business-to-business (B2B) refers to the process of selling the goods and services a business makes to another business, either for their own use or for resale to consumers. d. B2C: The term business-to-consumer (B2C) refers to the process of selling products and services directly between a business and consumers who are the end-users of its goods or services. e. Coaching: Coaching is a development or training technique typically used for executives or new managers/leaders. It often involves a professional coach or external consultant who works with employees to develop their skills and achieve business goals. Coaches 26 support employees by understanding their personal and career goals and providing feedback and direction to facilitate performance improvement (Training Industry, 2020). f. Executive in Residence: experts from industry, such as a retired executive, who lend their specialized knowledge and experience to organizations on a temporary or part-time basis. g. Institutional Advancement: a division or department of the institution focused on the improvement of the entity, primarily involved in fundraising activities. h. KSAs: Knowledge, skills and abilities (KSA) is a term referring to the qualifications that an individual needs to perform a specific job. Usually, an employer will create the KSA as a guideline to seek out the most qualified applicants for a position (Training Industry, 2020). i. Major/Minor/Specialization: For a Bachelors degree, a major is a primary focus of study and a minor is a secondary focus of study. At the university in this study, a minor typically requires 18 hours of study. A specialization or concentration differs between universities. At the university in this study, a specialization typically requires 12 credit hours of study. Some universities use the term concentration as an alternative. j. Mentoring: Mentoring is a type of training in which a more experienced employee (the mentor) provides expertise, support and guidance to a less experienced employee (the mentee), in an effort to advance his or her career growth and success in the workplace (Training Industry, 2020). k. Part-time or Adjunct Faculty: An adjunct position is a part-time job. Adjuncts are limited in the number of courses taught, and the compensation is on a per course basis. l. SEF: The Sales Education Foundation, a non-profit organization that promotes the benefits of university sales education both in academia and the corporate world. 27 m. Stakeholder: People affected by the challenge and in need of a solution, sometimes called front-line stakeholder. n. Student Coach: A type of coaching using peer feedback rather than an industry professional. Used to build skills in both the student being coached and the student coach. o. Top Box Score: The top box score used in this study is the sum of percentages for the top two points, such as strongly agree and agree, on a five-point scale. p. USCA: The University Sales Center Alliance is a group of 35 member and 23 associate member universities who meet specific criteria to join the alliance. All member universities have formal Sales Centers with a faculty director. The USCA connects marketing and sales faculty members who advocate for the continuing advancement of the sales profession through teaching, research and outreach (USCA, 2020). 28 Chapter 2: Literature Review The literature review was undertaken to assist in identification of the knowledge and skills needed by students to build sales competency, along with gathering information to support the design and development of a new professional selling program. General Background of Sales and Sales Programs Market Trends for the Sales Profession and University Sales Education Sales and marketing employment is projected to remain strong through 2029, yet sales and marketing positions are among the most challenging to fill (BLS, 2020; Manpower Group, 2018). The BLS Occupational Outlook Handbook provides comparative data across occupations and can help inform data-driven decision making about the current and future demand for a job category or even specific job types. The 2019 data indicated that a sales manager position requires a bachelors degree and that growth for this job is at 4% through 2029. This information lent credence to the argument that a sales career is more than vocational training and is a legitimate academic area of study. More specific categories such as insurance sales, advertising sales, and sales engineers, defined in the BLS as including B2B sales, which is what most academic programs teach, also showed strong rates of job growth through 2029. At the same time, CSO Insights, the research division of Miller Heiman Group, a sales training and consultancy firm, reported that only 16% of sales leaders believe that they have the talent they need to succeed in the future (HR Daily Advisor, 2018). In 2012, author and sales expert Daniel Pink, in his book, To Sell is Human, claimed that the number of salespeople in the United States outnumbers the federal government workforce by five to one. He also cited increasing demand and high turnover rates for salespeople that continue to fuel high sales employment rates. 29 This information lent strong support to the viability of a sales program and its ability to support university enrollment goals through 2029. Further confirmation for this opportunity was found by examining key milestones in higher education related to sales education, as presented in Table 1. These developments continued to spur interest in sales education and provided legitimacy to the field. Growth of sales programs at the university level, although gaining slow momentum among large universities, has lagged behind other business areas in terms of new program growth at both the undergraduate and MBA levels (Cespedes & Weinfurter, 2016). They reported that only about 150 colleges and universities in the U.S. offered some type of sales program. Of those, 31 offered a major in sales. Their research revealed that of those schools with a major, 21 were USCA members and 23 were AACSB accredited. Of the 17 without AACSB accreditation, two held IACBE accreditation, and the remainder did not report any specialized business school accreditation (Cespedes & Weinfurter, 2016; SEF Annual, 2020). Using data reported in the SEF Annual (2020), the number of students participating in the sales program at schools offering a sales major is shown in summary form in Table 2. Student numbers range from a low of 15 at Hodges University in Naples, Florida, an IACBE accredited university, to a high of 500 students at both Florida State University in Tallahassee and Weber State University in Ogden, Utah. Both of these schools are members of USCA. Figure 2 presents a visual representation of the location of all 31 universities offering a sales major (SEF Annual, 2020). 30 Table 1 Key Milestones in Sales Education Year 1980 1984 Key Milestone Journal of Personal Selling & Sales Management was created American Marketing Association (AMA) sponsored the Selling and Sales Management Consortium 1985 The first university sales center (Center for Professional Selling) to open at Baylor University 1989 The second university sales center to open at Northern Illinois University 1999 First National Collegiate Sales Competition (NCSC) held at Kennesaw State University by the Coles College of Business Center for Professional Selling. It is the longest running university sales role play competition in existence. 2002 University Sales Center Alliance (USCA) was formed by the leaders of the then eight established university Sales Centers or Sales Institutes with the purpose to connect faculty from different universities and to advocate for the continuing advancement of the sales profession through teaching, research, and outreach. 2007 Sales Education Foundation (SEF) was established as a nonprofit organization dedicated to promoting the benefits of university sales education both in academia and the corporate world. They publish the annual Top Sales Universities report. Source: Adapted/Updated from Spiller et al, (2019); NCSC, (2020) Table 2 Number of Students Participating in Sales at Universities Offering a Sales Major Number of Students Number of Universities < 49 50 99 100-199 200-299 300+ Source: SEF Annual (2020) 5 7 5 6 8 31 Figure 2 Map of Universities offering a Sales Major Source: SEF Annual, (2020) The combination of market opportunity, employer demand, support from the USCA, and the associated program growth among large and highly respected university marketing departments combined to create a substantial opportunity for those universities that embraced the challenge of creating a sales program (Weilbaker, 2014). The application of digital technologies and artificial intelligence to sales automation and customer relationship management further supported the need for strong sales research and curriculum at the university level (Spiller, et al., 2019). The rapid changes continuing to occur in digital sales technology have strong implications for universities who must navigate these challenges and find ways to integrate this information into the education of the next generation of sales professionals (Singh, Flaherty, Sohi, Deeter-Schmelz, Habel, Le Meunier-FitzHugh, Malshe, Mullins, & Onyemah, 2019). Finally, research by Bolander et al. (2014) provided convincing evidence that connecting the relationship between sales education and the performance of new graduates in their first sales job was an important step toward confirmation of the value of university sales education programs. Although quantitative studies are limited, Bolanders research demonstrated that 32 salespeople hired from universities with formal sales programs outperformed their non-sales program counterparts after the first year of employment. Sales education has also been shown to have a significant, positive impact on salesperson attitude toward his or her company (Bolander et al., 2014). The implications of these findings were significant for universities that offered or planned to develop sales programs in terms of meeting the needs of employers and leveraging those employer relationships for strong partnerships with the university. Why Students Study Sales While market demand is strong and there was evidence that sales education leads to stronger outcomes in a sales career, many young college students hold a negative perception of sales and lack consideration for studying sales or even taking a sales class (Allen, Kumar, Tarasi, & Wilson, 2014; Barat & Spillen, 2009; Peltier, Cummins, Pomirleanu, Cross, & Simon, 2014). It was therefore important to understand those perceptions and what factors might influence positive perceptions of a sales career. An understanding of what does and does not resonate with these students was needed so that marketing educators could better bridge the gap between students and potential employers. Four areas emerged in the literature as the primary contributors to students intent to pursue a professional selling career. Student perceptions and attitudes were shaped by the sales profession, salespeople, sales knowledge, and sales ethics (Bristow, Gulati, Amyx, & Slack, 2006; Bristow et al., 2011; Karakaya, Quigley & Bingham, 2011; Peltier et al., 2014). Karakaya et al. (2011) reported a significant relationship between a students feelings toward selling and the intent to pursue sales as a career. Positive perceptions and attitudes about the sales profession were found with students who perceived sales jobs as valuable, creative, and having opportunities for growth (Allen et al., 2014). Positive feelings were found among students who perceived sales as fun and exciting while requiring intelligence, confidence, and talent 33 (Karakaya et al., 2011). Allen et al. (2014) also reported that advisors had a significant influence on a students intent to study sales, as well as prior interest in the field. At the same time, it was important to ensure that students were forming realistic expectations about a sales career (Allen et al., 2014; Bush, A. J., Bush, V. D., Oakley, & Cicala, 2014). Research suggests that negative perceptions can be overcome through a variety of interventions. For students already studying some area of business as either a major or minor, an introduction to sales in a principles of marketing course was recommended (Peltier et al., 2014). Peltier et al. (2014) proposed the implementation of an Intent To Pursue Selling Career Scale in the principals of marketing course as a means to motivate interest in professional selling early in students educational careers (p. 18). For students in an introductory sales class, exposing them to practical applications and activities in the classroom, such as case studies, role plays, and interactions with sales professionals through mentoring, job shadowing, guest speakers, and networking events was found to increase the positive perception of a sales career (Allen et al., 2014; Bush et al., 2014; Cepedes & Weinfurter, 2016; Gafford, 2019). The use of these activities, along with sales internship experiences and competitions, supported the findings by Bush et al. (2014) which recommended that educators provide a realistic picture of the expectations and behaviors associated with success on the sales job. This information should also be of value in student recruitment to marketing and sales programs, as well as when assisting prospective college students in deciding what to study. Industry Partnerships Nielson and Cummins (2019) indicated that experiential activities and quality involvement by sales professionals in the academic environment were techniques used to influence intent to study sales and provide realistic expectations about a selling career. They also 34 wrote that in addition to learning about sales from practitioners, students required internships and post-graduation employment. Building relationships with companies as part of sales courses could build the networks that will lead to those opportunities. With regard to financial support, Nielson and Cummins (2019) also stated that industry partners could be a very strong source of needed funding, as well as provide brand building for the university. With these factors in mind, it was important to understand how to identify, create, and manage partnerships with sales professionals and their companies so that these experiential activities, job placement opportunities, and university support can become a reality. SEF (2019) summarized three key recommendations for activating corporate-sales center partnerships: Ensure a mutually beneficial strategy and equity in the partnership Maintain an open dialogue allowing for idea sharing Be flexible and collaborative to maximize the impact for university sales programs, corporate partners, and the next generation of sales professionals Fortunately, industry partnerships also provide substantial benefits to the companies who participate (Cepedes & Weinfurter, 2016). Industry professionals have identified the need for involvement in the academic environment as a means to attract top talent to their sales recruitment programs. Weilbaker (2004) suggested that the upward trend in demand for university sales programs was driven primarily by industry, its need to reduce training and other costs associated with the sales force, and overall greater demand for qualified sales talent. This conclusion was supported by the large number of corporations sponsoring university-related sales competitions and other sales program initiatives across the United States. Corporate partners may also have an interest in partnering on research projects that are relevant to issues they face (SEF, 2019). Sales centers must be doing research that is relevant to their corporate 35 partners and that cutting-edge research informs the classroom, according to Deva Rangaranjan, Ph.D., Former Director Center for Professional Selling, Ball State University (SEF, 2019, p. 40). Dr. Charles Ragland, Executive Director, Indiana Universitys Center for Global Sales Leadership, said, We recommend a three-year time horizon for corporate partner engagement (SEF, 2019, p. 40). This timeline is driven by the need for companies to build their brand on campus and develop relationships with program faculty and students. Partners may also wish to be part of the sales programs advisory board as a way to develop faculty and student relationships and steer curriculum (SEF, 2019). According to Lilly and Stanley (2016), these partnerships provided a win-win-win scenario; corporations saw the value, and, as a result of the decline of government funding for higher education, colleges were able to use the additional corporate funding to further their own goals and missions. Their work reinforced the idea that universities with the foresight to develop sales programs and offer specific sales curricula have benefited from higher levels of corporate involvement. Evidence indicated that students saw the benefit in connecting with industry partners to solve business problems, even when the interaction was held virtually (Coco, Crumbly, & Jackson, 2014). Students were more satisfied with courses that had this type of involvement than those that did not and tended to have a more favorable opinion of sales as a discipline than did students who took sales courses without this involvement (Lilly & Stanley, 2016). All stakeholders generally benefited from these offerings due to the tension between the low supply of qualified sales talent and the continuing high demand for qualified entry-level salespeople (Lilly & Stanley, 2016; Weilbaker, 2004). Industry partnerships, while valuable, are not without drawbacks, according to Lilly and Stanley (2016). They summarized the need for additional funding as a result of implementing the 36 activities related to the involvement of industry practitioners. They outlined field trips, networking events, and other sponsorship needs as part of such programs. Faculty time was another potential concern they raised. The focus on behavioral exercises such as role plays required significant coordination with multiple practitioners in terms of logistics and coordination on the part of the faculty involved. Faculty are often tasked with matching students to potential employers, as well as meeting with partners to discuss successes and failures of students hired, and general assessment of benefits to each partner. Sales Program Development How to Run, Manage, and Grow a Sales Program Academic literature on creating and managing a university sales program was very limited (Forbes et al., 2014). Two organizations exist that provide guidance to universities on this topic, the USCA and SEF. Most notably, the USCA (2020) provides guidance on what they feel is essential for the creation and management of a sales program as part of their membership criteria. Although academic literature was limited outside of these two organizations, an early article on the subject referencing the USCA guidelines highlighted the trend to develop subspecialties in marketing departments, such as one in professional sales (Weilbaker, 2004). Weilbaker (2004) reported on how to present the idea of a sales specialization to the marketing department or college of business. His perspective was based on the notion at the time that sales was considered more of a trade school profession. He outlined the transition in sales education and its gains in legitimacy as mirroring the changes that occurred in industry sales organizations. While sales was historically based in retail or B2C areas, by 2004, the rise of B2B selling was clear, and academics, as well as students, began to view the sales profession with more legitimacy. This legitimacy was driven by the need for more logic and analysis in the B2B 37 selling environment and formed the basis for the argument that sales should be considered a viable sub-specialty within university marketing curricula. When making the case for a sales subspecialty, Weilbaker (2004) identified three key stakeholder groups that should be included in a presentation of the benefits of a sales program: students, companies, and the marketing department or business school faculty. SEF published information on the topic based on their research. They provide this research and numerous other resources to educators and industry sales professionals. Another academic publication was in the Journal of Selling. The article provided recommendations on how to establish, run, and grow a sales program. It was authored by five large university sales center directors and summarizes their research with 30 USCA members (Forbes et al., 2014). As an authority on sales programs and as the only organization that certifies sales programs, the USCA provides essential guidance. They have strict membership requirements with specific criteria that must be met for membership in the organization. These criteria are outlined in Appendix A. The USCA requires a 5-year associate member period that includes mentoring by the USCA members while a university works to qualify for full membership (USCA, 2020, Membership section). SEF (2017) provided their recommendations for starting a new program based on research with over 70 university-recognized sales programs. They recommended that a new program launch should be in the form of a sales concentration or specialization, and include a minimum of three courses. They cautioned that a program can take many years to build, with both internal and external hurdles for engaging stakeholders in such an endeavor (SEF, 2019). While both organizations provide guidance on the topic, SEFs guidelines are recommendations, while USCA has set forth requirements based on their membership criteria. For the summary 38 below, the word recommendation was used, however, where USCA is noted, these are membership requirements rather than just recommendations. Both the USCA and SEF recommend a minimum of two faculty in a sales program, with at least one at the academically qualified (AQ) rank. SEF provides further detail on the faculty roles, recommending that one faculty manage research and academic requirements and the other manage corporate relationships, fundraising, and other sales functions of the program (SEF, 2019). Both the USCA and SEF organizations recommend sales courses, including an introductory and a more advanced course. SEF recommends at least one additional specialized course. Also recommended by both organizations are that sales research is performed, programs should be available to students of all majors throughout the university, have external financial support, and hold official recognition as a sales program within the university. Recommendations on recognition provided by Forbes et al. (2014) were similar, stating that buy-in from university and college administrations are requirements for a Center to be formed (p. 8). Finally, both organizations, as well as both the Forbes and Weilbaker articles recommended an advisory board (Forbes et al., 2014; SEF, 2019; USCA, 2020; Weilbaker, 2004). Forbes et al. (2014) provided more in-depth recommendations for a successful sales program based on a USCA member survey. For example, while USCA and SEF recommend external funding, the Forbes article proposed potential funding models and highlighted that the relationship between the sales center and the university fundraising department can impact the amount of funding that could be expected. Models summarized included various combinations of endowments, external fundraising by the sales center director, funds budgeted by the university, and other private funding. Center directors in the survey indicated that nearly all (97%) 39 participate in external fundraising activities, while 87% indicated that they were directly involved in 80% or more of all fundraising for the Center. The study acknowledged that donations were generally filtered through the university fundraising department or foundation and that for this reason, Center Directors need to develop a good working relationship with the Foundation (Forbes et al., 2014, p. 12). They suggested meeting with the foundation to explain how the Center will work with donors, and that this type of meeting should result in a better relationship with the foundation. Forbes et al. (2014) also supported several additional USCA requirements such as a Sales Advisory Board, a stipend for the Center Director for work beyond that of a typical faculty member, administrative help, and specific sales role play facilities and associated technology. A commonality among the research related to the content of a sales program was the need to align the curriculum with industry needs. While advances in technology may not be industryspecific, some industries may have requirements about selling techniques or customer knowledge. These industries may be regionally focused so it is important for a university to understand their marketplace, as well as have a realistic view of their abilities and limitations. Sales Program Curriculum Development Types of Courses Courses that should be included in a sales program have been reviewed historically, but older recommendations were of less relevance for this paper. Although useful to review for foundational course needs, these older articles were outdated due to the rapid technology changes that were occurring within sales. For this reason, publications from 2015 to the present were used to identify contemporary curriculum needs, other than specific instances for which older manuscripts added value. Two important sources in course identification were the USCA and 40 SEF websites. The mission of these organizations is to help further sales education and they are highly respected among universities committed to professional selling programs. Both organizations provide recommendations on the curriculum for new programs and both are widely used by marketing departments for guidance on a variety of sales program topics, including new or revised program design and course selection. Both organizations supply sound, research-based evidence and authority among university programs when envisioning the development of a sales program. USCA (2020) offers membership to universities that meet their specific criteria. USCA requires applicants for membership to first apply for associate membership by meeting 13 criteria. Among the associate member criteria set forth by USCA are the specifications for the required curriculum. To become an associate member, a candidate university must teach a minimum of two sales classes, with two levels of sales training (e.g., a personal selling and advanced selling course) with one being a prerequisite course for the other. To apply for full membership, a university must teach a minimum of three sales courses. SEF (2017) also provides recommendations for curriculum for universities interested in developing a professional sales program. Like the USCA, SEF curriculum recommendations for new programs are to include a minimum of three sales-specific courses. SEF provides guidance on the three courses, listed below. They also recommend at least one specialized course. Their guidance is for every program to offer the following: Introduction to Professional Selling Advanced Sales Sales Management 41 They encourage new programs to offer at least one specialized sales course. Specialized courses identified by SEF include the following: Financial Selling Key Account Management Sales Technology Sales and Social Media Supply Chain Sales Ethics in Selling Sales-specific internships are strongly encouraged by SEF. They further stated that a general education course such as public speaking and standard marketing courses such as Introduction to Marketing are not recognized as courses in the sales-specific curriculum. A systematic review of the literature related to sales curriculum was published in a comprehensive manuscript in the Journal of Marketing Education (Spiller et al., 2019). This publication, as well as others that are related to sales program curriculum design, were reviewed to ascertain current curriculum recommendations by authors from some of the most highly rated sales programs in the U. S. Some articles also identified courses that were predicted to be increasing in importance based on rapid technology changes. Spiller et al. (2019) provided results from a systematic review of sales education literature from 2014-2018, resulting in a comprehensive summary of findings from 53 articles. The publication also included curriculum information gleaned from a website review of all USCA member institutions, as well as results from a survey to university professors that included 57 SEF members and 30 USCA members and associate members. The website results 42 provided the most insight as to common sales program types and course lists. The program types identified are summarized in Table 3. Table 3 Common Sales Program Types at USCA Member Universities Sales Program Type Major and Minor Major only Minor only Certification Concentration/Specialization/Track Source: Spiller et al. (2019) % USCA Members (n=49) 22.4 28.6 42.9 26.5 24.5 Spiller et al. (2019) also reported that the majority of programs offer four to six courses in their sales program. The five most common courses among the 49 USCA members and associate members reviewed are the following: Professional Selling Advanced Selling Sales Management Negotiation Consumer Behavior Programs that consisted of a major or minor nearly always included an introductory sales course, an advanced sales course, and a sales management course (Spiller et al., 2019). A combination Selling and Sales Management course was more common in the 1980s, with nearly one-quarter of schools offering such a course, and the intent to offer such a course was growing among universities at that time (Kellerman & Hekmat, 1989). The combined course fell out of favor by 2000, when recommendations to split such a course into an introductory selling course 43 and a higher level sales management course were made by over 100 academics and business people at the AMA Faculty Consortium on Professional Selling and Sales Management (Marshall & Michaels, 2001). Spiller et al. (2019) also reported that additional specialty courses were offered depending on the total number of courses to meet requirements for the major or minor. Examples included some combination of Negotiations, Consumer Behavior, and Business-to-Business Management. Sales Analytics and Sales Technology were listed as relatively rare courses, but growing in popularity (Fogel, Hoffmeister, Rocco, & Strunk, 2012; Spiller et al., 2019). In addition, Spiller et al. reported that 34.7% of programs required a sales internship, and 44.9% offered online courses. While insightful, the Spiller et al. information had drawbacks. It summarized the state of USCA universities which were primarily large, well-established, and well-funded programs. These programs were not known for being nimble when it came to rapid curriculum changes to meet market needs. It also provided a view from late 2018, and with the rapid changes in digital sales technologies, may have overlooked new course opportunities that recently emerged. Therefore, this information was viewed as developmental, but not necessarily as distinctive or viewed as a requirement for a small university preparing to launch a new program. The lack of information on the accessibility of the program to students outside of a business school was also a drawback. A review of Stetson Universitys sales program provided a benchmark for a private university that implemented a sales program. Stetson launched a new sales program in the fall of 2018 and summarized their experience in a presentation at the 7th Sales Educators Academy in May of 2018 (Cox, 2018). Stetson has approximately 3,100 undergraduate students and is 44 located in Florida. They are an NCAA Division 1 School, and their business school is AACSB accredited. They offer both a major and a minor in Professional Sales, with the following requirements shown in Table 4. Table 4 Stetson Universitys Sales Major and Minor Course List Sales Major Six Courses SALS 230 Professional Selling and Communication SALS 330 Advanced Professional Selling and Communication SALS 410 Sales Management SALS 450 Executive Sales Strategy Plus 2 electives from a list of standard marketing courses Sales Minor Four Courses available to business and non-business majors SALS 230 Professional Selling and Communication MKTG 315 The Marketplace and Consumers SALS 330 Advanced Professional Selling and Communication SALS 410 Sales Management Stetson launched the sales program with both a major and a minor in just one year. They held an open house with 26 companies in attendance. Although financing was not addressed, they discussed the construction of five small role play rooms with video recording capabilities and one group role play room, as well as new faculty for the program. In their first year of launch, they reported that eight sections of the introductory sales course were offered, along with three sections of the advanced sales course. It did not appear that Stetson had applied for membership to the USCA (USCA, 2020, Membership List section). A second benchmark from the literature was provided by Newberry and Collins (2015), focused on Winona State University in Minnesota and their recent review and update of their sales program. The College of Business offers a minor in Professional Sales as part of their marketing program. The minor is open to business and non-business majors. Their sales program 45 is hosted in the Strauss Center for Sales Excellence. They are AACSB accredited, a USCA member, and are recognized as a SEF Top Sales University. Newberry and Collins reported that in 2015, the Center Director and one of the sales professors implemented a review of their existing sales program to identify critical instructional content and pedagogy. Their methodology included focus groups and in-depth interviews with sales practitioners, as well as a literature review. In addition to numerous content updates, a result of the study was to add a fourth course, a capstone experience, to their sales minor curriculum (Newberry & Collins, 2015). The resulting curriculum and course titles are shown in Table 5. Table 5 Winona State Universitys Sales Courses for a Minor Professional Sales Minor Requirements, 21 credit hours MKTG 265 Introduction to Professional Selling MKTG 300 Principles of Marketing (required as part of the Marketing major) MKTG 345 Sales Management MKTG Advanced Professional Selling Program Capstone (Independent Study on the Sales Competition Team or an Internship) Two additional electives from a list of marketing or communications courses Source: Newberry & Collins, 2015 Expected Knowledge and Skills and Typical Pedagogy in Sales Education Many publications outline the importance of technical and soft skills needed by marketing students (Bolander et al., 2014; Pink, 2012; Spiller et al., 2019). Sales research has shown that these are important skills to prepare students for success in professional selling positions (Healy, Taran, & Betts, 2011; Lilly & Stanley, 2016; Spiller et al., 2019). Sales education has been shown to produce these required outcomes (Jelinek, 2018; Lilly & Stanley, 2016; Loe & Inks, 2014; Spiller et al., 2019). Spiller et al. (2019) provided an inventory of skills, gathered from syllabi of 63 sales courses from SEF member professors. Their results showed that 46 sales educators heavily emphasized selling skills, communication skills, and relationship/ organizational skills, with over 70% of all syllabi reporting these course objectives. Other skills identified in the syllabi included networking, managerial, life, critical/analytical thinking, negotiation, problem solving, and technology. In addition to the educator perspective, it was important to understand the practitioner perspective when identifying skills needed for new graduates pursuing a professional selling career. The Newberry and Collins (2015) study provided a comprehensive view related to knowledge and skills in sales education from an industry standpoint. Using extensive background research, along with the qualitative research methods of focus groups and interviews of sales practitioners across different companies and industries, they identified the key selling skills that were preferred by sales practitioners. Their practitioner inventory of skills, categorized as critical, important, and useful, are shown in Table 6. Table 6 Sales Practitioner Skill Inventory Categorized as Critical, Important, and Useful Skill Inventory by Type Critical Communication Nonverbal: body language, facial expressions, grooming and fashion, proxemics Listening: focus, responsiveness, active listening Questioning: SPIN (Rackham, 1988), motivational interviewing Communication Styles: social style Writing Persuasion Self-Perception/Personal Expression: positive image, attitude, motivation, self-reliance, accountability, energetic, sociable, optimistic Professional Development: competitive, practice positive behaviors, entrepreneurial, coachable, organized, work efficiently, adaptable, demonstrate integrity Presentation: use of visuals, touching audience emotions, nonverbal, organization, providing a big finish, engaging the audience, voice clarity and use of tone/volume, being prepared. Important Networking: purpose, tools, processes 47 Rapport Building: meet and greet, use of questions and listening, communication styles Collaboration: teamwork, leadership, gaining by compromising, supporting others, with internal departments Critical Thinking: problem solving Negotiation: overcoming resistance, reaching a favorable commitment, conflict management Empathy Expression: focus on customer, use of communication skills, expressing understanding Time/Territory Management: goals and objectives, results oriented, establishing a plan, organization and scheduling, prioritizing customers/prospects, account planning Useful Research Techniques: leads, qualifying, identifying decision makers Closing Techniques Source: Reproduced from Newberry & Collins (2015) The previous studies supported that both sales educators and practitioners highly value communication skills such as listening, questioning, presentation, and persuasion or influence. Customer influence has further been documented in literature as including the ability to establish rapport, create a connection with buyers, uncover customer needs and wants through effective communication, presentation skills, and questioning to move the relationship forward (Bolander et al., 2014; Deeter-Schmelz & Kennedy, 2011; Marshall & Michaels, 2001; Moncrief & Marshall, 2005). Client entertainment activities also require skills related to communication and influence. Sales students, sales faculty, and sales practitioners rated entertainment activities, specifically those where alcohol is present, as important in achieving sales objectives and as a way to bond with co-workers, suggesting that managing these situations effectively was also a needed skill in sales education (Rodriguez, Honeycutt, & Ragland, 2015). Experiential teaching methods are well established in sales education, and research has established their effectiveness in achieving the desired learning outcomes (Inks & Avila, 2008). Spiller et al. (2019), in their review of prior sales literature, reported the most common teaching methods as lecture, role play, class discussions, in-class exercises, student presentations, and 48 projects, with 75% or more reporting those methods. Fifty to 74% reported assessments, group/team activities, and guest speakers. Additional teaching methods were reported less frequently overall but at a higher frequency in specific courses, such as videotaping, competitions, and coaching (Spiller et al., 2019). Deeter-Schmelz and Kennedy (2011) also reported that from a survey of 47 universities offering sales programs, 97.4% of universities used role play, 97.4% lecture, 86.8% guest speakers, 85.9% videotaping, and 84.2% in-class exercises. The USCA (2020) lent credence to the importance of role play, suggesting the use of mock selling situations and reporting that the top selling schools have established role play rooms where students are able to apply influence tactics in videotaped sessions. Newberry and Collins (2015) reported on pedagogy from the standpoint of sales practitioners. Their practitioner inventory of pedagogy, categorized by critical, important, and useful, is shown in Table 7. Table 7 Sales Practitioner Pedagogy Inventory Categorized as Critical, Important, and Useful Pedagogy Inventory by Type (*denotes an experiential pedagogy) Critical Role Play: competitive, own creation, given scenarios, computer generated* Actual Sales Experience: internships, fundraising sales calls, class projects, mentors* Job Shadow/Observation* Extra-Curricular: competitive sales team, networking events, seminars/workshops* Important Speakers: live, online Useful Simulations: online, computerized* Lecture: traditional, interactive Exercises/Assignments Exams/Quizzes Research/Independent Study Source: Newberry & Collins (2015) 49 Sales Technology and Ethics in Professional Selling Programs There appeared to be agreement on three courses for a sales curriculum (e.g. professional sales, advanced professional sales, sales management), general acceptance of at least one more (e.g. negotiation), and an acknowledgment among sales program leaders that the rapid changes emerging as a result of technology would create a need for inclusion of sales technology-oriented content. Topics such as sales analytics, social media in sales, and technologies such as sales force automation (SFA), customer relationship management (CRM), and artificial intelligence (AI) are part of the new sales technologies in use in corporations. Several of the larger USCA member universities with a major in sales offer some type of sales technology content, but this type of course was uncommon among universities offering only a minor or specialization (Spiller et al., 2019). Jelinek (2018) explained that the lack of technology in sales programs was because sales technology is inherently fast-changing, (p. 80) and instructors face a significant challenge when it comes to handling SFA in the classroom (p. 80). This challenge is predicted to increase with the emergence of the latest sales digitalization technologies like chatbots, social media, mobile apps, and AI, which are likely to be more significant and more far-reaching than previous sales technologies (Kunkle & Peterson, 2018; Singh et al., 2019). In a team-based workshop and collaborative process at the Setting the Research Agenda in Sales session at the 2018 AMA New Horizons Faculty Consortium in Selling and Sales Management, teams concluded that digitization and AI technologies pose a disruptive force and that AI technologies were triggering fundamental changes to the value creation function in sales (Singh et al., 2019, p. 2). 50 This was explained in part through a concept called sales enablement. Singh et al. (2019), reports that: Sales enablement groups are being established in many firms with a focus on providing salespeople with technology and information to sell more efficiently. These sales enablement roles are primarily responsible for drawing insights from sales activity data and creating digital content for sales leaders and salespeople to improve performance (p. 10). These technological changes also impact the idea that sales professionals may become knowledge brokers and improvisational thinkers who must integrate intellectual capital from digital technologies with their knowledge of the selling process (Kunkle & Peterson, 2018; Singh et al., 2019). Kunkle and Peterson (2018) also surmised that skills such as empathy, listening, critical thinking, and problem solving, often displayed by elite B2B sales professionals, would increase in value when combined with these new sales technologies. Due to the advances in technology, especially related to AI, and the resultant customer information that is resident in these systems, sales ethics is also a growing concern (Singh, 2019). Few universities offer specialized content in this area, yet student perceptions of selling ethics have been known to be a primary predictor of intent to pursue a selling career (Peltier et al., 2014). A review of USCA programs offering the Bachelor of Science in professional selling revealed no instances of a standalone sales ethics course (USCA, 2020). Others reported that among AACSB schools, 19 25% required a business ethics course of all business students (Bowe, 2015). 51 There has been a long-standing gap in ethics content within sales education, yet salespeople, due to their cross-functional interactions within and outside the company, are highly likely to confront complex ethical dilemmas (Anderson, Dixon, Jones, Johnston, LaForge, Marshall, & Tanner Jr., 2005; Bush et al., 2014). As early as 2004, the American Marketing Association called for increasing sales ethics content and application within the classroom (AMA Summer Conference, 2004). Most sales educators agreed that any sales program should include some significant focus on sales ethics, yet how that content was to be incorporated at the program level was not well explored. What educators did appear to agree on was that ethical content should include a variety of experiential tools that allow opportunities for students to analyze, apply, and discuss ethics and the importance of ethical decision making (Anderson, et al., 2005; Bowe, 2015; Castleberry, 2014; Cummins, Peltier, Erffmeyer, & Whalen, 2013; Cummins, Peltier, Pomirleanu, Cross, & Simon, 2015; Donoho & Heinze, 2011). There was also evidence in the literature that students who completed courses in ethics showed stronger ethical decision making skills and more positive perceptions of salespeople, yet also that these results could be achieved through business ethics courses aimed at all business majors, and not necessarily sales ethics specific courses (Bowe, 2015). It is unknown if this conclusion will continue to hold under the rapidly changing landscape of technology, namely AI and machine learning (ML). Singh et al. (2019) recommended research on the ethical implications of AI-based value creation and sales communication and as this research becomes available, it may highlight additional sales-specific ethics training needs. What did appear to be clear was that sales technology and sales ethics were two areas of potential interest in new program development, and as the two continue to become more intertwined through the emergence of AI and ML technologies applied to the sales function, sales programs will have to adapt. 52 Evaluation of Sales Education Programs within Business Schools Accrediting Agencies for evaluation Sales Program in Higher Education Schools offering a sales major, e.g. a degree in professional selling, may be subject to evaluation of that degree from external business accreditation agencies. There are three Councils for Higher Education Accreditation (CHEA) recognized business school or business programmatic accreditation agencies in the U.S. used by business schools. Common membership profiles are summarized in Table 8. Table 8 Summary of Business School Accreditation Agencies Agency Description AACSB International Association to Advance Collegiate Schools of Business International. Very high presence among research oriented mid to large sized State universities. ACBSP Accreditation Council for Business Schools and Programs. Most members are mid-sized teaching-oriented institutions. Significant representation of for-profit private colleges (Most members are small teaching focused private liberal arts colleges. For-profit colleges, thanks to increased scrutiny over their practices, seem to consider ACBSP accreditation as critical to validating their quality). IACBE International Assembly for Collegiate Business Education. Most members are small teaching-focused private liberal arts colleges. Source: George (2018) The majority of sales programs reside in business schools holding AACSB accreditation (SEF Annual, 2020). AACSB (2020) has specific evaluation standards for any major leading to a business degree, such as a Bachelor of Science in Professional Selling. The other two accreditation agencies utilize similar standards for program (major) evaluation. AACSB standards classify a minor, concentration, specialization, certificate, or track as a microcredential. They expect that these types of credentials should be reviewed for quality; however, a formal Assurance of Learning (AoL) systematic process with competency goals in 53 each program is not required. The other two accrediting agencies have similar requirements in that only programs resulting in degrees (e.g. majors) are formally evaluated, not minors or specializations. For this reason, programs that offer anything other than a major in Professional Selling are not required to publicly report accreditation results related to their sales programs (AACSB, 2020). The AACSB accreditation process can be a useful tool for the development of an educational evaluation framework. Lakhal and Sevigny (2015) summarized the AACSB AoL process as these five steps: 1. Define student learning goals and objectives 2. Align curricula with the adopted goals 3. Identify instruments and measures to assess learning 4. Collect, analyze, and disseminate information 5. Use information for continuous improvement Step 3, the identification of instruments and measures to assess learning can be accomplished using any of three acceptable approaches under AACSB standards. These are selection, course-embedded measures, and demonstration through stand-alone testing or performance. Selection is accomplished if students are selected to participate in the program if the learning goals have already been developed and met through their prior academic experiences. Course-embedded assessment is the process of using artifacts generated through classroom activities to assess achievement of student learning objectives at a program level. It builds on the daily work (assignments, tests, projects, etc.) of students and faculty members, (McConnell, Hoover, & Miller, 2008. p. 20). The option of demonstration occurs at the end of 54 the academic program and utilizes assessment tasks outside the normal classroom that are generally designed by the faculty, according to Lakhal and Sevigny (2015). In practice, however, they reported that demonstration is frequently measured in the classroom unless it is being accomplished by a standardized exam, such as a graduation exam. No matter the choice, they concluded that a benchmark must be determined to evaluate acceptable performance. Other Examples of Sales Program Evaluation in Higher Education Aside from formal accreditation requirements for programs with sales majors, academic literature regarding the evaluation of sales education programs based on outcomes was very limited. Bolander et al. (2014) offered the following statement when writing about the lack of outcomes studies in this area: Nevertheless, one subfield of marketing, professional selling, has for years proffered anecdotal claims that performance of graduates with formal sales education is higher, and turnover much lower, than their peers (Fogel, Hoffmeister, Rocco, & Strunk, 2012). However, a search of the scholarly, peer-reviewed literature reveals no empirical validation of these claims. One is left to wonder to what extent these assertions hold beyond the wishful thinking of the sales educators that make them! (p. 169) Kirkpatricks four-level model (1996) has been applied in non-academic, organizational settings as a method to evaluate sales training programs, notably in the retail industry (Tan & Newman, 2013). Praslova (2010) proposed the use of Kirkpatricks model in academic program evaluation, suggesting it served as a comprehensive and systematic approach to aligning criteria for educational effectiveness with specific indicators of achievement of these criteria... (p. 215). Blooms taxonomy (1956) has been utilized at the course level as a means for the evaluation of skills within sales courses. Healy et al. (2011), in the Journal of Instructional 55 Pedagogies, utilized Blooms taxonomy and experiential learning theory to examine two courses in a successful professional sales program. Experiential learning is rooted in hands-on practical exercises and experiences, while Blooms taxonomy was used by grouping objectives into three groups: low level knowledge or understanding; medium level application or analysis; and high level synthesis or evaluation. They combined the two frameworks to show how experiential learning in the sales classroom could incrementally move students to higher levels of learning. They used the professional selling and negotiation courses as examples. This combined framework, shown in Table 9, provided specific components mapped to the phases and modes of experiential learning and the levels of Blooms taxonomy. Table 9 Mapping Course Components for Experiential Learning and Blooms Taxonomy Grasping Experience Transforming Experience Concrete Experience Abstract Conceptualization Reflective Observation Active Experimentation Knowledge and Understanding Past Experience Course Readings, Lectures Homework, Quizzes n/a Application and Analysis In-Class Exercises, Early Role Plays Class Discussions, Early Role Play Debriefings Role Play Write-Up In-Class Exercises, Early Role Plays Advanced Role Later Role Plays, Later Role Play Play Write-Up, Sales Challenge Debriefings, Sales Sales Challenge Participation Challenge Feedback Term Paper Source: Reproduced from Healy, Taran, and Betts, 2011 Synthesis and Evaluation Later Role Plays, Sales Challenge Participation At the assignment level within a single course, Mullen and Larson (2016) reported on a set of learning outcomes related to a sales shadowing program at Georgia Southern University. Seven learning outcomes were identified based on Blooms taxonomy and students were 56 evaluated through the use of rubrics. Although the outcomes were developed specifically for this job shadowing exercise, they provided an example of how Blooms taxonomy could be used in the development of an evaluation framework. The seven learning outcomes for the sales shadowing assignment were as follows: 1. Student will be able to prospect, select, prepare, and plan a sales call 2. Student will demonstrate evidence of completed research and successful application of planning a sales call 3. Student will apply business communication skills by constructing a letter to the sales manager 4. This assignment simply ensures that students are on schedule, and can coincide with a time management discussion 5. Student will critique and reflect on the sales meeting. Student will apply concepts and skills taught in classroom to observations from a real-world experience 6. This assignment supports discussion on professionalism, relationship management and follow-up, and can coincide with a CRM discussion 7. Student will be able to receive constructive criticism on their performance outside of the classroom instructor. The evaluation provides feedback from the managers perspective on the interaction, and also asks for the managers interest in future shadowing projects Mullen and Larson (2016) also reported percent job placement and percentage of internship offers among participating students as evaluation measures for the exercise. An increase in support from the companies involved, in terms of time commitment, financial support, and expressions of desire to strengthen the partnership with the university were reported as additional positive measures resulting from the job shadowing (Mullen & Larson, 2016). A methodology for the development of program outcomes was suggested by Bolander et al. (2014). They utilized survey methodology and statistical analysis to assess the relationship between what is learned in university sales programs and the actual selling behaviors of recent 57 program graduates. The methodology used allowed a comparison between students who did and those who did not receive formal sales education in their undergraduate programs. The results supported the relationship between formal sales education and sales performance and the authors concluded that salespeople hired from universities with a formal sales program outperform their non-sales program counterparts after the first year of employment (pp. 176-177). Wood (2020) also used a survey to evaluate the sales career success of graduates of a sales leadership course. Both the students who had completed the course and their supervisors were surveyed and results showed that the sales leadership course resulted in increased leadership behaviors demonstrated post-graduation. Surveys have also been used to evaluate course pedagogy such as role plays and video recording, concluding that these techniques for sales education were perceived as effective or more effective than other teaching methods (Deeter-Schmaltz, and Norman Kennedy, 2011). Beyond these few publications and examples, established outcomes, measurements, assessment instruments, or benchmarks at a sales program level are very limited in the existing literature. Specifically, research related to an overall evaluation framework for a sales program within higher education, and the criteria for measuring student outcomes throughout such a program, was lacking, creating a significant gap in the literature. 58 Chapter 3: Methodology This project used a mixed methods methodology. Mixed methods research is a type of research in which a researcher or team of researchers combines elements of qualitative and quantitative research approaches (e.g., use of qualitative and quantitative viewpoints, data collection, analysis, inference techniques) for the broad purposes of breadth and depth of understanding and corroboration (Johnson, Onwuegbuzie, & Turner, 2007, p. 123). Mixed methods research provided several advantages for this study. For example, qualitative research was used to explain, clarify, and provide meaning to the information supplied in a literature review. Quantitative research provided student perspectives that were useful in supplementing the qualitative interviews and confirming the curriculum framework developed as a result of those interviews. Mixed methods was also useful since there were different stakeholders needed for data collection, all were small populations, and there were differences in specific areas of expertise for each group. Mixed methods research allowed for data collection from each group while adding to the credibility and validity of the findings and limiting bias related to a single methodology. For these reasons, mixed methods research allowed for a greater depth of knowledge from each group to inform the resulting recommendations for program development. No ethical concerns were identified with this methodology. Potential bias existed in the interpretation of the data by the researcher, who also created the program being evaluated. The methodology outlined in the study was developed to assist in mitigating this issue. The use of standardized rubrics completed by multiple different faculty or external sales experts, and objective evaluation data collected independently from sources other than the researcher are part of the methodology that addressed bias concerns. All data were held securely per the universitys institutional research policy and any identifying information, if collected, was eliminated from 59 the data files before they were stored on any electronic device or media. The methodology included four parts: Part 1: Benchmarking best practice universities Part 2: Qualitative inquiry for program development Part 3: Quantitative survey of program students Part 4: Program evaluation o Level 1: Reaction o Level 2: Learning o Level 3: Behavior o Level 4: Results The analysis of Parts 1 - 3 was used to create the program design. It included the identification of the knowledge and skills as outlined in the research question, as well as verification of the courses that were needed to deliver the identified knowledge and skills. Part 4 consisted of the program evaluation and was based on the adapted version of Kirkpatricks four level model (Kirkpatrick, 1996, 2005; Kurt, 2016; Praslova, 2010). Methodology Part 1: Benchmarking best practice universities Five institutions were benchmarked using secondary information, primarily university and other public websites. Programs selected for benchmarking included two regional, medium to large sized state universities, two small to medium Midwestern Catholic liberal arts universities, and one aspirational university. The aspirational university is a medium to large sized private university with a well-respected, successful professional selling program. The universities are labeled in this study as Regional, R1 and R2, Catholic, C1 and C2, and 60 Aspirational, A1. Benchmarking data supplemented information found in the literature review. The benchmarking process also informed interview questions in the interview guide development process for Part 2: Qualitative inquiry for program development. Methodology Part 2: Qualitative inquiry for program development Six qualitative interviews assisted in defining the knowledge and skills needed by sales program graduates. Two sales program directors at other schools were interviewed to gain perspectives on how other sales programs were developed, what they teach, and how they assess their programs. Three interviews were with current business school partners. These interviews allowed for a deeper understanding of the needs of local businesses regarding the desired student preparation prior to hiring. The final interview was performed with a senior member of a nonprofit organization that supports research and the promotion of university sales programs. This interview was used to gather a broader and research-based perspective on knowledge, skills, and coursework. The six interviews were implemented using in-depth semi-structured interviews. The interview guide used a logical flow from general to more specific questions that were discussed in a consistent manner, with probes designed to elicit more elaborate responses. This type of interview is flexible, accessible, and considered to be ideal for qualitative interviews (Adhabi & Anozie, 2017). The semi-structured interview guide was prepared and conducted by the researcher, who has been trained as a professional in qualitative research applications and techniques such as the following: interview guide development moderator training through the Burke Institute (Burke, 2020) part of coursework in the researchers doctoral program 61 The research instruments are included in Appendix B. Interview participants were sent an email invitation informing them of the purpose of the email, the purpose of the request for an interview, and the request for their voluntary participation. Five interviews were held in-person. For the remaining interview, the invitation to participate specified a telephone appointment to be scheduled. The appointment was scheduled for one hour and it lasted approximately 45 minutes. There was no compensation for the interviews. Interview sessions were not recorded. The interview guide was not sent to the participants before or after the interview. The six interviews were held with the following groups. Faculty at Other Schools. Two sales program directors were interviewed to gain insight from their experiences running successful sales programs at their universities. Both of their universities were USCA members. The interview topics included key learnings from their experience, sales program management, partnership sources and management, curriculum, pedagogy, knowledge and skills, and future plans. Those selected in this sample were chosen based on convenience, proximity, and willingness to participate. Business Partners. Three sales executives previously involved in the sales program were interviewed. Interviews focused on partnership needs and wants, partnership management, experience with other universities, and knowledge and skills of new graduates. Two interviews were conducted with executives at Fortune 1000 insurance and investment firms. A managing director was interviewed during the spring of 2020 and a Director of Internship Recruitment from a large regional office was interviewed in February of 2021. The third interview was held in the spring of 2020 with a District Manager at a nationally recognized logistics and distribution company. Both companies involved in the interviews were actively promoting internships and full time positions to sales students in the business school at the time of the interviews. 62 Non-Profit Organization Supporting University Sales Programs. One interview was held with a senior member of this organization. This interview focused on starting and growing a sales program, key learnings from other universities, curriculum and pedagogy, student knowledge and skills development, the value of their organizations to universities, how small universities compete and grow, sourcing and managing partnerships, and upcoming changes and advances that may impact sales programs. Methodology Part 3: Quantitative survey of program students A quantitative survey was administered to current sales students in the business school. The survey instrument consisted of 40 data points, and a copy is included in Appendix B. The sales program benefits section of the survey consisted of one question with six data points. This question was developed by the researcher due to its institution-specific content. Another section of the survey used the Intent to Pursue a Sales Career (ITPSC) 13 item short-form scale (Peltier et al., 2014). The ITPSC instrument used for this study was approved by the ITPSC author and copyright holder (Peltier et al., 2014). An additional survey section related to learning specific skills within courses taken in the business school was designed by the researcher and used a standard five-point Likert-type scale from strongly agree to strongly disagree. For this section, question wording was informed from previous sales education surveys identified in the literature (Barat & Spillan, 2009; Bolander, et al., 2014; Bristow et al., 2006; Peltier et al., 2014). The survey was fielded using the Qualtrics survey tool in the online format. Survey invitations were sent using the university email addresses of all members of the relevant population of current students who had taken or were taking a sales class during the spring 2021 or fall 2021 semester. All invitations informed the potential respondent of their rights, estimated survey completion 63 time, and individual respondent confidentiality. All other Institutional Research Board procedures were followed. No incentives were offered. The goal of the student survey was to assess current perceptions of the knowledge and skills gained through sales courses that had been recently implemented at the university in this study, and their alignment to the knowledge and skill needs that were identified in Part 1: Benchmarking best practice universities, and Part 2: Qualitative inquiry for program development. Survey results also helped confirm that the overall program objectives were being met, and assisted in the identification of any needed adjustments within individual courses. Additional survey questions were used to assist in future program development and were limited in their applicability to the research question under study. These additional questions centered on perceptions of sales careers and the identification of needs and wants relative to course topics, skills, competitions, facilities, external partnerships, and future career aspirations. Sample Size. The population surveyed was a census sample of current university sales course students, with the sample list drawn from all sales course rosters. The population was 109 students, comprised of students from the spring 2021 and fall 2021 semesters. Email invitations were sent during the final four weeks of the semester for spring and fall 2021. Participants were given two weeks to respond, with a reminder email sent to all non-responders one week after the initial invitation. The total number of usable survey responses was 63, resulting in a 57.8% response rate, a good representation of the participants. Methodology Part 4: Program evaluation The program evaluation model that was implemented was based on Kirkpatricks four level model (Kirkpatrick, 1996, 2005; Kurt, 2016). This model was chosen based on its 64 traditional use in business and organizational contexts, as well as its adaptation specific to higher education program evaluation (Praslova 2010). Arthur, Tubre, Paul, & Edens, (2003) utilized Level 1 and 2 of the four levels model for evaluation of teaching effectiveness in higher education. Level 1: Reaction, and Level 2: Learning, share similar characteristics across all training and educational settings and were therefore easily applicable to higher education. Applying Level 3: Behavior, and Level 4: Results, to higher education required some adaptation of the model. Praslova (2010) proposed this adaptation specific to the context and purposes of the university setting. In this adapted model, the four levels remained the same, Reaction, Learning, Behavior, and Results, but the evaluation criteria and suggested evaluation methods differed for Level 3 and Level 4. Table 10 shows the adapted model used for this program evaluation. Table 10 Sales Program Evaluation Model using the Adapted Kirkpatrick Four Levels Criteria Level 1: Reaction Level 2: Learning Level 3: Behavior Learning in Higher Educationa Student affective reactions and utility judgments Direct measures of learning outcomes, knowledge tests, performance tasks or other graded work Evidence of student use of knowledge and skills learned in the program and applied in subsequent work, e.g., internship, development of a professional resume, and other behaviors outside the context in which initial learning occurred Alumni career success, graduate school admission, service to society, personal stability a Taken from Praslova (2010). Level 4: Results b Indicators and Instruments Student evaluations of instruction all sales courses included Pre and post-evaluation of knowledge and skills in the classroom setting in the introductory and advanced sales courses End-of-course final project in 400-level sales coursesb, supervisor internship evaluation of all students participating For all sales program students: Job Placement Rate for Graduates, Starting Salaries, Job Titles Advanced Sales course competition results; Negotiation final project. 65 For this project, the Level 2 and 3 results were used to measure sales competencies and answer the research question being studied. Level 1 and 4 results did not explicitly address the research question. They were identified as necessary for a comprehensive review of the new program being implemented and are reported for that purpose. Sampling The sample for Part 4: Program Evaluation, Levels 1 through 4, was a census sample of all students participating in the sales program between May 2020 and May 2022. Research Instruments and Data Collection For Level 1: Reaction, the student evaluation of instruction questions are included in Appendix C. Data were collected as part of the university course evaluation process. All students were invited to participate in the student evaluation of instruction for each course. For Level 2: Learning, pre- and post-evaluation of knowledge and skills was assessed using a rubric for signature role play assignments in the introductory sales class and in the advanced sales class. These rubrics are included in Appendix C. Data collection was performed using direct observation by the course instructors during the first five weeks of class and then again during the final four weeks of class to provide the opportunity to evaluate growth during the semester (within group) as well as growth between the introductory and advanced course levels. All students in the courses were included. Level 3: Behavior, included three types of measures. First, end-of-course final projects in two senior-level courses were used. The advanced selling course used an external sales competition rubric and result as the measure of knowledge and skills applied. The negotiation course used the final negotiation role play scores. Rubrics used are included in Appendix C. The next measure in Level 3 was an internship evaluation done by supervisors of student interns. For 66 each sales student who had completed a sales internship, the results of the supervisor evaluation for the two skills relevant to the sales program were collected and recorded. These data are collected by the universitys career services office as part of the internship experience. The rubric used for this evaluation is included in Appendix C. For Level 4: Results, data collected included job placement rate, starting salary, and job title for each sales program graduate. These data were collected by the career services office. Data were summarized in an Excel spreadsheet as follows: Job placement rate of sales program graduates reported as a percentage of students obtaining a full time sales position within six months of graduation Average starting salary of sales program graduates, reported as a mean value and a median value of student-reported salary, collected from the university survey of new graduates within six months of graduation Job titles of sales program graduates who obtained a full time sales position within six months of graduation in a list format Data Analysis Plan Part 1: Benchmarking best practice universities The benchmarking results were reviewed for similarities and differences. Findings were used to inform other parts of the methodology for the project. The data collected were summarized in tabular form. Part 2: Qualitative inquiry for program development Qualitative data from the interviews were analyzed in three steps: data reduction, data display, and conclusions and verification (NSF, 1997). 67 In data reduction, written notes from the interviews were transcribed. Common patterns and themes related to knowledge and skills were identified and categorized. The knowledge and skill categories and themes were then grouped according to common terminology and displayed in a matrix to identify patterns and critical points of agreement and difference to understand the needs of each stakeholder type. Verification of the findings from this step consisted of revisiting the original interview notes to cross-check and verify the emergent conclusions. The display matrix was assessed for meaning and used in the final program design. Part 3: Quantitative survey of program Survey analysis was completed after downloading all survey results into a Microsoft Excel spreadsheet. The analysis consisted of frequencies, cross-tabulations, measures of central tendency, and measures of variance. Statistical treatments, such as a t-Test of means, were implemented where relevant. Open-ended comments were not relevant and therefore not utilized in this analysis. All results from Parts 1 through 3 assisted in creating the professional selling program design and the development of the needed knowledge and skills to be included in each course. Part 4: Program evaluation The data analysis plan for the sales program evaluation was implemented for each of the four levels in the Kirkpatrick model. Level 1: Reaction. Data were provided through the universitys course evaluation software. These evaluations consisted of seven questions related to the course and six questions related to the course instructor. Each question was evaluated by the student using a five-point verbal scale of strongly agree, agree, neither agree nor disagree, disagree, and strongly disagree. Results were analyzed by reviewing the means and standard deviations shown by question for 68 each sales course offered. Differences between semesters, where available, and between introductory and advanced course levels were compared. Mean scores were used to assess student affective reaction and utility judgments on the seven criteria rated for each course. Level 2: Learning. This evaluation utilized a pretest-posttest design for the introductory level and the advanced level sales courses. The nine core selling skills were evaluated using role play scenarios and a rubric that measured five sales concepts during the role play. The rubric included levels of evaluation ranging from Mastery to Not used at all. The maximum points (e.g. Mastery) for each category was 10 and the maximum point for the overall score was 50, or the sum of the five category points earned. The analyses included the following: descriptive statistics, including frequencies, for each sales concept percent of students achieving the mastery level within group (within a class section by level) comparisons, pre- and post-intervention, using a paired sample t-Test where pre and post level was nominal and total points earned on the rubric was interval level between group comparison of end of semester scores (introductory versus advanced sales course), using an independent samples t-Test Level 3: Behavior. This evaluation consisted of two different analyses including end-of- program projects and internship performance. The end-of-program final project data were gathered from rubrics for evaluating the final projects in the two 400-level courses offered. Each student was evaluated by a mix of sales professionals and course professors. The analysis consisted of the following: Descriptive statistics, including frequencies, for each knowledge or skill assessed Percent of students achieving the mastery level 69 Identification of areas of strength and areas with opportunities for improvement among the course participants overall Each student completing a sales internship was evaluated by their supervisor on communication skills and teamwork. Mean scores were reported along with a comparison to all business school student internship scores. Level 4: Results. This evaluation consisted of obtaining and reporting on the following metrics, along with comparisons to business students overall and the industry averages from the National Association of Colleges and Employers (NACE) First Destination Dashboard (2019): Job placement rate, defined as obtaining full time, professional placement in a business role within six months of graduation, as reported by the universitys career center Starting salary, self-reported by the graduate in the student exit survey or as reported by the universitys career center Job title, as self-reported by the graduate in the student exit survey or as reported by the universitys career center 70 Chapter 4: Study Results To prepare students for success in professional selling careers, universities need to deliver a curriculum that teaches the desired knowledge and skills. This study was designed to examine the curriculum needs when creating a new selling program. The goals of this study were to a) identify the knowledge and skills needed by a graduating student pursing a sales career, b) implement the findings in current or newly formed courses that form a professional selling program of study, and c) evaluate the effectiveness of the program. A four-part, mixed methods study design was utilized. First, secondary research was performed to benchmark five successful programs in Part 1. This was followed by Part 2, a qualitative step using in-depth interviews to gather insights and perceptions from academic experts heading sales programs, business executives who hire and train new graduates in sales roles, and an industry expert from an organization focused on university sales education research. Parts 1 and 2 were used for the initial program design. Part 3 consisted of two occurrences of a quantitative survey of sales students within the program under study. Part 3 of the study was used to confirm the initial program design, including the knowledge and skills to be introduced and reinforced in each course. Part 4 of the study was the evaluation. The study was guided by the research question: What knowledge and skills do students need in preparation for a professional selling career, and how does implementing these in a formal curriculum improve students' sales competencies? Sales Program Design and Implementation, Parts 1 through 3 Program Design Part 1 Results: Benchmarking Similar, Regional, and Aspirational Programs Five university sales programs were selected for benchmarking. A review of the university and program websites were used to document the main characteristics of each 71 program, the knowledge and skills taught in the introductory and advanced sales courses at each school, and the types of sales courses offered. Two schools were selected based on their characteristics of Catholic, small to medium in size, and private liberal arts focus. They are labeled C1 and C2 in the following tables. Two regional schools were selected based on proximity to the university being studied in this project and are labeled R1 and R2 in the following tables. One school was selected as an aspirational school and is labeled A in the following tables. The aspirational program is well known as a national leader in undergraduate sales education. All schools chosen are members of the USCA. The main characteristics of each program are shown in Table 11, including an indication of which schools offer a sales major, minor, certificate, or specialization as part of their program. Table 11 Sales Program Characteristics School C1 C2 R1 R2 A Years program has existed 23 5 25 31 35 Sales Accreditation Students 40 70 250 150 452 ACBSP ACBSP AACSB AACSB AACSB Major Minor Certificate Specialization Table 12 documents the stated knowledge and skill outcomes for the introductory level sales course and the advanced level sales course at each benchmarked university. Two courses at each university were identified for this comparison based on the USCA guidelines stating that member programs must offer an introductory and an advanced sales course, and one must be a 72 pre-requisite for the other. Table 12 indicates when the knowledge or skill was specifically stated as a learning outcome for the course. Table 12 Knowledge and Skills Outcomes at Comparison Schools Knowledge/Skill C1 C2 Buying Motives A IA Persuasive Communication IA IA Sales Process IA IA Professionalism I Sales Ethics A Presentation Skills A A Teamwork A Time Management A Career Management I I Negotiation A I Salesforce Management A Relationship Management A IA Sales Technologies A A Note. I = Introductory Course A=Advanced Course. R1 I IA I I I IA I IA A A R2 I I I A A A I A A IA IA I I A A I A Table 13 summarizes the course types offered in each benchmarked program. Only courses that included the word sales or negotiation in the course title were included in this comparison. Course types were identified through both course titles and descriptions. Specific titles varied slightly by the university. The number of sales courses ranged from three to six. In addition to the introductory and advanced course, three schools also offered a sales management course and three offered a negotiation course. 73 Table 13 Sales Course Types Offered by Program Course Type Introduction to Selling/Principles of Selling Advanced Selling Sales Management/Salesforce Management Negotiation Sales Internship Professional Sales Practices Sales Ethics Specialty Sales (healthcare) Sales Technologies Sales Strategy TOTAL OFFERING C1 6 Comparison School C2 R1 R2 A 5 5 3 5 Sales Program Design Part 2 Results: Qualitative Inquiry Six in-depth qualitative interviews were completed to gain additional insight into sales program design and curriculum development. Coding was performed manually using the transcribed notes from all interviews. Additional documentation was provided by two of those interviewed. The curriculum, knowledge, skills, and pedagogy recommendations summarized from the interview notes can be found in Appendix D. Descriptions of knowledge and skills were classified using common terminology. For example, phrases or words describing body language, attire, expression, and use of professional language were grouped into a professionalism knowledge area. Meaning was extracted through conceptually meaningful words and phrases and then four themes were derived through a process of refinement and revision. These four identified themes were: 1. Curriculum Development; 74 2. Knowledge and Skills needed in sales education; 3. Pedagogy; 4. Funding of Sales Programs. Three of these themes were identified as relevant to this study. One theme, funding of sales programs, was less relevant to the research question being studied, and therefore was not analyzed further. Sales Program Curriculum Development and Implementation Results from Part 1, benchmarking, and Part 2, qualitative inquiry, were evaluated to define the sales program. This definition included three overall program learning outcomes, courses to be offered, and learning objectives for each course. The evaluation included assimilating all Parts 1 and 2 findings to determine a course framework. This framework specified nine core knowledge and skill attributes needed in the introductory course. These are: listening, questioning, the sales process, building trust, persuasion, communication, professionalism, building rapport, and solving problems. Ten additional attributes were identified for inclusion in the advanced sales courses, with a natural progression of introduction, reinforcement, and assessment as students move from the introductory course to the more advanced courses. Suggested pedagogies were identified to support the progression of fundamentals in the introductory course and more advanced selling skills and attributes in the advanced sales courses. The course framework can be found in Appendix D. The proposed program was then compared to the Part 1 and Part 2 detailed notes, documents, and findings to ensure consistency. Finally, an in-person review of the program was held with the two faculty members teaching courses in the sales program to confirm that all information was understood and accepted for implementation. This ensured that all course 75 revisions and new course objectives would be met during the next course offering. After agreement among the sales faculty, a review and approval was completed with the remaining business faculty, the dean, and the assistant dean. The program was then submitted for the formal university approval process. A nine-credit hour specialization in professional selling was approved by the university in early 2020. The three overall program learning outcomes were that students who complete the specialization would: 1. Understand the sales process. 2. Understand sales concepts and how to apply them. 3. Know how to structure a sales presentation. The approved program required the completion of the introductory sales course, which was already being offered once per academic year, an advanced sales course, and one additional course from a list of options. The additional course option included a negotiation course offered for the first time in spring of 2020, and a sales technology course that was offered for the first time in the fall of 2021. Students could also elect to complete a sales-specific internship for three credit hours. Sales Program Design Part 3 Results: Quantitative Survey of Students To confirm the sales program framework that was implemented, a quantitative survey was performed with students completing and of the courses. The survey was fielded at two different time periods, April 2021, November 2021. This allowed for a representation of students completing all courses during fall 2020, spring 2021 and fall 2021 semesters. Sample size and response rates for the survey are shown in Table 14 and survey demographics are shown in Table 15. 76 Table 14 Sampling Data from Student Survey Survey Date Apr 2021 Sample N 62 Completed Responses n % 34 54.84% Nov 2021 47 29 61.70% Total 109 63 57.80% Table 15 Survey Participant Demographics Characteristic Gender Female Male Prefer not to say Year in school Graduate in 2021 Graduate 2022 or after Note. n = 63. n % 23 39 1 36.51 61.90 1.59 15 48 23.80 76.20 Survey results are presented in the following tables using totals from both survey occurrences. Some students completed the survey after taking multiple sales courses. For this reason, the introductory sales course analysis included only those respondents who had not yet completed any other sales course. The advanced sales course analysis included students who had completed, at minimum, both the introductory and the advanced sales courses. This allowed for a comparison of knowledge and skills obtained in the first sales course, as compared to the growth in knowledge and skills from completion of the advanced sales course. For the other two courses shown, students may have completed any combination of courses. Survey results show that eight of the 63 respondents, or 12.7%, completed more than two courses at the time of the survey. 77 Students selected all of the courses they had completed previously or were completing during the survey period. Table 16 shows these results. Table 16 Sales Courses Completed Students Course Introductory Sales Advanced Sales Negotiation Sales Technology n 49 18 22 17 % 46.23 16.98 20.75 16.04 Students were asked to indicate the types of university-sponsored sales competitions in which they had competed. These results are shown in Table 17. Table 17 Competition Experience Students Competition Type N % Internal 20 25.00 External 30 37.50 None 30 37.50 Note. Students were able to select both internal and external. Perceptions regarding the three overall program learning outcomes were included in the survey. Table 18 shows the results for all respondents in all sales courses. The introductory and advanced level course results were compared and are shown in Table 19. For the introductory course results, only those students who had not yet taken any of the other sales courses were included. All questions utilized a 5-point Likert scale ranging from 1 = strongly disagree to 5 = strongly agree. 78 Table 18 Sales Program Outcomes Agreement with Statement Mean Std Dev I understand the sales process 4.17 0.76 I understand sales concepts and how to apply them 3.98 0.96 I know how to structure a sales presentation Note. n = 63. 3.95 0.95 As shown in Table 19, there was an observed difference in the mean scores between the introductory (M=3.85, SD=0.78) and the advanced course (M=4.43, SD=0.42) for understanding the sales process, with the advanced course students showing significantly higher agreement. The two-sample t-Test demonstrated a significant effect for agreement, t(39,17) = -2.50, p = .008. There was an observed difference in the mean scores between the introductory (M=4.00, SD=0.63) and the advanced course (M=4.60, SD=0.40) for understanding sales concepts and how to apply them, with the advanced course students showing significantly higher agreement. The two-sample t-Test demonstrated a significant effect for agreement, t(39,17) = -2.62, p = .006. The t-Test demonstrated no significant effect for knowing how to structure a sales presentation, t(39,17) = -1.35, p = .091. The difference between mean ratings among the introductory course students (M = 4.05, SD = 0.73) and advanced course students (M = 4.40, SD = 0.69) were not significant. 79 Table 19 Sales Program Outcomes: Introductory and Advanced Sales Course Comparison Agreement with Statement Mean Introductory Advanced Sales Sales I understand the sales process I understand sales concepts and how to apply them 3.85 4.43** 4.00 4.60** I know how to structure a sales presentation 4.05 4.40 t(39,17) p -2.50 .008 -2.62 .006 -1.35 .091 Note. n=39 Introductory Sales, n=17 Advanced Sales. *Significant difference between Intro and Advanced Sales courses at p=.05 ** Significant difference between Intro and Advanced Sales courses at p=.01 Students were asked to rate their agreement with statements about the selling skills taught in the sales program. All questions utilized a 5-point Likert scale ranging from 1 = strongly disagree to 5 = strongly agree. Table 20 shows these results by course. The two-course series of the introductory level and the advanced level sales course were directly compared using a twosample t-Test of means. The sample for the introductory sales course included only those students who had not yet taken the advanced sales course. The sample for the advanced sales course included students who had completed both the introductory and the advanced courses. As shown in Table 20, there was an observed difference in the mean scores between the introductory (M=4.05, SD=0.99) and the advanced course (M=4.63, SD=0.62) for sales presentations, with the advanced course students showing significantly higher agreement, t(39,16) = -2.13, p = .019. There was an observed difference in the mean scores between the introductory (M=3.95, SD=0.76) and the advanced course (M=4.44, SD=0.73) for rapport building, with the advanced course students showing significantly higher agreement, t(39,16) = 80 -2.19, p = .016. There was an observed difference in the mean scores between the introductory (M=4.11, SD=0.69) and the advanced course (M=4.56, SD=0.51) for consultive communication, with the advanced course students showing significantly higher agreement, t(39,16) = -2.39, p = .010. There was an observed difference in the mean scores between the introductory (M=3.97, SD=1.04) and the advanced course (M=4.56, SD=0.51) for dealing with objections, with the advanced course students showing significantly higher agreement, t(39,16) = -2.15, p = .018. There was an observed difference in the mean scores between the introductory (M=3.77, SD=0.74) and the advanced course (M=4.50, SD=0.52), for gaining commitment for the sale, with the advanced course students showing significantly higher agreement, t(39,16) = -3.59, p = <.001. There was an observed difference in the mean scores between the introductory (M=3.62, SD=0.75) and the advanced course (M=4.81, SD=0.54) for collaboration and teamwork, with the advanced course students showing significantly higher agreement, t(39,16) = -5.79, p = <.001. There was an observed difference in the mean scores between the introductory (M=3.21, SD=0.98) and the advanced course (M=4.50, SD=0.73) for negotiation, with the advanced course students showing significantly higher agreement, t(39,16) = -4.77, p = <.001. The t-Test demonstrated no significant effect for asking questions, t(39,16) = -1.58, p = .060. The difference between mean ratings among the introductory course students (M = 4.67, SD = 0.48) and the advanced course students (M = 4.88, SD = 0.34) were not significant. The tTest demonstrated no significant effect for effective listening, t(39,16) = -1.07, p = .144. The difference between mean ratings among the introductory course students (M = 4.67, SD = 0.48) and the advanced course students (M = 4.81, SD = 0.40) were not significant. Mean scores for those who had taken the negotiation course were all between 4.33 and 4.89. Mean scores for the sales technology course ranged from a low of 3.94 for negotiation 81 skills to a high of 4.35 for sales presentation skills. There was no pre-requisite sales course for these two courses so no comparison of mean scores was performed. Table 20 Mean Results for Selling Skills The sales course(s) helped me build selling skills in: Introductory Sales Advanced Sales Negotiation Sales Technology Sales Presentations 4.05 4.63* 4.33 4.35 Rapport Building 3.95 4.44* 4.39 4.06 Consultive 4.11 4.56* 4.72 4.24 communication Dealing with 3.97 4.56* 4.56 4.12 objections Gaining commitment 3.77 4.50** 4.56 4.12 Collaboration and 3.62 4.81** 4.72 4.24 Teamwork Negotiation 3.21 4.50** 4.72 3.94 Asking Questions 4.67 4.88 4.83 4.41 Effective listening 4.67 4.81 4.89 4.35 Note. n=39 Introductory Sales, n=16 Advanced Sales, n=22 Negotiation, n=19 Sales Technology. *Significant difference between Intro and Advanced courses at p=.05 ** Significant difference between Intro and Advanced courses at p=.01 Students were asked to rate their agreement with six statements about self-management and customer relationship management skills or knowledge learned in the sales courses. All questions utilized a 5-point Likert scale ranging from 1 = strongly disagree to 5 = strongly agree. Table 21 shows these results by course type. The two-course series of the introductory level and the advanced level sales courses were directly compared using a two-sample t-Test of means. The sample for introductory sales included only those students who had not yet taken advanced 82 sales. The sample for advanced sales included students who had completed both the introductory and the advanced courses. There was an observed difference in the mean scores between the introductory (M=3.07, SD=1.33) and the advanced course (M=3.67, SD=0.95) for sales technologies, with the advanced course students showing significantly higher agreement, t(39,16) = -1.75, p = .043. There was an observed difference in the mean scores between the introductory (M=4.33, SD=0.62) and the advanced course (M=4.73, SD=0.46) for problem solving, with the advanced course students showing significantly higher agreement, t(39,16) = -2.26, p = .014. There was an observed difference in the mean scores between the introductory (M=3.36, SD=0.99) and the advanced course (M=4.07, SD=1.16) for dealing with ethical issues, with the advanced course students showing significantly higher agreement, t(39,16) = -2.25, p = .014. The t-Test demonstrated no significant effect for professionalism, t(39,16) = -0.87, p = .193. The difference between mean ratings among the introductory course students (M = 4.74, SD = 0.50) and advanced course students (M = 4.87, SD = 0.35) were not significant. The t-Test demonstrated no significant effect for networking, t(39,16) = 0.83, p = .206. The difference between mean ratings among the introductory course students (M = 4.72, SD = 0.46) and the advanced course students (M = 4.60, SD = 0.51) were not significant. The t-Test demonstrated no significant effect for time management, t(39,16) = -1.54, p = .065. The difference between mean ratings among the introductory course students (M = 4.23, SD = 0.84) and the advanced course students (M = 4.60, SD = 0.63) were not significant. Mean scores for those who had taken the negotiation course ranged from a low of 3.56 for sales technologies to a high of 4.72 for problem solving. Mean scores for the sales technology course ranged from lows of 4.28 for networking with others to highs of 4.62 for problem solving 83 and professionalism. There was no pre-requisite sales course for these two courses so no comparison of mean scores was performed. Table 21 Mean Results for Self-Management and Customer Relationship Management The sales course(s) helped me build skills in: Introductory Sales Advanced Sales Negotiation Sales Technology Sales Technologies 3.07 3.67* 3.56 4.33 Problem Solving 4.33 4.73* 4.72 4.62 Dealing with Ethical 3.36 4.07* 4.39 4.33 Issues Professionalism 4.74 4.87 4.83 4.62 Networking 4.72 4.60 4.39 4.51 Time Management 4.23 4.60 4.33 4.28 Note. n=39 Introductory Sales, n=16 Advanced Sales, n=22 Negotiation, n=19 Sales Technology. *Significant difference between Intro and Advanced Sales courses at p=.05 Seven questions to gauge perceptions of the selling profession were included in the survey. The first three questions were positively worded and the other four questions were negatively worded. These questions were included to assess perceptions after completing the introductory course, and if those perceptions changed as students were exposed to courses beyond the introductory level. Table 22 shows the mean results by course type. All questions utilized a 5-point Likert scale ranging from 1 = strongly disagree to 5 = strongly agree. The twocourse series of the introductory level and the advanced level sales course were compared using a two-sample t-Test of means. The sample for the introductory sales course included only those students who had not yet taken the advanced sales course. The sample for the advanced sales 84 course included students who had completed both the introductory and the advanced courses. No significant differences were observed between the introductory and the advanced course results. Table 22 Mean Results for Perceptions of the Selling Profession Salespeople Are respected by others Introductory Sales Advanced Sales Negotiation Sales Technology 3.74 3.53 3.61 3.40 Are admired by others 4.08 4.13 4.33 Are perceived favorably by others 3.77 3.93 3.83 Take advantage of uneducated 3.21 2.87 2.94 buyers Stretch the truth to make a sale 3.23 2.87 2.78 Make something up when they do 2.85 2.47 2.39 not know the answer to a question Misrepresent guarantees and/or 2.87 2.73 2.89 warranties Note. n=39 Introductory Sales, n=17 Advanced Sales, n=22 Negotiation, n=19 Sales Technology. 3.87 3.40 2.93 3.27 3.00 3.27 Three questions to gauge perceptions about a selling career were included in the survey. These questions were included to assess perceptions after completing the introductory course, and how those perceptions may change as students are exposed to courses beyond the introductory level. Table 23 shows the mean results by course type. All questions utilized a 5point Likert scale ranging from 1 = strongly disagree to 5 = strongly agree. The two-course series of the introductory level and the advanced level sales course were compared using a two-sample t-Test of means. The sample for the introductory sales course included only those students who had not yet taken the advanced sales course. The sample for the advanced sales course included students who had completed both the introductory and the advanced courses. No significant differences were observed between the introductory and the advanced course results. 85 Table 23 Mean Results for Perceptions about a Selling Career Course Advanced Negotiation Sales 4.53 4.56 Introductory Sales A sales career/selling Sales Technology Gives a sense of accomplishment 4.36 4.07 Is doing something worthwhile on 4.18 4.40 4.33 4.07 the job Is personally satisfying 4.21 4.40 4.28 4.00 Note. n=39 Introductory Sales, n=17 Advanced Sales, n=22 Negotiation, n=19 Sales Technology. Students were asked to state their interest in pursuing a sales career after graduation. Results are shown in Table 24. This question utilized a 5-point scale where 1 = not at all interested and 5 = very interested. Nearly 62% of students overall and 60% of those who had taken only the introductory course rated their interest as a 4 or 5. Just under 71% of those who had completed the advanced sales course rated their interest as a 4 or 5. Table 24 Interest in a Sales Career All Respondents n % 3 4.76 8 12.70 13 20.63 19 30.16 20 31.75 Introductory Sales n % 2 5.13 6 15.38 8 20.51 13 33.33 10 25.64 Advanced Sales Interest in Sales Career n % 1=Not at all interested 0 0 2 0 0 3 3 17.65 4 4 23.53 5 = Very interested 8 47.06 No Response 2 11.76 TOTAL 63 100 39 100 17 100 Note. Negotiation and Sales Technology counts are included in All Respondents To gauge student interest in current or potential future sales program features, students were asked to rate the importance of six opportunities. This question utilized a 5-point Likert- 86 type scale ranging from 1 = very important to 5 = very unimportant. Results in Table 25 indicated the strongest interest in involvement from sales professionals, followed by the opportunity to earn a certification in sales, facilities for role plays and video recording, participation in competitions, a sales minor, and a sales major. Table 25 Mean Results for Importance of Sales Program Features Benefit or Opportunity Mean Std Dev Involvement from Sales Professionals 4.47 0.60 Nationally recognized sales certification available 4.36 0.76 Role Play and other Video recording Facilities 4.32 0.80 Participating in Competitions Ability to earn a Sales MINOR Ability to earn a Sales MAJOR Note. n = 63. 4.28 4.25 4.07 0.90 0.79 0.94 Sales Program Evaluation, Part 4 of the Methodology Part 4 was based on the Kirkpatrick model for the evaluation of training and learning programs. Sales program results were assessed using four levels of criteria: reaction, learning, behavior, and results. Level 1, Reaction, showed the results for the student evaluations of instruction for sales courses and was used as part of the overall program evaluation. Level 2, Learning, summarized mastery of skills utilized in role play scenarios. Level 3, Behavior, examined the results from end of course projects in two courses, and student internship results. The Level 2 and 3 results were used to evaluate sales competencies. Level 4, Results, showed the outcomes measures of job placement percentage, average starting salary, and job titles for students who had recently graduated from the sales program. 87 Level 1 Reaction The results of the student evaluation of instruction for the sales courses are shown in Table 26. These results represented the aggregated sales program course evaluation results from the spring 202, and fall 2021 semesters. Using the standard university course evaluation instrument and process, students evaluated each course on the attributes listed using a 5-point Likert scale where 5=strongly agree, 4=agree, 3=neither agree nor disagree, 2=disagree, and 1=strongly disagree. Mean scores were used to assess student affective reaction and utility judgments on the criteria rated for each course. All four of the courses showed very positive results, with mean ratings at 4.5 or higher among all attributes for the three advanced courses, and all attributes above 4.2 for the introductory sales course. Exceptionally high ratings on several questions are noteworthy. For the introductory course, high scores were recorded for instructor mastery of the subject matter and instructors behavior reflecting concern for learning success. For the advanced course, instructors behavior reflecting concern for learning success and recommendation of instructor to another student were exceptionally high. In the negotiation results, areas with the highest scores were instructor mastery of subject matter, instructors behavior reflecting concern for learning success, and recommendation of instructor to another student. Prompt and useful feedback, instructor availability, and recommendation of instructor to another student were the areas with the highest scores. 88 Table 26 Student Evaluation of Instruction Results Attribute Introductory Sales Std M Dev Advanced Sales Std M Dev Negotiation Std M Dev Sales Technology Std M Dev Syllabus communicated 4.42 0.63 4.5 0.58 4.86 0.38 4.82 0.26 learning outcomes Readings, discussions, lectures, projects helped me 4.58 0.52 4.5 1.00 4.86 0.38 4.69 0.37 attain learning outcomes Multiple instructional 4.52 0.52 4.5 0.58 4.86 0.38 4.69 0.26 methods were used Activities and assignments 4.42 1.03 4.75 0.50 4.86 0.38 4.75 0.27 supported learning outcomes Activities and assignments challenged me to think more 4.58 0.97 4.5 1.00 4.86 0.38 4.63 0.36 deeply/critically about the course subject matter Would recommend course to 4.23 1.42 4.75 0.50 4.86 0.38 4.50 0.52 another student Instructor demonstrated 4.69 0.53 4.75 0.50 5.00 0.00 4.75 0.35 mastery of subject matter Instructor provided well4.54 0.79 4.75 0.50 4.86 0.38 4.69 0.37 organized learning activities Instructor provided prompt, 4.34 1.14 4.75 0.50 4.86 0.38 4.82 0.37 useful feedback Instructor was available on a 4.23 0.95 4.75 0.50 4.86 0.38 4.82 0.37 regular basis Instructor behavior clearly reflected genuine concern for 4.64 0.7 5.00 0.00 5.00 0.00 4.75 0.53 my learning success I would recommend 4.39 1.10 5.00 0.00 5.00 0.00 4.82 0.37 Instructor to another student Note. n=18 Introductory Sales, n=4 Advanced Sales, n=7 Negotiation, n=14 Sales Technology n=14. 89 Level 2 Learning This evaluation utilized a pretest-posttest design for the introductory level and the advanced level sales courses. Five sales concepts measuring the nine core selling skills were evaluated in each course using role play scenarios. The maximum points for each concept were 10 and the maximum points for the overall score was 50, or the sum of the points earned. The analyses included the following: 1. Mean and standard deviation for each concept; 2. Percent of students achieving the mastery level; 3. Within group comparison, pre- and post-intervention, using a paired sample t-Test where pre and post level was nominal and total points earned on the rubric was interval level; 4. Between group comparison of end of semester scores for the introductory versus advanced course, using an independent samples t-Test. The evaluation scale used was Mastery (10 9), Partial Mastery (8 7), Progressing (6 5), and Emerging (<4). The introductory course results are shown in Table 27. Two sections of the course, one in the spring of 2021 and one in the fall of 2021 were included in the total for this analysis. The courses were taught by two different instructors. The rubrics between semesters had slight wording differences. The five concepts evaluated showed a significant increase in mastery between the initial role play and the final role play evaluation. By the end of the semester, mastery levels ranged from a low of 43% of students for building trust and listening skills to a high of 80% of students for the sales approach, which measured the sales process, professionalism, and rapport building skills. 90 Table 27 Introductory Sales Course; Within Group Role Play Evaluation Introductory Course Initial Role Play Final Role Play Concept M M SD t(42) p SD Sales Approach 6.82 1.14 9.43 0.12 -13.65 Building Trust 6.74 1.00 8.62 0.14 -12.40 Effective Questioning 6.75 1.04 8.78 0.12 -13.40 Product Presentation 6.68 0.94 8.61 0.14 -15.05 Obtain Commitment 6.64 1.03 8.91 0.13 -16.28 Note. Paired t-Test of means (one-tailed). *** Significant difference between initial and final role plays at p<.001 <.001*** <.001*** <.001*** <.001*** <.001*** % students achieving mastery 81 43 55 50 52 The advanced course results are shown in Table 28. One section of the course from the fall of 2021 was included in this analysis. The rubric used the language of overcoming objections, unlike the introductory course rubric using the terminology of building trust. All five skills evaluated showed a significant increase in skill level between the initial role play and the final role play evaluation. By the end of the semester, mastery levels ranged from a low of 50% for obtaining commitment, which measured skills in persuasion and solving customer problems, to a high of 80% for the sales approach, which measured the sales process, professionalism, and rapport building skills. 91 Table 28 Advanced Sales Course; Within Group Role Play Evaluation Advanced Course Concept Initial Role Play Final Role Play M M SD t(10) p SD Sales Approach 6.60 1.65 8.90 1.10 -5.44 Trust/ Overcoming Objections 6.80 1.55 8.80 0.79 -6.00 Effective Questioning 6.70 1.06 8.70 0.95 -9.49 Product Presentation 6.90 1.45 8.90 0.99 -7.75 Obtain Commitment 6.60 0.65 8.40 1.07 -4.32 Note. Paired t-Test of means (one-tailed). *** Significant difference between initial and final role plays at p<.001 <.001*** <.001*** <.001*** <.001*** <.001*** % students achieving mastery 80 60 60 70 50 A between group comparison of the end of semester scores for the introductory and advanced courses was performed using an independent samples t-Test (two-tailed). These results are shown in Table 29. No significant differences were observed between the introductory and the advanced end-of-course role play comparison results. Table 29 Introductory and Advanced Course Between Group Final Role Play Comparison Introductory Concept Approach Building Trust/Overcoming Objections Effective Questioning Product Presentation Obtain Commitment Advanced M SD M SD t (42,10) 9.43 8.62 8.78 8.61 8.91 0.12 0.14 0.12 0.14 0.13 8.90 8.80 8.70 8.90 8.40 1.10 0.79 0.95 0.99 1.07 1.76 -0.58 0.27 -0.92 1.64 p 0.08 0.57 0.78 0.36 0.11 Level 3 Behavior This evaluation consisted of the end of course project in two courses and supervisor evaluations of student internships. 92 End of Course Projects. The end of course final project data were gathered using rubrics designed for evaluating the final projects in the two 400-level courses offered. In the advanced course, students were evaluated by industry experts and sales program faculty members. For the negotiation course, students were evaluated by a part-time faculty member and an external sales executive. For the advanced sales course, the end of course project used for evaluation was a national sales competition where the ten students in the course competed against six other nationally recognized sales programs. Students worked on one or more parts of the competition, including the pitch preparation, research, marketing, financial analysis, and live pitch to the judges. Each university team was evaluated and ranked by the industry sales experts serving as competition judges. The student team being studied for this project was ranked as number two by the external judges, against a total of seven teams. The competition scores were not able to be obtained. Four of the six other teams were recognized as USCA full member programs. Four of the six teams had 120 or more sales students each. One school reported 75 students and one reported 30 students. Using the competition rubric, the student team was evaluated and scored by the course instructor and a second part-time faculty member in the sales program. These scores were averaged and are reported in Table 30. The rubric used a scale of 1 to 5, with a 4.5 or above considered at the Mastery level and 3.5 to <4.5 considered to be Partial Mastery. The students were rated as achieving mastery in their approach, presentation of the solution, overcoming objections, and their action plan. Students were rated as achieving partial mastery in confirmation of needs and sales technology and analytics. 93 Table 30 Advanced Sales End of Course Team Project Evaluation Advanced Course Mean Score Mastery Observed Approach/Took the Lead 5.00 Yes Confirmation of Needs 4.25 Partial Technology & Analytics 4.25 Partial Communication/Presentation of Solution 5.00 Yes Overcame Objections 4.75 Yes Developed Action Plan 4.5 Yes Note. Mean score computed as the average of two instructor evaluations for the team. For the negotiation course, the end of course project used for evaluation was a final culminating negotiation project where each of the 20 students in the course participated as an individual negotiator against another student in the course. Students were assigned randomly. Scores were obtained for each student using a rubric to evaluate five different concepts on a 10 point scale, where 9 10 was considered mastery of the concept, 7 8 was considered partial mastery, 6 was considered emerging, 5 was considered still developing, and less than five was considered weak or limited use of the concept. The mean results are shown in Table 31. Fortyfive percent of the students were judged to have achieved mastery (n=9) and 40% achieved partial mastery (n=8). Two students were judged to have demonstrated skills that are emerging and one student was not able to demonstrate the needed skills and was judged as still in development. 94 Table 31 Negotiation End of Course Project Evaluation Negotiation Course Concept Evaluated Preparation Understanding Issues Questioning Generated Options/Solving Problems Built Trust Note. n=20 Mean Score out of 10 7.65 7.95 7.50 7.95 7.50 Sales Internship. Each student completing a sales internship was evaluated by their supervisor on communication and teamwork. Communication was one of the nine core selling skills in the selling skills framework established for the sales program being evaluated. Teamwork was one of the ten additional skills in the framework. The questions utilized a 5-point Likert scale ranging from 1 = strongly disagree to 5 = strongly agree. Fifteen sales program students were evaluated and their scores were compared to all 288 business students completing any business internship. Sales students were evaluated as performing well for both communication and teamwork skills, with both mean ratings at 4.60 or higher. While directionally higher than the results for all business students, there was no significant difference between the two groups. The results are shown in Table 32. Table 32 Sales Internship Supervisor Evaluation Attribute Mean Sales All Business Students Students Communication 4.60 4.52 Teamwork 4.69 4.61 Note. n=15 sales students; n=288 for all business students 95 Level 4: Results The fourth level of the evaluation consisted of obtaining and reporting on three student outcomes metrics. The three outcomes measures were the following: 1. Job placement rate, defined as obtaining full-time, professional placement in a business role within six months of graduation, as reported by the universitys career center. 2. Starting Salary, self-reported by the graduate in the student exit survey. 3. Job Title, as self-reported by the graduate in the student exit survey. Comparisons were made to all graduates of the business school as well as industry averages from the National Association of Colleges and Employers (NACE) First Destination Dashboard (2020). Six students completed the program and graduated with the sales specialization in 2021. No students dropped out of the program. All six students were male. The job placement rate was 100%. The median starting salary for these students was $65,750 and the mean was $75,028. Comparison data for job placement and starting salary are shown in Table 33. All six graduates obtained a sales-related professional position. The job titles they reported were: 1. Account Manager 2. Account Specialist 3. Assistant District Manager 4. Associate, Commercial Real Estate 5. Associate District Manager 6. Entrepreneur, Real Estate 96 Table 33 Career Outcomes and Comparison Outcome Job Placement % Mean Starting Salary $ a Sales Program Graduates (n=6) Universitys Business School Graduates (n=70) National Business School Graduates 2020 (n=95,233) 100 100 87a 75,028 53,299 55,572 NACE Class of 2020 Graduate Outcomes Report. The placement percentage includes employed, continuing education, military, and anyone else not seeking employment. 97 Chapter 5: Discussion and Evaluation of the Capstone Project Summary Summary of the Project and Significance of the Outcome This capstone project addressed the research question: What knowledge and skills do students need in preparation for a professional selling career, and how does implementing these in a formal curriculum improve students' sales competencies? There was some prior research on the knowledge and skills needed by business graduates in preparation for a sales career but nothing specific to small school programs. Research on measurement or evaluation of skill attainment had centered on specific courses or pedagogies rather than an entire curriculum. Addressing a comprehensive sales program development and evaluation process relative to a small university was nonexistent in the literature. The start of this project centered on determining the skills needed by sales graduates to develop sales competency and the curriculum required to deliver this result. Once the new curriculum was defined and implemented as a new program in professional selling, the second part of the project, an evaluation of the new program, was performed. The newly developed curriculum consisted of a nine credit hour specialization in professional selling. This new specialization was designed so that all students completed an introductory selling course, an advanced selling course, and an additional elective sales course from a list of options. To address the question of what selling skills were needed, a rigorous process was completed. It included an extensive literature review, including documentation from SEF and the USCA, interviews with experts who ran successful university sales programs, sales executives who routinely hired new graduates for sales roles, and an executive from an organization dedicated to sales research in universities. This process identified nine core skills 98 required to build sales competency. These core skills were introduced and practiced in the introductory level sales course. The core skills were reinforced in the advanced level course. Ten additional skills were identified as useful skills and these were introduced in the advanced level course. Discussions with internal marketing and sales faculty were used to confirm the courses to be included in the program, the skills and pedagogies to be incorporated in the sales coursework, along with how the skills would be introduced, reinforced, and assessed throughout the program. Following the initial program implementation, a quantitative survey was completed with all students in the program to assess their perceptions. Survey results suggested a high level of satisfaction among the students in the program and the perception that they had increased their competency in sales knowledge and skill attributes. This student survey was implemented prior to the formal evaluation of learning and was used as an early indicator of the program design. Student perceptions of satisfaction were later verified as part of the evaluation process using the universitys course analysis results. These results were part of the Level 1, Reaction, measures. The second part of the project was the formal evaluation of sales competency, defined as the ability of the students to implement the nine core skills. This was measured by a modified Kirkpatrick framework using Level 2 and 3 criteria, as explained in Table 10. This evaluation occurred over a three-semester period. A significant increase in sales competency was observed using this evaluation framework. This increase was measured by 1) within group knowledge and skill development gains using a pretest; post-test role play exercise in the introductory and advanced sales courses; 2) end of course project evaluations in two courses; and 3) internship supervisor evaluations. Further supporting this evidence were the career outcomes among program graduates. The conclusion was that the new professional selling program did provide the knowledge and skills students needed to build sales competency in preparation for a selling 99 career. Given the lack of research in this area, along with the strong market demand for graduates with selling skills, the result of this project was noteworthy. It contributed to the growing body of sales research, it documented a program design and evaluation process, and it provided supporting documentation for both students pursuing a selling career and businesses that require sales graduates. Discussion of the Knowledge and Skill Requirements Quantitative Survey For the nine core skills taught in the program, six showed a significant increase in perceived skills level between the introductory and advanced sales students on the initial quantitative survey of students in the sales program. The three core skills that did not show a significant difference, professionalism, asking questions, and effective listening, were directionally higher among the advanced students. A possible explanation for these results was the program design itself, which incorporated a substantial time commitment and focus on ensuring the attainment of these three skills at the introductory course level. Supporting this explanation were the mean and top box scores. These three skills had the highest mean values among the introductory students at 4.67 on a 5-point scale for asking questions and effective listening and 4.74 on a 5-point scale for professionalism. In addition, 100% of the introductory student responses indicated they agreed or strongly agreed that they had developed all three of these skills by the end of the introductory course. The advanced sales course students reported higher perceived skills attainment versus introductory students in 14 of the 15 skill areas on the survey, with 10 of those differences being statistically significant. While the student perception survey was not part of the formal course evaluation procedure and was not indicative of true learning, the results were valuable in understanding student perceptions of sales competency growth between the introductory and advanced level 100 course. The survey results also provided an initial benchmark for perceived attainment of the selling skills taught in the program and the three program learning objectives. As shown in Chapter 4, Table 19, student perceptions around the three program learning objectives were positive, with scores of 3.85 or higher on a 5-point Likert scale for agreement with meeting each objective. To further support the positive perceptions of meeting the program objectives, top box scores were assessed. Top box scores are defined in this paper as the sum of percentages for the top two points, such as strongly agree and agree, on a five-point scale. For the introductory course, 85% of respondents agreed or strongly agreed that they understood sales concepts, 85% agreed or strongly agreed they understood the sales process, and 76% agreed or strongly agreed that they knew how to structure a sales presentation. Only 3% disagreed with understanding sales concepts and the sales process, while 7% disagreed with knowing how to structure a sales presentation. Top box scores for the students who had completed both the introductory and the advanced course showed even more positive results, with 93% agree or strongly agree responses to sales concepts and the sales process. These results showed a statistically significant difference between the introductory and advanced level students (p=.01), with students at the advanced level reporting higher agreement. Top box scores among advanced level students for the third program objective were 94% agree or strongly agree for knowing how to structure a sales presentation. For all three of the program objectives, all remaining responses from the advanced students were in the neutral category and no students disagreed with any of the three program objectives statements. While these results suggest that the program has been successful in meeting its learning objectives, assessing these perceptions among the introductory level course cohort once they have completed the advanced course could provide stronger insight as to perceived growth among students who are progressing through the program. 101 It should be noted that the survey results were based on student perceptions and not actual skill attainment. For this reason, it was difficult to draw any specific conclusions based on these results alone. The findings were valuable as an initial check on course content rather than for the formal evaluation. The results suggested that all courses were perceived favorably. If repeated in the future, the survey results may serve as a valuable benchmark against which to measure changes in student perceptions. Additional questions were included in the student survey to gain insight into the future needs of the program. These results were reported in Chapter 4. They were not perceived to be of specific value to the knowledge and skill identification process and were therefore not addressed further. Discussion of the Program Evaluation Results The Level 2 and 3 evaluation results were used for the formal program evaluation. They provided the framework to measure sales competency among students in the new professional selling program. The results of this evaluation showed that the implementation of the nine core skills in the formal curriculum did improve students sales competencies. Level 2, Learning, was used to evaluate the introductory and advanced sales courses using a pre-test, post-test within group role play in each course. The use of role play is regarded as one of the most popular and effective methods to teach sales related academic curricula by both professors and sales representatives (Parker, Pettijohn, & Luke, 1996; Sellars, D. 2005). The scoring rubrics covered the sales process by evaluating students on five concepts that encompassed the nine core skills. Both the introductory and advanced course students showed significant growth (p<.001) on all five sales concepts using paired sample t-Tests, demonstrating improvement in sales competency between the pre to the post-test. Every student in both courses 102 showed growth between the pre and post-test, with the exception of one student in the advanced course who scored at the highest level, or mastery, on both the pre and post-test. All students performed at the mastery or near mastery level in each category by the end of the course. It is unlikely that the growth observed between the pre and post-test could be due to the mere practicing of a role play experience. The role plays used for the pre and post-tests were unique scenarios that required the application of the complex core skills that had been taught and reinforced throughout the semester. Application of these skills to each unique scenario was required to demonstrate mastery in the role play experience. The use of multiple evaluators, including qualified faculty and industry professionals, provided further evidence for the objectivity of the results. The Level 2 evaluation also included a between group comparison of the final role play results in the introductory and advanced courses. This was performed using an independent samples t-Test. There was no significant difference observed on any of the five concepts when comparing the final role play scores in the introductory course to the final role play scores in the advanced course, as shown in Table 29. There are several possible explanations for this result. It is possible that there was no observed increase in sales competency between the introductory course and the advanced course. Supporting this explanation was the similarity in initial role play scores between the two groups. A comparison of the between group initial role play scores in each course indicated a similar starting point among both groups, as shown in Tables 27 and 28. In fact, a mean of means for the five concepts that were evaluated showed an introductory course initial role play value of 6.73 and a nearly identical advanced course value of 6.72. Interpreted on its own, this could point to learning loss between the conclusion of the introductory course and the beginning of the advanced course. However, this conclusion assumes 103 that the two initial role play scenarios used, different for each course, were of a similar challenge level. This was not the case. The introductory course role plays used for both pre and post-test were beginner level role plays with a focus on information gathering skills such as questioning and listening. The advanced course role plays, both initial and final, were significantly more challenging, commensurate with an advanced level sales course. While the role play rubrics measured the same core skills, the application of these skills was more difficult in the advanced course role plays due to more complex scenarios. In addition, the advanced course role play scenarios were designed with less transparency surrounding the desired role play outcome than what was provided in the introductory course scenarios. For these reasons, learning loss between courses was not believed to be the most likely explanation for the lack of between group sales competency growth. A better explanation was that a between group comparison was not an appropriate measure due to the difference in the level of difficulty between the role play scenarios. Therefore, the conclusion is that a between group comparison measure is not valid for the data collected in this study. True learning among one student cohort, or a within group measurement from one course to the next may be a more reliable indicator of growth. This measure was not able to be assessed within the time frame of this study. A within group evaluation of growth from the conclusion of the introductory course to the conclusion of the advanced course, once the current students have progressed to the advanced course, is recommended as a future measure of competency. Level 3, Behavior, included two different measures. End of course projects were evaluated in the advanced sales course and the negotiation course. The advanced sales course students were evaluated by industry sales experts as part of a national sales competition. They ranked second among seven nationally-recognized sales programs, placing just behind a program 104 that was typically ranked in the top three nationally on any metric and has graduated more than 100 students majoring in professional selling annually for many years. When judged internally using the same rubric, the students in the course achieved mastery, or a score of five, on four concepts covering 14 skills. They achieved near mastery, or a score of 4.25 on the five-point scale, for the two concepts dealing with skills in interpretation of needs and use of technology and analytics. These two skills were introduced for the first time in the advanced course. The competition ranking among industry experts and in comparison to other established programs, along with the internal evaluation using the competition rubric, suggests sales competency among the advanced course students. Competition organizers typically provide video of the teams participation following the competition. Use of this video for further evaluation by businesses that recruit from the school in this study is recommended as a future metric for judging sales competency among the advanced course students. Although not all students in the program completed the elective course in negotiation, eight of the nine core skills were reinforced in this course, along with advanced skills related to negotiation. For this reason, the end of course project in the negotiation course was evaluated as part of the Level 3, Behavior, measure. The end of course project was a negotiation role play scenario. Five concepts covering the core and advanced skills were evaluated by the course instructor and an industry sales executive using a rubric. Forty-five percent of the students achieved the mastery level overall and the remainder achieved partial mastery. Negotiation requires a combination of advanced level skills, several of which are not introduced in the introductory course. Therefore, many of these students were exposed to these negotiation concepts for the first time in this course. The outcome obtained was believed to be acceptable for a newly developed course and aligned with the goals and objectives of the selling program; 105 however, no mastery goal was initially set for this course. This initial result should be used to evaluate the proportion of students achieving mastery in future semesters. As the only course where negotiation skills are the primary focus, continuing to track and evaluate this metric is important and will help in understanding student mastery of negotiation skills. Student internships were evaluated as another part of the Level 3 measurement. All students formally enrolled in the sales specialization and who had completed any internship following their junior year with sales as a responsibility were included in the analysis. Each internship supervisor provided the evaluation at the end of the students internship. To minimize differences in inter-rater agreement among the supervisors, each was provided with a standardized rubric for the evaluation and instructions for its completion. Although inter-rater agreement is not validated further, these data were valuable in that they provided an external evaluation measurement from individuals who were not closely affiliated with the sales program or its faculty and were therefore unlikely to be influenced by such an association. Communication, one of the five attributes measured by the internship supervisor, was one of the nine core selling skills. The mean result for this was 4.60 on the five-point scale, indicating that communication was a strength among the sales students who had completed an internship. In comparison, communication was rated at 4.52 for all business students. While statistically this difference was not significant, it was insightful that the sales student score was slightly higher. Teamwork, another one of the five attributes measured by the internship supervisor, was one of the ten additional selling skills introduced in the advanced sales course. The mean result for this attribute among the sales students was 4.69 on the five-point scale. This indicated that teamwork skills were a strength for sales students who had completed an internship. In 106 comparison, the mean result for teamwork among all business student interns was 4.61. While directionally higher, the difference was not statistically significant. The small sample size of sales students (n=15) may have been a contributor to the lack of significance in the result. As more students enter the sales program and complete sales internships, these data should be reexamined for significance and additional insight into sales competency judgments. The level 2 and 3 results provide evidence to suggest that there was an observed increase in sales competency among the sales program students. Therefore, the conclusion was made that the program did provide the knowledge and skills students need in preparation for a successful selling career, and implementing these skills in a formal curriculum did improve sales competency. The Level 1 and Level 4 measures did not directly assess sales competency but were identified as important in the program evaluation process for the university. Level 1, Reaction, provided student perspectives of each course using the universitys course evaluation instrument. Student affective reactions indicated a high level of satisfaction with all four of the sales courses. Scores for all attributes measured, for every course, were well above 4.0 on a five-point Likert scale. The mean of means for the four sales courses combined was 4.63 (n=37), as compared to the mean of means for all business school courses of 4.45 and all courses at the university of 4.41. The number of respondents at the school and university level was unknown. Student satisfaction may be an important indicator of student persistence in the program and therefore would be worthwhile to track in the future. Level 4, Results, consisted of measures related to student outcomes six months postgraduation. While not directly measuring sales competency, these data were insightful in terms of student success in the short term and these results bode well for the sales program. This 107 information could potentially be monitored as a leading indicator of post-graduation success as these graduates progress in their careers. While only six students were included in the Level 4 results, all of them reported successful outcomes as of the six-month post-graduation measurement period. All achieved their goal of a sales-related professional position, reported job titles commensurate with a professional selling role, and starting salaries equal to or above the mean salary for all of the schools business school graduates (n=70) from May 2021. This result suggested that graduates of the sales program were obtaining high quality professional selling positions. The program evaluation results in total supported a positive outcome as a result of the design, development, and implementation of the professional selling program. Limitations As noted previously, there were limitations to this study. Studying a small population in a single university may limit the generalizability of the findings to other schools. The small sample size of students in this study, especially the small sample of students available for inclusion in the graduate outcomes salary results, should be considered when evaluating the findings relative to other universities or programs. The study results also relied on a limited number of instructors for the courses that were part of the evaluation. Evaluation of the Intervention This project generated a strong interest within the business school. The administration, faculty, and students were supportive of the initiative from the outset. This support was driven by the desire to provide a relevant curriculum that aligned with the needs in the marketplace and the needs of the student population in the business school. The administration recognized the value related to recruitment of new students, retention of current students, and engagement and 108 potential funding support from businesses. Students were enthusiastic about the job prospects in professional selling and the course content to prepare them for those future roles. This project also fulfilled a need among local businesses searching for new sales talent and the program has resulted in financial support from the business community. Leadership issues surrounding interactions with and support from local businesses created the most significant organizational issue during the project. Some early interest and financial support from businesses created leadership conflicts between departments. To facilitate the work of the project and ensure cross-functional and peer support, an understanding of the conditions and culture within other departments and between the business school and other groups was required. These challenges were addressed through the application of adaptive leadership principles. The researcher utilized these tools to develop the leadership capacity necessary to manage the process. Frequent meetings with collaborators and other stakeholders were required to understand sources of conflict and differing values and norms, clearly communicate needs and constraints, and evaluate limitations within the organization, its structures, and its processes. Implications and Reflection This project can be used by the university and the business school in several ways. It provides a model for designing, implementing, and evaluating new specializations in the curriculum. This includes not only the technical requirements for new program development but also in understanding the leadership implications that arise when there are multiple stakeholders and collaborators and not all share the same vision or values. The adaptive leadership model provides an opportunity to build leadership capacity among a broader group of individuals. 109 This project resulted in a positive experience for the stakeholders and collaborators involved. They were able to realize a successful outcome, as measured not only by the results of the project evaluation but in terms of additional goals held by these individual members. Namely, a primary goal that was outside of the project but very important to multiple stakeholders was to generate external funding for the program. In 2021, as external business partners became exposed to students in the program and saw success by students in national competitions, significant external funding was realized from multiple donors. Reflecting on the leadership challenges of the project provided additional insight. Specifically, the diagnostic process created the most significant challenge. It required collaboration and compromise. An important adaptive leadership tool was to build the emotional capacity to cede control in parts of the process. This required self-reflection about the role of the researcher as well as the role of the other stakeholders and their positions. The most critical tools used in the diagnosis were elegant questioning and maintaining a high level of curiosity. These skills allowed for better anticipation of difficulties and the ability to accept compromise when needed. Practice over many meetings built these capacities. Future Research Given the result of the between group role play comparison, further evaluation of this area using within group growth from the introductory to the advanced course would be of interest. An evaluation of the advanced course students using a role play scenario similar in difficulty level to the introductory course rubric would minimize the bias from the more challenging role play in the advanced course, as occurred in this project. Another area for follow up research would be to evaluate program graduates one year post graduation. Understanding their strengths and opportunities for development during their first year in sales would provide valuable insight into the program and its curriculum. 110 Feedback from direct supervisors of the new graduates would also be of value and would provide additional detail that could be relevant to specific sales roles or industries. Assessment of non-business students who complete a university sales curriculum, such as a minor or specialization is another area for future research. Program funding and financial support from industry were raised as a theme during the qualitative interviews. Gaining more detailed information on the needs and requirements of businesses that would be willing to support sales programs, as well as information about other potential funding sources, could be another subject for future study. 111 References AACSB (2020). 2020 Interpretive guidance for business accreditation standards. AACSB. https://www.aacsb.edu/accreditation/standards/business Adhabi, E. & Anozie, C.B. (2017). Literature review for the type of interview in qualitative research. International Journal of Education. 9(3) 1-12. Allen, C., Kumar, P., Tarasi, C. & Wilson, H. (2014). Selling sales: Factors influencing undergraduate business students decision to pursue sales education. Journal of Marketing Education. 36(2) 94-104. American Marketing Association. (2004). Revolution in selling and sales management education: Market-driven change in the classroom. AMA Summer Educators Conference. Anderson, N. & Douglas-Gabriel, D. (2022). Colleges scramble for students as enrollment plunges. The Washington Post. Anderson, R.E., Dixon, A.L., Jones, E., Johnston, M.W., LaForge, R.W., Marshall, G.W. & Tanner Jr., J.F. (2005). The Scholarship of Teaching in Sales Education, Marketing Education Review, 15:2, 1-10, DOI: 10.1080/10528008.2005.11488899 Arthur, W., Jr., Tubre, T. C., Paul, D. S., & Edens, P. S. (2003). Teaching effectiveness: the relationship between reaction and learning criteria. Educational Psychology, 23, 275 285. Barat, S., & Spillan, J.E. (2009). A cross country comparative analysis of students perceptions of the sales profession: A look at U.S., Peru, and Guatemala. Marketing Management Journal, 19 (2), 52-63. Belkin, D. (2020). Is this the end of college as we know it? The Wall Street Journal. 112 Bloom, B.S., Engelhard, M.D., Furst, E.J., Hill, W.H., & Krathwohl, D.R., (1956). Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive domain. New York: David McKay. Bolander, W., Bonney, L., & Satornino, C. (2014). Sales education efficacy: Examining the relationship between sales education and sales success. Journal of Marketing Education, 39, 169-181. Bowe, J.J. (2015). Ethical decision making by future salespeople: When resolving sales dilemmas, do students in sales schools apply ethical philosophies differently from students in other business majors? Anderson University Doctoral Program. Bristow, D., Amyx, D., Castleberry, S. B., & Cochran, J. J. (2011). A Cross-Generational Comparison of Motivational Factors in a Sales Career Among Gen X and Gen-Y College Students. Journal of Personal Selling & Sales Management, 31(1), 77-85 Bristow, D.N., Gulati, R., Amyx, D. & Slack, J. (2006). An empirical look at professional selling from a student perspective. Journal of Education for Business, 81:5, 242-249, DOI: 10.3200/JOEB.81.5.242-249. Bureau of Labor Statistics Occupational Outlook Handbook (2019). Bureau of Labor Statistics. (2020, April 10). Occupational Outlook Handbook. https://www.bls.gov/ooh/management/home.htm. Accessed 10-26-20 Burke Institute. (2020). Seminars. https://www.burkeinstitute.com/ Bush, A. J., Bush, V. D., Oakley, J., & Cicala, J. (2014). Formulating undergraduate student expectations for better career development in sales: A socialization perspective. Journal of Marketing Education, 36, 120-131. 113 Castelberry, S. (2014). Salesperson ethics: An interactive computer simulation. Journal of Marketing Education 36(2), 209-216. Cespedes, F. V., & Weinfurter, D. (2016, April 26). More universities need to teach sales. Harvard Business Review. https://hbr.org/2016/04/more-universities-need-to-teach-sales Coco, C. M., Crumbly, J., & Jackson, F. H. (2014). The role of industry and university partnerships in developing significant learning outcomes related to business concepts. Journal of Academic Administration in Higher Education, 10(1), 1-5. Cox, B.T., Fyles, J., Nicholson, C. & Riggs, J. (2018). Professional Sales at Stetson University. Proceedings from the 7th Sales Educators Academy. May 16, 2018. Craig, R. (2020) How to close the skills gap when employers are from Mars, colleges are from Venus. Forbes. Nov 20, 2020. Crawford, J. & Lumpkin, J. R. (1983). The choice of selling as a career. Industrial Marketing Management. DOI:10.1016/S0019-8501(83)80005-4 Cummins, S., Peltier, J.W., Erffmeyer, R., & Whalen, J. (2013). A critical review of the literature for sales educators. Marketing & Entrepreneurship Faculty Publications. 5. Cummins, S., Peltier, J. W., Pomirleanu, N., Cross, J., & Simon, R. (2015). Evaluating educational practices for positively affecting student perceptions of a sales career. Marketing & Entrepreneurship Faculty Publications. 7. Deeter-Schmelz, D. R., & Norman Kennedy, K. (2011). The Journal of Personal Selling and Sales Management, Winter 2011, Vol. 31, No. 1, pp. 55-75 Donoho, C., & Heinze, T. (2011). The personal selling ethics scale: Revisions and expansions for teaching sales ethics. Journal of Marketing Education, 33(1) 107-122. 114 Emsi. (2020). Labor market data to help your institution, organization, or community thrive. Economicmodeling.com Fogel, S., Hoffmeister, D., Rocco, R. & Strunk, D. (2012). Teaching Sales, Harvard Business Review, 90, 94-99. Forbes, L., Loe, T., Patterson, R., & Erffmeyer, R. (2014). Establishing, growing, and running a sales program: An analysis of certified university sales centers, Journal of Selling, 14, 619. Gafford, J. M. (2019). Profit-Sharing vs Gainsharing: What drives millennials to a career in professional selling? Dissertation Manuscript. North Central University, School of Business and Technology Management. San Diego, CA George, B. (2018). Choosing the Right Kind of Accreditation for a Business School: A Comparison between AACSB, ACBSP, and IACBE. Journal of Research in Higher Education 2(2):45-61 DOI: 10.24193/JRHE.2018.2.3 Healy, W. J., Taran, Z., & Betts, S. C. (2011). Sales course design using experiential learning principles and Blooms taxonomy. Journal of Instructional Pedagogies, 6. Heifetz, R. A., Grashow, A., & Linsky, M. (2009). The practice of adaptive leadership: Tools and tactics for changing your organization and the world. Harvard Business Press. HR Daily Advisor Editorial Staff. (2018, September 19). Skilled sales talent is hard to come by, expensive to replace. HR Daily Advisor. https://hrdailyadvisor.blr.com/2018/09/19/skilled-sales-talent-is-hard-to-come-byexpensive-to-replace/ 115 Inks, S. A. & Avila, R. A. (2008). Preparing the next generation of sales professionals through social, experiential, and immersive learning experiences. Journal for Advancement of Marketing Education, 13(4), 47-55. Jelinek, R. (2018). Integrating SFA technology into the sales curriculum: Helping students understand what, why, and when. Marketing Education Review.Summer2018, Vol. 28 Issue 2, p80-88. DOI: 10.1080/10528008.2018.1464397 Job Placement Statistics (2019). Internal University Metrics. [Author Redacted]. Johnson, R. B., Onwuegbuzie, A. J., & Turner, L. A. (2007). Toward a definition of mixed methods research. Journal of Mixed Methods Research, 1(2), 112-133. Karakaya, F., Quigley, C. & Bingham, F. (2011). A cross-national investigation of student intentions to pursue a sales career. Journal of Marketing Education. https://doi.org/10.1177/0273475310389151 Kellerman, B. J. & Hekmat, F. (1989). Personal selling and sales management in the marketing curriculum: A status report. The Journal of Personal Selling and Sales Management, 9 (3) 35-45. Kirkpatrick, D. (1996). Revisiting Kirkpatricks four-level-model. Training & Development, 1, 54-57. Kirkpatrick, D. L., & Kirkpatrick, J. D. (2005). Transferring learning to behavior: using the four levels to improve performance (1st ed.). Berrett-Koehler. Retrieved from https://ebookcentral.proquest.com/lib/marianuin-ebooks/reader.action?docID=335397 Kunkle, M. & Peterson, R. (2018). Implications for business to business selling in the age of artificial intelligence. Sales Education Foundation. pp 1-8. Kurt, S. (2016, October 24). Kirkpatrick Model: Four Levels of Learning Evaluation. 116 Educational Technology. https://educationaltechnology.net/kirkpatrick-model-four-levelslearning-evaluation/ Lakhal, S. & Sevigny, S. (2015). The AACSB assurance of learning process: An assessment of current practices within the perspective of the unified view of validity. International Journal of Management Education, 13, 1-10. Lilly, B. & Stanley, S. M. (2016). Leveraging partnerships with local companies as a method of teaching sales: A constructivist approach. Journal for Advancement of Marketing Education, 24. Loe, T. W. & Inks, S. A. (2014). The advanced course in professional selling. Journal of Marketing Education, 36: 182. Manpower Group. (2018). Solving the talent shortage: Build, buy, borrow and bridge. 2018 Talent Shortage Survey. https://go.manpowergroup.com/talent-shortage-2018 Marshall, G. W. & Michaels, R. E. (2001). Teaching selling and sales management in the next millennium: An agenda from the AMA faculty consortium. Marketing Education Review, 11 (1). McConnell, C., Hoover, G., & Miller, G. (2008). Course embedded assessment and assurance of learning: Examples in business disciplines. Academy of Educational Leadership Journal, 12(3), 19-34. Moncrief, W. C., & Marshall, G. W. (2005). The evolution of the seven steps of selling, Industrial Marketing Management, (34)1, pp 13-22. Mullen, L. G., Larson, L. R. (2016). Analyzing the impact of a sales shadowing program: Process and outcomes. Journal for Advancement of Marketing Education, 24. 117 National Association of Colleges and Employers (NACE). (2019). First destinations for the college class of 2019. https://www.naceweb.org/job-market/graduate-outcomes/firstdestination/class-of-2019/interactive-dashboard/ National Association of Colleges and Employers (NACE). (2020). First destinations for the college class of 2020. https://www.naceweb.org/job-market/graduate-outcomes/firstdestination/class-of-2020/interactive-dashboard/ National Collegiate Sales Competition (NCSC). (2020). About us. https://www.ncscksu.org/about-us National Science Foundation. (1997). Analyzing Qualitative Data. User Friendly Handbook for Mixed Methods Evaluations. Ch. 4. https://www.nsf.gov/pubs/1997/nsf97153/start.htm Newberry, R. & Collins, M. K. (2015). Aligning sales curriculum content and pedagogy with practitioners needs. Research in Higher Education Journal, 27. 1-14. Nielson, B., & Cummins, S. (2019). Recruiting sales students: The value of professionals in the classroom. Marketing Education Review, 29(1), 65-74. Nietzel, M. T. (2020). Three options for reforming college faculty tenure. Forbes. Dec. 6, 2020. Office of Institutional Research (2020). Project Real Cost Financial Metrics. [Author Redacted]. Parker, R.S., Pettijohn, C.E. & Luke, R.H. (1996) Sales Representatives and Sales Professors: A Comparative Analysis of Sales Training Perceptions, Topics and Pedagogy, Marketing Education Review, 6:3, 41-50, DOI: 10.1080/10528008.1996.11488557 Peltier, J. W., Cummins, S., Pomirleanu, N., Cross, J., & Simon, R. (2014). A parsimonious instrument for predicting students intent to pursue a sales career: Scale development and validation. Journal of Marketing Education, 36, 62-74. 118 Pink, D. H. (2012). To sell is human: The surprising truth about moving others. New York, NY: Penguin Books. Praslova, L. (2010). Adaptation of Kirkpatricks four level model of training criteria to assessment of learning outcomes and program evaluation in Higher Education. Educational Assessment, Evaluation, and Accountability, 22, 215-225. Rodriguez, M., Honeycutt, E. D., & Ragland, C. (2015). Preliminary investigation of entertainment strategies involving alcohol: Implications for professional sales education and training in business markets. Journal of Business-to-Business Marketing. Oct-Dec2015, Vol. 22 Issue 4, p257-268. https://doi.org/10.1080/1051712X.2015.1115700 Sales Education Foundation. (2017). Building a university sales program: An opportunity to expand sales at the university level. Sales Education Foundation. https://salesfoundation.org/wp-content/uploads/2017/09/Building-A-University-SalesProgram.pdf Sales Education Foundation. (2019). Successfully activating corporate partnerships. Sales Education Foundation. https://salesfoundation.org/wpcontent/uploads/2019/11/Successfully-Activating-Corporate.2019-Annual.pdf Sales Education Foundation. (2020). Key statistics about university sales education. Sales Education Foundation. https://salesfoundation.org/wp-content/uploads/2020/06/2020Key-Statistics-Sales-education.pdf Sales Education Foundation. (2020). SEF Annual Magazine 2020. Sales Education Foundation. https://salesfoundation.org/SEFAnnual/ Sellars, D. (2005). Developing and Rote Playing Effective Sales Presentations, Canada: South Western. 119 Singh, J., Flaherty, K., Sohi, R., Deeter-Schmelz, D., Habel, J., Le Meunier-FitzHugh, K., Malshe, A., Mullins, R., & Onyemah, V. (2019). Sales profession and professionals in the age of digitization and artificial intelligence technologies: concepts, priorities, and questions. Journal of Personal Selling & Sales Management, 39 (1), 2-22. Spillan, J. E., Totten, J. W., & Chaubey, M. D. (2011). Exploring personal selling as a career option: A case study of the perceptions of African-American students. Academy of Marketing Studies Journal, suppl. Special Issue 2; Arden Vol. 15, 93-106. Spillan, J. E., Totten, J. W., & Ziemnowicz, C. (2007). What are students perceptions of a personal selling as a career? Journal for Advancement of Marketing Education, 11, 19-30 Spiller, L. D., Kim, D., & Aitken, K. (2019). Sales education in the United States: perspectives on curriculum and teaching practice, Journal of Marketing Education, Volume: 42, (3), 217-232. https://doi.org/10.1177/0273475319852756 Tan, K., & Newman, E. (2013). The evaluation of sales force training in retail organizations: a test of Kirkpatrick's four-level model. International Journal of Management, 30(2). p. 692. Accessed 27 Jan. 2021. Training Industry. (2020) Glossary Items. https://trainingindustry.com/glossary/knowledgeskills-and-abilities-ksa/ University Sales Center Alliance. (2020). Home. https://www.universitysalescenteralliance.org/content.aspx?page_id=0&club_id=554502 US News and World Report (2020). Sales representative overview. What is a sales representative? Money. (https://money.usnews.com/careers/best-jobs/sales-representative 120 Weilbaker, D. C. (2004). What to consider before developing a specialization (track, emphasis or center): personal selling as a template. Journal for Advancement of Marketing Education, 4 (Summer), 68-73. Williams June, A. (2020, November 12). Undergraduate-Enrollment Picture Worsens as Pandemic Drags On. Chronicle of Higher Education. https://www.chronicle.com/article/undergraduate-enrollment-picture-worsens-aspandemic-drags-on Wood, J. (2020). Indications that a course in sales leadership enhances early sales career success. Marketing Education Review, 30(3), 150 158. https://doi.org/10.1080/10528008.2020.1716805 121 Appendix A USCA Criteria for Membership 1. Become an Associate Member of the USCA for at least 12 months in good standing and meet all requirements for Associate membership. 2. Establish a university recognized stand-alone entity that will serve as the sales center. The entity must have a name and that name must have the word sales in some form (e.g., Sales Center; Selling Institute). 3. The sales center must have a budget. It must be able to receive funding, and the entity Director must have discretion to use funding and manage the budget. 4. Submit a document to the USCA President or Membership Chairperson which verifies the following for your school: a. Meeting the criteria outlined in #1, #2, and #3 above. b. Employs a minimum of 3 sales faculty (faculty must teach within sales curriculum), where 1 professor holds a full-time contract and a doctorate degree c. Operates a dedicated role play facility with recording capability d. Maintains a maximum of 30 students in entry-level sales classes, and a maximum of 20 students in advanced sales classes e. Provides 2 levels of sales training (e.g., basic and advanced) f. Offers a minimum of 3 sales courses 5. Full member status can be obtained without meeting one criteria if the University can demonstrate substantially exceeding requirements for other criteria 6. Pass a site visit Verification The President or other member of the USCA executive board shall conduct a site visit to verify status of program in accordance with the stated criteria for Full Membership. The site visit will include interviews with the Dean of the School and the Chair of the Department in which the Center is housed. 7. Attend the fall meeting each year Any full or associate member not attending the fall meeting will receive a letter of censure from the President of the USCA. Any full or associate member not attending two years in a row will be moved to suspended status (the University will be removed from USCA website). 122 Appendix B Research Instruments Qualitative Interview Guide Interview Guide: Sales Center Faculty/Sales Partners/Sales Research Organization Introduction Introduce self, my role, and purpose of the discussion. Review interview time (45 min to 1 hr). Restate information from invitation that interview is not recorded, that I will take notes, and that I will be documenting findings by organization type and job title, but not name of person. (Remind about my goals for this discussion) Role and Current Interactions with Partners Tell me a little bit about your role at ____________ and how you interact with partners (other universities or businesses). Probes: Interface with faculty, with students, others? Probes: Internship recruitment, FT job recruitment, speaking to classes/visits. Do you do all the recruiting or are there others with this responsibility? Do you use the college recruiting office or are you more likely to go directly to professors in the discipline from which you typically hire? Do you use any third party testing to assess potential candidates and if yes, who? What works well with that process today? Do you have any thoughts on how to improve your interactions with ________(faculty, students, career services, partners, etc.)? Probes: any partnerships where things work very well? What do they do? Any examples of what not to do? Probe: What is important when evaluating a potential university partnership? Knowledge and Skills needed by Graduates When thinking about what is required for your graduate/interns/new hires, what are the top skills that come to mind? Probes: Any others? If not mentioned, ask, What about (asking questions, listening, professionalism, presentations, telephone, time management, buying process, negotiation)? 123 Is there other knowledge that you believe is critical for new graduates to have, or are you seeing any new knowledge or skills starting to emerge or gaps in knowledge that are becoming more critical? Probe: If not mentioned, What about: CRM experience, AI, Virtual Selling, Sales Technology, Ethics Probe: Do you have any plans to include this/any of these in your own (curriculum/sales training/recommendations)? Businesses/Partners Only (Skip this section for faculty and other experts) Is there anything you would like to share about partnerships with universities? Do you have any advice, specifically to (university name) and our new sales program, about how we can best meet the needs of partners? Thank and end for Businesses/Partners Interview. Curriculum and Pedagogy Can you tell me a little about your curriculum and your suggestions for what courses you suggest for a new program? Probe: levels of classes, pre-reqs. What are your thoughts on curriculum needed for a new program that is just starting and has only a small number of students? Where would you focus your time? Which authors if any do you use to teach the selling process to your entry level sales employees? (e.g Spin Selling Neil Rackham) Is your sales curriculum open to non-business students? Why or why not? I know that role plays are commonly used in sales programs. Do you have any thoughts on that or any suggestions on other pedagogies that you have found to work well? Probes: What pedagogies have you found to be of most value? Why is that? Do you have any sources that I might reference to help me better understand those methods or how they are evaluated? Managing a Sales Program Id like to talk a little bit about what it takes to start and manage a sales program. Can you tell me how your program or typical programs are managed, especially when just getting started? 124 Probes: At what point (size of program/number of students/etc.) is a program director needed? Is that person responsible for fundraising? What other duties does that person have? What can you tell me about the fundraising process? How do you interface with your university fundraising group? What role, if any, does the Dean play in fundraising? (contacting partners, follow up, receipt and documentation of funds, budget authority, etc.). Anyone else involved? What suggestions do you have for a new program that is looking for partners? Probe: At what point do you ask for financial commitments? Do you typical ask for multi-year commitments? What level ($) of support would you feel a new, small program might achieve? Wrap up Do you have any other advice on ways to manage and grow the sales program, or on anything else weve discussed today? Thank and end Survey Instrument for Sales Students Thank you for participating in our student sales education survey. The results will be used as part of our sales program development and evaluation process. The survey should take about 5 minutes and is only for students who have completed at least one course in sales while in the (school name). No personally identifying information is being collected as part of this survey and your responses are completely anonymous. Please check all of the sales courses you have completed, including any course you are currently taking, while in the school. Which of the following types of sales competitions did you participate in? (Select all that apply) If you have not participated in any sales competitions, please select "None of these." Based on your experiences with the sales courses you have completed or are currently taking, please indicate your level of agreement with the following statements, from Strongly Agree to Strongly Disagree for each skill used in the selling process. The sales course(s) helped me build selling skills in: (list of skills) Based on your experiences with the sales courses you have completed or are currently taking, please indicate your level of agreement with the following statements, from Strongly Agree to Strongly Disagree for the knowledge and skills used in Self-Management and Customer Relationship Management. The sales course(s) helped me build knowledge and skills in: (list of skills) 125 Please indicate your level of agreement with each statement below from Strongly Agree to Strongly Disagree. I understand sales concepts and how to apply them I understand the sales process I know how to structure a sales presentation Have you previously or are you now completing an internship where at least part of your responsibilities are related to sales? What is your status in the school? I am a current student who will graduate in 2021 or earlier I am a current student who will graduate in 2022 or later What is your gender? Thank you for completing the survey. You may use the space below for any comments you wish to share about the sales program. 126 Appendix C Rubrics Level 1: Reaction, Student Evaluation of Instruction Questions (5-point scale, from Strongly Agree to Strongly Disagree) The syllabus clearly communicated the learning outcomes of the course. The readings, discussions, lectures, labs, and/or projects helped me attain the stated learning outcomes of this course. Multiple instructional methods were used in the course (e.g. lectures, problem solving, case studies, hands-on-activities, experiments, discussions, etc.). The instructional activities and assignments supported the course learning outcomes. The activities and assignments challenged me to think more deeply/critically about the course subject matter. Overall, (the Universitys) values of peace and justice, responsible stewardship, dignity of the individual, and reconciliation were reflected in the class. I would recommend this course to another student. The instructor demonstrated mastery of the subject matter. The instructor provided well-organized learning activities. The instructor provided prompt useful feedback. The instructor was available on a regular basis. The instructors behavior clearly reflected his/her genuine concern for my learning outcomes. I would recommend this instructor to another student. 127 Level 2: Learning, Role Play Rubric Student Name:____________________________________________ Concept/ Skills Approach Sales Process Professionalism Rapport Building Role Play Name: Ratings Pts 10 - 9 pts Mastery; great use 8 7 pts Partial Mastery; Used well but still room for improvement 6 pts Emerging; Gave it a good shot but not efficient yet 5 pts Still Developing; Attempted but felt weak 4 - 1 pts Tried but use was very limited at best 0 pts Not used at all 10 pts Mastery, great use 8 pts Used well but still room for improvement 6 pts Gave it a good shot but not efficient yet 4 pts Attempted but felt weak 2 pts Tried but use was very limited at best 0 pts Not used at all Effective Questioning Questioning 10 pts Mastery, great use 8 pts Used well but still room for improvement 6 pts Gave it a good shot but not efficient yet 4 pts Attempted but felt weak 2 pts Tried but use was very limited at best 0 pts Not used at all Product Presentation Communication/ Speaking 10 pts Mastery, great use 8 pts Used well but still room for improvement 6 pts Gave it a good shot but not efficient yet 4 pts Attempted but felt weak 2 pts Tried but use was very limited at best 0 pts Not used at all 10 pts Mastery, great use 8 pts Used well but still room for improvement 6 pts Gave it a good shot but not efficient yet 4 pts Attempted but felt weak 2 pts Tried but use was very limited at best 0 pts Not used at all Trust/Overcoming Objections Listening Building Trust/Ethics Obtain Commitment Persuasion/ Influence Solving Customer Problems Total Points: (out of 50) 128 Level 3: Behavior/Transfer, Negotiation Final Project Rubric Name___________________________ Date_______ Role Assigned_____________________ CONCEPT RATING 10 9 pts Mastery of the concept Prepared for the case Interests BATNA 8 7 pts Partial Mastery; Good attempts but room for improvement 6 pts Not efficient yet 5 pts Attempted but weak PTS 4 - 1 pts Very limited but some 0 pts Not used at all Grasped the issues of the case Asked questions to understand the interests of the other party Generated options to problem solve and grow the pie Behaved in ways to build trust Level 3: Behavior/Transfer, Competition Rubrics are not included due to the proprietary information they contain. 129 Level 3: Behavior/Transfer, Supervisor Internship Evaluation Final Internship Assessment By Internship Supervisor Student Name: _________________________________ Date: ________________ Intern Title: __________________________ Company Name: _____________ PART I: To be completed at the end of the students internship. Please provide your candid evaluation of this students performance or skill level in each of the following areas based on this internship experience. This evaluation is not confidential and will be shared with the student and (Study) University. 1=Lacks this skill, 2=Limited/minimal skill level, 3= Adequate/average skill level, 4=Above average skill level, 5=Exceptional skill level, N/A Not Applicable. Teamwork 1. Makes a positive impact on work team by establishing rapport and credibility. 2. Shares information/resources with others. 3. Assists/cooperates with co-workers. 4. Able to addresses business challenges in a team or collaborative setting. Communication Skills 1. Demonstrates oral communication skills required for the job. 2. Writes clearly and concisely. 3. Is willing to speak up, communicate information, and ask for clarification. 4. Listens to feedback and acts to improve. 5. Properly uses the specialized terms associated with the interns discipline. 130 Appendix D Display Matrix of Knowledge, Skills, and Pedagogy Findings Respondent Group Faculty at Other Schools Curriculum Recommendations Minimum two levels of sales courses, an introductory and an advanced; both should cover the sales process. Sales competitions are important so that students can apply sales concepts. Business Partners Sales Organization Supporting Statements USCA requires two levels and this is the only certifying organization for sales. Internships and competitions will help students with job placement. Sales technology continues to grow in importance, including virtual selling. Consider a technology course that Virtual selling is growing and students need to understand how to develop and includes presentation technologies/virtual presentations. deliver a good presentation in-person and remotely. Would like a dedicated sales This provides a good path for companies to course at the introductory level recruit students. where partners can interact with Introductory courses help students students. understand if sales is the right career Sales internships and choice for them. competitions are important; they Our other partner universities support sales expose students to real sales internships and/or competitions. scenarios. Minimum two levels of sales This is the recommendation from USCA. courses, an introductory and an Top programs have up to five sales advanced; more course are better. courses. Consider specialty areas in sales, This will attract more business partners and such as healthcare, finance, could help differentiate from other insurance. programs. Internships and competition Competitions allow students to apply what participation are needed. they have learned in the classroom. 131 Display Matrix of Knowledge, Skills, and Pedagogy Findings Respondent Group Faculty at Other Schools Business Partners Sales Organization Skills/Knowledge needed Listening Questioning Persuasion Networking Rapport building Sales Process Time Management Teamwork Solving Problems Sales Technology Sales Presentations Listening Speaking Writing Sales Process Networking Rapport Building Telephone/email etiquette/Social Persuasion Attitude/ Motivation Financial Analysis Questioning Listening Build rapport Interpreting/ Assessing Competitive Persuasive Detail oriented Pedagogy Suggestions Supporting Statements Videotaped Role Plays Competitions (internal and external) Class exercises Group Projects/ Presentations Sales pitches to class Sell something on campus Sales rep shadowing Online technology Involve business partners Mock interviews Pitches/Presentations Competitions Internships Partner interactions during class/outside of class Use skills assessment tools Use the Culture Index Have seen success in our programs (job placement, at competitions). Use skills assessment tools The Chally Assessment One on one coaching Competitions Classroom learning Role Plays USCA recommendation Graduate interview/survey data. Funding by industry from involving them in the program. Seen at other partner universities We use these tools with all new hires and they work to identify strengths and weaknesses. If you can teach them these skills, we can teach them about our specific industry/product. Most top programs use these tools. Research Publications Knowledge from other university sales programs 132 Course Framework for Knowledge and Skills Knowledge or Skill Listening Questioning Sales Process/TAO/SPIN Building Trust/Ethics Persuasion/Influence Communication/Speaking Professionalism Rapport Building Solving Customer Problems Networking Collaboration/Teamwork Sales Presentations Negotiation Time Management Interpreting Technology/Sales Tools CRM Software & Process Writing for Sales Financial Analysis/ Forecasting Introductory Course I, A I, A I, A I, A I, A I, A I I, A I, A Advanced Course R, A R, A R, A R, A R, A R, A R, A R, A R, A I, A I, A I, A I I, A I I I, A I I, A Sales Technology R R R I, A R, A I, A R, A R, A R, A I R, A Negotiation R, A R, A R, A R, A R R R, A R R, A R, A R, A Course Framework with Pedagogy Use Pedagogies Used Interaction with & Coaching from Sales Professionals Guest Speakers Role Plays Presentations Exams/Quizzes Case Studies Lecture Competitions - Regional/National Competitions - Internal/Local Resources, e.g., videos, online examples Class Projects Assignments/Writing Assignments Peer Coaching In-field job shadowing Introductory Course Advanced Course Sales Negotiation Technology  ...

... The item referenced in this repository content can be found by following the link on the descriptive page.  ...